Direct answer: The largest savings across multiple LLMs comes from routing, not from renegotiating vendor rates. Most teams overpay because they send easy jobs to expensive models out of habit, not because their per-token price is high. Unify your usage into one view first, then move each job to the cheapest model that can do it. That single change usually beats any discount a vendor will offer you.
I have watched a lot of engineering teams try to cut their LLM bill by starting at the wrong end. They open a conversation with Anthropic or OpenAI about volume discounts, or they stand up a proxy and start comparing published per-token rates across providers. Both of those are real levers, and both are small. In a multi-LLM setup the real money moves with which model you handed each job to, and the price of a token barely touches it by comparison.
The frame I use for this is a spend ladder with three rungs, and they only work in order. The bottom rung is visibility: one place where every model call from every tool is counted together. The middle rung is routing: deciding, per job, the cheapest model that can actually finish it. The top rung is rate: what you pay per token once the first two rungs are set. Teams instinctively grab for the top rung because it feels like the negotiation, but you cannot route what you cannot see, and you cannot judge a rate until you know the mix of work it applies to. So the ladder has to be climbed from the bottom.
Why unify usage before anything else
Start with the bottom rung because a multi-LLM bill is almost never one bill. Your coding agents run through Claude Code and Cursor. Your application calls OpenAI and Gemini directly. A batch job hits a self-hosted model on vLLM. Each of those shows up in a different console with a different unit and a different definition of a request, and none of them agree on what a "call" even is. Until those streams land in one place with a shared unit, every savings decision you make is a guess about a system you can only partly see.
The reason this matters is that the savings you are hunting for lives in the comparison between streams, not inside any one of them. A single provider console can tell you that your Anthropic spend rose this month. It cannot tell you that half of it was work that Gemini Flash or a small local model would have finished for a fraction of the cost, because it has no idea those other models exist. The unified view is what turns "spend went up" into "here is the specific work that was overpriced." That is the whole reason to build the bottom rung first.
Routing is where the money actually is
Once you can see all of it together, the middle rung is where you find the recurring failure that costs teams the most. I call it the expensive-model-for-cheap-job problem, and it is almost universal. An engineer wires a workflow to a frontier model during development because it is the strongest thing available and they want the thing to work. The workflow ships. Then it runs ten thousand times a day, and nine thousand of those runs are classification, extraction, short rewrites, or routing decisions that a much cheaper model handles at the same quality. The team keeps paying frontier prices for supermarket work, and because it is buried across several providers, nobody ever sees the line where it adds up.
Fixing this is genuinely mechanical once you can see it, and the tools already exist. A proxy like LiteLLM sits in front of every provider and lets you set routing rules in one place, so a job tagged "cheap" can go to Gemini Flash or a hosted small model while a job tagged "hard" still goes to a frontier model. The routing logic is not the hard part. Knowing which jobs belong in which bucket is the hard part, and that knowledge comes straight off the unified view from the bottom rung. This is the dependency that makes the ladder an order and not a menu: routing is easy to implement and impossible to target without visibility underneath it.
There is a real caution here, so route on measured outcomes rather than on hope. A cheaper model is only a saving if it finishes the task, and status in your logs tells you a call completed, not that the answer was good enough to use. Move a workflow down to a cheaper model and watch its first-try success and its retry rate for a week before you call it a win. A cheap model that fails a third of the time and forces a retry on a frontier model is more expensive than the frontier model you were trying to replace. The savings are real, but only the ones that survive the quality check are real, and completion is not quality.
Where rate negotiation finally fits
The top rung earns its place last, and it earns it for a specific reason. Once your work is routed well, your remaining spend is concentrated on the models that genuinely need to be expensive, doing work that genuinely needs them. That is exactly the volume that makes a rate conversation worth having, because you are now negotiating the price of tokens you were always going to buy. Negotiate before you route and you often lock in a committed-use discount on a pile of misrouted traffic, which quietly rewards you for the very waste you should be removing.
The order also protects you from the caching trap that hides underneath routing, and it is worth naming because it changes the math. Frontier providers charge far less for cached input tokens than for fresh ones, so a workflow that reuses a long stable prompt on one provider can be cheaper than the same workflow scattered across three providers that each recompute the context cold. Routing purely on sticker price without accounting for cache behavior can move a job to a "cheaper" model and raise its true cost. This is the last thing you tune, and you can only tune it once the streams are unified and the routing is in place.
So the honest answer to saving money across multiple LLMs is to climb the ladder in order. See it all together, route each job to the cheapest model that actually finishes it, verify the routing against real success rates, and only then take your concentrated, well-shaped volume into a rate conversation. Skip to the rate and you save a little on a mess. Fix the routing and the bill falls because the work finally costs what it should.
We built Oberhahn to give you that bottom rung, one live view of every model call across every tool and provider attributed to the workflow that made it, so the misrouted jobs are visible before you touch a routing rule. You can walk a live demo with sample data before connecting anything.
Frequently Asked Questions
How do I save money when I use several different LLMs?
Unify all of your usage into one view first, then route each job to the cheapest model that can actually complete it. Most overspend comes from sending easy jobs to expensive frontier models out of habit. Fixing that routing usually saves more than any per-token discount a vendor will give you, and you can only find the misrouted work once every provider's usage is counted together.
Is it cheaper to negotiate rates with LLM vendors or to switch models?
Switching the right jobs to cheaper models almost always beats rate negotiation, and it should come first. Once your traffic is routed well, your remaining spend is concentrated on models that genuinely need to be expensive, and that is the right volume to negotiate. Negotiating before you route often locks in a discount on traffic you should have moved off entirely.
What is the expensive-model-for-cheap-job problem?
It is the pattern where a workflow was wired to a frontier model during development, then runs at scale doing classification, extraction, or short rewrites that a much cheaper model handles at the same quality. The team keeps paying frontier prices for simple work, and because it is spread across providers, the cost rarely shows up as a single visible line.
Will a cheaper model actually save money?
Only if it still finishes the task. A cheaper model that fails often and forces retries on an expensive model can cost more than what it replaced. Move a workflow down, then watch its first-try success and retry rate before counting the saving. Completion status tells you a call ran, not that the output was good enough to use.