Cost vs LiteLLM: an honest comparison

LiteLLM is free, MIT licensed, and solves a problem Cost does not touch. If you have six services talking to three providers and you want one API, one place for keys, and automatic fallback when a provider degrades, that is a real need and LiteLLM is the standard answer to it. Nothing in Cost replaces a gateway.

Its spend tracking is also better than most people assume. Per request, per key, per team, per user, per customer, per tag and per model, written to a table you own and can query directly. For a platform team that already runs the proxy, that is a lot of visibility for nothing, and the data stays entirely in your infrastructure.

Being open source matters here in a way it does not always. A gateway sits in your request path, so being able to read it, fork it and run it without a vendor is not an ideological preference, it is a sensible availability requirement.

The attribution units are different in a way that decides who can use the answer. LiteLLM tells you that the key named backend-prod spent nine thousand euro. Cost tells you that the document summarisation route spent nine thousand euro, and that it did so because the system prompt grew in March. One of those is a fact. The other is something an engineer can act on this afternoon.

Cost does not go in your request path. A proxy that tracks spend is also a proxy that can be down, rate limit you, add latency, or lose requests. Teams who are already nervous about their LLM bill are rarely looking to add a dependency to every call, and the security review for an out-of-band SDK is shorter than the one for a new hop.

The verification step has no equivalent. LiteLLM can route traffic to a cheaper model. It cannot tell you whether the cheaper model's answers were good enough, which is the only question that actually blocks the saving. Cost shadow runs the cheaper model on your real traffic, has an independent judge score the answers, and surfaces the recommendation only when it matches.

If you are a platform engineer standardising how a dozen services reach three providers, run LiteLLM. It is the right tool and it is free. Use its spend logs for chargeback between teams.

If you are an engineering lead who needs to cut a bill this quarter without a migration, Cost is the faster path. It is a library change rather than an infrastructure change, and it arrives at a ranked list of routes with a verified recommendation against each.

If you are in a regulated environment where nothing may leave your network, LiteLLM self-hosted keeps everything inside, which is the strongest position available. Cost keeps bodies in your network by default but the dashboard is hosted, so the metadata leaves. Which of those matters depends on your policy, and it is worth reading ours before assuming.

If you are a small team with one service and one provider, you probably need neither a gateway nor a dashboard yet. Wrap one client with Cost's free tier, look at where the money goes, and come back to the gateway question when you have a second provider.

Yes, and plenty of teams should. LiteLLM gives you one API and per-team budgets. Cost gives you per-route attribution and a verified answer on whether the cheaper model holds up. They do not conflict: Cost wraps the client your code calls, which can be the LiteLLM SDK pointing at your own proxy.

The one thing to decide deliberately is which system is the source of truth for invoice reconciliation, because having two spend numbers that disagree by four percent is worse than having one. We would use the provider's own invoice as truth and treat both tools as explanations of it.