Three layers catch three different failures. Use all three; none replaces the others.
| Layer | Catches | Misses |
|---|---|---|
| Provider console limit and alerts | Everything, even when your code is wrong; it is the last line | Who or what spent it: one user or one agent run stops everyone; depending on the setting it may alert rather than refuse |
| Static checks before code ships (LLM Call Cost Scan) | Calls with no output cap, calls inside loops, oversized models, before they run | Anything that depends on real traffic |
| A guard in the request path (this tool) | One user, feature, key or run crossing its own budget; loops; requests with no cap; unpriced models | Calls that bypass it, which is why the provider limit stays on |
A guard that counts spend after responses come back is too late under concurrency: twenty parallel calls all see the same remaining budget. LLM Spend Guard prices each call's worst case before it is sent: a deliberately high input estimate (bytes of the request divided by three) plus the full output cap. It holds that amount against every budget the call belongs to. The real cost from the response's usage then replaces the hold. A call that never reports back is charged at its hold, never at zero.
Each budget has a scope (global, user, feature, key, run), a window (UTC minute, hour, day, month, or total) and a dollar limit, a call limit or both. A per-run total call limit is a step cap for an agent. Calls with no user still count, in one shared (none) bucket, so anonymous traffic cannot slip past a per-user cap.
A budget eventually stops a loop, but only after the loop has spent the budget. Breakers watch a rolling window and open early:
maxRepeats: the same request (same model and input) more than n times in repeatWindowSec. This catches an agent re-sending one failing tool call.maxCallsPerMinute and maxSpendPerMinuteUsd: a burst far above normal.An open breaker refuses calls in its scope for cooldownSec. The proxy answers 429 with Retry-After, so a well-behaved client waits instead of hammering.
By default a model with no configured price is refused (so a new model name cannot slip past every budget at $0), and a request with no output cap is refused. Set defaultMaxOutputTokens to add a cap instead, and maxOutputTokens to clamp any cap to a ceiling.
{
"mode": "observe",
"prices": { "your-model-name": { "in": 0, "out": 0, "cacheRead": 0 } },
"defaultMaxOutputTokens": 1024,
"maxOutputTokens": 8192,
"budgets": [
{ "id": "org-day", "scope": "global", "window": "day", "limitUsd": 25 },
{ "id": "org-month", "scope": "global", "window": "month", "limitUsd": 400 },
{ "id": "user-day", "scope": "user", "window": "day", "limitUsd": 2 },
{ "id": "run-total", "scope": "run", "window": "total", "limitUsd": 3, "limitCalls": 150 }
],
"breakers": [
{ "id": "loop", "scope": "run", "maxRepeats": 20, "repeatWindowSec": 300, "cooldownSec": 900 },
{ "id": "burst", "scope": "global", "maxCallsPerMinute": 120, "cooldownSec": 120 }
]
}
Fill in your models' current per-million-token prices from your provider's pricing page; the zeros above are placeholders, not rates. Run it in observe mode first: it logs every call it would refuse and refuses none. Replay last month's log through it on the simulator, adjust, then set "mode": "enforce".
Gateways and observability platforms such as LiteLLM, Portkey and Helicone offer budgets, rate limits or spend dashboards in various forms; check their current documentation for what each enforces and on which plan. Use this tool if you want a dependency-free file you can read in an hour. It holds worst-case cost before a call, breaks on repeated requests per agent run, refuses unpriced models by default, and replays a log offline. Not affiliated with, endorsed by, or sponsored by LiteLLM, Portkey or Helicone; their names are trademarks of their owners, used here only to describe what this tool is and is not.
Keep it on: it is the backstop that works even when your code is wrong. But it sees the whole organisation or project, not one user, feature or agent run, and depending on the provider and setting it may alert rather than refuse, or act after usage is counted. A guard in your own path can refuse one runaway user or run while everyone else keeps working.
Counting spend after responses arrive lets every concurrent call see the same remaining budget, so twenty parallel calls can all start under a limit that only one of them fits. Holding each call's worst case (input estimate plus full output cap) first means the budget is never promised twice.
A global daily and monthly dollar cap, a smaller per-user daily cap, a per-run total and step cap for agents, an output-token ceiling, a repeat breaker per run, and a calls-per-minute breaker globally. Start it in observe mode for a week, read what it would have refused, then switch to enforce.