Agent cost control: FinOps for autonomous AI

Because an agent is not one call but a loop: it plans, calls a tool, reads the result, decides and repeats, often many times before a single task is done. Every one of those passes spends tokens, both on the model's reasoning and on re-reading whatever context it carries forward, so the bill scales with the number of steps rather than with the apparent size of the question. Two things amplify it further. A large context re-sent on every iteration multiplies token spend across the whole loop, and multi-agent systems add inter-agent messages on top, where agents spend tokens talking to each other. The upshot is that one complex task can cost orders of magnitude more than the single completion that seems to sit at its core, and because the path varies, the same task can cost very differently from run to run. This is exactly why cost has to be a monitored production signal for agents, not a back-of-envelope estimate from per-token pricing.

Loops and retries, closely followed by re-reading a large context on every iteration. A healthy agent makes progress on each pass; an unhealthy one repeats steps, retries the same failing action, or circles without converging, and because it does not error it can quietly run up a large bill before anyone looks at the invoice. Context re-reads are the quieter cousin: if every iteration re-sends the full history and a big retrieved document, you pay for those tokens again and again even when the loop is behaving. Both are insidious precisely because nothing crashes — the agent looks busy and the cost accumulates in the background. The controls that catch them are loop detection, which stops an agent that is repeating without progress, and per-task budgets, which cap what any single run may spend and fail it safely when the cap is hit. Together they convert an open-ended worst case into a bounded one.

Attach token and cost attributes to every span in your traces, then aggregate that data per task, per agent and per tool so you can see not just the total but where it concentrates. The per-span discipline matters because a single number on the monthly invoice tells you nothing actionable; cost broken down by step tells you that one tool, or one looping sub-agent, is responsible for most of the spend. Once cost sits in your dashboards next to latency and completion rate, it stops being a surprise that arrives weeks later and becomes a metric you manage in the moment, with alerts on anomalies and budgets you can enforce. This is why cost is fundamentally an observability problem: you cannot control a number you cannot see, and the same tracing that lets you debug a run is what lets you cost it. In many teams, making cost visible is also what gets the whole observability effort funded, because finance understands a runaway bill faster than an engineering abstraction.

Often yes, if you route by difficulty rather than using one large model for everything, because much of what an agent does on each loop is routine — classifying, extracting, deciding the next step — and does not need your most capable model. The pattern is model right-sizing: use a small, cheap model for the frequent, easy steps and reserve the largest model for the genuinely hard reasoning, with a clear rule for when to escalate. Done well, this cuts cost substantially while leaving end-to-end quality essentially unchanged, because the hard steps still get the strong model. The risks are real and worth managing: a too-aggressive downgrade can quietly lower completion rate, and a bad routing decision can send a hard step to a weak model, so you need your evals and completion-rate metrics in place to detect regressions before they reach users. Treat it as an optimisation you measure, not a blanket switch — change the routing, then watch quality and cost together, and keep the cheaper path only where it holds quality.