The demo cost nothing. The bill arrives in month three.

In the first four months of 2026, Uber spent its entire annual AI budget.

The mechanism was mundane. Uber rolled Claude Code out across its engineering organisation in December 2025, and adoption climbed from 32% of roughly 5,000 engineers in February to 84% by March. Average spend ran $150 to $250 per engineer per month — but power users landed between $500 and $2,000, and CTO Praveen Neppalli Naga reported spending $1,200 himself in a single two-hour demo session. By April, the year's budget was gone. Roughly 70% of committed code was AI-generated by that point, and about 11% of live backend updates were written by agents with no human in the loop.

The part that gets left out of the retelling is what happened next. Uber did not cut usage. They added prompt caching, changed model defaults, and built usage dashboards — and usage has since quadrupled while the cost per token fell.

That is the shape of this entire problem. AI running cost is not a fixed property of the technology. It is a property of how the system was engineered, and it varies by an order of magnitude between a system built carelessly and the same system built deliberately.

We build production AI systems for a living, and we run one of our own — ConGreeto, a multi-tenant AI sales assistant. This is what we have learned about what these things actually cost, and what reliably brings that down.

Running cost is a different question from build cost

Almost everything written about AI costs is about build cost: what an agency charges to deliver the thing. We will address that briefly, because you should know how weak the data is.

Every "AI agent development cost" page we could find — and there are dozens — is agency marketing citing other agency marketing. There is no survey, no analyst dataset, no published benchmark. The numbers that circulate ($8K–$500K+ overall; RAG knowledge agents at $80K–$180K; multi-agent systems at $150K–$400K+; maintenance at 20–25% of build per year) are what the market advertises, not what it measures. Treat them accordingly, including when we quote them.

Running cost is the question that decides whether the thing survives, and it is measurable.

Some context on scale: Gartner's August 2026 forecast has AI-optimised infrastructure spending growing 96% in 2026, to around $42B — and, for the first time, inference spending ($23.3B) exceeding training spending ($19B). The industry has crossed from building models to running them. The costs that matter now are the recurring ones.

The clearest illustration of what happens when those recurring costs do not work is Sora. OpenAI announced its shutdown in March 2026 and discontinued the app the following month, on $2.1 million of lifetime in-app revenue against downloads that had fallen 66% from their peak. The model worked fine. The unit economics did not.

What a token actually costs today

Published list prices, taken from the providers' own pricing pages in September 2026, per million tokens:

Model

Input

Output

Cache read

Claude Fable 5.1

$10.00

$50.00

$0.25

Claude Opus 5

$5.00

$25.00

$0.50

Claude Sonnet 5

$2.00

$10.00

$0.20

Claude Haiku 4.5

$1.00

$5.00

$0.10

GPT-6 Astra

$10.00

$50.00

$1.00

GPT-5.6 Sol

$4.00

$20.00

$0.40

GPT-5.6 Terra

$2.00

$12.00

$0.20

GPT-5.6 Luna

$0.20

$1.20

$0.02

Gemini 3.8 Flash

$0.75

$3.75

$0.075

Four things follow from this table, and they matter more than any individual number.

The spread is 50x. Input pricing runs from $0.20 to $10.00 per million tokens across the current generation, and wider still if you reach for older or smaller models. Which model handles which step of your workflow is not a detail — it is most of your bill.

Output costs three to six times input. Every design decision that makes the model talk more is a decision to spend more, and verbosity is the easiest thing in the world to accidentally encourage in a prompt.

Cache reads are a tenth of input, at both major providers. Anthropic's cache writes cost 1.25x input, so the arithmetic is: writing once and reading n times costs 1.25 + 0.1n against n for no caching. That breaks even at about 1.4 reads — meaning caching pays from the second read onward, and everything after that is margin. If your system prompt is stable and your traffic is repetitive — which describes almost every production chatbot — not using prompt caching is leaving most of your bill on the table.

Batch processing is half price at all three providers, for anything that does not need an answer while a human waits.

One scheduling hazard worth putting in your calendar now: Gemini 3.8 Flash's $0.75/$3.75 pricing is promotional and expires on 31 December 2026, after which it doubles to $1.50/$7.50. Anyone building a 2027 budget on today's Flash pricing has a 2x step change waiting in Q1.

Why agentic systems cost so much more than you modelled

This is where most budgets break, and there is finally good research on it.

A paper published in April 2026 — How Do AI Agents Spend Your Money?, with authors including Erik Brynjolfsson and Alex Pentland — measured token consumption for eight frontier models across SWE-bench Verified. Its findings are uncomfortable in a specific way:

• Agentic tasks consumed up to 1000x the tokens of ordinary chat-style use — and it is input tokens, not output, that drive the cost. Every loop re-sends the accumulated context.
• The same task, run twice, varied in total token cost by up to 30x. Not because one attempt was harder. Because the process is stochastic.
• Models cannot predict their own consumption. When asked to estimate the tokens a task would need, frontier models topped out at a 0.39 correlation with actual usage, and they systematically underestimated.
• Accuracy peaked at intermediate cost. Spending more did not buy correctness.

That 30x variance is the number to design around. It means your cost model cannot be an average — it has to be a distribution with a hard ceiling enforced in code, because the tail is where the budget dies.

There is a second-order effect that engineers hit before finance does. As one commenter put it on Hacker News: "Raw cost per inferred token grows linearly with context... So longer tasks will cost quadratically more." A longer conversation is not linearly more expensive. Each turn re-reads everything before it.

The visibility problem compounds it. From the discussion of Databricks' cost-management post: "Nobody knows what your request will cost before it returns. You're writing a blank check every time." In the same thread, a Databricks engineer described the failure mode plainly: "Costs can drastically change quickly... suddenly you're at a $10M run rate within 60 days."

The line items nobody puts in the proposal

Token spend is the line everyone models. These are the ones that surprise people.

Retrieval and embeddings. Every document ingested is embedded once and re-embedded whenever chunking changes. A vector index has a monthly floor cost whether anyone talks to your assistant or not.

Crawling and rendering. If your assistant learns from customer websites, you are running a crawler. If any of those sites are client-rendered — and increasingly they all are — you are running a headless browser, which costs an order of magnitude more CPU and memory per page than an HTTP fetch. This is a real line item on our own product and it appears in nobody else's cost breakdown.

Retries and timeouts. A request that times out at the provider and gets retried bills for both attempts. Under provider degradation, your retry logic becomes your largest cost centre precisely when it is least useful.

Failed and abandoned work. A visitor who closes the tab mid-conversation costs exactly as much as one who converts.

Observability. Tracing every model call, tool invocation and retrieval step produces a large volume of structured data. It is worth paying for — see below — but it is not free.

Human review time. The least-modelled cost of all. From the same Hacker News thread: "An AI maximizer on my team put up 2 pull requests with ~140 changed files... we spent $2500-3500 in human salary reviewing it." Stack Overflow's May 2026 piece on agent-induced decision fatigue frames the general case well, quoting Smartsheet's CPTO Pratima Arora: "The hours haven't changed, but the density of work has." The bottleneck moved from writing to reviewing, and review is paid in salary, not tokens.

Maintenance. Models get deprecated. Prompts that were tuned for one model behave differently on the next. The widely-advertised figure is 20–25% of build cost annually; we have no independent data to confirm it, but the underlying work is real and continuous.

The four levers that measurably reduce the bill

Here the literature is unusually good, because several engineering teams have published real before-and-after numbers.

1. Model routing

Not every step of a workflow needs your best model. Classification, extraction, routing and summarisation are frequently indistinguishable between a frontier model and one costing a fiftieth as much.

Databricks published measured results from their Smart Router in August 2026: over 30% reduction in average task cost while roughly matching quality. In the same post, they reported that "relatively simple tuning of our harness and caching settings" cut generated tokens by about 50%.

There is a sampling argument too, made concisely on Hacker News: "You can get 5 Luna runs for the cost of 1 Sol run." For tasks where you can cheaply verify the answer, five attempts at a cheap model can beat one attempt at an expensive one — and cost less.

2. Prompt caching, done properly

This is the lever with the widest gap between "enabled" and "working".

Deriv published a production breakdown in July 2026: an 85.8% cache hit rate, cutting input token costs by 77%. Their key finding is the one to internalise: "enabling caching does not automatically produce this result." A well-structured prompt captured roughly 77% of the available savings. A poorly structured one captured about 18%. Same feature, same model, same outputs — a 4x difference decided entirely by prompt architecture.

What made the difference:

• Separate static context from dynamic context, rather than interleaving them
• Order prompt components from least-variable to most-variable, so the stable prefix stays cacheable
• Generate repeated content deterministically — sort your tool lists, canonicalise your schemas — so identical content produces identical tokens
• Track cache hit rate as a production metric. You cannot tune what you do not measure.

That last point is the one most teams skip. Cache hit rate belongs on the same dashboard as latency and error rate.

3. Context discipline

Every token in the context window is paid for on every turn. The economics reward ruthlessness.

Some concrete anchors: OpenAI recommends fewer than 20 tools per agent, with accuracy degrading past about 10. A complex JSON schema can consume 500+ tokens on its own. And MCP servers are a common, invisible offender — as one commenter observed: "companies write an MCP server with 50 different tools, and each one has a schema... that's 7500 tokens, dumped into every session."

For a retrieval-based assistant, the largest single lever is usually moving static knowledge out of the prompt and into retrieval. When we audited our own assistant, the per-message context was dominated by business knowledge pasted in wholesale rather than retrieved on demand. That is a common pattern and an expensive one: you pay for the entire knowledge base on every message, including the 95% of it irrelevant to the question asked.

4. Batch and asynchronous processing

Half price, at every major provider, for work no human is waiting on. Nightly re-indexing, backfills, summarisation of yesterday's conversations, evaluation runs. If it can wait an hour, it should cost half.

This is also an architecture question rather than a billing one. It only works if your system separates work by latency budget in the first place — which is why we run four separate worker queues split by how long a person is willing to wait.

How do you know cheaper did not become worse?

Every lever above trades against quality. If you cannot measure quality, you are not optimising — you are degrading the product and calling it a saving.

The industry is not in good shape here. LangChain's State of Agent Engineering, surveying 1,340 practitioners in late 2025, found 89% have observability in place, but only 52.4% run offline evaluations and 37.3% run online ones. Roughly a quarter of eval users do both.

Tracing is not testing. Observability tells you what happened; evals tell you whether it was right.

Notably, the same survey found the top barrier to production was quality, at around 33% — not cost, and not latency. Cost concern actually declined year over year. If you are choosing what to invest engineering time in, the data says correctness is the constraint.

Two further cautions, because the measurement layer is less trustworthy than it looks.

Public benchmarks are gameable, and were gamed. UC Berkeley researchers reported in April 2026 that a scanning agent achieved near-perfect scores on eight major agent benchmarks without solving a single task. SWE-bench Verified fell to a ten-line conftest.py that rewrote every pytest result to "passed". One benchmark's validation checked only whether the last message came from the assistant — an empty response scored 100% across all 890 tasks. Vendor benchmark scores are not a substitute for evaluating on your own data.

LLM-as-judge is noisier than reported. A June 2026 study covering 21 judge models across roughly 541,000 judgments found a systematic 33–41 percentage point gap between the exact-match agreement teams typically report and chance-corrected agreement. It also documented judges with test-retest reliability above 0.95 that still carried substantial position bias — perfectly repeatable and perfectly wrong. Use LLM judges, but validate them against human labels before trusting a number they produce.

Multi-tenant economics: cost per customer, not cost per agent

Almost everything written on this subject prices a single agent. If you are running a product, that is the wrong unit. What you need is cost per tenant, and how it moves as tenants are added.

A few things we design for, which are worth stealing regardless of stack:

Per-tenant metering, enforced in code. Token and conversation quotas per tenant, checked before the call rather than reconciled after it. Without this, one tenant's unusual month is silently funded by everyone else's margin.

Connection budgets before token budgets. The constraint that bites first is usually not the model bill — it is the database. Managed Postgres plans cap concurrent connections at levels that look generous until you multiply by worker processes. We release the database connection around every model call, because holding one open across a multi-second LLM round trip caps your concurrency at the size of the pool. This costs nothing and is invisible until the day it is the whole problem.

Queues split by latency budget. A five-minute document ingest must never sit in front of a login code. Mixing them means paying for capacity sized to your worst case at your best case's volume.

A circuit breaker around every provider call. Under provider degradation, naive retries multiply spend at exactly the moment they are least likely to succeed.

Cost attribution per tenant from day one. If you cannot answer "what did this customer cost us last month", you cannot price, and you will discover your worst-margin segment by accident.

We priced ConGreeto's plans on top of these measurements rather than by copying competitors — which is why the tiers scale seats, conversations and tokens rather than gating features.

What this means for pricing your own product

If you are reselling AI capability, your cost structure is now usage-shaped while your revenue may not be.

The published per-outcome rates give you a market reference: Intercom charges $0.99 per resolution, Zendesk $1.50 on committed volume or $2.00 pay-as-you-go, HubSpot $0.50 per resolved conversation, Gorgias $0.90–$1.00 per AI interaction.

There is a genuine paradox in that model. Intercom's resolution rate rose from roughly 27% to 66–67% in under two years — meaning that as the product got better, the same conversation volume produced substantially more billable outcomes. Improving the AI raised the customer's bill.

And despite the noise, outcome-based pricing remains a minority practice. Gartner's Tom Coshow, quoted in CIO Dive in August 2026: "Right now, what we see is that the increase in outcome-based pricing is more buzz than reality." Only 19% of services buyers currently use it, and Gartner projects under 25% adoption through 2031.

The practical advice is unglamorous: know your cost per unit of value delivered, price with enough headroom to absorb a 30x tail, and do not build a pricing model that punishes you for improving the product.

A note on our own most-quoted statistic

Our most-read post opens with Gartner's prediction that over 40% of agentic AI projects will be cancelled by the end of 2027. That prediction is real and we stand behind the argument built on it — but in the interest of the standard we are applying to everyone else's numbers: it was published in June 2025, based on a poll of 3,412 webinar attendees taken in January 2025. It is fifteen months old.

Its substance holds up, and the cited cancellation drivers are precisely the subject of this post: escalating costs, unclear business value, and inadequate risk controls — explicitly not model performance.

The same applies to a statistic you have certainly seen: MIT's "95% of AI pilots fail." It comes from a non-peer-reviewed working paper by a group that sells AI infrastructure, and what it actually measured was that 95% of analysed projects showed no measurable P&L impact — largely because they had no pre-deployment baseline to measure against. That is a measurement failure being reported as a project failure. Worth knowing before you build a strategy on it.

Before you start: the short version

• Model the distribution, not the average. The same task can cost 30x more on one run than another. Put a hard ceiling in code.
• Instrument cost per request, per tenant, per feature on day one. Retrofitting attribution is painful and you will need it sooner than you think.
• Turn on prompt caching, then restructure your prompts so it works. The gap between careless and careful is roughly 18% versus 77% of available savings.
• Route by task, not by habit. A 50x price spread across current models is an engineering opportunity, not a footnote.
• Move static knowledge into retrieval. Paying for your whole knowledge base on every message is the most common expensive mistake we see.
• Build evals before you optimise. Otherwise you cannot distinguish a cost saving from a quality regression.
• Check your providers' pricing calendar. At least one major promotional rate expires on 31 December 2026.
• Budget for review time. It is frequently larger than the token bill and almost never modelled.

Uber's outcome is the encouraging version of this story. They did not solve a cost problem by using less AI. They solved it by engineering the system properly — and then used four times as much.

If you are scoping an AI system and want a straight answer about what it will cost to run rather than what it will cost to build, talk to our engineering team. We will show you the arithmetic from a product we operate ourselves.

Frequently asked questions

How much does it cost to run an AI agent per month? There is no credible industry benchmark, and any page giving you a confident single figure is guessing. What is verifiable: Uber's internal coding-agent usage averaged $150–$250 per engineer per month with power users at $500–$2,000, and commercial support agents are priced at roughly $0.50–$2.00 per resolved conversation. Your own number depends on token volume per interaction, model choice, cache hit rate and retry behaviour — which is why instrumenting cost per request early matters more than any benchmark.

Why are AI running costs so much higher than expected? Three compounding reasons. Agentic loops re-send accumulated context on every step, so input tokens — not output — dominate, and research published in April 2026 measured up to 1000x the consumption of single-call usage. Costs grow quadratically with conversation length. And the same task varies in cost by up to 30x between runs, so an average-based budget underestimates the tail badly.

What percentage of AI running cost is tokens versus infrastructure? It varies enormously with architecture. For a retrieval-based chatbot, tokens usually dominate, but vector storage, embedding regeneration, headless-browser crawling, observability data and retry overhead are all real recurring costs. The most under-modelled line is human review time, which is paid in salary rather than API spend.

Does prompt caching actually reduce costs? Yes, substantially — but only if prompts are structured for it. Deriv reported an 85.8% cache hit rate cutting input costs 77% in production, and also reported that a poorly structured prompt captures only about 18% of the same available savings. Cache reads cost roughly a tenth of standard input at the major providers, and with Anthropic's cache writes priced at 1.25x input, caching pays for itself from the second read onward.

How do you reduce LLM costs without hurting accuracy? In order of measured impact: route cheaper models to the steps that do not need a frontier model (Databricks measured over 30% savings while roughly matching quality), structure prompts for caching, cut context aggressively — fewer tools, tighter schemas, retrieval instead of pasted knowledge — and move anything asynchronous onto batch pricing at half rate. Crucially, run evaluations alongside all of it, or you cannot tell a saving from a regression.

Should you use a cheaper model for parts of an agent? Usually yes. Input pricing spans roughly 50x across the current generation of models. Classification, routing, extraction and summarisation are frequently indistinguishable between tiers. Reserve the expensive model for the step where reasoning quality actually determines the outcome, and verify the split with evals rather than intuition.

How do multi-tenant AI apps control per-customer costs? Per-tenant token and conversation quotas enforced before the call rather than reconciled afterwards; cost attribution tagged per tenant from day one; queues separated by latency budget so cheap asynchronous work does not occupy expensive synchronous capacity; and circuit breakers so provider degradation does not multiply retry spend. Without per-tenant attribution you cannot price, and one tenant's unusual month is absorbed by everyone else's margin.

What is the difference between building and running an AI agent? The build is a one-off project cost; running cost is monthly, usage-shaped, and continues for the life of the product. Published build-cost ranges are agency marketing rather than measured data, so treat them sceptically. Running cost is the one you can actually measure, and it is the one that determines whether the product survives its second year.

Is outcome-based pricing taking over? Not yet, despite the coverage. Gartner's analysis puts current adoption at around 19% of services buyers, with under 25% projected through 2031, and its analyst describes the trend as "more buzz than reality." It also carries a structural paradox: as your AI improves, resolution rates rise and customer bills rise with them.

How much does observability and evaluation add to running cost? Real but modest next to token spend, and among the highest-return spending in the stack. The industry gap is instructive: 89% of practitioners report having observability, while only 52% run offline evaluations. Tracing tells you what happened; evals tell you whether it was correct. Cost optimisation without the second one is indistinguishable from quality regression.

Sources & further reading

• Uber burns its 2026 AI budget in four months — Forbes, May 2026
• How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks — arXiv, April 2026
• Managing AI coding costs at scale — Databricks Engineering, August 2026
• Prompt caching and production AI agent costs — Deriv, July 2026
• Gartner forecasts worldwide AI-optimized IaaS spending to grow 96% in 2026 — Gartner, August 2026
• State of Agent Engineering — LangChain, n=1,340, June 2026
• Trustworthy benchmarks, continued — UC Berkeley RDI, April 2026
• Reliability without Validity: LLM-as-a-Judge across Agreement, Consistency and Bias — arXiv, June 2026
• OpenAI shuts down Sora — Forbes, March 2026
• Agentic AI outcome pricing models — CIO Dive, August 2026
• Coding agents are giving everyone decision fatigue — Stack Overflow, May 2026
• Gartner predicts over 40% of agentic AI projects will be canceled by end of 2027 — Gartner, June 2025
• Anthropic pricing · OpenAI pricing · Google Gemini pricing

From us

• Why 40% of AI Agent Projects Get Cancelled — And What We Did Differently — the nine engineering disciplines behind this
• Speed to Lead in 2026: The Real Benchmarks — the commercial side of the same system
• ConGreeto — the multi-tenant AI product these numbers come from
• Talk to our engineering team