The real cost of AI: what the invoice shows and what it hides
Price per million tokens is only the start. The real bill depends on caching, reasoning effort, retries and architecture decisions made before the first prompt is written. Here is how we work it out.
In 2026 AI prices moved two ways: a premium tier at $10 / $50 per million tokens appeared (Claude Fable 5.1, GPT-6 Astra) while mid-tier prices fell (GPT-6 Sol at $2 / $10, Claude Opus 5.5 20% below Opus 5). The bill, though, depends less on list prices than on caching, reasoning tokens, batch discounts and surcharges.
Our rule: measure cost per solved task, not price per token, apply the levers in order and run the local-versus-API numbers with your own volumes before buying hardware.
How AI is billed in 2026
Seven meters run on every invoice; most teams watch two.
| Meter | How it works | Examples |
|---|---|---|
| Input and output | Output costs five to six times input | Claude Opus 5.5 $4 / $20; GPT-6 Sol $2 / $10; Gemini 3.5 Flash $1.50 / $9 |
| Reasoning tokens | Billed as output, even when you only see a summary | OpenAI and Google document it; effort sets the volume |
| Cached input | Reused prefixes at a fraction of input; writing the cache costs extra | Reads at 10% of input at Anthropic (5% on Opus 5.5), OpenAI's GPT-5 family and Gemini 3.5 Flash; Anthropic writes at 1.25x or 2x |
| Batch | Asynchronous jobs at half price | 50% off at Anthropic, OpenAI and Google |
| Long context | Surcharge above a threshold, by vendor | OpenAI (GPT-5.5 and later) 2x input and 1.5x output above 272K; Gemini 3.1 Pro $4 / $18 above 200K; Claude 4.6 and later none |
| Data residency | Premium to keep inference in a region | OpenAI +10% on models since March 5, 2026; Anthropic US-only 1.1x; Bedrock and Vertex regional endpoints +10% |
| Media and tools | Per image, per second or per call | Nano Banana 2 $0.067 per 1K image; Veo 3.1 $0.40 per second; Claude web search $10 per 1,000 |
Prices are splitting, not just falling
The trend depends on the tier you buy.
2026 created a premium tier at $10 / $50: Claude Fable 5 (June 9), Fable 5.1 (September 1) and GPT-6 Astra (September 3). The middle got cheaper: GPT-6 Sol ($2 / $10) and Luna ($0.10 / $0.50) launched on September 22 at half or less of their GPT-5.6 predecessors' promotional prices.
Per unit of capability, the fall is steep. Epoch AI found the price of reaching a fixed performance level dropped 9x to 900x a year depending on the threshold, with the fastest drops the least certain to last. On ARC Prize data, Claude Opus 4.6 scored 64.6% on ARC-AGI-2 for $2.25 per task in February 2026; DeepSeek V4 Flash scored 61.4% for $0.042 in July.
Token counts move too: Anthropic's tokenizer, used since Claude Opus 4.7, produces about 30% more tokens for the same text, so a lower price per token can still mean a higher bill.
Cost per solved task, not price per token
The unit that matters is the finished job.
A cheaper token that needs two retries is not cheaper. Our working formula: cost per solved task = spend on a route ÷ tasks that pass your evaluation. It captures retries, reasoning and tool calls that a price sheet ignores.
Effort shows how far the two diverge. On ARC Prize's ARC-AGI-2 board, Claude Opus 5.5 costs $0.241 per task at low effort (70.1%), $0.408 at high (93.3%) and $1.85 at max (91.7%): max costs 7.7 times low and 4.5 times high, for a lower score than high.
Treat any cost-per-task figure you haven't measured with suspicion: verify it with your own evals and tune effort as explained in when paying for thinking pays off.
The levers, in the order we apply them
Quality-neutral levers first; trade-offs last, and only with a before-and-after eval.
| Lever | Documented effect | Watch out for |
|---|---|---|
| 1. Caching | Reads at 10% of input (5% on Opus 5.5); in a 2026 case study, a 99.3% cache hit rate cut API spend 88.6% | Needs a stable prefix; on Anthropic, changing effort invalidates the cache |
| 2. Prompt and tool hygiene | Anthropic cut an agent workflow from 150,000 to 2,000 tokens by calling MCP tools from code | Tool definitions are billed: Claude's computer-use toolset adds about 4,500 tokens per request |
| 3. Output length | OpenAI's verbosity setting gave 560, 849 or 1,288 tokens on one task | On Opus 5, effort doesn't reliably shorten answers: ask for length in the prompt |
| 4. Batch | 50% off at the three big vendors; stacks with caching on Anthropic | Asynchronous; not combinable with Anthropic's fast mode |
| 5. Effort | Opus 5.5 on ARC-AGI-2: $0.241 per task at low, $1.85 at max | Less effort can cost quality; set it per route |
| 6. Routing and cascades | RouteLLM cut cost over 85% on MT Bench keeping 95% of GPT-4's quality; Cursor says its router beats Opus 4.8 at 41% lower cost | Routers need their own evals and a fallback |
| 7. Smaller models | GPT-6 Luna costs a twentieth of GPT-6 Sol; gpt-oss-120b is $0.15 / $0.60 on Groq and Together | Gaps on hard tasks and in Spanish or French |
| 8. Distillation | DeepSeek says its 8B distilled model matches Qwen3-235B-thinking on AIME 2024 | Needs data, training and evals; Anthropic blocks extraction of Opus 5.5's reasoning |
Costs that don't show on the token bill
Budget them from day one.
- Evals and observability. Every model or prompt change needs a re-run; our evals guide puts a full run at $16 to $32 in its example.
- Engineering time. In the same 2026 case study, the local open-weight setup had a fix-commit ratio of 74.9% against 45.9% with the API model.
- Retries, refusals and guardrails. Failed attempts are billed, some Fable 5 refusals have been billed since September 24, 2026, and each guardrail is another model call.
- Runtime fees. Claude Managed Agents bills $0.08 per session-hour and web search $10 per 1,000 searches.
- Lock-in. A US export-control directive forced Anthropic to pull Fable 5 for all users from June 12 to July 1, 2026.
- Scheduled price changes. Gemini 3.6 to 3.8 Flash double to $1.50 / $7.50 on January 1, 2027; GPT-5.6 Sol's $4 / $20 is promotional through at least November 21, 2026.
Local or API: a break-even method
It depends on what you compare against and how busy the machine is.
Monthly local cost = hardware ÷ months of life + power + operations time. Break-even = monthly local cost ÷ API cost per request. Then check that the machine can serve that volume and that the local model passes the same eval.
An illustrative example, not a quote: a workstation with an NVIDIA RTX 5090 (launch price $1,999, 575 W) running gpt-oss-20b, which LMSYS measured on that card at 8,519 tokens per second reading and 205 writing. Our assumptions: $1,000 for the rest of the machine, 36 months of life, 0.7 kW, 8 hours a day for 20 days a month at $0.15 per kWh (as in Pan et al.), 8 hours of operations a month at $50 an hour, and requests of 2,000 input and 500 output tokens.
| API compared | Cost per request | Requests a month to match $500 | Fits local capacity? |
|---|---|---|---|
| Claude Opus 5.5 ($4 / $20) | $0.018 | About 27,800 | Yes |
| GPT-6 Sol ($2 / $10) | $0.009 | About 55,600 | Yes |
| GPT-6 Luna ($0.10 / $0.50) | $0.00045 | About 1.1 million | No |
| gpt-oss-20b on Groq ($0.075 / $0.30) | $0.0003 | About 1.7 million | No |
Against a frontier API, the workstation pays for itself at tens of thousands of requests a month, if a 20B model passes your evals. Against the same open model served by an API, it doesn't at this utilization. Operations is the biggest line: without it, break-even against GPT-6 Sol falls to about 11,100 requests.
Studies agree on the shape. Pan et al. (2025) see on-premise paying off mainly above 50 million tokens a month or under residency mandates, and the 2026 case study found cached API calls at about $0.57 per million tokens, below the $2.83 of a shared on-premise GPU slice. Epoch AI adds that AI-chip performance per dollar rises about 49% a year: owned hardware ages fast.
Redo this calculation with your volumes, eval pass rates and labor costs. Compliance can justify local inference on its own; cost alone rarely does below high, steady volume. Options in our local AI hardware guide.
FinOps for AI: budgets, alerts and reviews
Cost control is a routine, not a project.
- Tag every call with route, feature, customer, model, effort and cache hit.
- Dashboards per route: cost per solved task, cache hit rate, output tokens per task, retries.
- Alerts and quotas: daily spend against a baseline, hard caps per key and per team. Agents burn budgets fast: OpenAI says its median researcher's daily token use, at API prices, exceeds $600.
- Effort discipline: set it per route, not per request; on Anthropic a change invalidates the cache.
- A monthly review of model mix, eval pass rates, price changes and retirement dates, with one owner who can say no.
Procurement notes for Colombia and Europe
Currency, residency and exit terms matter as much as the rate card.
Currency. Rate cards are in US dollars, so a budget in pesos or euros adds exchange-rate risk to usage risk. Budget in USD with a buffer, review monthly and have your tax advisor confirm how foreign digital services are taxed in your case; we don't model taxes here.
Residency. Keeping inference in a region costs more: +10% for OpenAI regional processing and for Bedrock or Vertex regional endpoints, 1.1x for Anthropic's US-only inference. Buying Claude through AWS, Google Cloud or Microsoft lets you use existing cloud commitments.
Retirements and exit. Anthropic publishes minimum availability dates (Opus 5.5 not before September 22, 2027; Haiku 4.5 only until at least October 15, 2026). Negotiate notice periods for retirements and price changes, migration support and data-retention terms, and keep a tested fallback from a second vendor. We help with this in AI consulting.
Key takeaways
- Price per token is only the base: caching, reasoning billed as output, batch and surcharges for long context or residency change the bill.
- 2026 split the market: a $10 / $50 premium tier appeared while mid-tier prices fell, and price per capability keeps dropping.
- Measure cost per solved task: on ARC-AGI-2, Opus 5.5 at max effort costs 7.7 times low and scores below high.
- Apply caching and batch first; effort, routing, smaller models and distillation trade quality for cost and need a before-and-after eval.
- Local beats the API only at high, steady volume or for compliance; operations time is usually the biggest line.
Sources
- Pricing
- Effort
- Models overview
- API pricing
- Introducing GPT-6 Sol and GPT-6 Luna
- Gemini API pricing
- ARC-AGI leaderboard
- LLM inference prices have fallen rapidly but unequally across tasks
- RouteLLM: An Open-Source Framework for Cost-Effective LLM Routing
- Code execution with MCP
- A Cost-Benefit Analysis of On-Premise Large Language Model Deployment: Breaking Even with Commercial LLM Services
- GeForce RTX 5090
Editorial note: this analysis reflects the public information available on the review date. Models, prices and rules change fast; every third-party figure links to its source, and our opinions are labeled as such. Spotted an error? Write to contact@slash-digital.io.
The questions we hear often
Why is output more expensive than input?
Vendors charge more for generating than for reading: on current models output costs five to six times input (Claude Opus 5.5 $4 / $20, GPT-6 Sol $2 / $10, Gemini 3.5 Flash $1.50 / $9). OpenAI and Google bill reasoning tokens as output, so effort settings weigh on the bill.
How much does prompt caching save?
Cache reads cost 10% of the input price at Anthropic, on OpenAI's GPT-5 family and on Gemini 3.5 Flash, and 5% on Claude Opus 5.5. Writes cost extra (1.25x or 2x at Anthropic), so caching pays when the same prefix is reused while the cache is alive.
Is the Batch API worth it?
For anything that doesn't need an immediate answer, yes: Anthropic, OpenAI and Google charge 50% less for batch jobs, and at Anthropic the discount stacks with caching.
When does local inference beat the API?
At high, steady volume against expensive APIs, or when compliance requires it. In our illustrative example, one RTX 5090 workstation matches GPT-6 Sol at about 55,600 requests a month but never the same open model served by an API.
How should a company budget for AI billed in dollars?
Budget in USD with a buffer for the exchange rate, set quotas and alerts per route, review monthly with announced price changes in view, and have your tax advisor confirm how foreign digital services are treated.
More analysis to read next
Claude Opus 5.5: what changes for businesses
Launched September 22, 2026: cheaper than Opus 5, Fable 5.1-level results per Anthropic. Pricing, benchmarks, breaking changes and when to use it.
ModelsReasoning models: when paying for thinking pays off
Thinking costs tokens and seconds. How Anthropic, OpenAI and Google expose effort, where it pays, where it doesn't, and how to tune it by cost per solved task.
Local AI and open weightsLocal AI in 2026: which hardware to buy and what runs on it
RTX 5090, RTX PRO 6000, DGX Spark, Ryzen AI Max+ and Mac Studio M5: memory math, what fits where, buy-vs-rent costs and picks by team size.
Where to next
Let's put it in production
Tell us your challenge. We reply within 24 business hours with an honest first read: if we can help, we'll say how; if not, we'll say who can.
I reply personally. No endless forms, no canned replies.