Cheap tokens do not guarantee cheap work. Measure AI by the full cost of an accepted outcome, then route each task to the least expensive system that can complete it reliably.
The easiest number to compare in AI is often the least useful one.
Model providers publish prices per million input and output tokens. Procurement teams copy them into spreadsheets. Engineering teams choose the apparently cheaper model. Then the application reaches production and costs more than expected—not necessarily because token volume was miscalculated, but because the cheaper model needed more attempts, more tool calls, longer outputs, and more human correction to finish the job.
Token price is an ingredient cost. Businesses pay for completed work.
This summer’s model releases made that distinction harder to ignore. OpenAI launched three GPT‑5.6 tiers positioned around different cost and capability profiles. Anthropic’s Claude Sonnet 5 added adjustable effort and presented performance as a curve rather than a single point. Google continued expanding context-caching support to reduce the cost of repeated prefixes.
The market is no longer offering one “best” model. It is offering a portfolio of intelligence, speed, and price. The architecture has to decide what each task deserves.
Suppose two models process 1,000 support escalations.
Model A costs €0.08 per run and produces an acceptable result 70% of the time. Model B costs €0.14 per run and succeeds 92% of the time. If a failed attempt is retried once and every result needs human review, the cheaper call may not produce the cheaper resolution.
The relevant equation is:
Cost per accepted outcome = (model + tools + infrastructure + retries + review + failure cost) / accepted outcomes
Each term matters.
For a low-risk internal summary, failure cost may be negligible. For an incorrect payment instruction or production change, it can dominate the equation.
This is why one global “cost per AI request” metric is misleading. Different workflows need different denominators and different quality thresholds.
Model routing without evaluation is automated guesswork.
Start with a representative set of tasks and an explicit acceptance rule. The test set should include ordinary work, edge cases, incomplete inputs, adversarial inputs, and examples where the correct behaviour is to stop or ask for help.
For each candidate configuration, measure:
A model can be more expensive per token and cheaper per accepted result. It can also be more capable on a benchmark and worse for your workflow because it is slower, overly verbose, or inconsistent with a required format.
OpenAI’s July guidance on managing AI investments makes a similar point: measure the full cost of reaching the required standard, including attempts, completion rate, latency, tool usage, and human review. The advice is commercially convenient for a frontier-model provider, but the measurement principle is sound—and it should be applied across vendors.
“Use the strongest model for finance” is not a routing strategy. Finance contains deterministic reconciliation, document extraction, ambiguous investigation, narrative explanation, and high-risk approval. These tasks require different systems.
A practical routing ladder looks like this:
The router should use observable features: input length, task type, risk class, required tools, uncertainty, and whether a previous attempt failed. It should not infer everything from a free-form prompt if the product already knows the workflow.
Escalation is often more valuable than perfect first-time routing. Begin with the least expensive configuration that regularly passes the acceptance threshold. If validation fails, uncertainty is high, or the task crosses a risk boundary, move up one level. Do not repeatedly call a weak model in the hope that randomness will eventually produce a strong answer.
Agentic systems waste money when the model performs operations that code can handle better.
Imagine an agent retrieving 200 records, filtering them by known rules, deduplicating them, calculating totals, and then writing a conclusion. Only the final interpretation may require model judgment. Passing every intermediate record through repeated reasoning turns predictable data processing into expensive probabilistic work.
Use code for filtering, joining, sorting, validation, aggregation, and schema conversion. Give the model the reduced evidence it needs for the semantic decision. This improves cost, latency, and reproducibility at the same time.
OpenAI’s July programmatic tool-calling guidance formalises this split for bounded, tool-heavy workflows. The general principle is provider-independent: models should decide where judgment is required; deterministic systems should perform deterministic operations.
The boundary needs testing. If intermediate material contains nuance the code cannot preserve, aggressive reduction may lower cost while silently reducing answer quality. Compare both approaches on the same task set.
Many production prompts repeat large blocks: policies, schemas, product catalogues, repository maps, tool instructions, or examples. Paying to process that prefix from scratch on every request is avoidable.
Prompt and context caching can reduce repeated-input cost, but only when the application is designed for stable prefixes. Put reusable material before variable task data. Version it deliberately. Track cache-hit tokens separately from ordinary input. Avoid invalidating the cache with timestamps, request IDs, or user-specific fields placed too early.
Provider behaviour differs. Google’s context-caching documentation describes automatic savings on cache hits. OpenAI’s GPT‑5.6 documentation distinguishes cache writes from discounted reads. Those billing details will change. The durable lesson is to measure the full cache lifecycle, including writes, retention, misses, and the operational complexity of keeping context current.
Do not cache your way around bad retrieval. Shipping an enormous policy manual with every task may be cheaper after caching, but a well-designed retrieval layer can still be more accurate and easier to govern.
Routing creates leverage only if switching models is technically and operationally possible.
Keep the business task, evaluation criteria, tool contracts, and audit record separate from a provider-specific prompt. Use an internal capability interface such as classify_case, draft_response, or investigate_discrepancy rather than scattering model names through the application. Normalize usage and error telemetry. Preserve the raw provider response where needed, but convert it into a stable internal result type.
Portability does not mean pretending all models behave identically. They do not. Prompts, tool use, safety behaviour, latency, and output structure vary. Every switch requires regression testing. The goal is not instant replacement; it is a controlled migration that does not require rebuilding the product.
This also protects the organisation from the quiet form of lock-in: evaluations that exist only inside a vendor dashboard, orchestration stored in a visual builder, or tool definitions that cannot be exercised elsewhere.
Own the acceptance test. It is the one asset that tells you whether a new model is actually better for your work.
For each production workflow, report five numbers:
Then break the cost down into model, tools, infrastructure, retries, and review. Engineering can use the detail to improve routing and architecture. Finance can compare the cost with the value or capacity created. Product can see whether lower cost is degrading the user experience.
Avoid converting every outcome into a speculative revenue number. Some work is valuable because it reduces risk, shortens a decision, or releases scarce expert time. Name that value honestly and choose a measure that can be observed.
The most important change is conceptual. AI is moving from a novelty budget measured in tokens to an operating capability measured in work. The teams that understand its unit economics will not necessarily use the cheapest model. They will use the least expensive system that reliably produces an acceptable result—and they will have the evidence to know the difference.
If you need to benchmark models, design a routing layer, or build an evaluation harness around a production workflow, talk to our Co‑Build team.