Models are getting cheaper. Buying API capacity is getting harder.
A low price does not solve migration, reliability, or real-task performance. The point of a price war is not to predict a winner; it is to avoid getting trapped.
Price per million tokens is an input, not total cost. Caching, long-context tiers, tool calls, retries, rate limits, and migration work all change the bill.
Replay real traffic
Replay representative traffic and measure input, output, cache hits, concurrency, and retries. Do not multiply every provider price by one ideal token count; tokenizers and prompt templates differ.
Put price and quality together
Compare cost per successful task, not cost per call. A cheap model can cost more through retries, corrections, or verbose output. A premium model pays only when quality removes downstream work.
Preserve an exit
Put model names, parameters, and response translation behind an adapter and maintain a regression set. Require quality, latency, and cost gates before switching production.
Pre-release checks
- Cost is measured per successful task
- Pricing includes cache and long-context rules
- A fallback model passes the core regression set
Sources
Related
Writing, extraction, long reasoning, and high-volume support do not need the same model. Cost, latency, privacy, and reliability rarely point to one name.
GPUs, operations, inference tuning, and downtime all land on the same ledger. Self-hosting only makes sense after you have done the math on volume and people.