Your bill says tokens. Where did the money actually go?
A total invoice cannot show which feature burns the budget or which workflow loses money. Costs need to be visible per task and per call.
A provider invoice shows total spend but rarely which product workflow created it. Optimization requires attributing usage to a recognizable user task.
Attribute every call
Record task ID, feature, tenant, environment, model version, input and output tokens, cache usage, retries, and tool cost. Keep user content out of the cost table.
Calculate cost per successful task
Aggregate retrieval, model calls, and retries by task ID, then join with success, human correction, and revenue. Inspect P50, P95, and expensive outliers, not only averages.
Remove waste first
Remove waste first: loops, duplicated context, verbose output, poor cache use, and development traffic on production models. A smaller model or shorter prompt is a saving only if task quality holds.
Pre-release checks
- Internal usage reconciles with provider invoices
- Every core feature has unit task cost
- Budget alerts identify task and version
Sources
Related
Writing, extraction, long reasoning, and high-volume support do not need the same model. Cost, latency, privacy, and reliability rarely point to one name.
GPUs, operations, inference tuning, and downtime all land on the same ledger. Self-hosting only makes sense after you have done the math on volume and people.