Stop asking which model is best. Describe the job first.
Writing, extraction, long reasoning, and high-volume support do not need the same model. Cost, latency, privacy, and reliability rarely point to one name.
Model selection is not a search for a permanent champion. It is choosing a version that meets a task-specific quality floor, latency target, and budget. Public leaderboards only narrow the field.
Define the task and failure cost
Evaluate classification, extraction, generation, RAG, and agent actions separately. Define critical failures such as missing a contraindication, inventing a citation, or calling the wrong tool. High-risk tasks need a higher bar than marketing copy.
Compare on representative samples
Fix prompts, tools, timeouts, and output format and repeat candidates on the same sample. Report success, critical failures, P50/P95 latency, and cost per successful task rather than one average score.
Keep version control after launch
Pin the production version, test upgrades on shadow traffic, and roll out gradually. Maintain rollback and fallback options, and rerun the regression set after model changes.
Pre-release checks
- Core tasks have a quality floor
- Latency and cost use real distributions
- Upgrades are gradual and reversible
Sources
Related
GPUs, operations, inference tuning, and downtime all land on the same ledger. Self-hosting only makes sense after you have done the math on volume and people.
Small models are often faster and cheaper for classification, extraction, and formatting. Pairing them with frontier models is usually more practical than picking a side.