Valid JSON is not the same as a result you can trust
Parsing is only the first gate. Meaning, business rules, and a safe failure path decide whether structured output belongs in a real workflow.
Structured output constrains syntax, not business truth. A response can match JSON Schema while dates, amounts, enum combinations, or object references remain wrong.
Make the schema express real boundaries
Required versus nullable must have business meaning. Constrain enums, lengths, ranges, and additional properties. Version schemas so old consumers do not silently misread changes.
Add deterministic business validation
After parsing, validate cross-field rules, database references, permissions, and freshness. End time must follow start time and product IDs must exist; deterministic rules belong in code.
Treat failure as a normal branch
Distinguish refusal, truncation, schema mismatch, and business validation failure. Retry only repairable errors with a limit and return validation feedback; high-risk actions still need confirmation.
Pre-release checks
- Schemas have versions and compatibility rules
- Code revalidates business rules
- Failures cannot leave partial state
Sources
Related
Writing, extraction, long reasoning, and high-volume support do not need the same model. Cost, latency, privacy, and reliability rarely point to one name.
GPUs, operations, inference tuning, and downtime all land on the same ledger. Self-hosting only makes sense after you have done the math on volume and people.