RAG did not stop hallucinations. It made them easier to hide.
A demo that works on three PDFs says little about a real corpus. Splitting, embeddings, retrieval, and stale sources can all produce an answer that only looks right.
A RAG demo does not guarantee reliability after corpus size, permissions, and update rate grow. Production diagnosis needs evidence from ingestion through retrieval, reranking, and generation.
Confirm the source exists
Check document version, parsed text, OCR, tables, and access labels to confirm the correct content exists in the active index. Store hashes, timestamps, and index versions.
Inspect candidates at every stage
Record original and rewritten queries, vector and keyword candidates, filter reasons, reranking scores, and final context. This distinguishes missing retrieval from later filtering.
Verify answer-to-evidence alignment
Check each claim against cited passages and ensure sources are current and authorized. After a fix, regress the whole failure class, not only one example.
Pre-release checks
- Index versions trace to source files
- Candidates are inspectable at each stage
- Permission filtering precedes model context
Sources
Related
Writing, extraction, long reasoning, and high-volume support do not need the same model. Cost, latency, privacy, and reliability rarely point to one name.
GPUs, operations, inference tuning, and downtime all land on the same ledger. Self-hosting only makes sense after you have done the math on volume and people.