A context window is not an agent's memory
Context fills up, costs money, and disappears. An agent that lasts needs to separate what it is thinking about now from what it should remember later.
A context window is working space for one inference, not durable memory. Replaying every conversation increases cost, lets stale facts interfere, and expands the privacy boundary.
Use three memory layers
Use three layers: working memory for the current task, long-term memory for user-confirmed preferences, and external systems of record for documents and facts. Each needs its own retention and access policy.
Make writes stricter than reads
Write only reusable, stable, authorized information. Store provenance, creation time, last-confirmed time, and a deletion path. Do not persist inferred personality or sensitive attributes by default.
Verify after retrieval
Retrieved memory is candidate context, not truth. Recheck changing facts such as permissions, prices, addresses, and identity against the system of record; prefer newer and better-provenanced records when conflicts occur.
Pre-release checks
- Users can inspect and delete long-term memory
- Stale records cannot silently override current facts
- A task can run without loading the full history
Sources
Related
Writing, extraction, long reasoning, and high-volume support do not need the same model. Cost, latency, privacy, and reliability rarely point to one name.
GPUs, operations, inference tuning, and downtime all land on the same ledger. Self-hosting only makes sense after you have done the math on volume and people.