Context windows are not memory

Retrieval-augmented generation combines a language model with an external document index. The original RAG work paired parametric memory in a pretrained model with non-parametric memory in a dense Wikipedia index. Retrieval made the evidence set explicit and allowed the generator to condition on selected passages.

Retrieval is not truth by itself. A production system still needs document freshness, access control, chunking, ranking, source display, and an answer policy for weak evidence. The goal is to place the smallest useful evidence set into context, not to fill the window because space is available.

A retrieval pipeline normally turns the query into a search representation, selects candidate passages, reranks or filters them, and formats the selected evidence for the model. Each stage can fail independently. A relevant document may be absent from the index, split across poor chunk boundaries, ranked below noise, or presented without enough surrounding context.

Generative Agents explored an architecture that stored a stream of experiences, retrieved records based on relevance, recency, and importance, and produced higher-level reflections used for planning. It is a research prototype, but it illustrates why durable agent memory is an application design, not a synonym for context length.

Durable memory needs a write policy as much as a read policy. Storing every message creates noise and privacy risk; storing only a summary can erase important detail. A useful design states what may be remembered, how long it is kept, who can inspect or delete it, and what evidence is required before an old record is updated.

Prompt assembly is an information architecture problem. High-priority instructions should be clear and stable. The current task should be easy to distinguish from historical discussion. Retrieved evidence should be grouped with its source, and the final portion of the prompt should restate the immediate output requirement without introducing a competing goal.

Compression should preserve decisions and evidence relationships. A useful summary says which conclusion was reached, what source or test supported it, what constraints remain, and what was intentionally excluded. A weak summary preserves fluent prose but loses the facts needed to continue safely.

For code work, a compact context might contain the issue, relevant interfaces, the current diff, failing test output, and repository rules. It usually should not contain every file inspected earlier. The agent can retrieve those files again if a later decision makes them relevant.

When a session grows, preserve the current goal, accepted constraints, changed files, verification results, and open questions. That compact decision state is usually more useful than a generic summary of every turn.

For facts that must remain exact, use structured storage with explicit updates. For large knowledge collections, retrieve and cite. For the immediate task, keep the prompt focused. Calling all three things memory hides the design choices that determine reliability.

Test memory as a system. Check whether relevant records are retrieved, outdated records are replaced, access rules are respected, deleted information stays deleted, and the model can continue when retrieval returns nothing. These cases matter more than demonstrating that a long conversation can be replayed.