Over the last few days, the same agent went through increasingly elaborate versions of prompt, context, and memory.
The first version sent a prompt on every turn. It was easy to diagnose because nothing carried forward. The model answered the current request and forgot the rest.
Adding previous messages fixed continuity. It also made every turn heavier than the one before it. Old tool output stayed in the request. Failed approaches sat beside successful ones. The useful parts of the conversation disappeared into its own history.
The next step was context. The model needed the repository, the relevant files, the current state of the work, and the rules governing what it could do.
That exposed another failure. The agent could maintain a conversation and retrieve information, yet still produce plausible answers unsupported by the local files. Memory had been confused with knowledge of the workspace.
Each fix exposed a different layer. The system became difficult to reason about whenever one layer was used as a substitute for another.
A prompt defines the current job
The prompt is the instruction surface for one model call.
It carries the user's request, the applicable rules, and the task-specific direction. It establishes the immediate objective and the constraints around it.
That is useful, but narrow. A prompt can tell an agent to inspect a failing test. It cannot prove which test is failing. It can say that repository files are authoritative. It cannot put those files in the model's working set.
When a model invents the contents of a file, another paragraph about files is unlikely to help. The missing input is evidence. Prompt tuning is an attractive place to hide a retrieval problem because the change is quick and the failure remains ambiguous.
History is the record
Conversation history answers a separate question: what happened between the user and the agent?
It preserves continuity. The user should not have to repeat the goal, and the agent may need to see an earlier decision before taking the next step.
It is also a poor default working set. A complete transcript grows with every turn, taking cost and latency with it. Some messages explain the current task; others are stale, superseded, or merely procedural.
Keep history for replay, audit, and recovery. Select from it when building the next call. Retention and inclusion are different decisions.
Context is the working set
Context is the evidence admitted into a particular model call.
The assembler might include the current request, a few relevant turns, repository instructions, selected source files, search results, tool observations, and a durable memory that applies to this project. The result is bounded by model capacity, latency, and cost.
The naive implementation sends the whole repository. That works until the repository becomes real. A project directory contains dependencies, generated files, build output, caches, vendored code, and unrelated packages. More input can make the relevant input harder to find.
Whole-repository text is inventory. It is not context.
The useful working set usually starts with a search for the symbol, failure, or configuration key in question. Read the matching file when the result is relevant. Follow neighboring modules when the dependency requires it. Exclude node_modules, target, build output, and caches unless the task specifically concerns them.
Sometimes a complete file is the right evidence. Sometimes one search result and a test failure are enough to choose the next step. Selection is an evidence decision, not a serialization decision.
The selection itself needs a trace. For each call, the runtime should be able to show what it included, what it omitted, where each item came from, how fresh it was, and how much budget it consumed. Otherwise a hallucinated answer has too many possible causes: omission, staleness, truncation, a bad search boundary, or a model that ignored correct evidence.
Memory decides what survives
Memory is durable state, not a second transcript.
A project convention may deserve to survive a session. So might a confirmed user preference, a decision made after investigation, or a recurring failure mode. Those records need scope and provenance. A rule learned in one repository should not silently become a global rule everywhere else.
The tempting design is to keep everything and retrieve the most similar fragments later. That creates a noisy archive. Summaries lower the size but can remove the evidence that made a fact trustworthy. Embeddings improve discovery; they do not establish authority. A graph can expose useful relationships, but it can also turn similar labels into false connections.
Memory therefore needs a promotion decision. Most observations should remain session history or searchable project evidence. Only facts that are durable, reusable, and reliable enough should cross into memory. The source should remain visible after promotion.
Memory can point toward a file. The current file still has to establish what is true.
One runtime, several state stores
The layers are distinct even though the user experiences one agent.
flowchart LR
Request[Current request] --> Assemble[Context assembly]
Prompt[Instructions] --> Assemble
History[Conversation history] --> Assemble
Memory[Scoped durable memory] --> Assemble
Workspace[Selective workspace retrieval] --> Assemble
Assemble --> Model[Model call]
Model --> Tools[Tools and observations]
Tools --> History
Tools --> Workspace
Tools --> Promote{Promote durable fact?}
Promote -->|yes| Memory
Promote -->|no| History
A turn starts with the request. The assembler adds the rules, selected history, applicable memory, and workspace evidence. The model proposes an answer or an action. Tools return observations. Those observations may shape the next call; only a small subset should become durable memory.
The runtime also owns the budget. A large history selection can crowd out the file that matters. An oversized tool result can push out the user's request. A memory hit can be relevant in topic but wrong in scope. These are assembly failures, not necessarily reasoning failures.
That is why one more prompt paragraph rarely fixes a broken agent. The problem may sit in retrieval scope, revision freshness, memory promotion, or budget accounting. The model only sees the final assembly, so a coherent answer can still be unsupported.
Diagnose the assembly
When an agent gives a bad answer, the output is the symptom. Start with the exact request sent to the model.
Separate system instructions, current input, retained history, retrieved memory, file contents, search results, tool schemas, and previous observations. Check which directories were searched and which were excluded. Confirm that selected files match the current revision. Look for truncation before blaming the model.
The useful question is not only "why did the model hallucinate?" It is "what did the runtime tell the model, and what did it leave out?"
That turns prompt tuning into an inspectable systems problem. The fix might be a narrower search boundary, a smaller history window, a provenance field on a memory, or a reserved context budget for the file under investigation.
The model is one component in the loop. Retrieval brings local reality into it. Context decides what the model can see now. Memory determines which lessons survive later. The agent becomes reliable when those responsibilities stay distinct and their handoffs remain visible.
The one-line version
Reliable agents assemble the evidence needed for the next decision and preserve only the lessons worth carrying forward.