Each call to a model is stateless. It receives a constructed input, produces a response, and has no independent record of the task, previous tool executions, or authority granted on an earlier call.
An agent is stateful because its runtime carries information across those calls. It retains the task, selected context, tool results, pending approvals, retry count, and current phase of the work. The next model call receives only the parts of that state the runtime chooses to include.
That makes most agent systems state machines with a model in the loop. Model output makes the machine less obvious because it selects actions at runtime and expresses them in natural language. The runtime still tracks the current phase and the transitions that can follow it.
An agent receives a request, assembles context, asks a model for a next action, validates that action, executes it if permitted, records the result, and either continues or stops. That is a state machine. Calling the action-selection step "reasoning" does not remove the transition that follows it.
The distinction matters as soon as the agent can do more than answer a question. A model can propose delete the stale deployment, but the system needs to know whether it is inspecting, awaiting approval, applying an approved change, recovering from a failed action, or finished. Those states carry different authority. They cannot be inferred safely from a fluent sentence after the fact.
The model is a variable transition selector
Traditional state machines often encode the next transition in code. Given AwaitingApproval and an approval event, move to Applying. Given Applying and a command failure, move to Recovering or Failed.
An agent inserts a model where the next event or action is chosen. It might decide to search the repository, read a file, call a service, ask for clarification, or finish. The runtime does not need to predict that choice when it is written.
The runtime validates the proposed transition against the current state, executes only permitted actions, and records the resulting observation for the next turn.
stateDiagram-v2
[*] --> AssemblingContext: request received
AssemblingContext --> SelectingAction: bounded evidence ready
SelectingAction --> AwaitingApproval: proposed write needs approval
SelectingAction --> ExecutingTool: permitted tool call
SelectingAction --> Completed: answer is sufficient
AwaitingApproval --> ExecutingTool: explicit approval
AwaitingApproval --> Completed: request withdrawn
ExecutingTool --> ObservingResult: tool returned
ExecutingTool --> Recovering: timeout or failure
ObservingResult --> AssemblingContext: more work needed
ObservingResult --> Completed: goal met
Recovering --> AssemblingContext: safe retry or revised plan
Recovering --> Failed: retry budget exhausted
Completed --> [*]
Failed --> [*]
Model output can vary across runs. The runtime must still enforce which transitions are legal, record the result of each one, and reject actions it cannot safely perform. An execution result needs a defined owner. A timeout needs a defined next state. An approval must bind to the proposed action it authorizes, rather than being treated as general permission for whatever the model says next.
These rules determine whether the agent can resume after a restart, reject a late result after cancellation, or prove which action an approval covered. They belong in the runtime rather than in model instructions.
Where agents hide their state poorly
A naive agent implementation is a while true loop that appends messages until the model stops asking for tools. It represents state as an unbounded transcript and leaves the transition rules implicit.
The current tool result is in the history. So is a failed plan, an old instruction, a partial write, and perhaps an approval that applied to an earlier proposal. The model has to reconstruct which of those facts are still active on every turn. When it gets that reconstruction wrong, the failure is called a hallucination even though the runtime never gave the state a durable shape.
This is one reason context growth produces strange agent behavior. The same transcript is being used as conversation record, working memory, recovery log, and workflow state. It becomes difficult to tell whether a fact is active, historical, superseded, or failed output that should never be replayed.
Prompt, context, and memory are different systems describes the data side of this boundary. Retain the transcript for audit and retrieval, but track the currently valid phase separately.
State constrains the lifecycle, not the investigation
An agent does not need a fixed sequence of investigations or a prewritten answer. The model can choose among the actions available in the current state.
The runtime defines the boundary around that choice. The model can choose which allowed read tool to use. It cannot decide that a proposed destructive action no longer needs approval. It can suggest a retry. It cannot turn an exhausted retry budget into an infinite loop. It can summarize an observation. It should not decide whether a failed write committed.
A browser tool can be registered as another action from SelectingAction; it does not need a second lifecycle. A new provider can emit a different tool-call representation, but its adapter should still produce the same validated transition. The trace then records one event shape regardless of which provider or tool produced it.
Tool selection is an interface contract is the same idea at the model-to-tool boundary. Before judging whether the model made a good choice, the system has to recognize the event it emitted and place it in the correct state.
Recovery requires a known state
The happy path can succeed with only implicit state. A tool call succeeds, the model sees the result, and the agent returns a useful answer. The transcript happens to contain enough information for the next turn.
Failures expose the missing transitions. A tool times out after starting work. The model repeats the same call. The user rejects an approval request. The process restarts halfway through a multi-step change. A result arrives after the task has been cancelled.
Each case asks a question a prompt cannot answer reliably: what state is the system in now? A recovery path needs to know whether an action was proposed, authorized, started, completed, or merely described by the model. Without that record, systems guess from chat history or repeat the operation and hope it is safe.
The loop-recovery issue in MUST Is Useful Because Everything Shouldn't Be a MUST was a small example. The harness had detected repeated generation, then replayed the bad output into the next context. Recovery had to change the agent's state, not merely ask the model to behave differently while preserving the same immediate example.
Terminal states must be specified alongside tool use. The runtime needs distinct transitions for completion, failure, waiting for input, cancellation, and handoff to a person. Without them, the only available transition is another model call.
State must record authority and evidence
Agent state may be stored in prompts, queues, databases, and half-finished side effects. The runtime needs an explicit representation of the state that governs authority and recovery.
An operator must be able to inspect why the agent was allowed to act, resume a restart from a known phase, distinguish a bad model decision from a parser failure, and verify that cancellation prevents late results from changing state.
The model can then explore, select tools, and adapt to unfamiliar work within a lifecycle the runtime can inspect, replay, and test.
Previous: Prompt, Context, and Memory Are Different Systems
Next: You Can't Dashboard What You Didn't Decide
The one-line version
A model can select the next action, but the runtime must represent and enforce the state transition that action creates.