Strap on in, we're going deep.
"Harness" is often used as a vague label for the application around a model. That application performs specific work. It assembles the model's context and exposes tools through a permission boundary. It executes approved actions and checks their results before responding to the user.
An AI harness is the runtime that turns a user's intent into a controlled and observable sequence of model calls. It prepares the model's working set and governs how proposed actions execute. It also manages completion and preserves the evidence from the run.
Start with one user story
Consider a routine coding-agent request:
As an engineer, I want the agent to find and fix a failing test so that I get a verified change and can inspect what happened.
Completing this interaction requires a series of decisions across the runtime.
journey
title Find and fix a failing test
section Request
Describe the failure: 5: Engineer
Establish workspace and authority: 3: Harness
section Investigate
Select relevant context: 3: Harness
Inspect code and test output: 4: Model, Harness
section Change
Propose and apply a patch: 3: Model, Harness
Recover from bad actions: 2: Harness
section Verify
Run the relevant tests: 4: Harness
Inspect the result and trace: 5: Engineer
Each handoff needs an owner. Unnamed responsibilities usually become scattered application code that is difficult to inspect as one execution path.
The harness is a set of boundaries
A deployment can keep these responsibilities in one process or distribute them across several services. The boundaries identify the decisions and data that must pass between them.
flowchart TB
User[User request] --> Intake[1. Request and session boundary]
subgraph Harness[AI harness]
Intake --> Context[2. Context assembly]
State[6. Working state and memory] --> Context
Context --> Adapter[3. Model adapter]
Adapter --> Loop[4. Orchestration loop]
Loop --> Policy[5. Policy and permissions]
Policy --> Tools[7. Tool execution]
Tools --> Observation[Structured observation]
Observation --> State
Observation --> Loop
Loop --> Verify[8. Verification and evaluation]
Verify --> Result[9. Result delivery]
Trace[10. Trace and observability]
Context -. record .-> Trace
Adapter -. record .-> Trace
Policy -. record .-> Trace
Tools -. record .-> Trace
Verify -. record .-> Trace
end
Result --> User
The failing-test request provides a concrete path through each part.
1. Request and session boundary
"Find and fix this failing test" provides the intent. Execution needs an identified workspace and an authenticated user. The session supplies the applicable authority policy and any history from earlier work.
The session attaches those facts to one run. Its budget limits resource use and elapsed time so stalled or repeating work can terminate cleanly.
2. Context assembly
The context starts with the request and governing instructions. The application adds source files that explain the failure and describes the tools available for the task. Relevant observations and memory can then carry earlier evidence into the current model call.
Assembly accounts for relevance and trust within the token budget. Selection quality determines whether the model has enough evidence without obscuring the failure. Instruction priority keeps repository text below system and user constraints.
3. Model adapter
The adapter translates that context into a provider request and normalizes the response. It contains provider-specific message formats and streaming behavior. It also presents usage and transient failures in a form the orchestration loop understands.
The trace should retain raw output beside the normalized form. Those two records show whether a failed action originated in model output or response parsing.
4. Orchestration loop
A coding task usually moves through investigation and modification before it reaches verification. The loop sends context to the model and interprets the proposed step. After policy and tool execution return an observation, the loop updates state and decides whether to continue.
The loop also owns stopping conditions. Verification can complete the run; blocked approval can pause it. Exhausted budgets and repeated failures can end it. A completion claim from the model is one input to that decision.
5. Policy and permissions
A tool call is an untrusted proposal. Policy validates its schema against the session's authority. The action proceeds directly or enters an approval step according to its risk.
The agent may read and edit files inside the workspace. Broader access can require additional authority when an action reaches an external system or has destructive effects. Prompt instructions document the rules; credentials and tool boundaries enforce them.
6. Working state and memory
Working state records how the current run is progressing. It connects the plan to evidence already gathered and changes already applied. It also tracks authority decisions and verification status. Durable memory carries relevant conventions or prior decisions across runs.
Explicit records support retry and crash recovery while preserving provenance. Memory retrieval can then select the relevant history for each turn.
7. Tool execution
Tools are controlled interfaces to the local environment and external systems. A useful contract defines its input schema and operational scope. Its result describes the outcome in a structured form, including failure and side effects.
Suppose the model requests pytest and the executable is unavailable. The tool result should identify the attempted command and its exit status. Error output explains the failure, while the side-effect record confirms whether anything changed. The next model call can use that evidence to find the repository's actual test command.
8. Verification and evaluation
The model then applies a plausible null check and declares the test fixed. The ordering bug remains and the test fails.
The repository contract determines the required verification. It may require a targeted test along with a build or static analysis. A failed check returns its assertion and output to the loop. The current diff gives the next model call the state that produced the failure.
Evaluation measures behavior across a task suite. It can compare the quality and reliability of different configurations while tracking their resource cost. Verification supplies the per-run evidence behind those measurements.
Following failures through the runtime
The failed tool call passes through tool execution and recovery. The incorrect patch passes through verification and another orchestration cycle.
sequenceDiagram
actor E as Engineer
participant H as Harness loop
participant C as Context builder
participant M as Model
participant P as Policy
participant T as Tools
participant V as Verifier
participant R as Trace
E->>H: Find and fix the failing test
H->>C: Assemble working context
C->>M: Send current context
M-->>H: Request test command
H->>P: Validate command and authority
P->>T: Execute allowed command
T-->>H: Failure with exit status and stderr
H->>R: Record decision and observation
H->>C: Add structured failure to context
C->>M: Ask for the next step
M-->>H: Use repository command and propose patch
H->>P: Validate reads and writes
P->>T: Apply patch
H->>V: Run required verification
V-->>H: Test still fails with assertion evidence
H->>R: Record failed verification and diff
H->>C: Add failed hypothesis and evidence
C->>M: Continue diagnosis
M-->>H: Propose corrected patch
H->>T: Apply patch within policy
H->>V: Run verification again
V-->>H: Required checks pass
H-->>E: Return verified result and summary
The sequence records the evidence from each failure, adds it to working state, and includes it in the next model call.
9. Result delivery
Runtime state provides the source for the final response. It identifies the change and the evidence used to verify it. It also preserves recovered failures and remaining warnings.
A useful result reports the artifacts and exact verification, including anything left unchecked. Authority increases as a prepared patch moves toward a commit and eventually a merge. The result should identify where that process stopped.
10. Trace and observability
The trace begins with the request and records how context was assembled for each model call. It retains the raw response alongside its normalized form. The same execution path then links policy decisions to tool results and state changes. Verification closes the trace with the outcome and resource usage.
That evidence locates failures. Retrieval may omit a required file. The adapter may misread a tool call. Policy may reject a valid action. A command may time out after a partial write. The verifier may run the wrong suite. Similar final answers can hide each of these causes.
Traces also make evaluations explainable. The score records the outcome. The trace locates the behavior that produced it within context assembly or model execution. It also captures any effect from parsing and tool policy.
A working definition of the harness
Products package these responsibilities differently. Every implementation needs an input boundary and a defined model context. It needs an execution path for authorized proposals and a state model that survives each transition. Finally, it needs a completion rule and an inspectable record of the run.
Together, these responsibilities define the harness as an application runtime with explicit boundaries and inspectable transitions.
The one-line version
An AI harness governs the path from model output to verified work.