Agent harness: why the software around the model decides whether your AI agent works

For years, nearly every AI discussion came down to one question: which model is best? Faster, smarter, a bigger context window. But anyone building agents that do real work finds that something else often decides the outcome: the software around the model.

That layer is called the agent harness. It decides what the model gets to see, what it remembers, which tools it may use and what happens when something goes wrong. My argument: whether an agent works reliably depends at least as much on the harness as on the model. And the best harnesses are often surprisingly bare.

What is an agent harness?

An agent harness is all the code around a language model that turns it into a working agent: context management, memory, tools, error handling and logging. The model reasons and writes; the harness decides what the model sees, what it's allowed to do and how the result is checked. Claude Code, Codex CLI and Mistral Vibe are examples of harnesses around a model.

Think of a car. An 800-horsepower engine, perfectly tuned, sitting on the ground with no chassis, no steering wheel, no brakes and no dashboard. An impressive piece of engineering that gets you nowhere.

That's a language model without good infrastructure. It can reason, write text and make plans. But on its own it can't:

• keep track of what it did yesterday;
• decide what should be kept;
• recover from errors systematically;
• stay consistent across multiple sessions;
• work safely with tools and external systems.

The harness is the rest of the car: steering, brakes, navigation, fuel supply, service log and safety systems.

What does a harness actually do?

A harness usually has five parts. Each answers a question the model can't answer by itself.

• Context management: what does the model see and not see? Stuffing everything into the prompt seems convenient, but it adds noise.
• External memory: instead of keeping everything in the context window, you use files, a Git repo or a database as long-term memory.
• Tools: which tools may the agent use, what do they look like, and what happens when a call fails?
• Error recovery: what if a test fails or an API times out? A good harness catches that and corrects course, instead of letting the agent build on a mistake.
• Logging and observability: without logs you don't know why an agent failed. The harness makes every step traceable.

Picture a smart intern. Without supervision he does all sorts of things; with supervision and clear agreements he delivers work you can use. Same intern.

Why fewer tools often work better

You'd think the more specialised tools an agent has, the better. In practice it often works the other way round.

With dozens of tools, the model first has to pick the right one, then fill in the parameters correctly, then understand that particular tool's error messages. Every step is a chance for a misunderstanding. Limit the agent to a few simple, predictable tools, such as ordinary shell commands like ls, cat and grep, and its behaviour often becomes more consistent.

Vercel described exactly this in December 2025. Their internal agent d0 turns plain-language questions into SQL. It had more than fifteen specialised tools and a lot of prompt engineering, and according to the team it was fragile, slow and needed constant maintenance. They threw most of it away and instead gave the agent (running on Claude Opus 4.5) a shell, access to the files that describe their data layer, and the ability to run SQL. On their test set, the success rate went from 80 to 100 percent. The agent was 3.5 times faster, used 37 percent fewer tokens and needed 42 percent fewer steps.

Two caveats. The test set was five questions, so this is a strong hint, not proof. And it worked because their data layer was already well documented, with consistently named YAML files. Their own warning: if your data layer is a mess, raw file access won't save you.

The trap: too many tools

Overengineering is tempting. You build routers, special API wrappers, complex schemas and evaluation modules. It feels professional. But for a language model it becomes a maze.

What happens then:

1. The agent picks the wrong tool.
2. The output gets misinterpreted.
3. The next step builds on that mistake.
4. Errors pile up.

There's a sneakier side too. Much of that extra machinery was built to compensate for a weakness of an older model. When the model improves, the machinery gets in its way. Vercel put it that way themselves: they removed constraints that protected against model limitations that no longer existed.

Long-running agents: the progress file

One of the hardest problems with agents is memory. A model remembers nothing outside the current context window. Put everything in it and it gets messy. Put too little in it and the next session has no idea where it left off.

A simple fix is a progress file, for example progress.md.

The routine:

1. The agent reads the progress file at every start.
2. It carries out one concrete task.
3. It updates the file with the new state.

In November 2025, Anthropic described a worked-out version of this for agents that work on one project across many sessions. A first "initializer" agent sets up the environment. It writes a list of about 200 features (in JSON, all starting as "not done"), an init.sh to start the development server, a claude-progress.txt file and a first Git commit. Every following session reads the progress and the Git history, picks one feature, tests it end to end and finishes with a commit.

That setup fixes two failures Anthropic kept seeing. The agent declared the project finished too early, and it tried to build everything at once. The feature list prevents the first; the "one feature per session" rule prevents the second.

Example of a progress file

A minimal version, sketched in Python (call_model stands for your own call to the model):

# Read progress
with open("progress.md", "r") as f:
    state = f.read()

response = call_model(task, state)

# Write the updated state back
with open("progress.md", "w") as f:
    f.write(response.updated_state)

It's almost shocking how well such a simple technique works.

A large context window doesn't solve it

Large context windows are impressive, but they don't solve everything. In July 2025, Chroma tested eighteen models, including GPT-4.1, Claude 4, Gemini 2.5 and Qwen3. The conclusion: performance degrades as input gets longer, "often in surprising and non-uniform ways", even on simple tasks such as retrieving something or copying text.

With external memory in files, a repo or a database, you put only the relevant pieces into the prompt. The rest stays outside the model, and you can simply open it and read it. You can check a progress file. You can't check what a model "remembers" in a long context.

The Bitter Lesson, applied to agents

In 2019, AI researcher Richard Sutton wrote the essay The Bitter Lesson. Its opening sentence:

"The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin."

His point: methods that scale with computation win in the long run over clever, human-designed tricks.

For agents, I read it like this. Don't keep building more elaborate prompts, and don't pile up rules and special tools to mask the weaknesses of the current model. Design infrastructure that is simple, repeatable and verifiable: a shell, files, Git, tests and logs. With a harness like that, you benefit from every better model. A harness full of tricks has to be taken apart again with every new model.

Who builds the best car?

The model still matters. But it's also the part a builder has the least say over: you pick one, and in a few months there's a better one. The harness is yours. That's where the difference lies between a demo that works once and an agent you can leave running for a week.

The engine keeps getting more powerful. You win the race with the car.