Running coding agents for hours without babysitting: notes on harness engineering

A model only produces text. Everything that turns it into an agent (the system prompt, tools, hooks, state files, the checks that run after each edit) is the harness. I run several background agents on personal projects, and most of my failures came from the harness, not the model. This post covers the pieces I added, in the order they paid off.

Guides and sensors

I think about a harness along two axes. A guide steers the agent before it acts: a short CLAUDE.md, skills loaded on demand. A sensor checks after it acts: lint, type check, tests, a reviewer agent. Guides alone mean nobody verifies the rules. Sensors alone mean the agent repeats the same mistake on every run. I need both, plus state written to files so work survives across sessions.

guides --> agent loop --> sensors
             |   ^           |
             v   |           v
           state files <-- reviewer

Public evidence says harness changes matter. LangChain kept the model fixed, changed only the harness, and moved its Terminal-Bench 2.0 score from 52.8 to 66.5. That is their own report on a public benchmark, so I read it as “the effect can be large”, not as a number to expect.

Guard: stop destructive commands before they run

The first hook I wrote was a PreToolUse guard. It reads the pending tool call and denies a short list: recursive deletes outside the project, force pushes, writes to credential files. Inside a background subagent, anything that would normally need my approval is denied outright. Nobody is there to click “allow”, and a hung job is worse than a refused one.

Read-only roles also get a tools: allowlist in their definition. A researcher agent has no edit tool, so it cannot “helpfully” fix something it was only asked to look at. That took ten minutes and removed a whole class of surprises.

Handoff gate: make “done” mean something

My worst habit was trusting a subagent that said “done”. Now every subagent must end with a fixed block:

Exit: ok | blocked
Tag: portfolio
Path: reports/2026-11-11-summary.md

A SubagentStop hook reads the last message, checks that all three fields exist and that Path points to a real file. If not, it returns a block decision whose reason becomes the agent’s next instruction:

{"decision": "block",
 "reason": "Missing Path or file does not exist. Write the report, then end with the Exit/Tag/Path block."}

I tested this end to end with claude -p. The first reply was a plain sentence and got blocked. The second came back with a valid block. Two cautions from that run. A vague reason makes the agent guess, so the message has to say exactly what is missing. And the hook must check stop_hook_active, or it can loop forever. I ran it in log-only mode for a week before letting it block.

Loop detection: agents cannot see themselves looping

An agent stuck on a wrong hypothesis edits the same file again and again, and each edit looks reasonable in isolation. A PostToolUse hook counts Edit and Write calls per file in the session. At the fifth edit it injects a message into the context: stop, re-read the original request and the first error, and state a different hypothesis. The pattern comes from LangChain’s loop-detection middleware.

I skip markdown files in the counter because notes and todo lists get edited many times for good reasons. The counter also needs a weekly cleanup.

I verified the snippet with the limit set to 3. It printed the reminder on the third edit. I have not measured whether it improves outcomes, and I would not claim it does yet.

State: survive compaction

Long sessions get compacted, and the agent loses its thread. I keep a short ## In progress section in a file, save it in a PreCompact hook, and re-inject it from a SessionStart hook on the compact event. The agent resumes from what the file says, not from what it vaguely remembers. Git history and a progress file do the rest.

Sensors that stay quiet

After each edit in an app repo, a hook runs the linter or compiler on the file that changed. Green output prints nothing. Red output prints the error plus the fix, because an error without a fix invites a guess. Anything noisier trains the agent to ignore it.

The ratchet

When the agent ignores a convention twice, I stop writing it harder in the prompt and turn it into a check: a hook, a lint rule, a pre-commit grep. A line of prose is unreliable. A deterministic check does not forget. Each mistake gets fixed once and the harness only tightens. I use “second occurrence” as the threshold, since a rule born from one odd incident is usually clutter.

What I do not know yet

• I have measured nothing on my own setup: how often subagents return the wrong format, how often the guard fires.
• Whether ask decisions behave correctly in a background subagent running with bypassed permissions is untested.
• An eval set (5 to 10 fixed tasks run before and after each harness change) is the missing piece. Without it I am tuning by feel.

If you build one, start with the guard and the handoff gate. Both take about 30 minutes, need no new infrastructure, and I noticed the difference the first day.