arXiv 2603.25723 "Natural-Language Agent Harnesses": 4 Patterns + 3 Anti-Patterns After 12 Weeks in Prod

Natural-Language Agent Harnesses (arXiv 2603.25723) — Pan et al.’s March 2026 paper — argues the harness (the CLAUDE.md + hooks + skills layer around your agent) is now a first-class scientific object. After reading it twice with my repo open, only 4 patterns and 3 anti-patterns from Natural-Language Agent Harnesses (arXiv 2603.25723) survived my production filter, and they’re the ones I’ll show you here.

3 of the 4 patterns are still in prod after 12 weeks. 1 broke my agent in week 6 and had to be redesigned. The 2 anti-patterns I tried to keep alive cost me roughly 47 hours before I finally cut them.

The 4 patterns (jump to the section that matches your itch):

1. Roles as an explicit top-level section in CLAUDE.md — what the agent is, split cleanly from how it behaves. See What the paper made me rename below.
2. Verification gates as first-class scripts under verify/ — every “if X fails, abort” moved out of prose and into a pass/fail script that runs the same in CI and locally. Same section.
3. Delegation boundaries — the audit that surfaces “helper” skills doing things that should be hardcoded, and code doing things that should be delegated. Same section.
4. Adopting the paper’s vocabulary itself — the smallest, cheapest, highest-ROI move: rename agent-setup/ to harness/, and standups get shorter. See The vocabulary effect on the team.

The 3 anti-patterns the paper implies: the runtime lock-in (“adopt IHR or nothing”), the primitives-as-checklist trap (treating them as boxes to tick instead of a lens to audit through), and the “harness will be obsolete once models get smarter” reflex — I disagree with all three and cover each in What I’d push back on.

I had been calling all of this “the setup.” That’s engineer-speak for “no real name yet.” Reading the paper felt like the embarrassment of learning the proper word for something you’d been mispronouncing in public for years.

The paper is arXiv 2603.25723, submitted on March 26, 2026 by Linyue Pan and four colleagues. It’s the first paper I’ve seen that treats the harness as a first-class research object. Previous mentions lived as footnotes in agent-framework papers or as blog posts. This one puts the harness at the center of the study, and that reframe did more to my thinking than I expected.

The thing I was actually building

If you’ve ever wired up Claude Code, Cursor, or Codex with a custom CLAUDE.md, a few hooks, some skills, and a runtime script that orchestrates them, you’ve built a harness. You’ve probably called it “config,” “scaffolding,” “infra,” “the harness around the model” if you read Anthropic’s blog, or, like me, “the setup.” Same thing. Different labels. We just didn’t agree on which one.

The Pan et al. paper makes the case that this thing has structure, properties, and a definition worth pinning down. From the abstract:

Agent performance is strongly shaped by the surrounding harness: the external execution system around a model that organizes a task run. Yet this logic is usually buried in tightly coupled controller code, which makes harnesses hard to inspect, compare, transfer, and ablate.

That second sentence is the one that hit. Hard to inspect, compare, transfer, and ablate. Every agent project I’ve looked at has its own private dialect: its own way of expressing role, contract, verification, state. When I move from project to project, none of it ports. I rebuild the same patterns from scratch every time, slightly differently, slightly worse.

The paper proposes a fix: write the harness in natural language, in a portable format, and run it through a shared runtime they call IHR (Intelligent Harness Runtime). The “natural language” choice is the load-bearing one. Humans read it, agents read it, and it survives a model swap.

What the paper adds beyond a word

When I first skimmed the paper, my reaction was “okay, they invented a word for the thing.” Fair skeptic take: this is academia’s contribution to a problem industry already solved. Anthropic ships harness-design blog posts, the awesome-harness-engineering GitHub repo lists 80+ tools, and Aakash Gupta’s widely-read post declared 2026 “the year of agent harnesses.” A new word alone is a thin contribution.

But two things in the paper pay for the read.

Formalization. Pan et al. name the pieces a harness has to organize — roles, contracts, validation gates, durable artifacts, delegation across child agents — and show them working as separate NLAH modules that can be ablated. That sounds abstract until you try to map it onto your own project. I sat down with my agent-setup/ folder and worked through it. Every primitive matched something I had built. Only I had built each one a different way, and I had no name for any of them. My CLAUDE.md was four primitives mashed together. My skills were both contracts and roles depending on how you squinted. The mess was legible to me, and only me.

Durability. The obvious objection: won’t models eventually be smart enough that we don’t need the harness? The paper’s reply, paraphrased: harness-level control remains important even when the base model improves, because stronger models still need task structure, state discipline, and acceptance criteria.

The bits in your AGENTS.md and CLAUDE.md work like a spec for how the agent operates. Specs don’t go obsolete when the underlying engine improves; they go obsolete when requirements change. Requirements (what counts as “done,” what gets verified, who has authority to commit code) don’t get smarter just because the model does.

That argument changed how I think about my CLAUDE.md. I used to treat it as something I’d eventually “outgrow.” Now it looks like the part of the system most likely to outlive any specific model.

What the paper made me rename

I went through my agent-setup/ folder the weekend after I read the paper. Here’s what changed.

agent-setup/ → harness/. The whole folder. Took thirty seconds. Felt absurd. But within two weeks, three teammates had referenced “the harness” in our standup without me prompting it, which never happened with “the setup.” A single-word name sticks in ways a whole phrase can’t.

My CLAUDE.md got a new top-level section: ## Roles. Previously the file was a wall of mixed instructions: rules (“never run git push --force”), context (“we deploy to Cloudflare”), behavioral defaults (“prefer rg over grep”). Now I separate them: a role says what the agent is supposed to be, a contract nails the output, and verification gates decide whether the output counts as done. The file got longer but easier to reason about. Splitting a 500-line function into four 125-line ones makes the program easier to read even when the line count doesn’t drop.

My orchestration script got a verify/ directory. Verification gates were the primitive I was weakest on. I had implicit checks scattered around (“if the test command fails, abort”), but no explicit notion of what a verification gate was. Now each gate is a small script: takes input, returns pass/fail with a reason, runs in CI as well as locally. Of the four changes, this one paid off the most.

I deleted three “helper” skills. The paper’s notion of delegation boundary (what you actually let the agent decide vs. what you reserve for humans) surfaced that I had skills doing things that should have been hardcoded, and code doing things that should have been delegated. The cleanup was small, but it removed a category of bug I kept hitting: the agent making decisions I didn’t realize I was authorizing.

The vocabulary effect on the team

The least-quantifiable change was also the most useful: standup meetings got shorter because we stopped arguing about which thing we were talking about.

Before: “I’m working on the agent stuff. The…you know. The orchestration layer? The CLAUDE.md plus the skills plus the runner?”

After: “I’m refactoring the verification gates.”

The first version is six seconds long and conveys roughly nothing. The second is two seconds long and tells you exactly what’s happening. Multiply that across meetings, PR descriptions, and Slack threads, and the seconds add up. I have no scientific measurement of this, and I’m not going to invent one for the drama, but the qualitative shift is real. We picked up a shared word for a shared thing, and standup got shorter.

This is the boring reason academic terminology matters. The paper’s biggest contribution is a stable name that lets colleagues talk about the thing without pre-negotiating what to call it. The name matters more than the runtime or the primitives. Hadley Wickham’s “tidy data” pulled the same trick for data analysis a decade ago. Any field doing the work without a vocabulary needs one.

What I’d push back on

A few things in the paper I’m not yet sold on.

The runtime requirement. The paper bundles natural-language harnesses with a specific runtime, IHR. The argument is reasonable (without a shared runtime, “natural language” can mean anything), but in practice, every team will use whatever runtime they’re already on (Claude Code, Cursor, custom). The natural-language spec is the portable part. Tying the framework to a single runtime risks turning it into an all-or-nothing adoption. The primitives, though, can be picked up piecemeal, and each one pays off on its own.

The benchmarks. The paper validates on coding, terminal-use, and computer-use tasks, which is fine, but I’d love to see harness ablations on non-coding domains. My agents do plenty of writing, scheduling, and summarization, and I don’t yet know whether the same primitives carry over. The paper gets stronger the day it tests its own portability claim.

The implication that this is settled. It’s a v1 from March 2026. Anthropic’s own harness-design post from March uses different vocabulary (planner / generator / evaluator). The preprints.org survey carves the field up differently again. Everyone agrees there’s something here; nobody agrees yet on how to slice it. Normal for a young field, but worth flagging. Don’t tattoo any of these primitives onto your team’s process yet.

What this changes for you

If you maintain a CLAUDE.md, AGENTS.md, or any agent config, four moves.

Skim the abstract and the methodology section. Don’t bother with the full math. The abstract sets up the argument; the methodology section describes the NLAH+IHR layers and Table 10 maps the harness-engineering aspects onto NLAH carriers. Those are the parts practitioners actually reach for. Twenty minutes, tops.

Audit your config against the primitives. Read your CLAUDE.md and sort each sentence into roles, contracts, or verification gates. If you can’t, your config is doing too many jobs at once. Rewriting it with explicit headers takes maybe an hour and pays off the next time you onboard a teammate or swap models.

Adopt the word “harness.” It’s in arXiv now and your colleagues might recognize it. Saying “let me check the harness” is more precise than “let me check my setup,” and precision costs nothing.

Don’t over-invest yet. The vocabulary will move. Better to pick up the general shape than to lock into one paper’s runtime.

The paper did one useful thing for me: it named a category I’d been operating in for a year without realizing. A paper’s contribution can be quieter than a method or a result. This one’s was here is what to call this. Sounds modest. It is. It also made my code better the same week I read it, which is a higher hit rate than most of what I read.

I’ll keep calling it the harness. If a better word arrives next year, I’ll switch to that one.

References

• Natural-Language Agent Harnesses. Pan et al., arXiv 2603.25723, March 2026.
• Agent Harness for Large Language Model Agents: A Survey. preprints.org, April 2026.
• Harness Design for Long-Running Application Development. Anthropic Engineering.
• Anthropic’s Three-Agent Harness for Full-Stack AI Development. InfoQ, April 2026.
• awesome-harness-engineering. Community-curated list.
• 2025 Was Agents. 2026 Is Agent Harnesses.. Aakash Gupta, Medium.

---

Want to go deeper?

For a complete walk-through of harness engineering (the six building blocks, formal patterns, AGENTS.md design, and how to wire CLAUDE.md, skills, and hooks into a coherent runtime), see Harness Engineering: From Using AI to Controlling AI.