OpenAI Codex AGENTS.md: 1M Lines Shipped, 3 Harness Lessons

I spent a week blaming the model.

My Codex runs kept blowing past their budgets, my agents kept editing the wrong files, and every morning I woke up to another PR that “almost” worked. I bought the hype, swapped Sonnet for GPT-5, then swapped it back, then tried a bigger context window, then tried a smaller one. Nothing moved.

The bug was one paragraph deep in my AGENTS.md.

OpenAI published “Harness engineering: leveraging Codex in an agent-first world” in February 2026, and the Codex team’s public numbers tell the same story I lived that week. Between August 2025 and January 2026, three engineers drove roughly 1,500 merged PRs and shipped on the order of a million lines of production code with zero lines written by hand. The model did not change halfway through. The AGENTS.md did, twice.

Here are the three things the experiment proved.

Lesson 1: The “one big AGENTS.md” collapses under its own weight

The Codex team’s first move was one big AGENTS.md covering the whole repo. Coding conventions, module boundaries, review checklist, deployment runbook, all in one file. It reads like the right thing to do.

It failed in the most predictable way possible. Context is a scarce resource, and a giant instruction file crowds out the task, the code, and the documentation the agent actually needs to look at. The team wrote about this directly: the encyclopedia approach lost, because the encyclopedia was the whole reason the agent stopped reading the code.

The fix was to shrink AGENTS.md to a table of contents (roughly 100 lines) and move the real knowledge into a docs/ directory treated as the system of record. AGENTS.md no longer answers questions; it points at where to look.

I ported the same shape into my own harness the day I read the post. My previous AGENTS.md was 940 lines. My new one is 118 lines and looks like this:

# AGENTS.md

## Overview
FastAPI + Postgres backend for the internal billing service.

## Structure
- `app/` → FastAPI application
- `tests/` → pytest suite
- `docs/skills/` → task-specific skills

## Key Skills
- Add an endpoint → docs/skills/add-endpoint.md
- Write a migration → docs/skills/write-migration.md
- Debug a production incident → docs/skills/oncall-triage.md

## Constraints
- All endpoints must have contract tests
- Never inline SQL — go through the repository layer
- PRs are squash-merged

Every constraint in the old file that used to sit in AGENTS.md now lives in the skill file that actually needs it. The agent loads it only when it enters that task.

The regressions I had been fighting stopped that afternoon. Not because the model got smarter. Because I stopped hiding the code from it.

Lesson 2: Swapping models fixes nothing when the harness is the bug

This is the uncomfortable one.

For a week I A/B-tested models. I tried GPT-5, I tried Claude Sonnet 4.6, I tried Haiku for the cheap runs and Opus for the hard ones. My completion rate on the same fifteen tasks moved by less than four percentage points across all four.

I switched to a smaller model but a cleaner harness: a 118-line AGENTS.md, hooks that ran linters before every commit, and a sandbox that refused writes outside app/. Completion jumped by 31 points on the same fifteen tasks.

The Codex team’s phrasing landed differently after that: “Agents aren’t hard; the harness is hard.” I read it a month before I believed it, which is roughly the industry-average delay.

Louis Bouchard put it more bluntly in his 2026 write-up: stop saying “the model is dumb” and start saying “my system tolerated this failure.” That reframe is why the Codex team keeps the same model checkpoint for weeks at a time and treats every regression as a docs/ update, not a model swap. When the harness is the thing that changes, the model becomes a stable variable, and you can actually reason about what broke.

Lesson 3: AGENTS.md, Anthropic Skills, and .cursorrules do different jobs

The vocabulary is confusing on purpose. Every vendor picked a different noun for a similar shape.

Here is what actually differs, based on what I ship with each:

Property	OpenAI AGENTS.md	Anthropic Skills	Cursor .cursorrules
Primary role	Repo-level index for any agent	Task-scoped, lazy-loaded playbook	Editor-level rules per project
Scope	One per repo	Many; one per task type	One per project
When loaded	Every session, top of context	Only when the task matches	Every prompt in this repo
Failure mode if oversized	Context crowded, agent ignores code	None — it never fires	Rules bleed into unrelated prompts
Best for	Where-to-look pointers	How-to-do-X procedures	Style + safety rails

They compose. On my own harness I keep a slim AGENTS.md as the top-level index, Anthropic-style skill files under docs/skills/ for anything task-specific, and a short .cursorrules for the two or three editor-scope conventions that survive across every task (never introduce any, never touch main.py without permission). The AGENTS.md is not competing with skills; it is announcing where the skills live.

The failure mode I still see most often in other people’s repos is putting the skill content inside AGENTS.md “so the agent always has it.” That is the encyclopedia problem, one level down. If it always has it, it never reads the code.

What actually changed for me

Three concrete moves that came from reading OpenAI’s post and porting the shape:

1. AGENTS.md down from 940 lines to 118 lines. Everything else moved into docs/skills/.
2. Model held constant for four weeks. Any regression writes a new line into a skill file, not a switch to a new checkpoint.
3. A harness/hooks/pre-commit.sh that runs the linter and the type checker before the agent is allowed to move on. This one I stole from Anthropic’s long-running-agent post from April, but it belongs on this list.

Completion rate on my personal fifteen-task benchmark went from 41% to 78% across the three changes. The model has been the same Sonnet 4.6 checkpoint for that entire span.

The uncomfortable truth of the Codex experiment is that “which model” was the wrong question the whole time. The right question is which harness this model is wearing. If you have been swapping checkpoints and blaming the vendor, try shrinking your AGENTS.md first. It’s cheaper, and it’s usually the actual bug.

---

If you want the longer version of this (the six components of a harness, why AGENTS.md is only one of them, and how the same shape applies to CLAUDE.md and hooks), that is what my book is about.

Harness Engineering: Building the Environment That Makes AI Agents Reliable

Sources: OpenAI — Harness engineering, Anthropic — Harness design for long-running app development, InfoQ — Anthropic three-agent harness.