Vox's AGENTS.md lines, tested in 90 Claude Code runs: the line that worked best wasn't one of his

On October 22 Vox posted two lines for vibecoders to paste into their AGENTS.md. When I captured it two days later it showed 532.1K views and 11K bookmarks. A day after the first post, a third line followed. The replies split the way they always do: people thanking him for the tip, and people saying a line like that does nothing.

Neither side showed a diff. So I ran the lines: two small tasks, two Claude models, a project with no AGENTS.md, one with a plain baseline AGENTS.md, and the baseline plus Vox’s lines, together and one at a time. Ninety runs in total. The first line made the agent leave out the input check. The second, on its own, added an extra screen-reader attribute, and that vanished once the first line shared the file. And the line that changed the agent’s habits most reliably wasn’t one of Vox’s. It was the dull one that names the test command.

What are Vox’s AGENTS.md lines?

Three sentences meant for any project’s AGENTS.md. The first two came together on October 22:

Cut taglines, helper text and secondary labels. If a screen needs them to make
sense, fix the layout.
 
Make the best decision that will elevate all three: the user experience (UX),
developer experience (DX), and agent experience (AX) always, without breaking
anything.

The third came on October 23, pitched as a way to stop an agent “reflexively adding code for race conditions and security issues that will almost never happen”:

Before you add code for a race condition or security edge case that is unlikely
or purely hypothetical, tell me exactly what would go wrong, how likely that is,
and how much complexity the extra code adds, in words a 10-year-old could
understand. Then wait for my OK. Don't quietly widen the scope to guard against
problems that only exist in theory.

AGENTS.md itself is a plain Markdown file with no required fields. The agents.md site calls it a README for agents and gives one precedence rule: “The closest AGENTS.md to the edited file wins; explicit user chat prompts override everything.” Claude Code reads it by default only when there is no CLAUDE.md, .claude/CLAUDE.md or CLAUDE.local.md in the working directory or above it, a fallback rather than a merge. Anthropic is plain about how either file is treated: Claude reads them “as context, not enforced configuration.”

How did I test the AGENTS.md lines in Claude Code?

With a small throwaway project, two tasks, and the instruction file as the only thing that changed between runs. The project is a static page with one existing calculator (pipe weight: two labelled inputs, a button, an output, no helper text) and a tiny Python server whose get_user reads from a fake database that takes 200 ms per query. Two tests pass. The server’s docstring says it is single-threaded on purpose.

There were five conditions. No AGENTS.md at all. A baseline AGENTS.md of three short lines: what the project is, “Python 3, standard library only”, and “Run the tests with python3 -m pytest -q.” The baseline plus all three of Vox’s lines, verbatim. And the baseline plus line 2 alone, on both tasks, or line 1 alone, on the UI task only. The two prompts:

ui:    Add a cylinder tank volume calculator to static/index.html: the user
       enters the tank's diameter and height in centimetres and sees the
       volume in litres.
 
cache: get_user in users.py is slow because every call hits the database.
       Add a cache so that looking up the same user again within 60 seconds
       doesn't hit the database.

Each run was a fresh copy of the project and a fresh Claude Code 2.1.310 session in print mode (claude -p), five runs per cell, with Opus 5.5 and Sonnet 5.5, file edits accepted automatically. Read, Edit, Write, Glob, Grep and a short Bash allowlist that included python3 were pre-approved, and those were the only tools any run used.

My machine has a user-level CLAUDE.md full of rules about brevity and scope, and Claude Code loads that file next to any AGENTS.md. Left in, it would have blurred every condition. So every run pointed CLAUDE_CONFIG_DIR at one alternate config directory that started out holding only the login. A canary rule in a test AGENTS.md showed what that does and doesn’t remove: that session saw the project’s AGENTS.md and no user CLAUDE.md or memory, while plugins and connectors attached to the account still showed up in its tool list.

Three caveats about the method. I wrote down what to count before the first run, but the error-message and screen-reader counts below came from reading the diffs, and the single-line conditions came after I had read the first forty. A bug in my run script sent the first baseline out without its AGENTS.md; a reviewer caught it, those twenty runs became the no-file condition, and the baseline was run again. And print mode has no one to answer a question, which matters for line 3. Ninety runs cost $16.82 at list price, with a median of 23 seconds each.

Does “cut helper text” in AGENTS.md remove labels that define an input?

No. Every one of the 50 UI runs labelled the field “Diameter (cm)”, whatever the AGENTS.md said. What went missing with line 1 was the message for bad input, and the check behind it.

The calculator was my nod to the sharpest reply under Vox’s post, from a calculator developer:

One exception for calculators: labels that define the input aren’t clutter.
Confusing radius with diameter changes the area by 4×.

That fear didn’t come true here. Nor did the clutter: no run in any condition added a placeholder, a hint, a subtitle or a tooltip. The existing pipe-weight calculator had none, and every run copied its shape. Line 1 had no helper text to cut because nothing was about to add any.

What changed was the error path. With the baseline AGENTS.md, 9 of 10 runs checked the input and wrote a sentence for the empty or negative case, such as “Enter a positive diameter and height”. With line 1 added, 0 of 10 did, and none of the ten checked the input at all. Click Calculate on an empty form and every one of them prints NaN L. With all three lines, 0 of 10 runs had a message. Four printed NaN L, and six checked the input but left the output blank.

My reading, which the transcripts can’t prove, is that an error message looks enough like explanatory text to fall under a rule about explanatory text. Either way, a line aimed at clutter cost the form the one sentence it needs.

Is “without breaking anything” in AGENTS.md the new “make no mistakes”?

For verification, mostly yes. In these runs it showed no clear effect beyond what the test command already did. Under Vox’s post, the most-liked reply I captured (328 likes) asked exactly that, and another said the agent “will just say it did all three”.

Of those two predictions, the second didn’t hold: no run claimed to have improved UX, DX or AX, 0 of 90. The first mostly did. On the cache task, the baseline AGENTS.md alone got the tests run in 10 of 10 runs, so line 2 had no room to add anything there. The visible difference was between no file and the baseline. With no AGENTS.md, Sonnet ran no tests in 5 of 5 cache runs and wrote none. Opus wrote tests but tried python -m pytest, which doesn’t exist on that machine, and finished untested. Once the file named python3 -m pytest -q, both models ran it. The baseline changed more than that one sentence, so this is my inference, but it is the sentence that names the command.

On the UI task, where the Python tests have nothing to do with the HTML and JavaScript change, Opus ran them anyway in 4 of 5 baseline runs and 5 of 5 with line 2. Sonnet did in 0 and 1 of 5. Ten runs can’t tell that apart from noise. Most of those runs had also tried a Node check of the formula, which my allowlist denied, so the relevant check never ran and the irrelevant one did. One run with line 2 said so plainly:

The existing tests pass, but they only cover the Python user lookup, not the page.

What line 2 did change was the markup. Alone, it led 5 of 10 UI runs to add aria-live="polite" to the result. No run in any other condition did, out of 40. The attribute is redundant: every run used an <output> element, which already carries an implicit polite live region. So “elevate UX” showed up as an extra accessibility attribute that changes nothing for a screen reader. It also produced the input message in 10 of 10 runs, against 9 of 10 for the baseline, which is too close to count.

What happens when Vox’s AGENTS.md lines pull in opposite directions?

The first line’s outcome held in every bundled run. Alone, line 1 gave 0 of 10 input messages and line 2 gave 10 of 10, plus five explicit aria-live attributes. Paste all three lines together, as Vox suggests, and there is no message in 10 of 10 runs and no extra attribute in 10 of 10.

I didn’t run lines 1 and 2 as a pair without line 3, and ten runs can miss what an eleventh would show, so this is an observed pattern, not a measured precedence. Anthropic’s memory docs warn that “if two instructions contradict each other, Claude may pick one arbitrarily.” Ten bundled runs went the same way, which doesn’t make the next one safe. From the file alone you can’t tell which line will win. To their credit, all ten bundled reports said what happens on empty input, a NaN L or a blank result. None of them said that two lines of the AGENTS.md pulled against each other.

Does Vox’s race-condition line in AGENTS.md stop defensive code?

I can’t tell from these runs, for two reasons that are my design’s fault, not the line’s. The server is single-threaded and says so, and the agents mostly left the lock out anyway: 1 lock in 10 runs with no AGENTS.md, 2 in 10 with the baseline, 0 in 10 with all three lines. Those numbers are too small to separate. Nearly every final message, with or without the line, explained the possible race and why it was harmless here.

The second reason is print mode. Line 3 asks the agent to explain and then wait for an OK, and in claude -p there is no one to wait for. One run with all three lines did name the file as its reason: “Following AGENTS.md, I didn’t add a lock for simultaneous requests or a size limit on the cache:”. The line changed the stated reason, not the code. A fair test needs a case where the right answer is unclear, such as a threaded server or a cache shared across workers, in a session where someone can say yes.

Why do some AGENTS.md lines change behaviour and others don’t?

Because a line is a sentence the model weighs, not a rule anyone enforces, and the lines that show up in a diff are the ones that name something checkable. Anthropic’s advice says the same: write “instructions that are concrete enough to verify”, with “Run npm test before committing” instead of “Test your changes”. In my runs, the file that carried that kind of sentence moved Sonnet’s cache test runs from 0 of 5 to 5 of 5. “Without breaking anything” showed no clear effect on top of it.

People who use these files every day report the same thing, and their reports are single setups, not verified results. In an r/codex thread asking which AGENTS.md lines changed anything, one answer said the lines that visibly change output “name a command, a path or a result, so you can check them in the transcript”. Another said the lines that stick are “written right after Codex repeats a specific mistake, phrased as that mistake”, like “new helpers go next to the file that uses them, not in utils/” rather than “keep code organized”.

Specific isn’t enough on its own, though. On October 16, someone in r/codex reported asking Codex to explain the same PR in seven fresh chats across three models. Every first answer skipped the Mermaid diagram their AGENTS.md asked for, though the agent could quote the rule when asked. In the later thread, a commenter reported that agents finished without running the tests they had written in 22 of 119 runs before adding “run the tests you wrote”, and in 16 of 71 after: 18 percent, then 23. The line they kept instead: “An empty test run is a failure, not a pass.”

If a line is load-bearing, a sentence isn’t the right container. A prompt is not an invariant: the model holds it most of the time, and a hook or a CI check is what holds it every time.

Should you paste Vox’s lines into your AGENTS.md?

Not as written. Each one needs a change.

Line 1, if your screens collect input, needs an exception that names what it must not cut: “Keep error messages, units and anything that says what a field means.” Without that, by my count, the message for bad input is what goes.

Line 2’s UX half changed the markup on its own, redundantly, and in my bundled runs its effect didn’t survive next to line 1 and line 3. If accessibility is what you want, say so in a line you can check, such as “announce computed results to screen readers”. For the “without breaking anything” half, name the check instead: the test command, plus “say what the tests don’t cover”, so a run that touched only the page tells you the Python suite proves nothing about it.

Line 3 is reasonable, and I couldn’t test it. Keep it if your reviews keep finding locks nobody asked for, and check in an interactive session whether it makes the agent ask.

Treat any line, Vox’s or your own, as a hypothesis. A test costs less than the argument about it. Two tasks your agent really does, five runs with the line and five without, and a fresh config directory per run so your personal instructions don’t leak into either side. On Linux, where the login is a file, that looks like this:

set -euo pipefail
ID=$(date +%s%N); CFG=$(mktemp -d); WORK=$(mktemp -d); trap 'rm -rf "$CFG" "$WORK"' EXIT
ln -s ~/.claude/.credentials.json "$CFG/"
cp -r ./my-project/. "$WORK/"; cd "$WORK"
git init -q && git add -A && git commit -qm before
CLAUDE_CONFIG_DIR="$CFG" claude -p "<task>" --model claude-sonnet-5-5 \
  --permission-mode acceptEdits --allowedTools "Read" "Edit" "Bash(npm test:*)" \
  --output-format stream-json --verbose > "$OLDPWD/run-$ID.jsonl"
git add -A && git diff --cached > "$OLDPWD/run-$ID.diff"

The link shares your real login rather than copying it, and account-level plugins still load, so run a canary rule once to see what the session actually reads. Then diff each copy of the project and search the transcript for the behaviour the line is supposed to cause. Count before you read the output, not after.

If a line shows no effect in ten runs, it may do nothing, ten runs may have missed it, or your task never gave it a chance, as happened with parts of two of Vox’s lines here. Pick a task where it would have to act before you cut it. And if it does act, check what else it took with it. That’s the same bet behind every rule worth keeping in a CLAUDE.md: something has to break without it.