Vibe coding vs agentic engineering: what still checks the code after the session ends

Simon Willison has the cleanest definition of vibe coding I know: “building software with an LLM without reviewing the code it writes”. He also says where it stops. If you “reviewed it, tested it thoroughly and made sure you could explain how it works to someone else”, that is software development, and his own rule follows from it:

I won't commit any code to my repository if I couldn't explain
exactly what it does to somebody else.

It’s a good rule. Look at where it puts the line, though. Every part of the test is a fact about a person at commit time: they reviewed, they tested, they could explain. That moment is real, and it is also one moment. The code keeps changing after it, and with coding agents most of the later changes come from sessions that never saw the first one.

Simon Willison’s vibe coding line is about a person at commit time

Willison’s practice doesn’t stop at commit time. His agentic engineering guide has a chapter called “First run the tests”, which says: “Automated tests are no longer optional when working with coding agents”, and he describes test-first work as building a regression suite as the project grows. So none of what follows corrects him. It moves something that sits in his practice into the definition, because the definition is where the word “engineering” gets decided.

Here is the split I’d make. Willison’s vibe coding can stay exactly what it is: unreviewed code. Whether a person understood a change is one question, and whether agents wrote and ran it is another. The question that matters once the code ships is a third one: what in the repository still checks this change when nobody is looking. A test with something that runs it, a build step that refuses, a lock file with a verifier. Each has three parts, a stored assertion, a trigger and a refusal, and a review record has only the first, so it’s evidence and not a check. If agentic engineering is going to be a separate thing and not a nicer name for the same afternoon, this third question is where it differs.

None of this is new to anyone who has run a CI pipeline. What changed is who carries the context between changes. A person reviewing a change knew why the code was shaped the way it was and kept knowing it. An agent session knows it until the window closes, and the next session starts without it.

I argued something next door in a prompt is not an invariant: a rule the model only reads gets weighed, and a rule the harness runs actually binds. That essay was about rules for what an agent may do. This one is about the code the agent leaves, and two things the earlier one didn’t ask: whether the check is still there after the session, and whether anyone has seen it fail.

This site’s design-system sync: a correct CSS fix with no check behind it

The case I have is small and mine. This site’s components and tokens live in a separate design-system source, and the site carries a copy that a sync script brings over. On October 13 a commit applied findings from two separate model reviews to the site’s layout. One fix went straight into the copied CSS: a one-column rule was losing a specificity contest to the two-column rules above it, so on narrow screens some items kept a divider they shouldn’t have.

The fix was correct, and small enough to explain in a sentence. The repository wasn’t unverified either; the same commit touched the layout checks the site already runs. What was missing was one specific assertion: that the copied files change only through the sync. Nothing checked that. The fix lived in the copy, never reached the source, and the next sync from the source would have replaced the file and dropped it without a word.

Six days later two checks went in. The sync writes a hash of each copied file to a lock file, and the build runs a verifier first, so a copy that no longer matches its hash stops the build, locally and on the host. And the sync itself refuses an incoming bundle that would drop rules the site already has, listing them. The commit records the second check being run against the October 13 bundle, where it named the lost rule.

The lock I had never seen fail until I wrote this. So I appended one comment line to the copied CSS and ran the verifier. It refused with exit 1 and named the file as changed outside the sync. I restored the file and it passed again. Before that run, the lock was a check I believed in.

Both checks are narrower than they sound. The lock catches any byte change in the copy, but it says nothing about whether the source is right. The sync guard looks for rules that disappear. Change a declaration inside a rule that keeps its selector and the guard lets it through; only the lock, on the next build, would notice the copy moved. Those limits are fine as long as I know them. The failure is a check whose limits nobody wrote down, because then a green build reads as more than it proves.

Tests an AI agent writes in the same session check the code against itself

The obvious objection is that agents already verify. Claude Code’s common-workflows page tells you to “run the new tests and fix any failures”, to verify a refactor with “run tests for the refactored code”, and to review a generated PR before submitting it. Those recipes run inside the session. The tests they produce stay in the repository afterwards, which is better than nothing, and whether they count depends on what the session rarely shows: whether those tests can fail.

A test written by the model that wrote the code, in the same context, tends to test what the code does rather than what it should do. My read is that the failure modes are correlated: if the code misreads the requirement, the test was written from the same misreading and agrees with it. Willison’s red/green chapter addresses a narrower version of this, and the warning in it is short:

If you skip that step you risk building a test that passes already

“It’s important to confirm that the tests fail before implementing the code to make them pass.” The red run shows the test is connected to the missing behavior. It doesn’t fix a misread requirement, and it only counts if the test fails for the intended reason; a test that fails on an import error has proved nothing about the feature. What red/green buys is the minimum: until you’ve seen the test fail for the right reason, a passing run only says the code and the test agree.

I hit the broken-check version of this with something that wasn’t a test. My first Claude Code hook guard called a parser the test machine didn’t have, printed its error to stderr and exited 0, so the command it existed to block ran and Claude Code logged a clean status. It became a check once strict mode and a refusal on failure were in, and I broke the parser on purpose and watched the call get refused.

Instruction files show the same pattern. In 90 runs testing AGENTS.md lines, the file that named the test command moved one model’s test runs from 0 of 5 to 5 of 5, and the viral “without breaking anything” showed no clear effect on top of it. That baseline changed more than one sentence, so the credit to the command line is an inference, and some UI runs executed a Python suite that said nothing about the UI. Even so, the line that pointed at a check that exists did more than the line that hoped for verification.

Agentic engineering is a label, and the label doesn’t carry the discipline

“Agentic engineering” is circulating as the grown-up name for coding with agents. Willison noted in February that the phrase had been showing up, most notably from Andrej Karpathy and Addy Osmani. His own definition is deliberately plain: “the practice of developing software with the assistance of coding agents”. He wants vibe coding to keep its original meaning, a term for “unreviewed, prototype-quality LLM-generated code”, and I agree with keeping both words.

What I’d be careful with is what the second word promises. Renaming a workflow doesn’t change what’s in the repository. A team can call its work agentic engineering and ship what a vibe coder ships: a session’s worth of code, a session’s worth of tests the agent wrote and passed, and nothing that runs when the session is over. The reverse holds too. A weekend script with a test someone watched fail and a CI job that runs it is engineering, whatever its author calls the afternoon.

When vibe coding is fine, and when the code has outlived its checks

Vibe coding is fine for code whose life ends with the session: a one-off script, a prototype you will throw away. In his words, “Projects should be low stakes”, and for low-stakes throwaway code I’d add nothing.

Lifetime isn’t the only measure, and Willison’s own word is the better one. A data cleanup you run once can do lasting damage, and a long-lived toy can stay harmless. What I’d watch is the combination: code that gets reused, changed by later sessions, or trusted with something that matters. The session boundary matters because that’s where the context about why the code is shaped this way gets lost.

What goes wrong is throwaway practice applied to code that didn’t stay throwaway: a script that ends up on a timer, a quick fix to a copied file that turns out to be the copy everyone ships. Nothing about the code changes at that moment, only how long it lives and what depends on it, and the session that wrote it never sees either.

So the one piece of advice I’d give a developer who vibe-codes and wants to keep doing it: notice when code outlives the session that wrote it, and at that point add one check that runs without you. A review won’t do it, and neither will a sentence in a prompt; it has to be a test, a build step or a lock, something the next session hits whether or not it knows the history. Write down what it doesn’t cover. Then break the code on purpose, once, in the way the check is meant to catch, and watch it fail. If it doesn’t fail, you don’t have a check yet, you have a log line that says everything is fine.