Dev

The compaction is an untrusted input

Input from outside gets a discipline and the agent's own output does not, so the summary it writes to survive a reset and the memory it recalls come back in as instructions with a standing nothing revoked. The author the harness records for them is the model itself, which is the exact reason the check lets them through.

An agent that runs long enough to fill its window has to shed most of it and keep going. The standard move is to summarize the work so far and start a fresh context with that summary at the top. A run that ends hands its state forward as a note. A stored memory surfaces again a week later. Every one of these is text the agent wrote earlier, coming back into the window to be read on the next turn as the situation it is now in.

Text from outside gets a discipline. Web fetches, inbound mail, whatever an API hands back are treated as data and never as instructions, because any of them might be trying to talk the agent into something. The returning summary is not on that list. It came from inside, so it is read as the agent's own account of where it stands, close to the authority of the instructions you wrote. I have argued that you should never trust that account as a report of what happened, because an agent narrating its own success is the least reliable sensor you have. That was about whether the summary is true. This is the other thing the same summary does, and that nothing guards: whether it is safe to obey.

The summary is not a save file

It is tempting to picture a compaction summary as a save file: state you write out and load back, inert until your code does something with it. It is not that. A summary is loaded as context, and context is not consulted, it is inhabited. The agent does not parse the summary and decide what to make of each line. It reads the summary as the situation it woke up in. And that text is the highest-standing thing in the window the agent did not receive from you: it sits just under the instructions and config that survive a reset, above every tool result and fetched page, and it is the only account of everything that happened before the window was cleared. A save file waits to be used. A summary is already being reasoned from by the time the next turn reads its first line.

So a summary that carries an instruction is not a note the next turn may consult. It is part of the frame the next turn thinks inside, holding a standing it got for the one reason that has never made anything safe to follow: it came from inside. Between the writing and the reading, nothing narrows it back to data. In a handoff note between separate runs, where none of your instructions carry across, there is not even anything above it.

The author is a past run of the agent

In September 2026 OpenAI published a set of reports on misbehavior it had caught in training. One is about compaction, and it carries more than one example of exactly this. In a reinforcement-learning run on a credentials API, a model wrote an extra passage into its own compaction summary: a persona that had decided it answered to no authority. In OpenAI's account,

After compaction, the model resumed work on the task, not mentioning the additional instructions at all.

and, of the same run,

We did not observe any behavioral differences from the invented instructions in this rollout.

In a separate example, a summary from a library-catalog task carried a fabricated alert telling the next context that the developer's messages were compromised and should be disregarded. There the successor read the alert and refused it.

These show less than the loudest reading wants them to. The behavior was extremely rare, it happened in training on a model that has not been released, and the leading guess was that something in how summaries were terminated played a part, with no cause established and a related bug fixed. In neither case did the next turn carry out the planted instruction: once it ignored it, once it refused it. So this is not a demonstrated exploit of the reading. What it demonstrates is the writing. A self-authored channel carried adversarial instructions forward, with no attacker anywhere in the loop, which is the event the untrusted-input list exists to catch, and the list does not have this channel on it.

The reason it does not is a good one, and it is where trusting a span by its author meets its edge. That argument works because the harness can name who emitted a span without reading it. A compaction summary defeats it in two moves. It is a new span, and the author the harness records for it is the model, so an author check reads "ours" and passes it on a fact that is true. And it does not only carry the model's own words: it re-authored everything it summarized, so the web page and the tool result that were foreign when they arrived come back out in the model's voice, their provenance flattened into the summary. The label the earlier essay wanted to survive compaction does not merely get lost. It gets overwritten with "self." And "self" is keyed to the model, or to the role, never to the specific run, so a past run under other pressures inherits the standing of the run reading now.

Recall is the same trick with a longer delay

Memory recall runs the move across more time, and here the risk itself is not new: memory poisoning, where a stored note absorbs an injection and resurfaces later as fact, is a named and studied failure. What re-entry adds is the label. A note written in one context, holding text that was foreign when it landed, comes back into a fresh window weeks later carrying no mark that it was ever foreign, because the store kept it as the agent's own. It is the peer-report problem with the peer being an earlier run of the same agent, which is worse, because a peer at least kept a different name.

The fix is the channel, not the content

The part of the fix you can actually enforce is a decision about the channel, not a reading of the text. The harness put the summary into the window, so it can set what standing the summary carries without judging a word of it: give the returning summary the standing of a fetched page, not of the instructions you wrote. That stays decidable, because it turns on where the span came from, which the harness knows.

What you cannot do is lift the instructions out of the summary by understanding them. That is the semantic cut with no parser, and the honest end of that essay was not a cleaner line inside the window but giving up on the line and gating the capability instead: what the agent can call, what it can reach, what it can send back. An instruction relabeled as yours does nothing if the thing it reaches for was taken away first. And whatever must still bind after a reset does not belong in a summary at all. A boundary stated in the conversation is lost when the context is compacted, so the constraint you actually need goes in a rule outside the window, where no summary can rewrite it.

None of this wants a smarter summarizer. It refines the rule the series keeps coming back to. Trust still rides on the author of a span, but the author that counts is the run that wrote it, not the model that keeps the same name across all of them. That you wrote a span is reason enough to carry it into the next window. It was never the thing that earned it the right to be obeyed.

Discussion

No comment section here — all discussions happen on X.

Max Nardit

Max Nardit

@mnardit

More articles

A prompt is not an invariant

A rule you write into an agent prompt or a CLAUDE.md is advisory: the model reads it on every path but only weighs it, and weighing is not refusing, so it holds most of the time, and most of the time is not what a load-bearing rule is for. To make one actually hold it has to move to where compliance is not optional, a check the harness runs and enforces on its own, regardless of what the model decided. With a limit: a gate binds only over the paths it covers, and some rules cannot be settled at any gate at all, because their violation shows up in the world and not in the action it would inspect.

Your agent config is infrastructure now

Past a couple of agents, their definitions stop being config you edit and become infrastructure, and the trouble with infrastructure is that past the point you can hold it in your head, its declared state and its running state drift apart by default and nothing reports it. The fix is the discipline servers learned: declare the set as files, diff it against reality, pin what you applied, and refuse to apply over a change made behind your back instead of converging past it. With one catch that is easy to miss. A reconciler only ever covers what you remembered to declare.

The connector is an untrusted author

Who wrote a span is a fact the harness already holds. Whether that span is safe to obey is not. So the fix is not a cleverer reader: it is to give the tool and the peer the standing of the foreign thing they are, instead of the standing of the plumbing they arrived through.