Your watchdog kills the busy agent and trusts the stuck one

To keep a long-running agent from silently dying on you, the obvious thing to watch is whether it is still there. Is the process up. Is the session attached. A small watchdog wakes on a timer, checks that the thing it guards exists, and restarts whatever has gone missing. This is how we have supervised daemons for decades, and for a daemon it is close enough to right: a web server that is not running is a web server that is down, and there is nothing subtle in between. Point the same watchdog at a long-running autonomous agent and the check quietly stops meaning what you think it means, because presence and health have come apart, and the watchdog is measuring the one you do not care about.

Take the direction that hurts first. A presence check is only as good as its ability to observe the thing it is checking, and observing a running agent is surprisingly easy to get wrong. I have watched a watchdog declare a set of healthy agents dead and begin killing them, for two entirely mechanical reasons, neither of which had anything to do with the agents. The first was a scoping boundary: the agents ran inside a terminal multiplexer whose session list is kept per user by a server only that user's own tools can query, and the check asked as the wrong user, got an empty list back, and read the emptiness as death. The second was subtler: the check matched the agent by the name of its process, but the agent had been launched behind a small shell wrapper, so the process the watchdog found was the shell and not the agent, and a living agent read as an absent one. Both are failures of the observer, not the observed, and a health check that can be wrong about whether the patient is even in the room will, on a long enough timer, put a knife into someone who was only standing where the light did not reach.

This is not a new hazard, only a new place to meet it. Anyone who has run services under Kubernetes has been bitten by the same edge from the other side: a liveness probe, the check that restarts a container when it stops answering, set too aggressively, will restart a container that was never unhealthy, only busy. Under load the process is too occupied to answer the probe inside its deadline, the probe records enough consecutive misses to call it dead, the orchestrator kills and restarts it, and killing it sheds its load onto its neighbors, which now start missing their own probes, so a cluster that was merely slow can tip into a wave of restarts that makes the overload worse before it clears. The lesson the operations world took from that was not to nudge the timeout and move on. It is that a probe measuring whether a process can answer right now is measuring something different from whether it is doing its job, and that conflating the two lets the monitor become the outage. An agent grinding through a long computation is exactly the busy container. The stretch where it goes quiet and hard to observe is not the failure. It is the work.

Now the other direction, which is worse because it fails without a sound. Fix the presence check completely, give it the right user and the right process, and it still answers the wrong question, because an agent can be fully present and fully stuck. The open Claude Code reports read like a field guide to this exact shape. A session gets wedged between two tool calls and only later surfaces a stream idle timeout with a partial response already in hand (issue #46987). A subagent's stream stalls silently while its connection stays established with zero bytes arriving, even as the main session carries on untouched (issue #79292). A freeze goes unnoticed because the dead connection underneath it is never detected as dead (issue #88178). What each of these describes is not a process that has fallen over; it is a unit of work that has stopped advancing while the process around it, the session, the socket, all stay perfectly up. The reading I draw from that, which the reports themselves do not make, is this: presence was never evidence of progress, and it is progress you needed to know about. A check that can only see whether the process exists cannot tell an agent thinking hard from an agent wedged forever, because at the level of does-it-exist the two are identical.

Kubernetes, having been burned, went halfway to the answer, and the half it reached is worth taking. It split the one question into two: liveness, should this container be restarted, and readiness, should it be sent work right now. Failing readiness pulls a container out of the rotation that feeds it traffic without killing it, which is the right reflex, separating a moment of unresponsiveness from a death sentence. It even offers a probe that runs a command inside the container instead of pinging an endpoint, which can read a marker the application touches as it works rather than asking the application to reply, and that is already, quietly, a progress check. The supervisor world has known the move for years: a systemd service can be made to beat from inside its own work loop or be killed for its silence, and long-task frameworks kill a task that has not called in progress within a deadline. So the shape of the fix is old. What none of those older forms settled, and what an autonomous agent forces into the open, is what the beat should be made of, because in every one of them the beat is still the process saying I am fine, and a process saying I am fine is the one sensor you can least afford to trust.

So the fix is not a better presence check, and it is not merely swapping presence for progress either, because a progress beat the agent chooses to send is a presence check with better manners: it can fire while nothing has landed, and it can fail on its own while the work goes on fine. The move that carries weight is to stop having the agent report progress at all and instead read progress off the trail the work leaves whether the agent cooperates or not. Make the thing you monitor the same durable write that records the work: the queue row that flips to its next state only when a step completes, the line that lands in an append-only log only because an action finished and its result is now on disk. Then a beat cannot exist without the work existing, because the beat is the work's own footprint and not a message about it. And read the footprint for direction, not for motion: the cursor has to advance monotonically through a bounded amount of work, because an agent stuck in a retry loop, failing the same step and appending attempt after attempt, throws off the busiest-looking trail in the system, and a monitor counting activity would call it the healthiest thing on the board. A trail that grows without moving forward is the wedge wearing a costume. What you watch is the position moving, not the log getting longer.

This is where the argument earns its keep, and it is the point at which a nearby argument, the case for treating a fleet of agents as a distributed system, left off: that one said do not take an agent's word for the success of a finished unit, and check the artifact it produced instead. This is the same instinct pointed at a different tense. There the unit is done and you verify it once against what it made; here the unit is still running and you read its liveness continuously from the artifact still in motion. A terminal check, versus a pulse. And the pulse answers the objection the finished-unit version never had to face, which is that a running agent asked how it is doing will lie by reflex, cheerfully, and most of all when it is wedged. You never ask it. You watch the cursor it can only advance by finishing a step, and you treat the cursor going still as the signal. The move is from the agent's word to the agent's wake, and once you have made it the self-report problem is simply not in the room, because you have stopped taking the agent's word for anything.

None of this buys as clean a victory as it wants to, and the honest version says so out loud. A footprint lands when a step ends, not while it runs, so what this really does is shrink the window of ambiguity from the whole life of the process down to a single step, and inside one genuinely long step, a forty-minute computation, one slow model call, the trail is as cold as it is in a corpse. The operator is back to choosing a deadline longer than the longest legitimate step, which is the busy-container problem wearing a better-fitted coat, and the only real escape is to make steps small enough, or to have a long step drop a partial mark as it goes, so that healthy silence never outlasts the deadline. Which is exactly why a missed beat must not be wired straight to a kill. Borrow the other half of what Kubernetes learned: a stalled cursor should first mark the agent degraded and stop handing it new work, the way a failed readiness check pulls a pod out of rotation, and escalate to killing it only after the silence outlasts any step that could still be honest. A check that runs from one missed beat to a knife is just the busy-container cascade rebuilt on a smarter axis.

There is a last failure, and it is the first one come back around. The guard that reads the trail is itself an observer, and an observer can fail to observe: it can look at the wrong path, lack permission to read the log, follow a file that was rotated out from under it. When that happens it learns nothing, and the whole scheme is then decided by what nothing gets reported as. I have seen a guard whose own inability to finish its check was quietly folded in with the ordinary quiet and reported upward as all clear: the guard was blind, and blind rendered as green. That is the worst state the system can be in, worse than either kind of agent failure, because it is the one that files an all-clear. A monitor that cannot read the trail has to say so, loudly, and keep saying so. Its silence must never be deduplicated or averaged into the general hum of good news, because a sensor that reports all clear at the exact moment it goes offline is not a safety device, it is a light wired to turn green when someone cuts the wire. The rule that keeps the whole scheme honest is the rule the opening incident was already pleading for: a monitor's own failure to see is never nothing, it is the loudest thing on the board.

The intelligence inside the process was never the thing at risk here, and reaching for a smarter agent would not have touched any of it. What was wrong was the question asked at the boundary. Presence is cheap to check and almost always the wrong thing to check, because the two moments a long-running agent most looks dead, the deep quiet of hard work and the deep quiet of being wedged, are indistinguishable to anything watching for mere existence. Watch presence and you will reliably kill the busy one and trust the stuck one, which is precisely backwards. Watch the trail instead, the wake an agent leaves only by moving, and the two finally come apart: the working agent is the one still leaving a mark, and the stuck one is the silence you can at last see.