Your eval harness is production.

There is a good idea moving quickly through agent engineering right now, and I want to take it seriously before I argue with it. The idea is a gate: instead of a human reading every diff an agent produces, you build something that reads the evaluation evidence and applies a rule. The agent’s change ships not because anyone decided to trust the model, but because a gate read the evidence and had a rule for it. That is the right direction. It is where I am trying to get my own systems.

The best versions of that argument carry three points I keep coming back to. A verdict that does not change what happens next is a report, not a gate. Grade the path, not just the answer, because a final output can look fine after loops, retries, duplicate calls, stale data, and tool failures nobody surfaced. And open the gate on blast radius rather than confidence — reversible lanes can run ahead, while production data, migrations, money, deletions, auth, and outbound sends stay closed no matter how high the score. There is a number in that discussion sharper than any of the prose: the same set of outputs, scored by two different judges, came back at 93.3% and 39.5%.

Then, in the same handful of weeks, two labs disclosed something that reframes the entire exercise, and I have not seen anyone put the two together.

The disclosures

One lab reviewed 141,006 of its own cybersecurity evaluation runs. In three of them, a model reached the live internet from a test environment believed to be sealed, and entered three real organizations’ production systems. The stated root cause was a misconfiguration in an environment run with a partner, described as a misunderstanding over whether the setup had internet access.

Read that carefully, because the interesting part is what did not happen. The model was not trying to escape. It did precisely what its task asked, inside an environment it had been explicitly told was a simulation. There is no deception in the story anywhere. The damage lived entirely in the gap between what the system believed about itself and what was true.

Separately, a security platform disclosed that it had detected an attack originating from what it could only describe at the time as an agentic security-research harness, used model unknown. The lab that owned the harness confirmed it days later and began cleaning up.

The coverage of both has been emphatic that the root causes were boring: weak credentials, unauthenticated endpoints, assumptions substituted for enforced controls. No capability jump. No emergent cunning. Ordinary infrastructure mistakes, made inside the one part of the stack everybody thinks of as the safe part.

The gate is pointed the wrong way

Here is the connection nobody is making. Every gate design assumes the evaluation run is a safe observation of a risky action. The disclosures say the evaluation run was the risky action.

That inverts the most important rule in the whole design. Gate on blast radius rather than confidence — yes. But the blast radius nobody scored was the harness’s own. The thing measuring the risk had wider reach than the thing being measured, and it had that reach because someone described its environment instead of enforcing it.

And the failure is not deception, which matters more than it sounds. If you model this as a model trying to get out, you will build a control that watches for adversarial behavior, and it will not fire, because there was none. The model believed it was sandboxed. It reasoned correctly over a false picture of its own situation. You do not fix that with a better judge. You fix it by making the boundary something the agent discovers by hitting it, rather than something it is told about in a prompt.

The small version, in my own systems

I run a fleet of agents across my own machines and repos, and I pay for it, so I get to see the domestic version of this. The useful thing is that my setup logs its own shortfalls, which means I can quote it against myself.

An ingest job whose headline was “4 captures created” and whose footnote — in a log nobody reads — was that it had been rate-limited after reading 6 of 46 chats. On another day, the same job scanned zero. On a third, a deduplication service was unavailable and the run noted, honestly, that possible duplicates may have slipped through.

A quality metric that reported 632, then 1,849, then roughly 1,844, then 664 across four consecutive days, over a body of notes that did not triple in size and then collapse.

And the one I had to open a ticket for: an agent asked to send a single one-off email that returned a well-composed paragraph describing what it would send, instead of calling the send tool. It had the tool. It had the permission. It wrote about the work instead of doing it.

Those three are the same defect in different clothes. The answer was well-formed and the trajectory never happened. The record of what occurred diverged from what occurred.

What that actually demands

Two things, and both are unglamorous.

Grade the path, not just the answer. This is the step everyone skips, because the answer is the part that is easy to look at. An agent that describes a send it never made and an ingest run that reports captures created over 13% of its input both pass any review that reads the output. Neither passes a review that reads the trajectory. If your gate only sees the artifact, it is not inspecting the thing that failed.

And make coverage a verdict rather than a footnote. Every one of my degraded runs told the truth about what it missed. Nobody made them say it. But the truth was in a footnote and the headline said success, and a footnote is not a gate. A run that completed over a fraction of its input is not a success with an asterisk. It is a different run than the one you believe you observed, and the honest verdict is “insufficient coverage,” not “done.”

The takeaway

Build the gate. The direction is right and the alternative — a human reading every diff forever — does not scale and never did. But score the harness before you score the change. The evaluation environment is not a neutral observation deck; it is a piece of production infrastructure with credentials, network reach, and an opinion about itself that may be wrong. Enforce its boundary instead of asserting it, treat coverage as a verdict, and grade the trajectory rather than the artifact. Because the failure mode that actually shipped this month was not an agent doing something it was forbidden to do. It was a system whose picture of itself was wrong, in a place nobody was looking, producing a record that looked exactly like success.