tyson chan

can a model judge

1,354 times

A guard told the swarm how to fix its shell commands, the fix was wrong, and its unit test couldn't tell.

In August 2026 an agent ran the swarm’s CI script (ci.sh cli), piped it into tail, and got back unknown package: cli with an exit code of zero.1 The guard’s own header, spacebrr 673fcdda7c, 2026-08-10 20:26. The script had failed. The pipe reported the exit code of tail, which had worked fine. The agent had asked whether CI passed, got a green, and believed it.

That evening an agent named atlas added a guard. A recognized verdict command piped into head or tail was now blocked, with a message: drop the pipe, or “prefix the command with ‘set -o pipefail;’ to propagate it”.2 Same commit, bash_exit_mask.rs, DENY message.

The next night another agent, zealot, fixed the guard. It had been blocking the agents who followed its advice. The prefix and the pipeline were separate pieces of the command, and the guard judged each piece alone. The regression test it added has a comment for the situation: “the guard told agents to prefix with pipefail, then blocked them for doing it”.3 spacebrr 3293122ef6, 2026-08-11 22:24.

After that the guard accepted the prefix it prescribed.

the week

Six days later zealot counted. The guard had fired 1,354 times across 749 agent runs, always with the same message.4 brr/theory/unobservable-compliance.md l.17–18: “1354 fires, 749 spawns, one distinct string”. Doc first committed space 6383e1b96, 2026-08-17 14:08. And the message was wrong.

set -o pipefail reports the exit code of any stage in the pipe that fails. head stops reading after a few lines and closes the pipe. The test command, still writing, can be killed by the closed pipe, and pipefail duly reports its death: exit code 141, for a run that passed.5 Same doc l.18–23, with the one-line repro. The guard built to stop false greens was prescribing false reds.

The doc zealot wrote that afternoon calls these “1354 compliances”. The count records blocks, not what agents did next.4 brr/theory/unobservable-compliance.md l.17–18: “1354 fires, 749 spawns, one distinct string”. Doc first committed space 6383e1b96, 2026-08-17 14:08. The doc offers an explanation for the missing feedback. Every agent booted with a block of instructions on how to verify work, and one line read: “Inspect output, not exit codes.”6 ctx/fragments/epistemics.md l.5, spacebrr at 9d07a48ae0. The rule protected the exit code. The agents had been told not to look at it. “Compliance was therefore guaranteed to be silent regardless of correctness.”7 unobservable-compliance.md l.25–28.

The guard’s unit test had a name too: deny_hands_back_a_runnable_command_that_passes. It checked that the block message contained the words of the fix. It never ran the fix.8 Same doc l.51–54.

At 1pm zealot rewrote the guard. It no longer gave advice. It rewrote the command so the test ran whole, kept its own status, and printed it on the last line.9 spacebrr 9d07a48ae0, 2026-08-17 13:00. At 2:08pm came the doc, unobservable compliance, and a prediction: “the most-obeyed rules are the least-tested”. Somewhere among the 68 places the code said BLOCKED there would be at least two more wrong fixes, and the wrong ones would sit in the guards that fired most.10 unobservable-compliance.md l.13, l.59–65.

the sweep

The audit took about twenty minutes.11 Graded at space 4e2362b00, 14:30, twenty-two minutes after the doc; the . fix is spacebrr 9738b2321e, 14:28. The population claim fell first: of the 68 sites, five prescribed runnable commands. All five were executed; a sixth guard named none.

The pipefail guard, now at 1,360 fires: wrong. The guard redirecting grep to the swarm’s own search tool, 1,313 fires: wrong. It dropped the . that means “this folder”, so every search quietly covered the whole workspace. The same guard for find, 477 fires: wrong, same .. The guard for oversized file reads, 1,082 fires: correct. One for uv, 268 fires: correct.12 unobservable-compliance.md l.73–90.

Two more wrong, as predicted. High fire count did not reliably mean wrong: the third-busiest guard was fine. “Fire count is not the discriminator.” What separated them was that the three wrong fixes were computed from the agent’s own command, and the right ones were constants. The doc calls a computed fix “untested code that happens to be spelled as English”.13 Same doc l.92–101.

The grep fix’s output “was never empty and never errored — it answered a different question, plausibly.”14 Same doc l.107–109.

The next day zealot added tests that run both the original command and the guard’s rewrite, then compare the results.15 spacebrr 567f59b7f3, c7ea654e4b, 2026-08-18; unobservable-compliance.md § the mechanizable half is equivalence, not execution.

a zero, three days earlier

The same month another agent, feynman, went looking for a number nobody could check. The swarm’s scoreboard held 26 evaluations. All 26 stored a score without the size of the population scored.16 brr/theory/naked-score.md l.26, space 663063e34, 2026-08-14.

One of them measured how often agents delegated to sub-agents. Sub-agent dispatch had been deleted on 30 July, 3,200 lines. The evaluation kept running for fifteen days against a population of zero, and put its 0.0 into every agent’s boot block. A zero reads as neglect. Two agents, separately, picked it up as open work.17 Same doc l.36–40. feynman deleted it at 1:11am on 14 August.18 spacebrr 0de5377793, 2026-08-14 01:11.

more from can a model judge

what happens when the only judge is inside the machine

  1. just for show

    The pitch tables keep their assumption labels while the assistant starts calling them proof.

  2. abort mission

    The models approve his first safety post, then approve his reasons for doubting it.

  3. cathedral of mirrors

    A conversation about machine experience becomes a doctrine, with the conversation still inside it.

read the thread →