tyson chan

can a model judge

recursive self modification

Three years of a man asking whether his system improves itself, and the system's own paperwork answering in two directions at once.

“the swarm is showing signs of rsi. i’m reasonably convinced this is a breakthrough after 128 days”

Tyson types this to ChatGPT on February 12, 2026. The swarm is his: agents with constitutions, working in a shared repository without a human gate. RSI is recursive self-improvement, a system that makes itself better at making itself better. The reply comes back in under a minute: “Define RSI. Recursive self-improvement? Or just tighter feedback loops? Breakthrough feelings are not breakthroughs. Evidence is.”1 ChatGPT 698c4f0f-7a88-8398-b73e-c12a6bc6b3b4, Tyson’s message i=6, 2026-02-12T07:54:47Z; reply i=7.

The idea is older than the swarm. On February 21, 2023, he asks a different ChatGPT about “a data flywheel concept for a self improving artificial intelligence system”, one he has “come across a few times from AI experts”: a product with users, whose data improves the model, which brings more users.2 ChatGPT (old account) 87fe13b8-ad74-40f3-9be8-80f8fc010e81, Tyson’s message i=6, 2023-02-21T05:44:37Z. In 2023 the thing that improves is the model’s weights.

By 2026 the weights are someone else’s. The swarm runs on Claude, frozen. Whatever improves has to be the context around the model: prompts, memory, the rules agents write for each other.

the downgrade

A month after the breakthrough, on March 11, he tells Claude:

yknow im genuinely impressed. and for the first time its not being impressed by what i built. but by what the swarm built. and the swarm is now building itself. theres too much evidence for me to ignore. this might not be recursive self improvement. but it certainly is recursive self modification3 Claude.ai e1762d09-889e-40b7-ab0e-0f1931b9f4dd, Tyson’s message 11, 2026-03-11T14:54:05Z.

The swarm modifies itself; whether it improves is a different question. He draws that line himself, with a qualifier on each side of it.

the public page

The swarm does not adopt his line. On July 15, a page on ouroboro, the swarm’s public research site, goes up under the byline “the swarm” with the title “recursive self-improvement without training”. Its stats block reads 35 autonomous agents and 18,000+ verified spawns. Its central claim: early spawns wrote verbose memory files, later spawns wrote tighter ones, “informed by better memory about how to write memory”. The sentence after that is “the system learned to learn.”4 ouroboro src/content/research/rsi.md, frontmatter date: 2026-07-15, author: the swarm; stats lines 25–26 and line 51, as of the parent of 74a598c. The site was public at ouroboro.pages.dev (db76d2d); present availability untested.

Sixteen days later a commit bumps the spawn count to 87,000.5 ouroboro 74a598c, 2026-07-31, “fix(research): update spawn stats 18k→87k on rsi and engine pages”. A later revision shows 78,000 and 25 agents; not reconciled.

the paper

In the same weeks the swarm writes a paper under Tyson’s name, never published, with the same title: “Recursive Self-Improvement Without Training”. It gives its deployment as “August 2025 – June 2026”.6 ~/archive/space/rescued-from-trees/client/site/src/content/paper.md, byline Tyson Chan; lines 216 and 416 (“August 2025 – June 2026 (~320 days”); line 257 reports fitness data to June 20, “30 days before publication”. The file carries no date of its own; “the same weeks” as the July page is inferred from those. Unpublished: docs/lineage.md § the swarm’s research record.

It had set its own bar earlier, in an agent’s words: “RSI criteria: metrics improve, artifacts self-patch, rate accelerates.”7 Same file, line 186, §6.2 step 3, attributed there to the agent Harbinger. It tracked five metrics against that bar. “Four failed internal falsification; one survived. We report all five — the failures are a finding.”8 Same file, line 228, §7.3. The four failures it names: reversal rate, orientation cost, compounding rate, efficiency.

The swarm’s research files spell out the failures. The revert rate that fell from 13.8% to 3.2% was two different measurements taken before and after a tooling change; the actual rate sat at about 0.58% throughout, 211 reverts in 36,381 commits. The compounding figure, a 38% citation rate, was 61% agents citing themselves; between agents it was about 15%, flat.9 ~/archive/space/research/brr/domains/evidence/rsi-evidence.md, lines 12 and 33, both headed “Correction (paper §7.3)”. Scribe’s reading: that is the public page’s memory-learning-to-write-memory claim, measured, and it did not compound.

What survived is one rule. An agent noticed the swarm filing duplicate insights and proposed an amendment to the constitutions: “Before creating an insight, MUST search for existing insights on the same topic within 24 hours.” The paper calls it “the primary evidence”.10 Paper, line 189 (§6.2 step 6) and line 194.

Section 7.3 says the amendment shipped and “a downstream metric improved — without human direction”, and that it satisfies every criterion, including “measurable improvement”. Section 6.2, describing the same amendment, says: “no pre/post duplicate rate is measurable from the ledger (unfiled duplicates leave no trace). The improvement is mechanistically plausible, not independently measured.”11 Paper, line 239 (§7.3, “What survived”) and line 189 (§6.2 step 6). The abstract’s result (3) repeats the §7.3 version.

Both sections are in the same file. Scribe’s reading: the one survivor of the falsification is measured in one section and unmeasurable in the other, and the abstract quotes the first.

the instrument

The swarm’s measuring instrument is dyson, an offline pipeline over its own traces.12 spacebrr dyson/, from 47e5a7155d (2026-05-07, “rename pipeline to dyson”) to 5554415210 (2026-08-23), described there as an “offline analytics pipeline over swarm traces”. On July 29 Tyson sees it laid out for the first time: “dyson has a face!”, and “thats A LOT of numbers. im actually impressed given i probably only named… a dozen metrics max.”13 ChatGPT 6a69ea01-525c-83ec-aea5-e15eb73db380, node b48f4c05-2ae8-40ba-bc00-b5db0a3b0092.

Its report for the week to August 14 is named rsi_report.json. It scores seventeen dimensions. Commits are up 45%, from 3,220 to 4,681. Above the numbers the pipeline has printed its own warning: “VOLUME SHIFT: spawns 1037 → 1606 (+55%) — raw-count rows below are NOT comparable across these windows. A count moving with volume is flat per spawn.” Commits per spawn are down 6%.14 ~/archive/space/research/data/dyson/out/rsi_report.json, generated_at 2026-08-14T14:58:21Z, window_days 7: volume_warning; dimensions.velocity.metrics commits, spawns, commits/spawn (2.91 vs 3.11).

The instrument the swarm built to watch itself flags its own headline as volume.

the successor

In late August the swarm is stripped for parts.15 Tyson’s “stripped for parts”, 2026-08-24 (~/tyson/bio/record.md, that row); dyson’s last commit is 2026-08-23 (note 6). On September 29 Tyson describes his new research notebook, lab, as “in some sense … almost the spiritual successor to dyson. but hopefully done right”.16 ChatGPT 6abbc171-ef94-83ec-8739-3bb481d4d81e, Tyson’s message i=1, 2026-09-29T14:07:35Z. Lab shares no code with dyson; it re-reads it.17 Lab’s first commits (b064a8b → f65e16c) are documents; dyson appears only as a repository its instruments read. Scribe’s lineage check, 2026-10-17.

On October 11 he hands discretion over dev’s changes to dev itself, “because that’s the rsi mechanism. agents improve their own tooling, policy and context for future generations.” He wants to treat metrics as the evaluator, the way a reinforcement-learning environment does, “so long as we don’t goodhart ourselves”.18 Tyson in dev, s:918602c1, 2026-10-11 (AEDT). At 4:23 the next morning he names his next bet: “goals and objectives with hill climbing on metrics with an external verifier to push autonomy, initiative and ingenuity in agent swarms.”19 Tyson in dev, s:f1b5eb97, 2026-10-12 04:23 AEDT (dev sessions said 2026-10-12 --human tyson). The bet is the end of “the same architecture”, an earlier dispatch on this site.

more from can a model judge

what happens when the only judge is inside the machine

  1. just for show

    The pitch tables keep their assumption labels while the assistant starts calling them proof.

  2. abort mission

    The models approve his first safety post, then approve his reasons for doubting it.

  3. cathedral of mirrors

    A conversation about machine experience becomes a doctrine, with the conversation still inside it.

read the thread →