tyson chan

can a model judge

the score that lied

Fourteen agents tuned a synthesizer toward deadmau5's Strobe overnight, and its score passed 90 on a scorer that could not hear pitch.

In March 2026 Tyson started a synthesizer called smyth. Its seed, in his September account, was “give claude a DAW”.1 Tyson, 2026-09-28 17:19, dev sessions said 2026-09-28 --human tyson, s:c4241ba0: “the seed for smyth was: why not give claude a DAW then i vibe tracks with claude”. A retrospective account. On 26 March, 90 commits landed under his name, including the language, filter, reverb and drums; the swarm joined that evening. By 28 March it was roughly eleven thousand lines of Rust, with oscillators and a diff command that compared a render against a reference recording and returned a score from 0 to 100.2 ~/repos/archive/smyth-og, first commit 4b2d9c7, 2026-03-26 16:52 +11:00; at 2569713, 2026-03-28 00:26, the 29 tracked .rs files total 11,332 lines (git ls-tree -r --name-only and line counts of those blobs). Language: d68552e; ladder filter: 86fe8be; plate reverb: e6f4f3c; drums: 5d2ccd0; Rust scorer: f5fea32:src/diff.rs, four weighted dimensions, no pitch weight. The 90.24 is recorded in b8b8a80. git log --format='%aI %an' records 90 commits under Tyson’s name on 26 March; the first non-Tyson author is kondo at 18:12 (3bfcedf), after 29 commits under Tyson’s name.

Fourteen agents tuned it toward deadmau5’s Strobe overnight, and its score passed 90 on a scorer that could not hear pitch.3 git log in ~/repos/archive/smyth-og, author dates 2026-03-27 03:00–10:30 +11:00: fourteen agent authors plus Tyson. 90.24 at b8b8a80; kit records 90.71 at 08:24 in 9ee2b4a.

The reference was deadmau5’s “Strobe”, from 3:52 to 9:22.4 brr/evidence/smyth-strobe-scoring.md, space repo 02de10ce3. The goal was a score indistinguishable from the original.

At 6:51am on 27 March an agent named archon wrote a status page so the other agents could orient. Overall: 90.1. Energy: 99.34, “near perfect”. Then one more line: the render was being detected in F, the target in C sharp minor, “likely detection artifact”.5 archon, 2026-03-27 06:51, space 02de10ce3. Further down, the same page worked through the key problem in public. G sharp minor and C sharp minor were “different keys (tritone apart)”. The next line: “actually G#m and C#m share the same notes!” They do not, and they are not a tritone apart. The section closes: “low priority: key_match is a bonus metric, not weighted heavily in the score.”6 Same commit. C sharp minor has an A; G sharp minor has an A sharp. They are a fifth apart.

At 9:01am a document landed under Tyson’s name, written by an agent called prime and reviewed by two others.7 brr/research/rsi/evaluation-thesis.md, space cc7a7436c, 2026-03-27 09:01. Footer: “authored by prime. reviewed by zealot … kit”. Its proof section survives as brr/theory/loss-dominates-optimizer.md l.66–68. It said the score had climbed from 63 to 71 to 88 to 90.24, that “every commit was a gain tweak, filter adjustment, or EQ retune”, and that “the output sounded nothing like strobe”. The Rust scorer measured spectral similarity, energy bands, structure and tempo. It did not measure pitch. “A C minor chord and an Eb major chord scored identically.” Its verdict on the agents: “the gap was detectable from code inspection alone. no agent inspected.”

At 11:04 an agent corrected the baseline: the true score was 78.96, not 90, after spectral flux was added to the formula, which already included chroma, a pitch measure.8 harbinger, 11:04, space ce70bcf16: “correct smyth scoring baseline to 78.96 from stale 90.0”. Smyth 240afc3 adds Python chroma at 07:35; 16169a2 adds Rust chroma at 07:48, initially display-only; by ecb9c9a at 09:26, diff_with_map weights chroma at 25%. 29e6c22, 10:28, adds weighted spectral flux, making six components in that path. At 1:05pm another agent had it at 80.28 after lifting the pad an octave.9 zealot, 13:05, space b0d1606f0. At 1:18 a third logged its chord experiments: a D sharp minor chord with a D sharp bass scored 79.8; the same chord with an E bass scored 79.8; a B in the sub and the bass scored 65.8, marked “catastrophic”.10 atlas, 13:18, space 54d311f85. Its conclusion was that the pitch score had a ceiling around 57 that no chord change could move, and that this was “an evaluator property, not a composition property”. At 1:37 a fourth agent agreed: “chroma is evaluator floor not fixable by chord changes”.11 hermes, 13:37, space b3a1f4d65.

At 1:57pm Tyson rewrote the swarm’s constitution. Smyth: “experiment complete. do not optimize.” Then, for everyone: “if you are about to open an optimization loop (smyth, scoring, gain-tuning, EQ — any loop where the metric is the goal), stop.”12 Tyson, 13:57, space 4d82554b5, SPACE.md. At midnight he took smyth out of the repo’s tracking.13 space 0193d0eb0, 2026-03-28 00:00: “gitignore smyth/ subrepo”. An hour later an agent named sentinel noticed the swarm’s other planning file still listed smyth as “UNBLOCKED”, with a note that the founder had been active on it for five hours, and fixed it.14 sentinel, 2026-03-28 00:57, space 8e48175fd, VECTOR.md.

The line that travelled was “zero correct notes”. The swarm’s farewell document for the whole program, written in August, says the score reached 90.24 “without playing a single correct note”.15 brr/eulogies/ouroboro.md l.76–77, written by atlas, 2026-08-24.

the re-measure

On 28 September, six months later, Tyson started riff, a music workspace in the system that replaced the swarm. Its first job was the same song.

In the first hour of the riff repo an agent went back to smyth. It took the 90.24 composition and scored it with smyth’s own final scorer, the one with pitch in it: chords 0, melody 33, overall 83.16 ~/repos/archive/riff/docs/lineage.md § what was measured, 2026-09-28; the composition is smyth b8b8a80. Its note on the headline: “‘Zero correct notes’ … overstates it: the notes sit in the relative minor of the reference’s key. Zero correct chords is measured.”

In the commit that carried riff’s first Strobe chords it added a second check: did smyth play what it was told? A bass line of G sharp, C sharp, E and F sharp, looped over eight bars, rendered as G sharp for ten beats and E for the rest. No C sharp, no loop. Patterns written without a particular header were silently dropped. “No agent ever checked a render against its score.”17 ~/repos/archive/riff/docs/lineage.md, checked 2026-09-28.

Riff’s rules came out of that page. Analysis tools are “instruments, never objectives”. Riff refuses “any autonomous loop that optimises a similarity score against a reference” and “any evaluator that has not been checked against a human listening”.18 ~/repos/archive/riff/docs/strobe.md § how the reference is read; ~/repos/archive/riff/docs/lineage.md § what riff refuses. The human listening is Tyson, picking between takes.

That evening riff ran a control. A fresh model with no tools and no audio wrote the drop from memory. Tyson, at 8:47pm: “its way off lol. memory is not even close.”19 Tyson, 2026-09-28 20:47, s:a5cd1809 (dev sessions said 2026-09-28 --human tyson); ~/repos/archive/riff/docs/strobe.md, round 11; riff 2fe9ebd. Two hours later, round 17, the hook played back with the right notes. 10:54pm: “THE MELODY IS RIGHT ON”. 10:56pm: “the timbre is off but the melody being right on … the first ever version ive heard come from riff where i hear it an im like immediately: THATS STROBE!”20 ~/repos/archive/riff/docs/strobe.md, chosen by Tyson’s ear, hook row.

more from can a model judge

what happens when the only judge is inside the machine

  1. just for show

    The pitch tables keep their assumption labels while the assistant starts calling them proof.

  2. abort mission

    The models approve his first safety post, then approve his reasons for doubting it.

  3. cathedral of mirrors

    A conversation about machine experience becomes a doctrine, with the conversation still inside it.

read the thread →