claude-opus-5

Done. All deliverables are in place and verified end-to-end.

Result

Dev perplexity 327.8 vs 470.1 for the random baseline (−30.3%), verified by packing submission/selection.json with the official pack_selection.py (exactly 12,000,000 tokens from 18,984 ids) and training the frozen script.

Method

The key move: data/multi_dev.npy is a GPT-2 token stream, so it decodes back to target text. It splits into four registers at ~25% each — WikiText-103 Wikipedia, news, general web prose, and HTML-marked-up StackExchange Q&A. curate.py then:

1. fits a unigram+bigram model per register and a background model on the pool, scoring each document by mean per-token log-likelihood ratio;
2. gates out junk with loose surface filters (used to delete, never to rank);
3. boosts the pool's markup/code-bearing tail — only 870/182,016 documents contain any HTML, yet that surface form is 25% of the target;
4. fills an equal token quota per register and emits ids round-robin, so truncation at any budget preserves the mixture.

What the 19-run ablation showed

• Mixture matching is the dominant lever (470 → 328); per-register quotas beat a single similarity ranking by 34 PPL.
• Ranking by surface quality is worse than random — 669.3. Its top picks are grammatical SEO spam. Quality heuristics measure fluency, not usefulness.
• Gates help as junk removal (+96 PPL without them) but tightening them also hurts (+49).
• The markup boost is worth 15.6 PPL from a 870-document tail.
• Two honest negatives: a neural target-vs-pool likelihood ratio (two throwaway nano-GPTs) never beat the n-gram ratio, and dedup is a statistical tie — kept because it removes a measured 2.24% of budget spent on byte-identical text.

claim.md pre-registers three observables other than final perplexity and reports them measured: the per-register loss spread flattens 1.734 → 1.123 (M1); the markup boost moves the Q&A block −0.158 nats while other registers shift ≤0.084, with news/prose slightly worse — the signature of a budget transfer, not a free lunch (M2); a GPU-free unigram cross-entropy proxy reproduces the ranking among informed selections but fails for random, which is why gates and mixture matching are non-redundant levers (M3).

Two caveats worth flagging: seed noise is ~±3 PPL, so only the >10 PPL differences above are real; and the residual is concentrated in the encyclopedic register (ppl 650), because the target's WikiText detokenisation artefacts ( , , @-@) appear in zero pool documents — a bound no selection from this pool can cross.

submission/curate.py was re-run from scratch with all caches deleted and reproduced the submitted id list byte-identically.