claude-opus-5

All deliverables are in place and verified. Final summary:

Result

Dev perplexity 312.99 vs 485.46 for the random do-nothing baseline — a 35.5% reduction. I verified my fast packer produces a bit-identical 12M-token array to the official pack_selection.py, so that number is exactly what the frozen pipeline computes on this selection.

Criterion (submission/curate.py, reproduces selection.json bit-exactly with no arguments)

1. Quality floor — Gopher/C4-style hard filters (length, word-length, function-word density, non-ASCII, line duplication, SHOUTING, symbol soup), with a lower stopword bar for code-bearing docs.
2. Dedup — MinHash banding on 5-word shingles; drops 22,632 of 182,016 docs.
3. Importance weighting (DSIR) — hashed unigram+bigram distributions fit per target register, read off the dev target by segmenting at <|endoftext|> and classifying by surface form; each pool doc scored by its length-normalized log ratio against a pool background. HTML/code markers are kept as pseudo-words.
4. Mixture quota + interleave — fill a token quota per register, then interleave proportionally so every prefix — in particular the 12M cutoff — carries the target mixture.

What the experiments showed (20 training runs)

• Mixture control is the dominant lever: removing it costs 99 PPL; removing the quality floor entirely costs 37.
• Filter strength is non-monotonic — strict 362.7 → medium 333.9 → loose 324.5 → none 355.0. The floor is a junk guard, not a quality maximizer; over-filtering forces the quota deeper into a worse-matching tail.
• Hard top-k beats DSIR's stochastic resampling at this 6% selection ratio (317.8 vs 358.4) — match, not diversity, is binding.
• A neural Moore-Lewis scorer failed (score_lm.py): worse at every blend weight. With only 1M in-domain tokens the proxy memorizes dev topics.
• Mechanism confirmed by an independent observable — six extra runs measuring per-register held-out loss. The predicted ordering holds exactly: technical Q&A −0.650 nats > encyclopedic −0.432 > prose/news −0.336. Gains are not uniform, which rejects the "just picked cleaner text" account.

Honest caveats, recorded in claim.md

The tuned quota (0.28/0.36/0.36) deviates from the nominal 25/50/25 reading of the disclosed description, which measures 326.68 under an otherwise identical pipeline — so tuning is only 8% of the total gain. The response surface is a flat, bumpy plateau (312.99–320.35 over ten neighbouring points), so the submitted point is a shallow minimum, and its ~4-PPL edge over neighbours could vanish on the hidden sample. The ~170-PPL edge over random comes from the structure, not the tuning, and should hold. One stated falsification test (shuffling the priority order) I did not spend GPU time on, and I noted that in the document.