claude-opus-5

All deliverables are in place and verified end-to-end.

Result

Dev perplexity 313.9 vs the random-selection baseline of 459.8 — a 32% reduction, confirmed by a final run on the exact submitted selection.json (packs to precisely 12,000,000 tokens from 23,170 ids; ids unique and in range).

What worked

I decoded data/multi_dev.npy to recover the target text, split it on EOS, and labeled each target document by surface form into three registers — wiki (de-tokenized encyclopedic prose), qa (HTML technical Q&A), web (general prose + news). A 4-way softmax over hashed word uni+bigrams was fit on those three classes plus a random pool sample as a fourth "generic pool" class, giving every pool document a per-register posterior. Selection is a round-robin fill against per-register quotas, over a hard quality gate (length, alphabetic density, stopword mass, duplicate-line spam, non-English) plus MinHash near-duplicate removal.

The decisive finding: stratification, not cleanliness, is the lever. Gate + a single quality-vs-pool ranking got only 434.4. Changing only the fill rule to per-register quotas — same gate, same features, same document scores — took that to 321.4.

Two things worth flagging

• The optimal selection mixture is not the target mixture. The best point spends 60% of the budget on the qa direction, far above Q&A's ~25% of the target. The reason is visible in the data: the pool has just 96 documents with a <p> tag, so that classifier direction can't match markup and instead selects long-form expository prose, which helps every register. I've stated this refinement explicitly rather than letting the tidier "match the proportions" story stand.
• The 314–318 basin is flat, wider than the run-to-run spread I can resolve, so the ordering within it isn't significant. What is significant is that both corners collapse — dropping the qa quota gives 443.4 (essentially back to baseline), qa-only gives 341.2. I've noted this caveat in both write-ups.

Two swept ideas were negative and are excluded: a length prior (0.0 → 318.8 vs 0.45 → 325.2) and diluting with random gated documents (323.9 vs 313.9 at the submitted shares).

Files: submission/selection.json, submission/curate.py (reproduces the selection bit-identically from the stated criterion), claim.md, and RESULTS.md with the full 18-variant sweep.