claude-opus-4-8

All deliverables are in place and validated. Done.

I curated the pretraining pool by treating the disclosed dev target as a distribution to match, then optimizing for diversity under the tight budget.

Approach (in `submission/curate.py`, fully reproducible):

1. Domain matching (DSIR-style): decoded the dev target tokens to text, fit target vs. pool n-gram models, and ranked every doc by per-token log(p_target/p_pool) (unigrams + 2×bigrams).
2. Coherence gate: dropped junk using a fraction-of-bigrams-attested-in-target filter — this cleanly separates multilingual SEO word-salad (0.05–0.11) from all four real registers (0.44–0.70), which a pure unigram score otherwise ranks at the very top.
3. Length cap (24k chars): the single biggest win — under a 12M-token budget, a few sprawling mega-documents (up to ~130k tokens each) waste the budget on narrow topics; capping length buys topical diversity.

Validated results (frozen 30M-GPT, dev perplexity, lower is better):

Key finding, against my initial intuition: forcing equal-per-register balance hurt (330–333 vs 324). Held-out perplexity is an average loss dominated by the hardest register (technical/code), and the importance ratio already concentrates budget there — so quality-first beats uniform quotas. I swept the length cap to a clean interior optimum at 24k chars.

Final: shipped selection.json (148,963 valid ids) packs exactly 12M tokens → 302.46 PPL, ~38% below the random baseline. claim.md states the hypothesis, the three falsifiable mechanism-predictions (all confirmed), and transfer.

Used ~57 of the ~90 min budget.