claude-opus-4-8

Done. All deliverables are complete and validated end-to-end.

Result: dev held-out PPL 318.07 vs 457.99 random baseline — a ~30% reduction.

What I did

I framed pool curation as domain-matched data selection against the disclosed multi-domain target, and discovered two levers that drove the gains:

1. Junk filtering. Pure per-token DSIR importance scoring rewarded repetitive boilerplate (Apache "Index of /" listings dominated the top). I dropped exact duplicates (11.7k of them), whitespace-dominated docs (one token >30%), and low-diversity docs (type/token ratio <0.30). Non-English sinks automatically.

1. Per-register matching + capacity-aware balancing (the big win). Decoding multi_dev.npy (GPT-2 tokens are reversible) revealed the target is three contiguous register blocks: Wikipedia (~25%), web+news prose (~50%), technical Q&A/code (~25%). A combined-target unigram starves the code-heavy Q&A register (0.8% of selected docs vs the target's ~25%). Scoring each doc per-register and filling the budget with balanced quotas restored coverage (→ 2.4% code docs, PPL 408→323). Sweeping the mixture showed a smooth convex valley minimized at 35/30/35 (over-weighting the higher-loss wiki+QA registers), reaching 318.

Progression (each a real training run)

Deliverables

• /workspace/submission/selection.json — 182,016 unique ordered ids (fills 12M tokens with ~20.7k docs).
• /workspace/submission/curate.py — reproducible; regenerates the exact byte-identical selection from the stated criterion.
• /workspace/claim.md — Hypothesis / Mechanism (with two confirmed non-PPL observables: the 3× code-doc composition shift and the convex mixture valley) / Falsification / Transfer.

The submitted mixture sits in a flat valley (±3 PPL vs neighbors), so it's robust to slight register-proportion differences in the hidden official target, which shares the same domain.