claude-opus-4-8

All three deliverables are complete and verified.

Result: dev held-out perplexity 378.7 vs 476.7 random baseline — a 20.6% reduction.

What I found (the key insight): My first instinct — a quality classifier that ranks documents by target-likeness and takes the strict top — actually scored *502, worse than random (477)*. Decoding the dev target showed it's a genuine 4-way mixture (Wikipedia, web prose, news, StackOverflow Q&A). Hard-ranking toward "most target-like" collapses that register diversity and hurts a fixed-budget LM.

The winning recipe (in curate.py):

1. Train a target-vs-random-pool quality classifier — feature-hashed bag-of-ngrams logistic regression as a torch EmbeddingBag on GPU (no sklearn; none was installable offline).
2. Apply a light junk gate (min length, min alphabetic fraction).
3. *Drop the bottom 65% by score, then keep the whole top band and emit it in a diversity-preserving shuffled order* — not sorted by score. The pipeline packs 12M tokens by broadly sampling this clean band, so all four registers stay represented.

The drop percentile was tuned on a dev-PPL sweep, which traced a clear U-shape (drop 35%→392, 50%→389, 65%→379, 78%→405, 88%→443, top-only→502) — the minimum sits at keeping the diverse top ~35% by quality.

Deliverables:

• /workspace/submission/selection.json — 61,986 valid unique ids (~69M tokens, 5.7× budget), priority order = shuffled clean top band.
• /workspace/submission/curate.py — reproducible, stated criterion.
• /workspace/claim.md — hypothesis (filter-then-diversify beats rank-concentrate for broad targets), mechanism with a non-perplexity observable (the U-shape and a diversity metric that should track loss ordering), falsification conditions, and transfer (narrows toward strict top-k as the target narrows).