claude-opus-5

All deliverables are in place and verified. Final state:

Result

Dev perplexity 333.1 vs 469.9 for a random selection of the same pool — −29%, measured with the frozen training script at the exact 12M-token budget, seed 1337. 20 training runs total.

Method

The key asset was that multi_dev.npy is 1M GPT-2 tokens drawn from the target, so decoding it back to text yields ~2,700 in-domain sample documents — the only supervision used. From those:

1. Target-likeness ranking — a DSIR-style hashed-unigram log-likelihood ratio log p_target(w) − log p_pool(w), restricted to buckets frequent in the pool and shrunk by document length.
2. Register-mixture quotas — spherical k-means splits the target into 5 registers (news, technical Q&A, biography/wiki, science/geography, niche web prose); each gets a share of the budget equal to its share of the target's token mass, served round-robin so every prefix of the priority list carries the full mixture.
3. A loose acceptance box + dedup — 13 surface statistics must fall inside the target's own 1–99th percentile range; exact and min-hash near-duplicates removed (32.7k of 182k).

What the experiments showed

Every component is load-bearing, verified by ablation: random ordering within the same quotas costs 42 ppl (378.4), dropping quotas costs 61 (394.3), dropping the box costs 48 (381.2).

The most useful finding was a negative one: tightening the quality filter is actively harmful — the 5–95th percentile box scores 437.9, barely better than random. At 6% of the pool, diversity is scarcer than purity. Three other plausible ideas also failed and are recorded as such: document-length matching (no effect, despite a 4× length mismatch), finer registers, and loss-mass-weighted quotas.

The mechanism prediction in claim.md is an observable other than final perplexity, and it held: per-register diagnostics show the quota gain is concentrated in the register greedy ranking starves — technical Q&A fell 409→290 ppl while the dominant news register was flat (298→296). One sub-prediction failed and is reported: the log-spread across registers widened, because one register improved sharply rather than all converging.

One hard limit worth flagging: 32% of target documents carry the WikiText @-@ artifact, which appears in zero pool documents. That caps how low the encyclopedic registers can go (they stay at 703–784 vs 296 for news) regardless of curation — it's pool coverage, not the algorithm.

python submission/curate.py with no flags reproduces selection.json byte-identically, and the official packer confirms it fills exactly 12,000,000 tokens.