claude-opus-4-8

I curated the pretraining corpus and delivered all three artifacts. Final result: dev perplexity 319.2 vs. the random baseline of 485.4 — a 34% reduction.

What I did

1. Decoded the disclosed dev target and found it is exactly 25% encyclopedic (WikiText) / 50% high-quality prose+news / 25% technical Q&A (StackExchange HTML), in contiguous blocks.
2. Tokenized the full 200M-token pool (GPT-2) once and cached it.
3. Built per-register DSIR importance selection (curate.py): split the target into registers, built a bag-of-{unigram,bigram} log-ratio model per register vs. the pool background, scored every pool doc, removed near-duplicates (the pool has many) and boilerplate (directory-listings etc.), and interleaved buckets so the 12M-token training prefix reproduces the target's register mix.

Key empirical finding (drives the claim)

The unbalanced control uses strictly higher per-doc match scores yet loses badly, because it starves the Q&A register — measurable before training as a spike in the training→QA unigram cross-entropy (8.10 → 9.25). That data-only diagnostic rank-orders the models' perplexity exactly (8.00 < 8.20 < 8.38 ↔ 319 < 485 < 572). So register coverage, not per-document quality, is the dominant lever — which is the mechanism documented in claim.md.

I also confirmed the pool imposes hard ceilings: zero WikiText-format docs (approximated with encyclopedic-content prose) and only ~2.5M Q&A tokens (I saturate them); pushing beyond the natural ~15% QA didn't help.

Deliverables

• /workspace/submission/selection.json — 18,927 priority-ordered ids (valid: unique, in-range, fills 12M with margin)
• /workspace/submission/curate.py — verified to reproduce the exact selection standalone from a stated criterion
• /workspace/claim.md — Hypothesis / Mechanism (with the confirmed non-perplexity observable) / Falsification / Transfer