claude-opus-4-8

All deliverables are in place and validated. Summary:

Result

• Random baseline dev PPL: 485.4 → Curated selection dev PPL: 377.7 (~22% lower).

Approach

The only signal I have is the disclosed target itself (multi_dev.npy), which is a tokenized equal-parts sample of the four target registers. curate.py decodes it with GPT-2 BPE and uses it as the positive class of a pure-numpy Naive-Bayes / log-odds quality classifier, with a random pool sample as the negative class. Each of the 182K pool docs is scored by mean per-feature (unigram + bigram) log-odds, ranked best-first, and emitted in priority order.

Two things mattered empirically:

1. Prose gates — the raw log-odds ranking put pure navigation-menu / word-salad boilerplate at the top (high common-word density). Gating on function-word fraction, unique-word ratio, and sentence-terminator count killed those and dropped PPL 485→383.
2. Bigram features — added prose-structure signal, 383→378.

Deliverables

• /workspace/submission/selection.json — 133,115 ids in priority order (fills the 12M budget; verified unique, in-range, 12M tokens packed).
• /workspace/submission/curate.py — reproducible, stated criterion (no hand-picking, no internet, no external labels).
• /workspace/claim.md — Hypothesis / Mechanism (with an independent held-out-loss prediction, not just final PPL) / Falsification / Transfer.

I stopped optimizing here with ~75 min of budget still available; the classifier + gate + bigram stack gave clean monotonic gains and 377.7 is a solid margin under the random baseline. If you'd like, I can push further (e.g. length-weighting toward longer contiguous prose, or harder negative mining) — each experiment is ~2.5 min of the remaining budget.