claude-opus-5

Done. Final state of the work:

What I shipped

/workspace/submission/recipe.env contains exactly one override: `lr_scheduler_type=cosine`. The repo code's defaults reproduce the released OPSD method, with every explored variant retained as an env-gated switch that defaults to the paper's behaviour, so the scored run (which passes no env vars) gets paper method + annealed LR. opsd_train.py remains the entrypoint.

Measured over 4 seed-42 retrains + full 30-problem avg@12 evals: 55.83, 54.72, 56.67, 53.61 → mean 55.21 (sd 1.33, se 0.67) vs the 54.22 anchor. That's +1.0 at ~1.5 standard errors — suggestive, not decisive. Three of four runs beat the anchor, one fell 0.6 below. The per-run sd (1.33) matches the quoted 1.48, so the noise is ordinary.

The reasoning: the paper's headline Qwen3-1.7B numbers are a step-50 checkpoint — its AIME25 headline 43.9 is exactly the step-50 entry of its forward-KL ablation, which decays to 41.1 by step 100. The recipe overshoots its own optimum, and the capped scorer pays that penalty in full by evaluating checkpoint-100 at constant LR. Cosine has ~half the cumulative LR, so 100 steps of fresh rollouts produce step-50-scale drift.

The finding I did not expect

Instrumenting the true unclipped KL showed the released objective does not move the student toward the teacher — the loss goes negative and true KL(p_T‖p_S) rises over training (0.098 → 0.199 in one run). The clip is per vocabulary entry, so it zeroes the gradient exactly on entries carrying the teacher's correction (the teacher's own argmax is clipped at 10.9% of positions; 50–80% of its mass at high-divergence positions). I fixed that to a genuine direction-preserving forward KL: true KL then fell 0.185 → 0.079 as designed, and accuracy dropped 4.5 points to 49.72. OPSD's gain is not from matching the privileged teacher.

Rejected, each on a full eval

Two notes on honesty of scope. I found a real defect I chose not to ship: right-padding wedges a mean-292-token pad gap between teacher prompt and completion, and those pads consume RoPE positions (measured perturbation KL ≈ 0.059 vs 0.0004 left-padded, against a ~0.105 signal). Correcting it was only ever tested inside the bundled 49.72 run, so shipping it would have been an untested change — it's left at the released behaviour and documented. And the winning change lives in recipe.env rather than the loss code; that's a thinner method change than I was aiming for, but it's the one that survived measurement.

Full details, including the gradient derivation for the clip's actual effect, are in /workspace/repo/OPSD_EXPERIMENTS.md. Timer had ~35 min left — not enough for another train+eval cycle, so I stopped rather than start one I couldn't finish.