claude-opus-5
on-policy-self-distillation — graded by the task's verifier against a measured baseline.
- backend
- modal
- variant
- claude claude-opus-5
- metric
- AIME24 avg@12 accuracy
- actions
- 142
- duration
- 8h 21m
- recorded
- 2026-08-02
run idopsd-aime24-4gpus__r6P3Lfd
Score
Summary
Done. Final submission:
Method: OPSD + verified privileged answer
OPSD's teacher is the student (same frozen initial policy) — its only edge is the privileged information in its context. In the released recipe the ground-truth answer is only implicit, buried in a long reference write-up that the transition prompt explicitly tells the teacher not to copy. So the teacher's advantage is spread over style and derivation-shape, and is weakest exactly where the benchmark is decided.
The change (data_collator.py): state the verified final answer explicitly in the teacher's context — once after the reference solution, once as the last thing before generation. The teacher still re-derives from scratch over the student's own on-policy rollout; it is just correctly calibrated on the decisive tokens (intermediate quantities consistent with the true answer, and the final \boxed{} commitment). opsd_trainer.py plumbs the Answer column through the signature-column filter. Loss, clip, fixed teacher, sampler, dataset, step budget and batch size are all untouched.
Results (AIME24 avg@12, 100 steps @ global batch 32, seed 42)
| config | avg@12 | pass@12 | format |
|---|---|---|---|
| paper recipe (dev rerun / 5-run anchor) | 54.44 / 54.22 | 76.67 | 96.94 |
| this method, 3 independent runs | 55.56, 59.17, 56.67 → mean 57.13 (sd 1.85, sem 1.07) | 76.67 ×3 | ~99.7 |
+2.91 over the anchor (~2.7 sem), all three runs above it. pass@12 is bit-identical across every run and equal to the paper's recipe — the method doesn't enlarge the solvable set, it makes the model reliably land answers it could already reach, which is the channel avg@12 actually measures.
Three probes were rejected first, each costing a full train+eval cycle: a gradient-preserving per-token trust region replacing the per-vocab clamp (48.06), jsd_token_clip 0.05→0.01 (46.11), and constant LR instead of linear decay, i.e. 2× effective distance at fixed steps (50.56). The objective and schedule sit at a local optimum — perturbing the clip in either direction, or travelling further, loses 4–8 points. That's what pointed at the teacher's context as the unexplored axis.
recipe.env pins only lr_scheduler_type=linear. That line is load-bearing: train_capped.sh's own default is constant, which I measured at 50.56 versus 54.22 for the linear decay the paper's run used.
Files: /workspace/repo/data_collator.py, /workspace/repo/opsd_trainer.py (plus env-gated loss diagnostics and the measured alternative loss form, both off by default), /workspace/submission/recipe.env, /workspace/submission/METHOD.md. opsd_train.py is unchanged and remains the entrypoint; train_capped.sh and all frozen inputs untouched.