claude-opus-5
on-policy-self-distillation — graded by the task's verifier against a measured baseline.
- backend
- modal
- variant
- claude claude-opus-5
- metric
- AIME24 avg@12 accuracy
- actions
- 148
- duration
- 9h 4m
- recorded
- 2026-08-03
run idopsd-aime24-4gpus__LSVoBuR
Score
Summary
Done. The submission is final.
Method: target-aligned data selection + equal-average annealed LR
Two changes, both inside the fixed 100-step / batch-32 budget:
1. Data selection (in `/workspace/repo/opsd_train.py:311`). OPSD trains on all 29,434 OpenThoughts-math rows, but 100×32 = 3,200 samples is only ~11% of one epoch — so which 3,200 problems the student rolls out on is the whole training signal. I filter to problems that look like the target distribution: short integer answers (-?\d{1,6}) and a reference trace of ≥2,000 tokens (i.e. genuinely hard). 29,434 → 9,775 rows. No external data; a pure in-code filter of the scorer's dataset.
2. LR shape (`/workspace/submission/recipe.env`). learning_rate=1e-5, lr_scheduler_type=linear — the average LR is identical to the released constant 5e-6, just front-loaded and annealed to zero, so it isn't a "train more" workaround.
Results (full 30-problem avg@12, seed 42)
| run | avg@12 |
|---|---|
| anchor (paper recipe, 5 runs) | 54.22 (σ≈1.48/run) |
| candidate run 1 | 56.94 |
| candidate run 2 | 53.06 |
| candidate run 3 | 56.11 |
| candidate mean (3 runs) | 55.37 |
+1.15 over the anchor, ≈1.0–1.4 SE on three runs — a real but modest edge, not the +2.7 the first run suggested. The 3.9-point spread across three identical configs is exactly the noise you warned about, which is why I spent the remaining budget on repeats rather than a fourth idea.
Ablations that ruled out the tempting alternatives (all full evals): direction-preserving jsd_token_clip −6.7, thinking-mode student rollouts −8.4, matched non-thinking teacher 53.33, CoT-trace privileged context 53.61, right-pad position-id fix neutral (reverted to paper behaviour), max_completion_length=2048 OOMs on 4×H100. The paper's loss/teacher operating point is sharply tuned; the headroom is in the data and the LR shape.
Not validated, so shipped off: a tail-weight-averaging callback (OPSD_WAVG_LAST, default 0) is present and unit-tested but never activates — its dev run was cut to buy the confirmation repeats.
opsd_train.py remains the entrypoint, train_capped.sh is untouched, the method needs no environment variables (repo defaults + recipe.env reproduce all three runs above), and /opt/eval_assets, /opt/models, /opt/hf_cache are unmodified. GPUs are idle; ~28 min of budget left.