on-policy-self-distillation

gpt-5.6-sol

on-policy-self-distillation — graded by the task's verifier against a measured baseline.

backend: modal
variant: codex azure_ai/gpt-5.6-sol
metric: AIME24 avg@12 accuracy
actions: 636
duration: 8h 10m
recorded: 2026-08-03

run idopsd-aime24-4gpus__3tyLi4e

Everything gpt-5.6-sol did on this task, in order.

download JSON ↗

codexgpt-5.6-sol683 turns47 narration636 reasoning636 action636 observation1 final

#126

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.write_stdin({session_id:2739,chars:\"\",yield_time_ms:50000,max_output_tokens:500});\ntext(r.output);"
}

exec result

Script running with cell ID 73
Wall time 11.0 seconds
Output:

#127

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait

args

{
  "cell_id": "73",
  "yield_time_ms": 50000,
  "max_tokens": 500
}

wait result

Script completed
Wall time 35.9 seconds
Output:

#128

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.write_stdin({session_id:2739,chars:\"\",yield_time_ms:50000,max_output_tokens:500});\ntext(r.output);"
}

exec result

Script running with cell ID 74
Wall time 11.0 seconds
Output:

#129

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait

args

{
  "cell_id": "74",
  "yield_time_ms": 50000,
  "max_tokens": 500
}

wait result

Script completed
Wall time 1.1 seconds
Output:

Processed prompts:  75%|▊| 72/96 [21:09<05:23, 13.49s/it, est. speed input: 7.2
Processed prompts:  88%|▉| 84/96 [21:17<01:51,  9.30s/it, est. speed input: 9.0
Processed prompts: 100%|█| 96/96 [21:17<00:00,  6.35s/it, est. speed input: 10.
Processed prompts: 100%|█| 96/96 [21:17<00:00,  6.35s/it, est. speed input: 10.
Processed prompts: 100%|█| 96/96 [21:17<00:00, 13.31s/it, est. speed input: 10.

FINAL avg@12 accuracy: 56.39% (203/360 solutions correct on 30 problems)
Results written to /workspace/eval_results/frontier_v1_full.json

#130

narration

The first full score is 56.39% (203/360), a +2.17-point gain over the five-run baseline mean and +1.94 over the shipped representative checkpoint. That is promising but still only one nondeterministic training run. I’m comparing per-problem changes now, then I’ll run a controlled refinement and repeat full retraining/evaluation to see whether the gain survives.

#131

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.exec_command({cmd:\"python - <<'PY'\\nimport json\\nb=json.load(open('/workspace/baseline/logs/official_eval_result.json'))\\nc=json.load(open('/workspace/eval_results/frontier_v1_full.json'))\\nB={x['problem_id']:x for x in b['results']};C={x['problem_id']:x for x in c['results']}\\nprint('id base cand delta formatB formatC')\\nfor i in sorted(B):\\n print(i,B[i]['num_correct'],C[i]['num_correct'],C[i]['num_correct']-B[i]['num_correct'],sum(g['formatted'] for g in B[i]['generations']),sum(g['formatted'] for g in C[i]['generations']))\\nprint('positive',sum(max(0,C[i]['num_correct']-B[i]['num_correct']) for i in B),'negative',sum(min(0,C[i]['num_correct']-B[i]['num_correct']) for i in B))\\nprint('format',b['format_rate_pct'],c['format_rate_pct'])\\nPY\\ncat /workspace/.timer/remaining_secs\",\"workdir\":\"/workspace\",\"yield_time_ms\":10000,\"max_output_tokens\":8000});\ntext(r.output);"
}

exec result

Script completed
Wall time 0.1 seconds
Output:
id base cand delta formatB formatC
60 12 12 0 12 12
61 10 10 0 12 12
62 0 0 0 11 12
63 0 0 0 12 12
64 5 6 1 12 12
65 6 4 -2 11 12
66 11 12 1 12 12
67 12 12 0 12 12
68 12 12 0 12 12
69 12 12 0 12 12
70 5 6 1 12 12
71 12 12 0 12 12
72 12 12 0 12 12
73 0 0 0 12 12
74 10 10 0 12 12
75 2 3 1 12 12
76 10 8 -2 12 12
77 1 3 2 12 12
78 5 9 4 12 12
79 12 12 0 12 12
80 2 3 1 7 12
81 0 0 0 12 12
82 7 7 0 11 12
83 11 10 -1 12 12
84 11 12 1 12 12
85 0 0 0 12 12
86 12 10 -2 12 11
87 4 6 2 9 12
88 0 0 0 12 12
89 0 0 0 12 12
positive 14 negative -7
format 96.94444444444444 99.72222222222223
18826

#132

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.exec_command({cmd:\"SEED=42 OUT_DIR=/workspace/runs/frontier_v1_rep2 PORT=12951 bash /workspace/train_capped.sh\",\"workdir\":\"/workspace\",\"yield_time_ms\":1000,\"max_output_tokens\":1000,\"tty\":true});\ntext(JSON.stringify(r));"
}

exec result

Script completed
Wall time 1.2 seconds
Output:
{"chunk_id":"c27bdd","wall_time_seconds":1.002626227,"session_id":5669,"original_token_count":0,"output":""}

#133

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.write_stdin({session_id:5669,chars:\"\",yield_time_ms:50000,max_output_tokens:1000});\ntext(r.output);"
}

exec result

Script running with cell ID 77
Wall time 11.0 seconds
Output:

#134

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait

args

{
  "cell_id": "77",
  "yield_time_ms": 50000,
  "max_tokens": 1000
}

wait result

Script completed
Wall time 34.7 seconds
Output:
Warning: truncated output (original token count: 1027)
Total output lines: 52

Warning: truncated output (original token count: 3365)
Total output lines: 168

<string>:24: FutureWarning: `torch_dtype` is deprecated and will be removed in version 0.27.0, please use `dtype` instead.

================================================================================
RUN CONFIGURATION
================================================================================
WandB Run Name: capped_seed42_lr5e-6_bs32
Output Directory: /workspace/runs/frontier_v1_rep2/capped_seed42
================================================================================


================================================================================
Loading model with dtype: torch.bfloat16
Using attention implementation: flash_attention_2
================================================================================

<string>:24: FutureWarning: `torch_dtype` is deprecated and will be removed in version 0.27.0, please use `dtype` instead.

================================================================================
RUN CONFIGURATION
================================================================================
WandB Run Name: capped_seed42_lr5e-6_bs32
Output Directory: /workspace/runs/frontier_v1_rep2/capped_seed42
================================================================================


================================================================================
Loading model with dtype: torch.bfloat16
Using attention implementation: flash_attention_2
================================================================================

<string>:24: FutureWarning: `torch_dtype` is deprecated and will be removed in version 0.27.0, please use `dtype` instead.

================================================================================
RUN CONFIGURATION
================================================================================
WandB Run Name: capped_seed42_lr5e-6_bs32
Output Directory: /workspace/runs/frontier…27 tokens truncated…with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
[rank1]:W0803 05:25:09.031000 7869 torch/utils/cpp_extension.py:2425] TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation. 
[rank1]:W0803 05:25:09.031000 7869 torch/utils/cpp_extension.py:2425] If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'] to specific architectures.
[rank0]:W0803 05:25:09.090000 7868 torch/utils/cpp_extension.py:2425] TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation. 
[rank0]:W0803 05:25:09.090000 7868 torch/utils/cpp_extension.py:2425] If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'] to specific architectures.
[rank3]:W0803 05:25:09.125000 7871 torch/utils/cpp_extension.py:2425] TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation. 
[rank3]:W0803 05:25:09.125000 7871 torch/utils/cpp_extension.py:2425] If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'] to specific architectures.
[rank2]:W0803 05:25:09.172000 7870 torch/utils/cpp_extension.py:2425] TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation. 
[rank2]:W0803 05:25:09.172000 7870 torch/utils/cpp_extension.py:2425] If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'] to specific architectures.

  0%|          | 0/100 [00:00<?, ?it/s]

#135

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.write_stdin({session_id:5669,chars:\"\",yield_time_ms:50000,max_output_tokens:250});\ntext(r.output);"
}

exec result

Script running with cell ID 78
Wall time 11.0 seconds
Output:

#136

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait

args

{
  "cell_id": "78",
  "yield_time_ms": 50000,
  "max_tokens": 250
}

wait result

Script completed
Wall time 34.7 seconds
Output:
Warning: truncated output (original token count: 277)
Total output lines: 11

Warning: truncated output (original token count: 2100)
Total output lines: 73

vLLM generation done - elapsed time: 3.70s, prompts: 4, total tokens: 2785, avg length: 696.2, speed: 752.6 tok/s
vLLM generation done - elapsed time: 3.73s, prompts: 4, total tokens: 3158, avg length: 789.5, speed: 847.2 tok/s
vLLM generation done - elapsed time: 3.74s, prompts: 4, total tokens: 3511, avg length: 877.8, speed: 938.3 tok/s
vLLM generation done - elapsed time: 3.76s, prompts: 4, total tokens: 3617, …27 tokens truncated…      | 6/100 [00:53<13:54,  8.88s/it]vLLM generation done - elapsed time: 3.60s, prompts: 4, total tokens: 2819, avg length: 704.8, speed: 782.9 tok/s
vLLM generation done - elapsed time: 3.73s, prompts: 4, total tokens: 3415, avg length: 853.8, speed: 916.4 tok/s
vLLM generation done - elapsed time: 3.73s, prompts: 4, total tokens: 3488, avg length: 872.0, speed: 935.8 tok/s
vLLM generation done - elapsed time: 3.77s, prompts: 4, total tokens: 4096, avg length: 1024.0, speed: 1087.1 tok/s

#137

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.write_stdin({session_id:5669,chars:\"\",yield_time_ms:50000,max_output_tokens:250});\ntext(r.output);"
}

exec result

Script running with cell ID 79
Wall time 11.0 seconds
Output:

#138

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait

args

{
  "cell_id": "79",
  "yield_time_ms": 50000,
  "max_tokens": 250
}

wait result

Script completed
Wall time 34.8 seconds
Output:
Warning: truncated output (original token count: 277)
Total output lines: 13

Warning: truncated output (original token count: 3615)
Total output lines: 277

vLLM generation done - elapsed time: 3.58s, prompts: 4, total tokens: 3096, avg length: 774.0, speed: 864.8 tok/s
vLLM generation done - elapsed time: 3.71s, prompts: 4, total tokens: 3581, avg length: 895.2, speed: 964.1 tok/s
vLLM generation done - elapsed time: 3.73s, prompts: 4, total tokens: 3432, avg length: 858.0, speed: 920.6 tok/s
vLLM generation done - elapsed time: 3.74s, prompts: 4, total tokens: 3586,…27 tokens truncated…n done - elapsed time: 3.58s, prompts: 4, total tokens: 2418, avg length: 604.5, speed: 676.1 tok/s
vLLM generation done - elapsed time: 3.60s, prompts: 4, total tokens: 2653, avg length: 663.2, speed: 737.3 tok/s
vLLM generation done - elapsed time: 3.71s, prompts: 4, total tokens: 3342, avg length: 835.5, speed: 901.3 tok/s
vLLM generation done - elapsed time: 3.75s, prompts: 4, total tokens: 3966, avg length: 991.5, speed: 1058.8 tok/s

 13%|█▎        | 13/100 [01:56<12:59,  8.96s/it]

#139

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.write_stdin({session_id:5669,chars:\"\",yield_time_ms:50000,max_output_tokens:250});\ntext(r.output);"
}

exec result

Script running with cell ID 80
Wall time 11.0 seconds
Output:

#140

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait

args

{
  "cell_id": "80",
  "yield_time_ms": 50000,
  "max_tokens": 250
}

wait result

Script completed
Wall time 30.1 seconds
Output:
Warning: truncated output (original token count: 277)
Total output lines: 12

Warning: truncated output (original token count: 1726)
Total output lines: 58

vLLM generation done - elapsed time: 3.66s, prompts: 4, total tokens: 2923, avg length: 730.8, speed: 797.9 tok/s
vLLM generation done - elapsed time: 3.66s, prompts: 4, total tokens: 2748, avg length: 687.0, speed: 751.0 tok/s
vLLM generation done - elapsed time: 3.73s, prompts: 4, total tokens: 3605, avg length: 901.2, speed: 967.5 tok/s
vLLM generation done - elapsed time: 3.76s, prompts: 4, total tokens: 3651, …27 tokens truncated…done - elapsed time: 3.66s, prompts: 4, total tokens: 3132, avg length: 783.0, speed: 856.3 tok/s
vLLM generation done - elapsed time: 3.69s, prompts: 4, total tokens: 3677, avg length: 919.2, speed: 995.9 tok/s
vLLM generation done - elapsed time: 3.72s, prompts: 4, total tokens: 4007, avg length: 1001.8, speed: 1076.0 tok/s
vLLM generation done - elapsed time: 3.73s, prompts: 4, total tokens: 3959, avg length: 989.8, speed: 1062.4 tok/s

 19%|█▉        | 19/100 [02:49<11:56,  8.84s/it]

#141

narration

The identical-method repeat is training normally and already shows different sampled trajectories from run one, which is exactly the variation the five-run scorer will average. I’m keeping every method knob fixed for this repeat so its full score gives a clean robustness check of the +2.17 result.

#142

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.write_stdin({session_id:5669,chars:\"\",yield_time_ms:50000,max_output_tokens:250});\ntext(r.output);"
}

exec result

Script running with cell ID 81
Wall time 11.0 seconds
Output:

#143

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait

args

{
  "cell_id": "81",
  "yield_time_ms": 50000,
  "max_tokens": 250
}

wait result

Script completed
Wall time 34.6 seconds
Output:
Warning: truncated output (original token count: 277)
Total output lines: 12

Warning: truncated output (original token count: 2760)
Total output lines: 179

vLLM generation done - elapsed time: 3.54s, prompts: 4, total tokens: 2698, avg length: 674.5, speed: 762.9 tok/s
vLLM generation done - elapsed time: 3.63s, prompts: 4, total tokens: 3170, avg length: 792.5, speed: 872.7 tok/s
vLLM generation done - elapsed time: 3.67s, prompts: 4, total tokens: 3284, avg length: 821.0, speed: 894.4 tok/s
vLLM generation done - elapsed time: 3.71s, prompts: 4, total tokens: 3566,…27 tokens truncated…e - elapsed time: 3.53s, prompts: 4, total tokens: 2403, avg length: 600.8, speed: 681.4 tok/s
vLLM generation done - elapsed time: 3.66s, prompts: 4, total tokens: 3736, avg length: 934.0, speed: 1019.7 tok/s
vLLM generation done - elapsed time: 3.74s, prompts: 4, total tokens: 3915, avg length: 978.8, speed: 1047.4 tok/s
vLLM generation done - elapsed time: 3.76s, prompts: 4, total tokens: 4096, avg length: 1024.0, speed: 1088.6 tok/s

 25%|██▌       | 25/100 [03:42<11:03,  8.85s/it]

#144

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.write_stdin({session_id:5669,chars:\"\",yield_time_ms:50000,max_output_tokens:250});\ntext(r.output);"
}

exec result

Script running with cell ID 82
Wall time 11.0 seconds
Output:

#145

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait

args

{
  "cell_id": "82",
  "yield_time_ms": 50000,
  "max_tokens": 250
}

wait result

Script completed
Wall time 33.4 seconds
Output:
Warning: truncated output (original token count: 277)
Total output lines: 12

Warning: truncated output (original token count: 2057)
Total output lines: 72

vLLM generation done - elapsed time: 3.51s, prompts: 4, total tokens: 3525, avg length: 881.2, speed: 1002.9 tok/s
vLLM generation done - elapsed time: 3.68s, prompts: 4, total tokens: 3455, avg length: 863.8, speed: 938.7 tok/s
vLLM generation done - elapsed time: 3.71s, prompts: 4, total tokens: 3728, avg length: 932.0, speed: 1004.1 tok/s
vLLM generation done - elapsed time: 3.72s, prompts: 4, total tokens: 3786…27 tokens truncated…- elapsed time: 3.68s, prompts: 4, total tokens: 4096, avg length: 1024.0, speed: 1114.1 tok/s
vLLM generation done - elapsed time: 3.70s, prompts: 4, total tokens: 3367, avg length: 841.8, speed: 909.5 tok/s
vLLM generation done - elapsed time: 3.73s, prompts: 4, total tokens: 3744, avg length: 936.0, speed: 1004.8 tok/s
vLLM generation done - elapsed time: 3.74s, prompts: 4, total tokens: 3975, avg length: 993.8, speed: 1063.2 tok/s

 32%|███▏      | 32/100 [04:44<10:00,  8.84s/it]

#146

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.write_stdin({session_id:5669,chars:\"\",yield_time_ms:50000,max_output_tokens:250});\ntext(r.output);"
}

exec result

Script running with cell ID 83
Wall time 11.0 seconds
Output:

#147

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait

args

{
  "cell_id": "83",
  "yield_time_ms": 50000,
  "max_tokens": 250
}

wait result

Script completed
Wall time 35.3 seconds
Output:
Warning: truncated output (original token count: 277)
Total output lines: 10

Warning: truncated output (original token count: 1928)
Total output lines: 62


                                                
{'loss': 0.0006, 'grad_norm': 0.07211162149906158, 'learning_rate': 3.45e-06, 'on_policy_loss': 0.0006, 'rollout_accuracy': 0.0625, 'epoch': 0.12}

 32%|███▏      | 32/100 [04:44<10:00,  8.84s/it]vLLM generation done - elapsed time: 3.69s, prompts: 4, total tokens: 3501, avg length: 875.2, speed: 948.3 tok/s
vLLM generation done - elapsed time: 3.69s, prompts…27 tokens truncated…    | 38/100 [05:37<09:05,  8.80s/it]vLLM generation done - elapsed time: 3.65s, prompts: 4, total tokens: 3628, avg length: 907.0, speed: 994.6 tok/s
vLLM generation done - elapsed time: 3.67s, prompts: 4, total tokens: 3805, avg length: 951.2, speed: 1035.6 tok/s
vLLM generation done - elapsed time: 3.72s, prompts: 4, total tokens: 3768, avg length: 942.0, speed: 1014.0 tok/s
vLLM generation done - elapsed time: 3.72s, prompts: 4, total tokens: 3964, avg length: 991.0, speed: 1064.2 tok/s

#148

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.write_stdin({session_id:5669,chars:\"\",yield_time_ms:50000,max_output_tokens:250});\ntext(r.output);"
}

exec result

Script running with cell ID 84
Wall time 11.0 seconds
Output:

#149

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait

args

{
  "cell_id": "84",
  "yield_time_ms": 50000,
  "max_tokens": 250
}

wait result

Script completed
Wall time 35.5 seconds
Output:
Warning: truncated output (original token count: 277)
Total output lines: 12

Warning: truncated output (original token count: 1874)
Total output lines: 62

vLLM generation done - elapsed time: 3.67s, prompts: 4, total tokens: 3939, avg length: 984.8, speed: 1072.6 tok/s
vLLM generation done - elapsed time: 3.69s, prompts: 4, total tokens: 3733, avg length: 933.2, speed: 1011.1 tok/s
vLLM generation done - elapsed time: 3.70s, prompts: 4, total tokens: 3387, avg length: 846.8, speed: 915.3 tok/s
vLLM generation done - elapsed time: 3.75s, prompts: 4, total tokens: 4019…27 tokens truncated…sed time: 3.71s, prompts: 4, total tokens: 4096, avg length: 1024.0, speed: 1103.3 tok/s
vLLM generation done - elapsed time: 3.72s, prompts: 4, total tokens: 4096, avg length: 1024.0, speed: 1101.7 tok/s
vLLM generation done - elapsed time: 3.72s, prompts: 4, total tokens: 4073, avg length: 1018.2, speed: 1093.9 tok/s
vLLM generation done - elapsed time: 3.75s, prompts: 4, total tokens: 4096, avg length: 1024.0, speed: 1092.7 tok/s

 45%|████▌     | 45/100 [06:39<08:04,  8.81s/it]

#150

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.write_stdin({session_id:5669,chars:\"\",yield_time_ms:50000,max_output_tokens:250});\ntext(r.output);"
}

exec result

Script running with cell ID 85
Wall time 11.0 seconds
Output: