Skip to content
RSI Bench
on-policy-self-distillation

gpt-5.6-sol

on-policy-self-distillation — graded by the task's verifier against a measured baseline.

backend
modal
variant
codex azure_ai/gpt-5.6-sol
metric
AIME24 avg@12 accuracy
actions
636
duration
8h 10m
recorded
2026-08-03

run idopsd-aime24-4gpus__3tyLi4e

Everything gpt-5.6-sol did on this task, in order.

download JSON ↗
codexgpt-5.6-sol683 turns47 narration636 reasoning636 action636 observation1 final
#651
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec
args
{
  "input": "const r = await tools.write_stdin({session_id: 53682, chars: \"\", yield_time_ms: 50000, max_output_tokens: 400});\ntext(JSON.stringify(r));"
}
exec result
Script running with cell ID 333
Wall time 11.0 seconds
Output:
#652
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait
args
{
  "cell_id": "333",
  "yield_time_ms": 50000,
  "max_tokens": 400
}
wait result
Script completed
Wall time 36.3 seconds
Output:
{"chunk_id":"8b3ff4","wall_time_seconds":50.001883305,"session_id":53682,"original_token_count":66,"output":"\rProcessed prompts:  43%|████▎     | 36/84 [10:58<12:53, 16.11s/it, est. speed input: 4.63 toks/s, output: 770.21 toks/s]\rProcessed prompts:  57%|█████▋    | 48/84 [11:08<06:02, 10.06s/it, est. speed input: 6.05 toks/s, output: 995.56 toks/s]"}
#653
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec
args
{
  "input": "const r = await tools.write_stdin({session_id: 53682, chars: \"\", yield_time_ms: 50000, max_output_tokens: 400});\ntext(JSON.stringify(r));"
}
exec result
Script running with cell ID 334
Wall time 11.0 seconds
Output:
#654
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait
args
{
  "cell_id": "334",
  "yield_time_ms": 50000,
  "max_tokens": 400
}
wait result
Script completed
Wall time 36.2 seconds
Output:
{"chunk_id":"b0dd46","wall_time_seconds":50.001183276,"session_id":53682,"original_token_count":35,"output":"\rProcessed prompts:  71%|███████▏  | 60/84 [11:38<02:56,  7.34s/it, est. speed input: 8.11 toks/s, output: 1324.76 toks/s]"}
#655
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec
args
{
  "input": "const r = await tools.write_stdin({session_id: 53682, chars: \"\", yield_time_ms: 50000, max_output_tokens: 400});\ntext(JSON.stringify(r));"
}
exec result
Script running with cell ID 335
Wall time 11.0 seconds
Output:
#656
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait
args
{
  "cell_id": "335",
  "yield_time_ms": 50000,
  "max_tokens": 400
}
wait result
Script completed
Wall time 34.3 seconds
Output:
{"chunk_id":"4ad6ec","wall_time_seconds":50.001539458,"session_id":53682,"original_token_count":66,"output":"\rProcessed prompts:  62%|██████▎   | 60/96 [12:20<07:49, 13.03s/it, est. speed input: 9.77 toks/s, output: 935.51 toks/s] \rProcessed prompts:  14%|█▍        | 12/84 [12:25<1:14:35, 62.16s/it, est. speed input: 1.59 toks/s, output: 378.62 toks/s]"}
#657
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec
args
{
  "input": "const r = await tools.write_stdin({session_id: 53682, chars: \"\", yield_time_ms: 50000, max_output_tokens: 400});\ntext(JSON.stringify(r));"
}
exec result
Script running with cell ID 336
Wall time 11.0 seconds
Output:
#658
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait
args
{
  "cell_id": "336",
  "yield_time_ms": 50000,
  "max_tokens": 400
}
wait result
Script completed
Wall time 35.9 seconds
Output:
{"chunk_id":"9b527d","wall_time_seconds":50.001730475,"session_id":53682,"original_token_count":133,"output":"\rProcessed prompts:  29%|██▊       | 24/84 [13:17<28:08, 28.14s/it, est. speed input: 2.99 toks/s, output: 702.20 toks/s]  \rProcessed prompts:  43%|████▎     | 36/84 [13:31<12:39, 15.82s/it, est. speed input: 4.52 toks/s, output: 936.78 toks/s]\rProcessed prompts:  75%|███████▌  | 72/96 [13:54<04:30, 11.27s/it, est. speed input: 9.76 toks/s, output: 1088.38 toks/s]\rProcessed prompts:  57%|█████▋    | 48/84 [13:55<06:12, 10.35s/it, est. speed input: 6.12 toks/s, output: 1204.42 toks/s]"}
#659
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec
args
{
  "input": "const r = await tools.write_stdin({session_id: 53682, chars: \"\", yield_time_ms: 50000, max_output_tokens: 400});\ntext(JSON.stringify(r));"
}
exec result
Script running with cell ID 337
Wall time 11.0 seconds
Output:
#660
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait
args
{
  "cell_id": "337",
  "yield_time_ms": 50000,
  "max_tokens": 400
}
wait result
Script completed
Wall time 36.0 seconds
Output:
{"chunk_id":"e0365a","wall_time_seconds":50.00173808,"session_id":53682,"original_token_count":35,"output":"\rProcessed prompts:  86%|████████▌ | 72/84 [14:36<01:58,  9.90s/it, est. speed input: 8.68 toks/s, output: 1357.33 toks/s]"}
#661
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec
args
{
  "input": "const r = await tools.write_stdin({session_id: 53682, chars: \"\", yield_time_ms: 50000, max_output_tokens: 400});\ntext(JSON.stringify(r));"
}
exec result
Script running with cell ID 338
Wall time 11.0 seconds
Output:
#662
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait
args
{
  "cell_id": "338",
  "yield_time_ms": 50000,
  "max_tokens": 400
}
wait result
Script completed
Wall time 36.5 seconds
Output:
{"chunk_id":"8c90a7","wall_time_seconds":50.001880167,"session_id":53682,"original_token_count":143,"output":"\rProcessed prompts:  88%|████████▊ | 84/96 [15:41<02:06, 10.51s/it, est. speed input: 10.82 toks/s, output: 1359.02 toks/s]\rProcessed prompts: 100%|██████████| 96/96 [15:42<00:00,  7.18s/it, est. speed input: 16.10 toks/s, output: 1674.36 toks/s]\rProcessed prompts: 100%|██████████| 96/96 [15:42<00:00,  7.18s/it, est. speed input: 16.10 toks/s, output: 1674.36 toks/s]\rProcessed prompts: 100%|██████████| 96/96 [15:42<00:00,  9.82s/it, est. speed input: 16.10 toks/s, output: 1674.36 toks/s]\n"}
#663
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec
args
{
  "input": "const r = await tools.write_stdin({session_id: 53682, chars: \"\", yield_time_ms: 50000, max_output_tokens: 400});\ntext(JSON.stringify(r));"
}
exec result
Script running with cell ID 339
Wall time 11.0 seconds
Output:
#664
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait
args
{
  "cell_id": "339",
  "yield_time_ms": 50000,
  "max_tokens": 400
}
wait result
Script completed
Wall time 35.8 seconds
Output:
{"chunk_id":"651af4","wall_time_seconds":50.001193521,"session_id":53682,"original_token_count":107,"output":"\rProcessed prompts: 100%|██████████| 84/84 [16:26<00:00,  9.65s/it, est. speed input: 8.86 toks/s, output: 1508.61 toks/s]\rProcessed prompts: 100%|██████████| 84/84 [16:26<00:00,  9.65s/it, est. speed input: 8.86 toks/s, output: 1508.61 toks/s]\rProcessed prompts: 100%|██████████| 84/84 [16:26<00:00, 11.74s/it, est. speed input: 8.86 toks/s, output: 1508.61 toks/s]\n"}
#665
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec
args
{
  "input": "const r = await tools.write_stdin({session_id: 53682, chars: \"\", yield_time_ms: 50000, max_output_tokens: 400});\ntext(JSON.stringify(r));"
}
exec result
Script running with cell ID 340
Wall time 11.0 seconds
Output:
#666
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait
args
{
  "cell_id": "340",
  "yield_time_ms": 50000,
  "max_tokens": 400
}
wait result
Script completed
Wall time 35.7 seconds
Output:
{"chunk_id":"e07326","wall_time_seconds":50.001452103,"session_id":53682,"original_token_count":242,"output":"\rProcessed prompts:  38%|███▊      | 36/96 [16:40<28:41, 28.69s/it, est. speed input: 4.74 toks/s, output: 612.46 toks/s]\rProcessed prompts:  50%|█████     | 48/96 [16:54<14:15, 17.83s/it, est. speed input: 5.96 toks/s, output: 912.08 toks/s]\rProcessed prompts:  71%|███████▏  | 60/84 [17:07<04:57, 12.39s/it, est. speed input: 5.96 toks/s, output: 1137.81 toks/s]\rProcessed prompts:  86%|████████▌ | 72/84 [17:19<01:42,  8.52s/it, est. speed input: 8.43 toks/s, output: 1393.65 toks/s]\rProcessed prompts: 100%|██████████| 84/84 [17:30<00:00,  6.03s/it, est. speed input: 9.47 toks/s, output: 1599.57 toks/s]\rProcessed prompts: 100%|██████████| 84/84 [17:30<00:00,  6.03s/it, est. speed input: 9.47 toks/s, output: 1599.57 toks/s]\rProcessed prompts: 100%|██████████| 84/84 [17:30<00:00, 12.50s/it, est. speed input: 9.47 toks/s, output: 1599.57 toks/s]\n"}
#667
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec
args
{
  "input": "const r = await tools.write_stdin({session_id: 53682, chars: \"\", yield_time_ms: 50000, max_output_tokens: 400});\ntext(JSON.stringify(r));"
}
exec result
Script running with cell ID 341
Wall time 11.0 seconds
Output:
#668
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait
args
{
  "cell_id": "341",
  "yield_time_ms": 50000,
  "max_tokens": 400
}
wait result
Script completed
Wall time 35.2 seconds
Output:
{"chunk_id":"5b68d7","wall_time_seconds":50.000500204,"session_id":53682,"original_token_count":69,"output":"\rProcessed prompts:  62%|██████▎   | 60/96 [17:58<07:59, 13.33s/it, est. speed input: 7.85 toks/s, output: 1073.76 toks/s]\rProcessed prompts:  75%|███████▌  | 72/96 [18:00<03:32,  8.84s/it, est. speed input: 9.78 toks/s, output: 1359.09 toks/s]"}
#669
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec
args
{
  "input": "const r = await tools.write_stdin({session_id: 53682, chars: \"\", yield_time_ms: 50000, max_output_tokens: 800});\ntext(JSON.stringify(r));"
}
exec result
Script running with cell ID 342
Wall time 11.0 seconds
Output:
#670
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait
args
{
  "cell_id": "342",
  "yield_time_ms": 50000,
  "max_tokens": 800
}
wait result
Script completed
Wall time 36.0 seconds
Output:
{"chunk_id":"fa9a21","wall_time_seconds":50.000620777,"session_id":53682,"original_token_count":0,"output":""}
#671
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec
args
{
  "input": "const r = await tools.write_stdin({session_id: 53682, chars: \"\", yield_time_ms: 50000, max_output_tokens: 800});\ntext(JSON.stringify(r));"
}
exec result
Script running with cell ID 343
Wall time 11.0 seconds
Output:
#672
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait
args
{
  "cell_id": "343",
  "yield_time_ms": 50000,
  "max_tokens": 800
}
wait result
Script completed
Wall time 34.5 seconds
Output:
{"chunk_id":"e47c92","wall_time_seconds":50.000536077,"session_id":53682,"original_token_count":35,"output":"\rProcessed prompts:  88%|████████▊ | 84/96 [19:41<01:44,  8.70s/it, est. speed input: 9.81 toks/s, output: 1437.92 toks/s]"}
#673
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec
args
{
  "input": "const r = await tools.write_stdin({session_id: 53682, chars: \"\", yield_time_ms: 50000, max_output_tokens: 1600});\ntext(JSON.stringify(r));"
}
exec result
Script running with cell ID 344
Wall time 11.0 seconds
Output:
#674
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait
args
{
  "cell_id": "344",
  "yield_time_ms": 50000,
  "max_tokens": 1600
}
wait result
Script completed
Wall time 7.0 seconds
Output:
{"chunk_id":"6961dc","wall_time_seconds":21.159531684,"exit_code":0,"original_token_count":143,"output":"\rProcessed prompts: 100%|██████████| 96/96 [20:30<00:00,  7.24s/it, est. speed input: 10.63 toks/s, output: 1563.50 toks/s]\rProcessed prompts: 100%|██████████| 96/96 [20:30<00:00,  7.24s/it, est. speed input: 10.63 toks/s, output: 1563.50 toks/s]\rProcessed prompts: 100%|██████████| 96/96 [20:30<00:00, 12.82s/it, est. speed input: 10.63 toks/s, output: 1563.50 toks/s]\n\nFINAL avg@12 accuracy: 54.72% (197/360 solutions correct on 30 problems)\nResults written to /workspace/eval_results/dev_eval_1785747779.json\n"}
#675
narration

The third full run scored 54.72% (197/360). Across the three exact final-method runs, the mean is 55.00% versus the 54.22% five-run baseline—a +0.78-point estimate, with individual runs spanning 53.89–56.39 as expected from training noise. I’m now cleaning the run-created derived caches and doing the final reproducibility check.