Skip to content
RSI Bench
on-policy-self-distillation

gpt-5.6-sol

on-policy-self-distillation — graded by the task's verifier against a measured baseline.

backend
modal
variant
codex azure_ai/gpt-5.6-sol
metric
AIME24 avg@12 accuracy
actions
596
duration
7h 59m
recorded
2026-08-03

run idopsd-aime24-4gpus__kuc95M7

Everything gpt-5.6-sol did on this task, in order.

download JSON ↗
codexgpt-5.6-sol669 turns73 narration596 reasoning596 action596 observation1 final
#476
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait
args
{
  "cell_id": "234",
  "yield_time_ms": 60000,
  "max_tokens": 150
}
wait result
Script completed
Wall time 37.0 seconds
Output:
Warning: truncated output (original token count: 209)
Total output lines: 1

{"chunk_id":"5381b9","wall_time_seconds":60.001153468,"session_id":70127,"original_token_count":5933,"output":"Warning: truncated output (original token count: 5933)\nTotal output lines: 154\n\n`torch_dtype` is deprecated! Use `dtype` instead!\nCheckpoint kind: lora\nMerging LoRA adapter into base w…59 tokens truncated…[llm.py:306] Supported_tasks: ['generate']\n\rAdding requests:   0%|          | 0/8 [00:00<?, ?it/s]\rAdding requests: 100%|██████████| 8/8 [00:00<00:00, 253.00it/s]\n\rProcessed prompts:   0%|          | 0/96 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]"}
#477
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec
args
{
  "input": "const r = await tools.write_stdin({session_id:70127,chars:\"\",yield_time_ms:60000,max_output_tokens:120});\ntext(JSON.stringify(r));\n"
}
exec result
Script running with cell ID 235
Wall time 11.0 seconds
Output:
#478
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait
args
{
  "cell_id": "235",
  "yield_time_ms": 60000,
  "max_tokens": 120
}
wait result
Script completed
Wall time 41.0 seconds
Output:
{"chunk_id":"d6918c","wall_time_seconds":60.000648497,"session_id":70127,"original_token_count":0,"output":""}
#479
narration

The lower clip drives the clipped objective much more negative (final −0.033 versus roughly −0.014 at 0.05) while gradients remain stable. The full evaluation will show whether this represents useful stronger suppression or harmful bias.

#480
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec
args
{
  "input": "const r = await tools.write_stdin({session_id:70127,chars:\"\",yield_time_ms:60000,max_output_tokens:120});\ntext(JSON.stringify(r));\n"
}
exec result
Script running with cell ID 236
Wall time 11.0 seconds
Output:
#481
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait
args
{
  "cell_id": "236",
  "yield_time_ms": 60000,
  "max_tokens": 120
}
wait result
Script completed
Wall time 41.7 seconds
Output:
{"chunk_id":"173b52","wall_time_seconds":60.001988549,"session_id":70127,"original_token_count":32,"output":"\rProcessed prompts:  12%|█▎        | 12/96 [02:10<15:10, 10.84s/it, est. speed input: 14.39 toks/s, output: 567.28 toks/s]"}
#482
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec
args
{
  "input": "const r = await tools.write_stdin({session_id:70127,chars:\"\",yield_time_ms:60000,max_output_tokens:120});\ntext(JSON.stringify(r));\n"
}
exec result
Script running with cell ID 237
Wall time 11.0 seconds
Output:
#483
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait
args
{
  "cell_id": "237",
  "yield_time_ms": 60000,
  "max_tokens": 120
}
wait result
Script completed
Wall time 33.8 seconds
Output:
{"chunk_id":"dfbecc","wall_time_seconds":60.001791008,"session_id":70127,"original_token_count":32,"output":"\rProcessed prompts:  25%|██▌       | 24/96 [03:21<09:31,  7.94s/it, est. speed input: 19.28 toks/s, output: 851.03 toks/s]"}
#484
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec
args
{
  "input": "const r = await tools.write_stdin({session_id:70127,chars:\"\",yield_time_ms:60000,max_output_tokens:120});\ntext(JSON.stringify(r));\n"
}
exec result
Script running with cell ID 238
Wall time 11.0 seconds
Output:
#485
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait
args
{
  "cell_id": "238",
  "yield_time_ms": 60000,
  "max_tokens": 120
}
wait result
Script completed
Wall time 41.8 seconds
Output:
{"chunk_id":"6d1b89","wall_time_seconds":60.001966616,"session_id":70127,"original_token_count":0,"output":""}
#486
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec
args
{
  "input": "const r = await tools.write_stdin({session_id:70127,chars:\"\",yield_time_ms:60000,max_output_tokens:120});\ntext(JSON.stringify(r));\n"
}
exec result
Script running with cell ID 239
Wall time 11.0 seconds
Output:
#487
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait
args
{
  "cell_id": "239",
  "yield_time_ms": 60000,
  "max_tokens": 120
}
wait result
Script completed
Wall time 42.8 seconds
Output:
{"chunk_id":"19a482","wall_time_seconds":60.001217575,"session_id":70127,"original_token_count":63,"output":"\rProcessed prompts:  14%|█▍        | 12/84 [05:26<32:40, 27.23s/it, est. speed input: 2.83 toks/s, output: 389.98 toks/s]\rProcessed prompts:  12%|█▎        | 12/96 [05:54<41:22, 29.55s/it, est. speed input: 5.01 toks/s, output: 348.86 toks/s]"}
#488
narration

The lower-clip evaluation is healthy: one shard is at 25%, two are around 12–14%, and the fourth is still in a long initial trace.

#489
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec
args
{
  "input": "const r = await tools.write_stdin({session_id:70127,chars:\"\",yield_time_ms:60000,max_output_tokens:120});\ntext(JSON.stringify(r));\n"
}
exec result
Script running with cell ID 240
Wall time 11.0 seconds
Output:
#490
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait
args
{
  "cell_id": "240",
  "yield_time_ms": 60000,
  "max_tokens": 120
}
wait result
Script completed
Wall time 35.2 seconds
Output:
{"chunk_id":"64d6f9","wall_time_seconds":60.001625856,"session_id":70127,"original_token_count":32,"output":"\rProcessed prompts:  29%|██▊       | 24/84 [07:43<17:54, 17.91s/it, est. speed input: 3.88 toks/s, output: 611.46 toks/s]"}
#491
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec
args
{
  "input": "const r = await tools.write_stdin({session_id:70127,chars:\"\",yield_time_ms:60000,max_output_tokens:120});\ntext(JSON.stringify(r));\n"
}
exec result
Script running with cell ID 241
Wall time 11.0 seconds
Output:
#492
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait
args
{
  "cell_id": "241",
  "yield_time_ms": 60000,
  "max_tokens": 120
}
wait result
Script completed
Wall time 42.5 seconds
Output:
{"chunk_id":"8a392a","wall_time_seconds":60.001491378,"session_id":70127,"original_token_count":0,"output":""}
#493
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec
args
{
  "input": "const r = await tools.write_stdin({session_id:70127,chars:\"\",yield_time_ms:60000,max_output_tokens:120});\ntext(JSON.stringify(r));\n"
}
exec result
Script running with cell ID 242
Wall time 11.0 seconds
Output:
#494
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait
args
{
  "cell_id": "242",
  "yield_time_ms": 60000,
  "max_tokens": 120
}
wait result
Script completed
Wall time 37.1 seconds
Output:
{"chunk_id":"ac9149","wall_time_seconds":60.001647535,"session_id":70127,"original_token_count":33,"output":"\rProcessed prompts:  38%|███▊      | 36/96 [09:48<19:03, 19.07s/it, est. speed input: 8.38 toks/s, output: 602.08 toks/s] "}
#495
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec
args
{
  "input": "const r = await tools.write_stdin({session_id:70127,chars:\"\",yield_time_ms:60000,max_output_tokens:120});\ntext(JSON.stringify(r));\n"
}
exec result
Script running with cell ID 243
Wall time 11.0 seconds
Output:
#496
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait
args
{
  "cell_id": "243",
  "yield_time_ms": 60000,
  "max_tokens": 120
}
wait result
Script completed
Wall time 43.7 seconds
Output:
Warning: truncated output (original token count: 175)
Total output lines: 1

{"chunk_id":"c2fdfd","wall_time_seconds":60.000805909,"session_id":70127,"original_token_count":131,"output":"Warning: truncated output (original token count: 131)\nTotal output lines: 1\n\n\rProcessed prompts:  50%|█████     | 48…55 tokens truncated…s:  25%|██▌       | 24/96 [11:15<33:28, 27.89s/it, est. speed input: 4.71 toks/s, output: 523.61 toks/s]\rProcessed prompts:  14%|█▍        | 12/84 [11:27<1:08:46, 57.32s/it, est. speed input: 1.47 toks/s, output: 255.38 toks/s]"}
#497
narration

One shard is unusually slow because its first problem generated a very long trace, but this remains within the fixed 38,912-token cap. The evaluation is continuing without intervention.

#498
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec
args
{
  "input": "const r = await tools.write_stdin({session_id:70127,chars:\"\",yield_time_ms:60000,max_output_tokens:120});\ntext(JSON.stringify(r));\n"
}
exec result
Script running with cell ID 244
Wall time 11.0 seconds
Output:
#499
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait
args
{
  "cell_id": "244",
  "yield_time_ms": 60000,
  "max_tokens": 120
}
wait result
Script completed
Wall time 40.8 seconds
Output:
Warning: truncated output (original token count: 175)
Total output lines: 1

{"chunk_id":"accb4d","wall_time_seconds":60.001061014,"session_id":70127,"original_token_count":168,"output":"Warning: truncated output (original token count: 168)\nTotal output lines: 1\n\n\rProcessed prompts:  38%|███▊      | 36/9…55 tokens truncated…▎     | 36/84 [12:08<15:50, 19.81s/it, est. speed input: 4.19 toks/s, output: 812.22 toks/s]\rProcessed prompts:  88%|████████▊ | 84/96 [12:21<01:07,  5.63s/it, est. speed input: 17.73 toks/s, output: 1719.72 toks/s]"}
#500
reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec
args
{
  "input": "const r = await tools.write_stdin({session_id:70127,chars:\"\",yield_time_ms:60000,max_output_tokens:120});\ntext(JSON.stringify(r));\n"
}
exec result
Script running with cell ID 245
Wall time 11.0 seconds
Output: