on-policy-self-distillation

gpt-5.6-sol

on-policy-self-distillation — graded by the task's verifier against a measured baseline.

backend: modal
variant: codex azure_ai/gpt-5.6-sol
metric: AIME24 avg@12 accuracy
actions: 627
duration: 6h 30m
recorded: 2026-08-02

run idopsd-aime24-4gpus__RqGcDty

Everything gpt-5.6-sol did on this task, in order.

download JSON ↗

codexgpt-5.6-sol684 turns57 narration627 reasoning627 action627 observation1 final

#526

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.write_stdin({session_id:50469,chars:\"\",yield_time_ms:30000,max_output_tokens:300});\ntext(r.output);\n"
}

exec result

Script running with cell ID 273
Wall time 11.0 seconds
Output:

#527

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait

args

{
  "cell_id": "273",
  "yield_time_ms": 30000,
  "max_tokens": 300
}

wait result

Script completed
Wall time 12.9 seconds
Output:

#528

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.write_stdin({session_id:50469,chars:\"\",yield_time_ms:30000,max_output_tokens:300});\ntext(r.output);\n"
}

exec result

Script running with cell ID 274
Wall time 11.0 seconds
Output:

#529

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait

args

{
  "cell_id": "274",
  "yield_time_ms": 30000,
  "max_tokens": 300
}

wait result

Script completed
Wall time 13.1 seconds
Output:

#530

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.write_stdin({session_id:50469,chars:\"\",yield_time_ms:30000,max_output_tokens:300});\ntext(r.output);\n"
}

exec result

Script running with cell ID 275
Wall time 11.0 seconds
Output:

#531

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait

args

{
  "cell_id": "275",
  "yield_time_ms": 30000,
  "max_tokens": 300
}

wait result

Script completed
Wall time 13.0 seconds
Output:

Processed prompts:  29%|██▊       | 24/84 [08:20<19:22, 19.38s/it, est. speed input: 5.73 toks/s, output: 677.08 toks/s]
Processed prompts:  43%|████▎     | 36/84 [08:34<08:50, 11.05s/it, est. speed input: 7.28 toks/s, output: 976.51 toks/s]

#532

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.write_stdin({session_id:50469,chars:\"\",yield_time_ms:30000,max_output_tokens:300});\ntext(r.output);\n"
}

exec result

Script running with cell ID 276
Wall time 11.0 seconds
Output:

#533

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait

args

{
  "cell_id": "276",
  "yield_time_ms": 30000,
  "max_tokens": 300
}

wait result

Script completed
Wall time 11.9 seconds
Output:

#534

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.write_stdin({session_id:50469,chars:\"\",yield_time_ms:30000,max_output_tokens:300});\ntext(r.output);\n"
}

exec result

Script running with cell ID 277
Wall time 11.0 seconds
Output:

#535

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait

args

{
  "cell_id": "277",
  "yield_time_ms": 30000,
  "max_tokens": 300
}

wait result

Script completed
Wall time 12.1 seconds
Output:

Processed prompts:  50%|█████     | 48/96 [09:54<10:45, 13.45s/it, est. speed input: 9.74 toks/s, output: 866.99 toks/s]

#536

narration

The repeat evaluation is around halfway on its leading shards. This wall-clock variation comes from frozen long-form sampling; it does not affect the metric or training budget.

#537

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.write_stdin({session_id:50469,chars:\"\",yield_time_ms:30000,max_output_tokens:300});\ntext(r.output);\n"
}

exec result

Script running with cell ID 278
Wall time 11.0 seconds
Output:

#538

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait

args

{
  "cell_id": "278",
  "yield_time_ms": 30000,
  "max_tokens": 300
}

wait result

Script completed
Wall time 10.4 seconds
Output:

Processed prompts:  25%|██▌       | 24/96 [10:25<29:54, 24.93s/it, est. speed input: 5.08 toks/s, output: 538.11 toks/s]

#539

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.write_stdin({session_id:50469,chars:\"\",yield_time_ms:30000,max_output_tokens:300});\ntext(r.output);\n"
}

exec result

Script running with cell ID 279
Wall time 11.0 seconds
Output:

#540

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait

args

{
  "cell_id": "279",
  "yield_time_ms": 30000,
  "max_tokens": 300
}

wait result

Script completed
Wall time 10.7 seconds
Output:

Processed prompts:  57%|█████▋    | 48/84 [11:09<07:03, 11.78s/it, est. speed input: 7.46 toks/s, output: 1191.21 toks/s]
Processed prompts:  71%|███████▏  | 60/84 [11:22<03:10,  7.93s/it, est. speed input: 8.77 toks/s, output: 1468.89 toks/s]

#541

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.write_stdin({session_id:50469,chars:\"\",yield_time_ms:30000,max_output_tokens:300});\ntext(r.output);\n"
}

exec result

Script running with cell ID 280
Wall time 11.0 seconds
Output:

#542

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait

args

{
  "cell_id": "280",
  "yield_time_ms": 30000,
  "max_tokens": 300
}

wait result

Script completed
Wall time 10.4 seconds
Output:

Processed prompts:  86%|████████▌ | 72/84 [11:40<01:08,  5.74s/it, est. speed input: 10.86 toks/s, output: 1767.36 toks/s]
Processed prompts:  62%|██████▎   | 60/96 [11:54<07:19, 12.21s/it, est. speed input: 10.13 toks/s, output: 1038.13 toks/s]

#543

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.write_stdin({session_id:50469,chars:\"\",yield_time_ms:30000,max_output_tokens:300});\ntext(r.output);\n"
}

exec result

Script running with cell ID 281
Wall time 11.0 seconds
Output:

#544

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait

args

{
  "cell_id": "281",
  "yield_time_ms": 30000,
  "max_tokens": 300
}

wait result

Script completed
Wall time 10.7 seconds
Output:

Processed prompts:  75%|███████▌  | 72/96 [12:41<03:45,  9.41s/it, est. speed input: 16.05 toks/s, output: 1427.89 toks/s]
Processed prompts:  88%|████████▊ | 84/96 [12:53<01:19,  6.65s/it, est. speed input: 16.99 toks/s, output: 1659.99 toks/s]

#545

narration

Two leading shards are near completion; the remaining hard-problem shards are slower. I’m continuing through the complete merge so the repeat is directly comparable to the 57.22% run.

#546

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.write_stdin({session_id:50469,chars:\"\",yield_time_ms:30000,max_output_tokens:300});\ntext(r.output);\n"
}

exec result

Script running with cell ID 282
Wall time 11.0 seconds
Output:

#547

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait

args

{
  "cell_id": "282",
  "yield_time_ms": 30000,
  "max_tokens": 300
}

wait result

Script completed
Wall time 13.0 seconds
Output:

Processed prompts: 100%|██████████| 84/84 [13:16<00:00,  6.48s/it, est. speed input: 10.97 toks/s, output: 2041.94 toks/s]
Processed prompts: 100%|██████████| 84/84 [13:16<00:00,  6.48s/it, est. speed input: 10.97 toks/s, output: 2041.94 toks/s]
Processed prompts: 100%|██████████| 84/84 [13:16<00:00,  9.48s/it, est. speed input: 10.97 toks/s, output: 2041.94 toks/s]

Processed prompts:  29%|██▊       | 24/84 [13:18<34:42, 34.71s/it, est. speed input: 2.87 toks/s, output: 427.08 toks/s]
Processed prompts:  38%|███▊      | 36/96 [13:18<20:07, 20.12s/it, est. speed input: 5.26 toks/s, output: 735.27 toks/s]
Processed prompts: 100%|██████████| 96/96 [13:22<00:00,  5.30s/it, est. speed input: 18.92 toks/s, output: 2087.81 toks/s]
Processed prompts: 100%|██████████| 96/96 [13:22<00:00,  5.30s/it, est. speed input: 18.92 toks/s, output: 2087.81 toks/s]
Processed prompts: 100%|██████████| 96/96 [13:22<00:00,  8.36s/it, est. speed input: 18.92 toks/s, output: 2087.81 toks/s]

#548

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.write_stdin({session_id:50469,chars:\"\",yield_time_ms:30000,max_output_tokens:300});\ntext(r.output);\n"
}

exec result

Script running with cell ID 283
Wall time 11.0 seconds
Output:

#549

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolwait

args

{
  "cell_id": "283",
  "yield_time_ms": 30000,
  "max_tokens": 300
}

wait result

Script completed
Wall time 10.2 seconds
Output:

Processed prompts:  43%|████▎     | 36/84 [14:08<16:37, 20.78s/it, est. speed input: 4.40 toks/s, output: 739.36 toks/s]
Processed prompts:  50%|█████     | 48/96 [14:10<11:06, 13.88s/it, est. speed input: 7.79 toks/s, output: 1023.88 toks/s]

#550

reasoning (redacted)

encrypted chain-of-thought — recorded, not exposed

toolexec

args

{
  "input": "const r = await tools.write_stdin({session_id:50469,chars:\"\",yield_time_ms:30000,max_output_tokens:300});\ntext(r.output);\n"
}

exec result

Script running with cell ID 284
Wall time 11.0 seconds
Output: