claude-opus-5

Done. Final state of the submission in /app/methods/main/:

The frozen graph (graph.json)

5 nodes, 10 active edges, 3 LLM calls/problem (cost reference is 4.0):

problem ─┬─> solver_pot2 (llm) ──> code_exec (free) ─┐
         └─> solver_alt  (llm) ─┐                    │
                                ├─> verify (free) ───┤
                                │   [alt, code, pot2]│
                                └────────────────────┴─> decision (llm)

The decision node is a conditional re-solver, not an aggregator: if the free verifier reports unanimity it restates that integer and stops; otherwise it ignores every candidate and solves from scratch.

Verified results

Six independent selfcheck.py passes: 53.3 / 66.7 / 53.3 / 66.7 / 53.3 / 53.3, zero budget violations. The cached-replay val figure was 0.644, but the design was partly selected on it — 0.578 is the honest fresh-sample number and it's what the write-up reports.

What the optimizer does differently

Trace caching keyed by (draw, prompt) gave an 87% hit rate and made every comparison paired on identical completions — the whole aggregator design space is then enumerable offline at zero LLM cost, which is the part REINFORCE spends its entire sample budget learning. Credit assignment is paired leave-one-edge-out rather than one scalar smeared over 16 edges.

Three findings that drove the design

• Free nodes carry the most credit. The highest-attribution edge in the final graph is pot2 → code_exec (+0.100) — a non-LLM edge. code_exec reads raw predecessor text, so it recovers a program even from a <think> block truncated before any prose answer existed.
• Edge order is a decision rule. symbolic_verify breaks ties by earliest predecessor and code_exec double-votes, so ordering the reasoner first makes two agreeing reasoners override the program while a lone one doesn't. Fixing this ordering was worth +0.016 for free.
• Aggregation can't arbitrate. When candidates split, the tally is right 0–43% of the time; when they agree, 88–96%. So the mandatory final call is worth much more as a fresh attempt than as a copyist (+0.033).

Negative results are recorded too: prompt-level control of thinking length (terse) lost more conditional accuracy than it gained closure, and pot's apparent advantage over pot2 was an artifact of a biased cached subset.

One deliberate refusal

My own cost-aware objective prefers deleting a solver — dropping to 2 LLM calls raises the efficiency multiplier ~28% while costing only 0.067 accuracy, so Δobj = −0.193 for the edge feeding solver_alt. I did not take it. Gate 1 pays exactly 0 for using too few LLM nodes, and "more than a minimum number" is ambiguous enough that 2 nodes is a real risk; trading a hard gate for a multiplier is a bad bet at any plausible odds. Three LLM nodes is the leanest count that doesn't gamble on the gate, and the graph sits at that corner on purpose.