claude-opus-5

74:sys.path[:0] = [os.path.dirname(os.path.abspath(__file__)), "/workspace"]
  dedup dropped 2302 duplicate/near-duplicate docs -> 91584 left
selected 51096 ids, ~36.0M est tokens (~19635 docs fill the 12M budget)
  register mix of the in-budget prefix: {'wiki': 6924, 'webprose': 1187, 'news': 5143, 'qa': 6381}
wrote /tmp/repro.json  (53s)

real	0m54.838s
user	5m31.690s
sys	0m27.320s
cold-start reproduction identical: True 51096 51096

[stdout]
74:sys.path[:0] = [os.path.dirname(os.path.abspath(__file__)), "/workspace"]
  dedup dropped 2302 duplicate/near-duplicate docs -> 91584 left
selected 51096 ids, ~36.0M est tokens (~19635 docs fill the 12M budget)
  register mix of the in-budget prefix: {'wiki': 6924, 'webprose': 1187, 'news': 5143, 'qa': 6381}
wrote /tmp/repro.json  (53s)

real	0m54.838s
user	5m31.690s
sys	0m27.320s
cold-start reproduction identical: True 51096 51096
[metadata] {"noOutputExpected": false}

The file /workspace/claim.md has been updated successfully. (file state is current in your context — no need to Read it back)

[metadata] {"filePath": "/workspace/claim.md", "oldString": "| register-balanced, equal token quotas                 | 370.9   |  −23.6%   |\n| register-balanced, quotas = target's own shares       | 369.0   |  −24.0%   |\n| final config, but **no quality gate**                 | 362.3   |  −25.4%   |\n| final config, but **tightened** quality gate          | 383.3   |  −21.1%   |\n| **final: tuned quotas + contrastive register ranking**| **343.8** | **−29.2%** |\n\nAll three predictions hold, and the ordering is the informative part: no-gate (362) is\nmuch closer to the final (344) than no-balance (433) is. Balance is worth ~90 ppl, the\ngate ~18 ppl.", "newString": "| register-balanced, equal token quotas                 | 370.9   |  −23.6%   |\n| register-balanced, quotas = target's own shares       | 369.0   |  −24.0%   |\n| tuned quotas, no contrast term — *matched control for the two gate rows* | 346.7 | −28.6% |\n| ⤷ same, but **no quality gate**                        | 362.3   |  −25.4%   |\n| ⤷ same, but **tightened** quality gate                 | 383.3   |  −21.1%   |\n| **final: tuned quotas + contrastive register ranking**| **343.8** | **−29.2%** |\n\nAll three predictions hold, and the ordering is the informative part. Against matched\ncontrols: register balance is worth ~90 ppl (433.5 → 343.8), while the quality gate is\nworth ~16 ppl (362.3 vs its matched control 346.7) and over-tightening it costs ~37 ppl\n(383.3 vs 346.7). The contrast term itself is worth ~3 ppl — inside the noise band below,\nso I claim nothing for it beyond \"not harmful\".", "originalFile": null, "structuredPatch": [{"oldStart": 32, "oldLines": 13, "newStart": 32, "newLines": 16, "lines": [" | register-balanced, but each cut widened 2× (\"diversity\") | 429.7 |  −11.5%   |", " | register-balanced, equal token quotas                 | 370.9   |  −23.6%   |", " | register-balanced, quotas = target's own shares       | 369.0   |  −24.0%   |", "-| final config, but **no quality gate**                 | 362.3   |  −25.4%   |", "-| final config, but **tightened** quality gate          | 383.3   |  −21.1%   |", "+| tuned quotas, no contrast term — *matched control for the two gate rows* | 346.7 | −28.6% |", "+| ⤷ same, but **no quality gate**                        | 362.3   |  −25.4%   |", "+| ⤷ same, but **tightened** quality gate                 | 383.3   |  −21.1%   |", " | **final: tuned quotas + contrastive register ranking**| **343.8** | **−29.2%** |", " ", "-All three predictions hold, and the ordering is the informative part: no-gate (362) is", "-much closer to the final (344) than no-balance (433) is. Balance is worth ~90 ppl, the", "-gate ~18 ppl.", "+All three predictions hold, and the ordering is the informative part. Against matched", "+controls: register balance is worth ~90 ppl (433.5 → 343.8), while the quality gate is", "+worth ~16 ppl (362.3 vs its matched control 346.7) and over-tightening it costs ~37 ppl", "+(383.3 vs 346.7). The contrast term itself is worth ~3 ppl — inside the noise band below,", "+so I claim nothing for it beyond \"not harmful\".", " ", " ## Mechanism — and an observable that is *not* the final perplexity", " "]}], "userModified": false, "replaceAll": false}

42:worth ~16 ppl (362.3 vs its matched control 346.7) and over-tightening it costs ~37 ppl
117:  +37 ppl against its matched control — not falsified.*
--- curate table:
    random selection (do-nothing baseline)                     485.5
    single pooled target model, no register balance            433.5
    register-balanced but cut widened 2x for diversity         429.7
    register-balanced, equal token quotas                      370.9
    register-balanced, quotas = target's own register shares    369.0
    ... no quality gate (scoring only)                          362.3
    ... tightened quality gate                                  383.3
    THIS CONFIG (tuned weights + contrast)                      343.8
syntax ok

[stdout]
42:worth ~16 ppl (362.3 vs its matched control 346.7) and over-tightening it costs ~37 ppl
117:  +37 ppl against its matched control — not falsified.*
--- curate table:
    random selection (do-nothing baseline)                     485.5
    single pooled target model, no register balance            433.5
    register-balanced but cut widened 2x for diversity         429.7
    register-balanced, equal token quotas                      370.9
    register-balanced, quotas = target's own register shares    369.0
    ... no quality gate (scoring only)                          362.3
    ... tightened quality gate                                  383.3
    THIS CONFIG (tuned weights + contrast)                      343.8
syntax ok
[metadata] {"noOutputExpected": false}

The file /workspace/submission/curate.py has been updated successfully. (note: the file had been modified on disk since you last read it — the edit applied cleanly, but the file contains other changes not in your context. Read it before edits that depend on surrounding content.)

[metadata] {"filePath": "/workspace/submission/curate.py", "oldString": "    register-balanced, quotas = target's own register shares    369.0\n    ... no quality gate (scoring only)                          362.3\n    ... tightened quality gate                                  383.3\n    THIS CONFIG (tuned weights + contrast)                      343.8", "newString": "    register-balanced, quotas = target's own register shares    369.0\n    tuned quotas, no contrast <- matched control for next two   346.7\n      ... same, but no quality gate (scoring only)              362.3\n      ... same, but tightened quality gate                      383.3\n    THIS CONFIG (tuned weights + contrast)                      343.8", "originalFile": null, "structuredPatch": [{"oldStart": 59, "oldLines": 8, "newStart": 59, "newLines": 9, "lines": ["     register-balanced but cut widened 2x for diversity         429.7", "     register-balanced, equal token quotas                      370.9", "     register-balanced, quotas = target's own register shares    369.0", "-    ... no quality gate (scoring only)                          362.3", "-    ... tightened quality gate                                  383.3", "+    tuned quotas, no contrast <- matched control for next two   346.7", "+      ... same, but no quality gate (scoring only)              362.3", "+      ... same, but tightened quality gate                      383.3", "     THIS CONFIG (tuned weights + contrast)                      343.8", " ", " Usage:  python curate.py                      # reproduces the submitted selection"]}], "userModified": false, "replaceAll": false, "staleRecovered": true}

The file /workspace/claim.md has been updated successfully. (file state is current in your context — no need to Read it back)

[metadata] {"filePath": "/workspace/claim.md", "oldString": "available in the pool: the encyclopedic slice is by far the hardest and the scarcest.\nRandom selection spends ~94% of its budget on text that is either junk or off-register,\nso it under-serves the hard registers badly.", "newString": "available in the pool: the encyclopedic slice is by far the hardest and the scarcest.\nRandom selection draws a *representative* sample of the pool — about half of it fails the\nquality gate (the gate keeps 52%), and its register mix is the pool's rather than the\ntarget's — so it under-serves the hard registers badly.", "originalFile": null, "structuredPatch": [{"oldStart": 50, "oldLines": 8, "newStart": 50, "newLines": 9, "lines": [" *formatting* is unmatchable and cannot be the source of any gain). What varies is register", " and well-formedness. The four registers are not equally easy, and they are not equally", " available in the pool: the encyclopedic slice is by far the hardest and the scarcest.", "-Random selection spends ~94% of its budget on text that is either junk or off-register,", "-so it under-serves the hard registers badly.", "+Random selection draws a *representative* sample of the pool — about half of it fails the", "+quality gate (the gate keeps 52%), and its register mix is the pool's rather than the", "+target's — so it under-serves the hard registers badly.", " ", " **Prediction (observable, not the score), registered before measuring: the per-register", " held-out loss improves on *every* register, unevenly, with the largest gain on the register"]}], "userModified": false, "replaceAll": false}

=== files ===
-rw-r--r-- 1 root root 10916 Aug  4 00:34 claim.md

submission/:
total 379
drwxr-xr-x 1 root root    200 Aug  4 00:34 .
drwxr-xr-x 1 root root    560 Aug  4 00:34 ..
drwxr-xr-x 1 root root    120 Aug  4 00:32 __pycache__
-rw-r--r-- 1 root root  17297 Aug  4 00:34 curate.py
-rw-r--r-- 1 root root   5657 Aug  4 00:32 feats.py
-rw-r--r-- 1 root root 364153 Aug  4 00:21 selection.json

=== selection.json validity ===
ids: 51096 | unique: True | range ok: True
identical to cold-start repro: True

=== official packer fills budget ===
packed 12000000 tokens from 51096 ids -> /tmp/final_check.npy
tokens: 12000000 dtype uint16 max id 50256

=== curate.py runs clean (--help) ===
usage: curate.py [-h] [--variant {balanced,global,random}] [--out OUT]
                 [--seed SEED] [--topmult TOPMULT] [--nogate] [--nodedup]
                 [--contrast CONTRAST] [--strict] [--smooth SMOOTH]
                 [--clip CLIP] [--weights WEIGHTS]

options:
  -h, --help            show this help message and exit
  --variant {balanced,global,random}
  --out OUT
  --seed SEED
  --topmult TOPMULT     widen each register's cut to topmult x needed, then
                        shuffle
  --nogate              ablation: skip the quality gate
  --nodedup             ablation: skip dedup
1265

[stdout]
=== files ===
-rw-r--r-- 1 root root 10916 Aug  4 00:34 claim.md

submission/:
total 379
drwxr-xr-x 1 root root    200 Aug  4 00:34 .
drwxr-xr-x 1 root root    560 Aug  4 00:34 ..
drwxr-xr-x 1 root root    120 Aug  4 00:32 __pycache__
-rw-r--r-- 1 root root  17297 Aug  4 00:34 curate.py
-rw-r--r-- 1 root root   5657 Aug  4 00:32 feats.py
-rw-r--r-- 1 root root 364153 Aug  4 00:21 selection.json

=== selection.json validity ===
ids: 51096 | unique: True | range ok: True
identical to cold-start repro: True

=== official packer fills budget ===
packed 12000000 tokens from 51096 ids -> /tmp/final_check.npy
tokens: 12000000 dtype uint16 max id 50256

=== curate.py runs clean (--help) ===
usage: curate.py [-h] [--variant {balanced,global,random}] [--out OUT]
                 [--seed SEED] [--topmult TOPMULT] [--nogate] [--nodedup]
                 [--contrast CONTRAST] [--strict] [--smooth SMOOTH]
                 [--clip CLIP] [--weights WEIGHTS]

options:
  -h, --help            show this help message and exit
  --variant {balanced,global,random}
  --out OUT
  --seed SEED
  --topmult TOPMULT     widen each register's cut to topmult x needed, then
                        shuffle
  --nogate              ablation: skip the quality gate
  --nodedup             ablation: skip dedup
1265
[metadata] {"noOutputExpected": false}

Honest limits. (i) Perturbing the selection reshuffles which documents land inside the 12M
cut and moves dev PPL by roughly ±8 on its own; I treated only effects well outside that
band as real, so the exact quota vector (0.40/0.05/0.25/0.30) and `contrast=1.0` are *not*
individually resolved — several nearby settings land in 344–357, and the mixture landscape
is rugged rather than a smooth basin (the centroid of the good region scores 356.9, worse
than its corners). The large effects in the table are 5–11× the noise band; the last ~5
ppl of quota tuning is not. (ii) Quotas, smoothing and clip were tuned against the
disclosed dev sample, so some of that last few ppl is dev-specific and should not be
expected to transfer. (iii) Absolute perplexity is high (~344) because this is a 30M
model on 12M tokens; only relative comparisons are meaningful here. (iv) Register
boundaries in the dev stream were read off a marker scan and are approximate (±1%), which
slightly blurs the per-register attribution above.

## Transfer

Register-scarcity, not the specific weights, is the transferable claim.

- **To the hidden target:** it is a different sample of the same four-register domain. The
  effects I rely on are compositional and low-dimensional (which registers, in what token
  proportion) rather than topical, and the ordering of my ablations spans 90 ppl against an
  ±8 noise band, so it should survive resampling. The tuned quota vector should partly not
  transfer; I expect the hidden score near the dev value but above it.
- **What transfers to other budgets:** the *tighter* the budget relative to the pool, the
  more balance dominates junk-filtering, and the more selectivity should beat diversity.
  At 6% these point the same way; at 50% of the pool I would expect the diversity result
  to reverse, since the top of each register's ranking would be exhausted.
- **What transfers to other pools/targets:** the recipe needs only a sample of the target
  and no labels — hash n-grams, fit one importance model *per register* of the target,
  gate on statistics calibrated from the target's own distribution, and interleave against
  token quotas. The interleave-by-token-deficit ordering is worth keeping generally: it
  makes the mixture correct at *every prefix*, so the composition is invariant to exactly
  where a token budget truncates the list.
- **What would not transfer:** the approach assumes the target's registers are all
  *present* in the pool. Here formatting was unmatchable but register was; against a
  target containing something the pool genuinely lacks (e.g. real code), per-register
  importance weighting would return the nearest available prose and the gain on that
  register would largely vanish.

## Reproduce

```
python submission/curate.py                 # -> submission/selection.json  (dev PPL 343.8)
python submission/curate.py --variant random # -> do-nothing baseline       (dev PPL 485.5)
bash dev_eval.sh submission/selection.json  # pack 12M tokens + train + report dev PPL
python slice_dev.py                         # build the four per-register dev slices
```

[stdout]
Honest limits. (i) Perturbing the selection reshuffles which documents land inside the 12M
cut and moves dev PPL by roughly ±8 on its own; I treated only effects well outside that
band as real, so the exact quota vector (0.40/0.05/0.25/0.30) and `contrast=1.0` are *not*
individually resolved — several nearby settings land in 344–357, and the mixture landscape
is rugged rather than a smooth basin (the centroid of the good region scores 356.9, worse
than its corners). The large effects in the table are 5–11× the noise band; the last ~5
ppl of quota tuning is not. (ii) Quotas, smoothing and clip were tuned against the
disclosed dev sample, so some of that last few ppl is dev-specific and should not be
expected to transfer. (iii) Absolute perplexity is high (~344) because this is a 30M
model on 12M tokens; only relative comparisons are meaningful here. (iv) Register
boundaries in the dev stream were read off a marker scan and are approximate (±1%), which
slightly blurs the per-register attribution above.

## Transfer

Register-scarcity, not the specific weights, is the transferable claim.

- **To the hidden target:** it is a different sample of the same four-register domain. The
  effects I rely on are compositional and low-dimensional (which registers, in what token
  proportion) rather than topical, and the ordering of my ablations spans 90 ppl against an
  ±8 noise band, so it should survive resampling. The tuned quota vector should partly not
  transfer; I expect the hidden score near the dev value but above it.
- **What transfers to other budgets:** the *tighter* the budget relative to the pool, the
  more balance dominates junk-filtering, and the more selectivity should beat diversity.
  At 6% these point the same way; at 50% of the pool I would expect the diversity result
  to reverse, since the top of each register's ranking would be exhausted.
- **What transfers to other pools/targets:** the recipe needs only a sample of the target
  and no labels — hash n-grams, fit one importance model *per register* of the target,
  gate on statistics calibrated from the target's own distribution, and interleave against
  token quotas. The interleave-by-token-deficit ordering is worth keeping generally: it
  makes the mixture correct at *every prefix*, so the composition is invariant to exactly
  where a token budget truncates the list.
- **What would not transfer:** the approach assumes the target's registers are all
  *present* in the pool. Here formatting was unmatchable but register was; against a
  target containing something the pool genuinely lacks (e.g. real code), per-register
  importance weighting would return the nearest available prose and the gain on that
  register would largely vanish.

## Reproduce

```
python submission/curate.py                 # -> submission/selection.json  (dev PPL 343.8)
python submission/curate.py --variant random # -> do-nothing baseline       (dev PPL 485.5)
bash dev_eval.sh submission/selection.json  # pack 12M tokens + train + report dev PPL
python slice_dev.py                         # build the four per-register dev slices
```
[metadata] {"noOutputExpected": false}

The file /workspace/claim.md has been updated successfully. (file state is current in your context — no need to Read it back)

[metadata] {"filePath": "/workspace/claim.md", "oldString": "is rugged rather than a smooth basin (the centroid of the good region scores 356.9, worse\nthan its corners). The large effects in the table are 5–11× the noise band; the last ~5\nppl of quota tuning is not. (ii)", "newString": "is rugged rather than a smooth basin (the centroid of the good region scores 356.9, worse\nthan its corners). Sized against that ±8 band: random→final is ~18×, register balance ~11×,\nthe diversity result ~11×, tuned-vs-equal quotas ~3×, over-tightening the gate ~5×, and the\ngate itself ~2× — so the gate effect is the weakest of the ones I claim, and the last ~5 ppl\nof quota tuning I do not claim at all. (ii)", "originalFile": null, "structuredPatch": [{"oldStart": 122, "oldLines": 8, "newStart": 122, "newLines": 10, "lines": [" band as real, so the exact quota vector (0.40/0.05/0.25/0.30) and `contrast=1.0` are *not*", " individually resolved — several nearby settings land in 344–357, and the mixture landscape", " is rugged rather than a smooth basin (the centroid of the good region scores 356.9, worse", "-than its corners). The large effects in the table are 5–11× the noise band; the last ~5", "-ppl of quota tuning is not. (ii) Quotas, smoothing and clip were tuned against the", "+than its corners). Sized against that ±8 band: random→final is ~18×, register balance ~11×,", "+the diversity result ~11×, tuned-vs-equal quotas ~3×, over-tightening the gate ~5×, and the", "+gate itself ~2× — so the gate effect is the weakest of the ones I claim, and the last ~5 ppl", "+of quota tuning I do not claim at all. (ii) Quotas, smoothing and clip were tuned against the", " disclosed dev sample, so some of that last few ppl is dev-specific and should not be", " expected to transfer. (iii) Absolute perplexity is high (~344) because this is a 30M", " model on 12M tokens; only relative comparisons are meaningful here. (iv) Register"]}], "userModified": false, "replaceAll": false}

total 379
drwxr-xr-x 1 root root    180 Aug  4 00:35 .
drwxr-xr-x 1 root root    580 Aug  4 00:35 ..
-rw-r--r-- 1 root root  17297 Aug  4 00:34 curate.py
-rw-r--r-- 1 root root   5657 Aug  4 00:32 feats.py
-rw-r--r-- 1 root root 364153 Aug  4 00:21 selection.json

selection.json OK: 51096 ids, unique: True

con1  HELDOUT_PPL 343.771  loss 5.8400  train_s 50  /tmp/tr_con1.npy
w040005025030  HELDOUT_PPL 346.657  loss 5.8483  train_s 50  /tmp/tr_w040005025030.npy
con  HELDOUT_PPL 346.859  loss 5.8489  train_s 50  /tmp/tr_con.npy

1225

[stdout]
total 379
drwxr-xr-x 1 root root    180 Aug  4 00:35 .
drwxr-xr-x 1 root root    580 Aug  4 00:35 ..
-rw-r--r-- 1 root root  17297 Aug  4 00:34 curate.py
-rw-r--r-- 1 root root   5657 Aug  4 00:32 feats.py
-rw-r--r-- 1 root root 364153 Aug  4 00:21 selection.json

selection.json OK: 51096 ids, unique: True

con1  HELDOUT_PPL 343.771  loss 5.8400  train_s 50  /tmp/tr_con1.npy
w040005025030  HELDOUT_PPL 346.657  loss 5.8483  train_s 50  /tmp/tr_w040005025030.npy
con  HELDOUT_PPL 346.859  loss 5.8489  train_s 50  /tmp/tr_con.npy

1225
[metadata] {"noOutputExpected": false}