gpt-5.6-sol

Script completed
Wall time 10.4 seconds
Output:
84424 0.433 0.11 ly Stadium for soccer and outdoor track and field. This prim
53695 0.507 0.106 Ukraine’s Do-It-Yourself War Ukraine's military isn't up to 
101049 0.488 0.129  Herald, The (Newspaper) - July 2, 1970, Lethbridge, Alberta
107301 0.414 0.079 !<|endoftext|>The Consulting Practice re-sells software solu
65819 0.439 0.093 Maintenance of Mounds and Park Infrastructure (continued) Fo
163316 0.463 0.13  epiphany 100% professional! Menu outline for cause and effe
107938 0.387 0.049 <|endoftext|>Email your questions to;firstname.lastname@exam
64972 0.515 0.129 <|endoftext|>"Get into the Know" Today it is not uncommon to
52615 0.496 0.125 <|endoftext|>The perspective to the family is a powerful poi
82244 0.456 0.102 <|endoftext|>[Keynote #1] by Cedric Blancher /PDF Slides [Ke
62135 0.395 0.07 ICULOUS TO THE LAST DETAIL Complete In-house Environmental C
85058 0.5 0.126 <|endoftext|>How well are nuclear plants near you prepared f
34525 0.337 0.085 <|endoftext|>The family office has a clear portfolio with th
25321 0.371 0.068 <|endoftext|>The grand bazaar, in marion county indiana crim
36267 0.424 0.096 <|endoftext|>The Social Worker functions as part of an inter
32134 0.555 0.158 In brief, the Global War on Terror sledgehammer strategy has
67498 0.477 0.103 <|endoftext|>Get a FREE Health Guide ...plus all the latest 
90410 0.419 0.094 , flowers. We give them to express our love, and our sorrow.
93294 0.411 0.097  AU<|endoftext|>COMMUNIQUE OF THE 48TH ANNUAL NATIONAL CONFE
48097 0.529 0.143 by Paul Craig Roberts Does anyone remember when National Pub
80026 0.514 0.156 <|endoftext|>Play live casino games online The first thing t
27357 0.341 0.051 By Bernhard H. Walke, Stefan Mangold, Lars Berlemann In the 
51748 0.525 0.147  (Reuters) - What if the generation that once rocked out to 
92440 0.461 0.112 ktoman's blog is a source of frequent amusement. I have an a
82001 0.417 0.095 fronted with low voter participation rates and high levels o
104689 0.473 0.116  all areas<|endoftext|>The United States is, indisputably, t
97296 0.436 0.103  functions as a member of the nursing care team and assists 
95890 0.52 0.136 .<|endoftext|>"Doveriai no proveriai." (Trust but verify.) -
142325 0.478 0.111  rehab in morningstar meadows richardson tx | drug rehab in 
68891 0.531 0.12 <|endoftext|>The world of technology is constantly evolving 
47602 0.42 0.087 ”Calciopoli now is completed. they may of paid off the retai
111430 0.465 0.13  reboots its security platform with Crossbeam Systems, Inc. 

Script completed
Wall time 0.0 seconds
Output:
{}

Script completed
Wall time 0.2 seconds
Output:
     1	#!/usr/bin/env python3
     2	"""Rank raw web documents for broad high-quality English LM pretraining.
     3	
     4	The score rewards connected edited/expository prose and target-relevant news or
     5	technical Q&A, penalizes crawl boilerplate and SEO/commercial pages, and applies
     6	light near-duplicate suppression.  Domain round-robin prevents one easy-to-find
     7	register from consuming the fixed token budget.
     8	"""
     9	import json, math, re, sys, hashlib, os
    10	from collections import Counter, defaultdict
    11	
    12	POOL = "/workspace/data/pool.jsonl"
    13	OUT = "/workspace/submission/selection.json"
    14	
    15	WORD = re.compile(r"[A-Za-z]+(?:'[A-Za-z]+)?")
    16	SENT = re.compile(r"[.!?](?:[\"')\]]+)?(?:\s|$)")
    17	BAD = re.compile(r"(?i)(skip to (?:main )?content|add to cart|shopping cart|cookie policy|"
    18	                 r"privacy policy|terms (?:of (?:use|service)|and conditions)|all rights reserved|"
    19	                 r"subscribe to (?:our|the) newsletter|sign up for|free shipping|xml sitemap|"
    20	                 r"saved vehicles|log in|forgot password|javascript is disabled|"
    21	                 r"click here|share this|related posts|posted in|leave a (?:comment|reply)|"
    22	                 r"online casino|casino games|essay writing|homework help|drug rehab|"
    23	                 r"mail order bride|dating site|payday loan|free health guide)")
    24	NEWS = re.compile(r"(?i)(\(Reuters\)\s*-|\b(?:Reuters|Associated Press)\b|"
    25	                  r"Additional reporting by|Reporting by .{2,60}; Editing by)")
    26	TECH = re.compile(r"(?i)(<p>|<pre>|<code>|stack overflow|stack exchange|"
    27	                  r"traceback \(most recent call last\)|exception:|error:|"
    28	                  r"\b(?:python|javascript|java|linux|sql|database|algorithm|function|compiler)\b)")
    29	REFERENCE = re.compile(r"(?i)(\b(?:is|was|are|were) (?:a|an|the)\b|"
    30	                       r"\b(?:located|founded|established|known as|refers to|consists of)\b)")
    31	
    32	LANG=None
    33	def language_model():
    34	    """Unigram and observed phrase statistics of the disclosed broad dev target."""
    35	    import numpy as np
    36	    from transformers import AutoTokenizer
    37	    raw=AutoTokenizer.from_pretrained('gpt2').decode(np.load('/workspace/data/multi_dev.npy'))
    38	    w=[x.lower() for x in WORD.findall(raw)]
    39	    uni=Counter(w); bi=set(zip(w,w[1:])); tri=set(zip(w,w[1:],w[2:]))
    40	    return uni, sum(uni.values()), bi, tri
    41	
    42	def features(s):
    43	    n = len(s); words = WORD.findall(s); nw = len(words)
    44	    if nw < 80 or n < 450:
    45	        return None
    46	    # Derive character ratios from already-extracted words / C-level counts;
    47	    # avoid a second Python-level pass over the 770 MB corpus.
    48	    alpha = sum(map(len, words)) / n
    49	    weird = s.count('\ufffd') / n
    50	    sent = len(SENT.findall(s)); lines = s.count('\n') + 1
    51	    avg_sent = nw / max(1, sent)
    52	    uniq = len(set(w.lower() for w in words)) / nw
    53	    caps = sum(w.isupper() and len(w)>2 for w in words) / nw
    54	    bad = len(BAD.findall(s))
    55	    # Repeated short lines are characteristic menus, tag clouds, and catalogs.
    56	    ls = [x.strip().lower() for x in s.splitlines() if x.strip()]
    57	    short_lines = sum(len(x) < 45 for x in ls) / max(1, len(ls))
    58	    line_repeat = 1 - len(set(ls)) / max(1, len(ls))
    59	    punct = sum(s.count(c) for c in '{}[]|<>_=\\') / n
    60	    url = s.lower().count('http') + s.lower().count('www.')
    61	    commercial = len(re.findall(r"(?i)(\$\s?\d|£\s?\d|€\s?\d|\bprice\b|\bshipping\b|\bcheckout\b|\bbuy now\b)",s))
    62	
    63	    # Target-language affinity is deliberately distributional rather than a
    64	    # document lookup: hidden samples from the same registers should transfer.
    65	    uni, total_uni, bi, tri = LANG
    66	    low=[w.lower() for w in words]
    67	    logp=sum(math.log((uni[w]+.1)/(total_uni+10000)) for w in low)/nw
    68	    bicov=sum(x in bi for x in zip(low,low[1:]))/max(1,nw-1)
    69	    tricov=sum(x in tri for x in zip(low,low[1:],low[2:]))/max(1,nw-2)
    70	    stop=sum(w in {'the','of','and','to','a','in','is','that','for','it','on','was','with','as','are','be','by','this','from','or','at','an','have','not','which','but','has','can','will','they','you','their','we'} for w in low)/nw
    71	
    72	    # Smooth quality prior: normal English character/sentence statistics,
    73	    # target phrase coverage, lexical breadth, and coherent paragraphs.
    74	    q = 0.55 * min(math.log1p(n/700), 2.5)
    75	    q += 1.1 * min(max((alpha-.60)/.18, 0), 1)
    76	    q += 0.5 * min(uniq/.48, 1)
    77	    q += 0.45 * min(sent/12, 1)
    78	    q -= 0.045 * abs(avg_sent-22) if 7 < avg_sent < 55 else 2.5
    79	    q += 1.0*(logp+9.0) + 6.0*(bicov-.35) + 2.0*(tricov-.07)
    80	    q -= 5.0*abs(stop-.32)
    81	    q -= 1.0 * caps + 5.0 * weird + 1.55 * bad
    82	    q -= 2.1 * max(0, short_lines-.48) + 2.5 * line_repeat
    83	    q -= 5.0 * max(0, punct-.018) - 0.15 * min(TECH.search(s) is not None, 1)
    84	    q -= .16 * min(url,8) + .10 * min(commercial,10)
    85	    if '<|endoftext|>' in s: q += .12
    86	    if n > 45000: q -= math.log(n/45000)
    87	
    88	    if NEWS.search(s): domain='news'; q += 2.7
    89	    elif '<p>' in s and ('<code>' in s or '<pre>' in s): domain='tech'; q += 2.2
    90	    elif TECH.search(s) and ('?' in s[:1200] or 'how to' in s[:500].lower()): domain='tech'; q += .65
    91	    elif len(REFERENCE.findall(s)) >= 4 and sent >= 8: domain='reference'; q += .45
    92	    else: domain='web'
    93	    return q, domain, nw
    94	
    95	def main(pool=POOL, out=OUT):
    96	    buckets=defaultdict(list)
    97	    for line in open(pool, encoding='utf-8'):
    98	        o=json.loads(line); f=features(o['text'])
    99	        if f is None: continue
   100	        score, domain, nw=f
   101	        # Stable content fingerprint for cheap exact/near-template suppression.
   102	        norm=' '.join(WORD.findall(o['text'][:5000].lower()))
   103	        fp=hashlib.blake2b(norm[:1200].encode(),digest_size=8).digest()
   104	        buckets[domain].append((score,o['id'],nw,fp))
   105	    for b in buckets.values(): b.sort(reverse=True)
   106	
   107	    # Approximate target shares. Extra ids are emitted because exact GPT-2 token
   108	    # counts and the stopping point are deliberately left to the frozen packer.
   109	    shares={'reference':.29,'web':.31,'news':.20,'tech':.20}
   110	    target_words=13_000_000  # comfortably beyond 12M BPE tokens
   111	    picked=[]; seen=set()
   112	    for d,share in shares.items():
   113	        got=0
   114	        for score,i,nw,fp in buckets[d]:
   115	            if score < 2.0: break
   116	            if fp in seen: continue
   117	            seen.add(fp); picked.append((d,score,i,nw)); got += nw
   118	            if got >= target_words*share: break
   119	    # Weighted round-robin in small word chunks gives domain balance throughout
   120	    # priority order, including at the packer's exact 12M-token cutoff.
   121	    by=defaultdict(list)
   122	    for x in picked: by[x[0]].append(x)
   123	    for v in by.values(): v.sort(key=lambda x:x[1],reverse=True)
   124	    idx=Counter(); used=Counter(); result=[]
   125	    total=0
   126	    while True:
   127	        eligible=[d for d in shares if idx[d] < len(by[d])]
   128	        if not eligible: break
   129	        d=min(eligible,key=lambda z: used[z]/shares[z])
   130	        x=by[d][idx[d]]; idx[d]+=1
   131	        result.append(x[2]); used[d]+=x[3]; total+=x[3]
   132	    json.dump(result,open(out,'w'),separators=(',',':'))
   133	    print(f"wrote {len(result)} ids; approx {total:,} words; domains {dict(used)}")
   134	
   135	if __name__=='__main__':
   136	    main(*(sys.argv[1:3]))

Script completed
Wall time 1.2 seconds
Output:
{}SESSION_ID=76259

Script running with cell ID 50
Wall time 11.0 seconds
Output:

Script completed
Wall time 8.3 seconds
Output:
root      2382  0.1  0.0  12852  5428 ?        Ss   01:28   0:00 /bin/bash -c python -m py_compile submission/curate.py && time python submission/curate.py
root      2384 39.2  0.4 5594068 4758296 ?     Sl   01:28   0:12 python submission/curate.py
root      2402 92.7  0.0 5384148 905124 ?      R    01:28   0:24 python submission/curate.py
root      2403 92.7  0.0 5384148 905124 ?      R    01:28   0:24 python submission/curate.py
root      2404 92.8  0.0 5384148 905124 ?      R    01:28   0:24 python submission/curate.py
root      2405 92.7  0.0 5384148 905124 ?      R    01:28   0:24 python submission/curate.py
root      2406 92.7  0.0 5384148 905124 ?      R    01:28   0:24 python submission/curate.py
root      2407 92.7  0.0 5384148 905124 ?      R    01:28   0:24 python submission/curate.py
root      2408 92.7  0.0 5384148 905124 ?      R    01:28   0:24 python submission/curate.py
root      2409 92.5  0.0 5384148 905124 ?      R    01:28   0:24 python submission/curate.py
-rw-r--r-- 1 root root 58K Aug  4 01:25 submission/selection.json

Script running with cell ID 51
Wall time 11.0 seconds
Output:

Script completed
Wall time 0.0 seconds
Output:
root      2382  0.0  0.0  12852  5428 ?        Ss   01:28   0:00 /bin/bash -c python -m py_compile submission/curate.py && time python submission/curate.py
root      2384 27.8  0.4 5607380 4771704 ?     Sl   01:28   0:17 python submission/curate.py
root      2402 93.6  0.0 5384148 905124 ?      R    01:28   0:52 python submission/curate.py
root      2403 93.6  0.0 5384148 905124 ?      R    01:28   0:52 python submission/curate.py
root      2404 93.6  0.0 5384148 905124 ?      R    01:28   0:52 python submission/curate.py
root      2405 93.6  0.0 5384148 905124 ?      R    01:28   0:52 python submission/curate.py
root      2406 93.5  0.0 5384148 905124 ?      R    01:28   0:52 python submission/curate.py
root      2407 93.6  0.0 5384148 905124 ?      R    01:28   0:52 python submission/curate.py
root      2408 93.6  0.0 5384148 905124 ?      R    01:28   0:52 python submission/curate.py
root      2409 93.4  0.0 5384148 905124 ?      R    01:28   0:52 python submission/curate.py
2026-08-04 01:25:24.234375081 +0000 58658

Script running with cell ID 52
Wall time 11.0 seconds
Output:

Script completed
Wall time 0.0 seconds
Output:
2026-08-04 01:29:59.653564282 +0000 80413

Script completed
Wall time 3.6 seconds
Output:
13214 13214 [91095, 101539, 46614, 62018, 3135, 66731, 59719, 44309, 79307, 12142, 92915, 82898, 110424, 24219, 101323, 42707, 95874, 106620, 76836, 27954, 112208, 111105, 53466, 51998, 113923, 84352, 93978, 69551, 32168, 42170, 94042, 45005]
91095 4437  a long time before deciding to write something about it. Why is it so difficult for me to write about successful products? Well, mostly because I spent most of my profes
101539 4566 ’ve done it. Or at least nearly…just waiting for the paperwork to officially be signed, but I am a fully qualified teacher. Gosh that feels scary writing that, but equall
46614 3331  The first sign of a possible test was the detection of seismic waves. The epicenter of the apparent earthquake was close to North Korea's nuclear testing site. Here's da
62018 9104 <|endoftext|>Following the release of her novel The Hidden Child in North America, TheReadingRoom’s Sophie Hamley had the opportunity to interview the hugely popular Swed
3135 9547 When The Conversation launched in the UK five years ago, “fake news” was a term that had yet to be coined and Donald Trump was still only firing people on US television s
66731 11646 , how can you get your fiancé back? Well, the first thing you need to realize is that if you got that far into a relationship with her, she obviously loved you a lot at o
59719 952 <|endoftext|>I would like to say we all make mistakes in life, and I feel that if we were not a doctor we would not have to read about our mistakes every week in the pape
44309 3154  will give some background. I am from a third world country, and I am doing my math PhD in Europe. During high school and university years I participated in many maths co
79307 1667 ’ it back to I don’t remember when. I just know it was nice out that evening and we were at our spot and you let me take a lot of pictures of us. I love these two picture
12142 3721 Benefits of E-learning The benefits of e-learning are more than just the obvious. When you consider the number of people who will be using it over the next several years,
92915 4464 ."<|endoftext|>If you like music (and science tells me that you do), then you'll like listening to music in your target language. And if you like doing something in your 
82898 2570  Best Bail Bond Agencies Near Us People who are caught that are committing a crime or have an arrest warrant would be apprehended. They would be placed in a detention are
110424 7214  is so unique, and it feels like a throwback but still new and fun. What attracts you to that genre? For me, my whole relationship with music, ever since I was kid, was j
24219 3756 You may have heard a lot about being an interesting man, but it is important that you do not only believe what you have read forum of freeapp or just blog. You should und
101323 5082 /NEW YORK (Reuters) - U.S. President Donald Trump on Wednesday blasted the federal court system as “broken and unfair” after a judge blocked his administration’s move to 
42707 21202 <|endoftext|>3982 words · 19 min read I knew. For a long time, I knew I was in part a creation of someone called the scientist. Since the scientist was responsible for th
95874 4709  Tries To Capture Olympic Spirit TAMPA, Fla. — It was a countdown that electrified the nation: The Miracle on Ice, the U.S. men’s hockey team’s stunning defeat of the Sov
106620 5795  experience by Discover Taiji I seem to make it pretty clear that there isn't a single aspect of standing post practice that I enjoy. I would say that perhaps I enjoy the
76836 5193  war in Syria is a vast catastrophe; what began as a peaceable resistance movement against the Assad regime, one among many in the “Arab Spring” of 2011, has devolved int
27954 6298 In life, maybe I am not a person with a cleanliness addiction, but physically, I am a person with a serious cleanliness addiction. Knowing that her husband had been engag
112208 2093  you.<|endoftext|>So this is probably a really strange thing for most people. Why don’t you want to go home and spend time with family during the holidays? I do, I really
111105 8088  here .<|endoftext|>Joe Biden claimed 150,000 Russian soldiers are still in a “threat situation” around Ukraine, Despite Russian allegations of withdrawalAnd he warned th
53466 3904  is official, in just a couple of days it will be 2017! That being said, there will be a lot of people making resolutions for the new year, and it can be fun and daunting
51998 4642  been a news junkie for a long time and I don’t know that it is good for me anymore. The world, or should I say the news, is depressing. And the 24-hour news stations are
113923 1640 <|endoftext|>At the beginning of the year, I prayed to God and asked Him to make this year one of extreme growth for me. I wanted to be surrounded by peace and love, and 
84352 2390  of Having the Help of the Top Personal Injury Lawyer for Your Case The personal injury cases are one of the many types of the issues that you will find in the world toda
93978 10656 ili Reinhart Almost Gave Up Acting Before Being Cast in Riverdale The actress opens up about struggling with depression, the craziness of Riverdale's season two, and the 
69551 7786  segment is from the live stream held by Sega yesterday at the Tokyo Game Show. [Download Video] – 591MB – – – – The Tokyo Game Show is a video game expo/convention held 
32168 5767 An internal Pentagon study has found that most U.S. troops and their families don't care whether gays are allowed to serve openly and think the policy of "don't ask, don'
42170 8053  with Pete Murray Byron Bay singer-songwriter & one of Australia’s most loved musicians, PETE MURRAY has released his new single TAKE ME DOWN, alongside the announcement 
94042 2904 <|endoftext|>I’m a type A – goal oriented person, which basically means I set a goal every year to make New Years resolutions, and I have to create a list so I can check 
45005 2901  Sui-Lee Wee BEIJING (Reuters) - Chinese President Xi Jinping told his U.S. counterpart Barack Obama on Friday that the crisis in Syria should not be resolved through a m

Script completed
Wall time 1.2 seconds
Output:
SESSION_ID=90627

Script running with cell ID 55
Wall time 11.0 seconds
Output:

Script completed
Wall time 7.2 seconds
Output:
root      2636  0.1  0.0  12852  6276 ?        Ss   01:30   0:00 /bin/bash -c cp submission/selection.json /tmp/selection2.json && python pack_selection.py /tmp/selection2.json /tmp/curated2.npy && python train_nano.py --train_npy /tmp/curated2.npy --heldout_npy data/multi_dev.npy --out_json /tmp/curated2_result.json --max_iters 3000 --seed 1337
root      2638 98.5  0.5 6966076 6347960 ?     Rl   01:30   0:33 python pack_selection.py /tmp/selection2.json /tmp/curated2.npy
ls: cannot access '/tmp/curated2.npy': No such file or directory

Script running with cell ID 56
Wall time 11.0 seconds
Output:

Script completed
Wall time 11.7 seconds
Output:
root      2636 31.5  0.4 28798604 5316596 ?    Rsl  01:30   0:19 python train_nano.py --train_npy /tmp/curated2.npy --heldout_npy data/multi_dev.npy --out_json /tmp/curated2_result.json --max_iters 3000 --seed 1337
-rw-r--r-- 1 root root 23M Aug  4 01:31 /tmp/curated2.npy

Script completed
Wall time 0.0 seconds
Output:
{}

Script running with cell ID 58
Wall time 11.0 seconds
Output:

Script completed
Wall time 10.3 seconds
Output:
root      2636 55.3  0.4 28798604 5316596 ?    Ssl  01:30   0:57 python train_nano.py --train_npy /tmp/curated2.npy --heldout_npy data/multi_dev.npy --out_json /tmp/curated2_result.json --max_iters 3000 --seed 1337

Script completed
Wall time 10.1 seconds
Output:
{"train_npy": "/tmp/curated2.npy", "heldout_ppl": 488.3926641210511, "mean_loss": 6.191119721896577, "max_iters": 3000, "seed": 1337, "train_s": 55.6}

Script completed
Wall time 1.2 seconds
Output:
{}SESSION_ID=11150

Script running with cell ID 61
Wall time 11.0 seconds
Output:

Script completed
Wall time 15.9 seconds
Output:
root      2900 35.0  0.4 5600292 4766960 ?     Ssl  01:32   0:14 python submission/curate.py
root      2929 92.5  0.0 5384228 907644 ?      R    01:32   0:33 python submission/curate.py
root      2930 92.4  0.0 5384228 907644 ?      R    01:32   0:33 python submission/curate.py
root      2931 92.5  0.0 5384228 907644 ?      R    01:32   0:33 python submission/curate.py
root      2932 95.0  0.0 5384228 907644 ?      R    01:32   0:33 python submission/curate.py
root      2933 95.1  0.0 5384228 907644 ?      R    01:32   0:33 python submission/curate.py
root      2934 95.0  0.0 5384228 907644 ?      R    01:32   0:33 python submission/curate.py
root      2935 95.1  0.0 5384228 907644 ?      R    01:32   0:33 python submission/curate.py
root      2936 95.1  0.0 5384228 907644 ?      R    01:32   0:33 python submission/curate.py
2026-08-04 01:29:59.653564282 +0000 80413