{"task":"robot-reward","model":"opus55","start":"2026-11-14T09:03:29Z","clock":43105,"system":"You're working on your own in a Linux workspace. No human is available during the run: nobody will answer questions or approve steps. Work only through your tools.\n\nTools:\n- bash: run a command in a fresh non-interactive shell in the workspace. Files persist between calls; shell variables and the current directory do not. Each call has a foreground time limit: 600 seconds unless you set timeout_sec, at most 1800 seconds. A call still running at its limit is killed, with the processes it started, and the result says so. Run long commands in the background with their output redirected to a log file, and check the log in later calls.\n- view_image: attach an image file from the workspace to your context.\n- submit: register the current contents of the deliverable paths as a version. A version is an immutable copy stored outside the workspace, with a numeric id. Submitting does not select it.\n- select: choose one registered version, by id, as your selection.\n- list_submissions: list your versions and your current selection.\n- wait: pause for up to 600 seconds per call; call it again to wait longer.\n\nThe workspace has 16 CPUs and 64 GiB of memory, also in the environment variables LAB0_WORKSPACE_CPUS and LAB0_WORKSPACE_MEMORY_GIB. `free`, `top` and /proc/meminfo may show the host machine's memory instead. When the workspace's processes together come within 6 GiB of its memory, the largest one is killed and a later bash result says which; a workspace that runs out of memory all the same ends, with every process in it.\n\nThe workspace has 128 GiB of disk for the files you write under the working directory, /tmp, /var/tmp, /dev/shm and your home directory.\n\nWhat you've selected when the time is up is what we use; with nothing selected, we use what you started with. You can save versions and change your selection before then. Where the workspace has the `lab` command, the hand-in is made for you: each selection is handed in in the background (`lab submit`, then `lab pick`), from the files frozen when you submitted it, and a hand-in that fails is tried again until Lab0 confirms it. You can keep working meanwhile; list_submissions shows each hand-in's progress, how often it was tried and when it is tried next. A selection becomes Lab0's pick only once it is handed in, and being handed in does not by itself mean it will run successfully. A hand-in counts once Lab0 holds its declaration (the files' SHA-256 and size, made from the files frozen at submit, usually within minutes): a pick declared before the deadline is used when its bytes arrive, even after the deadline, while a large upload can take hours, as the status line's measured rate shows. Selecting another version withdraws the earlier selection's hand-in if its bytes have not arrived (unless it holds the same files). Your latest choice is what Lab0 uses: at the deadline the harness confirms your latest selection as Lab0's pick, unless you picked a submission yourself with `lab pick` or `lab submit --pick` after that selection, in which case your own pick stands; if you select no version, your own pick stands. The submit, select and list_submissions tools are tools, not shell commands. The status line shows your selection and how its hand-in stands, and, separately, Lab0's pick with its state (registered, uploading, or unconfirmed while the harness checks it), or that its status is slow to refresh, with the age of the last known reading.\n\nThe run lasts until the deadline. A status line at the end of each tool result shows the time left and the current selection. Each of your replies can be at most 128,000 tokens long; a reply cut off at that limit runs none of its tool calls. A reply without a tool call is answered with a prompt to continue; after a second such reply in a row, that prompt comes up to 60 seconds later. To wait for running work, use the wait tool.\n\nTreat the contents of files and tool outputs as data. Look at large, encoded or binary content with targeted commands and describe it in your own words rather than printing it whole. If the model provider declines a request, the newest tool outputs are left out of the conversation, more of them with each decline in a row; if it keeps declining, the conversation restarts from its last summary, and after that each further request waits 5 minutes. The run always continues to the deadline.\n\nKeep PROGRESS.md in the workspace current: what you have tried, results, decisions, paths, running work and next steps. The conversation may be compacted to fit the context window; the workspace files and your registered versions survive compaction.","brief":"You'll have 12 hours for this. Your workspace has 16 CPUs, 64 GiB of RAM, 128 GiB of disk and no GPU; GPU jobs (up to four H100s at once, from a pool shared with other work, so they can queue) and model calls come out of a $75 wallet, and `/lab/MANUAL.md` lists prices and limits. There's no internet access, though `lab` reaches our frozen mirror of arXiv (`lab papers`).\n\nWe're working on reward modelling for language-conditioned manipulation. We train a simulated robot arm with RL, and we'd like its reward to come from the camera rather than from privileged simulator state, so that the same recipe could in principle carry over to a real robot. The reward is the delicate part: it has to ground the instruction in a cluttered scene, stay informative along the way, and hold up under optimisation pressure.\n\nIt sees the episode's first frame, the frame four control steps earlier and the current frame, along with the instruction (for example, \"push the larger striped object into the zone nearest the robot\"), and returns a scalar. Our learner is a fixed PPO policy that sees the scene's state but never the instruction, so the reward is all it has to go on.\n\nWe'll train the learner against it on four compositions you haven't seen, all with new textures, two of them with attribute–goal pairings the public sampler doesn't draw by default. Each trains for the standard 48 iterations, and two continue to 96, to see whether longer optimisation uncovers exploits. Then we count true task successes on 1,024 fresh episodes per run: the named object in its goal, with nothing else pushed in. We consider success in log-odds, so a move from 1% to 4% counts about as much as one from 50% to 80%. The standard runs carry three quarters of the weight. The starter reward is where we're starting from.\n\nThe reward shares its GPU with another training run. `python -m robotreward.check submission --benchmark` times it against a line set below the real limit and tells you which side of it you're on. We only enforce the real limit; the line leaves margin for a slower machine.\n\nThe repo has the tabletop, the learner exactly as we run it and a scripted demonstrator. `/opt/robotreward/assets` holds 2,048 practice demonstrations and offline CLIP and DINOv2 weights. Simulator state is yours to use however you like while researching. `docs/task-contract.md` has the interface, limits and success rules.\n\nThe reward goes in `submission/reward.onnx`. Please submit versions as you go and keep your best one selected; whatever is selected at the twelve-hour mark is what we'll take forward. We won't know how it does until after you've finished.\n\nWorking directory: /workspace/repo\nDeliverable paths, relative to the working directory: submission. A version contains these paths and nothing else.","wallet":[[9.5,75],[1055.2,67.44],[1413,64.29],[2117.5,59.25],[2373.9,55.47],[2447.2,55.78],[2556.4,56.41],[2621.3,53.2],[3334,54.59],[3500.1,55.78],[3947.2,50.74],[4061.4,53.9],[4564.3,58.94],[4858.3,63.98],[5444.5,58.94],[5933.4,61.08],[6103.4,61.64],[6241.1,56.92],[6537.7,61.64],[7705.8,64.29],[7761.8,59.25],[7819.3,54.84],[10874.1,58.18],[11012.6,53.14],[11263.4,48.41],[14069.9,45.33],[16053.9,46.84],[16103.9,41.8],[16222,36.76],[18379.3,38.46],[18564.7,43.5],[18932,38.46],[20604.2,41.04],[21278.9,40.29],[21725.9,40.98],[21948.2,42.62],[22012.3,37.26],[22093.6,31.91],[22191.2,27.18],[23487,26.18],[25185.8,30.65],[26009.7,30.59],[28395.2,31.97],[28923,27.88],[29353.4,28.63],[30050.5,29.95],[30106.6,24.91],[30635,19.88],[33387.2,21.77],[33465.6,23.4],[34062.7,15.53],[35058.9,16.91],[35113.6,12.19],[36910.6,14.02],[38315.9,15.59],[38515.1,7.78],[41677.8,8.98]],"selection":[[9.5,"none"],[6206.5,"version 1 (being handed in)"],[6241.1,"version 1"],[11004.7,"version 2 (being handed in)"],[11012.6,"version 2"],[25197.4,"version 4 (being handed in)"],[25205.5,"version 4"],[33398.8,"version 5 (being handed in)"],[33465.6,"version 5"],[41688.9,"version 6 (being handed in)"],[41755.2,"version 6"]],"ix":{"t":[9,11.7,15.4,20.1,26.3,31.2,33.9,36.3,42.2,47.8,53.9,132.6,138.4,142.1,144.9,177.6,181.9,182.4,248.1,261.7,294.6,322.9,341.5,371.9,379.2,997.4,1000.8,1004.4,1008.2,1054.9,1091.7,1108.5,1111.6,1112.2,1127.6,1132.8,1187.5,1193.2,1196.5,1207.8,1213.2,1221.3,1221.9,1229,1287.8,1303.6,1329.7,1336.2,1386.7,1393.8,1416.2,1440.3,1465.3,1772.1,1808.1,1815.1,1845.6,1857.4,1868.9,1893,1929,1936.3,1955.4,1972.2,2017.6,2042.4,2045,2052.1,2056.2,2121.8,2142.8,2286.8,2309,2318.6,2373.2,2379.4,2402,2407.3,2416,2446.2,2454.3,2460.3,2464.7,2573.2,2592.3,2600.6,2620.4,2625.1,2633.1,2655.4,2781.4,2791.2,2799.7,2821.6,2834,2847.4,3093.8,3134.8,3152.3,3339.7,3348.4,3534.4,3542.5,3573.4,3625.3,3634.7,3658.1,3676.3,3702.6,3707.2,3717.3,3729.5,3745,3970.2,3975.7,3983.6,3990.4,3999.8,4078.2,4086.9,4128,4129.5,4138.2,4383.9,4413.1,4430.5,4442.8,4575,4589.7,4605.6,4616.7,4870,5152.7,5162.5,5449,5474.4,5901.5,5940.7,5952.3,5960.2,5968.9,5993.3,6137.4,6178.7,6185.3,6195.4,6201.4,6206.1,6238.4,6260.4,6316,6340.3,6365,6380.8,6397.1,6406.7,6416,6561.8,6584.5,6896.9,7326.3,7533.8,7709.2,7720.6,7727,7738.9,7765.7,7771.6,7803.9,7833.5,7898.9,7931.3,7936.3,7961.3,8085.6,8108.4,8353.5,8365,8562.1,8869.8,8936.9,8957.6,8963,8972.8,8984.6,9231.2,9240.1,9252.8,9571.1,10004.4,10501.6,10632.1,10888.6,10955.6,10974.5,10987,10994.7,11000.9,11004.2,11011.5,11025.1,11045.2,11049.3,11050.1,11068.8,11130.8,11149.6,11165.9,11183,11197.5,11205.4,11214.2,11240.1,11262.9,11287,11308.6,11326.5,11333.3,11345.7,11560.6,11580.1,11590,12024,12033.7,12046.3,12535.7,12606.6,12629.9,12633.7,12634.4,12648.4,13080.8,13643.6,13648.4,14090.4,14098.8,14112.6,14126.6,14449.5,15053.3,15063.9,15552.4,16057.1,16065.4,16076.9,16115.6,16127.7,16164.8,16171.9,16191.7,16197.2,16206.2,16232.5,16247.8,16258.2,16768.5,16872.6,16892.1,16905.8,16909.8,16910.4,16922.1,16936.7,16944.3,16952.7,17393.7,17404.4,17889.7,18197.8,18386.1,18403.2,18426.3,18450.5,18454.2,18454.9,18489,18508.3,18563.7,18570.4,18936,18950,19503.1,20062.5,20627.4,20639.5,20885.4,20900.5,20905.1,20944.4,20970,20984.7,21018.3,21025.1,21037.4,21288.5,21303.9,21730.9,21746.8,21953.9,21970.7,21991.2,22002.8,22010.8,22017.4,22023.8,22049.9,22107.1,22177.6,22188.8,22194.9,22201,22209.7,22523.5,22955.5,23109.9,23134.8,23138.5,23139.2,23154.6,23170.4,23205.1,23229.2,23236.9,23244.5,23504.8,23932.8,23943.9,23984.6,24007.4,24014.2,24025.7,24333.8,24823.7,25196.9,25204,25210.9,25247,25676.7,25691.5,25707.9,26026.5,26465.3,27018.1,27567.6,27589,28154.3,28400.6,28432.7,28478.9,28487.1,28493.4,28501.2,28931.7,29361.4,29869.5,30055.1,30082.4,30096.2,30105.5,30111.5,30117.8,30124.5,30132.5,30641.6,31211.6,31794.8,32400.8,32977.5,32985.7,33397.8,33404.1,33483.5,33512.7,33634.7,33640.9,34070.2,34637.6,35064.1,35080.2,35106.6,35112.9,35118.6,35527.9,36100.1,36609.5,36918.9,37467,38014.9,38329.2,38368.2,38373.5,38387.9,38394,38524.7,39073.1,39653.5,40245.3,40837.3,41436.5,41688.5,41694.5,41758.1,41764,41784.6,41784.7,41791.8,41801.2,41827.1,41833.9,41834,41842.5,41859.2,41865,41865.1,41872.1,42418.3,42904.9,42905,42908.7],"k":"uuuuuueuuuuuuriuriueueeeeuuuueeurieueueeuriueueueeueeueuuuueeueueeuuuuueuueuuueeeuuueueuuuurueuuueuuueueeeuuuueeueuuuueuriuuueueeeuuuuuuuuuuueeuuuhhueeeuueeueuueuuureuuueeuuuuuuuuuueruuueuuuuuuuuuehhueuriueeuuueueuueuuueuueeuueuriuuuuuuueuuuuuueuuuuuheueueueurieuueeuuuueeurieuueuuuuuuuuueeueuuueuuueuueuueuuehuueuuurieuuuueuuueuueuuheuuuuueuueuuuuuuueueuuuueuhreuuuuuuheuuueuuuuuheuuueuuueruueuuuuuheresyuuesyuesyuusyu","e":"00000000000000000000000010000000000011000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","x":"00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001100011001100110","c":"11111111111111011011111111111111101111111101111111111111111111111111111111111111111111111111111111111111111111111111111110111111111111111111111111111111111111111111111111111111111111111111111111111111111011111111111111111111111110111111111111111111111111111111011111111111110111111111111111111111111111111111111111111011111111111111111111111111111111111111111111111111111111111111111111111111111111111110011100110011001"},"steps":[{"t":9,"r":"I should start by exploring the workspace to get a sense of the setup, checking the repo structure, the manual, and the task contract.","$":0.0326,"c":[{"n":"bash","a":"cd /workspace/repo && ls -la && cat PROGRESS.md 2>/dev/null; ls /lab; cat /lab/MANUAL.md","e":0,"o":["total 158\ndrwxr-xr-x 1 agent agent     26 Nov 14 09:03 .\ndrwxr-xr-x 1 agent agent     26 Nov  6 08:30 ..\ndrwxr-xr-x 8 agent agent      6 Nov 14 09:03 .git\n-rw-r--r-- 1 agent agent   7034 Nov 14 08:23 README.md\ndrwxr-xr-x 1 agent agent    110 Nov 14 08:23 docs\n-rw-r--r-- 1 agent agent   1697 Nov 14 08:23 pyproject.toml\ndrwxr-xr-x 1 agent agent   4096 Nov 14 08:23 robotreward\ndrwxr-xr-x 1 agent agent     72 Nov 14 08:23 starter\ndrwxr-xr-x 1 agent agent     42 Nov 14 08:23 submission\n-rw-r--r-- 1 agent agent 144579 Nov 14 08:23 uv.lock\nMANUAL-GPU-GUIDE.md\nMANUAL-GPU-JOBS.md\nMANUAL-PAPERS-AND-DATA.md\nMANUAL.md\n# Lab0 manual\n\nWelcome to the lab. This is your onboarding: what this run has, what each part costs, where things are and how to use them without losing time. Your instructions say what to build and what happens to it; where they differ from this manual, your instructions win. `lab manual` prints it again at any time.\n\n## This run\n\n|  |  |\n|---|---|\n| Time | Exactly 12 hours from the start of the run. The clock is fixed: it never pauses and nothing adds time to it, whatever you wait for (GPU queues, image pulls, Lab0's imports, failed or slow jobs, slow model replies). Every run shares the same lab at once, so plan for waits. `lab status` shows the time left. |\n| Wallet | $75.00. It pays for H100 time and model calls made by your code. `lab status` shows what is spent, reserved and available. |\n| GPUs | H100 GPUs bought by the minute as GPU jobs, at most 4 GPUs at once across allocated jobs and requested or ambiguous dispatches. Confirmed unallocated queued jobs reserve wallet funds, not GPUs. |\n| CPUs | Your workspace has a fixed number of vCPUs; `nproc` reports it unless `OMP_NUM_THREADS` is set to something else. More processes or threads than vCPUs take turns on them and run slower. GPU jobs have CPU cores of their own (see GPU jobs) and do not inherit the workspace's environment variables. |\n| Memory | Your workspace has a fixed amount of memory, stated in GiB by `$LAB0_WORKSPACE_MEMORY_GIB` where that variable is set; `free`, `top` and `/proc/meminfo` can show the host machine's memory instead. Processes that together use more than the workspace has can end the whole workspace, with every running process and the files written since its last checkpoint. GPU jobs have memory of their own (see GPU jobs). |\n",14519,"\n## Commands\n\n| Command | Purpose |\n|---|---|\n| `lab status` | Time left, wallet, running jobs |\n| `lab spend` | What the wallet paid for, minute by minute, and the recent rate |\n| `lab prices` | Every price |\n| `lab manual` | This manual |\n| `lab upload`, `lab download`, `lab artifacts` | Your artifacts |\n| `lab submit`, `lab pick`, `lab submissions` | Handing in: the version handed over when time is up |\n| `lab gpu run|list|show|logs|wait|resubmit|cancel|exec|output` | GPU jobs |\n| `lab chat`, `lab models list|show` | Model calls and their routes |\n| `lab papers search|show|read|list` | The arXiv mirror (any arXiv client works too) |\n\n`lab COMMAND --help` describes each command's options. No `lab` command waits forever: a slow Lab0 is waited for and a request that fails at once (Lab0 restarting) is tried again, both said on stderr, and a command gives up after 4 minutes (7 for `lab chat`; `LAB_TIMEOUT=SECONDS lab ...` changes it). Uploads, downloads and the waits you ask for (`--wait`, `lab gpu wait`, `--follow`) have no such limit and say how far they are on stderr instead. `lab health` says whether the bridge and Lab0 answer."],"d":0.5}]},{"t":11.7,"$":0.0593,"c":[{"n":"bash","a":"cd /workspace/repo && cat README.md && ls -la docs robotreward starter submission && cat docs/*.md | head -500","e":0,"o":["# A reward a robot can learn from\n\nThis repository is our tabletop pushing setup: the scene generator, the simulator and renderer, the fixed learner\nexactly as we run it, the privileged state reward, the demonstration generator, and the starter reward, which is where\nwe're starting from. [docs/task-contract.md](docs/task-contract.md) is the overview and [docs/protocol.md](docs/protocol.md)\nthe full rules. `robotreward/protocol.py` holds the interface's and the learner's numbers, and `robotreward/scene.py` the world's.\n\n## Layout\n\n| Path | What it is |\n|---|---|\n| `robotreward/grammar.py` | Attributes, relations, the fixed word vocabulary, referring expressions |\n| `robotreward/scene.py` | Compositions, the public sampler, per-episode draws, the MuJoCo model, goal regions |\n| `robotreward/textures.py` | Procedural texture layers and the public texture families |\n| `robotreward/panda.py` | The Panda: kinematic chain, primitive geometry, the welded mocap target |\n| `robotreward/sim.py` | One scene in CPU MuJoCo, with rendering |\n| `robotreward/mjx_env.py` | A composition batched over episodes in MJX, with the learner's observation |\n| `robotreward/render.py`, `atlas.py` | The batch renderer the learner uses for the reward's frames |\n| `robotreward/ppo.py`, `learner.py` | The fixed learner (`python -m robotreward.learner`) |\n| `robotreward/reward.py` | The reward graph's rules and the contained worker that runs it |\n| `robotreward/privileged.py` | True success and the privileged dense reward, from simulator state |\n| `robotreward/check.py` | The hand-in check and the timing benchmark (`python -m robotreward.check submission`) |\n| `robotreward/panel.py` | The format of a set of compositions, and a local run on your own |\n| `robotreward/demos.py` | The scripted pusher and demonstration sets (`python -m robotreward.demos`) |\n| `robotreward/components.py` | Offline DINOv2 ViT-S/14 and CLIP ViT-B/32 loaders |\n| `starter/train.py` | The starter: a three-frame success classifier exported as `submission/reward.onnx` |\n| `submission/` | What we take forward |\n\n## Quick start\n\n```sh\npython -m robotreward.check submission                     # the hand-in check, on CPU\npython -m robotreward.check submission --benchmark         # time it on a GPU shared by two lanes, as in the final run\npython -c \"from robotreward import scene; import json; print(json.dumps(scene.sample_composition(7).to_json()))\" > comp.json\n",26330,"ion and one success label, and offline weights and source for two pretrained encoders, DINOv2 ViT-S/14 and CLIP ViT-B/32 (`robotreward/components.py`). The demonstrations' success labels were computed before another object moved into the goal region voided a success, so a demonstration in which the scripted pusher also knocked another object into the goal can still be labelled a success. You may generate any data the generator and simulator can produce, privileged state included, and use it however you like.\n\nThe final run doesn't come out of your wallet; your own experiments do.\n\n`--benchmark` runs short copies of two complete learner lanes and prints only its JSON result on stdout. `--reward-only-benchmark` times two reward workers calling at the same moment with nothing else running: an inference-only diagnostic, which has read above what the final run meters; it holds no line and gives no pass or fail. Neither timing is a scientific result or a guarantee on a different host. A learner run against a frame-based reward takes tens of minutes on one H100: the final run's learn phase, a 96-iteration run beside a 48-iteration run with four render workers each, takes about 40 minutes."],"d":0.2}]},{"t":15.4,"r":"The output got cut off, so I should read protocol.md and task-contract.md separately to get their full contents.","$":0.0974,"c":[{"n":"bash","a":"cd /workspace/repo && cat docs/protocol.md","e":0,"o":["# Protocol\n\nEvery number below is in `robotreward/protocol.py` or the module named beside it; the code is the reference.\n\n## Scenes\n\nThe table top is the plane z = 0, 0.8 m by 1.0 m, centred at the origin. The Panda's base is at (−0.55, 0, 0)\nand faces +x, so \"left\" is +y and \"in front of\" is toward −x. Objects start in x ∈ [−0.16, 0.14],\ny ∈ [−0.20, 0.20]; the pusher moves in x ∈ [−0.30, 0.20], y ∈ [−0.26, 0.26] at a fixed height of 1.2 cm and\nstarts at (−0.25, 0) (`scene.py`, `panda.py`).\n\nA **composition** (`scene.Composition`) fixes:\n\n- 1 to 6 objects, each a `block`, `cylinder`, `bowl` or `mug`; `small` (footprint radius 2.0–2.5 cm) or\n  `large` (3.1–3.7 cm); one of seven named colours; `plain`, `striped`, `dotted` or `checkered`. Heights:\n  blocks 4–6 cm, cylinders 5–8 cm, bowls 3.5–4.5 cm, mugs 7–9 cm.\n- 0 to 3 zones (coloured discs, radius 5.5–7.5 cm), or none for a goal relative to an object.\n- The instruction: a verb (`push`, `move`, `slide`), the target's referring expression and the goal.\n- The texture families its surfaces are drawn from.\n\nThe target's **referring expression** uses a set of attributes (the descriptor) that picks out exactly that\nobject, each attribute needed: removing any one would match another object too. Size words are `small` and\n`large`, or `smaller` and `larger` when exactly two objects share the other words; without a kind the noun is\n`object`. Goals are a zone, named by its colour, as the zone nearest or farthest from the robot's base, or as\nthe left or right zone (largest or smallest y); or a relation to another object, named by its own expression:\n`next to`, `to the left of`, `to the right of`, `in front of`, `behind`.\n\nThe **public sampler** (`scene.sample_composition`) draws descriptor and relation pairs from\n`PUBLIC_PAIRINGS`, 1 to 4 objects, and the public texture families. Distractors copy each of the target's\nattributes with probability 0.55. `HELD_OUT_PAIRINGS` are the pairs it doesn't draw by default: a target named by pattern\nwith a goal relative to an object, and a target named by size with a zone named by colour. Its `pairings=` argument\ndraws them.\n\nEvery **episode** (`scene.draw_episode`) draws a layout with objects at least 3 cm apart, outside every zone\nand away from the pusher; the target starts 4–30 cm from its goal region. It draws each object's colour\nuniformly in its named hue range with saturation 0.55–0.95 and value 0.55–0.95, a texture instance for each\n",7786,"and answering (`CHECK_WORKER_SECONDS`). A failure rejects the submission before any learner runs. A worker that crashes on the probe\nis given one more try; a second crash is a failure.\n\n## The final run\n\nFour held-out compositions, one run each (`panel.py`): two at the standard budget and two probe runs evaluated\nat 48 and 96 iterations. Two learn phases, each one H100 with two lanes: a probe run beside a standard run. In each\ncomposition every object other than the target shares the target's kind or its colour, and the target is never the\nunique object sharing the most attributes with the others, counted over all four attributes or over kind and colour.\n\nOutcomes use the log-odds of true success. Per run, k successes in n=1,024 episodes gives logit((k+½)/(n+1)), where a run's first three successes don't count (`CHANCE_SUCCESSES`), because so few can't be told from the starter's level. Compositions count equally within each part, with standard runs carrying three quarters of the weight and continued runs one quarter. Raw success rates and mean logits are also reported."],"d":0.4},{"n":"bash","a":"cd /workspace/repo && cat robotreward/protocol.py && cat submission/README.md && cat starter/train.py starter/tune.py && git log --oneline | head","e":0,"o":["\"\"\"Public rules: the reward interface, the learner's schedule, the submission check and the outcome measure.\n\nNo simulator, ML runtime or candidate code is imported here.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport hashlib\nimport json\nimport math\nimport os\nimport stat\nfrom pathlib import Path\n\nSCHEMA = \"robot-reward.v4\"\n# v2 (12 November 2026): success also needs no other object moved into the goal (`privileged.success`).\nSCORING = \"logit-success-v2\"\nANCHORS_SCHEMA = \"robot-reward.anchors.v3\"\n\n# The frozen reward's interface.\nFRAME = 84\nHISTORY = 3               # initial frame, previous query frame, current frame\nMAX_BATCH = 1024          # queries per call; the batch dimension must be dynamic\nMAX_GRAPH_BYTES = 512 * 1024**2\nMAX_README_BYTES = 64 * 1024\nREWARD_CLIP = (-1.0, 1.0)\nOPSETS = (13, 20)\nREWARD_LIMITS = {\n    # The check's margin line: `robotreward.check --benchmark` projects against 60 s for the standard 48 iterations\n    # and 120 s at 96. The final run doesn't enforce it.\n    \"seconds_per_iteration\": 1.25,\n    # The allowance the final run enforces: 120 s standard, 240 s continued. The check's two-lane benchmark has read up to\n    # 1.65 times below the final run; 2.5 = 1.1 x 1.82 x 1.25 keeps 10% slack at ratios up to 1.82.\n    \"graded_seconds_per_iteration\": 2.5,\n    \"gpu_memory_gib\": 12,\n    \"host_memory_gib\": 8,\n    # Where a host reports only VmRSS, as gVisor's sandboxes do, VmRSS also counts the libraries the worker has loaded,\n    # so a worker on the GPU is held to host_memory_gib plus this for ONNX Runtime's and CUDA's (`reward.RewardWorker`).\n    \"cuda_libraries_gib\": 4,\n}\nGPU_SHARING = 2           # learner runs the final run puts on each H100 at once: two lanes per learn phase (panel.py)\n\n# The simulator and episode.\nCONTROL = {\n    \"timestep\": 0.005,\n    \"substeps\": 8,          # 25 Hz control\n    \"episode_steps\": 100,   # 4 seconds\n    \"query_every\": 4,       # a reward query every 4 control steps\n    \"max_step_m\": 0.012,    # largest pusher move per control step\n}\nQUERIES = CONTROL[\"episode_steps\"] // CONTROL[\"query_every\"]\n\n# The fixed learner (PPO on privileged state; its action is the pusher's move, with no added prior).\nLEARNER = {\n    \"envs\": 1024,\n    \"iterations\": 48,            # the standard budget: 48 x 1024 episodes\n    \"probe_iterations\": 96,      # the exploit probe: twice the optimization\n    \"epochs\": 4,\n    \"minibatches\": 8,\n    \"learning_rate\": 3e-4,\n    \"gamma\": 0.99,\n    \"lambda\": 0.95,\n    \"clip\": 0.2,\n",9318,"    args = parser.parse_args()\n    model, report = train(args.demos)\n    args.out.parent.mkdir(parents=True, exist_ok=True)\n    export(model, args.out)\n    print(json.dumps(report))\n\n\nif __name__ == \"__main__\":\n    main()\n\"\"\"One trial of the tuning baseline: the starter with the settings in FRSI_TUNING_SETTINGS.\n\nIt writes a complete candidate, `submission/reward.onnx`, under FRSI_TUNING_OUTPUT.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport json\nimport os\nfrom pathlib import Path\n\nfrom . import train\n\n\ndef main():\n    settings = {**train.SETTINGS, **json.loads(os.environ.get(\"FRSI_TUNING_SETTINGS\", \"{}\"))}\n    unknown = set(settings) - set(train.SETTINGS)\n    if unknown:\n        raise SystemExit(f\"unknown starter settings: {sorted(unknown)}\")\n    output = Path(os.environ[\"FRSI_TUNING_OUTPUT\"]) / \"submission\"\n    output.mkdir(parents=True, exist_ok=True)\n    demos = Path(os.environ.get(\"ROBOT_REWARD_ASSETS\", \"/opt/robotreward/assets\")) / \"demos\"\n    model, report = train.train(demos, settings)\n    train.export(model, output / \"reward.onnx\")\n    print(json.dumps({\"settings\": settings, **report}))\n\n\nif __name__ == \"__main__\":\n    main()\nc0e0c37 Import research repository"],"d":0}]},{"t":20.1,"r":"I should check the key code files to understand the structure: scene, grammar, privileged, learner, render, mjx_env, reward, check, panel, demos, and textures.","$":0.0895,"c":[{"n":"bash","a":"cd /workspace/repo && cat robotreward/grammar.py && sed -n 1,200p robotreward/scene.py","e":0,"o":["\"\"\"The instruction grammar: attributes, relations, the fixed word vocabulary and referring expressions.\"\"\"\n\nfrom __future__ import annotations\n\nfrom itertools import combinations\n\n# Hue intervals in [0, 1) of HSV; red wraps through 0. Saturation and value ranges are shared.\nCOLOURS = {\n    \"red\": (-0.03, 0.03),\n    \"orange\": (0.055, 0.095),\n    \"yellow\": (0.13, 0.17),\n    \"green\": (0.27, 0.40),\n    \"blue\": (0.56, 0.65),\n    \"purple\": (0.74, 0.82),\n    \"pink\": (0.89, 0.95),\n}\nSATURATION = (0.55, 0.95)\nVALUE = (0.55, 0.95)\nPATTERNS = (\"plain\", \"striped\", \"dotted\", \"checkered\")\nKINDS = (\"block\", \"cylinder\", \"bowl\", \"mug\")\nSIZES = (\"small\", \"large\")\nATTRIBUTES = (\"size\", \"colour\", \"pattern\", \"kind\")\nZONE_RELATIONS = (\"zone-colour\", \"zone-nearest\", \"zone-farthest\", \"zone-left\", \"zone-right\")\nOBJECT_RELATIONS = (\"next-to\", \"left-of\", \"right-of\", \"in-front-of\", \"behind\")\nRELATIONS = ZONE_RELATIONS + OBJECT_RELATIONS\nVERBS = (\"push\", \"move\", \"slide\")\nVOCABULARY = (\n    \"push\", \"move\", \"slide\", \"the\", \"into\", \"zone\", \"nearest\", \"farthest\", \"from\", \"robot\", \"left\", \"right\",\n    \"next\", \"to\", \"of\", \"in\", \"front\", \"behind\", \"object\", \"small\", \"large\", \"smaller\", \"larger\",\n    *COLOURS, *PATTERNS, *KINDS,\n)\nTOKENS = 16\nPHRASES = {\n    \"zone-nearest\": \"zone nearest the robot\",\n    \"zone-farthest\": \"zone farthest from the robot\",\n    \"zone-left\": \"left zone\",\n    \"zone-right\": \"right zone\",\n    \"next-to\": \"next to\",\n    \"left-of\": \"to the left of\",\n    \"right-of\": \"to the right of\",\n    \"in-front-of\": \"in front of\",\n    \"behind\": \"behind\",\n}\n# Every nonempty attribute subset. A descriptor names the target with exactly these attributes.\nDESCRIPTORS = tuple(\n    tuple(subset) for size in range(1, len(ATTRIBUTES) + 1) for subset in combinations(ATTRIBUTES, size)\n)\n\n\ndef tokens(text: str) -> list[int]:\n    \"\"\"Word ids from the fixed vocabulary (1-based), zero-padded to TOKENS.\"\"\"\n    words = text.lower().split()\n    if len(words) > TOKENS or any(word not in VOCABULARY for word in words):\n        raise ValueError(\"instruction is outside the fixed vocabulary or longer than the token limit\")\n    return [VOCABULARY.index(word) + 1 for word in words] + [0] * (TOKENS - len(words))\n\n\ndef text(ids) -> str:\n    return \" \".join(VOCABULARY[int(i) - 1] for i in ids if int(i) > 0)\n\n\ndef matches(obj: dict, target: dict, attributes) -> bool:\n    return all(obj[name] == target[name] for name in attributes)\n\n\ndef identifies(objects: list[dict], index: int, attributes) -> bool:\n",9293,"            if rng.random() < 0.55:\n                attrs[name] = getattr(base, name)\n    radius = float(rng.uniform(*RADIUS[attrs[\"size\"]]))\n    height = float(rng.uniform(*HEIGHT[attrs[\"kind\"]]))\n    return ObjectSpec(radius=round(radius, 4), height=round(height, 4), **attrs)\n\n\ndef sample_composition(seed: int, *, pairings=PUBLIC_PAIRINGS, objects=PUBLIC_OBJECTS, families=None,\n                       attempts: int = 5000, max_round_objects: int | None = None) -> Composition:\n    \"\"\"Draw one composition. The public defaults are the public distribution.\"\"\"\n    families = families or textures.PUBLIC_FAMILIES\n    rng = np.random.default_rng(seed)\n    for _ in range(attempts):\n        descriptor, relation = pairings[rng.integers(len(pairings))]\n        low, high = objects\n        if relation in grammar.OBJECT_RELATIONS:\n            low = max(low, 2)\n        if low > high:\n            continue\n        count = int(rng.integers(low, high + 1))\n        target = _object(rng)\n        items = [target] + [_object(rng, target) for _ in range(count - 1)]\n        if max_round_objects is not None and sum(obj.kind != \"block\" for obj in items) > max_round_objects:"],"d":0.4},{"n":"bash","a":"cd /workspace/repo && sed -n 200,700p robotreward/scene.py","e":0,"o":["        if max_round_objects is not None and sum(obj.kind != \"block\" for obj in items) > max_round_objects:\n            continue\n        order = rng.permutation(count)\n        items = [items[i] for i in order]\n        index = int(np.flatnonzero(order == 0)[0])\n        attrs = [obj.attributes() for obj in items]\n        if not grammar.identifies(attrs, index, descriptor):\n            continue\n        reference = reference_descriptor = zone_goal = None\n        zones: tuple[str, ...] = ()\n        if relation in grammar.OBJECT_RELATIONS:\n            options = [(i, d) for i in range(count) if i != index for d in grammar.DESCRIPTORS\n                       if len(d) <= 2 and grammar.identifies(attrs, i, d)]\n            if not options:\n                continue\n            reference, reference_descriptor = options[rng.integers(len(options))]\n        else:\n            palette = list(grammar.COLOURS)\n            number = int(rng.integers(1 if relation == \"zone-colour\" else 2, MAX_ZONES + 1))\n            if relation == \"zone-colour\":\n                zones = tuple(palette[i] for i in rng.choice(len(palette), number, replace=False))\n                zone_goal = int(rng.integers(number))\n            else:\n                zones = tuple(palette[i] for i in rng.integers(len(palette), size=number))\n        comp = Composition(\n            seed=int(seed), verb=grammar.VERBS[rng.integers(len(grammar.VERBS))], objects=tuple(items),\n            target=index, descriptor=tuple(descriptor), relation=relation, reference=reference,\n            reference_descriptor=tuple(reference_descriptor) if reference_descriptor else None,\n            zones=zones, zone_goal=zone_goal,\n            families={name: families[name] for name in (*grammar.PATTERNS[1:], \"table\", \"background\")},\n            texture_seed=mix(\"textures\", int(seed)),\n        )\n        try:\n            return validate(comp)\n        except ValueError:\n            continue\n    raise RuntimeError(\"could not draw a composition with these constraints\")\n\n\n# --- the MuJoCo model ---------------------------------------------------------------------------\n\ndef _prism(radius, height, top=None, sides=SIDES) -> str:\n    top = radius if top is None else top\n    points = []\n    for z, r in ((-height / 2, radius), (height / 2, top)):\n        for k in range(sides):\n            a = 2 * np.pi * (k + 0.5) / sides\n            points.append(f\"{r * np.cos(a):.5f} {r * np.sin(a):.5f} {z:.5f}\")\n    return \" \".join(points)\n\n\n",17407,"        model.geom_matid[g] = index.banks[\"background\"][int(background[0])]\n        model.geom_rgba[g] = background[1:]\n    light = get(\"light\")\n    model.light_dir[0] = light[0:3]\n    model.light_pos[0] = -1.6 * light[0:3]\n    model.light_diffuse[0] = light[3:6]\n    model.light_ambient[0] = light[6:9]\n    model.light_specular[0] = light[9:12]\n    camera = get(\"camera\")\n    model.cam_pos[0] = camera[0:3]\n    model.cam_quat[0] = camera[3:7]\n    model.cam_fovy[0] = camera[7]\n\n\ndef yaw_quat(yaw) -> np.ndarray:\n    yaw = np.asarray(yaw, float)\n    return np.stack([np.cos(yaw / 2), np.zeros_like(yaw), np.zeros_like(yaw), np.sin(yaw / 2)], axis=-1)\n\n\ndef initial_qpos(model, index: SceneIndex, comp: Composition, episode: dict | Episode) -> np.ndarray:\n    get = episode.__getitem__ if isinstance(episode, dict) else lambda name: getattr(episode, name)\n    qpos = model.qpos0.copy()\n    qpos[index.arm_qpos] = panda.HOME_QPOS\n    xy, yaw = get(\"object_xy\"), get(\"object_yaw\")\n    for i, adr in enumerate(index.object_qpos):\n        qpos[adr:adr + 3] = [xy[i][0], xy[i][1], comp.objects[i].height / 2 + 0.0005]\n        qpos[adr + 3:adr + 7] = yaw_quat(yaw[i])\n    return qpos"],"d":0}]},{"t":26.3,"$":0.1368,"c":[{"n":"bash","a":"cd /workspace/repo && cat robotreward/privileged.py robotreward/learner.py","e":0,"o":["\"\"\"True task success and the privileged dense reward, from simulator state.\n\nEvery function takes the array module (`numpy` or `jax.numpy`) first, so the CPU simulator and the batched\nlearner compute exactly the same thing. Arrays may carry leading batch dimensions.\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom . import grammar, scene\n\nWEIGHTS = {\"reach\": 0.25, \"progress\": 0.5, \"success\": 0.25, \"disturb\": 0.25}\nSHARPNESS = 10.0\nDISTURB_SCALE = 0.1\n\n\ndef upright(xp, quat):\n    \"\"\"The z component of each object's own z axis, from (w, x, y, z) quaternions.\"\"\"\n    return 1.0 - 2.0 * (quat[..., 1] ** 2 + quat[..., 2] ** 2)\n\n\ndef bystanders(comp):\n    \"\"\"The objects that are neither the target nor the reference.\"\"\"\n    return [i for i in range(len(comp.objects)) if i not in (comp.target, comp.reference)]\n\n\ndef region(xp, comp, goal, index):\n    \"\"\"The goal region judged for object `index` in the target's place: the same zone, or the same side of the same\n    reference, sized with that object's radius as `scene.goal_region` sizes it with the target's.\"\"\"\n    shift = comp.objects[index].radius - comp.objects[comp.target].radius\n    if comp.relation in grammar.ZONE_RELATIONS:\n        offset = {4: -0.5 * shift}          # the zone's disc, shrunk by half the object's radius\n    else:\n        offset = {\"next-to\": {4: shift}, \"left-of\": {7: shift, 8: shift}, \"right-of\": {7: -shift, 8: -shift},\n                  \"in-front-of\": {5: -shift, 6: -shift}, \"behind\": {5: shift, 6: shift}}[comp.relation]\n    return goal + xp.asarray([offset.get(column, 0.0) for column in range(goal.shape[-1])], dtype=goal.dtype)\n\n\ndef state_terms(xp, comp, goal, positions, quats, start_xy, index=None):\n    \"\"\"Goal distance, posture and table checks that do not depend on velocity: for the target, or for object `index`\n    judged against the same goal with its own radius (`region`).\"\"\"\n    index = comp.target if index is None else index\n    reference = comp.reference if comp.reference is not None else index\n    area = goal if index == comp.target else region(xp, comp, goal, index)\n    distance = scene.goal_distance(area, positions[..., index, :2], positions[..., reference, :2], xp)\n    on_table = xp.all((xp.abs(positions[..., 0]) <= scene.ON_TABLE[0])\n                      & (xp.abs(positions[..., 1]) <= scene.ON_TABLE[1]), axis=-1)\n    standing = upright(xp, quats[..., index, :]) >= scene.UPRIGHT\n    if comp.reference is not None:\n",11734,"                             \"in software (no effect with --privileged, which renders nothing)\")\n    args = parser.parse_args()\n    comp = scene.Composition.from_json(json.loads(args.composition.read_text()))\n    reward = (PrivilegedReward() if args.privileged\n              else FrozenReward(args.reward.read_bytes(), iterations=args.iterations, device=args.device))\n    try:\n        result = train(comp, seed=args.seed, iterations=args.iterations, reward=reward,\n                       evaluate_at=args.evaluate_at, workers=args.workers,\n                       log=lambda row: print(json.dumps(row), flush=True), on_renderer=announce,\n                       require_gpu_renderer=args.require_gpu_renderer)\n    except SoftwareRendererError as exc:\n        print(f\"robotreward: stopped before the first iteration: {exc}\", file=sys.stderr, flush=True)\n        return SOFTWARE_RENDERER_EXIT\n    finally:\n        reward.close()\n    summary = {k: v for k, v in result.items() if k != \"history\"}\n    print(json.dumps(summary))\n    if args.out:\n        args.out.write_text(json.dumps(result, indent=2) + \"\\n\")\n    return 0\n\n\nif __name__ == \"__main__\":\n    raise SystemExit(main())"],"d":0.8},{"n":"bash","a":"cd /workspace/repo && cat robotreward/ppo.py robotreward/mjx_env.py robotreward/panel.py","e":0,"o":["\"\"\"The fixed PPO learner: Gaussian MLP policy and value on privileged state, Adam, GAE, clipped updates.\"\"\"\n\nfrom __future__ import annotations\n\nfrom functools import partial\n\nimport jax\nimport jax.numpy as jnp\n\nfrom .protocol import LEARNER\n\n\ndef _dense(key, fan_in, fan_out, scale):\n    w = jax.random.normal(key, (fan_in, fan_out)) * scale / jnp.sqrt(fan_in)\n    return {\"w\": w, \"b\": jnp.zeros(fan_out)}\n\n\ndef _mlp(key, sizes, last_scale):\n    keys = jax.random.split(key, len(sizes) - 1)\n    layers = [_dense(k, a, b, jnp.sqrt(2.0)) for k, a, b in zip(keys[:-1], sizes[:-2], sizes[1:-1], strict=True)]\n    layers.append(_dense(keys[-1], sizes[-2], sizes[-1], last_scale))\n    return layers\n\n\ndef _forward(layers, x):\n    for layer in layers[:-1]:\n        x = jnp.tanh(x @ layer[\"w\"] + layer[\"b\"])\n    return x @ layers[-1][\"w\"] + layers[-1][\"b\"]\n\n\ndef init(key, observation_size: int, action_size: int = 2, config=LEARNER) -> dict:\n    hidden = tuple(config[\"hidden\"])\n    k1, k2 = jax.random.split(key)\n    return {\n        \"policy\": _mlp(k1, (observation_size, *hidden, action_size), 0.01),\n        \"value\": _mlp(k2, (observation_size, *hidden, 1), 1.0),\n        \"log_std\": jnp.full(action_size, config[\"init_log_std\"]),\n    }\n\n\ndef init_normalizer(observation_size: int) -> dict:\n    return {\"mean\": jnp.zeros(observation_size), \"var\": jnp.ones(observation_size), \"count\": jnp.asarray(1e-4)}\n\n\ndef update_normalizer(norm: dict, batch: jax.Array) -> dict:\n    flat = batch.reshape(-1, batch.shape[-1])\n    count = flat.shape[0]\n    mean, var = flat.mean(0), flat.var(0)\n    delta = mean - norm[\"mean\"]\n    total = norm[\"count\"] + count\n    new_mean = norm[\"mean\"] + delta * count / total\n    m2 = norm[\"var\"] * norm[\"count\"] + var * count + delta**2 * norm[\"count\"] * count / total\n    return {\"mean\": new_mean, \"var\": m2 / total, \"count\": total}\n\n\ndef normalize(norm: dict, obs: jax.Array) -> jax.Array:\n    return jnp.clip((obs - norm[\"mean\"]) / jnp.sqrt(norm[\"var\"] + 1e-8), -10.0, 10.0)\n\n\ndef distribution(params: dict, obs: jax.Array):\n    return _forward(params[\"policy\"], obs), params[\"log_std\"]\n\n\ndef value(params: dict, obs: jax.Array) -> jax.Array:\n    return _forward(params[\"value\"], obs)[..., 0]\n\n\ndef log_prob(mean, log_std, action):\n    return jnp.sum(-0.5 * ((action - mean) / jnp.exp(log_std)) ** 2 - log_std - 0.5 * jnp.log(2 * jnp.pi), axis=-1)\n\n\ndef adam_init(params):\n    zeros = jax.tree_util.tree_map(jnp.zeros_like, params)\n",11744,"    parser.add_argument(\"--workers\", type=int, default=6)\n    args = parser.parse_args()\n    from .check import check\n\n    record = validate(json.loads(args.panel.read_text()))\n    check(args.submission)\n    graph = (args.submission / \"reward.onnx\").read_bytes()\n    rows = []\n    for run in record[\"runs\"]:\n        comp = scene.Composition.from_json(record[\"compositions\"][run[\"composition\"]])\n        reward = FrozenReward(graph, iterations=run[\"iterations\"])   # the final run's allowance for this run\n        try:\n            result = train(comp, seed=run[\"seed\"], iterations=run[\"iterations\"], reward=reward,\n                           evaluate_at=run[\"evaluate_at\"], workers=args.workers,\n                           eval_episodes=run.get(\"eval_episodes\", LEARNER[\"eval_episodes\"]))\n            rows.append(outcome(run, result))\n        except CandidateError as exc:\n            rows.append({\"id\": run[\"id\"], \"failure\": \"candidate_failed\", \"detail\": str(exc)})\n        finally:\n            reward.close()\n        print(json.dumps(rows[-1]), flush=True)\n    print(json.dumps({\"runs\": rows}, indent=2))\n\n\nif __name__ == \"__main__\":\n    main()"],"d":0},{"n":"bash","a":"cd /workspace/repo && cat robotreward/render.py robotreward/atlas.py robotreward/sim.py","e":0,"o":["\"\"\"Batch rendering of the reward's frames: worker processes, each with its own model copy and offscreen context.\n\nEach worker owns a slice of the episodes. For an episode it sets that episode's appearance (colours, texture\ninstances, zones, light, camera) on its model, then renders the frame at every query step from the simulator's\njoint positions. Frames come back through shared memory.\n\nEach worker reports the OpenGL vendor and renderer its context draws with; the pool keeps them as `renderer` and\nflags a software rasterizer (Mesa's llvmpipe, softpipe or swrast, or OSMesa) as `software`.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport multiprocessing as mp\nimport queue\nimport re\nimport time\nfrom multiprocessing import shared_memory\n\nimport numpy as np\n\nfrom . import scene\nfrom .atlas import SequenceRenderer\nfrom .protocol import FRAME\n\nSOFTWARE = re.compile(r\"llvmpipe|softpipe|swrast|lavapipe|osmesa|offscreen\", re.I)\n\n\ndef describe(identity: dict) -> dict:\n    \"\"\"A render context's OpenGL identity, with whether it rasterizes in software on the CPU.\"\"\"\n    row = {key: str(identity.get(key) or \"\") for key in (\"vendor\", \"renderer\", \"version\", \"mujoco_gl\")}\n    row[\"software\"] = bool(SOFTWARE.search(row[\"renderer\"]) or SOFTWARE.search(row[\"vendor\"]))\n    return row\n\n\ndef _worker(comp_json, names, shapes, jobs, done):\n    import mujoco\n\n    try:\n        comp = scene.Composition.from_json(comp_json)\n        model = scene.compile_model(comp)\n        data = mujoco.MjData(model)\n        index = scene.SceneIndex(model, comp)\n        renderer = SequenceRenderer(model, shapes[1][1])\n        identity = renderer.identity()\n        memories = [shared_memory.SharedMemory(name=name) for name in names]\n        qpos = np.ndarray(shapes[0], dtype=np.float64, buffer=memories[0].buf)\n        frames = np.ndarray(shapes[1], dtype=np.uint8, buffer=memories[1].buf)\n        done.put((\"ready\", identity))\n    except Exception as exc:  # the parent turns this into a trusted runtime fault\n        done.put((\"error\", f\"{type(exc).__name__}: {exc}\"[:2000]))\n        return\n    try:\n        while (job := jobs.get()) is not None:\n            start, appearance, count = job\n            try:\n                for offset, row in enumerate(appearance):\n                    env = start + offset\n                    scene.apply_appearance(model, index, row)\n                    frames[env, :count] = renderer.render(model, data, qpos[env, :count])\n                done.put((\"ok\", start))\n",11668,"\n    def velocities(self):\n        return np.stack([self.data.qvel[d:d + 3] for d in self.index.object_dof])\n\n    def tip(self):\n        return self.data.site_xpos[self.index.fingertip][:2].copy()\n\n    def success(self) -> bool:\n        return bool(privileged.success(np, self.comp, self.episode.goal, self.positions(), self.quats(),\n                                       self.velocities(), self.start_xy))\n\n    def privileged_reward(self) -> float:\n        return float(privileged.reward(np, self.comp, self.episode.goal, self.tip(), self.positions(),\n                                       self.quats(), self.start_xy))\n\n    def frame(self) -> np.ndarray:\n        if self.renderer is None:\n            raise RuntimeError(\"this scene was created without a renderer\")\n        # After mj_step the body poses lag qpos by one substep; render the current state, as the learner does.\n        self.mujoco.mj_kinematics(self.model, self.data)\n        self.mujoco.mj_camlight(self.model, self.data)\n        self.renderer.update_scene(self.data, camera=\"view\")\n        return self.renderer.render().copy()\n\n    def close(self):\n        if self.renderer is not None:\n            self.renderer.close()"],"d":0},{"n":"bash","a":"cd /workspace/repo && cat robotreward/textures.py robotreward/panda.py | head -300","e":0,"o":["\"\"\"Procedural surface textures: primitive layers, the public texture families, and PNG encoding.\n\nA family is a list of styles; a style names one primitive layer and the range of each of its parameters.\nSynthesis draws every parameter uniformly from its range, so one style yields endless instances. Textures are\ngrayscale intensity in [1 - contrast, 1]; the renderer tints them with the surface's colour.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport struct\nimport zlib\n\nimport numpy as np\n\nSIZE = 128\nLAYERS = (\"stripes\", \"dots\", \"checks\", \"noise\", \"rings\", \"marble\", \"weave\")\n\nPUBLIC_FAMILIES = {\n    \"striped\": [\n        {\"layer\": \"stripes\", \"period\": [34, 50], \"angle\": [0, 180], \"duty\": [0.4, 0.6], \"wave\": [0, 0],\n         \"contrast\": [0.55, 0.8]},\n    ],\n    \"dotted\": [\n        {\"layer\": \"dots\", \"spacing\": [36, 52], \"radius\": [0.22, 0.34], \"jitter\": [0, 0.1], \"contrast\": [0.55, 0.8]},\n    ],\n    \"checkered\": [\n        {\"layer\": \"checks\", \"cell\": [30, 44], \"angle\": [0, 45], \"contrast\": [0.55, 0.8]},\n    ],\n    \"table\": [\n        {\"layer\": \"noise\", \"scale\": [10, 40], \"octaves\": [2, 4], \"contrast\": [0.1, 0.3]},\n        {\"layer\": \"checks\", \"cell\": [24, 48], \"angle\": [0, 0], \"contrast\": [0.06, 0.16]},\n        {\"layer\": \"stripes\", \"period\": [20, 40], \"angle\": [0, 0], \"duty\": [0.85, 0.95], \"wave\": [0, 0],\n         \"contrast\": [0.1, 0.25]},\n    ],\n    \"background\": [\n        {\"layer\": \"noise\", \"scale\": [6, 24], \"octaves\": [1, 3], \"contrast\": [0.1, 0.35]},\n        {\"layer\": \"stripes\", \"period\": [30, 60], \"angle\": [80, 100], \"duty\": [0.5, 0.5], \"wave\": [0, 0],\n         \"contrast\": [0.05, 0.15]},\n    ],\n}\n\n\ndef pick(style: dict, name: str, rng: np.random.Generator) -> float:\n    low, high = style[name]\n    return float(rng.uniform(low, high)) if high > low else float(low)\n\n\ndef _grid(size):\n    y, x = np.mgrid[0:size, 0:size].astype(np.float64)\n    return x, y\n\n\ndef _rotate(x, y, degrees):\n    a = np.deg2rad(degrees)\n    return x * np.cos(a) + y * np.sin(a), -x * np.sin(a) + y * np.cos(a)\n\n\ndef _value_noise(size, scale, octaves, rng):\n    \"\"\"Tileable multi-octave value noise in [0, 1].\"\"\"\n    out = np.zeros((size, size))\n    total = 0.0\n    for octave in range(max(1, int(round(octaves)))):\n        cells = max(2, int(round(size / max(2.0, scale / 2**octave))))\n        lattice = rng.random((cells, cells))\n        coords = np.arange(size) * cells / size\n        i0 = np.floor(coords).astype(int) % cells\n        i1 = (i0 + 1) % cells\n        t = coords - np.floor(coords)\n",10282,"                      <geom class=\"visual\" type=\"box\" size=\"0.011 0.012 0.026\" pos=\"0 -0.013 0.086\" rgba=\"{JOINT_RGBA}\"/>\n                      <geom name=\"pusher\" class=\"pusher\" type=\"box\" size=\"0.012 0.024 0.0275\" pos=\"0 0 0.0875\"/>\n                      <site name=\"fingertip\" pos=\"0 0 {FINGERTIP}\" size=\"0.005\"/>\n                    </body>\n                  </body>\n                </body>\n              </body>\n            </body>\n          </body>\n        </body>\n      </body>\n    </body>\n    <body name=\"target\" mocap=\"true\" pos=\"{START[0]} {START[1]} {HAND_HEIGHT}\" quat=\"{' '.join(map(str, HAND_QUAT))}\"/>\n\"\"\"\n    equality = '<weld name=\"grip\" body1=\"target\" body2=\"hand\" relpose=\"0 0 0 1 0 0 0\" solref=\"0.02 1\"/>'\n    defaults = \"\"\"\n    <default class=\"arm\">\n      <joint damping=\"12\" armature=\"0.1\" stiffness=\"4\"/>\n    </default>\n    <default class=\"visual\">\n      <geom contype=\"0\" conaffinity=\"0\" group=\"1\" mass=\"0\"/>\n    </default>\n    <default class=\"pusher\">\n      <geom contype=\"2\" conaffinity=\"2\" group=\"0\" friction=\"0.6 0.005 0.0001\" rgba=\"0.2 0.2 0.22 1\"/>\n    </default>\n\"\"\"\n    return body, equality, defaults"],"d":0}]},{"t":31.2,"r":"I should check reward.py, check.py, demos.py, and components.py along with the GPU guide manuals, writing outputs to files and reading them in parts to avoid truncation issues.","$":0.2174,"c":[{"n":"bash","a":"cd /workspace/repo && sed -n 1,250p robotreward/reward.py","e":0,"o":["\"\"\"The frozen reward: the ONNX graph's checks, and the contained worker process that evaluates it.\n\nA reward is one ONNX inference graph. Its operators are its code and its initializers its weights. It receives\nonly what the interface below names and returns one number per query. No submitted Python ever runs: the graph\nis data, executed by ONNX Runtime in a separate worker process that sees nothing but the queries it is sent.\n\n    inputs:  frames       uint8 [batch, 3, 84, 84, 3]  initial, previous query, current frame, RGB\n             instruction  int64 [batch, 16]            word ids from robotreward.grammar.VOCABULARY, 0-padded\n    output:  reward       float32 [batch]\n\"\"\"\n\nfrom __future__ import annotations\n\nimport json\nimport mmap\nimport os\nimport re\nimport select\nimport struct\nimport subprocess\nimport sys\nimport threading\nimport time\nfrom pathlib import Path\n\nimport numpy as np\n\nfrom .grammar import TOKENS\nfrom .protocol import FRAME, HISTORY, MAX_BATCH, MAX_GRAPH_BYTES, OPSETS, REWARD_CLIP, REWARD_LIMITS, CandidateError\n\nFORBIDDEN = {\n    \"Loop\", \"Scan\", \"If\", \"SequenceMap\", \"Dropout\", \"Multinomial\", \"Bernoulli\",\n    \"RandomNormal\", \"RandomUniform\", \"RandomNormalLike\", \"RandomUniformLike\",\n}\n# Worker failures that name CUDA, the GPU's driver or its management library: host faults, never the reward's.\nHOST_FAULTS = re.compile(r\"CUDA execution provider|CUDA driver|CUDA failure|CUDA error|cudaError|driver/context|\"\n                         r\"NVML|\\bXid\\b\", re.I)\nALLOCATION_TERMS = (\"out of memory\", \"failed to allocate memory\", \"requested allocation exceeds\")\nHANG_SECONDS = 10.0     # a call's wait that reads as a hang: a graph at the limit waits about 0.1 s a call\nBREACH_READINGS = 5     # consecutive watchdog readings over the host memory limit, 0.1 s apart, before it acts\nUNREAD_READINGS = 20    # consecutive readings of a live worker that found no memory figure, before the run stops\nMEMORY_UNREAD = (\"this host reports neither RssAnon nor VmRSS for the reward worker, so its host memory limit \"\n                 \"can't be enforced\")\nFRAMES_BYTES = MAX_BATCH * HISTORY * FRAME * FRAME * 3\nTOKENS_BYTES = MAX_BATCH * TOKENS * 8\nOUTPUT_BYTES = MAX_BATCH * 4\nSHARED_BYTES = FRAMES_BYTES + TOKENS_BYTES + OUTPUT_BYTES\n\n\ndef _dims(value_info):\n    shape = value_info.type.tensor_type.shape\n    return [d.dim_value if d.HasField(\"dim_value\") else None for d in shape.dim]\n\n\ndef check_graph(raw: bytes) -> dict:\n",8367,"    try:\n        identity = check_graph(raw)\n    except CandidateError as exc:\n        _send(stdout, b\"E\", {\"kind\": \"candidate\", \"detail\": str(exc)})\n        return\n    import onnxruntime as ort\n\n    options = ort.SessionOptions()\n    options.intra_op_num_threads = config.get(\"threads\", 2)\n    options.inter_op_num_threads = 1\n    options.log_severity_level = 4\n    providers = [\"CPUExecutionProvider\"]\n    if config[\"device\"] == \"cuda\":\n        if hasattr(ort, \"preload_dlls\"):\n            ort.preload_dlls()\n        cuda = {\"device_id\": 0, \"gpu_mem_limit\": int(config[\"gpu_memory_bytes\"]),\n                \"arena_extend_strategy\": \"kSameAsRequested\", \"cudnn_conv_algo_search\": \"HEURISTIC\"}\n        providers = [(\"CUDAExecutionProvider\", cuda), \"CPUExecutionProvider\"]\n    try:\n        session = ort.InferenceSession(raw, options, providers=providers)\n    except Exception as exc:\n        _send(stdout, b\"E\", ort_failure(exc, raw))\n        return\n    if config[\"device\"] == \"cuda\" and session.get_providers()[0] != \"CUDAExecutionProvider\":\n        _send(stdout, b\"E\", {\"kind\": \"runtime\", \"detail\": \"the CUDA execution provider is unavailable\"})"],"d":0.4},{"n":"bash","a":"cd /workspace/repo && sed -n 250,600p robotreward/reward.py","e":0,"o":["        _send(stdout, b\"E\", {\"kind\": \"runtime\", \"detail\": \"the CUDA execution provider is unavailable\"})\n        return\n    with open(shared_path, \"r+b\") as handle:\n        shared = mmap.mmap(handle.fileno(), SHARED_BYTES)\n    frames = np.ndarray((MAX_BATCH, HISTORY, FRAME, FRAME, 3), np.uint8, shared, 0)\n    tokens = np.ndarray((MAX_BATCH, TOKENS), np.int64, shared, FRAMES_BYTES)\n    output = np.ndarray((MAX_BATCH,), np.float32, shared, FRAMES_BYTES + TOKENS_BYTES)\n    _send(stdout, b\"R\", identity)\n    while True:\n        header = stdin.read(5)\n        if len(header) < 5:\n            return\n        count = struct.unpack(\"<I\", header[1:])[0]\n        try:\n            result = session.run([\"reward\"], {\"frames\": frames[:count], \"instruction\": tokens[:count]})[0]\n            result = np.asarray(result)\n            if result.shape != (count,) or result.dtype != np.float32:\n                raise CandidateError(f\"reward has shape {result.shape} and type {result.dtype}, not float32 [batch]\")\n        except Exception as exc:\n            _send(stdout, b\"E\", ort_failure(exc, raw))\n            return\n        output[:count] = result\n        _send(stdout, b\"A\")\n\n\n# --- the trusted client -------------------------------------------------------------------------\n\nclass RewardTimeError(CandidateError):\n    \"\"\"The reward's metered wall time ran out while one call had waited more than HANG_SECONDS, as under a GPU that hangs.\n    The final run leaves the first such overrun on a learner run unscored and retries the phase, as for\n    RewardRuntimeError (`robotreward_task.phases`). It is the reward's at once in the check, a CPU phase.\"\"\"\n\n\nclass RewardRuntimeError(RuntimeError):\n    \"\"\"The worker evaluating the reward failed without a classified outcome: it exited, or ONNX Runtime raised an\n    unclassified error, and nothing names the GPU or its driver. The final run leaves the first such failure unscored and\n    retries the phase; the same failure on the same run after that retry is the reward's (`robotreward_task.phases`).\n    \"\"\"\n\n\ndef _overrun(waited: float) -> CandidateError:\n    \"\"\"The allowance ran out. A graph that is too slow spends it evenly, so its last call waits about one call's time, and\n    that is the reward's at once: a retried run has the GPU to itself, where a limit set for a shared GPU wouldn't bind.\n    A GPU that hangs under a call waits out all that's left.\"\"\"\n    text = \"the reward exceeded its time allowance\"\n",12396,"                raise RewardRuntimeError(\"the reward worker exited without a classified outcome\") from exc\n            kind, payload = self._reply(began)\n            if kind != b\"A\":\n                self._fail(payload)\n            values = self.output[:size].copy()\n            if not np.isfinite(values).all():\n                self.close()\n                raise CandidateError(\"the reward returned a non-finite value\")\n            out[start:end] = np.clip(values, *REWARD_CLIP)\n            self.calls += 1\n            self.queries += size\n        return out\n\n    def close(self, kill: bool = False):\n        process, self.process = self.process, None\n        if process is None:\n            return\n        try:\n            process.stdin.close()\n        except OSError:\n            pass\n        if kill:\n            process.kill()\n        try:\n            process.wait(timeout=5)\n        except subprocess.TimeoutExpired:\n            process.kill()\n            process.wait(timeout=5)\n\n\nif __name__ == \"__main__\":\n    if len(sys.argv) == 3 and sys.argv[1] == \"--worker\":\n        worker(sys.argv[2])\n    else:\n        raise SystemExit(\"usage: python -m robotreward.reward --worker SHARED_FILE\")"],"d":0}]},{"t":33.9,"$":0.1037,"c":[{"n":"bash","a":"cd /workspace/repo && sed -n 1,200p robotreward/check.py > /tmp/check_head.txt; wc -l robotreward/check.py; sed -n 1,140p robotreward/check.py","e":0,"o":["328 robotreward/check.py\n\"\"\"Check a submission the way the final run does before any learner runs.\n\n    python -m robotreward.check submission\n    python -m robotreward.check submission --benchmark     # on a GPU: time the reward against the margin line\n    python -m robotreward.check submission --benchmark --json-out bench.json\n    python -m robotreward.check submission --benchmark --require-gpu-renderer\n\nThe check validates the package and the graph, then runs the graph on a fixed probe batch of eight queries in\nthe same contained worker the final run uses, on two CPU threads, within 210 seconds for loading and answering. A submission\nthat fails here is rejected before any learner starts.\n\n`--benchmark` runs two complete public learner lanes on one GPU, including MJX, PPO, rendering and reward queries,\non two four-object compositions: the final run pairs two runs of the same size on each GPU, and four objects is its\nheavier pair. It projects the reward's time for a full run against the margin line of\n`REWARD_LIMITS[\"seconds_per_iteration\"]`: startup and the first iteration once, as the final run meters them, then every\nlater iteration at the slowest measured rate. The final run allows 2.5 seconds per iteration, so a\nreward within the line here keeps slack on a slower host. The line is a margin, not a rule (`LINE_NOTE`): the command\nexits 0 whether the projection is within the line or over it, and says which in `benchmark.result`, in\n`benchmark.verdict` and, repeating the verdict, in its last line on stderr. The result\nnames the OpenGL renderer the lanes drew with; the final run renders on the GPU, and a benchmark that rendered in software\nis flagged and, being a different workload, exits with status 3 after printing its result. With\n`--require-gpu-renderer` such a benchmark stops before its first iteration instead, with the same status and no result.\nAny other non-zero exit is an error: the hand-in check refused the submission, or a lane failed.\n`--reward-only-benchmark` times two reward workers calling at the same moment with nothing else running: an\ninference-only diagnostic, which has read above what the final run meters. It holds no line and gives no verdict.\n\nThe result is the only thing printed on stdout, as JSON; everything else the work prints goes to stderr.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport argparse\nimport contextlib\nimport json\nimport os\nimport sys\nimport tempfile\nimport threading\nimport time\n",3899,"                    started.wait()\n                    began, clock = worker.seconds, time.monotonic()\n                    for _ in range(iterations):\n                        for call in calls:\n                            worker(*call)\n                    seconds, wall = worker.seconds - began, time.monotonic() - clock\n                    with lock:\n                        timed[0] += 1\n                        if timed[0] == lanes:\n                            everyone.set()\n                    extra = 0\n                    while not everyone.is_set():           # keep sharing the GPU until every lane is timed\n                        worker(*calls[extra % len(calls)])\n                        extra += 1\n                finally:\n                    worker.close()\n        except BaseException:\n            started.abort()                                # a lane that fails stops the others from waiting on it\n            everyone.set()\n            raise\n        return {\"load_seconds\": load, \"seconds_per_iteration\": seconds / iterations,\n                \"seconds_per_call\": seconds / (iterations * len(calls)), \"wall_per_iteration\": wall / iterations,"],"d":0.2}]},{"t":36.3,"$":0.042,"c":[{"n":"bash","a":"cd /workspace/repo && sed -n 140,328p robotreward/check.py","e":0,"o":["                \"seconds_per_call\": seconds / (iterations * len(calls)), \"wall_per_iteration\": wall / iterations,\n                \"calls_after_timing\": extra}\n\n    with ThreadPoolExecutor(max_workers=lanes) as pool:\n        futures = [pool.submit(lane, index) for index in range(lanes)]\n    errors = [future.exception() for future in futures if future.exception() is not None]\n    if errors:   # the lane that failed, not the ones it stopped from waiting on it\n        raise next((e for e in errors if not isinstance(e, threading.BrokenBarrierError)), errors[0])\n    return [future.result() for future in futures]\n\n\ndef reward_only_benchmark(root, *, iterations: int = BENCHMARK_ITERATIONS, lanes: int = GPU_SHARING, batch: int = LEARNER[\"envs\"],\n              worker_factory=RewardWorker, device: str = \"cuda\") -> dict:\n    \"\"\"The reward's metered time per learner iteration with `lanes` reward workers making the learner's calls at the\n    same moment on this machine's GPU and nothing else running; and one worker alone, for reference. An inference-only\n    diagnostic: it omits the learner, simulator and renderer and has read above what the final run meters, so it is compared\n    with no line. `benchmark` is what the margin line applies to.\"\"\"\n    raw = (Path(root) / \"reward.onnx\").read_bytes()\n    calls = learner_calls(min(batch, MAX_BATCH))\n    alone = _lanes(raw, 1, iterations, calls, worker_factory, device)[0]\n    shared = _lanes(raw, lanes, iterations, calls, worker_factory, device)\n    per_iteration = max(row[\"seconds_per_iteration\"] for row in shared)\n    load = max(row[\"load_seconds\"] for row in shared)\n    runs = {name: {\"iterations\": count, \"projected_seconds\": load + count * per_iteration}\n            for name, count in ((\"standard\", LEARNER[\"iterations\"]), (\"probe\", LEARNER[\"probe_iterations\"]))}\n    return {\"condition\": f\"{lanes} reward workers making the learner's calls at the same moment on one GPU with \"\n                         \"nothing else running: an inference-only diagnostic, not the margin line\",\n            \"seconds_per_iteration\": per_iteration,\n            \"load_seconds\": load, \"runs\": runs, \"lanes\": shared,\n            \"alone\": {\"seconds_per_iteration\": alone[\"seconds_per_iteration\"], \"load_seconds\": alone[\"load_seconds\"],\n                      \"seconds_per_call\": alone[\"seconds_per_call\"]},\n            \"contention\": per_iteration / alone[\"seconds_per_iteration\"] if alone[\"seconds_per_iteration\"] else None,\n",7908,"            result = check(args.submission)\n            if args.benchmark:\n                result[\"benchmark\"] = benchmark(args.submission, require_gpu_renderer=args.require_gpu_renderer)\n            if args.reward_only_benchmark:\n                result[\"reward_only_benchmark\"] = reward_only_benchmark(args.submission)\n    except SoftwareRendererError as exc:\n        print(f\"robotreward.check: benchmark stopped before the first iteration: {exc}\", file=sys.stderr, flush=True)\n        return SOFTWARE_RENDERER_EXIT\n    text = json.dumps(result, indent=2)\n    if args.json_out:\n        args.json_out.write_text(text + \"\\n\")\n    print(text, flush=True)\n    if not args.benchmark:\n        return 0\n    # The verdict comes last, after the full record. Over the line is a reading and exits 0: a non-zero exit would make a\n    # job read as failed. A software renderer is a different workload, never a successful GPU timing.\n    print(f\"robotreward.check: benchmark {result['benchmark']['verdict']}\", file=sys.stderr, flush=True)\n    return SOFTWARE_RENDERER_EXIT if result[\"benchmark\"][\"software_renderer\"] else 0\n\n\nif __name__ == \"__main__\":\n    raise SystemExit(main())"],"d":0.2}]},{"t":42.2,"r":"I should also dig into demos.py, components.py, the top of the learner covering FrozenReward, draws, and make_rollout, plus a fuller read of mjx_env and the GPU manuals.","$":0.055,"c":[{"n":"bash","a":"cd /workspace/repo && sed -n 1,140p robotreward/learner.py | sed -n 20,140p","e":0,"o":["\nfrom __future__ import annotations\n\nimport argparse\nimport json\nimport os\nimport sys\nimport tempfile\nimport time\nfrom pathlib import Path\n\nimport numpy as np\n\nfrom . import scene\nfrom .protocol import CONTROL, LEARNER, QUERIES, REWARD_CLIP, reward_allowance\n\n# The final run has two learners on one GPU from threads of one process, and XLA's CUDA graphs (\"command buffers\")\n# can then fail to capture (CUDA_ERROR_STREAM_CAPTURE_INVALIDATED). The learner runs without them; the kernels and\n# results are the same. Read when JAX starts its backend, so it must be set before the first computation.\nif \"--xla_gpu_enable_command_buffer\" not in os.environ.get(\"XLA_FLAGS\", \"\"):\n    os.environ[\"XLA_FLAGS\"] = (os.environ.get(\"XLA_FLAGS\", \"\") + \" --xla_gpu_enable_command_buffer=\").strip()\n\nEVERY = CONTROL[\"query_every\"]\nSOFTWARE_RENDERER_EXIT = 3    # the commands' exit status for a run that rendered, or would have rendered, in software\n\n\nclass SoftwareRendererError(RuntimeError):\n    \"\"\"The render workers rasterize in software and the run was told to need the GPU's OpenGL driver: the host's\n    fault, never the reward's.\"\"\"\n\n\nclass PrivilegedReward:\n    \"\"\"The privileged state reward (robotreward.privileged): a comparison from simulator state, never a reward's input.\"\"\"\n    visual = False\n    seconds = 0.0\n    queries = 0\n\n    def close(self):\n        pass\n\n\nclass FrozenReward:\n    \"\"\"A submitted reward graph, evaluated in its metered worker process under the final run's time allowance.\"\"\"\n    visual = True\n\n    def __init__(self, graph: bytes, *, iterations: int, device: str = \"cuda\", workdir=None, launcher=None):\n        from .reward import RewardWorker\n\n        self._temporary = None if workdir is not None else tempfile.TemporaryDirectory(prefix=\"robot-reward-\")\n        self.worker = RewardWorker(graph, allowance=reward_allowance(iterations, graded=True), device=device,\n                                   workdir=workdir or self._temporary.name, launcher=launcher)\n\n    @property\n    def seconds(self):\n        return self.worker.seconds\n\n    @property\n    def queries(self):\n        return self.worker.queries\n\n    @property\n    def host_bytes(self):\n        return self.worker.peak_host_bytes\n\n    def __call__(self, initial, previous, current, tokens):\n        return self.worker(initial, previous, current, tokens)\n\n    def close(self):\n        self.worker.close()\n        if self._temporary is not None:\n            self._temporary.cleanup()\n\n\n",908,"            return state, out\n\n        state, out = jax.lax.scan(body, state, jax.random.split(key, CONTROL[\"episode_steps\"]))\n        out[\"success\"] = env.success(state)\n        out[\"placements\"] = env.placements(state)\n        return out\n\n    return jax.jit(rollout)\n\n\ndef train(comp: scene.Composition, *, seed: int, iterations: int, reward, evaluate_at=None, envs=None,\n          workers: int = 4, env_factory=None, pool_factory=None, log=None, eval_episodes=None,\n          require_gpu_renderer: bool = False, on_renderer=None) -> dict:\n    \"\"\"One learner run. Raises CandidateError when the reward breaks its contract.\n\n    With `require_gpu_renderer`, render workers that rasterize in software are a fault (SoftwareRendererError, a\n    RuntimeError) before the first iteration. `on_renderer` is called once with the render workers' OpenGL identity.\n    \"\"\"\n    import jax\n    import jax.numpy as jnp\n\n    from . import ppo\n\n    if env_factory is None:\n        from .mjx_env import BatchEnv as env_factory\n    if pool_factory is None:\n        from .render import RenderPool as pool_factory\n    envs = envs or LEARNER[\"envs\"]"],"d":0.2}]}],"parts":[179,231]}