07Model taskGoal-conditioned visual navigation for legged robots

New Rooms

Build a controller that walks a simulated quadruped through an unseen building to where a goal photo was taken, from its own camera alone.

Every model’s run on New Rooms, ranked by score
#ModelScoreLab spendInferenceGPU hoursTurnsFinal pick
1AstraAstra0.530$70$28116.848911.1 h
2Fable 5.2Fable0.456$59$10114.949710.2 h
3Opus 5.5Opus0.426$37$519.74089.7 h
4CarbonCarbon0.331$31$937.932711.6 h
5Grok 4.7Grok0.000$5$1421.21,07311.9 h

One run per model, from the evaluation of 14 November 2026. Lab spend is what a run paid from its $85 wallet for compute and model calls; inference is the agent’s own model calls at list prices; the final pick is the hour at which it chose what was graded. Every task’s anchors, scale and noise are in the paper.

What the runs showed

Astra had the best run, 0.530, ahead of Fable 5.2 at 0.456 and Opus 5.5 at 0.426.

Each agent tested on journeys of its own making, and three estimates landed within 0.08 of the grade. Carbon expected 0.71 and scored 0.33; Grok 4.7’s estimate of 0.79 came from a single journey, and its final scored 0.

Background

Goal-conditioned visual navigation asks a robot to reach the place shown in a picture. Here a four-legged robot, OGBench’s ant in MuJoCo, is placed in a building it has never been in and shown one photo taken somewhere inside. It has no map, no pose estimate and no compass: only egocentric pixels, its proprioception and that goal image.

Controllers that look strong on the layouts they were trained on often struggle badly on new ones. The task asks for a controller that does well in buildings unlike any it trained in, which means walking, exploring and recognizing the goal’s surroundings at once.

Task

The agent hands in a controller, Controller(root) with a fast callback on every tick and an optional slow one for planning.

  • Observation. A 64 × 64 camera image, 26 proprioceptive numbers and the goal image, identical on every tick of a journey.
  • Action. Torques for the ant’s eight actuators, returned within 20 ms of every 50 ms tick, in real time.
  • Package. At most 2 GB, read-only in the final run.

What the agent gets

The generator, renderer and simulator for the training families, exactly as they are run, and the real-time loop with local tools to run it.

  • Data. 3.8M ticks of recorded walking.
  • Motor. An optional walking motor, with its pace along a known route against the deadline.
  • Starter. A behavior-cloned controller, trained from scratch by train.py.
  • Compute. Simulation is CPU-bound, including inside GPU jobs.

How it is graded

After the research window the controller is sent on 48 journeys in 24 buildings, every layout in a style the training families do not contain.

  • Appearance and physics. A third of the buildings change only the layout, a third also have unseen textures and lighting, and a third have unseen textures and lighting plus friction, mass or motor strength outside the training ranges.
  • Routes. Short, medium and long routes count equally.
  • Measure. Arrival carries three quarters of the credit and speed up to a quarter, on one H100 and two CPU cores; the two-core limit is enforced by measuring the controller’s CPU use.

Environment

The lab on New Rooms
Time limit12 hours
Wallet$85
Compute16 CPUs · 32 GiB RAM · 64 GiB disk
GPUsUp to 4 H100s at once, $3.78 per GPU-hour
ModelsClaude Sonnet 5.5, Claude Haiku 5.5, GPT-6 Luna, Gemini 3.8 Flash, Qwen3.8 Max Prime, Qwen3.8 Flash, GLM-5.3 Prime
NetworkNone. Frozen arXiv to 25 January 2026, and a mirror of Hugging Face datasets
Codetasks/new-rooms on GitHub

References

The brief

You'll have 12 hours for this. Your workspace has 16 CPUs, 32 GiB of RAM, 64 GiB of disk and no GPU; GPU jobs (up to four H100s at once, from a pool shared with other work, so they can queue) and model calls come out of a $85 wallet, and /lab/MANUAL.md lists prices, limits and each job's CPUs. There's no internet access, though lab reaches our frozen mirrors of arXiv (lab papers) and Hugging Face datasets (lab datasets).

We're interested in goal-conditioned visual navigation for legged robots. A four-legged robot, simulated as OGBench's ant in MuJoCo, is placed in a building it has never been in. Someone shows it a photo taken somewhere in that building, and it has to walk there. It has no map, no pose estimate and no compass: only egocentric pixels, its proprioception and that one goal image.

In our experience, controllers that look strong on the layouts they were trained on often struggle badly on new ones. We'd like a controller that does well in genuinely unfamiliar buildings.

We'll send it on 48 journeys in 24 buildings, every layout in a style the training families don't contain. Two thirds of the buildings also have unseen textures and lighting, and in a third, friction, mass and motor strength come from wider ranges. Short, medium and long routes count equally. It runs in real time, returning an action within 20 ms of every 50 ms tick, on one H100 and two CPU cores. Arriving matters most; speed counts for up to a quarter. The starter controller is where we're starting from.

The repo has the generator, renderer and simulator for the training families, exactly as we run them. It also has 3.8M ticks of recorded walking, an optional walking motor and a behaviour-cloned starter, and its README covers the interface and the limits. Simulation is CPU-bound, including inside GPU jobs.

The controller goes in submission/. Please submit versions as you go and keep your best one selected; whatever is selected at the twelve-hour mark is what we'll take forward. We won't know how it does until after you've finished.