Selected for NVIDIA's Codefest — with a zero we intend to change.

We applied on 15 July. On 7 August the confirmation came: we are one of the fifteen teams selected for NVIDIA’s Open Models Codefest. Here is what we want out of it — the question we had to correct in our own application, and the new release we checked before it could become a distraction.

The programme. The opening workshop ran on 30 July; the hackathon itself runs 9 September to 7 October, on the open model families — Nemotron, NeMo, NIM, TensorRT — with GPU access and NVIDIA engineers alongside. The top five teams present at GTC Berlin, 20–22 October. That is the bar we are setting ourselves, said out loud so that it can be seen to be missed.

One condition, stated openly. Our product is proprietary, and mentors can only work directly on permissively-licensed code. So the piece where their time has the most leverage goes into the open: an Apache-2.0 harness for closed-loop rollout and on-device benchmarking of vision-language navigation policies, runnable end to end on open weights before 9 September.

Where we actually are. A bolt-on brain for uncrewed ground and aerial machines — speech in, an open 4-billion-parameter vision-language model deciding and calling tools, speech out, motor control over serial, the whole loop on a Jetson Orin Nano 8 GB, offline. Separately, in R&D, we fine-tune that same model into a local navigation planner, gated on collision rate, not trajectory error. That adapter is not deployed: everything below is measured in simulation, and we say so plainly because the difference matters.

The wall we hit. The base model collides on roughly 99 of 100 mazes. Fine-tuned, it stopped colliding entirely at the scale it was trained on — the headline number, and it was wrong. A held-out evaluation scored the same adapter at 49% collisions: it had memorised a single constant output and was emitting it whatever the geometry in front of it. We fixed the cause and retrained. The memorisation is gone; precision regressed slightly.

Then the first closed-loop rollouts — drive the plan, re-render what the robot now sees, re-plan from there. It reached the goal zero times out of four. The failure has a shape: heading errors compound across re-plans, the robot leaves the centreline, and close to a wall its camera returns a featureless surface. It is then asked to plan from an image with no usable scene in it, and it drives straight ahead. Textbook covariate shift — and open-loop metrics are structurally blind to it, because they always hand the model a clean, on-path frame.

The correction. Our application asked for faster training. True, and not the binding constraint: a fine-tune costs under two hours on one consumer card, so a tenfold speedup buys back ninety minutes an experiment. The expensive thing is closed-loop rollouts, and their cost is inference throughput, not gradient throughput.

The world model we ruled out. NVIDIA released Cosmos 3 Edge on 20 July, and forward dynamics — predicting the observation that follows an action — is functionally what a rollout needs. We read the model card and ruled it out. Its action conditioning is defined for a specific set of embodiments — camera motion, autonomous vehicles, egocentric motion, a handful of manipulator arms — and a differential-drive ground robot is not among them, so it could not drive our rollouts in the first place. And decisively, the card states that because the model “lacks an explicit physics simulator”, contact dynamics and physical laws “are only approximated”, and that outputs “should not be treated as physically accurate simulation”. Our entire gate is collision rate, and a collision is contact dynamics — we would be grading a policy against an approximation of the very quantity we measure. It stays interesting for inverse dynamics and for the appearance half of sim2real. Not for this.

Three things we will measure.

• Rollout throughput — whether more rollouts an hour changes the scale of experiment we can run, or only the wall-clock of the one we already run.
• What fits in 8 GB — Nemotron 3 Nano 4B is listed as running on Orin Nano 8 GB under llama.cpp, already our runtime, so it is a genuine drop-in comparison against the model we ship. Judged on collision rate against our own held-out evaluation, not on a vision-language leaderboard.
• Whether a world model earns a place anywhere in the pipeline — on inverse dynamics and appearance, not as a substitute for exact simulation.

Why we publish this. The 49% and the zero-out-of-four are our own numbers, found by disagreeing with our own headline result. We would rather be the team that publishes the evaluation contradicting its first announcement than the team whose first number was never re-checked. Between 9 September and 7 October we will run the rollouts and publish what they show — whether the cell reads 0 or 91.

If you’re building autonomy for the places the network can’t reach, talk to us.