Demos

The players

The hosted models' latencies include the network from the Mac; OpenRouter's own added time was measured at 130 to 155 ms per call.

Browser agent (10 runs of each task)

On the 272 labelled steps (the operation question alone), the answer is inside the acceptable set on 243 (89%) for the Qwen base, 269 (99%) for Gemma 4 12B, 248 (91%) for Gemma 4 31B and 268 (99%) for Cygnet.

Driving

Lockstep waits for every decision, so it measures the decisions alone; realtime keeps the car moving while a request is in flight, as the demo does, so a slow answer arrives too late. Gemini and Luna lose drives in realtime to latency, not to judgement: about a second per decision, and the demo's 1.5 s timeout hands most of Luna's decisions to the rules driver. Over all of their drives, in lockstep and in realtime:

• the Qwen base passed 16 of 18 and 9 of 9 (in lockstep one collision in s2-4 and one drive off the road in s2-5; the rules driver alone passes both);
• Gemma 4 12B passed 18 of 18 and 9 of 9;
• Gemma 4 31B passed 18 of 18 and 8 of 9 (in realtime a collision in s2-2);
• Cygnet passed 6 of 6 and 9 of 9.

Pong (5 games of up to 45 s, seeds 20260917 to 20260921)

A game ends when the model's paddle misses five times; the ball advances one step per decision, so returns per game reward fast answers as much as right ones. Gemma 4 31B played all five games to the 45 s limit, which is why it was asked three times as often as the other bases.

Triage (400 held-out tickets over 20 queues)

Registration was repeated on each base and took 33 s, 35 s and 69 s. On the Qwen base it kept both the intent head and the calibration; on the Gemma bases it kept the head only (each is kept only when cross-validation on the examples shows a gain). The head costs latency: a registered answer reads the model's hidden state as well as its letters. The plain question is already right on 91% to 97% of these synthetic tickets, so the lift is small, and smallest where the plain answer is best.