SenseNova: robot benchmarks ignore inference latency

SenseNova evaluated GPT-6-Astra on 12 robotics benchmarks (arXiv:2610.22471) and deployed it on eight physical robot setups with low-level controllers such as π0.5. The report's key finding: simulated benchmarks pause time while the planner thinks, whereas a three-second delay in the real world means actions execute against stale coordinates. The authors propose evaluating agents against an unpaused world clock with zero in-domain demonstrations.

• The model was tested on 12 benchmarks and 8 physical robots
• Simulated benchmarks pause time while the planner thinks
• A 3-second delay leads to actions based on stale scene state
• Proposed benchmarks use a continuous clock and no demonstrations