VCIS-Bench: Claude Sonnet 5.5 beats Gemini 3.7 Flash and Gemma 4 on spatial grid puzzles
A developer released VCIS-Bench, a synthetic 7×7 grid benchmark where an agent must grab a key and unlock a door without hitting walls. Claude Sonnet 5.5 scored 100% on all three tasks, Gemini 3.7 Flash 0.97, and open-weights Gemma 4 26B A4B 0.85.
- Claude Sonnet 5.5: 100% on plan, coordinate and step-error checks
- Gemini 3.7 Flash: 100% on the first two tasks, 92% on step diagnosis
- Gemma 4 26B A4B: 96.7% on spatial grounding but only 76% on step-level analysis
- Gemini 3.7 Flash hit 0.97 at roughly half the cost and latency of Claude
Read next
AI