EdgeWise selective cloud delegation cuts inference costs by 87%

Researchers introduced EdgeWise, a system where a lightweight selector on a Jetson Orin decides which steps of a long-horizon AI agent go to the cloud. According to the paper (arXiv, October 13, 2026), only 12% of decisions in a 50-step task need GPT-4-turbo, yielding a 5–7× inference cost reduction with 95th-percentile latency under 200 ms.

• 12M-parameter selector runs in under 5 ms on Jetson Orin
• Only 10–20% of steps are delegated to the cloud while keeping 95% quality
• Inference cost drops 5–7×, edge-step latency is 190 ms
• SDK 0.3.1 is pip-installable with ready ARM64 Docker images