A $7 ESP32 and a local vision model control an iPhone without Xcode

Developer Henry Yuan built esp32-iphone-agent, which drives a stock iPhone using an ESP32-S3 posing as a USB mouse and keyboard plus any OpenAI-compatible vision model. The screen is captured over AirPlay via UxPlay and GStreamer, and the MIT-licensed repo has reached 2,975 GitHub stars.

• A ~$7 ESP32-S3 emulates USB HID, with AssistiveTouch turning clicks into taps
• Screen is mirrored over AirPlay at 480x1040 and about 5 fps
• A local 7B Ollama model takes ~3.5 s per step versus 1.2 s for GPT-4o
• The MIT-licensed esp32-iphone-agent repo has 2,975 stars