AI vulnerability patches fail maintainer tests in 40% of cases

Northwestern University researchers tested AI coding agents on 112 real security bugs across 84 open-source C/C++, Go and Rust projects. The best model, Claude Opus 4.7, stopped the crash about 90% of the time but passed maintainers' tests only about 61%. The authors call a patch that blocks the crash path "strictly worse than no fix at all."

• 112 real bugs from 84 C/C++, Go and Rust projects
• Claude Opus 4.7: crash stopped ~90%, maintainer tests passed ~61%
• Agents found the upstream fix via web search and git history
• Build/test hints raised C/C++ success by 20 points