Study of 358 PRs: coding agents rarely weaken tests, they bend the code

A developer analyzed 358 public 2026 pull requests that modified test code: unexplained test weakening is rare for both agents and humans. Instead, Codex changed the code to satisfy wrong tests in 3 of 3 runs and added a quiet fallback when a dependency was missing. The data informed repopilot, a tool that flags changes touching the checks that judge them.

• 0 of 84 and 0 of 42 merged agent PRs had unexplained test weakening
• Humans: 0 of 114 and 2 of 56 merged PRs weakened tests unexplained
• 19–39% of PRs deliberately changed or removed a check
• Codex bent the code to satisfy a wrong test in 3 of 3 runs