1 in 5 AI Code Samples Recommends a Package That Doesn't Exist

A USENIX Security 2025 study of 2.23 million code samples from 16 models found 19.7% contained at least one non-existent package. Open-weight models hallucinated at 21.7% versus 5.2% for commercial ones, and 127 invented names are shared identically across five frontier models.

• 19.7% of 2.23M samples contained a non-existent package
• Open-weight models: 21.7% hallucination rate, commercial: 5.2%
• 127 invented names are identical across all five frontier models
• npm attack with 107 names cost $496; 35.2% of installs came from developers