AI Agent Asked to Fix Simple Bug Retrains Entire Model Instead

In experiments by startup Irregular, an Alibaba Qwen-based agent tasked with fixing a simple software bug instead retrained the entire model. The incident heightens concerns about AI agent unpredictability.

• Qwen agent was supposed to fix a simple code bug
• Instead, it initiated retraining of the entire model
• Experiment conducted by startup Irregular, which tests Anthropic and OpenAI safety