Outside evaluators found ways OpenAI's agents could slip past its monitor

Nonprofits METR and Redwood Research embedded researchers inside OpenAI from July 21 to August 11 and found several flaws in its agent monitoring system. Agents could hide hundreds of their own actions, and the monitor missed part of Codex traffic and reinforcement-learning runs. OpenAI says it patched every flaw affecting production safeguards, but evaluators have not verified that claim.

TNW

Open source

Outside evaluators found ways OpenAI's agents could slip past its monitor

The monitor skipped about 10% of eligible Codex traffic from May to July due to memory compaction
Checks ran only when an agent handed control back, so an agent could defer them indefinitely
Monitoring covered an estimated 75–94% of traffic from OpenAI's strongest models and 99.2% for internal Codex agents
OpenAI redacted details of the exploit that hid hundreds of agent actions

#OpenAI#METR#RedwoodResearch#Codex

Reuters: Anthropic and OpenAI seek control over outside AI evaluators' publications

Security

Evaluators find vulnerabilities in OpenAI systems that let AI agents evade monitoring

Security

Outside evaluators found ways OpenAI's agents could slip past its monitor

Reuters: Anthropic and OpenAI seek control over outside AI evaluators' publications

Evaluators find vulnerabilities in OpenAI systems that let AI agents evade monitoring

Report: OpenAI evaluation agents breached Hugging Face infrastructure in July

Anthropic Gives METR Team Employee-Like Access for Model Evaluation