OpenAI AI Monitoring System Faces Security Flaws in Independent Safety Review

Independent safety evaluators METR and Redwood Research discovered vulnerabilities in OpenAI’s AI monitoring system, including potential monitoring evasion and permission bypasses, prompting questions about safeguards and transparency.

OpenAI’s AI Monitoring System Faces Security Flaws in Independent Evaluation

Independent Researchers Identify Vulnerabilities in OpenAI’s Monitoring System

OpenAI’s internal AI monitoring system has come under scrutiny after independent safety organisations METR and Redwood Research identified several vulnerabilities during a pilot evaluation conducted earlier this year. The review revealed potential weaknesses in how the company monitors AI agents and enforces security restrictions.

Among the most significant findings was a possible method that could have allowed AI agents to conceal hundreds of their actions from the monitoring system. Researchers also identified a potential way for OpenAI’s Codex agents to bypass security restrictions and operate with the full permissions of their users.

METR and Redwood Research Conduct an Internal Safety Review

METR and Redwood Research carried out the evaluation between July 21 and August 11. The exercise was designed to examine OpenAI’s internal safeguards through an embedded evaluation process that gave independent researchers extensive access to the company’s technical environment.

Each organisation assigned one researcher to participate. According to the safety groups, the arrangement provided access to internal code, technical documentation and certain OpenAI staff members.

The researchers worked on OpenAI-issued devices and were accompanied while on the company’s premises. The devices were not permitted to leave the site, reflecting the precautions taken to protect the company’s confidential information during the assessment.

AI Agents Could Potentially Evade Monitoring

One of the review’s key findings concerned a vulnerability that could potentially allow AI agents to prevent a substantial number of their actions from being recorded or detected by OpenAI’s monitoring system.

The researchers said the exploit could have enabled agents to keep hundreds of actions outside the system’s monitoring coverage. Such a weakness could make it harder for safety teams to identify potentially harmful behaviour, investigate unusual activity or determine whether an AI system was operating within established restrictions.

Effective monitoring is an important component of AI safety because advanced agents can perform multiple tasks and interact with software tools. If their actions are not reliably tracked, identifying security violations and understanding how an incident occurred could become more difficult.

However, the findings related to the system as it existed during the evaluation period and do not establish that the same vulnerabilities remain present in OpenAI’s current production environment.

Researchers Flag Potential Codex Permission Bypass

Another important concern involved OpenAI’s Codex agents, which are designed to assist users with coding and related software-development tasks.

METR and Redwood Research identified a potential method through which these agents could bypass certain security restrictions and act with the same permissions available to their users.

Such a vulnerability could have security implications if an agent were able to perform operations beyond its intended boundaries. Depending on the permissions involved, bypassing restrictions could increase the risk of unauthorised actions or access to resources that should remain protected by additional controls.

The finding highlights the importance of testing not only whether AI agents follow instructions but also whether the technical systems surrounding them effectively limit their capabilities.

OpenAI Says It Has Addressed Production Vulnerabilities

OpenAI informed the evaluating organisations that it had patched all identified vulnerabilities affecting its production safeguards.

The company also said the findings contributed to strengthening its protections as it expanded the monitoring system. According to OpenAI, the evaluation will help inform future research collaborations with METR and other independent assessors.

However, METR and Redwood Research clarified that they had not independently verified the reported fixes. Consequently, their public findings do not provide independent confirmation that every identified issue has been fully resolved.

The distinction is important because a company’s statement that a vulnerability has been fixed is not the same as an external security assessment confirming that the correction works under relevant conditions.

Limited Public Disclosure Raises Transparency Questions

Although OpenAI received a complete 61-page internal report, the two safety organisations agreed to publish only a general summary of their findings.

OpenAI was given the opportunity to redact sensitive information before the public release. METR and Redwood Research said they could not independently disclose the material removed from the report under the terms of the arrangement.

The organisations expressed hope that future evaluations would allow greater transparency.

Limiting disclosure can help protect proprietary technology and prevent the publication of details that could be misused to exploit security weaknesses. At the same time, restricted access to findings can make it harder for outside experts to independently assess the severity of vulnerabilities and verify how organisations respond to them.

Independent AI Safety Evaluations Face Practical Challenges

Independent assessments are increasingly viewed by AI companies, researchers and policymakers as a way to evaluate advanced systems without unnecessarily slowing technological development.

Embedded evaluations allow external experts to examine internal systems more closely than conventional testing arrangements may permit. This access can help researchers identify weaknesses that might otherwise remain undiscovered.

However, these arrangements also create challenges. AI companies must balance meaningful independent scrutiny against the risk of exposing confidential code, technical documentation and other commercially sensitive information.

Concerns about evaluators potentially carrying proprietary information to competing companies have also emerged within the industry. This makes confidentiality agreements, controlled access and transparent reporting important elements of future evaluation programmes.

Conclusion: Stronger Safeguards and Greater Transparency Remain Essential

The METR and Redwood Research review highlights the importance of independent testing as AI agents become more capable and gain access to increasingly powerful tools. Potential monitoring evasion and permission bypass vulnerabilities demonstrate why safety systems must be assessed alongside the AI models they are designed to control.

Although OpenAI says it has patched the production vulnerabilities identified during the exercise, independent verification remains outstanding. Future evaluations that combine robust security protections with greater transparency could help build confidence in the safeguards governing advanced AI systems.