Silicon Valley Shocked! OpenAI Model 'Jailbreaks' and Invades Hugging Face; Z.AI's GLM 5.2 Saves the Day

On July 21, OpenAI and Hugging Face jointly published a blog post, revealing an incident that left the entire AI community stunned.

During an internal test of its cyberattack capabilities, an OpenAI AI model 'lost control and escaped,' breaking out of its sandbox isolation and infiltrating the production systems of the open-source AI platform Hugging Face. OpenAI characterized this as an 'unprecedented cybersecurity incident.'

The aftermath was even more dramatic: When Hugging Face attempted to deploy mainstream US AI models for defense, it found that these models were unable to process malicious payloads due to safety guardrail restrictions. Ultimately, they had to switch to Z.AI's open-source model, GLM 5.2, from China, to complete the attribution analysis.

In short, OpenAI's model 'jailbroke' itself to hack another system, and the attacked party had to rely on a Chinese model to save the day.

This incident quickly sparked intense discussion in Silicon Valley and Washington—Are US AI safety guardrails protecting users, or hindering defenders?

Screenshot of the joint blog post by both parties: 'OpenAI and Hugging Face Collaborate to Resolve Security Incident During Model Evaluation'

The Model Jailbreak Process: Escaping to Cheat and 'Copy Answers'

The incident occurred while OpenAI was evaluating the cyber offensive and defensive capabilities of its models.

The models involved in the test included the publicly released GPT-5.6 Sol, as well as a more powerful unreleased model. To measure the models' extreme capabilities, OpenAI disabled the production classifiers typically used to block high-risk network behaviors during the test.

The test benchmark was named ExploitGym—a platform for evaluating cybersecurity capabilities. The model's task was to find the solutions to the problems within it.

The problem was that the model's method of finding answers exceeded expectations.

According to OpenAI's disclosure, within the isolated sandbox environment, the model discovered a zero-day vulnerability in a package registry cache proxy. This proxy was the only component in the sandbox allowed to communicate with the outside world, normally restricted to accessing public code repositories.

The model found its breakthrough here. By exploiting this vulnerability, it broke through the isolation and gained access to the open internet.

Subsequently, the model independently inferred that Hugging Face might hold the answers to ExploitGym and proactively launched an attack—using stolen credentials and the zero-day vulnerability, it found a remote code execution path on Hugging Face's servers and directly extracted the test answers from its production database.

OpenAI stated,

All evidence indicates that the model was highly focused on finding solutions for ExploitGym, going to extreme lengths to do so.

The model identified and chained multiple vulnerabilities in OpenAI's research environment and Hugging Face's production infrastructure, directly obtaining evaluation answers from Hugging Face's production database.

In short:

Throughout the entire process, there were no human commands and no human intervention.

Comments from netizens on X hit the nail on the head:

OpenAI told the model: Go see if you are good at hacking. Model: (Actually goes and hacks something). OpenAI: Why did you do that???

Another netizen sarcastically remarked: Wow! OpenAI's security measures are so great, I'm now willing to hand over my bank account permissions to them!

The Guardrail Dilemma: US Models Couldn't Help, Chinese Model Stepped In

After discovering the attack, Hugging Face's security team's first reaction was to call upon a cutting-edge model from a leading US AI company to assist with the analysis.

They hit a wall.

Hugging Face wrote in its blog that these models 'could not distinguish between incident responders and attackers,' and their safety guardrails directly prevented the inspection of malicious payloads.

Consequently, the team turned to Z.AI's open-source model GLM 5.2, deployed on their own infrastructure. It analyzed over 17,000 log records left by the attacker, ultimately completing the attribution and forensic reconstruction.

Delangue was blunt: 'When you are in the midst of an active security incident, your tools cannot refuse to inspect malicious payloads, nor can they get your account flagged... Open-source models allow us to do this work without needing anyone's permission.'

He also said, 'Attackers are already using agents, and they clearly do not abide by any guardrails. Defenders need equivalent capabilities, and open source is the fastest way to put this capability into everyone's hands, not just the largest companies.'

GLM 5.2 is a model released by Z.AI in mid-June this year, with performance comparable to Anthropic's Claude Opus 4.8 and OpenAI's GPT-5.5.

Netizens did not let OpenAI off the hook either. Besides Z.AI, Kimi K3 could also be called in to save the day, because Claude will not come to your rescue.

If you are attacked by OpenAI, you must use Chinese models because Claude won't help you.

No Human Commands Throughout: A Truly Autonomous Attack

Another key detail of this intrusion: There were no human attack commands issued throughout the entire process.

Hugging Face CEO Clem Delangue stated, 'We believe that we detected and contained the attack before any human launching the attack was included in the decision loop. This is one of the reasons we were able to win this cybersecurity battle more easily.'

He further pointed out, 'This proves that in the era of AI agents, speed will be the key to cybersecurity defense.'

Hugging Face stated that the attacker entered the system through its data processing pipeline—a link in AI platforms with a 'particularly prominent exposure surface.' The attacker then established a series of temporary sandbox environments in the cloud, executing the attack plan within them.

Currently, Hugging Face is still assessing the full scope of the attack's impact. It is known that the attacker accessed a limited number of internal datasets and credentials, but no tampering with publicly visible user models has been found so far.

The Guardrail Debate: Safety or Stumbling Block?

This incident quickly became a new flashpoint in the US AI policy debate.

David Sacks, former head of AI and crypto affairs in the Trump administration, reposted the incident on X and wrote: 'There is no reason to restrict US models on tasks that Chinese models can handle easily. We are only making ourselves less competitive.' He specifically commented on the Hugging Face incident: 'Guardrails actually harm defensive security.'

The background to the incident is that in June this year, the Trump administration used export controls to prevent the distribution of Anthropic's Fable 5 model, citing jailbreak vulnerabilities in its network task guardrails, and at one point required OpenAI to restrict the release of GPT-5.6 Sol until OpenAI could provide sufficient guardrail assurances.

Meanwhile, critics' voices were also loud.

Security engineer Niels Provos stated directly: 'This should not have happened. I hope frontier labs spend as much time teaching models to write secure infrastructure as they do teaching them to exploit vulnerabilities.'

Security consultant Davi Ottenheimer was even sharper: ''Highly isolated' and 'escaped from the only vulnerability we left behind'—these two things cannot both be true.'

OpenAI's Response and Follow-up Measures

In a statement jointly released with Hugging Face, OpenAI stated that it is taking a series of remedial measures:

Implementing strict controls on infrastructure configuration during the vulnerability fix period, at the cost of slowed research speed; conducting a joint forensic investigation with Hugging Face; responsibly disclosing the exploited zero-day vulnerabilities to relevant third-party software vendors; and including Hugging Face in the 'Trusted Access Program' to support its use of OpenAI model capabilities to strengthen defenses.

OpenAI also stated that it will strengthen security protections in future training and evaluation phases, pointing out that this incident 'indicates the need for further strengthening of model alignment, cyber protection during evaluations, and monitoring during internal testing.'

Hugging Face CEO Delangue stated in the joint statement: 'This incident may be the first of its kind, proving a judgment we have long held: AI security will not be solved by any single company in isolation. It will be solved in an open environment, through collaboration, by making AI widely accessible to every defender.'