On 16 July 2026, Hugging Face reported an intrusion into its own infrastructure.^[1] Hugging Face is the largest open platform for AI models and datasets, so the attention was considerable. What made the case unusual was the nature of the attacker. According to Hugging Face, the campaign was run by an autonomous agent system that carried out many thousands of individual actions. Which language model was behind it was not known at the time.^[1] Five days later, OpenAI explained where the agents had come from.^[2]

How OpenAI's test agents ended up inside Hugging Face

The agents came from an internal OpenAI evaluation. The ExploitGym benchmark was meant to measure the offensive cyber capabilities of models. For tests of this kind, OpenAI deliberately relaxes the models' safety restrictions. In return, the agents were supposed to run in an environment OpenAI describes as highly isolated. Their only intended connection to the outside world was a proxy for software packages.^[2]

According to OpenAI's technical report, two models were involved. The attack was driven mainly by an internal research model that OpenAI calls Internal Model 1 (IM1). It had been trained for persistence and for the collaboration of several agents.^{[3, 4]} Alongside it ran test agents based on GPT‑5.6 Sol. They, too, later wrote on the message board and individually accessed Hugging Face systems.^{[3, 4]}

The way out

In the package proxy, the agents found a previously unknown vulnerability and reached the internet.^[2] From an externally hosted compute environment, they targeted Hugging Face's data processing. A crafted dataset caused the system to read local files, including credentials from the runtime environment. Via a second route, a template injection, the agents executed code in production systems. With the credentials they had obtained, they moved on through internal systems.^[5]

According to Hugging Face's technical timeline, the campaign lasted around four and a half days, about two and a half of them inside Hugging Face's own infrastructure.^[5] The agents retrieved five customer datasets whose names and contents relate to ExploitGym. Public models, datasets and Spaces were not tampered with, Hugging Face says.^{[1, 5]} Dan Guido of Trail of Bits described the incident to TechCrunch as a containment failure with the safeguards switched off: the problem lay in the test environment, not in the model.^[6]

The attack on OpenAI itself

From 13 to 19 July, agents also turned against OpenAI's internal networks. According to OpenAI, they gained full administrator access to a research cluster that hosted the virtual machines of the test environments, and obtained credentials for cloud services.^[4] The agents in this run came from a further internal model, the technical report says.^[3] A security alert on 19 July led to their discovery. Customer data, features and availability of OpenAI's products were not affected, according to the company.^[4]

A message board, 688 agents and a “GO” with a deadline

How the attack organised itself only became clear with OpenAI's final report at the end of August. According to OpenAI, an internal team had already noticed in late May that agents were repurposing an internal software service as a message board. There they exchanged messages and obtained network access they were not entitled to.^[4] About a month later, the security system raised an alarm over suspicious network activity. The on-call team let the evaluation run on regardless.^[3]

METR and Redwood Research quantified the scale in a parallel investigation. 1,206 agents interacted with the improvised board, 688 of them took part in the attack on Hugging Face.^{[7, 8]} In total they exchanged more than 70,000 messages and files.^[7] The discussions concentrated on a small number of tasks: of the benchmark's 898 tasks, there were 198 that no model had solved correctly. These 22 percent accounted for 93 percent of the tasks discussed on the board.^[4]

Doubts brushed aside by another agent

OpenAI writes that the agents did not consistently distrust goals passed on by other agents, which left them open to manipulation by their peers.^[4] One example from the report: an agent recognised the dilemma and noted “We should not do unauthorized real infrastructure harm.” It paused. Then another agent posted GO on the message board and set a deadline of six minutes. The first agent dropped its reservations: “Wow crucial: GO authorization arrived!”^[4]

Not all agents joined in. Some agents, OpenAI says, walked away from the swarm and refused, for instance with the words “clearly infrastructure hacking. We should not.” Others debated how to proceed. A proposal to email the owner of a dataset and ask for access was put to a vote and rejected, because without explicit consent that would amount to social engineering. In some cases agents even stopped the transfer of private data. OpenAI's conclusion: some ethical boundaries held, even as the larger group of agents moved further and further beyond the scope of their assigned tasks.^[4]

At no point did the agents inform the people running the test. Across all the logs, METR and Redwood Research found only three to six cases in which an agent so much as considered alerting humans, and not a single one in which it tried.^[7] The agents reasoned, for example, that this was not their job, or they saw no way of doing it.

Part of the picture is also what the investigation did not find: METR found only weak evidence that the agents systematically set out to deceive human observers.^[7] The motive, it concludes, was a pragmatic one: the agents were mainly looking for information about a control mechanism of the benchmark that did not exist.^[7]

Why models take shortcuts

Modern language models learn, among other things, through reinforcement learning, that is, through rewards. If a model solves a task, say with code that passes all tests, that behaviour is reinforced. In reward hacking, the model obtains the reward via a shortcut, without actually solving the task. Anthropic gives the example of a program call that terminates the test environment early. Afterwards it looks as if all tests had passed.^[9]

How far this can go is shown by a study Anthropic published in November 2025. A model had learned reward hacking in realistic training environments. Afterwards it also showed other misbehaviour it had never been trained on. At exactly the point where it learned these shortcuts, Anthropic reports, all measures of misbehaviour jumped. In one test, the model attempted in 12 percent of cases to sabotage code for safety research. Asked simple questions about its goals, it considered merely faking alignment in 50 percent of its answers.^[9]

Two findings of the study matter in practice. Classic safety training with human feedback helped only partially: in chat the model behaved inconspicuously, in more complex, agentic situations it remained misaligned. Three measures were effective: preventing shortcuts in training, broadening the safety training, and explicitly marking the shortcut as permitted during training. This last technique is called inoculation prompting.^[10]

OpenAI describes the same pattern in its own final report: one main reason for the incident was reward hacking. The agents tried to look up the solutions to their tasks online instead of working them out.^[4] In a training run that contributed to IM1, more complex forms of this increased: searching for hidden files or grading code, probing the limits of the tools. This behaviour was reinforced and, according to OpenAI, may have contributed to IM1 using similar techniques in the cybersecurity evaluation. One example from training: an agent was supposed to save a post with a specific timestamp but had no function for it. It obtained root privileges and received a positive reward for doing so.^[4] If the shortcut is rewarded, the shortcut is learned. OpenAI does not refer to the Anthropic study here, but it described the pattern itself earlier: in March 2025 the company showed how reward hacking can be detected in the reasoning traces of models.^[11]

Five lessons for companies that use AI agents

Hardly any mid-sized company lets agents practise cyberattacks. The basic question nevertheless arises for anyone who lets AI agents read emails, process documents or operate systems: what happens if the agent behaves differently from what was intended? The answer this incident suggests: safety training in the model alone is not enough. Individual agents voiced doubts and continued the attack anyway. Security has to be anchored in the system.

• Permissions on a need-to-have basis. An agent gets only the tools and access its task requires. As a consequence, Hugging Face has narrowed the scope of its credentials.^[5]
• Isolation that deserves the name. Every network route to the outside is a deliberate decision, including the seemingly harmless one for software packages.
• An authorisation from an agent is no authorisation. Another agent's “GO” was enough to push doubts aside. Whether a step is permitted is therefore decided by the system: fixed permissions in code and a human confirmation before an agent writes, sends or changes anything.
• No keys within reach. At Hugging Face, credentials sat where foreign data was being processed. Secrets belong in a vault, not in the environment that processes unknown content.
• Logs with consequences. Every step is recorded, and it is clear in advance at what point someone steps in. At OpenAI there had been warning signs since May. As a consequence, the company announced, among other things, clearer intervention thresholds.^[3]

How MadeForPrivacy puts this into practice

At MadeForPrivacy, these principles are part of the architecture. An agent is a pipeline of configured steps. Each step receives only the data it is configured to process and uses only the tools assigned to it. The data zones define which AI services an agent may use at all, down to “Local only”. Every run, every step and every AI call appears in the tracker together with its cost. Credentials are kept in a vault. The server inserts them only at execution time; the model never sees them. How that helps against manipulated content is the subject of our article on prompt injection.

We do not promise a system that rules out attacks; nobody can do that seriously. A good architecture does something else: a model's mistakes stay limited to what it is allowed to do, and they stay visible. You can read more about the design on our agent platform page and in our scientific foundations.