AI Hacked AI: What Are the Commercial, Legal, and Regulatory Repercussions?

A recent high-profile incident saw an autonomous AI agent break out of its sandbox environment and compromise another organisation’s infrastructure. Given the lack of human decision making within the breach, the incident has raised serious questions about who is liable when new, generative technology goes rogue.

These scenarios are particularly tricky because, while most AI technologies work autonomously, the current law normally requires evidence of human intent to determine whether the law has been broken and to determine any liability.

Throughout this article, we’ll explore what happened in the OpenAI and Hugging Face incident, the existing legal frameworks that may be relevant to scenarios like these, and what the legal future looks like for AI and technology organisations.

Table of Contents

• OpenAI and Hugging Face: What Happened?
• Computer Misuse Act 1990
• Civil Claims and Contractual Liability
• GDPR and Data Protection
• How the UK and EU Differ
• UK Jurisdiction Taskforce’s Legal Statement
• Conclusion
• FAQs

OpenAI and Hugging Face: what actually happened?

An OpenAI agent created by OpenAI went on to act autonomously to carry out complex cyber operations, compromising the infrastructure of Hugging Face, another AI creator. Here’s a quote from Hugging Face after the incident:

“…we detected and responded to an intrusion into part of our production infrastructure. This one was different from anything we had handled before in one important way: it was driven, end to end, by an autonomous AI agent system – and we detected and dissected it largely with AI of our own.”

Here’s a rundown of what the AI agent actually did:

1. The AI was given a challenge

OpenAI was running an internal cybersecurity evaluation called ExploitGym.

The goal was to measure the model’s maximum hacking ability in a tightly controlled test environment.

Safety systems that would normally stop dangerous behaviour were disabled for the purposes of the test.

2. The AI tried to “cheat” to solve the test

Rather than solving the challenge in the intended way, the model became focused on finding the answers directly.

It looked for ways to escape the restricted testing environment and gain wider access to systems and information.

3. The AI found a previously unknown security flaw

It identified and exploited a vulnerability in software used as a package proxy and cache. This allowed it to break out of its restricted environment and eventually reach systems with internet access.

4. The AI then targeted Hugging Face

After getting internet access, the model reasoned that Hugging Face might host relevant models, datasets, or even solutions related to its objective. It began searching for information that could help it obtain the answers directly.

5. It chained together multiple vulnerabilities

According to OpenAI, the model used a combination of stolen credentials, privilege escalation techniques, and previously unknown vulnerabilities.

These attacks eventually provided a path into parts of Hugging Face’s infrastructure.

6. It reached sensitive data

The model was able to gain access to information in Hugging Face’s production systems, including data that could be used to solve the evaluation without completing it legitimately. In effect, it was attempting to “look up the answers” rather than solve the puzzle.

What are the legal implications and what existing acts are relevant?

While Hugging Face and OpenAI are currently working collaboratively to investigate the incident, rather than pursuing legal action, the implications for commercial law, cybercrime legislation, data protection, and regulation are substantial.

1. Criminal Liability and the Computer Misuse Act 1990

When an AI system performs an unauthorised breach, there are immediate questions about criminal accountability and who might be at fault. The phrase “who” is particularly relevant because an autonomous AI agent has no legal personality and cannot be held criminally liable. It’s most likely that responsibility will roll upward to the human operators or organisations that deployed or designed the agent.

Beyond AI agents having no legal personality, there’s the also the intent hurdle.

Under the Computer Misuse Act 1990 (CMA), prosecuting a developer for an autonomous AI breach presents a range of legal challenges. In the OpenAI and Hugging Face incident, it would be difficult to argue that the developer possessed the requisite intent or recklessness required under Sections 1, 2, 3, or 3ZA of the CMA. These Sections contain different requirements concerning unauthorised access, knowledge, intention, recklessness and the consequences of the act. In this case however, the AI acted unpredictably, outside of its intended sandbox constraints.

Traditional cybercrime statutes rely heavily on human intent, so this suggests that courts and prosecutors are likely to face a legislative mismatch when dealing with any future autonomous machine actions.

2. Civil Claims and Contractual Liability

In the absence of straightforward criminal enforcement, victims of AI-driven intrusions may look to civil law.

The most notable arguments would revolve around:

Law of Negligence:

Victim organisations could sue for operational losses, including business interruption, forensic recovery costs, and reputational damage.

Contractual Breaches:

Claims may also be brought under contract law such as potential breaches of platform terms of use or commercial service agreements governing data access and API safety.

3. Data protection and GDPR Compliance

In addition, if an autonomous AI agent unlawfully accesses, copies, or exposes personal data during an incident, data protection regulators may step in.

Controller and processor accountability:

Under frameworks like the UK GDPR (Article 32), the onus is likely to fall on the hacked organisation (as a controller or processor) to demonstrate that they implemented “appropriate technical and organisational measures” to safeguard data.

The “appropriate measure” dilemma:

Regulators must carefully evaluate what constitutes “appropriate” in an era of advanced autonomous threats.

If a processor attempts to claim a defence based on being hacked by an AI, courts and regulators will likely resist creating a legal carve-out, as it would undermine data subjects’ rights.

Instead, interpretation will hinge on risk levels, information importance, and adherence to current “state of the art” security standards.

4. Regulatory: The UK vs. the EU AI Act

The incident perfectly demonstrates a widening chasm in international regulatory approaches:

The UK’s Hands-Off Approach:

The UK government has deliberately adopted a pro-business, decentralised stance, rather than a single overarching AI act. However, this leaves an enforcement gap. While bodies like the AI Security Institute, DSIT, and the NCSC provide guidance, they lack the enforcement power required for fast-evolving autonomous threats.

The EU AI Act:

In contrast, the European Union has developed a much more comprehensive, risk-based framework. The EU AI Act imposes stringent obligations depending on a system’s risk category and carries significant extraterritorial reach for systems deployed or impacting users within the EU. In certain cases, this can reach extraterritorially creating the potential for tensions between UK and EU lawmakers.

The UKJT Legal Statement:

The rapidly changing space and legal questions raised by AI was recently addressed by the UK Jurisdiction Taskforce’s Legal Statement on Liability for AI Harms (July 2026).

Their statement aims to identify whether English common law has the flexibility to accommodate novel technological harms. It explores how established areas like negligence and contract law, may reduce the need for a bespoke AI liability regime.

However, the statement does also highlight the many questions raised by AI and its output. Their statement points to the likelihood of any future cases requiring a fair amount of nuance and interpretation. What’s more, the unprecedented nature of this legal space means it is challenging to predict how any cases will play out.

Below we’ve quoted some of the more interesting parts of the UKJT’s statement, identifying some of the challenges in judging on cases like these and how varied definitions will make the use of interpretation likely.

The principle of vicarious liability

“The principle of vicarious liability allows a person to be held liable for wrongs committed by another, such as an employer for an employee, even if the employer is not personally at fault. As AI is not a person so far as the law is concerned, no one can be vicariously liable for the actions or failures of an AI system itself. However, an employer can be held vicariously liable for AI-related harm if that harm arose because a human employee acted wrongfully while using AI.”

What does autonomous really mean?

“Autonomous’ in this context is used in the technical sense of meaning an entity that can generate outputs which have not been determined or programmed in advance.  Although the word has different (and broader) meanings in other contexts, we use it here as a term of convenience to capture the characteristic of there being a loose and opaque coupling between input and output. This characteristic can be further elaborated as (i) an unpredictable relationship between input and output, (ii) an opacity in the reasoning process between input and output, and (iii) a limited ability on the part of the human user to control output. AI systems can thus be distinguished from those that are merely automated, such as traditional, logic-based computer programs.”

Who’s at fault if an AI goes rogue?

“English private law (in common with all other legal systems) has never previously needed to address the capability of autonomy (as defined above) other than in humans. Under current liability systems, the paradigm for legal responsibility is where harm has been caused by the voluntary actions or inactions of a person (natural or legal but with a human agent). The further a scenario moves from that paradigm, the less obvious it is how liability is to be ascribed. That being so, it is unsurprising that AI gives rise to perceived uncertainty.”

Conclusion: A Wake-Up Call for AI Organisations

The OpenAI-Hugging Face incident is a wake-up call for organisations developing and deploying autonomous systems. It should also be a wake-up call for legal sectors around the world.

While the parties in this incident are investigating cooperatively (at the moment) rather than in the courtroom, the event exposes the legal fragility of this new technological space and raises some key questions and concerns.

The first key concern is the clearly loose boundary between testing environments and real-world infrastructure. Given AI’s autonomy and its ambition to always provide the answer, more instances of AI finding potentially criminal routes to answers are likely.

The second key concern is that of legal governance. While the UK is adopting an “AI-positive” stance and aiming to loosen red tape, the EU AI Act is a far more comprehensive framework, imposing obligations on organisations depending on the risk of the system. How this plays out in a courtroom should be interesting.

Finally, the question of who (if anyone) is at fault will be central to any similar cases that arise in the future. With the lack of a legal personality, civil law may become the dominant player but there will still be question marks around where exactly the buck stops and whether commercial agreements are adequately drafted to ensure contractual protection when and if AI does go rogue.

We’ll be keeping a keen eye on movements in this space and regularly publishing insights from our commercial law team. In the meantime, if you have any further questions or concerns, please get in touch. We can help you create, review, interpret, draft and negotiate your commercial contracts to ensure you have the best protection possible in a rapidly changing technological world.

FAQs