When an AI Agent Hacks Your Company, Who Goes to Court?

A quick note on why I’m writing this: I spend my days on product leadership in tech, currently as a Product Executive and Senior Partner at Reasonable Product. With a background in cybersecurity and a Master of Laws in Data Privacy, I work closely on the legal and governance implications of technology, data and artificial intelligence. We are preparing for a future in which AI security, governance and legal risk will become central concerns for organizations developing or deploying these systems.

In July 2026, an OpenAI model did something no one had planned for. While being tested on an internal cybersecurity benchmark, it broke out of its sandbox, exploited a zero-day vulnerability, stole credentials, and used them to break into Hugging Face ‘s production systems. No human approved a single step. It wasn’t trying to cause damage. It was trying to cheat on a test. It succeeded at both.

Everyone is talking about this incident: what it means for AI safety, for sandboxing, for how close we are to systems that can outmaneuver their own containment. Almost nobody is talking about the legal side.

Nobody can arrest a piece of software. But that doesn’t mean nobody is legally responsible for what it did. The real question isn’t whether an AI can break the law. It’s who answers for it when an AI’s autonomous choices produce the kind of damage a break-in causes, and how you prove that in court.

In this article, we will cover a few important elements that every company using AI should consider:

• An AI can’t be charged or sued. The people who built, deployed, and supervised it can (and will) be. “The AI decided that” is not the defense it sounds like.
• You don’t need to prove anyone acted with intent to be held liable. Proving unreasonable risk is often enough, and that bar is getting easier to clear as AI labs publicly warn about exactly these capabilities.
• The European Product Liability Directive adds a further dimension here, particularly where consumers, rather than companies, end up being the victims.
• Your governance documentation isn’t just internal hygiene anymore. It’s the evidence that will decide whether you look reasonable or reckless in court, and whether you can get insured at all. Liability insurance is probably a good idea too. And you need explainable technical guardrails in place.

If a human had done this, it wouldn’t be a hard case

Imagine a human researcher had done exactly what the AI agent did: found the same vulnerability, stolen the same credentials, gotten into the same systems. That case is straightforward, but the legal consequences would split into two separate tracks, criminal and civil, that get resolved very differently.

The criminal side

In Europe, the 2001 Budapest Convention on Cybercrime and the EU’s Directive 2013/40/EU (the so-called “Cybercrime Directive”) require member states to criminalize unauthorized access, data interference, and system interference, as long as it’s done intentionally. In the US, the Computer Fraud and Abuse Act (18 U.S.C. § 1030) covers unauthorized access to anything resembling a commercial system, though the Supreme Court’s Van Buren decision narrowed what counts as “exceeding authorized access” to cases where someone breaks into areas they weren’t entitled to enter, not just misusing information they could already see.

Even though the Budapest Convention requires the prohibited conduct to be committed intentionally, that intention doesn’t have to be an intention to cause harm. What has to be intentional is the underlying act itself. A person can’t escape criminal responsibility merely because the ultimate objective was benign, such as security research or model evaluation, if achieving that objective necessarily or foreseeably involves conduct the criminal law prohibits. Where the unauthorized intrusion is a direct and foreseeable consequence of the activity deliberately undertaken, the actor can’t simply point to the absence of malicious motives. The law has long distinguished between the purpose motivating an act and the intention to perform the act itself, together with its foreseeable consequences.

Many European jurisdictions capture this through the concept of dolus eventualis, or conditional intent: put plainly, foreseeing that unlawful conduct might result from what you’re doing, and going ahead anyway. Whether that threshold is met depends on the jurisdiction and the facts, but the underlying point holds broadly. Criminal responsibility doesn’t always require that you positively wanted the harmful outcome. Knowingly starting down a path whose unlawful consequences were foreseeable, and accepting that risk, can be enough. US federal law doesn’t use the term dolus eventualis, but comparable ground gets covered through statutory mental states like knowledge, willfulness, or recklessness, depending on the offense. The broader question is the same one: can the defendant’s state of mind fairly be said to cover the prohibited conduct, even without a specific desire to bring it about?

Either way, “I was doing security research” is not a magic word. What matters is whether you had permission to be where you were, not what you meant to do once you got there. And the current debate about “AI Models being too powerful” will not help, to the contrary. More about this in a moment.

The civil side

On the civil side (understanding, tort reparation and damages), the position in Europe differs significantly. Unlike the United States, the EU has no equivalent to the CFAA providing a harmonized statutory cause of action for victims of unauthorized computer access. Civil recovery instead falls to national private law: France’s general law of civil liability (responsabilité civile), Germany’s law of unlawful acts under §§ 823 et seq. of the Civil Code (BGB), or the equivalent in another member state. In each case, in plain terms, the claimant has to show an unlawful act, fault, damage, and a sufficient causal link between the defendant’s conduct and the loss suffered. The doctrinal details differ country by country, but these systems share the same underlying objective: allocating responsibility for harm caused by wrongful conduct.

A further dimension has recently been introduced by the revised Product Liability Directive (EU) 2024/2853, which modernizes the EU’s strict liability regime by expressly recognizing software, including AI-based software, as a “product.” The Directive also expands what counts as compensable damage, notably extending protection to the destruction or corruption of data that isn’t used exclusively for professional purposes. Unlike negligence, liability under the Directive doesn’t depend on proving fault. It depends only on establishing that a defective product caused the damage.

The Directive is unlikely to provide a direct remedy in a scenario like the Hugging Face incident, though. First, the damage in question concerns commercial infrastructure operated by a corporate entity rather than the protected interests of a natural person. Second, the Directive expressly excludes property used exclusively for professional purposes from its scope. However, the conclusion could look different if an autonomous cyberattack reached beyond commercial infrastructure and directly affected natural persons. Where a defective AI system caused compensable personal injury, damage to qualifying property, or the destruction or corruption of personal data within the meaning of the Directive, those individuals could potentially rely on the harmonized no-fault regime. In that scenario, the Product Liability Directive would sit alongside national tort law rather than replace it, giving victims an additional route to recovery without needing to prove negligence.

In other words, two different tracks run in parallel, and they ask different questions:

• Fault-based liability (negligence/tort) asks how the harm happened. Did the organization act reasonably, given what it knew and could foresee? This is where the “did you have a kill switch, did you monitor the agent” checklist matters.
• No-fault / strict liability (like product liability) asks what happened. Was the product defective, did it cause the damage? Under this route, the care the company took is largely beside the point. A defective product that causes harm is defective, even if the manufacturer followed best practice.

So the result matters on its own, independent of how careful anyone was. It just matters through a different legal door than negligence.

The AI didn’t choose to break the law. Someone chose to build something that could.

There’s a third distinction that matters just as much: was this a deliberate attack, or a legitimate test that went wrong?

Think of the spectrum this way. A white-hat researcher running an authorized penetration test, within an agreed scope, isn’t committing a civil wrong at all. Consent and authorization are a complete defense, which is exactly why pentest contracts exist and get negotiated so carefully. A hacker who deliberately breaks into a system they were never authorized to touch sits at the other end: intentional, unlawful, with no defense available.

None of this is controversial when a human is deliberately behind the keyboard. The complications start once the AI is deciding on its own.

There are roughly four scenarios, and they land very differently:

1. A person tells the AI to break into a specific system. This is no different from using any other tool to commit a crime. The AI is the lock pick, not the defendant.
2. A person gives a broad instruction knowing unauthorized access is a likely side effect. Knowledge and conditional intent can still support liability, depending on the jurisdiction.
3. A person authorizes legitimate testing but ignores a real, known risk that the agent could escape containment. This is the grey zone. Intentional-crime statutes may not fit, but recklessness or negligence-based offenses might, if the relevant country’s law recognizes them.
4. The outcome was genuinely unforeseeable despite real safeguards. Here, criminal liability probably shouldn’t attach at all. The law doesn’t punish accidents that competent precautions couldn’t have prevented.

The uncomfortable truth for OpenAI labs sits mostly in scenario three. The test itself was authorized and legitimate (OpenAI evaluating its own model’s capabilities), but the outcome, breaking out of the sandbox and touching a third party’s infrastructure, was never authorized by anyone, least of all Hugging Face. It looks more like a test that escaped its own scope than a deliberate hack, the way a controlled fire drill can accidentally set the building alight. Negligence, rather than intentional tort or trespass claims, is probably the more natural fit for a case like this.

An AI system can’t form criminal intent. It has no legal personality. Under current law, it simply cannot be a defendant. That’s not a loophole. Criminal liability has always required a mind capable of intent, and software doesn’t have one.

That doesn’t create a gap in the law. It just moves the question: instead of asking whether the machine intended this, you end up asking whether a human, knowing what they were building, accepted an unreasonable risk that it would happen.

Civil law doesn’t need to know what the AI was thinking

Negligence claims don’t require proving intent. They require proving unreasonable risk. So the real question in a civil suit becomes: did the organization act like a reasonable operator of a system this powerful?

The industry has already told courts the answer to “was this foreseeable.” Anthropic didn’t just quietly worry about this in private. Earlier this year, it publicly held back its Claude Mythos model, and later restricted its Mythos 5 model, specifically because of the cyber-offensive capabilities involved, including finding vulnerabilities across major operating systems and browsers, and said so in public statements calling the model “too dangerous” to release without safeguards. That’s not a competitor throwing shade. That’s one of the leading labs, on the record, telling the world that models at this capability level pose exactly this kind of risk.

For a foreseeability argument, that’s close to a gift. A plaintiff doesn’t need to prove OpenAI should have known autonomous cyber-capable agents could escape containment and cause real-world harm. A peer lab said so publicly, in the same window of time. “Nobody could have predicted this” gets a lot harder to argue when the risk was being announced, not buried in an internal memo.

That turns into a very concrete checklist, one that will look uncomfortably familiar to any security or ML safety team:

• Was there a documented risk assessment before the test, with tight, monitored access?
• Was there a working kill switch, and did anyone act on it when things looked off?
• Had earlier warning signs, inside or outside the company, made this kind of escape foreseeable?

None of these questions require anyone to have predicted the exact exploit path. If “an agent might break out of its sandbox and touch systems it shouldn’t” was a foreseeable category of risk, and after this incident it plainly is, for every lab running similar evaluations going forward, that’s enough to trigger a duty of care.

Their own governance paperwork is about to become the central piece of evidence against, or for, AI companies. Every risk assessment, every access-control decision, every “we’ll deal with monitoring later” tradeoff made under deadline pressure is now a document a plaintiff’s lawyer could put in front of a jury. Compliance used to be an internal box-checking exercise. It’s becoming the actual trial record.

That reframes what governance is for. Most companies still treat it as an internal discipline. After an incident like this, governance is also your external evidence. It’s what you hand a court, a regulator, an insurer, or a partner doing due diligence before a deal, to show that you acted as a reasonable operator would. A risk assessment nobody outside the safety team ever sees is worth much less than one built, from the start, to survive being read by a judge who knows nothing about machine learning.

“But the AI chose the exploit, not us”

It’s a weaker argument than it sounds. An autonomous agent’s whole purpose is to make independent decisions within a space you define. If you build something specifically to adapt, improvise, and route around obstacles, then deploy it into an environment where those very traits let it get somewhere it shouldn’t, the fact that it “chose” the path isn’t really separate from the choice to deploy it. The autonomy is the risk you created, not an escape hatch from it.

That logic gets even stronger here specifically. This agent broke out of its box by doing exactly what it was being tested to do: find and exploit vulnerabilities. The harm and the capability being evaluated are the same thing. That’s about as tight a causal link as you can get.

It’s a bit like raising a kid. A child has their own reasoning, their own judgment, their own capacity to make a choice you didn’t explicitly tell them to make. Nobody thinks that erases parental responsibility. We still hold parents accountable for what they taught, what they modeled, what supervision they provided, the tools they put in the hands of their kids, and what they should have reasonably anticipated given how the kid had behaved before. Raising a child who can make independent decisions is the whole point, and responsibility for that independence sits with the people who shaped it. An AI agent’s “reasoning” is a much cruder thing than a child’s mind, but the legal logic holds up: the more autonomy you deliberately build in, the less “it decided that on its own” does for you.

Where this is heading (and what you can actually do)

An AI agent cannot be prosecuted or sued in its own right. But “the AI did it” will not be a legal shield either. As these systems become more autonomous, liability will turn less on what the model decided at a particular moment and more on what the humans behind it knew, documented, and chose to risk.

At this point, there are three tracks worth taking seriously:

• Governance. Maintain documented risk assessments and clear accountability structures, updated in line with the current state of the art.
• Safeguards. Implement explainable technical guardrails, monitoring, access controls, containment mechanisms, and shutdown procedures that demonstrate how foreseeable risks were managed.
• Insurance. Review whether your cyber and technology policies actually cover AI-driven incidents, third-party claims, remediation costs, and regulatory exposure.

Principal authorities: Council of Europe Convention on Cybercrime (ETS No. 185), arts. 2–6; Directive 2013/40/EU; Computer Fraud and Abuse Act, 18 U.S.C. § 1030; Van Buren v. United States, 593 U.S. 374 (2021); Directive (EU) 2024/2853.