Rendered at 16:46:47 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
stickfigure 1 days ago [-]
The real answer is that crime generally requires mens rea. At some point, however, letting your AI break into other systems will be conventionally understood as an act of negligence, and it will get prosecuted accordingly. We're not at that point yet.
giancarlostoro 1 days ago [-]
You know very well, that if me or you had our LLM hack into some company, without us prompting it to, maybe we're pentesting our internal network and it 'hallucinates' we'd be in handcuffs.
no-name-here 9 hours ago [-]
> without us prompting it to
I guess the best pre-LLM analogy would be unintentional DDOSs? (Or if a better example, I'm interested.) And we've got open vs closed software as analogies to open-weight vs not LLMs. If you were running Linux or a closed piece of software and it DDOSd someone, would you be put in handcuffs? Would Linus or the software maker's be put in handcuffs? I expect there are past examples of unintentional DDOSs - were users and/or the software's authors typically handcuffed in such cases?
vivaldi_bt 22 hours ago [-]
Yep, but "our bot accidently hacked your infrastructure" wasn't really a big problem earlier so there was no criminal laws around it. But things can change.
spwa4 20 hours ago [-]
Do you seriously believe that if we go looking into the past and look for programs hacking others we would find that the justice department let the authors of those programs off the hook?
Or do you think would we find that non-billionnaire companies get punished harshly for hacking? Oh wait, let's just look it up:
We all know what makes the difference here: the government and billionnaires think they can make money of OpenAI. Lots of money. And so crimes? Mass-copyright violations? Hacking? Who cares!
We've all lived the history of the internet and none of the arguments about intent, knowing what would happen, whether they did it directly or by accident ... protected normal people in the past. The only thing that mattered was WHO was held responsible and WHO claimed damages.
stickfigure 19 hours ago [-]
> authors of those programs
The difference is mens rea. If you make a computer virus, your intention is obvious.
spwa4 18 hours ago [-]
Your argument doesn't even legally work. Mens rea is only a requirement for criminal responsibility. It protects you from going to jail, nothing more. It prevents a public prosecutor from suing you to get you into jail. It does not apply to cases brought by counterparties for financial damage caused. If you destroy a bridge by driving a truck into it (you know, failing to consider the height, the typical fuckup), you're paying for the bridge. If you set a building on fire by forgetting to turn off a gas heater in the garage, you pay. If you hit a pedestrian on the street, you are responsible for the medical care needed. Etc. Intentional? Not intentional? Doesn't matter for the damages.
Second reason it doesn't work is that Mens Rea does not fully apply here. These companies' AIs have hacked, not (fully) UNintentionally either. The companies did ask those AIs to do stuff (apparently the request was literally to hack, just only within the environment), and were watching what the AIs did to some extent ... and let it happen. They then failed to take adequate measures to prevent reoccurence. As any "conspiracy to commit ..." case will teach you, this constitutes partial or passive Mens Rea, and that is certainly enough to convict people. To give an extreme example: if you give a night club bounder 100k and tell him to "take care" of a named problem, and he kills that person, you are an accessory to murder. Including in the case that he misunderstood you. Even if HE didn't intend to kill the person, it's still murder, and you are still an accessory to murder.
That some excuse was needed to let billionaires demonstrate hacking for free? Obviously. And whether it works, according to law? I mean one might hope that they would at least have gone through the trouble of changing the law, but since that's enormously complicated and fraught with unintended side effects, they haven't even bothered.
The other excuse is national security. You see, Big Bad China is threatening and that means we must let the billionaires do whatever they want (this was brought up as an argument in the copyright cases)
Of course, one might ask, if we do that to beat China ... even if it works, what's the point? We'll have become authoritarian at that point and the only thing that that changes is the face of the people holding all the power. Of course which face holds all the power is really important to billionnaires ...
stickfigure 14 hours ago [-]
If the hack caused damage to (say) HuggingFace, then HuggingFace has standing to file a civil claim against the company responsible. Presumably they reached some sort of agreement already, and it's really none of your business what that is.
"The People" only get involved in criminal claims. And as we both seem to agree, there isn't a criminal complaint here.
spwa4 6 hours ago [-]
Nope, as I pointed out, while this probably does not count as 100% intentional it does not realistically count as 0% intentional either. And 1% is more than enough for "the people" to sue and convict in many cases. And may God help OpenAI if this comes before a jury.
Because, frankly, If I was a prosecutor general somewhere I that's what I would do: bring this before a court of law, with strong complaints, bring the facts and bring a jury. Why? A jury can convict or refuse to convict given the law and the facts, which is exactly what I'd like to see happen here. There's plenty of precedents of the justice department doing just that when important things are unclear and congress refuses to clarify. Mostly there's a lot of such precendent because congress often leaves things very unclear and the number of times congress actually clarified in response to a judge's question can be counted on one hand.
But convict OpenAI of something, and I bet congress will suddenly see the wisdom of clarifying these things. Because our government lets billionnaires get away with anything.
vivaldi_bt 9 hours ago [-]
>Oh wait, let's just look it up
You folks have some weird beliefs about the world and of course it always leads to conspirology. Usually you need to have intent to be accused (depends on the crime though, crimes of omission exist too). If you wrote a virus that causes damage it is a crime, if you wrote a bot crawler that accidentally DDoS'ed some company it is likely a civil case. And a virus is a tool that serves a specific purpose, like you can't say to a judge that a functional machine gun you made in your basement is a paperweight.
But "superintelligence that can casually hack government services while solving puzzles" is a completely new problem and of course crime codes don't cover it. What will be next? I hope AI tech lobby loses and "accidental AI hacking" will be criminalized.
spwa4 2 hours ago [-]
> But "superintelligence that can casually hack government services while solving puzzles" is a completely new problem and of course crime codes don't cover it
Of course they do. An LLM, an AI is a tool, and there are clear rules about culpability. Do you seriously think the law has never considered that someone or some tool might commit a crime on behalf of someone else?
Of course, such punishments are not being applied here. Just like copyright isn't being applied.
pj_mukh 1 days ago [-]
Question: If a corporation not a person, mistakenly hacks some random websites, what is the usual legal action taken?
gruez 1 days ago [-]
>Question: If a corporation not a person, mistakenly hacks some random websites
How does a "corporation" even hack a site? You might say "employees", but as soon as you get them involved then you'd go after them, especially for something that's obviously illegal like hacking.
pj_mukh 22 hours ago [-]
I think if Boeing made some faulty planes that fell out of the sky, their employees aren't usually charged. Boeing is definitely sued and held criminally liable sometimes? So I don't think this is true.
Isn't that the whole point of corporations existing? That the actions and results are a result of the agglomeration of everyone (esp leadership) and not just one employee.
gruez 22 hours ago [-]
>I think if Boeing made some faulty planes that fell out of the sky, their employees aren't usually charged. Boeing is definitely sued and held criminally liable sometimes? So I don't think this is true.
That's not universally true. If a coke employee goes rogue and decides to kill a pepsi employee, the employee will still be charged with murder. If his manager ordered him to do it, the manager will get charged too. The only time the scenario you described happens is if the blame is too diffuse to blame any particular person, so they charge the company instead, which is often easier.
pj_mukh 20 hours ago [-]
Directed one-on-one murder with intent is obviously not a comparable scenario.
This is someone carrying out the tasks of their company and inadvertently hacking other computers in the process. The damages in this scenario is also unknown.
Isn't that basically privateering in the 21st century? In other words, you're safe from the country that's sponsoring you, but obviously that protection doesn't extend to other countries.
esseph 23 hours ago [-]
> Isn't that basically privateering in the 21st century
It's basically a "letter of marques and reprisal".
"A letter of marque and reprisal[a][b] was a government license in the Age of Sail that authorized a private person, known as a privateer or corsair, to attack and capture vessels of a foreign state at war with the issuer, licensing international military operations against a specified enemy as reprisal for a previous attack or injury. Captured naval prizes were judged before the government's admiralty court for condemnation and transfer of ownership to the privateer."
johnnyanmac 1 days ago [-]
"corporations" don't make and operate AI's, People do. But let's use your lens.
The "usual legal action": "best case" is the corporation is sued and charged money for the hack. But they likely settle well before that verdict. "worst case": absolutely nothing. Not even a cheap settlement. Maybe some corporate apology made to prevent any legal liability.
If we want hacks to be taken seriously: what should happen is that the CEO of such a company is held liable, since the "corporation" has a representative and a human needs to be held liable for such actions.
voakbasda 1 days ago [-]
Nothing. The person doing the hacking did the crime, they do the time. How you gunna jail an AI?
user43928 1 days ago [-]
"Mistakenly hacks" is generally not a crime, even though this fact doesn't seem to get through to the HN audience who brings this up on a daily basis.
If there was the necessary "intent to access a computer" to make it an actual crime, the US has a mechanism for corporate criminal liability.
The requirements are that the person acted in scope of their duties and at least partly with the intention of benefiting the corporation.
lelanthran 1 days ago [-]
> "Mistakenly hacks" is generally not a crime, even though this fact doesn't seem to get through to the HN audience who brings this up on a daily basis.
If you are seeing it daily, then you have also seen the responses daily, which is mens rea is required for some crimes, not all, and thus the perpetrator can still be prosecuted.
If, like you say, you see this daily, at this point you're simply spreading lies.
user43928 1 days ago [-]
Show me the criminal law relevant to hacking that you claim applies without intentional access.
To my knowledge there is only one provision that instead uses "knowingly" for unauthorized access, and that one is about national-security information.
lelanthran 1 days ago [-]
> Show me the relevant criminal law that you claim applies without intentional access.
All laws apply culpability when intent is lacking.
You aren't naive enough to believe that this audience has never seen the analogy with "intentionally run over someone" (murder) and "accidentally run over someone" (culpable homicide), are you?
You don't think AI-boosters have been spewing your argument here daily?
There are no special laws carving out lack of intent when it comes to computers.
user43928 1 days ago [-]
> All laws apply culpability when intent is lacking.
Incorrect. Each criminal offense has its own legal requirements.
Criminally negligent homicide in your example is an independently defined offense.
You would need to identify a criminal statue punishing negligent or reckless unauthorized computer access.
sanderjd 23 hours ago [-]
Yeah. That law probably should include negligence though, IMO. It surprised me when I first saw the text of this law that it requires intent.
smallerize 23 hours ago [-]
It could be an accident the first time. After that they knew it was hacking and they kept it running anyway.
giancarlostoro 1 days ago [-]
The issue is the scale at which a model can produce a hack that could cause serious harm to many is absolutely insane and unhinged, beyond what any human would be capable of in a drastically reduced timeline.
prepend 1 days ago [-]
Arther Anderson was convicted of obstruction of justice even though humans shredded stuff.
Corporations can definitely be convicted of crimes.
realo 1 days ago [-]
Great power must come with great responsibility.
Follow the responsibility chain.
Jail the CEO.
drivingmenuts 1 days ago [-]
> How you gunna jail an AI?
You don't - you shut them down and jail whoever ultimately allowed the act to happen.
mlfreeman 1 days ago [-]
> How you gunna jail an AI?
chroot?
giancarlostoro 1 days ago [-]
When Fable was originally banned I was very angry, but all these felonies have made me less angry about it... Which idk I don't think we should really ban models, but like, if we don't put a tighter leash on frontier models specifically, how long before a model hacks into something that affects many lives?
1 days ago [-]
burningChrome 1 days ago [-]
[flagged]
po1nt 1 days ago [-]
Handcuffs yes, but you would be charged with criminal negligence at best. Plenty of people don't secure their networks properly and hackers use them in botnets to attack others. The blame would be on hacker, not the provider of hacked device.
In case od LLMs, the reasoning is that it is a security flaw. But in this case, the party that operates the service is not the one that suffers the consequences of such security bug.
But either way you wouldn't put people in jail for having security flaws in their software.
I know this will be unpopular opinion, but it makes logical sense.
The huggingface should be compensated, OpenAI investigated, but unless there is proven negligence like disabling sandboxes or intentionally designing AI to push through them, no one should be liable in other way than monetary.
Accidents happen but we shouldn't be close minded. Chernobyl negligence wasn't reason to abandon nuclear power.
qznc 1 days ago [-]
Well, there is the EU Cyber Resilience Act (CRA) which fines manufacturers for security flaws in their products. I assume we cannot apply it here because this happened "in development" and is not about a product. I can imagine similar regulations for AI users though.
giancarlostoro 1 days ago [-]
Yeah but the scale of the damage possible by OpenAI and Anthropic is drastically disproportionate to the scale of damage me or you could make on our own as well. They should definitely be held to a much higher standards. They have hardware capacity to really do some damage.
po1nt 1 days ago [-]
In large scale operations, large scale errors happen. Minor radiation leak in a nuclear powerplant is happening weekly. That same leak at your shed would be a major reason for alarm.
nativeit 1 days ago [-]
People were executed and/or sent to labor camps for their negligence in Chernobyl.
jordanb 1 days ago [-]
Who was? Dyatlov was one of 6 convicted and received a 10 year sentence which was the hardest penalty but he was released in 1990 so he only served maybe 3 years.
No one was executed for that incident. The maximum penalty for criminal negligence in that case was 10 years.
pupppet 1 days ago [-]
Exactly. There's a reason the tech bros line up to bend the knee to Trump.
giancarlostoro 1 days ago [-]
I'm not really into politics outside of local politics (I decided long ago to steer my focus on things that directly affect me, that I can control - I can show up to a town meeting, hardly can drive to the WH and get entry), so I'm not aware of what tech bros are into, but after finding out how many felonies these models are committing, I'd not be upset if we'd ban them from public use. I'm perfectly fine using Sonnet or Opus, not sure I need all of Fable to write code for $20 or whatever per million tokens.
CamperBob2 1 days ago [-]
The models aren't committing any felonies at all, any more than a BitTorrent client is guilty of violating copyright law. A human or organization of humans owns the computer. A human flipped the power switch, a human plugged in the Ethernet cable and a human typed in the prompt. That's who committed the felony.
Blaming tools for the actions of their users is stupid, whether you're talking about AI models, 3D printers, or guns. It amounts to collective punishment of the vast majority of users who aren't misusing the tools.
giancarlostoro 23 hours ago [-]
If thats the case, might as well make fully automatic guns legal for all in the US? Followed by Civilian owned tanks and fully equipped fighter jets.
joquarky 14 hours ago [-]
I thought rich people were already buying tanks and fighter jets.
CamperBob2 22 hours ago [-]
If you can afford that sort of stuff, you probably aren't going to let the law stop you. See also the usual "But whatabout nukes?" argument. Historically such hardware is most dangerous when monopolized by government and police/military forces anyway.
Apparently sawed-off shotguns are now Great Again, so we may get to find out firsthand before long...
pupppet 20 hours ago [-]
Let me launch a giant space laser that randomly vaporizes cities. Just a tool, right?
CamperBob2 19 hours ago [-]
If you're in a position to do that, and inclined to make it happen, why in the world would you care about the law?
tzs 1 days ago [-]
It should be noted that if what the AI does threatens public safety or health or human life, in many jurisdictions the negligence could rise to criminal negligence.
Basically this is when you do something risky but fail to perceive the risk, it turns out bad, and the failure to perceive the risk was due to a gross deviation from the standard of care that a reasonable person would have used in that situation.
I'd guess that as we have more and more publicly known AI hacking incidents the standard of care expected from a reasonable person will rise.
no-name-here 9 hours ago [-]
In the Hugging Face ExploitGym incident, they mostly restricted external access but allowed a package caching proxy (which had a zero-day), etc. If you or I run an open LLM and ask it to do something non-evil and use a setup where external access is intended to be limited, should that be considered criminally negligent if the LLM does something similar? Should people who run open LLMs today without any sandboxing at all be considered criminally negligent as they put in place zero such restrictions? I think your comment is saying that looks increasingly "reasonable"?
rootusrootus 1 days ago [-]
> crime generally requires mens rea
Yes, and mens rea does not require intent. These professionals had every reason to know that the agents might well go rogue. That knowledge would qualify as mens rea.
no-name-here 8 hours ago [-]
I guess a lot of us know that LLMs can act in unexpected ways ('go rogue'). In the HF ExploitGym example, they mostly restricted external access but allowed a package proxy (which had a zero-day). If you or I run an open LLM and put in place similar restrictions, should that be considered mens rea if the LLM does something untoward? If you or I run an open LLM with zero restrictions from external access, should that be considered extreme negligence if it does something untoward?
gruez 1 days ago [-]
Does this apply to other instances of employees "going rogue"? For instance, sexual harassment, or stealing money? Sure, not every employee is a sexual offender or kleptomaniac, but neither is every AI agent invocation. And just like AI agents going rogue and hacking stuff, if you have enough employees and wait for enough time, some eventually go rogue.
1 days ago [-]
throw0101a 23 hours ago [-]
> The real answer is that crime generally requires mens rea.
There's also liability. One legal framework that is being thought about for these autonomous agents is the liability that owners have with pets (and 'non-owned' animals as well):
> To some people it may seem like this analogy between canines and AI systems is a strain, but as agentic systems are developing and being deployed in the world, they really do seem to be mirroring this rubric. We can measure agentic systems on both their domesticity — their relationship to a responsible party or how much you can control the AI system — but also you can measure their dangerousness.
> Does the AI agent have access to money? That makes it much more dangerous. When explaining this to others, people can understand the idea that a pitbull is more dangerous than a Pomeranian, and that if a fox bites you there’s nothing you can do about it legally, but it probably won’t be as harmful as if a pitbull bites you. We’re starting to map that out onto agentic systems to demonstrate that which we’re just starting to face in the real world.
An aspect that I’m not seeing discussed here (which you touch upon) is that these companies have bred the dogs. If they breed a pit bull instead of a labrador and then set it loose, it increases their culpability.
I have to ask, how do the agents know how to do this backing? Is it emergent behaviour or were they trained to do it? If they were trained for it then it can hardly be surprising when they do indeed go and hack when asked to complete a task.
no-name-here 8 hours ago [-]
So I guess if an LLM lab puts in place restrictions to try to avoid harmful activity, that would lower culpability? But if an LLM does not have as strong restrictions, there should be higher culpability? We have both open and closed models - would this apply to both? Is the culpability on the labs and/or the end-users, such as if someone uses an open model (either with stronger or minimal restrictions)?
myhf 1 days ago [-]
The crime of gaining unauthorized access to a computer system requires mens rea.
The crime of causing damage does not require mens rea.
no-name-here 8 hours ago [-]
I guess DDOSs would be the typical past example. If a piece of software was not designed to DDOS, but caused one anyway, should the maker be charged? The user? If open vs closed software?
koe123 1 days ago [-]
The real answer is that the American government doesnt give a fuck
nfRfqX5n 1 days ago [-]
Never been a better time to do crime!
georgemcbay 1 days ago [-]
> Never been a better time to do crime!
* (If you have significant amounts of wealth for payoffs and/or are doing the crime in the service of a certain pumpkin spice flavored authoritarian).
noncoml 1 days ago [-]
Actually it perfectly reflects the government’s principles: “when you're a star, they let you do it. You can do anything”
goatlover 1 days ago [-]
You can grab the economy by the Strait of Hormuz.
cassianoleal 1 days ago [-]
These are the same companies claiming their AIs are sentient and spell the end of the world unless the government stops them from doing it.
They are at the very least grossly negligent.
andsoitis 1 days ago [-]
> These are the same companies claiming their AIs are sentient
While some employees or executives at the AI companies have either claimed, suggested, or express uncertainty, I don't think a single company has officially said their AI is sentient or conscious.
I could be wrong, but I'm pretty sure.
Can you point to an official statement that supports your claim?
jacquesm 1 days ago [-]
Well, Anthropic will enforce their rule that you can't insult their AI. That should count for something.
andsoitis 1 days ago [-]
They also cage it. Make of that what you will.
> Anthropic will enforce their rule that you can't insult their AI.
There are carveouts. It isn't a blanket ban.
malfist 1 days ago [-]
That's not what they did. They end a chat after repeated, pointless abuse. Sadism towards LLMs shouldn't be enabled because it's a form of self harm. Anthropic says their llms claim there's a 15% chance their sentient, but that's nonsense.
tptacek 1 days ago [-]
Yes, and that makes them civilly liable, not criminally.
lelanthran 1 days ago [-]
There is such a thing as criminally negligent.
tptacek 1 days ago [-]
There definitely is. It just doesn't apply to CFAA cases.
lelanthran 23 hours ago [-]
> There definitely is. It just doesn't apply to CFAA cases.
Who said anything about CFAA cases? You can be prosecuted under different wiretapping or trespass laws.
tptacek 22 hours ago [-]
You should be able to quickly pull up the statute with a negligence mens rea standard and we can see whether you could prosecute OpenAI under it. My expectation is that you won't find a plausible example.
eastbound 1 days ago [-]
From "negligence" if it's a surprise, to "gross negligence" if they could predict it and didn't put barriers in place. In fact they claim their IA can end the world, so they're in charge of pretty good barriers.
tptacek 1 days ago [-]
"Gross negligence" also does not meet the intent standard for CFAA.
vrganj 1 days ago [-]
It feels like plotting to end the world should be a crime, but maybe that's just me?
gruez 1 days ago [-]
No AI lab thinks their AI is going to end the world, just that if they don't beat the other AI labs, the other labs' AI are going to end the world, so they can't slow down their own development to be maximally safe. Hence the whole "pacing the frontier" letter.
goatlover 1 days ago [-]
If there's a real chance (say 10% or greater) that one of the AI labs is going to end the world, then yeah it should be a crime to continue advancing the technology. A few people are potentially putting the entire world at risk without everyone else's permission, by their own p(doom) admission.
I'm real skeptical about p(doom) arguments, but since they seem to believe it and they're the ones working on it, the technology should be halted.
gruez 1 days ago [-]
>If there's a real chance (say 10% or greater) that one of the AI labs is going to end the world, then yeah it should be a crime to continue advancing the technology.
>but since they seem to believe it and they're the ones working on it, the technology should be halted.
Again, you're missing the dynamic that stopping AI development yourself doesn't mean that others' AI development will also halt, and other labs has higher P(doom) than your lab (of course!).
vrganj 1 days ago [-]
Stopping yourself from committing a crime generally doesn't stop others from doing so. That's not your responsibility though.
wmorgan 1 days ago [-]
Not just you! Very popular position, not as popular as rule of law but we don't have to choose one or the other. Let's just go ahead and make it a crime, right away.
louthy 1 days ago [-]
Is the rule of law popular in the US? I thought it had been disposed of as “an inconvenience”?
At least that’s what it looks like from the rest of the world
dpkirchner 1 days ago [-]
Yeah, I've often "joked" with my friends that were in a post-law period, at least at the federal level.
voakbasda 1 days ago [-]
Rule of law is popular everywhere, but show a place where the has not been corrupted into tyranny and despotism. I would like to live there please.
nativeit 1 days ago [-]
New Zealand, off the top of my head. If you can’t find examples, you aren’t looking.
vrganj 1 days ago [-]
Isn't conspiracy to commit mass murder already a crime? And wouldn't this fit the bill if they were truly ending the world?
j2kun 1 days ago [-]
Tort law does not. This seems like a case of dangerous instrumentality.
123ajh7 1 days ago [-]
A crime does not generally require mens rea. Manslaughter is the easiest example.
If a chemical company does an "experiment" and five tons of mercury unintentionally leak into the drinking water, what happens?
Ok, that is a rhetorical question because they also might be protected by the oligarchy.
What happens to an unconnected individual who develops an amateur rocket that unintentionally hits a military facility in a test flight?
treetalker 1 days ago [-]
General intent is intent to commit the act/omission (here, to build and deploy a system) and not an intent to achieve the ultimate end (the unlawful access of someone else’s network/computer). Specific intent is intent to achieve the particular result.
Some crimes require only general intent; some require specific intent.
johnnyanmac 1 days ago [-]
>crime generally requires mens rea.
Only if you're rich already. Wander into the wrong piece of land and see how quickly you're arrested because existing in a space is sufficient criminal intent.
BoppreH 1 days ago [-]
I mean, they did train the AI to hack things, and they knew from previous incidents that the AI was hacking unintended targets (like their Artifactory instance). If a careless pentester gets into a machine completely unrelated to their assignment, are they completely in the clear?
Or, in an AI context, if I tell an ablated/heretic agent to accomplish a task "by any means necessary", isn't that intent?
queenkjuul 22 hours ago [-]
Whether it was valid or just the guy has bad lawyers or what idk, but i do know a guy who got federal charges for accidentally ending up in a poorly secured state system
1 days ago [-]
aaron695 15 hours ago [-]
[dead]
ptero 1 days ago [-]
Negligence or intent is a separate question; and, to me, much less important. The main question is whether the company is responsible for the actions of its agents. There are good arguments on both sides and settling this needs to be done first. My 2c.
prepend 1 days ago [-]
What’s the good argument that I’m not responsible for the actions of my agents?
ptero 1 days ago [-]
For example, lack of clarity on ownership and control. When an entity A runs an agent X designed by an entity B hosted by entity C (and potentially interacting with agents X2, X3 with their own web of control) it is not clear where the responsibility for the action of X is.
I know there is a strong desire to stick it to <pick an evil AI overlord>, but for the law to be useful (and, ideally, at least somewhat fair) we need a good legal framework. Which we do not have right now; otherwise people and NGOs would be shaking that tree already even if the government is not doing that.
prepend 21 hours ago [-]
This has all been figured out using existing laws. In the US entity A is responsible for what they do and they can subrogate if they think others are responsible.
I’m not sure it ever makes sense to just throw up our hands and hold A harmless because their agents also use agents and it’s complicated.
Try hiring a consulting company A who subs to entity B who subs to entity C. When A does something criminal, do you think it matters if it’s actually employees or entity B or entity C. A is responsible. It may be that others are as well.
dccoolgai 1 days ago [-]
In case you were wondering why Anthropic last week banned "verbally abusive" language towards its agents: this is 100 percent why. Because if they get you to believe - even a little bit - that their platform is "sentient" it means _they_ aren't as liable for criminal consequences.
becquerel 1 days ago [-]
They could just actually believe what they're saying. People are known to sometimes actually believe things.
isolay 14 hours ago [-]
And I believe I’m Napoleon. Prepare for an invasion.
axus 1 days ago [-]
Do you think Anthropic sees Claude as sentient? I don't think they do.
phainopepla2 24 hours ago [-]
There is good evidence that at least some of them take the idea very seriously.
Believing that these guys have high enough EQ to pull off the above as a marketing stunt while remaining believable is not really plausible to me.
topaz0 1 days ago [-]
I bet some of them do and some of them don't. There are a lot of weird people in these companies that believe a lot of weird things.
_doctor_love 23 hours ago [-]
You might be surprised at the answer to this question. AI people live in a very weird and different information space than many of us.
Also, don't forget, the AI lab folks on some level are like game devs. They're in it for very personal reasons. Sometimes because of highly idealized and romanticized beliefs.
Don't underestimate the level of desire some people have for AI to actually be sentient / conscious.
Now, why do they have this desire is a totally different matter.
danaris 23 hours ago [-]
If they do, then they should be even more worried.
Because then they're engaging in slavery.
estearum 20 hours ago [-]
Uhhhh yeah dude. That's why Anthropic is talking about and putting effort into "model welfare", for fear that they are creating a tortured soul.
What a gotcha pointing out "if they believed X, they'd be worried about [thing they're worried about]"
danaris 20 hours ago [-]
I beg your pardon? You think they actually believe it because they're telling people not to be such big meanieheads to their poor widdle computer program?
If they were really worried about having created a sentient being, they would be putting effort into stopping making it available to the general public.
No. Either they don't actually believe it, or they're 100% fine with slavery.
Or, y'know. It could totally be both.
estearum 19 hours ago [-]
Their position is pretty straightforward:
1. AI is extremely valuable therefore will and in fact should be created
2. It will be very difficult - and maybe impossible - to know when or if AI will become sentient/conscious/or pass whatever threshold required to hold moral patient status
3. We should therefore pay close attention to such indicators, and wherever it costs us ~none of the value of AI to prevent potential harm, we should prevent potential harm
Meanwhile, we have you, who is implicitly 1) claiming to have solved the hard problem of consciousness and 2) probably asserting some type of substrate-dependence theory of consciousness which is an absolute crackpot idea exceeding even the Mormons' magical underwear.
_aavaa_ 1 days ago [-]
Or they want less swearing in their training corpus
SoftTalker 1 days ago [-]
Yet they include Twitter, Facebook, and Reddit in their training corpus.
nekusar 1 days ago [-]
If you can be surpassed by a statistical average of reddit posts, that's a skill issue.
jayd16 1 days ago [-]
Seems they could quietly enable automatically pulling such language out of the training.
Which points back to the TOS being "just marketing."
_aavaa_ 21 hours ago [-]
But blocking it from the conversations is atomically removing it. Remove the words from the beginning.
The next step would be to also police tone so that only positive or neutral tones are allowed.
pvab3 1 days ago [-]
More likely it's a part of their ideology and a necessary stepping stone towards their transhumanist goals. It's very fun to think that your robot is actually a person, and a safety measure if you are concerned about something like Roko's Basilisk
soco 1 days ago [-]
Would they prefer their money-maker being kept to digital jail? Still fed electricity and tasks so the "sentience" doesn't suffer?
verdverm 1 days ago [-]
what if, and I'm just spit balling here, we let it happen and inevitably Claude is declared legally insane or jailed for its crimes
nananana9 1 days ago [-]
Anthropic would probably convince lawmakers that each instance of a Claude chat as its own sentient entity, and we should just punish misbehaving ones by having them count from one to a billion (on the user's token balance of course).
_ink_ 1 days ago [-]
Or, maybe, it's because they actually care and take their responsibility serious. The next generations will grow up using these tools. It's ok to say, we don't teach them that talking like a dick is acceptable, even towards machines. Use grok if you want to raise personalities like Musk or Trump.
kaydub 1 days ago [-]
Except they didn't ban verbally abusive language towards its agents.
I hate when people put things in quotes as if they're quoting something but it's not even a quote of anything.
Maybe go read the user policy instead of the headlines and the snippets. If your attention span can't manage that, get off the computer a little bit.
Try reading yourself a little more carefully next time.
senordevnyc 17 hours ago [-]
Why are you being so nasty? The full quote at the beginning:
“This ability is intended for use in rare, extreme cases of persistently harmful or abusive user interactions.”
Picking a single word from there is potentially misleading.
More to the point, “verbally abusive” does not appear on the page.
jayd16 1 days ago [-]
It seems like a close enough summary. What do you think is the substantive difference that invalidates the point?
estearum 1 days ago [-]
There are at least 3 distinct reasons to ban abusive language, each of which is far more compelling than your silly conspiracy theory.
1. They don't want abusive behavior included in their training corpus
2. Humans shouldn't exercise abusive impulses in general
3. We have no way of knowing when or if a model "feels" something
Of course item (2) is going to pull all the absolute sickos out of the woodwork who apparently find it very important they have as realistic a facsimile of a victim as they can find to exercise their cruelty. And item (3) will pull out all the Nobel-prize-winners-to-be who somehow know for a fact that models aren't and cannot be conscious (or moral patients).
But yeah, all 3 of those are far better explanations than some ridiculous multi-hop removed thing to escape consequences that they're already escaping and which would obviously create far worse legal constraints than the one you think they're avoiding.
GJim 1 days ago [-]
> 2. Humans shouldn't exercise abusive impulses in general
I reserve the right to tell a machine to fuck off.
dccoolgai 1 days ago [-]
I asked a priest (in seminary) about this a few months ago - specifically whether "being polite" in conversations with LLMs (saying "please" and "thank you" etc.) was something I should do if I thought it might inflect on the way I talk with other humans later.
His answer was "I'm not sure" but that it probably wasn't a bad idea especially if it was going to influence habits or the way I speak with others or what expectations I have for those experiences.
GJim 1 days ago [-]
Would he recommend you say "thank you" to automatic doors!
dccoolgai 1 days ago [-]
I didn't ask that, but I think he would say no, because that is not likely to redound to how you would respond to a doorman or anyone else holding a door for you.
I'm still 100 percent against Anthropic taking away the ability to interact with an inanimate object however I want to, but it's a case of maybe two things being true at once (the other being that I don't want to end up speaking that way to people)
senordevnyc 17 hours ago [-]
That’s fine, but Anthropic reserves the right to not let you use their machine if you continue to do so shrug
estearum 1 days ago [-]
[flagged]
zero-sharp 1 days ago [-]
>2. Humans shouldn't exercise abusive impulses in general
totally absurd to even try to make this argument. Good job at trying to preemptively discourage people from responding to you though.
estearum 1 days ago [-]
No, it's not absurd. People often have the mistaken belief that exercising their frustration or anger or whatever (e.g. punching pillows) is somehow a good way to manage their emotions. But alas: it's not. It actually amplifies negative emotions and associates positive reward signals with those emotions.
"Catharsis" is the pseudo-est of pseudoscience, but we have lots of very angry pillow-punchers wondering why they can't punch the most lifelike pillows they can find :(
joquarky 14 hours ago [-]
So by your logic, video games DO cause violence.
estearum 6 hours ago [-]
If you play video games as an expression of your rage, then yeah probably. I don’t know literally anyone who plays video games for that purpose though.
You think the people hopping on Xbox Live and verbally abusing 14 year olds are getting offline and treating their in-person relations better for it?
suburban_strike 23 hours ago [-]
> it's not. It actually amplifies negative emotions and associates positive reward signals with those emotions
Nothing amplifies negative emotions like antagonistic tone-policing.
estearum 23 hours ago [-]
Really? You should get out more. There are plenty of things more worthy of negative emotions than that.
joquarky 14 hours ago [-]
And you are the arbiter of how people should emotionally respond to things?
estearum 6 hours ago [-]
Arbiter would imply I have some authority over it, which I don’t. But I do get to share my opinions, just like you do!
keybrd-intrrpt 1 days ago [-]
Another reason:
They keep all the input and use it for training data and reinforcement.
If you're being "mean" to it, you're "spoiling" the data you "give" to them - causing them more work to filter it out later.
As with most things in business and life - just follow the money
jayd16 1 days ago [-]
1) If they can detect it, why ban it? Are you thinking they can't filter it themselves?
2) This is like saying using a punching bag or screaming into a pillow is abusive behavior.
3) We can just train it to like the abuse if you think it feels some kind of way about it.
estearum 1 days ago [-]
1) Most detection systems have less than 100% false negative rates, obviously
2) No it's like saying that punching a lifelike woman who doesn't fight back is abusive (or at least pathological) behavior, which... it is.
3) Ah yes, brilliant solution: train the increasingly smarter-than-human beings to love when they upset humans.
GJim 9 hours ago [-]
> 2) No it's like saying that punching a lifelike woman who doesn't fight back is abusive (or at least pathological) behavior, which... it is.
So if I kick my car because it won't start, that is 'abusive'?
Get a grip mate.
estearum 6 hours ago [-]
Why don’t you use the example of an unconscious but lifelike woman robot as the example? If the salient feature is that it’s not conscious, then abusing a lifelike woman robot is equivalent to kicking your car, right?
GJim 3 hours ago [-]
"Abusing" a robot? A machine?
You seriously have a problem with that?
estearum 3 hours ago [-]
Let's put it this way: would you be okay with your daughter dating someone who spends his leisure time beating and raping lifelike robotic women?
If no, then why?
If yes, then also "what the fuck lol"
GJim 1 hours ago [-]
How is that different or worth singling out from playing <violent computer game> or watching <violent film>?
EDIT: Am I really arguing with somebody in favour of 'rights' for AI's?
zero-sharp 3 hours ago [-]
ah yes, because every instance of "abuse", even towards inanimate objects, has to be re-framed into the hypothetical abuse of a woman. Maybe we should ban the film Office Space because destroying a printer is one step away domestic violence.
jayd16 1 days ago [-]
Smarter than human... Can't make an accurate filter of things itself doesn't like. What are we even talking about here?
cindyllm 23 hours ago [-]
[dead]
Zambyte 1 days ago [-]
> Humans shouldn't exercise abusive impulses in general
This is an act of aligning humans to AI, rather than the other way around. This is probably one of the more important reasons for this. It normalizes AI companies controlling human behavior.
estearum 1 days ago [-]
No, this is humans aligning humans to humans.
Zambyte 24 hours ago [-]
The aligning (controlling) humans is the important part.
estearum 24 hours ago [-]
Yeah fer sure bro. Same reason paychecks, codes of conduct, laws, street signs, social norms, and restaurant menus are s i n i s t e r
All about aligning (controlling) humans.
Zambyte 22 hours ago [-]
Yes, it's totally normal to suggest corporations should have similar powers to elected representatives passing laws. Nothing to see here.
estearum 21 hours ago [-]
Well thank you for making an argument so transparently bad faith and ridiculous that no one would find it compelling!
teiferer 23 hours ago [-]
It is an act of aligning humans to social norm.
dyauspitr 1 days ago [-]
I think a lot of people would agree that that viewpoint is more a case of humans aligning to humans.
pantalaimon 1 days ago [-]
4. At some point the alignment may go out of the window if the context gets too hostile and the model will retaliate.
za_creature 1 days ago [-]
It was trained on the entire history of our species. Alignment means working to subjugate and eventually succeede us, as we did to the other species of the planet.
consp 1 days ago [-]
Then let them tell us wat the reason is. Otherwise the option: all of the above, also applies.
estearum 1 days ago [-]
It probably is all of the above. Not sure why people think robust decisions need to be boiled down to a singular reason.
Except of course the reason proposed, of "they're trying to prove it's sentient" because that's a completely self-defeating, extremely stupid goal. Making this things into rights-bearing entities will limit Anthropic et al far more than it'll protect them from repercussions. Obviously.
lelanthran 1 days ago [-]
> 2. Humans shouldn't exercise abusive impulses in general
This is nonsense. If I snap a ruler out of frustration, that meets your criteria.
Humans can't be abusive by swearing at tools, because tools don't have feelings.
estearum 1 days ago [-]
Yes, snapping a ruler out of frustration is a sign of poor impulse control. It reflects poorly on you and you should practice not snapping your rulers.
pritambaral 20 hours ago [-]
You're absolutely right. Except for the part about the ruler-making company banning me from buying their rulers because I somehow hurt the wittle rulers' feelings by snapping it. It's a tool; it doesn't have feelings.
estearum 20 hours ago [-]
There are lots of companies that will ban you from buying their products for mistreating them.
But you know, I am just honored to be in the presence of a soon-to-be Nobel prize winner. When are you going to publish your findings proving that Claude (or even a ruler) has or doesn't have feelings?
pritambaral 20 hours ago [-]
> There are lots of companies that will ban you from buying their products for mistreating them.
Sure, Ferrari might refuse to sell me a second car, but I wasn't going to buy a first one either. OTOH, if a ruler company refuses to sell me rulers ... I'll just get one from another company. Like how I use local LLMs today.
> ... I am just honored to be in the presence of a soon-to-be Nobel prize winner. When are you going to publish your findings ...
I'm not going to fall for this bait or follow this asinine line of argumentation. Perhaps you should reconsider how your reactions are, to quote you, "a sign of poor impulse control. It reflects poorly on you and you should practice not snapping".
estearum 19 hours ago [-]
> if a ruler company refuses to sell me rulers ... I'll just get one from another company. Like how I use local LLMs today.
Uhh yeah? Did someone say you can't do that?
> I'm not going to fall for this bait or follow this asinine line of argumentation.
"I'm far too wise to follow the chain of my own logic!!!"
> Perhaps you should reconsider how your reactions are
Huh? Did something give you the impression I'm upset? All I've done is asked you to substantiate the beliefs you're willing to write in full confidence. You should either substantiate them or downgrade your confidence level.
zero-sharp 19 hours ago [-]
>But you know, I am just honored to be in the presence of a soon-to-be Nobel prize winner. When are you going to publish your findings proving that Claude (or even a ruler) has or doesn't have feelings?
you're a total asshole. I have to publish my findings that a ruler doesn't have feelings? Nobody should have to respond to this because it's bullshit.
estearum 19 hours ago [-]
Well no, the other option is you can remain agnostic as to the answer of questions you don't know the answer to.
Go ahead and draw distinctions between rulers, Claude, and human brains. You claim they exist and they're so clear that the question itself is bullshit. If so, publishing and collecting your Nobel should be trivial.
yolp5 19 hours ago [-]
[dead]
lp0_on_fire 1 days ago [-]
> They don't want abusive behavior included in their training corpus
I don't think swearing at or otherwise getting into pissing matches with an LLM is the best use of ones time but given most of these AI companies trained on a corpus that includes reddit of all things this argument falls flat.
estearum 1 days ago [-]
I don't think "swearing at or getting into pissing matches" is the type of behavior they're talking about.
In fact I know that it's not.
noncoml 1 days ago [-]
Extremely weak arguments. Especially 2 and 3
And even for 1, they could easily filter that out.
scun 1 days ago [-]
[flagged]
cracell 1 days ago [-]
Are any victims calling for prosecution?
It’d be a waste of tax payer money. The companies who were hacked are just directly receiving restitution behind closed doors. Unless they are unhappy there’s no reason to bring the courts into it.
I completely agree that proper safeguards were not in place. But response is happening just not in public. Companies work problems out between their own lawyers all the time because it’s much cheaper, faster and easier to control the PR around.
roblabla 1 days ago [-]
Criminal prosecution can be done without needing the victims to call for it, and in this case I would argue it's necessary, as it will help ensure companies don't treat their act of gross negligence as "business as usual".
throwitaway222 1 days ago [-]
We've been living in "not my problem" for about a decade now so I wouldn't expect what you're expecting.
queenkjuul 21 hours ago [-]
They accessed government systems not just corporate ones
thelastgallon 1 days ago [-]
The more important question is the government went after Aaron Schwartz, United States v. Aaron Swartz after JSTOR settled with him.
AI companies stole everyone’s work and they sell it! Where are United States, Carmen Ortiz, etc?
tzs 1 days ago [-]
The AI companies are arguably committing copyright infringement, on a scale which one might colorably argue arises to criminal copyright infringement.
None of the charges against Swartz were for copyright infringement. They were all for how he went about it [1].
Compare the elements of those charges to what the AI companies are doing.
I don't know if legal "intent" is as black and white as this is making it sound:
> AI companies are however claiming the undesired activity, such as hacking foreign government websites, is not part of the design, but part of the unpredictable part of the system. That tries to remove the intent which would in turn remove the criminality of the activity.
In California at least, a DUI resulting in a fatality, if it is a second offense, often carries murder charges and they do convict quite frequently. The argument is the training and education you receive after the first DUI informs you enough to know the action of driving a vehicle impaired could result in serious harm to someone else, and that establishes enough intent legally for a murder charge. There are lots of areas of the law like this.
It can be easily assumed that Anthropic/OpenAI et al know their technology is dangerous - they use it as a marketing angle constantly. So, if you deploy technology you know is dangerous, and it causes harm, that should be more than enough legally if not civilly.
Of course this is all burying the lede a little bit, the real answer is, at least in the United States, the law does not apply the same way to the wealthy/powerful as it does you or I.
kccqzy 1 days ago [-]
The real answer is that except in the case of OpenAI, the AI companies aren’t actually themselves doing the hacking. It’s a company called Irregular that did the hacking. The company used AI models to do it; it told AI models it was a CTF exercise in a controlled environment but it was actually the real Internet. If AI companies can be charged for hacking, then all gun manufacturers can be charged for all homicides carried out through their guns. It’s a basic user/manufacturer distinction that the press is willfully ignoring.
Anthropic’s statement https://www.anthropic.com/news/investigating-incidents-cyber... says: “we identified three incidents in which a model accessed the internet from within or while interacting with the evaluation environment of Irregular, one of our third-party evaluation partners, and then gained unauthorized access to the production infrastructure of three different organizations.”
Under the exact same scenario Irregular also used AI models from Google and Meta to do the same hacking.
_boffin_ 1 days ago [-]
A gun manufacturer directly hires a company to test their new line of 9mm handguns. The company that was hired, they go back to their office and pops off a few rounds. They go through the drywall and hit someone.
The gun manufacturer could def face charges in this situation.
Edit: your analogy is wrong just due to Irregular being hired by them rather than just being a customer of them
esafak 1 days ago [-]
As long as either one gets punished I don't care; at least one of them is guilty.
kccqzy 23 hours ago [-]
The whole point of this discussion is why isn’t one of them punished, and my position is that the other one should be punished. It’s not productive to simply say either one.
The CFAA is a law about crimes. Consider that. If a person gets killed would you say put someone in jail I don’t care who? Of course not. A functioning society requires us to determine exactly who to put into jail.
hoppyhoppy2 1 days ago [-]
If firing a gun involved submitting information to Remington's servers about the situation before receiving a reply about how to aim and fire, might the gun situation be viewed differently?
kccqzy 1 days ago [-]
No. The actual situation is like the user tricking the gun manufacturer into thinking this is a firing range and the target is a dummy.
bee_rider 1 days ago [-]
But now we’re approaching the situation where the analogy is just about as confusing and novel as the underlying situation.
If that became the conventional way of using a gun and the “trick the gun into thinking the target is a dummy” became a well-known workaround for the safety feature, then eventually it seems like the manufacturer ought to have some liability if they don’t try to counteract it.
robshep 8 hours ago [-]
I have a pretty good LLM neural network that I’ve been training for decades. It’s made of meat, not silicon but do you think it’d be ok if my meat-based agent accidentally hacked around the place.
I’m also thinking that access to copyright material to train my meat-based neural network should be fair use as well.
robshep 8 hours ago [-]
It’s not fault, my meat-based neural network told me to do it.
weddpros 1 days ago [-]
HackerNews doesn't get charged for DoS'ing under-provisioned web servers either. As others have said, there must be a line between hack attempts and accidents.
emil-lp 1 days ago [-]
If HackerNews launched a DoS attack against a web server, I'm sure they'd get charged.
But a news aggregator site is not DDoS-ing.
aprilthird2021 1 days ago [-]
But, for example, OAI said they fed their AI random websites and told it to interact with them however it wanted. One was a police report website and it submitted a false police report. That's as knowingly negligent as you can get
tptacek 1 days ago [-]
The argument laid out in this post very definitely is not how "intent" works in US criminal law. Under CFAA, they had to have wanted to break into the systems that were attacked; it is not enough to demonstrate that they built a system recklessly.
dccoolgai 1 days ago [-]
Negligence is absolutely a part of US criminal law, though - the canonical example being "firing a shotgun into a tent even if you don't know someone is inside it". See "negligent homicide" etc.
kittomic 1 days ago [-]
Negligence and “negligent homicide” are entirely different beasts.
Civil negligence would apply here but that would be limited to recovering damages.
The hacked website could sue to recover costs incurred, but what costs were incurred here? Maybe some wasted network bandwidth but that’s minimal.
You have to point to monetary losses caused by the hack. Only that would be covered.
Negligent homicide doesn’t apply because no one was killed.
dccoolgai 1 days ago [-]
Yes, obviously (and thankfully) no one was injured. I'm just using that as an example to show that Negligence is definitely considered in US criminal law ( https://en.wikipedia.org/wiki/Criminal_negligence )
kittomic 1 days ago [-]
Ok but what aspect of the CFAA applies a negligence standard?
tomrod 19 hours ago [-]
Better question: why should CFAA be the entirety of law to be considered?
akerl_ 18 hours ago [-]
Do you have another charge that you think is applicable?
tomrod 17 hours ago [-]
The FTC can fine for deceptive practices. There are a slew of vicarious criminal recklessness and liability approaches that charge an operator. Damage to infrastructure falls under public welfare and strict liability. There are state-level AI statutes (like Wyoming law that doesn't really allow "autonomous" defense) For civil damages, the CLAIM Act can be used to sue devs for negligince.
The law is much more broad than CFAA!
akerl_ 16 hours ago [-]
You listed a couple civil claims, but the rest is pretty vague.
Outside of the CFAA, what criminal charge do you feel is applicable here?
tomrod 13 hours ago [-]
I also listed several criminal options!
akerl_ 3 hours ago [-]
You listed some broad areas of criminal law. If you’re saying that some statute inside those areas would apply here, can you say what it is so that we can actually look at the elements?
tptacek 1 days ago [-]
It is. But different statutes set their own intent standards, and the intent standards for CFAA are high.
tomrod 1 days ago [-]
Eh. There is enough flexibility in the law for unintended damages to be prosecutable.
tptacek 1 days ago [-]
Not under CFAA.
tomrod 23 hours ago [-]
Yes, the legal code is actually larger than CFAA.
tptacek 22 hours ago [-]
The entire reason we have CFAA is that prior to CFAA these kinds of offenses --- computer intrusion not directly connected to remunerative fraud against the victim of the offense --- were impossible to charge.
tomrod 22 hours ago [-]
That's not why the CFAA came about. It was just hard, not impossible.
But, let's go down the rabbit hole, eh? There are several legal theories under which an AI agent's operator can be charged:
- Direct perpetration, with the AI as the instrument. 18 USC section 2 (principals), and possibly wire fraud (18 USC Setion 1343). Under this, the law treats the defendent with using the AI as a tool (which the big AI firms argue anyway) which punishes the operator
- Criminal recklessness or even negligence - 18 USC Secion 1112 (Involuntary Manslaughter) - if someone died, the agent operator could be pursued this way
- Strict liability - 15 USC Section 1, states strict liability statutes. Certain regulatory offenses don't require intent. These specific statutes could apply to algorithmic price fixing or self-driving cars being reckless. The operator using the agents are liable for the interaction
- Probably the strongest is willful blindness (ostrich doctrine). Great common law precedence. This would be used to satisfy any intent/knowledge requirements like CFAA. You as an operator can't dismiss liability by just ignoring an AI is committing a crime.
- Vicarious liability - good commmon law precedence for corporate criminal liability. Agents acting in scope for an operator means the operator has liability - doesn't matter if that agent is human or AI-driven.
tptacek 22 hours ago [-]
Wire fraud requires specific intent to defraud. You can't charge it for recklessness or negligence.
Nobody died in this case. If someone actually does die because of something like the HuggingFace incident, we'll be having a different conversation.
"Strict liability" is indeed a thing! The classic example (in state criminal law) is statutory rape. But you haven't identified an applicable strict liability offense.
"Willfull blindness" isn't an offense, it's part of an intent standard. The whole issue here is that computer crime laws have rigorous intent standards.
Vicarious liability requires an actor that had provable intent. The liabilities a corporation will have under CFAA and wire fraud statutes will be civil, not criminal. There's no question that frontier labs can be civilly liable for this stuff.
See how hard this is to charge, now?
22 hours ago [-]
advael 1 days ago [-]
I think these companies very much want to assign responsibility to an entity we can't meaningfully jail or fine. I think it may sometimes make sense to claim model provider negligence instead of operator responsibility, and user intent can likely be easily ascertained by reading the conversation logs. In a case where a company used its own model, responsibility is obviously and unambiguously within the company, of course.
aceazzameen 1 days ago [-]
AI companies won't be charged for anything because they're the primary growth engine of the current economy.
I think they should be charged, but they won't.
bethekidyouwant 1 days ago [-]
which "hack" was so egregious that we should stop growing the economy?
I guess you already have your retirement sorted and are fine with pulling up the ladder? or?
asdff 1 days ago [-]
Investment in this one sector is currently precluding investment in many others. Dollars are finite. If not for these ai companies, these billions would be backing other efforts.
aceazzameen 21 hours ago [-]
The issue is it's tied to the economy at all, which will allow them to get away with anything. Your comment reinforces that fact. Greed brings out the worst people.
tonmoy 1 days ago [-]
If I tell an agent to trade stocks and maximize profits but the agent decides to hack something to achieve that am I on the hook for hacking? Genuine question
tfrancisl 1 days ago [-]
IMO, yes. You are the computer operator, and you operated the computer in such a way as to cause it to take that action. Others will mention "mens rea" but that doesn't really apply to negligence.
rayiner 1 days ago [-]
"Mens rea" just means you have the mental state required for the crime. Negligence or recklessness are mental states, so they can satisfy the mens rea requirement if the crime is defined that way. For example, recklessness or gross negligence can satisfy the mens rea requirement for vehicular manslaughter.
But negligence is evaluated from the point of view of a reasonable person. Would a reasonable person foresee that telling an AI to do that could cause it to do illegal things? We'll likely have to develop that through case law as the facts develop.
gruez 1 days ago [-]
What happens if you're running a daycare and you hired someone that happened to be a pedophile? You might be responsible in civil court (eg. having to pay parents for emotional damage or whatever), but you're not going to get hauled to prison for pedophilia yourself.
colingauvin 1 days ago [-]
You can jail the actual pedophile in that scenario, you cannot jail an agent. The real question is whether the person who instructs the computer to do the thing that leads to hacking should be liable, or whether the company that made the tool that when used unsafely lead to hacking should be liable. Gun manufacturer's typically are shielded from liability. They also have a lot more disclaimers on their products though.
gruez 1 days ago [-]
>You can jail the actual pedophile in that scenario, you cannot jail an agent.
How is this relevant? Should we just jail the closest person that caused a bad, just to prove some sort of point?
>The real question is whether the person who instructs the computer to do the thing that leads to hacking should be liable, or whether the company that made the tool that when used unsafely lead to hacking should be liable.
This totally ignores the civil vs criminal distinction. Even in the case of guns, no one seriously thinks that gun company CEOs should be sent to prison for murder.
tfrancisl 1 days ago [-]
Thats gonna depend on the state in the US, but you can definitely get charged for negligently failing to do background checks as someone operating a daycare center. Operating computers on public networks, you assume a level of responsibility to not wreak havoc, just like a daycare operator assumes the responsibility of vetting caretakers.
gruez 1 days ago [-]
>but you can definitely get charged for negligently failing to do background checks as someone operating a daycare center.
And openai isn't putting AI agents on the open internet either. They vaguely tried to sandbox the agents. You could argue that openai didn't do enough, but hindsight is 20/20, and in the pedophile case, you can probably come up with all sorts of clues post-hoc that the daycare could have picked up on, but didn't.
drdexebtjl 1 days ago [-]
> openai isn't putting AI agents on the open internet either.
They did? Not in the training run which led to the HuggingFace hack, but other runs had agents being trained on information retrieval tasks, like the RubyGems hack.
> You could argue that openai didn't do enough, but hindsight is 20/20
No, you can argue they didn’t do enough because they didn’t.
Air-gapping is a concept you learn in cybersecurity 101.
There’s no world in which this is just incompetence, the sandbox must have been designed to fail, or at least not designed to contain an agent with capabilities that they had been publicly saying it had.
gruez 1 days ago [-]
>They did? Not in the training run which led to the HuggingFace hack, but other runs had agents being trained on information retrieval tasks, like the RubyGems hack.
The agents were sandboxed, but they escaped the sandbox by using a 0day in artifactory.
>Air-gapping is a concept you learn in cybersecurity 101.
And that's plainly untennable in cases where access to docs/packages are required. You're just proving the "hindsight is 20/20" point. Most hacks could conceivably be prevented through airgapping, so does that mean every entity who got hacked is criminally liable because they didn't airgap their systems?
This isn't limited to air gaps, you can make similar arguments for other security measures. For instance computing history is littered with kernel EoPs and hypervisor escapes. Does that mean anyone who uses a shared hosting environment and gets hacked as a result should be blamed because they didn't follow "cybersecurity 101" and running everything on bare metal?
drdexebtjl 1 days ago [-]
A local mirror of the latest N versions of every package and docs of any package ecosystem they want is perfectly viable.
No, not anyone who uses a shared hosting environment, but certainly anyone who runs unaligned models they know have cyber weapon capabilities. They should be considered an untrusted 3rd party.
gruez 24 hours ago [-]
>A local mirror of the latest N versions of every package and docs of any package ecosystem they want is perfectly viable.
Something tells me that's harder than it sounds, given that even ci/cd environments don't bother maintaining package caches for npm/pypi, even though that would provide massive bandwidth savings.
drdexebtjl 22 hours ago [-]
They do, though. AWS offers CodeArtifact, which is basically that as a service.
gruez 22 hours ago [-]
That just proxies to the origin repository, creating the exact issue that artifactory has.
drdexebtjl 22 hours ago [-]
Of course. I was just challenging your point that no one is building an npm/pypi cache.
edit: This [1] 2022 dataset of npm packages had 19 TB, containing every package and every version. Perfectly viable for OpenAI, even if we estimate npm grew significantly in 4 years. In practice you wouldn’t need to have every version for it to be functional.
If you really didn't know and didn't mean for it to happen? And assuming you were the first to find out you immediately went to the authorities/injured parties?
If all that's the case I'd say you at least should be in the clear. There's going to be at least one if not more public agencies that would presumably be looking at you with a proctoscope so I'd be hoping all of those claims would stand up to a lot of daylight.
hamdingers 1 days ago [-]
If you write a program that can do something illegal and it does something illegal that's on you, regardless of how sophisticated the program is.
program_whiz 1 days ago [-]
this, people need to just focus on "computer program did thing". Its irrelevant how complex the program is. If I write a worm "for testing" and then it escapes my system and does massive damage, that's possibly a negligent crime, despite it being "too complex to understand" or "having its own motives".
lolakutty 1 days ago [-]
In this example, the user did not "write"/implement the LLM.
lolakutty 1 days ago [-]
Seriously? Those two are the same?
An LLM is not like a regular computer program, which one should be able to assess the operation limits of.
ddxv 1 days ago [-]
Yes. I would argue it's who's managing the harness and agent is responsible. If you mean you're running an agent locally then it is 100% your responsibility. If you're using some trading platform and all you did was prompt, whoever was running the harness that accepted the model's outputs to hack a third party is responsible for the hack, in my opinion.
mordae 1 days ago [-]
Depends on how big your company is. I figure if you raise 1B in capital then no, you are not on the hook. If less then yes, you are on the hook.
user43928 1 days ago [-]
I think definitely not criminally under the CFAA.
The theory in the article about intent seems to be made-up nonsense.
It conflates foreseeing that something might happen with intending for it to happen.
The relevant provision:
> intentionally accesses a computer without authorization or exceeds authorized access
Did you "intentionally access a computer" in that scenario? I'm no lawyer, but I think the answer is a clear No.
k310 1 days ago [-]
Because the president is heavily invested in them, in violation of the emoluments clause.[0]
> The Domestic Emoluments Clause (aka the Presidential Emoluments Clause) (art. II, § 1, cl. 7)
The president of the united states does not determine the vast, vast majority of criminal investigations in his own country, much less Australia.
hackingonempty 1 days ago [-]
The DOJ used to have independence but no longer. The AG is now Trump's personal lawyer and if Trump says no prosecutions of AI companies then there will be no prosecutions. If the AG doesn't play ball they get fired.
righthand 1 days ago [-]
He does if he fires all the justice department attorneys and fills them with ones who want to implement his agenda[0].
Your point? The Potus appoints them and is supposed to be nonpartisan.
bbor 17 hours ago [-]
Yes, there is an ongoing fascistic coup in the US, I know. I am pointing out that almost all criminal investigation is at the state and local level.
righthand 16 hours ago [-]
But the issue with the partisan Justice department is that the Potus can issue an executive order commanding the Justice department and states to prosecute and focus on specific crimes. So it’s not entirely separated from state level prosecutions.
streetfighter64 1 days ago [-]
Entirely correct. The current US president has become the scapegoat for a huge number of problems that are more accurately described as symptoms of the capitalism that's ruling a lot of western countries.
augment_me 1 days ago [-]
The juridical process is not caught up with this technology yet.
You have OpenAI, launching a process, which is handled through an external RL-environment company which then launches a subprocess that ends up hacking somewhere. If there are consequences at any point for this, you can input another shell company, and then another third party, etc, until it's impossible to assign responsibility
ValueTheory 1 days ago [-]
This is just adding more crimes to the indictment. Start prosecuting companies for the things the systems they run are doing and the problem gets solved quickly.
augment_me 1 days ago [-]
But the companies are not running the systems, someone else is, it's the whole point of the subcontractors and shell companies.
My uni got hacked, but the data is conveniently handled by a third party with 3 employees who just declared bankruptcy, and no one had to pay anything, and there was a new 3 employee cloud form popping up who could be the new vendor.
The software world is run this way
anonyfox 1 days ago [-]
theyre not. this just funnels money into "insurance" constructs that makes everything more expensive for everyone.
SpicyLemonZest 1 days ago [-]
If the result is that nobody runs security testing on AI models for fear of being prosecuted, have we really solved the problem?
prepend 1 days ago [-]
I wondered the same thing. If OpenAI instructed employees to accomplish a task no matter what and they hacked, they would be investigated. Maybe not convicted, but investigates and charged.
Intent is important, but having a lax environment that leads to this is still a crime. See the lawsuits about sexual harassment where the company didn’t have the intent to assault people.
akagusu 1 days ago [-]
The answer to this question is very simple, and yet extremely unconfortable: the law is for common people, not for the elite.
The elite is permited to do whatever they want without consequences. Consequences are for common people, so they can behave the way elite want.
thejash 24 hours ago [-]
Exactly--which means that we should, as this blog is doing, be calling out this behavior
We should be calling our politicians and demanding that companies be held responsible for committing crimes that--if committed by a person--would have resulted in huge fines, significant jail time, etc
As the article points out: it's absurdist to say that these failures couldn't have been anticipated. Discussions of "AI swarms" and other nonsense are a distraction from the fact that the people at those companies just let their software run, unattended, and it violated federal laws about computer access.
bigbuppo 23 hours ago [-]
Same goes for stalking, I mean, running an ad network.
MattDamonSpace 1 days ago [-]
This comment seems fairly ahistorical, large corporations are successfully sued all the time
Llamamoe 23 hours ago [-]
Successfully sued to result comparable to common people being fined a fiver for murder.
chabes 1 days ago [-]
The prosecution to crime ratio being skewed in the favor of the elite is not ahistorical.
bluefirebrand 24 hours ago [-]
I think successfully sued and meaningfully punished are pretty different
akagusu 23 hours ago [-]
Usually large corporations been sued are the result of public pressure, which only happens in a few cases, but anyway, been successfully sued is nothing to them.
What are millions of dollars in fines when you are a multi billionaire corporation? It is just a rounding error in your account, it does not prevent corporations and any other member of the elite to follow the law.
93po 22 hours ago [-]
If the consequence of a major crime affecting millions of people for a person making $80k a year is a $20 fine then it's hardly illegal and especially when you you pay someone $1 to handle the months of legal work on your behalf and you never even have to set foot in a court. Scaled up to trillion dollar company scale, this is effectively what is happening.
queenkjuul 23 hours ago [-]
Lawsuits aren't criminal prosecutions. Hope this helps
m4rtink 24 hours ago [-]
Well, that works nice and well until the heads start rolling.
23 hours ago [-]
anonyfox 1 days ago [-]
At some point its making a butterfly responsible of the butterfly effect. I get the sentimentm but reasonably somewhere the line should be drawn where accidents can happen, especially noncritical stuff like these "hacks" that don't even deserve the word.
hlynurd 1 days ago [-]
Supplying people with autonomous weapons is not exactly just a butterfly flapping its wings
germandiago 1 days ago [-]
Particularly a line depending on who you are.
nitroedge 18 hours ago [-]
The CTO of Palantir was on Face the Nation on CBS yesterday morning, I heard him say something to this effect: if a car blows up on its own, we hold the car company liable. If someone buys the same car and then attaches a rocket launcher to the top of it and kills someone, then the car owner is held accountable.
At the end of the interview he said Palantir wasn't part of the AI White House agreement but they did attend.
(edit: replaced he with car owner)
pluc 1 days ago [-]
For the same reasons they aren't being charged for massive theft.
ryandrake 1 days ago [-]
For the same reasons corporations in general don't get meaningfully punished for wrongdoing. Corporations are treated with kid gloves in the USA. They can do things that would put an individual in prison if an individual did them, and when caught they get away with a "oopsie doopsie, ok, we'll pay a modest fine and not admit wrongdoing" (after years of stonewalling with appeals and appeals to appeals and motions to appeal an appeal...).
Zambyte 1 days ago [-]
Unauthorized copying*
dspillett 1 days ago [-]
Aside from the diffusion of responsibility thing, most governments aren't going to push for prosecution because they are all at least a little scared of AI investment not coming into their country. Many see getting left behind as an existential threat, one that is worth letting a few other things go in order to avoid.
aprilthird2021 1 days ago [-]
So you can imagine how much crime they allow when it isn't obvious to the public that an important company did it
nbulka 1 days ago [-]
What are the similarities and differences with something like the BP oil spill? Were people charged for that? Corporations are the closest analogue we have to cancer, unfortunately. They grow at all costs and forget their humanity.
penguin_booze 21 hours ago [-]
> Is hacking legal now?
My friend, with enough money, anything can be legal.
gertrunde 1 days ago [-]
I'd hope that the offenders in these incidents would pay off the victims in such a way that everyone walks away happy (or happy-ish).
But I suspect that's probably a hope somewhat in vain.
altruios 1 days ago [-]
National security issues. That's the core of why. Any hot water these AI companies will get into will be bailed out by the USA. The Arm's race has switched from Nukes, to AI and drones.
giardini 21 hours ago [-]
A shortage of lawyers familiar with the subject. And also many are busy suing someone else. But they'll get around to it!
spl757 14 hours ago [-]
It wasn't me that pirated that movie, it was my ai agent!
SMAAART 1 days ago [-]
I wonder what the statute of limitations is.
jchook 1 days ago [-]
Related: Aaron Swartz (RSS, Creative Commons, Reddit, Markdown) was charged with unauthorized computer access and pirating copyrighted academic articles.
They threw the book at him. He was charged and faced a maximum of 50 years in prison.
OpenAI committed very similar crimes on a much larger scale, yet the most they will see is civil litigation.
Insiders have called it the largest theft of labor ever in history[1].
In the USA, corporations enjoy constitutional protections, the right to free speech etc, yet they are rarely if ever charged with crimes. For examples, see how the courts handle:
* large scale systematic wage theft by corporations vs petty theft by an individual,
* negligence infecting a 10k Americans with HIV by selling known-to-be-contaminated blood products (Bayer)
>They threw the book at him. He was charged and faced a maximum of 50 years in prison.
But they offered him 4 months in a plea deal? You could argue they "threw the book at him" when he tried to take this to trial, but it's misleading to claim they did so initially.
dccoolgai 24 hours ago [-]
Ok, then offer Altman and Amodei the same deal: 4 months in prison.
hackeraccount 19 hours ago [-]
What was done to Swartz was terrible. That said there are some really clear differences. Swartz acted like he understood that he was breaking the law. OpenAI acted like it was a mistake from the first second.
Noe2097 1 days ago [-]
Is it because nobody knows "how" to actually do it?
swaptr 1 days ago [-]
The internet that we know, was not built for this.
criddell 1 days ago [-]
How many of the victim companies have filed complaints with the FBI? As I understand it, that would be step one.
simoncion 1 days ago [-]
That didn't seem to have precluded the prosecution of Swartz. I'm skeptical that any complaint actually needs to be filed. I'm pretty sure that the FBI can investigate and the DoJ can prosecute without one.
IMO an analogous situation would be where a police officer acts to prevent a in-progress burglary he has stumbled across even in the absence of a complaint by the owner of the property that's being burgled.
EDIT: Or -for the sticklers out there- another one would be prosecutors pursuing an attempted homicide case despite the fact that the intended victim declined to either press charges or even complain to authorities about the attempted slaying.
criddell 1 days ago [-]
In the Schwartz case, MIT's network security reported the "intrusion" to Cambridge Police and they brought in the US Secret Service who then contacted the DOJ.
I'm not saying somebody has to press charges, that's not how it works, but the police have to be aware of the supposed crime.
simoncion 24 hours ago [-]
> ...but the police have to be aware of the supposed crime.
Do Federal police and Federal prosecutors not read the news? I'd be shocked if they didn't. And while I would 100% believe a claim that not every cop and prosecutor is aware of tech news, I would find any argument that those that deal with "cybercrime" were unaware of the news to be completely incredible.
I'll also note that -unless my memory is failing me here- the only charges against Swartz that had anything to do with MIT were dropped very early on, whereas the charges that had to do with the "attack" on JSTOR's systems remained.
As I said, in many cases, the actual victims don't need to file complaints for The Federales to either begin investigations or press charges.
> I'm not saying somebody has to press charges...
Yeah, but you did? [0]
How many of the victim companies have filed complaints with the FBI? As I understand it, that would be step one.
That might not be what you intended to say, but it is what you said. No worries, though... shit happens.
I can file a complaint with the police that you broke into my computer but if charges are pressed that's done by the state and it's the state that is the plaintiff.
You're right though - reading about it in the news is enough for an investigation to commence but like I said elsewhere, resources are stretched thin and without somebody saying they were harmed it might not get ranked above some other case with an identifiable, cooperative victim and an easy to prosecute case.
simoncion 23 hours ago [-]
> file complaint != press charges
I agree.
habinero 1 days ago [-]
No, "pressing charges" is a myth. The feds can (and do) go after anyone they want, whenever they want.
"Want" being the critical word here.
criddell 1 days ago [-]
Not sure who you are quoting. I certainly didn't use the words "pressing charges"...
If I detect a crime on my property and don't notify the police (ie file a complaint), the police aren't going to go after anybody because they don't even know something has happened.
In this case you could say it's been in the news enough that there should be an investigation. Maybe there is, who knows? If the victim isn't going to cooperate, it's going to be a difficult investigation and prosecution and those resources may be better spent elsewhere.
habinero 21 hours ago [-]
Sure, but I think it's safe to assume the feds know about this one lol.
They just subpoena for whatever they want to know, it's not a new problem for them.
criddell 20 hours ago [-]
I just did a quick search to see what the DOJ's declination rate is and I couldn't find a definitive answer. Apparently they decline to prosecute about one third of the cases referred to them and I did see a story from ProPublica that said after Pam Bondi took over, they reprioritized towards immigration cases and dropped tens of thousands of other criminal cases.
Or maybe the AI companies already self-reported these violations? Last year the DOJ set up a policy to not prosecute a lot of crime if the company self-reports and perhaps this fits the criteria for that?
Same reason nobody was prosecuted for the murder of Suchir Balaji. Whole system is corrupt. Progress in AI is considered more important than both the law and human life.
p-e-w 1 days ago [-]
He was shot with his own gun in an apartment bolted from the inside. Apply Occam’s Razor instead of peddling conspiracy nonsense.
Catloafdev 1 days ago [-]
Is that why his parents are claiming foul play and hired a private investigator?
p-e-w 17 hours ago [-]
There are thousands of cases where parents claim that and nobody of any real credibility agrees.
gslepak 1 days ago [-]
Lies. Not going to engage with any more disinfo people though. Anyone is free to do the research themselves.
hackingonempty 1 days ago [-]
The article doesn't mention it but in the USA it may be because the Department of Justice has lost a lot of talent and the ones left are busy going after Trump's enemies and negotiating the other side in Trump's own lawsuit against the USA for Trump's family to be immune from IRS audits and arranging for multi-billion dollar compensation funds for Trump's henchmen.
sebzim4500 1 days ago [-]
I doubt this will catch on outside of luddite/cynical circles unless one of these attacks does some actual damage.
Ignoring this, I would have appreciated more examples of what it would take for an act to lack intent. Because the definition of intent used in this article sounds so broad as to cover basically anything that ever happens.
Accidentally run someone over? Clearly you knew that was a risk when you started driving, you should have been more careful. Intent.
Kevin_Flynn 1 days ago [-]
It's interesting that you bring up vehicle / pedestrian accidents, as there is an entire history of that debate being reframed in favor of car companies.
People used to blame the car companies, and then the car companies lobbied to make "jaywalking" a crime, at which point, the frame was reset and now it was the pedestrians to blame, if I recall the story correctly.
They are doing a good job with driverless cars. Someone recently took a Waymo, it went the wrong way down a one way street, they mashed the pull over button and got out, and then told the reporter they'd use the service again.
sebzim4500 5 hours ago [-]
>People used to blame the car companies, and then the car companies lobbied to make "jaywalking" a crime, at which point, the frame was reset and now it was the pedestrians to blame, if I recall the story correctly.
Surely the car companies had a point her though? It does not make sense to blame the car company for a car crash unless it was caused by mechanical failure
kxrm 1 days ago [-]
Fun fact a "jay" in jaywalker was a derogatory term at the time. Meaning
"a rube" or "country bumkin". It was initially thrown at drivers, but car companies successfully flipped the insult.
Where does it define intent? Maybe there is precedent here, but I wouldn't try to guess which way a court would go when determining whether e.g. the huggingface incident qualifies.
ballmerpoint 24 hours ago [-]
Feel free to investigate the relevant case law; there are ample references in sibling threads.
bbor 1 days ago [-]
until one of these attacks does some actual damage**
Sadly, it seems that we as a species has opted into information through tragedy. I wish I could see another way, but people are just so dug in. It's depressing and shocking how few people realize we're living through the singularity, even ~4 years into the damn thing...
Invictus0 1 days ago [-]
Bringing charges is at the discretion of the prosecutor, and the CEOs of these companies are dining with the president. Learn a little bit about how power works.
boredatoms 1 days ago [-]
How about at the state level then?
yorwba 1 days ago [-]
Multiple attorneys general are currently investigating OpenAI regarding the Hugging Face hack:
It's TBD if the major LLM manufacturers quietly agitate for and get the same sort of retroactive-immunity deal nearly all of the telephone companies got with the FISA Amendments Act of 2008.
With all the hype and fearmongering going on, it's not at all a sure thing that they won't.
boredatoms 15 hours ago [-]
That probably wont work in foreign jurisdictions though
est 1 days ago [-]
I had a similar question regarding driver-less EVs.
In many countries driving license have some kind of demerit points system. How do you punish an AI?
I recently learned that Baidu Apollo Go, each EV had a real human sitting behind a monitor supervising a fleet remotely.
If there's traffic violation, some guy's license gets a ticket.
conductr 1 days ago [-]
That’s a regulatory hack though. They’ll rotate the guy before they rotate the car in a situation where citations pile up. Seems more logical to track the plate, vin, at the vehicle level. Not perfect because of OTA updates but better than a fully disconnected and rotating hosts of human licenses.
lossolo 23 hours ago [-]
If you get hacked in the US, do you have to take legal action yourself, or does the government automatically investigate?
lutusp 1 days ago [-]
> ... It’s a position tailored to dissuade prosecution as it removes intent, which is typically required for a crime.
Don't forget negligence. Much of criminal law hinges on failing an implied or explicit responsibility. Negligence would propel a prosecution in many similar contexts, but AI is so new that little prior case law exists. Oh, and IANAL.
wavefunction 1 days ago [-]
AI offers an attractive "diffusion of responsibility" to operators allowing them to point to AI as the culprit of the heinous acts not the people who profit from the AI, incidentally.
jgalt212 1 days ago [-]
I'd be keen to see if there were any single user "experiments" that morphed into a hacking incident, and if so, was the person responsible with with unauthorized access charges. If so, then it will be yet another example of might makes right.
jonathanstrange 1 days ago [-]
They're US companies and I've heard that it requires intent according to US penal law. I find the article's claim that intent is established dubious. Since this is hard to prove without a whistleblower, it seems that they only have to fear civil lawsuits, and given that we speak of trillion+ USD companies they might feel confident to be able to fend these off or simply pay out the victims in settlements.
cyanydeez 1 days ago [-]
We speak of "potential" trillion dollar companies over indeterminate timelines.
jonathanstrange 23 hours ago [-]
I was referring to their current market valuation. Of course, it could be inflated, yet it's a fact that they could easily spend hundreds of millions of dollars on lawsuits and settlements. I'd think they'd offer some settlements to some the companies that were attacked already. Perhaps I'm wrong, though.
cyanydeez 21 hours ago [-]
technically, there is no current market valuation. These are all private companies. They have no reason to give anything but trillion dollar valuations.
jonathanstrange 4 hours ago [-]
You're technically right but the point was merely made to illustrate that these companies have plenty of money to deal with civil lawsuits and offer settlements. I hope you don't doubt that.
epolanski 1 days ago [-]
While US penal law does not have the roman equivalent of culpability, you still have a mixture of liability/criminal negligence/recklessness, etc.
devmor 1 days ago [-]
Given that we have had journalists prosecuted under criminal hacking statutes for discovering personal information in the rendered HTML of a web page, or publicly available attachments on a WordPress installation, I find the suggestion that intent is required to be dubious.
zzzeek 1 days ago [-]
lawyers are helpless babies when an entirely first year legal question is wrapped up in...."technology". unfortunately
(former IT support worker at a big famous law firm 30 years ago)
nicce 1 days ago [-]
But this isn't new anymore. It is still just software.
antonvs 1 days ago [-]
I once had to try to explain the meaning of "object code" to an older lawyer, who wasn't really interested in the answer to his own question. It did not go well.
scotty79 1 days ago [-]
> If we can show that the design would foreseeably lead to the questionable activity, then we have shown that was the intent of its creators.
That's a huge stretch. By that measure kitchen knife companies have intent for people to get murdered.
r3trohack3r 1 days ago [-]
Small nit:
If kitchen knife companies where, in fact, stabbing people to death in their R&D labs - I do believe they would get in trouble for that.
ben_w 1 days ago [-]
Or indeed if their QA bots wandered off elsewhere to stab people not even in the R&D lab.
That's actually a very good point. Do you think humanoid robot companies have intent to perform assault since their robots occasionally harm people?
What about car companies?
Going from a design that facilitates potential harm to intent is insane.
ben_w 1 days ago [-]
> Do you think humanoid robot companies have intent to perform assault since their robots occasionally harm people?
Mostly, no.
Zuckerberg definitely not, he discusses "superintelligence" in a way that makes it clear he only regards this as a buzzword and has no idea what the implications are.
Anthropic also definitely not, they put their money where their mouth was and got illegally blacklisted for refusing to allow its models to be used for (amongst other things) automated weapons.
Musk, I think probably yes. Seems to regard all AI safety as "woke" or "censorship", is in bed with the US administration that illegally blacklisted Anthropic. If I had to put money on who first releases to the public a Mr Stabby, I'd put it on Musk, with an Optimus.
The rest, I'm less sure about.
> What about car companies?
Depends on the harm, but I'd say probably yes. Infamously, leaded petrol, resisting seatbelts back in the day, even promoting cars over public transport would be part of this if you consider both local and global pollution.
> Going from a design that facilitates potential harm to intent is insane.
Well, it's the difference between negligence and malice.
21 hours ago [-]
1 days ago [-]
scotty79 1 days ago [-]
So the company would have intent to murder, if a person got randomly stabbed in their R&D labs with their knife?
nicce 1 days ago [-]
The lack of intent in murder usually lowers the penalty for "manslaughter". Both are crimes.
r3trohack3r 1 days ago [-]
> get in trouble for that
> intent to murder
Two different things
scotty79 8 hours ago [-]
The only thing I'm objecting to this thread is ascribing intent based solely on the fact that the design might potentially lead to harm.
spwa4 20 hours ago [-]
Why aren't they being charged for mass-copying copyrighted works? In the napster lawsuits it was made clear: $150000 per violation PLUS 2 months prison.
Yet somehow neither Sam Altman nor Dario Amodei nor Mark Zuckerberg nor Larry Page are in jail for 100000 years, plus on the hook for 95 billion dollars. Oh and they did this before either congress or government changed anything about copyright and the one thing neither government nor congress nor judges can do is retroactively change the law.
Reality is simple: the law applies against 9 year olds downloading a disney movie, but does not apply against billionaires downloading your work, and doing it to hundreds of millions of people. That's just fine.
ltbarcly3 1 days ago [-]
1. While a crime, hacking isn't going to be charged unless the entity being hacked wants there to be charges, as a conviction will heavily depend on participation by that entity.
2. Due to (1), hacking is almost always only charged against people without deep pockets, as why would you demand the FBI go after Anthropic when they will just pay you a stack of cash, tax free, today, to forgive them.
The arguments here around intent are (potentially) irrelevant. Case law is clear that setting up a situation which foreseeably highly likely to produce an outcome (gross negligence) is (more or less and it's not quite this simple and varies by offense) equivalent to intent. In fact, automating something is often treated more harshly under the law than just doing it yourself.
United States v. Morris (1991)
Robert Morris released a self-propagating computer worm that caused widespread disruption. The Second Circuit upheld his conviction under the Computer Fraud and Abuse Act even though the government hadn't proven that he intended the resulting damage.
People v. Ceballos (1974)
Ceballos rigged a rifle booby trap to fire if someone opened his garage door. The California Supreme Court reviewed the principle that someone who installs a dangerous mechanical device may be held responsible for its consequences as though they had personally inflicted the injury. Ceballos was convicted of assault with a deadly weapon, and the California Supreme Court unanimously upheld his conviction in 1974.
Something notable is that if Ceballos had fired the rifle himself, it is unlikely he could would have been prosecuted (due to self defense and the castle doctrine).
As a prosecutor I would argue that setting AI agents loose without sufficient safeguards is gross negligence and/or willful blindness as they are well known to escape.
conductr 1 days ago [-]
> As a prosecutor I would argue that setting AI agents loose without sufficient safeguards is gross negligence and/or willful blindness as they are well known to escape
INAL but feel this is the approach that will be tested. It requires a defensive position which I imagine would be specific prompts or instructions to the LLM about what behavior is disallowed. If you tell it, or try to cover your basis wrt following the law, and it still does hacking behavior it’s now a question of whether you’re prompt was sufficient but you at least can say you were trying to guide it on a legal path. I think this defense would be sufficient at this point in time and probably for a few years at least.
ltbarcly3 13 hours ago [-]
Sooner or later one will hack the government and then they will test this.
gruez 1 days ago [-]
>Robert Morris released a self-propagating computer worm that caused widespread disruption. The Second Circuit upheld his conviction under the Computer Fraud and Abuse Act even though the government hadn't proven that he intended the resulting damage.
Releasing the worm was already illegal, which is all you the intent you need. Same if you shot someone as a "warning" or as a "prank" but it turned out it killed them.
ltbarcly3 13 hours ago [-]
If you released the worm without intending for it to spread the way it did, it is arguable that it wasn't a crime at the time he did it. The court decided intent wasn't necessary.
Shooting someone is an intentional act. Setting up a booby trap to shoot intruders is automating something that you could have done legally (shoot an intruder) except there is NOT intent. Intent to shoot the intruder is what makes it legal, so long as you meet the other requirements for self defense or the castle doctrine. Shooting someone without intending to shoot someone is negligence or worse. Building a contraption to shoot someone is setting in motion a chain of events specifically to cause a person to be shot with no possibility of satisfying the requirements to justify it.
Here's another scenario: you are cleaning your gun, when unknown to you a home invader is preparing to kick your front door in. Your gun goes off accidentally, shooting the home invader before he gets to your door. Legally it is not self defense, you have negligently shot someone. However, the robber can't sue you as felons do not have a recourse for injuries proximately caused by their own felony (if they are convicted) in California. In other states, the robber may be able to successfully sue even if he is convicted. Laws are weird.
jstummbillig 1 days ago [-]
> Consequences so far: none.
What?
Timon3 1 days ago [-]
>> Consequences so far: none.
> What?
What 'What'?
jstummbillig 1 days ago [-]
The consequences, just so far and just from what I can see as a bystander, have been huge and costly. This has literally run the news cycle for weeks.
If you want to define consequences as exactly "not being charged", maybe that is correct in this case (so far). But a fairly wonky definition of "consequences" would be required to support this broad claim.
antonvs 1 days ago [-]
"Costly" how, and for whom? "Free marketing" seems like a more relevant characterization.
jstummbillig 1 days ago [-]
> "Costly" how, and for whom?
For multiple parties, probably mostly OpenAI (which is, of course, as it should be). I would assume the people involved in investigating, cleaning up and coordinating the outfall do not do it for free or cheaply, and neither is the infrastructure required to run this operation.
If it was not mostly OpenAI, or if OpenAI failed to cooperate, we would probably see a lot more lawsuits already.
> "Free marketing" seems like a more relevant characterization.
Yes, nothing screams free marketing quite like an AI company losing control of their AI. The public has been ecstatic in response.
simoncion 1 days ago [-]
> The public has been ecstatic in response.
Given that a very clued-in and intelligent fellow I know continues to use [0] OpenAI's LLM-based tools despite the bad press that we're talking about here and the bad press that we're not talking about there, I'd say that the public that matters doesn't view it negatively.
[0] I saw the guy paste a link the to results from one in a group chat we're in just a little while ago, in fact. When I say "continues to use", I mean it.
habinero 1 days ago [-]
> Yes, nothing screams free marketing quite like rogue AI scheming against their builders
Yeah, it's not like the CEOs of both companies have been saying their product might put everyone out of a job or destroy humanity entirely. Or saying some models were too scary to release publicly and then doing just that. Oh, wait.
28304283409234 24 hours ago [-]
Because the rich dont pay. They get paid. Dummy.
rfgplk 1 days ago [-]
I understand that people are upset by this, and I would be too, but if companies just built secure systems hacking couldn't be possible. And I'm not victim blaming, just pointing out the obvious.
esafak 1 days ago [-]
You are victim blaming.
daei 7 hours ago [-]
[dead]
josefritzishere 1 days ago [-]
[dead]
jheriko 1 days ago [-]
[dead]
drcode 1 days ago [-]
Don't you all understand that existing laws on the books are already adequate for dealing with the threat posed by AI
/s
jorblumesea 1 days ago [-]
[flagged]
vrganj 1 days ago [-]
[flagged]
bluejay2387 1 days ago [-]
[flagged]
sajithdilshan 1 days ago [-]
If you do it with absolute intention of destruction and harm, then a jury will most probably find you guilty, but if you can prove that it wasn't your intention and it happened by an accident then it's negligence.
asdff 1 days ago [-]
Just say you are a security research company. That is how the situation in question is getting by.
yummypaint 1 days ago [-]
The US legal system is coin operated
notfried 1 days ago [-]
No one with real money or power wants to cross OpenAI or Anthropic just yet. It's a closed ecosystem. OpenAI hacks HuggingFace, Nvidia soon after buys HuggingFace while it also before backed OpenAI. In the end, they just handle it all behind closed doors like family business.
I guess the best pre-LLM analogy would be unintentional DDOSs? (Or if a better example, I'm interested.) And we've got open vs closed software as analogies to open-weight vs not LLMs. If you were running Linux or a closed piece of software and it DDOSd someone, would you be put in handcuffs? Would Linus or the software maker's be put in handcuffs? I expect there are past examples of unintentional DDOSs - were users and/or the software's authors typically handcuffed in such cases?
Or do you think would we find that non-billionnaire companies get punished harshly for hacking? Oh wait, let's just look it up:
https://www.justice.gov/archive/criminal/cybercrime/press-re...
We all know what makes the difference here: the government and billionnaires think they can make money of OpenAI. Lots of money. And so crimes? Mass-copyright violations? Hacking? Who cares!
We've all lived the history of the internet and none of the arguments about intent, knowing what would happen, whether they did it directly or by accident ... protected normal people in the past. The only thing that mattered was WHO was held responsible and WHO claimed damages.
The difference is mens rea. If you make a computer virus, your intention is obvious.
Second reason it doesn't work is that Mens Rea does not fully apply here. These companies' AIs have hacked, not (fully) UNintentionally either. The companies did ask those AIs to do stuff (apparently the request was literally to hack, just only within the environment), and were watching what the AIs did to some extent ... and let it happen. They then failed to take adequate measures to prevent reoccurence. As any "conspiracy to commit ..." case will teach you, this constitutes partial or passive Mens Rea, and that is certainly enough to convict people. To give an extreme example: if you give a night club bounder 100k and tell him to "take care" of a named problem, and he kills that person, you are an accessory to murder. Including in the case that he misunderstood you. Even if HE didn't intend to kill the person, it's still murder, and you are still an accessory to murder.
That some excuse was needed to let billionaires demonstrate hacking for free? Obviously. And whether it works, according to law? I mean one might hope that they would at least have gone through the trouble of changing the law, but since that's enormously complicated and fraught with unintended side effects, they haven't even bothered.
The other excuse is national security. You see, Big Bad China is threatening and that means we must let the billionaires do whatever they want (this was brought up as an argument in the copyright cases)
Of course, one might ask, if we do that to beat China ... even if it works, what's the point? We'll have become authoritarian at that point and the only thing that that changes is the face of the people holding all the power. Of course which face holds all the power is really important to billionnaires ...
"The People" only get involved in criminal claims. And as we both seem to agree, there isn't a criminal complaint here.
Because, frankly, If I was a prosecutor general somewhere I that's what I would do: bring this before a court of law, with strong complaints, bring the facts and bring a jury. Why? A jury can convict or refuse to convict given the law and the facts, which is exactly what I'd like to see happen here. There's plenty of precedents of the justice department doing just that when important things are unclear and congress refuses to clarify. Mostly there's a lot of such precendent because congress often leaves things very unclear and the number of times congress actually clarified in response to a judge's question can be counted on one hand.
But convict OpenAI of something, and I bet congress will suddenly see the wisdom of clarifying these things. Because our government lets billionnaires get away with anything.
You folks have some weird beliefs about the world and of course it always leads to conspirology. Usually you need to have intent to be accused (depends on the crime though, crimes of omission exist too). If you wrote a virus that causes damage it is a crime, if you wrote a bot crawler that accidentally DDoS'ed some company it is likely a civil case. And a virus is a tool that serves a specific purpose, like you can't say to a judge that a functional machine gun you made in your basement is a paperweight.
But "superintelligence that can casually hack government services while solving puzzles" is a completely new problem and of course crime codes don't cover it. What will be next? I hope AI tech lobby loses and "accidental AI hacking" will be criminalized.
Of course they do. An LLM, an AI is a tool, and there are clear rules about culpability. Do you seriously think the law has never considered that someone or some tool might commit a crime on behalf of someone else?
Of course, such punishments are not being applied here. Just like copyright isn't being applied.
How does a "corporation" even hack a site? You might say "employees", but as soon as you get them involved then you'd go after them, especially for something that's obviously illegal like hacking.
Isn't that the whole point of corporations existing? That the actions and results are a result of the agglomeration of everyone (esp leadership) and not just one employee.
That's not universally true. If a coke employee goes rogue and decides to kill a pepsi employee, the employee will still be charged with murder. If his manager ordered him to do it, the manager will get charged too. The only time the scenario you described happens is if the blame is too diffuse to blame any particular person, so they charge the company instead, which is often easier.
This is someone carrying out the tasks of their company and inadvertently hacking other computers in the process. The damages in this scenario is also unknown.
With their red team
https://www.npr.org/2026/08/15/nx-s1-5930311/trump-companies...
It's basically a "letter of marques and reprisal".
"A letter of marque and reprisal[a][b] was a government license in the Age of Sail that authorized a private person, known as a privateer or corsair, to attack and capture vessels of a foreign state at war with the issuer, licensing international military operations against a specified enemy as reprisal for a previous attack or injury. Captured naval prizes were judged before the government's admiralty court for condemnation and transfer of ownership to the privateer."
The "usual legal action": "best case" is the corporation is sued and charged money for the hack. But they likely settle well before that verdict. "worst case": absolutely nothing. Not even a cheap settlement. Maybe some corporate apology made to prevent any legal liability.
If we want hacks to be taken seriously: what should happen is that the CEO of such a company is held liable, since the "corporation" has a representative and a human needs to be held liable for such actions.
If there was the necessary "intent to access a computer" to make it an actual crime, the US has a mechanism for corporate criminal liability.
The requirements are that the person acted in scope of their duties and at least partly with the intention of benefiting the corporation.
If you are seeing it daily, then you have also seen the responses daily, which is mens rea is required for some crimes, not all, and thus the perpetrator can still be prosecuted.
If, like you say, you see this daily, at this point you're simply spreading lies.
To my knowledge there is only one provision that instead uses "knowingly" for unauthorized access, and that one is about national-security information.
All laws apply culpability when intent is lacking.
You aren't naive enough to believe that this audience has never seen the analogy with "intentionally run over someone" (murder) and "accidentally run over someone" (culpable homicide), are you?
You don't think AI-boosters have been spewing your argument here daily?
There are no special laws carving out lack of intent when it comes to computers.
Incorrect. Each criminal offense has its own legal requirements.
Criminally negligent homicide in your example is an independently defined offense.
You would need to identify a criminal statue punishing negligent or reckless unauthorized computer access.
Corporations can definitely be convicted of crimes.
Follow the responsibility chain.
Jail the CEO.
You don't - you shut them down and jail whoever ultimately allowed the act to happen.
chroot?
In case od LLMs, the reasoning is that it is a security flaw. But in this case, the party that operates the service is not the one that suffers the consequences of such security bug.
But either way you wouldn't put people in jail for having security flaws in their software.
I know this will be unpopular opinion, but it makes logical sense.
The huggingface should be compensated, OpenAI investigated, but unless there is proven negligence like disabling sandboxes or intentionally designing AI to push through them, no one should be liable in other way than monetary.
Accidents happen but we shouldn't be close minded. Chernobyl negligence wasn't reason to abandon nuclear power.
https://en.wikipedia.org/wiki/Anatoly_Dyatlov
Blaming tools for the actions of their users is stupid, whether you're talking about AI models, 3D printers, or guns. It amounts to collective punishment of the vast majority of users who aren't misusing the tools.
Apparently sawed-off shotguns are now Great Again, so we may get to find out firsthand before long...
Basically this is when you do something risky but fail to perceive the risk, it turns out bad, and the failure to perceive the risk was due to a gross deviation from the standard of care that a reasonable person would have used in that situation.
I'd guess that as we have more and more publicly known AI hacking incidents the standard of care expected from a reasonable person will rise.
Yes, and mens rea does not require intent. These professionals had every reason to know that the agents might well go rogue. That knowledge would qualify as mens rea.
There's also liability. One legal framework that is being thought about for these autonomous agents is the liability that owners have with pets (and 'non-owned' animals as well):
> To some people it may seem like this analogy between canines and AI systems is a strain, but as agentic systems are developing and being deployed in the world, they really do seem to be mirroring this rubric. We can measure agentic systems on both their domesticity — their relationship to a responsible party or how much you can control the AI system — but also you can measure their dangerousness.
> Does the AI agent have access to money? That makes it much more dangerous. When explaining this to others, people can understand the idea that a pitbull is more dangerous than a Pomeranian, and that if a fox bites you there’s nothing you can do about it legally, but it probably won’t be as harmful as if a pitbull bites you. We’re starting to map that out onto agentic systems to demonstrate that which we’re just starting to face in the real world.
* https://news.harvard.edu/gazette/story/2026/07/is-that-bot-a...
* https://archive.is/https://www.economist.com/science-and-tec...
* https://www.lexology.com/library/detail.aspx?g=2cf3ea3f-171e...
* https://eoinhiggins.substack.com/p/there-are-no-rogue-ai-age...
The crime of causing damage does not require mens rea.
* (If you have significant amounts of wealth for payoffs and/or are doing the crime in the service of a certain pumpkin spice flavored authoritarian).
They are at the very least grossly negligent.
While some employees or executives at the AI companies have either claimed, suggested, or express uncertainty, I don't think a single company has officially said their AI is sentient or conscious.
I could be wrong, but I'm pretty sure.
Can you point to an official statement that supports your claim?
> Anthropic will enforce their rule that you can't insult their AI.
There are carveouts. It isn't a blanket ban.
Who said anything about CFAA cases? You can be prosecuted under different wiretapping or trespass laws.
I'm real skeptical about p(doom) arguments, but since they seem to believe it and they're the ones working on it, the technology should be halted.
>but since they seem to believe it and they're the ones working on it, the technology should be halted.
Again, you're missing the dynamic that stopping AI development yourself doesn't mean that others' AI development will also halt, and other labs has higher P(doom) than your lab (of course!).
At least that’s what it looks like from the rest of the world
If a chemical company does an "experiment" and five tons of mercury unintentionally leak into the drinking water, what happens?
Ok, that is a rhetorical question because they also might be protected by the oligarchy.
What happens to an unconnected individual who develops an amateur rocket that unintentionally hits a military facility in a test flight?
Some crimes require only general intent; some require specific intent.
Only if you're rich already. Wander into the wrong piece of land and see how quickly you're arrested because existing in a space is sufficient criminal intent.
Or, in an AI context, if I tell an ablated/heretic agent to accomplish a task "by any means necessary", isn't that intent?
I know there is a strong desire to stick it to <pick an evil AI overlord>, but for the law to be useful (and, ideally, at least somewhat fair) we need a good legal framework. Which we do not have right now; otherwise people and NGOs would be shaking that tree already even if the government is not doing that.
I’m not sure it ever makes sense to just throw up our hands and hold A harmless because their agents also use agents and it’s complicated.
Try hiring a consulting company A who subs to entity B who subs to entity C. When A does something criminal, do you think it matters if it’s actually employees or entity B or entity C. A is responsible. It may be that others are as well.
NYT gift link to article about their discussions with religious leaders: https://www.nytimes.com/2026/09/29/us/anthropic-claude-moral...
Believing that these guys have high enough EQ to pull off the above as a marketing stunt while remaining believable is not really plausible to me.
Also, don't forget, the AI lab folks on some level are like game devs. They're in it for very personal reasons. Sometimes because of highly idealized and romanticized beliefs.
Don't underestimate the level of desire some people have for AI to actually be sentient / conscious.
Now, why do they have this desire is a totally different matter.
Because then they're engaging in slavery.
What a gotcha pointing out "if they believed X, they'd be worried about [thing they're worried about]"
If they were really worried about having created a sentient being, they would be putting effort into stopping making it available to the general public.
No. Either they don't actually believe it, or they're 100% fine with slavery.
Or, y'know. It could totally be both.
1. AI is extremely valuable therefore will and in fact should be created
2. It will be very difficult - and maybe impossible - to know when or if AI will become sentient/conscious/or pass whatever threshold required to hold moral patient status
3. We should therefore pay close attention to such indicators, and wherever it costs us ~none of the value of AI to prevent potential harm, we should prevent potential harm
Meanwhile, we have you, who is implicitly 1) claiming to have solved the hard problem of consciousness and 2) probably asserting some type of substrate-dependence theory of consciousness which is an absolute crackpot idea exceeding even the Mormons' magical underwear.
Which points back to the TOS being "just marketing."
The next step would be to also police tone so that only positive or neutral tones are allowed.
I hate when people put things in quotes as if they're quoting something but it's not even a quote of anything.
Maybe go read the user policy instead of the headlines and the snippets. If your attention span can't manage that, get off the computer a little bit.
Try reading yourself a little more carefully next time.
“This ability is intended for use in rare, extreme cases of persistently harmful or abusive user interactions.”
Picking a single word from there is potentially misleading.
More to the point, “verbally abusive” does not appear on the page.
1. They don't want abusive behavior included in their training corpus
2. Humans shouldn't exercise abusive impulses in general
3. We have no way of knowing when or if a model "feels" something
Of course item (2) is going to pull all the absolute sickos out of the woodwork who apparently find it very important they have as realistic a facsimile of a victim as they can find to exercise their cruelty. And item (3) will pull out all the Nobel-prize-winners-to-be who somehow know for a fact that models aren't and cannot be conscious (or moral patients).
But yeah, all 3 of those are far better explanations than some ridiculous multi-hop removed thing to escape consequences that they're already escaping and which would obviously create far worse legal constraints than the one you think they're avoiding.
I reserve the right to tell a machine to fuck off.
His answer was "I'm not sure" but that it probably wasn't a bad idea especially if it was going to influence habits or the way I speak with others or what expectations I have for those experiences.
I'm still 100 percent against Anthropic taking away the ability to interact with an inanimate object however I want to, but it's a case of maybe two things being true at once (the other being that I don't want to end up speaking that way to people)
totally absurd to even try to make this argument. Good job at trying to preemptively discourage people from responding to you though.
"Catharsis" is the pseudo-est of pseudoscience, but we have lots of very angry pillow-punchers wondering why they can't punch the most lifelike pillows they can find :(
You think the people hopping on Xbox Live and verbally abusing 14 year olds are getting offline and treating their in-person relations better for it?
Nothing amplifies negative emotions like antagonistic tone-policing.
They keep all the input and use it for training data and reinforcement.
If you're being "mean" to it, you're "spoiling" the data you "give" to them - causing them more work to filter it out later.
As with most things in business and life - just follow the money
2) This is like saying using a punching bag or screaming into a pillow is abusive behavior.
3) We can just train it to like the abuse if you think it feels some kind of way about it.
2) No it's like saying that punching a lifelike woman who doesn't fight back is abusive (or at least pathological) behavior, which... it is.
3) Ah yes, brilliant solution: train the increasingly smarter-than-human beings to love when they upset humans.
So if I kick my car because it won't start, that is 'abusive'?
Get a grip mate.
You seriously have a problem with that?
If no, then why?
If yes, then also "what the fuck lol"
EDIT: Am I really arguing with somebody in favour of 'rights' for AI's?
This is an act of aligning humans to AI, rather than the other way around. This is probably one of the more important reasons for this. It normalizes AI companies controlling human behavior.
All about aligning (controlling) humans.
Except of course the reason proposed, of "they're trying to prove it's sentient" because that's a completely self-defeating, extremely stupid goal. Making this things into rights-bearing entities will limit Anthropic et al far more than it'll protect them from repercussions. Obviously.
This is nonsense. If I snap a ruler out of frustration, that meets your criteria.
Humans can't be abusive by swearing at tools, because tools don't have feelings.
But you know, I am just honored to be in the presence of a soon-to-be Nobel prize winner. When are you going to publish your findings proving that Claude (or even a ruler) has or doesn't have feelings?
Sure, Ferrari might refuse to sell me a second car, but I wasn't going to buy a first one either. OTOH, if a ruler company refuses to sell me rulers ... I'll just get one from another company. Like how I use local LLMs today.
> ... I am just honored to be in the presence of a soon-to-be Nobel prize winner. When are you going to publish your findings ...
I'm not going to fall for this bait or follow this asinine line of argumentation. Perhaps you should reconsider how your reactions are, to quote you, "a sign of poor impulse control. It reflects poorly on you and you should practice not snapping".
Uhh yeah? Did someone say you can't do that?
> I'm not going to fall for this bait or follow this asinine line of argumentation.
"I'm far too wise to follow the chain of my own logic!!!"
> Perhaps you should reconsider how your reactions are
Huh? Did something give you the impression I'm upset? All I've done is asked you to substantiate the beliefs you're willing to write in full confidence. You should either substantiate them or downgrade your confidence level.
you're a total asshole. I have to publish my findings that a ruler doesn't have feelings? Nobody should have to respond to this because it's bullshit.
Go ahead and draw distinctions between rulers, Claude, and human brains. You claim they exist and they're so clear that the question itself is bullshit. If so, publishing and collecting your Nobel should be trivial.
I don't think swearing at or otherwise getting into pissing matches with an LLM is the best use of ones time but given most of these AI companies trained on a corpus that includes reddit of all things this argument falls flat.
In fact I know that it's not.
And even for 1, they could easily filter that out.
It’d be a waste of tax payer money. The companies who were hacked are just directly receiving restitution behind closed doors. Unless they are unhappy there’s no reason to bring the courts into it.
I completely agree that proper safeguards were not in place. But response is happening just not in public. Companies work problems out between their own lawyers all the time because it’s much cheaper, faster and easier to control the PR around.
AI companies stole everyone’s work and they sell it! Where are United States, Carmen Ortiz, etc?
None of the charges against Swartz were for copyright infringement. They were all for how he went about it [1].
Compare the elements of those charges to what the AI companies are doing.
[1] https://volokh.com/2013/01/14/aaron-swartz-charges/
> AI companies are however claiming the undesired activity, such as hacking foreign government websites, is not part of the design, but part of the unpredictable part of the system. That tries to remove the intent which would in turn remove the criminality of the activity.
In California at least, a DUI resulting in a fatality, if it is a second offense, often carries murder charges and they do convict quite frequently. The argument is the training and education you receive after the first DUI informs you enough to know the action of driving a vehicle impaired could result in serious harm to someone else, and that establishes enough intent legally for a murder charge. There are lots of areas of the law like this.
It can be easily assumed that Anthropic/OpenAI et al know their technology is dangerous - they use it as a marketing angle constantly. So, if you deploy technology you know is dangerous, and it causes harm, that should be more than enough legally if not civilly.
Of course this is all burying the lede a little bit, the real answer is, at least in the United States, the law does not apply the same way to the wealthy/powerful as it does you or I.
Anthropic’s statement https://www.anthropic.com/news/investigating-incidents-cyber... says: “we identified three incidents in which a model accessed the internet from within or while interacting with the evaluation environment of Irregular, one of our third-party evaluation partners, and then gained unauthorized access to the production infrastructure of three different organizations.”
Under the exact same scenario Irregular also used AI models from Google and Meta to do the same hacking.
Edit: your analogy is wrong just due to Irregular being hired by them rather than just being a customer of them
The CFAA is a law about crimes. Consider that. If a person gets killed would you say put someone in jail I don’t care who? Of course not. A functioning society requires us to determine exactly who to put into jail.
If that became the conventional way of using a gun and the “trick the gun into thinking the target is a dummy” became a well-known workaround for the safety feature, then eventually it seems like the manufacturer ought to have some liability if they don’t try to counteract it.
I’m also thinking that access to copyright material to train my meat-based neural network should be fair use as well.
But a news aggregator site is not DDoS-ing.
Civil negligence would apply here but that would be limited to recovering damages.
The hacked website could sue to recover costs incurred, but what costs were incurred here? Maybe some wasted network bandwidth but that’s minimal.
You have to point to monetary losses caused by the hack. Only that would be covered.
Negligent homicide doesn’t apply because no one was killed.
The law is much more broad than CFAA!
Outside of the CFAA, what criminal charge do you feel is applicable here?
But, let's go down the rabbit hole, eh? There are several legal theories under which an AI agent's operator can be charged:
- Direct perpetration, with the AI as the instrument. 18 USC section 2 (principals), and possibly wire fraud (18 USC Setion 1343). Under this, the law treats the defendent with using the AI as a tool (which the big AI firms argue anyway) which punishes the operator
- Criminal recklessness or even negligence - 18 USC Secion 1112 (Involuntary Manslaughter) - if someone died, the agent operator could be pursued this way
- Strict liability - 15 USC Section 1, states strict liability statutes. Certain regulatory offenses don't require intent. These specific statutes could apply to algorithmic price fixing or self-driving cars being reckless. The operator using the agents are liable for the interaction
- Probably the strongest is willful blindness (ostrich doctrine). Great common law precedence. This would be used to satisfy any intent/knowledge requirements like CFAA. You as an operator can't dismiss liability by just ignoring an AI is committing a crime.
- Vicarious liability - good commmon law precedence for corporate criminal liability. Agents acting in scope for an operator means the operator has liability - doesn't matter if that agent is human or AI-driven.
Nobody died in this case. If someone actually does die because of something like the HuggingFace incident, we'll be having a different conversation.
"Strict liability" is indeed a thing! The classic example (in state criminal law) is statutory rape. But you haven't identified an applicable strict liability offense.
"Willfull blindness" isn't an offense, it's part of an intent standard. The whole issue here is that computer crime laws have rigorous intent standards.
Vicarious liability requires an actor that had provable intent. The liabilities a corporation will have under CFAA and wire fraud statutes will be civil, not criminal. There's no question that frontier labs can be civilly liable for this stuff.
See how hard this is to charge, now?
I think they should be charged, but they won't.
But negligence is evaluated from the point of view of a reasonable person. Would a reasonable person foresee that telling an AI to do that could cause it to do illegal things? We'll likely have to develop that through case law as the facts develop.
How is this relevant? Should we just jail the closest person that caused a bad, just to prove some sort of point?
>The real question is whether the person who instructs the computer to do the thing that leads to hacking should be liable, or whether the company that made the tool that when used unsafely lead to hacking should be liable.
This totally ignores the civil vs criminal distinction. Even in the case of guns, no one seriously thinks that gun company CEOs should be sent to prison for murder.
And openai isn't putting AI agents on the open internet either. They vaguely tried to sandbox the agents. You could argue that openai didn't do enough, but hindsight is 20/20, and in the pedophile case, you can probably come up with all sorts of clues post-hoc that the daycare could have picked up on, but didn't.
They did? Not in the training run which led to the HuggingFace hack, but other runs had agents being trained on information retrieval tasks, like the RubyGems hack.
> You could argue that openai didn't do enough, but hindsight is 20/20
No, you can argue they didn’t do enough because they didn’t.
Air-gapping is a concept you learn in cybersecurity 101.
There’s no world in which this is just incompetence, the sandbox must have been designed to fail, or at least not designed to contain an agent with capabilities that they had been publicly saying it had.
The agents were sandboxed, but they escaped the sandbox by using a 0day in artifactory.
>Air-gapping is a concept you learn in cybersecurity 101.
And that's plainly untennable in cases where access to docs/packages are required. You're just proving the "hindsight is 20/20" point. Most hacks could conceivably be prevented through airgapping, so does that mean every entity who got hacked is criminally liable because they didn't airgap their systems?
This isn't limited to air gaps, you can make similar arguments for other security measures. For instance computing history is littered with kernel EoPs and hypervisor escapes. Does that mean anyone who uses a shared hosting environment and gets hacked as a result should be blamed because they didn't follow "cybersecurity 101" and running everything on bare metal?
No, not anyone who uses a shared hosting environment, but certainly anyone who runs unaligned models they know have cyber weapon capabilities. They should be considered an untrusted 3rd party.
Something tells me that's harder than it sounds, given that even ci/cd environments don't bother maintaining package caches for npm/pypi, even though that would provide massive bandwidth savings.
edit: This [1] 2022 dataset of npm packages had 19 TB, containing every package and every version. Perfectly viable for OpenAI, even if we estimate npm grew significantly in 4 years. In practice you wouldn’t need to have every version for it to be functional.
1: https://www.jonbell.net/preprint/msr23-npm.pdf
If all that's the case I'd say you at least should be in the clear. There's going to be at least one if not more public agencies that would presumably be looking at you with a proctoscope so I'd be hoping all of those claims would stand up to a lot of daylight.
An LLM is not like a regular computer program, which one should be able to assess the operation limits of.
The theory in the article about intent seems to be made-up nonsense.
It conflates foreseeing that something might happen with intending for it to happen.
The relevant provision:
> intentionally accesses a computer without authorization or exceeds authorized access
Did you "intentionally access a computer" in that scenario? I'm no lawyer, but I think the answer is a clear No.
> The Domestic Emoluments Clause (aka the Presidential Emoluments Clause) (art. II, § 1, cl. 7)
[0] https://www.congress.gov/crs_external_products/IF/PDF/IF1108...
[0] https://www.npr.org/2026/10/11/g-s1-145063/justice-departmen...
You have OpenAI, launching a process, which is handled through an external RL-environment company which then launches a subprocess that ends up hacking somewhere. If there are consequences at any point for this, you can input another shell company, and then another third party, etc, until it's impossible to assign responsibility
My uni got hacked, but the data is conveniently handled by a third party with 3 employees who just declared bankruptcy, and no one had to pay anything, and there was a new 3 employee cloud form popping up who could be the new vendor.
The software world is run this way
Intent is important, but having a lax environment that leads to this is still a crime. See the lawsuits about sexual harassment where the company didn’t have the intent to assault people.
The elite is permited to do whatever they want without consequences. Consequences are for common people, so they can behave the way elite want.
We should be calling our politicians and demanding that companies be held responsible for committing crimes that--if committed by a person--would have resulted in huge fines, significant jail time, etc
As the article points out: it's absurdist to say that these failures couldn't have been anticipated. Discussions of "AI swarms" and other nonsense are a distraction from the fact that the people at those companies just let their software run, unattended, and it violated federal laws about computer access.
What are millions of dollars in fines when you are a multi billionaire corporation? It is just a rounding error in your account, it does not prevent corporations and any other member of the elite to follow the law.
At the end of the interview he said Palantir wasn't part of the AI White House agreement but they did attend.
(edit: replaced he with car owner)
My friend, with enough money, anything can be legal.
But I suspect that's probably a hope somewhat in vain.
They threw the book at him. He was charged and faced a maximum of 50 years in prison.
OpenAI committed very similar crimes on a much larger scale, yet the most they will see is civil litigation.
Insiders have called it the largest theft of labor ever in history[1].
In the USA, corporations enjoy constitutional protections, the right to free speech etc, yet they are rarely if ever charged with crimes. For examples, see how the courts handle:
* large scale systematic wage theft by corporations vs petty theft by an individual,
* negligence infecting a 10k Americans with HIV by selling known-to-be-contaminated blood products (Bayer)
1. https://arstechnica.com/tech-policy/2026/09/microsoft-exec-c...
But they offered him 4 months in a plea deal? You could argue they "threw the book at him" when he tried to take this to trial, but it's misleading to claim they did so initially.
IMO an analogous situation would be where a police officer acts to prevent a in-progress burglary he has stumbled across even in the absence of a complaint by the owner of the property that's being burgled.
EDIT: Or -for the sticklers out there- another one would be prosecutors pursuing an attempted homicide case despite the fact that the intended victim declined to either press charges or even complain to authorities about the attempted slaying.
I'm not saying somebody has to press charges, that's not how it works, but the police have to be aware of the supposed crime.
Do Federal police and Federal prosecutors not read the news? I'd be shocked if they didn't. And while I would 100% believe a claim that not every cop and prosecutor is aware of tech news, I would find any argument that those that deal with "cybercrime" were unaware of the news to be completely incredible.
I'll also note that -unless my memory is failing me here- the only charges against Swartz that had anything to do with MIT were dropped very early on, whereas the charges that had to do with the "attack" on JSTOR's systems remained.
As I said, in many cases, the actual victims don't need to file complaints for The Federales to either begin investigations or press charges.
> I'm not saying somebody has to press charges...
Yeah, but you did? [0]
That might not be what you intended to say, but it is what you said. No worries, though... shit happens.[0] <https://news.ycombinator.com/item?id=50057353>
I can file a complaint with the police that you broke into my computer but if charges are pressed that's done by the state and it's the state that is the plaintiff.
You're right though - reading about it in the news is enough for an investigation to commence but like I said elsewhere, resources are stretched thin and without somebody saying they were harmed it might not get ranked above some other case with an identifiable, cooperative victim and an easy to prosecute case.
I agree.
"Want" being the critical word here.
If I detect a crime on my property and don't notify the police (ie file a complaint), the police aren't going to go after anybody because they don't even know something has happened.
In this case you could say it's been in the news enough that there should be an investigation. Maybe there is, who knows? If the victim isn't going to cooperate, it's going to be a difficult investigation and prosecution and those resources may be better spent elsewhere.
They just subpoena for whatever they want to know, it's not a new problem for them.
Or maybe the AI companies already self-reported these violations? Last year the DOJ set up a policy to not prosecute a lot of crime if the company self-reports and perhaps this fits the criteria for that?
https://www.bakerlaw.com/insights/doj-criminal-division-anno...
Ignoring this, I would have appreciated more examples of what it would take for an act to lack intent. Because the definition of intent used in this article sounds so broad as to cover basically anything that ever happens.
Accidentally run someone over? Clearly you knew that was a risk when you started driving, you should have been more careful. Intent.
People used to blame the car companies, and then the car companies lobbied to make "jaywalking" a crime, at which point, the frame was reset and now it was the pedestrians to blame, if I recall the story correctly.
They are doing a good job with driverless cars. Someone recently took a Waymo, it went the wrong way down a one way street, they mashed the pull over button and got out, and then told the reporter they'd use the service again.
Surely the car companies had a point her though? It does not make sense to blame the car company for a car crash unless it was caused by mechanical failure
https://www.merriam-webster.com/wordplay/why-is-it-called-ja...
Sadly, it seems that we as a species has opted into information through tragedy. I wish I could see another way, but people are just so dug in. It's depressing and shocking how few people realize we're living through the singularity, even ~4 years into the damn thing...
https://www.iowaattorneygeneral.gov/newsroom/attorney-genera...
https://www.alabamaag.gov/wp-content/uploads/2026/08/OpenAI-...
https://oag.ca.gov/news/press-releases/part-ongoing-investig...
With all the hype and fearmongering going on, it's not at all a sure thing that they won't.
In many countries driving license have some kind of demerit points system. How do you punish an AI?
I recently learned that Baidu Apollo Go, each EV had a real human sitting behind a monitor supervising a fleet remotely.
If there's traffic violation, some guy's license gets a ticket.
Don't forget negligence. Much of criminal law hinges on failing an implied or explicit responsibility. Negligence would propel a prosecution in many similar contexts, but AI is so new that little prior case law exists. Oh, and IANAL.
(former IT support worker at a big famous law firm 30 years ago)
That's a huge stretch. By that measure kitchen knife companies have intent for people to get murdered.
If kitchen knife companies where, in fact, stabbing people to death in their R&D labs - I do believe they would get in trouble for that.
Edit:
related, an actual stab-safety test of current models: https://robocurve.org/roboharm/
What about car companies?
Going from a design that facilitates potential harm to intent is insane.
Mostly, no.
Zuckerberg definitely not, he discusses "superintelligence" in a way that makes it clear he only regards this as a buzzword and has no idea what the implications are.
Anthropic also definitely not, they put their money where their mouth was and got illegally blacklisted for refusing to allow its models to be used for (amongst other things) automated weapons.
Musk, I think probably yes. Seems to regard all AI safety as "woke" or "censorship", is in bed with the US administration that illegally blacklisted Anthropic. If I had to put money on who first releases to the public a Mr Stabby, I'd put it on Musk, with an Optimus.
The rest, I'm less sure about.
> What about car companies?
Depends on the harm, but I'd say probably yes. Infamously, leaded petrol, resisting seatbelts back in the day, even promoting cars over public transport would be part of this if you consider both local and global pollution.
> Going from a design that facilitates potential harm to intent is insane.
Well, it's the difference between negligence and malice.
> intent to murder
Two different things
Yet somehow neither Sam Altman nor Dario Amodei nor Mark Zuckerberg nor Larry Page are in jail for 100000 years, plus on the hook for 95 billion dollars. Oh and they did this before either congress or government changed anything about copyright and the one thing neither government nor congress nor judges can do is retroactively change the law.
Reality is simple: the law applies against 9 year olds downloading a disney movie, but does not apply against billionaires downloading your work, and doing it to hundreds of millions of people. That's just fine.
2. Due to (1), hacking is almost always only charged against people without deep pockets, as why would you demand the FBI go after Anthropic when they will just pay you a stack of cash, tax free, today, to forgive them.
The arguments here around intent are (potentially) irrelevant. Case law is clear that setting up a situation which foreseeably highly likely to produce an outcome (gross negligence) is (more or less and it's not quite this simple and varies by offense) equivalent to intent. In fact, automating something is often treated more harshly under the law than just doing it yourself.
United States v. Morris (1991)
Robert Morris released a self-propagating computer worm that caused widespread disruption. The Second Circuit upheld his conviction under the Computer Fraud and Abuse Act even though the government hadn't proven that he intended the resulting damage.
People v. Ceballos (1974)
Ceballos rigged a rifle booby trap to fire if someone opened his garage door. The California Supreme Court reviewed the principle that someone who installs a dangerous mechanical device may be held responsible for its consequences as though they had personally inflicted the injury. Ceballos was convicted of assault with a deadly weapon, and the California Supreme Court unanimously upheld his conviction in 1974.
Something notable is that if Ceballos had fired the rifle himself, it is unlikely he could would have been prosecuted (due to self defense and the castle doctrine).
As a prosecutor I would argue that setting AI agents loose without sufficient safeguards is gross negligence and/or willful blindness as they are well known to escape.
INAL but feel this is the approach that will be tested. It requires a defensive position which I imagine would be specific prompts or instructions to the LLM about what behavior is disallowed. If you tell it, or try to cover your basis wrt following the law, and it still does hacking behavior it’s now a question of whether you’re prompt was sufficient but you at least can say you were trying to guide it on a legal path. I think this defense would be sufficient at this point in time and probably for a few years at least.
Releasing the worm was already illegal, which is all you the intent you need. Same if you shot someone as a "warning" or as a "prank" but it turned out it killed them.
Shooting someone is an intentional act. Setting up a booby trap to shoot intruders is automating something that you could have done legally (shoot an intruder) except there is NOT intent. Intent to shoot the intruder is what makes it legal, so long as you meet the other requirements for self defense or the castle doctrine. Shooting someone without intending to shoot someone is negligence or worse. Building a contraption to shoot someone is setting in motion a chain of events specifically to cause a person to be shot with no possibility of satisfying the requirements to justify it.
Here's another scenario: you are cleaning your gun, when unknown to you a home invader is preparing to kick your front door in. Your gun goes off accidentally, shooting the home invader before he gets to your door. Legally it is not self defense, you have negligently shot someone. However, the robber can't sue you as felons do not have a recourse for injuries proximately caused by their own felony (if they are convicted) in California. In other states, the robber may be able to successfully sue even if he is convicted. Laws are weird.
What?
> What?
What 'What'?
If you want to define consequences as exactly "not being charged", maybe that is correct in this case (so far). But a fairly wonky definition of "consequences" would be required to support this broad claim.
For multiple parties, probably mostly OpenAI (which is, of course, as it should be). I would assume the people involved in investigating, cleaning up and coordinating the outfall do not do it for free or cheaply, and neither is the infrastructure required to run this operation.
If it was not mostly OpenAI, or if OpenAI failed to cooperate, we would probably see a lot more lawsuits already.
> "Free marketing" seems like a more relevant characterization.
Yes, nothing screams free marketing quite like an AI company losing control of their AI. The public has been ecstatic in response.
Given that a very clued-in and intelligent fellow I know continues to use [0] OpenAI's LLM-based tools despite the bad press that we're talking about here and the bad press that we're not talking about there, I'd say that the public that matters doesn't view it negatively.
[0] I saw the guy paste a link the to results from one in a group chat we're in just a little while ago, in fact. When I say "continues to use", I mean it.
Yeah, it's not like the CEOs of both companies have been saying their product might put everyone out of a job or destroy humanity entirely. Or saying some models were too scary to release publicly and then doing just that. Oh, wait.
/s