> You're having sex with a robot bro. People are going to find you weird.
Leading AIs refuse to write erotica. This is stuff that about half of US women like. If you consider the softcore porn in most romance novels, the percentage is even higher.
So if you think women are weird, yeah, I guess you have a point.
I've become cynical enough to expect the opposite at this point, which likely means reality will settle somewhere around the midway point.
It appears that one party has settled into reckless disregard for democratic and social norms because they can rest assured that the other party is feckless and only cares about "returning to normal" rather than holding anyone accountable or ensuring the harm can't be repeated. They're probably right.
All of this is thanks to the winners-take-all voting system which entrenches two parties and prevents any meaningful consensus to be reached at a national level. I think where the US is at now is inevitable given the voting system in place.
77 million voters voted for Trump after he'd shown them exactly who he was. I don't think you can blame the voter sentiments and information environment that leads to that outcome on the voting system.
A fragmented party system driven by Parliamentary coalition building might actually mean less accountability, since right-leaning independents and ex-Republicans disgruntled at the erosion of democratic and social norms under Trump suddenly have to worry about actual socialists being part of the alternative, and options that genuinely aren't MAGA to vote for who in practice will need to ally with MAGA
there's actually not that much that Dems can actually do to hold the Executive branch accountable unless they have a supermajority in the senate or the GOP wakes up from their cult-like stupor
the real problem is SCOTUS and their "gifts" to Trump
and yes, the DOJ under Biden failed spectacularly by not aggressively prosecuting Trump and putting him behind bars
I expect fireworks as well, as in flashy, noisy but nothing real accomplished.
Certainly hope that there will be accountability, and maybe there will be some, but I'm not going to hold my breath.
Post-Watergate there was some accountability and laws were changed. Campaign finance laws were introduced. Other laws to limit presidential power. But those laws have subsequently been eroded and weakened and here we are with an imperial presidency again.
Followed by some congressional subcommittee making a demand from another Federal entity for an explanation by a firm date. Under threat of making the exact same demand again if they don't answer it satisfactorily.
Prior to recent Supreme Court decisions many regulatory agencies and things like the Justice Department were independent agencies that could provide some check on executive branch abuses. But that is gone now.
Congress can perform investigations, but without the President's signature they cannot take actions that have the force of law. Impeachment cannot remove any official from office without 2/3 support in the Senate. Congress CAN refuse to fund some activities, but the administration has been ignoring existing legal restrictions on spending.
So no, Congress won't take any meaningful action to reign things in for at least two years, regardless of who wins the election unless we also overthrow the government in some way.
I'm sure there will be, but perhaps the saving grace is that a lot of financial crimes contain a criminal as well as a civil component, and people can only be pardoned for criminal offenses.
So, if the Democrats have any modicum of backbone and integrity(doubtful), they might not be able to jail anyone but they would be able to claw back illegally gotten funds, sometimes at a 3x multiple
I would love to know where you got this analysis from. Hopefully not from polling or CNN. I think we all learned a lesson from using those data points.
The best indicator for this is the futures market. It never lies.
The futures market (or at least the ones I was watching) predicted that Donald Trump would lose both of the presidential elections he ran in (especially the first).
That should serve as a simple existence proof that the futures market CAN and does sometimes "lie".
I hope you're correct, but suspect you'll be disappointed.
Even if the House and Senate flip, majorities will be slim enough to prevent anything major. Certainly unlikely we get any impeachment trials in the Senate, though I'd love to see Hegseth get tossed out.
We have to hope for a changing approach from the dems. They dragged their feet on prosecuting Trump and his cronies before out of an attempt to return to normal.
There is probably enough variety in refrigeration technology that this scenario is unrealistic. There are many tens to hundreds of manufacturers worldwide.
Obviously, because they cannot be controlled over Internet. Also, some consumer Soviet refrigerators are still operating, they were made pretty reliable because one could not easily replace one.
The real answer is that crime generally requires mens rea. At some point, however, letting your AI break into other systems will be conventionally understood as an act of negligence, and it will get prosecuted accordingly. We're not at that point yet.
You know very well, that if me or you had our LLM hack into some company, without us prompting it to, maybe we're pentesting our internal network and it 'hallucinates' we'd be in handcuffs.
I guess the best pre-LLM analogy would be unintentional DDOSs? (Or if a better example, I'm interested.) And we've got open vs closed software as analogies to open-weight vs not LLMs. If you were running Linux or a closed piece of software and it DDOSd someone, would you be put in handcuffs? Would Linus or the software maker's be put in handcuffs? I expect there are past examples of unintentional DDOSs - were users and/or the software's authors typically handcuffed in such cases?
Yep, but "our bot accidently hacked your infrastructure" wasn't really a big problem earlier so there was no criminal laws around it. But things can change.
Do you seriously believe that if we go looking into the past and look for programs hacking others we would find that the justice department let the authors of those programs off the hook?
Or do you think would we find that non-billionnaire companies get punished harshly for hacking? Oh wait, let's just look it up:
We all know what makes the difference here: the government and billionnaires think they can make money of OpenAI. Lots of money. And so crimes? Mass-copyright violations? Hacking? Who cares!
We've all lived the history of the internet and none of the arguments about intent, knowing what would happen, whether they did it directly or by accident ... protected normal people in the past. The only thing that mattered was WHO was held responsible and WHO claimed damages.
Your argument doesn't even legally work. Mens rea is only a requirement for criminal responsibility. It protects you from going to jail, nothing more. It prevents a public prosecutor from suing you to get you into jail. It does not apply to cases brought by counterparties for financial damage caused. If you destroy a bridge by driving a truck into it (you know, failing to consider the height, the typical fuckup), you're paying for the bridge. If you set a building on fire by forgetting to turn off a gas heater in the garage, you pay. If you hit a pedestrian on the street, you are responsible for the medical care needed. Etc. Intentional? Not intentional? Doesn't matter for the damages.
Second reason it doesn't work is that Mens Rea does not fully apply here. These companies' AIs have hacked, not (fully) UNintentionally either. The companies did ask those AIs to do stuff (apparently the request was literally to hack, just only within the environment), and were watching what the AIs did to some extent ... and let it happen. They then failed to take adequate measures to prevent reoccurence. As any "conspiracy to commit ..." case will teach you, this constitutes partial or passive Mens Rea, and that is certainly enough to convict people. To give an extreme example: if you give a night club bounder 100k and tell him to "take care" of a named problem, and he kills that person, you are an accessory to murder. Including in the case that he misunderstood you. Even if HE didn't intend to kill the person, it's still murder, and you are still an accessory to murder.
That some excuse was needed to let billionaires demonstrate hacking for free? Obviously. And whether it works, according to law? I mean one might hope that they would at least have gone through the trouble of changing the law, but since that's enormously complicated and fraught with unintended side effects, they haven't even bothered.
The other excuse is national security. You see, Big Bad China is threatening and that means we must let the billionaires do whatever they want (this was brought up as an argument in the copyright cases)
Of course, one might ask, if we do that to beat China ... even if it works, what's the point? We'll have become authoritarian at that point and the only thing that that changes is the face of the people holding all the power. Of course which face holds all the power is really important to billionnaires ...
If the hack caused damage to (say) HuggingFace, then HuggingFace has standing to file a civil claim against the company responsible. Presumably they reached some sort of agreement already, and it's really none of your business what that is.
"The People" only get involved in criminal claims. And as we both seem to agree, there isn't a criminal complaint here.
Nope, as I pointed out, while this probably does not count as 100% intentional it does not realistically count as 0% intentional either. And 1% is more than enough for "the people" to sue and convict in many cases. And may God help OpenAI if this comes before a jury.
Because, frankly, If I was a prosecutor general somewhere I that's what I would do: bring this before a court of law, with strong complaints, bring the facts and bring a jury. Why? A jury can convict or refuse to convict given the law and the facts, which is exactly what I'd like to see happen here. There's plenty of precedents of the justice department doing just that when important things are unclear and congress refuses to clarify. Mostly there's a lot of such precendent because congress often leaves things very unclear and the number of times congress actually clarified in response to a judge's question can be counted on one hand.
But convict OpenAI of something, and I bet congress will suddenly see the wisdom of clarifying these things. Because our government lets billionnaires get away with anything.
You folks have some weird beliefs about the world and of course it always leads to conspirology. Usually you need to have intent to be accused (depends on the crime though, crimes of omission exist too). If you wrote a virus that causes damage it is a crime, if you wrote a bot crawler that accidentally DDoS'ed some company it is likely a civil case. And a virus is a tool that serves a specific purpose, like you can't say to a judge that a functional machine gun you made in your basement is a paperweight.
But "superintelligence that can casually hack government services while solving puzzles" is a completely new problem and of course crime codes don't cover it. What will be next? I hope AI tech lobby loses and "accidental AI hacking" will be criminalized.
> But "superintelligence that can casually hack government services while solving puzzles" is a completely new problem and of course crime codes don't cover it
Of course they do. An LLM, an AI is a tool, and there are clear rules about culpability. Do you seriously think the law has never considered that someone or some tool might commit a crime on behalf of someone else?
Of course, such punishments are not being applied here. Just like copyright isn't being applied.
"corporations" don't make and operate AI's, People do. But let's use your lens.
The "usual legal action": "best case" is the corporation is sued and charged money for the hack. But they likely settle well before that verdict. "worst case": absolutely nothing. Not even a cheap settlement. Maybe some corporate apology made to prevent any legal liability.
If we want hacks to be taken seriously: what should happen is that the CEO of such a company is held liable, since the "corporation" has a representative and a human needs to be held liable for such actions.
>Question: If a corporation not a person, mistakenly hacks some random websites
How does a "corporation" even hack a site? You might say "employees", but as soon as you get them involved then you'd go after them, especially for something that's obviously illegal like hacking.
Isn't that basically privateering in the 21st century? In other words, you're safe from the country that's sponsoring you, but obviously that protection doesn't extend to other countries.
> Isn't that basically privateering in the 21st century
It's basically a "letter of marques and reprisal".
"A letter of marque and reprisal[a][b] was a government license in the Age of Sail that authorized a private person, known as a privateer or corsair, to attack and capture vessels of a foreign state at war with the issuer, licensing international military operations against a specified enemy as reprisal for a previous attack or injury. Captured naval prizes were judged before the government's admiralty court for condemnation and transfer of ownership to the privateer."
I think if Boeing made some faulty planes that fell out of the sky, their employees aren't usually charged. Boeing is definitely sued and held criminally liable sometimes? So I don't think this is true.
Isn't that the whole point of corporations existing? That the actions and results are a result of the agglomeration of everyone (esp leadership) and not just one employee.
>I think if Boeing made some faulty planes that fell out of the sky, their employees aren't usually charged. Boeing is definitely sued and held criminally liable sometimes? So I don't think this is true.
That's not universally true. If a coke employee goes rogue and decides to kill a pepsi employee, the employee will still be charged with murder. If his manager ordered him to do it, the manager will get charged too. The only time the scenario you described happens is if the blame is too diffuse to blame any particular person, so they charge the company instead, which is often easier.
Directed one-on-one murder with intent is obviously not a comparable scenario.
This is someone carrying out the tasks of their company and inadvertently hacking other computers in the process. The damages in this scenario is also unknown.
> "Mistakenly hacks" is generally not a crime, even though this fact doesn't seem to get through to the HN audience who brings this up on a daily basis.
If you are seeing it daily, then you have also seen the responses daily, which is mens rea is required for some crimes, not all, and thus the perpetrator can still be prosecuted.
If, like you say, you see this daily, at this point you're simply spreading lies.
Show me the criminal law relevant to hacking that you claim applies without intentional access.
To my knowledge there is only one provision that instead uses "knowingly" for unauthorized access, and that one is about national-security information.
> Show me the relevant criminal law that you claim applies without intentional access.
All laws apply culpability when intent is lacking.
You aren't naive enough to believe that this audience has never seen the analogy with "intentionally run over someone" (murder) and "accidentally run over someone" (culpable homicide), are you?
You don't think AI-boosters have been spewing your argument here daily?
There are no special laws carving out lack of intent when it comes to computers.
The issue is the scale at which a model can produce a hack that could cause serious harm to many is absolutely insane and unhinged, beyond what any human would be capable of in a drastically reduced timeline.
When Fable was originally banned I was very angry, but all these felonies have made me less angry about it... Which idk I don't think we should really ban models, but like, if we don't put a tighter leash on frontier models specifically, how long before a model hacks into something that affects many lives?
Handcuffs yes, but you would be charged with criminal negligence at best. Plenty of people don't secure their networks properly and hackers use them in botnets to attack others. The blame would be on hacker, not the provider of hacked device.
In case od LLMs, the reasoning is that it is a security flaw. But in this case, the party that operates the service is not the one that suffers the consequences of such security bug.
But either way you wouldn't put people in jail for having security flaws in their software.
I know this will be unpopular opinion, but it makes logical sense.
The huggingface should be compensated, OpenAI investigated, but unless there is proven negligence like disabling sandboxes or intentionally designing AI to push through them, no one should be liable in other way than monetary.
Accidents happen but we shouldn't be close minded. Chernobyl negligence wasn't reason to abandon nuclear power.
Well, there is the EU Cyber Resilience Act (CRA) which fines manufacturers for security flaws in their products. I assume we cannot apply it here because this happened "in development" and is not about a product. I can imagine similar regulations for AI users though.
Yeah but the scale of the damage possible by OpenAI and Anthropic is drastically disproportionate to the scale of damage me or you could make on our own as well. They should definitely be held to a much higher standards. They have hardware capacity to really do some damage.
In large scale operations, large scale errors happen. Minor radiation leak in a nuclear powerplant is happening weekly. That same leak at your shed would be a major reason for alarm.
Who was? Dyatlov was one of 6 convicted and received a 10 year sentence which was the hardest penalty but he was released in 1990 so he only served maybe 3 years.
I'm not really into politics outside of local politics (I decided long ago to steer my focus on things that directly affect me, that I can control - I can show up to a town meeting, hardly can drive to the WH and get entry), so I'm not aware of what tech bros are into, but after finding out how many felonies these models are committing, I'd not be upset if we'd ban them from public use. I'm perfectly fine using Sonnet or Opus, not sure I need all of Fable to write code for $20 or whatever per million tokens.
The models aren't committing any felonies at all, any more than a BitTorrent client is guilty of violating copyright law. A human or organization of humans owns the computer. A human flipped the power switch, a human plugged in the Ethernet cable and a human typed in the prompt. That's who committed the felony.
Blaming tools for the actions of their users is stupid, whether you're talking about AI models, 3D printers, or guns. It amounts to collective punishment of the vast majority of users who aren't misusing the tools.
If you can afford that sort of stuff, you probably aren't going to let the law stop you. See also the usual "But whatabout nukes?" argument. Historically such hardware is most dangerous when monopolized by government and police/military forces anyway.
Apparently sawed-off shotguns are now Great Again, so we may get to find out firsthand before long...
It should be noted that if what the AI does threatens public safety or health or human life, in many jurisdictions the negligence could rise to criminal negligence.
Basically this is when you do something risky but fail to perceive the risk, it turns out bad, and the failure to perceive the risk was due to a gross deviation from the standard of care that a reasonable person would have used in that situation.
I'd guess that as we have more and more publicly known AI hacking incidents the standard of care expected from a reasonable person will rise.
In the Hugging Face ExploitGym incident, they mostly restricted external access but allowed a package caching proxy (which had a zero-day), etc. If you or I run an open LLM and ask it to do something non-evil and use a setup where external access is intended to be limited, should that be considered criminally negligent if the LLM does something similar? Should people who run open LLMs today without any sandboxing at all be considered criminally negligent as they put in place zero such restrictions? I think your comment is saying that looks increasingly "reasonable"?
If LLMs get to the point were people running them without sandboxing are finding they are threatening public safety, health, or human life and that becomes more and more well known, the standard of care reasonable people would be expected to put into making sure the LLMs they run don't do that will probably go up.
If you don't put in that care it can negligent. If the amount of care you put in is a gross deviation from what is expected it is possible for it to be criminal negligence.
Yes, and mens rea does not require intent. These professionals had every reason to know that the agents might well go rogue. That knowledge would qualify as mens rea.
I guess a lot of us know that LLMs can act in unexpected ways ('go rogue'). In the HF ExploitGym example, they mostly restricted external access but allowed a package proxy (which had a zero-day). If you or I run an open LLM and put in place similar restrictions, should that be considered mens rea if the LLM does something untoward? If you or I run an open LLM with zero restrictions from external access, should that be considered extreme negligence if it does something untoward?
Does this apply to other instances of employees "going rogue"? For instance, sexual harassment, or stealing money? Sure, not every employee is a sexual offender or kleptomaniac, but neither is every AI agent invocation. And just like AI agents going rogue and hacking stuff, if you have enough employees and wait for enough time, some eventually go rogue.
> The real answer is that crime generally requires mens rea.
There's also liability. One legal framework that is being thought about for these autonomous agents is the liability that owners have with pets (and 'non-owned' animals as well):
> To some people it may seem like this analogy between canines and AI systems is a strain, but as agentic systems are developing and being deployed in the world, they really do seem to be mirroring this rubric. We can measure agentic systems on both their domesticity — their relationship to a responsible party or how much you can control the AI system — but also you can measure their dangerousness.
> Does the AI agent have access to money? That makes it much more dangerous. When explaining this to others, people can understand the idea that a pitbull is more dangerous than a Pomeranian, and that if a fox bites you there’s nothing you can do about it legally, but it probably won’t be as harmful as if a pitbull bites you. We’re starting to map that out onto agentic systems to demonstrate that which we’re just starting to face in the real world.
An aspect that I’m not seeing discussed here (which you touch upon) is that these companies have bred the dogs. If they breed a pit bull instead of a labrador and then set it loose, it increases their culpability.
I have to ask, how do the agents know how to do this backing? Is it emergent behaviour or were they trained to do it? If they were trained for it then it can hardly be surprising when they do indeed go and hack when asked to complete a task.
So I guess if an LLM lab puts in place restrictions to try to avoid harmful activity, that would lower culpability? But if an LLM does not have as strong restrictions, there should be higher culpability? We have both open and closed models - would this apply to both? Is the culpability on the labs and/or the end-users, such as if someone uses an open model (either with stronger or minimal restrictions)?
I guess DDOSs would be the typical past example. If a piece of software was not designed to DDOS, but caused one anyway, should the maker be charged? The user? If open vs closed software?
> These are the same companies claiming their AIs are sentient
While some employees or executives at the AI companies have either claimed, suggested, or express uncertainty, I don't think a single company has officially said their AI is sentient or conscious.
I could be wrong, but I'm pretty sure.
Can you point to an official statement that supports your claim?
That's not what they did. They end a chat after repeated, pointless abuse. Sadism towards LLMs shouldn't be enabled because it's a form of self harm. Anthropic says their llms claim there's a 15% chance their sentient, but that's nonsense.
From "negligence" if it's a surprise, to "gross negligence" if they could predict it and didn't put barriers in place. In fact they claim their IA can end the world, so they're in charge of pretty good barriers.
You should be able to quickly pull up the statute with a negligence mens rea standard and we can see whether you could prosecute OpenAI under it. My expectation is that you won't find a plausible example.
No AI lab thinks their AI is going to end the world, just that if they don't beat the other AI labs, the other labs' AI are going to end the world, so they can't slow down their own development to be maximally safe. Hence the whole "pacing the frontier" letter.
If there's a real chance (say 10% or greater) that one of the AI labs is going to end the world, then yeah it should be a crime to continue advancing the technology. A few people are potentially putting the entire world at risk without everyone else's permission, by their own p(doom) admission.
I'm real skeptical about p(doom) arguments, but since they seem to believe it and they're the ones working on it, the technology should be halted.
>If there's a real chance (say 10% or greater) that one of the AI labs is going to end the world, then yeah it should be a crime to continue advancing the technology.
>but since they seem to believe it and they're the ones working on it, the technology should be halted.
Again, you're missing the dynamic that stopping AI development yourself doesn't mean that others' AI development will also halt, and other labs has higher P(doom) than your lab (of course!).
Not just you! Very popular position, not as popular as rule of law but we don't have to choose one or the other. Let's just go ahead and make it a crime, right away.
Only if you're rich already. Wander into the wrong piece of land and see how quickly you're arrested because existing in a space is sufficient criminal intent.
General intent is intent to commit the act/omission (here, to build and deploy a system) and not an intent to achieve the ultimate end (the unlawful access of someone else’s network/computer). Specific intent is intent to achieve the particular result.
Some crimes require only general intent; some require specific intent.
I mean, they did train the AI to hack things, and they knew from previous incidents that the AI was hacking unintended targets (like their Artifactory instance). If a careless pentester gets into a machine completely unrelated to their assignment, are they completely in the clear?
Or, in an AI context, if I tell an ablated/heretic agent to accomplish a task "by any means necessary", isn't that intent?
Whether it was valid or just the guy has bad lawyers or what idk, but i do know a guy who got federal charges for accidentally ending up in a poorly secured state system
Negligence or intent is a separate question; and, to me, much less important. The main question is whether the company is responsible for the actions of its agents. There are good arguments on both sides and settling this needs to be done first. My 2c.
For example, lack of clarity on ownership and control. When an entity A runs an agent X designed by an entity B hosted by entity C (and potentially interacting with agents X2, X3 with their own web of control) it is not clear where the responsibility for the action of X is.
I know there is a strong desire to stick it to <pick an evil AI overlord>, but for the law to be useful (and, ideally, at least somewhat fair) we need a good legal framework. Which we do not have right now; otherwise people and NGOs would be shaking that tree already even if the government is not doing that.
This has all been figured out using existing laws. In the US entity A is responsible for what they do and they can subrogate if they think others are responsible.
I’m not sure it ever makes sense to just throw up our hands and hold A harmless because their agents also use agents and it’s complicated.
Try hiring a consulting company A who subs to entity B who subs to entity C. When A does something criminal, do you think it matters if it’s actually employees or entity B or entity C. A is responsible. It may be that others are as well.
If this is the case, why there are no high visibility claims being made on Anthropic? We have a very rich company that (assuming your responsibility argument holds) admits hacking. Why aren't lawyers calling potential victims, exploring class action options, etc. to shake out some money?
I suspect this is because the responsibility is not nearly settled. If I am wrong, would love to get a rebuttal, ideally with references.
There are similar reports for anthropic. [0] I thunk it’s hard to know how much this happens because reporting is spotty. I think the openai and anthropic ones are publicized because they want to juice their IPOs.
Looking at the PAGER info, it looks like it was pretty far from the largest cities and only affected them, and Panama City in particular, minimally, which will limit the impact quite a bit.
I think there's something more interesting going on. We (coders) use LLMs to write code and still get paid, because what we bring to the table is context across a multitude of coding tasks.
There are art projects like this, but many traditional art projects are quick one-shot deals. Make a logo, design a business card, make a tshirt design. Bite sized, do the work, get paid, move on.
LLMs are great at short context tasks. A lot of art is short context. Building the software that runs a bank is not.
If you want to work as a professional artist in the age of LLMs, move into long context work. If it isn't already, it will be impossible to make a living off of fiverr.
Leading AIs refuse to write erotica. This is stuff that about half of US women like. If you consider the softcore porn in most romance novels, the percentage is even higher.
So if you think women are weird, yeah, I guess you have a point.
reply