I wouldn't go that far. Assassin is giving the kid too much credit. Arsonist I think would be more appropriate. Maybe 'terrorist' under the strict definition of someone using terror as a means to influence policy. Otherwise an enthusiastic anti-AI activist
Let's not pretend Altman is a random innocent bystander.
We may object to someone trying to get him to a state of incandescence as an act of unjustified violence, but Altman is definitely a member of the tech elite, something as close to aristocracy as we have in our times.
How the future will have written the current events, I cannot say.
Let's look up the dictionary definition of terror, shall we?
> violence or the threat of violence used as a weapon of intimidation or coercion
> especially : violent or destructive acts (such as bombing) committed by groups in order to intimidate a population or government into granting their demands
Now, this could apply to, say, US actions in the Middle East, Iranian actions in the Middle East, MAGA riots, these protesters, the anti-Nazi Resistance and Yugoslav Partisans, pro-Nazi violence in pre-Anschluss Austria, the Suffragettes, etc etc.
Which one you cart out the scary label for is merely a reflection of your ideological biases. Amongst the same notes, the radicalization you see in others can be read as a reflection of what has already become normalized in your mind.
For my understanding: aren't "terrorists" meant those who target random bystanders to make a point? In this particular case Altman is no bystander (but his family is).
There's no consensus on the definition of terrorism though, and it's so broad that you could use it for calling any kind of violent protest as terrorism.
A thing to keep in mind: when people feel they have no voice they inevitably will resort to violence for their political aims. It's just a fact that terrorism, in its broadest sense, will happen if powerless people don't see any other possible avenue for their political opinions to be taken into account.
Far left radicalization is necessary to counteract right wing radicalization that directly caused it. The right wing radicalized first and right wing millionaires put a lot of money into supporting that radicalization.
Wouldn't the Trump right say they were a counter reaction to the radicalisation of the establishment left? Saying men are women, racial hiring quotas, record levels of immigration while respectable low to middle income manufacturing jobs offshored to China, the gleeful forcasting of America's "founding population/dominant culture" becoming a minority.
The reactionary right needs no more feeding, its belly is already full.
I studied history in a state that used to be part of the Nazi empire. I know how they rose to power. The key factor was von Papen, a weak conservative who enabled them, vaguely the equivalent of establishment Republicans today.
I can also tell you that the pre-Weimar warnings ring a bit hollow when we already have some post-Weimar circumstances, like far-right paramilitaries patrolling cities and shooting dissenters.
Political Violence has already undermined law and democracy. Right now, ICE goons are brutalizing a Latino family somewhere. Right now, a war with no democratic backing rages on - a war that has already seen a school bombed.
Why is it that one dude bombing a CEO's house is worthy of so much more condemnation than the above? That, to me, feels like a more interesting question to answer, because of what it says about the ruling ideology and how it's normalized its own violence.
> Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails.
This is the only concrete prediction in the entire essay.
And it simply cannot happen. For one, you will need billions worth of compute.
Why should we believe that a scaled out version of something that happened a few months ago "simply cannot happen"? How many dollars of compute do you believe were available to the swarm(s) behind the OAI-HF, German wiki, and Rubygems incidents?
Well a big reason that we criticize OpenAI for that is because they were the ones giving it access to the massive compute necessary for the LLMs to think. If they had been responsible about their experiments or what types of workloads they allow their LLMs to operate, it wouldn't have happened. Very few companies could enable those workloads.
Well sure, but access to that compute is gated by simple credentials (like API tokens). Those can be hacked.
Imagine for example if a model hacks into ~every Linux computer on the internet using an 0day and steals their OpenAI, anthropic and openrouter credentials. It now has access to billions of compute and the only way to fully stop that is for multiple major providers to shut down services entirely. That's already well into "billions of dollars of damage" territory.
> How many dollars of compute do you believe were available to the swarm(s)
At least two OOMs more than the dollar value of the damage they'd caused. (Also, as an aside, IIRC, the wiki servers weren't breached; it was just a lot of spam.)
Sure. And there's an OOM more compute coming online in the next year or two, while models at a given capability are getting cheaper. "Two OOMs" of scale relative to the HF swarm seems like a bit of a red herring to me, but also within the realm of possibility.
The Internet is big, but one can do quite a lot of damage with ordinary bots and worms that exploit individual widespread vulnerabilities, which LLMs are perfectly capable of writing. Most of the damage also doesn't rely on hitting every long-tail website.
I'm honestly not that concerned about cyber impacts of LLMs relative to other impacts. I just don't like to see the whole concept of being worried dismissed as obviously baseless on the basis of one pretty shaky scale argument.
I like to replace thes AI text with "virus manipulation"
"Given the acceleratung rate of virus manipulation in labs, its my worry that in 6-12 months a virus could scape a take over the world and collapse health systems."
If a CEO of a health company was saying this, the reactions would not be that chill.
The worst failure of our society is to call this technology AI, intead of something line "artificial general automation". The formes allows the creators to be somehow less responsible of the consequences. The later clearly moves the responsibility to the creator/user.
Your point was that its an unfair comparison because AI its a race. My point is that bioweaponds are also a race, but a less public one. You dont have the equivalent of Dario publishing an essay every month.
Hell, put an app on the app store (or dozens of apps on the app store) and youve got a massive network of computers with tons of resources right there if you can get past the scans and reviews.
Or doorbell cameras or IP cameras or or or or or
There's a lot of shitty stuff connected on the internet that up until now has been a feasible target for hackers but still required "effort" to set up and get things going. Not hard to imagine a self replicating slime mold of a botnet running on every device held by a Grandpa Joe because they thought "Candy Rush" is what they wanted to download
"Persistent botnet" here does not need to be the full-sized LLM, nor does it need to run at full scale inference to be a huge pain in the ass.
Yeah, I don't get it. Are they imagining this happening just with the open weights models running on however many GPUs the bad actors can cobble together?
For now, all the scary hacking things still require an API key to one of the LLM providers. Surely they should take some responsibility for how to turn off the tap.
Currently Qwen3.8 27B is roughly on Opus 4.6 level. In at most a year given the current pace, you could probably run such hacking bot nets out of a reasonably small local server, bootstrapping by hacking or acquiring login credentials for more compute.
I take it you haven't studied the details of the HuggingFace hack. It was millions of dollars worth of rogue compute running for months before anyone noticed, and THOSE agents weren't even really trying to evade human detection.
I've followed it enough to see the argument go in this same circle over and over again. The agents weren't "rogue", they were a neglected experiment by OpenAI who likewise allowed them to keep spinning GPUs without question.
The LLM vendors need to know who their high spend customers are, not allow malicious workloads, and especially not when those workloads are coming from inside the building.
"Just don't make mistakes" is naive. You have no appreciation for the scale of agents being run right now, finding the rogue agent is a needle in a haystack operation.
lolwut? This is Hacker News of all places do people not realize how much memory, and more importantly bandwidth, these systems need to work? The idea of a distributed botnet of AI using the compute of its victims to continue its inference is pure science fiction given how LLMs actually work.
> The idea of a distributed botnet of AI using the compute of its victims to continue its inference is pure science fiction given how LLMs actually work.
An attack like this doesn't really need the exploited computers to run inference, do they? If I was an LLM bent on destruction of the internet, I'd be writing programs to run on each computer, not turning each computer into an LLM itself.
A few programs to break in, install themselves and remain asleep until they are needed, another few to spread through grabbing every OpenAI, GLM, whatever key, another one to remain asleep on computers (whether hosted or desktops) that have adequate GPU, etc.
The GP was referring to the AI 'living off the land' so to speak by using its victims compute to avoid being shut down which is clearly laughable.
More to the point, so many people in this thread are making completely contrived and outlandish stories up about how AI might try go ruin our lives without any evidence backing them up in any way. It is hysterical. This is the most important technology in our lifetimes and people want to freak out and turn it into the next nuclear power, with progress banned in all but name.
It is really mysterious how awestruck folks are at the HF attack. I mean just watch one modern agent (qwen 3.8, deepseek 4, glm 5.3) rip apart a coding problem and nearly destroy your computer in the process, it's a wonder it took "months" and "millions" in the first place.
OpenAI has too much money, a common post-growth-stage issue that leads to pursuing a million stupid things with no clear plan. Like running a bunch of agents for months without any idea how to keep track of progress.
I agree with the idea that it's not probable but I do think it's important to point out-- we only need these elements to run with the bandwidth they have and the token rate because we want to see things in human-scale time. But slow things down to a 1tok/sec doesn't matter to this hypothetical anti-aligned LLM. Time is, after all, relative, and it's not like LLMs give a shit how long something takes. They don't have squishy stupid organs that fail after a certain amount of time, or those pesky glands that emit impatience hormones.
But yeah you're not going to be able to shard out the terabytes of Fable weights that are needed to run inference without addressing some fundamental physics problems.
How would you identify the computers to unplug? On whose authority will you unplug? How will anyone communicate when AI has the ability to intercept and impersonate?
Kids lack proper impulse control and sometimes make bad decisions if left unattended. It's preferable if we restrict certain dangerous actions from them.
Yes, LLMs can be dangerous for your own health if misused. Yes, I'd be furious if I was 14 and got my Claude access revoked.
As others have said in this thread, I'd add that this should primarily be the parents' responsibility since they know their child best.
It's not just Claude. It's being cut off from every part of society except a carefully curated "child ghetto". In which you stay until the day you turn 18. At which point you're shoved out into the outside world. Possibly cut off from all your existing contacts, in fact, since now you're an "adult" and it's presumed dangerous for you to interact with a "child" in any noninstitutional way.
Yes, most of society is online nowadays, or at least is hard to engage in if you can't access the online stuff that organizes it.
> Yes, most of society is online nowadays, or at least is hard to engage in if you can't access the online stuff that organizes it.
I think there's an analogy to physical roads there. Most of society is connecting by roads and adults driving cars, but that doesn't prevent children from participating in the things between the driving.
It's dehumanizing because it feels like you're an incomplete human with no agency.
When I was a teenager, there was nothing I hated more than obnoxious adults deciding for me what I could or couldn't do. I did literally everything in my power to circumvent them.
"Minors" should have less agency than adults, but the state of "minor" vs adult does not follow the plain age. (And of course there exist many dimensions of maturity.)
(In fact, in some educational systems, the final examination in pre-University schools is called "Maturity test".)
It's not dehumanization, they just dread a future where kids can learn from 24/7 tutoring from an entity smart enough to solve Millenium problems instead of from some B student who's also teaching 20 other kids at a government propaganda factory.
Some people cannot grasp how administrations appear to fight cultivation instead of promoting it, and given the unsightly appearance of many administrators, many people got to nurture the idea that administrations would want "masses of ignorant voters" to preserve their status as "elected".
The way they explain it implies they're using this at the sampler level and not trained into the weights themselves. So unless you're using an inference library that does this, the open models will not have this kind of a watermark.
> When watermarking is used, choices are still made at random, but the source of the randomness is different. Instead of using an arbitrary random number generator to pick the next word, watermaking uses the key and a few words that come before to settle what word the model should pick.
> the watermark only changes the source of the randomness used to pick among words.
It's a chicken and egg problem tho, whatsapp doesn't support smaller platforms, and smaller platforms can't grow without app support. I remember that the uptake of Windows Phone 7 and early versions of 8 was pretty low partly because whatsapp didn't support that OS.
I know, but my comment was more along the lines of Windows Phone 7 coming out in 2010 and whatsapp being available a year later, and of whatsapp not having a native app for Sailfish
They could do Telegram at least, which has a non-zero user base (not sure about India, though, which seems to be their target market), and supports third party clients. Rolling their own chat app is... well, good luck with that.
Good that they are keeping it, Kimis way of speaking and conveying some sort of EQ is absolutely the best. The other models might be better at certain things, but nothing comes close to how good Kimi is at understanding language, emotions and reading the room in conversations.
I should maybe also mention that I have not used the later models like Opus or Fable, so my opinion might be a bit outdated.
When I remember that this site even showed Kimi having the highest score at one point https://eqbench.com
They are one of the few labs (perhaps even the only one at this level) that are doing something both unique and useful, rather than simply imitating what the others are doing: https://thinkingmachines.ai/blog/interaction-models/
"protester"