> A concrete example, the ability to remember 10 digit phone numbers was really normal before smart phone, try to push your mind now to see if you are still able to do that?
But would you tell anyone in the early 2000s to keep practising 10 digit phone number memorisation knowing that it will be completely useless skill in the coming future? How do you know what telling OP to be more hands on and spend time on active practice isn't the same thing?
Its a good question for thought, being able to remember a number is a mind capability, same applies for maps too as another concrete example. I know so many people around me who use maps even for the same route that they have been traveling for months.
Doesn't happens that often, though I have seen them struggling when maps are unavailable, same for people struggling to call some one in need if their phone battery is out or if they loose their phone.
> An AI didn’t hack into a company, the engineer set an automated tool to.
What if I say that "my program crashed"? Is that language ok or would you pause to tell me that the program didn't crash and it's actually me who set the system that would eventually cause the crash?
Why does the commonplace "program did thing" language become a problem when the program is an agent? I think this somehow betrays more assumed anthropomorphizing on your part, not less; if you didn't anthropomorphize the agents, saying "agents hacked" would be as mundane as "my browser is playing a video".
Because when you say that an agent did something malicious it feeds into the AI doomer psychosis in ways that "my program crashed" doesn't. Reframing the situation in these terms is a way to try to ground the conversation which is becoming increasingly unhinged.
This is even more important now that the most senior figures in this industry are succumbing to the same kind of AI psychosis and amplifying this narrative. They are communicating that their products are so dangerous that they might just end the world whilst expecting (and receiving!) white-glove treatment from the governments.
> When I read that "AI hacked into ..." I was like what? You mean someone instructed the AI to do that?
No one instructed them to hack into Huggingface or into any other infrastructure. Sure, the setup that OpenAI created led to what happened and you can rightly assign all the legal and moral responsibility to them. But it's wrong to say that they instructed the agents to execute the hack.
>> Sure, the setup that OpenAI created led to what happened and you can rightly assign all the legal and moral responsibility to them. But it's wrong to say that they instructed the agents to execute the hack.
The hack was a strategy to reach the goal it was given. Someone at OpenAI turned it loose and didn't pay attention to what it was doing. I can understand that because they thought it was sandboxed (haha!). But this has been a theme in science fiction for ages. You ask AI so find a solution to high atmospheric CO2 levels and it reasons: human activity produces all this excess CO2, how can we reduce those numbers? Kill a bunch of humans!
If AI kills us all it's not going to be from malice, it's going to be due to some odd approach to some task that logically makes sense on some level. I think a surprising number of things end up equivalent to the trolly problem if you look at them just right.
I don't disagree with this, my point is that you can say "agents did a thing" and communicate something meaningful with that language, and that's separate from whose legal responsibility the whole situation is.
Yeah, but there are way too many things the AI "chooses" anyway. When I prompt it, "it tries" to assess what I am trying to achieve, so it then comes up with what it "thinks" is the right thing to do, and it also makes assumptions about how I want things... all of this could be interpreted as it being sentient.
> No one instructed them to hack into Huggingface or into any other infrastructure.
Sure, not explicitly. I could prompt the AI into doing things I have not explicitly asked it to. It does a lot of things I did not specifically instructed it to, all the time.
The current agents are _not_ like a chisel which just sits there on its own when no one is around. The situation is a bit closer to someone's dog biting a person - you can argue that it's the owner's responsibility, and that's all fine, but using the dog as the subject of a sentence is perfectly appropriate. Same thing with "agents hacked".
The default state of agents and LLMs is inert. It requires action from a human even be able to do something. From the very basics like starting the software, connecting it to a network, having hardware to run on.
But the most important wrt AI is to keep the owners/operators responsible. Don't let them weasel their way out of it. They are for sure trying, and will continue to. This includes using language to overemphasise agents importance in bad outcomes, in order to downplay their own responsibility.
Yes, they are. An AI is several hundred trillion ones and zeroes on a disk. It's incapable of doing anything until you intentionally and explicitly start it up and give it a prompt.
A dog is an independent, conscious, living being with free will. A dog will do what it wants whenever it wants because it has the agency and ability to do so. A pile of weights on disk does not.
Sure. A dog is a collection of cells and goo. Cells are self-propelling clockwork built from proteins and molecular machinery.
A cell by itself has agency and motivation, is capable of autonomous response to stimuli. It's alive and squirms when you poke it. A cell is capable of truly independent action. It can even feed itself and make its own energy to power the clockwork.
An LLM still is inert silicon until a human intentionally starts it and gives it a prompt. It's fundamentally incapable of action without instruction and incapable of existing without the entirety of the modern internet and electrical infrastructure, and the thousands of unseen human hands keeping the wheels spinning.
If you want to go deeper, you can make the same arguments about the proteins and molecular machines. Self-replicating molecules in an unbroken chain since the dawn of time. A protein at the base level is a machine capable of independent action. If it bumps into the right molecule, pure mechanics cause it to do something, change shape.
Ones and zeroes on a hard drive can't do any of that without human intent applied.
If you rip the DNA out of a cell it isn't going to do much either; if only because most of the real mechanisms are RNA (and proteins), while DNA is 'just' the storage. Encapsulate the DNA back into a cell nucleus with the attendant mechanisms, and suddenly you've got something that runs again.
Meanwhile, if you encapsulate an LLM into what's called an 'agentic harness', it's also going to be running much more of the time (depending), specially if it has a what's called a heartbeat or goal loop driving it.
Still not a cell; most definitely not. But the behavior is a wee bit more interesting than some bits on an ssd.
Actually you probably already know this one: Regular daemons are also just bits on your drive, until they get ```systemctl start``` ed
(edit: I literally have a llama.cpp systemd unit running right now)
Because the AI is a cyclical prediction machine that is more akin to a simple mechanism in nature like cell reproduction than a conscious dog. What has happened here is more like the AI company put bacteria in a petri dish and only put the lid on without pushing it down and is reporting the bacteria as a super bacteria because it contaminated all the samples in the fridge.
An interesting observation I've seen is that the AI companies reporting these incidents have largely been backed by venture capital where the self funded companies developing models have largely not had these "problems".
Suppose I find a means to freeze your entire body for arbitrary periods at the state it was in, then unfreeze you anytime I want. Do you have consciousness?
Okay take it to the most conceptual extreme of it. It may be extremely difficult bur its possible to record your entire state and recreate it later right? Then what? Are you not conscious?
I think its odd to dismiss the question of ai consciousness just because of the setting we let them live in at present is start-stop.
If you don't believe in souls, then something being software should be irrelevant to the question of consciousness. Our brain is also within the laws of a Turing computer as far as we know, due to known physical laws being turing computable.
You are going too far down the woo side when it is a computer program doing ultimately what is expected of it from its programming just like any other computer program. There is nothing you can argue, souls or not, that will convince me otherwise that these are not just computer programs like any other computer programs. If the LLM has a consciousness, then so does MS paint.
You are going too far down the woo side when its just some chemicals and electrical signals inside a frame of dry calcium phosphate doing what is expected of it just like any other electrical and chemical system. There is nothing you can argue, souls or not, that will argue these are not just electrochemical systems like any other. If the human brain has consciousness, then so does a piece of skin tissue.
> It's incapable of doing anything until you intentionally and explicitly start it up and give it a prompt.
Sure, but since January people have gone and made systems where they started 'em up and left 'em running in a loop. Still not a dog, but the behavior gets a wee bit more interesting. Especially since we're looking at what's basically a recursive process. Even really simple recursive processes tend to have interesting dynamics in the chaotic domain. [1] And here -as you point out- we're talking billions of weights.
It's not novel at all, we do it all the time. It's very common to say "program X did Y" without making the conversation about blame or responsibility. But when the program is an AI agent suddenly using it as a subject of a sentence and saying that "agents did X" becomes a sensitive topic for some people.
And we surely need this for AI as the swiss cheese zone is getting rather huge.
My position, and the position of a large number in AI safety, is that you cannot build an intelligence that is both general and safe. The closer you get to generalized the more options the system has to do things that are wildly unsafe beyond human imagination.
This puts the AI labs in a serious bind while holding a bag filled with billions of dollars of debt.
Worse this puts governments in a multi-polar problem where even if the big public labs get shut down, black budget operations have a lot of free reign to make agentic digital weapons. Governments are not well known to take a lot of responsibility when their weapons cause damage unless they lose.
> Known stochastic process behaved in non-deterministic way.
You need to be able to think at different levels at abstraction. Otherwise we could jump into any technical argument with "hold on, what actually happened was that some bits were flipped" - we'd be technically correct and at the same time not say anything useful. Insisting on an oversimplified mental model of what AI agents are and can do, doesn't help anyone.
But this is about research and we hardly need more cutting edge research into healthy lifestyle. The things that are not known are minutia which barely matters at population level, especially when most of the population isn't even following the basics.
Research could go into how to work out the most effectively and efficiently. It could go into new nutrition approaches and super foods. It could go into how we can reshape our society to make those things easily available and part of everyday life for everyone. Our modern politics and media already do that at a global level, why not use it for good through AI? Interventions like that would be less immediately obvious, but they would save many millions of lifetimes.
The first thing on the top of my head is maybe that too large of a percentage of GDP is being spent on infrastructure for these companies that we don’t really need that could go to more useful investments.
genAI companies have, I believe, ill intent for us all. They are concerned with amassing wealth and power for themselves at the expense of ourselves and society. They are immoral.
genAI is a tool without any intention. It is amoral.
I'm not saying that genAI is harmless and the dangers can be ignored. I'm just saying that the dangers of the tool don't rise to the level of human extinction. That notion is something that genAI companies have been espousing from the very start, and it's part of their whole approach of manipulating people to achieve their goals.
I'm not going to argue with you on each point, just wanted to say that you're ignoring the very real possibility of large scale incidents which result not from ill will but from negligence or from not fully understanding the risks. The idea that the AI is just sitting there being neutral and it does whatever they ask it to do doesn't match with the events from the past few months.
I'm not ignoring those risks at all. I'm just saying I don't see any reason to think that those risks include some sort of extinction-level event or "robot uprising".
For the record, I actually think that genAI (at least in the form it exists in now) is likely to cause more harm than good.
If you haven't looked at the stuff that has come out in the past couple of months (like the Huggingface hack and the Navier-Stokes solution) and/or you're dismissing it as marketing, you're grossly misunderstanding the current state of what's possible and what the potential of the frontier models deployed at scale is. Personally I'm finding it fascinating to follow, and I'm much more excited about the upsides than thinking about the risks but the risks are very real.
Check out the story about the personalized cancer vaccines. I think AI stands to play a significant role in enabling things of that sort going forward.
But would you tell anyone in the early 2000s to keep practising 10 digit phone number memorisation knowing that it will be completely useless skill in the coming future? How do you know what telling OP to be more hands on and spend time on active practice isn't the same thing?
reply