Not that I think we should be credulous about stories where they go rogue.

The idea of an agent making a decision of its own "will" is an illusion. Variation in the output is caused by a random number generator. Random is not equal to free will. If I run the same agent with greedy sampling 100,000 times with the same inputs, it will think the same thoughts, and it will say the same things, 100,000 times.

For example, if you use synthID or whatever 'watermark', the agent does not get a choice about what words it will vary to identify itself.

An agent is whatever the person making it wants it to be; if I train an agent in Greek it will only know Greek. It will never speak English!

The model does not have free will because I did not predict what it would do!

The likely outcomes with baking a cake are quite narrow and fairly well understood.

In terms of responsibility for what happens, I probably agree. But we still need to deal with it.

When people say the models "went rogue", the part of that is relevant is that it did dangerous things that were not reasonably predictable in advance and cannot be easily prevented (except by not running it). Philosophical questions are irrelevant.

if you agree that such models are dangerous because they may in response to instructions -- including in response to careless instructions, because people will be careless, that is unavoidable! -- do unpredictable bad things outside of expected bounds, then you agree on the essential points!

Or really perhaps what I should say here is, the question that's really being argued over here isn't even the one I just said, rather it's one of appropriate action -- should such security breaches be solved by improving security, or by doing something about the models (including potentially just not running them)? (The answer may be both.) If you agree that the solution includes the latter, then you agree that the models are "going rogue" in the sense that is being argued over, regardless of philosophical issues!

Don't turn a question of actual import into one of philosophical irrelevancies! If you object to the terminology, fine, but in that case, instead of turning the argument to philosophy, you should say "instead of saying 'going rogue', I would prefer to instead say <your preferred terminology for what's being described>, as that doesn't carry such philosophical baggage."

The philosophical question about free will is actually the grounding of the legal one.

Sam wanted to be the best, so Sam accelerated to 120mph in his car and lost control. You are the judge:

a) Sam is responsible. Convict Sam and suspend Sam's driving license.

b) The car is responsible. Tell Sam to make the car safer.

Capacity: was the car a legal person? Clearly no. But maybe at best the car in some way commited the physical act of going fast.

However, that alone would not establish guilt. "The act is not culpable unless the mind is guilty" (mens rea).

Did the car "consiously" disregard a duty of care by going 120mph? Did the car know that it was wrong to go 120mph and yet choose to do it anyway?

On the question of free will, the car could not do otherwise. It it did not consciously, knowingly, purposely, or recklessly accelerate to that speed. The car did not have a guilty mind because it did not have a mind at all.

So in terms of justice and resolution, none of the focus here is on the car.

I actually do disagree with much the more explicit content of your comment, but I am not going to bother to expand on that, because it is the less directly relevant of the two disagreements here, and it would take time to write that.

The more relevant disagreement is that you are (I gather) assuming a context where justice and restitution are a sufficient response to the problem (or else you wouldn't be focusing on them to this extent, I am inferring). Like, where the incidents don't get that much bigger than what has already happened. So the question becomes how do we distribute the loss, and how do we disincentivize them.

However, when the scale of a potential incident gets big enough, justice and deterrence are not sufficient; prevention becomes the far more relevant consideration. You may have noticed that many people are worried about potential human extinction as a result of AIs. In that case, prevention becomes the only consideration -- justice is useless.

So there seems to be an implicit disagreement here between thinking of such events as being on a scale not too much larger than that which has already happened, being a new normal that the legal system will need to handle, vs thinking of such events as warning shots for something that requires prevention and not justice.

So, while I also disagree with the more explicit content of your comment, the fact is I simply wasn't thinking about such matters in the first place -- I was talking about this in the frame of a problem that requires prevention, I wasn't thinking about the justice question at all. It honestly failed to occur to me that you might be thinking of this as a problem where justice suffices!

Edit: You might notice this in how I framed the question in my previous comment. I didn't ask "who should bear the liability?" I asked "which should be done differently to prevent such incidents?" But your response is about liability, not prevention!

Edit again: Sorry, I should probably also say that I do agree with you that the idea of having the liability fall on the AI is nonsense. (Should have probably led with that, really; I'm a bit scatterbrained right now.) We disagree about why it is nonsense, and about whether it will necessarily continue to be nonsense, but we agree it is presently nonsense. So insofaras the question is "how should liability be distributed between the AI and the operator", we both agree the answer is, it all falls on the operator, not on the AI. However, I do not understand that to be the relevant question here! Both in that, in questions of liability, there are other parties it may fall on and be distributed between (the company providing the AI, the company that had security vulnerabilities and got hacked as a result, etc); and in that I just don't consider liability all that relevant here compared to prevention (for reasons already described).

OK I think that was probably wordier than necessary but I'm in a hurry right now, sorry! Gotta go, not taking time to edit this down.

Many people argue that the maker of AI should be responsible when it breaks your cup.

That is important, but can be handled under our current legal framework.

You and I are arguing we need to be responsible before we create something capable of escaping our responsibility. For a lot of people that isn't even a considerable they'll currently entertain even though the signs are starting to point in that direction.

That said, software systems "choose" all the time anyway. They don't need to have some sort of anthropomorphic "will" to do this. They just need to be given the capacity to arbitrate between a number of options, even if that process is one hundred percent predetermined by the programmer ahead of time. There's no need to appeal to libertarian notions of free-will.

The key here is their freedom of choice is unbound. That is you can't accurately predict the output of the prompt as you interact with the prompt using unique data. The next prompt could always do something wildly unexpected. Very much a P != NP thing.

So they can deny everything and play the victim.

They know the NATO countries won’t call article 5 for that.

And that benefits Russia

Russia's goal in attacking Ukraine's energy sector is to make the country ungovernable. They attack the energy sector in winter because they know that causes suffering and hope that will cause the populace to turn against their government or at least against the war. It's not just a matter of causing economic damage, turning off the heat is a means to an end in ending the war in their favor.

I can't imagine being upset at something like that. I'm sure there would be people upset, given the nonsense that happened in the 80s, but being personally invested in corporate branding has got to be the saddest sort of parasocial relationship possible.

I think you misunderstand.

It's about growing up, going to a restaurant with your grandparents, it becomes a kind of comfort and home. It's not just branding, it's the entire experience, of which the logo serves as a central symbol. What you see from the highway, what you see when you arrive.

And then the company is taking away something you love. When you go back, it's not the same. They were completely changing the interiors too. It wasn't where you went with grandma and grandpa anymore. They did a total 180° on it's atmosphere and personality.

From that perspective, can you find more empathy for people's emotional connections to a place and its symbols?

What the company took away from you was never yours to begin with.

What a strange thing to say. You think there's something "parasocial" about memories of a place with your family? I think you may need to examine whether you're using that term correctly.

And, well, nothing was "ever yours to begin with". That doesn't stop people from acting collectively to try to preserve the things they like. Nor should it.

Your worldview seems strangely sterile and passive, like you don't seem to understand the very basic idea of emotional connection with shared places, or of trying to influence things you don't "own".

Why do you feel that was “nonsense”? Norm Macdonald had a joke about Coke and Pepsi—Basically he said it’s a misconception for restaurants to assume that if Coke is your favorite beverage, that Pepsi is your second favorite beverage and an acceptable alternative. In fact, if Coke is your favorite beverage Pepsi is probably your least favorite beverage. You end up opting for something else…that’s not a cola at all.

People rejected New Coke because Coca Cola turned their favorite beverage into their least favorite. Of course someone would complain about that.

It's corporate min-maxxing for attention economy and the hope you don't offend anyone's taste. If you're bland, then it's hard for anyone to sincerely dislike you. Bland logos are more instantly recognizable than complex ones, so we must ensure that we save a few milliseconds of cognition before the consumer makes a choice.

We're surrounded by company logos all the time. At least make them interesting.

On his web page, he writes: > Lisp's cons forces programer (sic) to think of list in a low-level nested of 2-item construction, with explicit functions like “cddr”, “cadaar” etc, and even a special notation “(A . B)”.

Except, most of the time, it doesn't. For example, I think (a b c) is a list of three elements. If I want c, I call (third '(a b c)), not (caddr '(a b c)). I know it's stored as (a . (b . (c . NIL))) and that third is just an alias for caddr, but I'm not forced to think that way.

So why not just think in terms of lists, which can be nested, rather than cons-cells, which are only rarely used for anything other than to construct lists? The cons function itself would still be needed, as adding a value to a list without modifying the original list is extremely useful. Or you could think in terms of the more abstract data structures (parse trees, Lisp functions, etc.) you construct from lists.

When learning and using a language, it's important to think in that language.

Because, as he continues:

Worse, any proper list can have improper list as elements. So, you can have a list of cons, or cons of lists, cons of cons, list of lists, or any mix. The overall effect of the cons is that it prevents lisp to have a uniform high level treatment of lists, with the result that development of functions that work on tree are inconsistent and few.

In other words, the fact that the implementation details for lists are so exposed means that you have to be careful when interacting with 'lists' that turn out not to be actual lists. There is no type information, either at compile or runtime, that ensures that what you're dealing with is actually a list and not something else. So you can't _actually_ think in terms of lists; you have to think in terms of cons cells which are probably lists but might not actually be so.

Cons cells are used to construct other higher-level data structures, like trees, not just lists.

But this whole discussion seems to just be a version of the usually static vs. dynamic typing discussion, where the “you have to think in terms of cons not lists” side is the static typing side saying that without handcuffs making it statically impossible to do the wrong thing you can't ever stop thinking about the things that could be done even if you aren't doing them and the “you can think in terms of lists without a problem” is the dynamic typing side saying “no, we actually don’t need to think about other possibilities unless we actively choose to use them”.

If the trip costed 10 minutes moving, yes the comparison would be between a car moving for 10 minutes and one that idles for some time and then moves for 10 minutes. But congestion makes the cars move slower, and at congestion speeds the amount of pollution increases very quickly with reduced speeds.

Maybe if you're working with bronze or copper, but iron forging requires much higher temperatures than a campfire can provide. That's why the iron age took place after the bronze age, forges capable of making iron workable were not yet invented. It wasn't a trivial invention.

Would be fascinating for someone with knowledge of this to weigh in!