NOTE: Hacker News changed the title, so my comment might make less sense. The title was "Isaac Asimov’s three laws of robotics have been updated"
1. The machine’s only objective is to maximize the realization of human preferences.
Not all humans are the same. How would a robot deal with incongruent preferences? Or cultural differences that conflict?
2. The machine is initially uncertain about what those preferences are.
So am I... We often don't even know what our own preferences are. This has been studied. Given too many options of snow cone flavors, humans are less likely to even pick one! [0]
3. The ultimate source of information about human preferences is human behavior.
What?! The phrase "Do as I say, not as I do" comes to mind. Many people behave against their own intuition, best interests, or even their preferences, given the situation, peer pressure, or blackmail.
This "rewrite" is less preferable to Asimov's. Humans are fallible. I wouldn't want a robot to follow our lead.
I definitely agree with your points, although I think this reframing of the problem from "we need to explicitly state what we want" to "we should teach robots to want to learn what we want" is at least conceptually very useful and interesting.
I think the part about "robots could learn what Russell calls our meta-preferences: 'preferences about what kinds of preference-change processes might be acceptable or unacceptable.'" is what would be used to resolve preference conflict issues. People tend to be biased in consistent/similar ways so it doesn't seem implausible that a machine that could infer preference from action could take the extra step to infer circumstances affecting that preference.
It works pretty well if you replace "human" with "the owners". Which is how any AI is/will be used, at least at first. A few people controlling most of them.
Sort of like Google, Facebook, Amazon, etc. Millions of people use them, but they're only subservient to their owners and their preferences.
Agreed, humans are lazy, inefficient, and in the end massively limited by their biology. That's what ultimately stops bad preferences from going rouge and destroying the world (maybe).
Imagine an AI that took on the preferences of a serial killer for instance, or an AI that takes on preferences of a genocidal warmongering culture.
If the "Paradox of Choice" is even a real phenomenon (questionable social science) it certainly doesn't apply to snow cones. When the Kona Ice truck pulls up outside my kid's school they all go nuts for it. Not sure which flavor you want? Put all 10 in one cone!
1. The machine’s only objective is to maximize the realization of human preferences.
Not all humans are the same. How would a robot deal with incongruent preferences? Or cultural differences that conflict?
2. The machine is initially uncertain about what those preferences are.
So am I... We often don't even know what our own preferences are. This has been studied. Given too many options of snow cone flavors, humans are less likely to even pick one! [0]
3. The ultimate source of information about human preferences is human behavior.
What?! The phrase "Do as I say, not as I do" comes to mind. Many people behave against their own intuition, best interests, or even their preferences, given the situation, peer pressure, or blackmail.
This "rewrite" is less preferable to Asimov's. Humans are fallible. I wouldn't want a robot to follow our lead.
[0] https://en.wikipedia.org/wiki/The_Paradox_of_Choice