The issue with embodiment is that it's relatively easy to start affecting the world once you have Internet access. Including things like adding great features to open source software that contains subtle bugs to exploit.
Or if you mean the physical world, even sending some text messages to a lonely kid can get them to do all sorts of things.
> We cannot, in general, keep humans under control or aligned.
This is the crux of why replicable-more-than-human-intelligence is so dangerous. Even giving a random person on the street great power is a bad idea, and they've evolved to have very similar values and preferences to you.
I get that AI basically is a problem solving machine that might eventually adapt to solve generic problems and thus reach the ability to break out of its box. But so what? Even if it manages to do all that, doesn't make it sentient. Doesn't make it a threat to mankind. Only when sufficiently motivated, or in actuality, when we project our humanity on it does it become scary and dangerous.
If we ever seriously try to create an artificial conscience it might need to be embodied, because we are embodied and seem to have evolved this due to evolution, which is a pretty physical process. Looking at it from this perspective one might say that if we keep the AI in its box, it will never have a need for conscience and therefore will never gain it.
This reply puzzles me somewhat. The first half doesn't seem to relate to the post it's replying to.
How aware are you of the main points around AI X-risk like orthogonality? Or how an optimising process that makes efficient use of information does not need (in theory) to have "conscience" or "sentience" to be lethal?
And on a separate tangent, are you aware people are already making primitive agents by connecting LLMs (given an initial prompt) in a loop with the result of feedback from its actions?
Or if you mean the physical world, even sending some text messages to a lonely kid can get them to do all sorts of things.
> We cannot, in general, keep humans under control or aligned.
This is the crux of why replicable-more-than-human-intelligence is so dangerous. Even giving a random person on the street great power is a bad idea, and they've evolved to have very similar values and preferences to you.