This whole thought experiment is actually presenting what is quite possibly a limitation specific to human minds - no reason to extrapolate to all possible minds. That is, human minds seem to have two very distinct ways of altering their internal state. "Raw" sensory data gets stored in one way ("experiences"), while higher order data gets stored in another way ("memories"). That we can, to some extent at least, create memories of facts we learn from books, but we can't create experiences from facts we learn in this way can very well be an artifact of how our minds are fundamentally put together.
An analogy with a computer makes this quite obvious. If we imagine an AI running on a CPU similar to those we know, and given free reign to interact with everything including the contents of the computer as well, it's easy to see that an AI-Mary would not need to leave the room to experience the color red - it could simply alter its input channels/memory to generate the exact experience that its sensors would detect if they were pointed at a red object.
If however we alter the AI such that it can't access/modify its sensors directly, and it's sensors write data to a piece of memory that the AI also can't write to, then the AI-Mary will not be able to put itself in exactly the same state it would be in were its sensors pointed at a red object, and so would have some kind of a new experience when leaving the room.
But, this would not be some fundamental fact about subjective experience or qualia, it would simply be a limitation of the architecture.
> This whole thought experiment is actually presenting what is quite possibly a limitation specific to human minds - no reason to extrapolate to all possible minds. That is, human minds seem to have two very distinct ways of altering their internal state.
How to define “mind”? If all human minds turn out to have a certain property, how do we decide whether that property is essential or accidental to “being a mind”? Obviously if the property is “being human”, it can’t be essential to the property of “being a mind” (unless someone wants to claim that only humans can have minds by definition-a claim very few would make). But, if the property is “having two very distinct ways of altering their internal state”, how do we know if that is just an accidental property that all human minds happen to have, but non-human minds might lack, or an essential property of being a mind, which all minds must have to be “a mind”?
Sure - I'm not claiming to have settled the theory of mind!
I'm just pointing out that nothing in the argument about Mary, even if we accept that she would learn something new when leaving the room, proves that qualia must be non-physical or non-computational or anything like that.
You could find arguments that such a separation between qualia and physical facts is indeed necessary for any mind that could really exist in the world. But they would have to be other arguments - this one doesn't work.
I think a version of this argument does succeed in showing that qualia are real as opposed to just some conceptual mistake.
The problem with arguing about whether qualia are “physical” or “non-physical”, is I’m not sure how much sense such a distinction makes if one is not a Cartesian dualist-which I’m not, and you don’t appear to be either
Personally, I'm of the opinion that the human/animal mind is a computational process happening in the brain, and that a sufficiently complex computer could also "run" such a mind. In that sense, I believe they are entirely physical objects, just as much as the Linux kernel running on my phone is a physical object.
As such, I believe that qualia are just how we would label some aspects of this computation, and I believe that concepts like a p-zombie are thus more or less non-sensical.
I'm not claiming to know for sure that this is the way. I do think it's, at least in principle, possible that a mind is a different thing, one that our understanding of physics can't (yet?) fully model. Hell, even though I'm an atheist, I wouldn't even claim that the concept of a divine soul is completely impossible.
A single celled organism lacks any sensory perception and so, from its perspective, it has explored the depths of the universe without having to ever even move. With each step upwards in complexity, the entire perceivable domain becomes wider. That humans can experience a facsimile of something and the "reality" of something differently, would seem to imply that we have a more sophisticated input system than something that would equate the two different "inputs."
> A single celled organism lacks any sensory perception and so, from its perspective, it has explored the depths of the universe without having to ever even move.
Single celled organisms do have a kind of sensory perception and can react to changes in their environment. Even our own individual cells have this ability (and I'm not talking about neurons, I'm talking about every single cell in our body, or at least the vast majority).
In fact, some single celled organisms (or at least, colonies of them) have even been proven to be capable of certain computations - for example, "slime mould", which is a bacterial colony, can grow in a way which finds solutions to simple mazes.
> That humans can experience a facsimile of something and the "reality" of something differently
That's not what I was talking about. I'm saying that we have no way to directly stimulate our sensory organs with with signals that mimic external stimulus, which would be quite trivial for an electronic brain. Also, I'm saying our memory of direct sensory experience (what I saw when I looked at a red thing) is separate from our memory of more conceptual subjects (what is the square root of 4, who conquered England from Normandy, what I ate this morning).
While it may turn out that this "architecture" is necessary for having consciousness/a mind, it doesn't seem a priori impossible to build an electronic mind that doesn't have these limitations/separations.
An analogy with a computer makes this quite obvious. If we imagine an AI running on a CPU similar to those we know, and given free reign to interact with everything including the contents of the computer as well, it's easy to see that an AI-Mary would not need to leave the room to experience the color red - it could simply alter its input channels/memory to generate the exact experience that its sensors would detect if they were pointed at a red object.
If however we alter the AI such that it can't access/modify its sensors directly, and it's sensors write data to a piece of memory that the AI also can't write to, then the AI-Mary will not be able to put itself in exactly the same state it would be in were its sensors pointed at a red object, and so would have some kind of a new experience when leaving the room.
But, this would not be some fundamental fact about subjective experience or qualia, it would simply be a limitation of the architecture.