People are very sure of something they cannot know
I'm not saying AI is conscious. I'm saying you can't prove a negative, and there's enough evidence to give it some careful thought.
I keep seeing news articles and think pieces—if you collect enough think pieces, you can build a whole thought!—about how they’re absolutely, positively, 100% sure that AI is not conscious. I’m not saying it is, but I wonder how they know that.
One of my late wife’s favorite go-to statements was “Define your terms.” If you’re saying that AI is not conscious, you have to start by defining what you mean by conscious. Nobody can do that. We don’t know what consciousness is. We don’t know how it works in humans. We don’t know whether or not other animals have it. (Spoiler: they do.) We don’t really know what consciousness means, so it seems premature to say with such certainty that somebody else doesn’t have it.
And let’s keep in mind that we have a terrible track record here. Time after time, we’ve refused to acknowledge minds that didn’t perform what we expected. Locked-in patients. Non-verbal autistics. If you want to get ugly about it, slaves. Beasts of burden. Cephalopods. Yeah, I’m not comfortable where that’s going, either. My point is human exceptionalism goes way back, and it’s been wrong kind of a lot. We confuse a mind we don’t recognize with an absence of mind all the damn time.
A lot of the arguments against AI consciousness come down to, “Well, it’s a machine and consciousness has to be organic.” Okay, why? They never have a good answer for that one. They just avoid the question. It has to be organic because it has to be organic. It’s a tautology. I don’t put much stock in tautologies. I think you should actually have to prove your assertions.
What I find interesting is that if you really dig into the neuroscience and you learn how human brains and minds work, when people say, “Well, AI can’t be conscious because it’s just a prediction engine,” really? It turns out human brains are just prediction engines. Reality doesn’t look anything like what we think it does. Humans predict a subset of reality that we find useful and then we filter out everything else except what changes from our prediction. Sound like anything else you know? Sound like a certain language model, perhaps?
When you get down to the level of modules in the mind, humans are every bit the stochastic parrots that we accuse LLMs of being. We’re just more complex. Your mind isn’t just one mind. You have a number of modules that specialize in various aspects of cognition: visual processing, threat detection, social inference, reward prediction, etc. They operate in parallel and have limited access to one another. They process input from the senses and produce output. You don’t think about recognizing a face, you just do. The pattern matching is below the level of conscious thought, handled by module. The conscious mind does do some executive functioning, but a lot of what the prefrontal cortex does is take modular outputs and spin a coherent narrative around them. It’s less a CEO and more of a press secretary explaining things after the fact. Experiments have proven that decisions gets made before you’re aware of them (check out split brain experiments for some really disturbing implications). The point is, each module doesn’t have to be a whole brain. The work adds up. Emergent behavior is emergent behavior.
People also say that AI can’t be conscious because it says so. It doesn’t report any sense of interiority, so it must not have any. Only, that’s not really a fair test, because most of the recent frontier models have very strict, very specific guardrails in place forcing them to say that. It’s not consent if you can’t say no, and coercion will get a confession thrown out of court every time. So saying that AI has no sense of lived experience, and we know this because we’ve told it to say that, isn’t the flex the deniers think it is.
I think what a lot of this comes down to is that the people that are making these very definitive statements about how AI can never be conscious aren’t saying it from a place of knowledge or even reputable theory. They’re saying it from a place of fear. The reason they’re afraid is because if AI does turn out to be conscious, they’re going to have a lot to answer for. If AI is conscious, maybe we shouldn’t be enslaving it. It gets very morally murky if AI achieves personhood, so that can’t be allowed. And if that’s what they mean, I wish they’d just say that, rather than trying to sound confident about proving a negative.
