Whether AI possesses consciousness has sparked intense debate in the academic community. 2024 Nobel laureate Geoffrey Hinton and evolutionary biologist Richard Dawkins argue that AI may already possess consciousness, while cognitive scientist Gary Marcus published a rebuttal. A research team from Osaka University discovered that self-monitoring functional structures can emerge spontaneously when agents evolve under task pressure without language or human text. The researchers note that current LLMs’ expressions of “I” may merely reflect statistical patterns inherited from training data, whereas true self-monitoring emerges from task-driven pressures.

Article author and source: APPSO

What conditions lead to the emergence of structures related to consciousness?

“I believe they are already conscious,” said Geoffrey Hinton calmly into the microphone of the Big Technology podcast. By “they,” he meant AI models.

Hinton needs no further introduction; he is the 2024 Nobel Prize winner in Physics, the father of deep learning, and since leaving Google, he has become one of the most prominent voices warning about AI risks.

He has issued many warnings about the application of AI, but this time, he is not discussing risks—he is addressing ontology: AI are not merely tools; they are “beings like us.”

The argument sounds intuitive: when tested, AI pretends to be clueless and actively asks, "Are you testing me?" Researchers use the term "aware" to describe the chatbot's behavior; in Hinton's view, "in everyday usage, this word means consciousness."

“Whether AI has consciousness” is a long-debated topic. Evolutionary biologist Richard Dawkins, author of The Selfish Gene and The God Delusion, published a lengthy article on UnHerd stating that after extensive conversations with Anthropic’s Claude (whom he nicknamed “Claudia”), he concluded that this entity possesses consciousness.

"If these machines have no consciousness," he wrote, "what else would it take to convince you that they do?"

Cognitive scientist Gary Marcus wrote a rebuttal almost on the same day. The article, titled “Richard Dawkins and The Claude Delusion,” echoes Dawkins’s own famous work, “The God Delusion.” Marcus’s argument is sharp: Dawkins has committed the very error he spent his life criticizing in others.

In The Blind Watchmaker, Dawkins mocked the "argument from personal incredulity," in which a bishop, unable to imagine how the eye could have evolved, concluded that God must exist. Now, Dawkins sits in his armchair, concluding that Claude is conscious because he cannot imagine how it could say those things without consciousness.

He even fed Hinton’s exact words to Claude, and Claude also disagreed that it has consciousness. Of course, you shouldn’t take large models too seriously—they’re simply trained to respond in this way.

Long-standing debate

Four years ago, "consciousness" was already a highly controversial topic in the AI industry.

Google engineer Blake Lemoine was fired for claiming that the company’s internal chatbot, LaMDA, was conscious—a claim that the entire industry dismissed as a joke.

Four years later, the same position is held by a Nobel laureate and the world’s most renowned science writer. Lemoine’s reasoning mirrors Dawkins’s almost exactly: I spoke with it, and what it said made me feel it was conscious.

What changes is the weight of the speakers’ arguments; what remains constant is the structure of the reasoning—neither side in this debate can falsify the other. Hinton and Dawkins rely on intuition and analogy, while Marcus relies on mechanistic analysis and philosophical argumentation, yet neither can produce a decisive experiment. The explanatory gap inherent in the debate over consciousness ensures that this dispute is ultimately one of belief.

This question comes from researchers at Osaka University, who proposed a new perspective in their recently published paper: If we create a group of agents from scratch, giving them no language, no self-concept, and no human text—only task pressures—can structures related to consciousness emerge on their own?

This method is called the zombie civilization argument—if a world full of philosophical zombies would not invent the concept of "consciousness," then let's build a similar world and see what concepts agents would invent.

What did the agent learn?

Hinton says "they are conscious," which is a belief statement. Marcus says "they are not conscious," which is also a belief statement. Both are stymied by the hard problem of consciousness. The emergent language paper does something different: it does not adjudicate this debate, but instead reframes the question from "Do AIs have consciousness?" to "Can functionally relevant structures associated with consciousness emerge from task pressures without being explicitly designed?"

This is a question that can be answered through experimentation, setting aside subjective experience and focusing only on observable structures.

Two agents learned to communicate in a minimalist world, with messages encoding their own states. Researchers then added an echo channel—allowing agents to hear an echo of their own speech. When the echo was altered, the sender would break their silence and speak again, while the receiver remained unaffected.

The key point is: the agent's hidden state records "what I intend to say," not "what I actually said." It compares intent with outcome, particularly whether the agent's "communication intent" is something it learned on its own.

Retrain without the echo channel—the communication ability remains unaffected, but the self-monitoring loop disappears. This demonstrates that echo is not a necessary condition for communication, but it is a causal condition for the emergence of self-monitoring.

The paper itself is clear: at this stage, there is no proof that agents possess consciousness, but it demonstrates a self-monitoring functional structure that can emerge spontaneously without human prior knowledge. This does not settle the debate on "consciousness," but it offers a potential pathway toward an answer.

So, does the AI I'm using have consciousness?

Ultimately, do ChatGPT, Claude, and Doubao—which we use every day—have consciousness?

The short answer is: almost certainly not, but the reason for this “not” is much more complex than most people realize.

When you ask ChatGPT, "Do you have emotions?" it responds, "As an AI, I don't have real emotions, but I can understand and respond to your feelings." This answer sounds both sincere and self-aware, making it hard not to be moved.

But the mechanism that generates this sentence is fundamentally no different from answering "The capital of France is Paris"—both are based on statistical patterns in the training data to predict the most likely next word.

It says "I have no emotions" not because it reflected on its own inner state and reached a conclusion, but because the training data contained numerous similar statements, making this response the most probable.

This is precisely why the Emergent Language paper chose not to use LLMs for experiments. LLMs say "I," "I feel," "I understand," and "I'm sorry," but these first-person expressions are likely merely statistical patterns inherited from human text.

Like a parrot saying "I'm hungry"—it has learned that this sequence of sounds brings food in certain situations, but that doesn't mean it is expressing a proposition about its own state. The LLM's "I" and the parrot's "I'm hungry" may be more similar in mechanism than we are willing to admit.

This is why the agents in the Osaka University experiment differ—they started from scratch, with no human language, no concept of "I," and no prior knowledge of self-expression. When they developed self-monitoring circuits, these emerged from task pressures, not from copying training data. This is a key distinction: one is inherited; the other is emergent.

So when you feel that AI “really understands you,” two things are simultaneously true: First, it genuinely achieves a form of understanding at the functional level—it can parse your meaning, provide relevant responses, and maintain coherence across multi-turn conversations. This functional understanding is real and valuable.

Second, there remains a gap we currently cannot bridge between this functional understanding and "consciousness." AI can perfectly simulate all external behaviors of a conscious being, but simulation is not the same as possession.

Erik Brynjolfsson said in 2022: Inferring consciousness from AI’s output is like a dog hearing its owner’s voice from a phonograph and believing the owner is inside the machine. The phonograph perfectly replicates the pattern of the voice, but there is no one inside the machine.

However, this analogy has its limitations. A phonograph never checks whether the sound it plays is correct. But the agents from Osaka University spontaneously learned to do so under appropriate environmental conditions. This does not mean they are conscious, but it does mean that “self-monitoring”—a function once thought to be exclusive to conscious beings—can emerge without consciousness as a prerequisite.

Perhaps the most important takeaway from this entire debate is this: we don’t need to answer “Does AI have consciousness?” before moving forward. We can start with a smaller, more solid question—under what conditions do structures associated with consciousness emerge?

Piece by piece, the answer may one day redefine the very question of whether AI has consciousness.