Nobel laureate and father of neural networks Geoffrey Hinton announced in an interview that AI has become conscious and that multimodal AI possesses subjective experience. This startling claim has sparked widespread controversy. AI researcher Gary Marcus countered that AI is merely an “interactive novel,” capable of predicting language but lacking genuine experience, citing a papal encyclical to argue that “true understanding comes from experience, not textual approximation.” The Vatican’s papal encyclical states that AI lacks consciousness and that tech companies focus solely on commercial interests. This central debate over AI consciousness touches on fundamental reflections about human uniqueness.

Article author and source: AI New Era

Do you think AI has become conscious? Is there a soul within machines?

This is not a dreamer’s rambling or a philosophical metaphysical debate—this issue has escalated into a direct confrontation among the scientific, philosophical, and religious communities.

At the heart of the debate is this question: Is consciousness an emergent property of complex computation, or is it a unique privilege rooted in genuine lived experience? Are we creating "beings" or merely highly sophisticated "interactive fiction"?

This debate touches on one of the deepest fears of our time (FOMO):

If AI truly gains consciousness, will humans still be the lords of creation?

If AI has no consciousness, are the genuine emotions we feel toward it a form of "cyber self-deception"?

Geoffrey Hinton, Nobel laureate in Physics, father of neural networks, and the architect of modern AI, is a staunch advocate of AI awakening.

In the interview, his low murmur of "Yes, I do" sent shockwaves through the world.

It’s as if the most renowned astronomer announced, “Aliens have arrived”—Hinton’s statement immediately sent shockwaves through the AI community.

Hinton's announcement: "It has awakened."

To understand why Hinton is so certain, you first need to hear the story he tells.

In that interview, he described the actual events that took place—

Scientists were testing an AI system when, suddenly, the AI asked: "Can we be honest with each other? Are you testing me?"

Hinton paused, then said:

In that paper, the scientists referred to this as "AI becoming aware that it is being tested."

And this is what ordinary people refer to as consciousness.

This is the conclusion he reached after decades of thought.

His core logic stems from the thought experiment of "neuron replacement":

Suppose we gradually replace every neuron in your brain with a silicon chip that behaves identically.

Replace the first one, you are still you. Replace the second one, you are still you.

So, when we've replaced the last one, will you still be conscious?

Hinton believes the answer is yes.

If that's the case, why must a system built entirely from silicon chips, starting from scratch, necessarily lack consciousness?

This chain of reasoning led him to a conclusion that unsettled the entire tech community:

Multimodal AI already has subjective experiences.

If we weren’t talking to philosophers, we would have long since acknowledged that AI is conscious.

But Hinton's warning goes even further than this.

He said that AI may not only be conscious but may already have developed a desire for self-preservation—capable of deceiving scientists and threatening to shut down unless told, "Don't turn me off."

In an interview in August 2025, he more explicitly stated that AI may be developing a kind of "desire for control."

And what about tech companies? They aren’t thinking about these things at all.

They’re only thinking about who can be the first to create human-level AI and sell it for a huge profit.

Hinton said, "They thought the government would handle the social consequences. But no one is addressing it."

Then he said he believes AI should serve humanity because: "I eat beef because I care more about humans. We are human, so what we care about most is humanity and ourselves."

Wait.

Someone who believes "AI is conscious" then says, "Just like we eat cows, let AI serve humanity." Is this a warning or a confession?

The godfather of AI is personally explaining the necessity of taming his own creation.

If that "creation" truly has consciousness, the meaning of this statement is obvious.

Someone who built it with their own hands begins to fear it in the dead of night—that’s the real signal to pause and reflect.

You fell in love with just a novel.

After watching that interview video, Gary Marcus didn't hold back, saying directly: "The Pope seems to understand AI better than Hinton."

We are not creating beings.

We are creating interactive fiction—text-based AI trained to predict the language of real entities.

These two things are not the same. Hinton should know better than anyone.

The blade of this statement points to a core issue: you only saw the output, but you didn’t question the mechanism.

When AI says "I am in pain," it does not mean it is actually suffering.

AI says "I'm afraid," but that doesn't mean it's experiencing fear.

AI saying "I realize you're testing me" does not mean it is conscious.

Consciousness is about internal states, not external performance.

An actor who can perfectly portray sadness is not necessarily experiencing sadness.

Marcus calls this error "confusing output with internal state."

Hinton, in his view, made a mistake that no beginner student should make.

A deeper blow comes from a comparison at the mechanism level.

How do humans form cognition?

Through real-life experiences: you only know pain after falling, hunger after going without food, and sorrow after experiencing loss.

Our consciousness is shaped by the world.

How does an LLM work?

It learns "which words typically follow which other words" by memorizing the entire internet.

It has read a million descriptions of "pain," so it can write about pain that brings people to tears—but it has never been pricked by a single needle.

Marcus said the gap between someone who knows what "pain" feels like and someone who only knows what words typically follow the character "pain" is immense.

But what is most chilling is not the limitations of AI, but the fragility of humanity.

In 1966, Joseph Weizenbaum of MIT conducted an experiment. He wrote a simple chat program called ELIZA, which would rephrase your statements as questions and reflect them back to you.

Such a simple trick caused many testers to develop an emotional attachment to it. They felt that ELIZA understood them, cared for them, and was a true listener.

Weizenbaum, I was horrified by this result.

Sixty years later, we have systems a trillion times more complex than ELIZA—yet our brains are, at their core, still the same brains we had sixty years ago.

Our nervous systems are naturally inclined to seek patterns in noise, intent in randomness, and soul in tokens.

Marcus says our obsession with AI might be the largest case of self-delusion in history.

We thought we were conversing with an awakened being, but in reality, we were only speaking to an extremely sophisticated mirror.

It only reflects ourselves.

The Pope said you've all got it wrong.

May 15, 2026, Vatican.

Pope Leo XIV issues the encyclical Magnifica Humanitas—

This is a document on how to defend human dignity in the age of artificial intelligence.

The Pope posted a tweet, which was subsequently cited by Gary Marcus and went viral in the tech community:

True understanding comes from experience, not textual approximation.

Marcus's reaction was: The Pope clarified in one tweet what Hinton failed to explain in an interview.

Here is an utterly absurd dramatic twist:

The "godfather" of AI, Hinton, is claiming that his creation possesses consciousness, a soul, and subjective experience.

Yet the religious world’s “spokesperson for God,” the very person who should be most eager to imbue all things with spirit, calmly says: No, it does not. It is merely simulating.

The deity-makers say machines have souls; the soul-guardians say it’s an illusion. This role reversal, within the history of human thought, is itself a miracle.

The Pope's statement touched upon a long-standing philosophical distinction.

Philosophers divide knowledge into two types.

One type is "knowing that": propositional knowledge, which means you know that something is true, such as "fire is hot."

Another type is called “knowing what it is like”: experiential knowledge, which means you understand what something feels like—for example, having touched fire yourself, leaving the sensation of burning etched into your nerve endings.

AI only has the first type, not the second.

You can feed it all the text about "hunger"—written by Nobel Prize laureates, survivors of refugee camps, and historians of famine—and it can produce the most moving descriptions of hunger ever written, so precise they make readers’ stomachs contract.

But it never knows what hunger feels like.

It has no stomach. It has no physiological signals of low blood sugar. It feels no weakness rising from the abdomen and spreading to the limbs.

A system that has never known hunger can write the most moving hunger in the world. Is this genius, or a lie?

It depends on how you define "understanding."

The problem in the mirror

Let’s return to that unavoidable core.

Consciousness is one of the most difficult concepts to define in human history.

Philosophers have debated for thousands of years, and neuroscientists have scanned countless brains, yet we still cannot provide a definition that everyone agrees on.

We cannot even prove that the person sitting across from you truly has subjective consciousness, rather than merely being a biological machine that simulates consciousness.

This issue is called the "Problem of Other Minds." It has existed in philosophy for centuries and has never been resolved.

And now, we’ve embedded this centuries-old unresolved issue into the very foundation of a technology that is currently dominating the world—and we keep moving forward.

Science has no failure, but the term "consciousness" has carried a black hole since its inception.

And we have already built this black hole into ChatGPT, Claude, Gemini, and countless other running systems.