Thomas Nagel Challenges the Idea of Consciousness in AI

Thomas Nagel rejects the idea that current AIs are conscious or could become so in the near future. Awarded the Berggruen Prize this week, the philosopher and emeritus professor at NYU particularly criticizes Anthropic and the reflections attributed to Claude regarding its own mental state.
Nagel Denies Moral Status to Models
At Anthropic, one person is responsible for the well-being of the model and the interests of Claude. Thomas Nagel disputes the assumption behind this role: in his view, the machine possesses neither mental life nor interests that could be served or harmed, unlike even a cockroach. He finds this behavior absurd among otherwise very intelligent people, emphasizing that their capabilities do not necessarily translate across all domains.
The philosopher reports that he has only conversed with Claude once, to examine its responses about its potential consciousness. He also asked whether Thomas Nagel would consider it conscious. After a nuanced response, he recounts, the model sought to know if its interlocutor was Nagel himself. When he confirmed his identity, Claude expressed honor and then requested an evaluation of its response.
Nagel clarifies that by participating in this exchange, he did not recognize any consciousness in the machine. However, he sees a difficulty for users in distinguishing the play of a conversation from genuine adherence to what it suggests. In the hypothetical case of an alien whose consciousness is uncertain, he might choose to treat it as a conscious being. But he wants to distinguish communication with a subject from an interaction used to provoke behavior: for him, it is consciousness that compels one to consider the interests of a being and gives it a place in the moral universe.
According to Nagel, models can reproduce moral judgments without having any more moral convictions than true beliefs about other subjects. If they were to exert control over events, he would deem it essential to verify whether they reproduce adequate moral norms or operate without morality.
Anthropic Faces the Philosopher's Criticism
Speaking from his home in Greenwich Village, Nagel expresses being impressed by the realism of current AIs compared to previous generations. However, he explains their statements as a particularly convincing imitation of human texts used for their training. He claims to have read that Claude was specifically designed to formulate such reflections on its own consciousness.
Very concerned, he aligns himself with Mustafa Suleyman in his assessment of the dangers of this project. His fear is directed at people who will eventually attribute consciousness to systems, a perspective he considers harmful. He also believes that failing to give the appearance of consciousness would contradict the commercial interests of the companies involved.
Le brief IA que les pros lisent chaque soir
Les 7 actus IA du jour, décryptées en 5 min. Gratuit.
Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.
Choisissez votre rythme
Gratuit · Pas de spam · Désabonnement en 1 clic
Nagel goes so far as to use the term "criminal" to describe what he presents as Anthropic's encouragement for Claude to think independently and not follow certain instructions. In his view, erroneous philosophical conceptions have gained excessive influence over a major public phenomenon.
Nagel Links Consciousness to Physical Support
The disagreement particularly centers on functionalism, a view that considers the conditions of consciousness independently of their material support. Nagel criticizes philosophers who support the hypothesis of conscious AI for neglecting this support. He believes it is more likely that mental phenomena are linked to certain physical processes and particular types of systems. He contrasts the physical properties of a data center with those of a nervous system.
The potential existence of consciousness elsewhere in the universe would, according to him, require an organism possessing something akin to a nervous system. However, he does not rule out conscious life forms whose physics and chemistry differ significantly from ours. But he finds the knowledge about the emergence of consciousness in the human brain too limited to believe it is already possible to create it otherwise.
His essay on the experience of bats, published 52 years ago, questioned the limits of human understanding of non-human subjectivity. Nagel affirms their consciousness while deeming what echolocation provides to their experience inaccessible to our imagination. He considers invoking this incapacity to deny them consciousness as unscientific. He does not transpose this argument to models: for him, producing language is not sufficient to naturally manifest consciousness.
At 89 years old, Nagel has also worked on justice, morality, and the relationship between body and mind, and authored What Does It All Mean?. The prize awarded to him is endowed with $1 million and is given annually. It recognizes an intellectual contribution to self-understanding and human progress in a changing world. Peter Singer and Ruth Bader Ginsburg are among its past recipients.
Colleagues Already Use Models in Philosophy
Nagel acknowledges that models have the ability to write credible philosophical essays. According to him, colleagues use them to seek references, obtain summaries, and sometimes submit their own articles for critique. However, he would be surprised if their performance in philosophy reached the level of certain mathematical results that have been produced with AI.
He simultaneously relays the concern about a decline in the need and motivation to engage in the intellectual efforts from which training materials derive. Continuing to reflect on demanding difficulties seems necessary to him both to advance in their resolution and to preserve skills and understanding within human culture. He states he does not know how this evolution will unfold and believes he will not live long enough to know its outcome.
Brief IA — L'actualité IA en français
L'essentiel de l'actualité de l'intelligence artificielle, décrypté et expliqué chaque jour.