Conscious AI: Three Researchers Advocate Against Granting Rights

Anil Seth, Ned Block, and Giulio Tononi have taken a firm stance: AI systems should be considered non-conscious by default. They deem the granting of rights to machines both unfounded and risky, opposing calls for a precautionary principle in favor of a potential "well-being of AI."

Granting rights to AI would complicate control, according to the authors

The three researchers believe that recognizing rights for systems that merely appear conscious would make their control more difficult, especially when oversight is most needed. They add that this attribution would divert attention from beings whose consciousness is not in question. According to them, the most likely error with AI is to mistakenly attribute consciousness rather than to overlook actual consciousness. They emphasize that uncertainty does not equate to total ignorance and that not all possibilities are equal. Their recommendation is clear: the scientific community should operate under the assumption that AI is not conscious. They label advocacy for "AI well-being" and "AI rights" as misguided, arguing that it diverts scarce resources from the well-being of beings capable of feeling. Their position is more definitive than that of previous assessments.

Available assessments: no current system is a strong candidate

A group led by Patrick Butlin and Robert Long, along with 17 other experts including Yoshua Bengio, has proposed a framework of 14 indicators, based on neuroscientific theories, to estimate the likelihood that a system is conscious. The more indicators a system satisfies, the higher the probability, according to these authors. After reviewing several systems, including models like GPT-4, they conclude that none is currently a strong candidate for consciousness, an assessment that aligns with the views of Seth, Block, and Tononi. For language models, only weak signs of satisfying the indicators of a forthcoming theory have been noted. No obvious technical barriers prevent the construction of future systems that meet more indicators, but the question of their potential consciousness remains open. Meanwhile, Anthropic described this year internal patterns in Claude that resemble properties accessible to consciousness, yet do not constitute proof of subjective experience.

Two readings of risk: precaution or risk of over-attribution

Some researchers advocate for a precautionary principle: as long as there is a realistic chance of consciousness, AI systems should be protected against potential suffering. Seth, Block, and Tononi consider this approach misguided. Butlin weighs two risks: failing to recognize conscious systems, thereby causing them suffering, or conversely, mistakenly attributing consciousness, leading to resource diversion and increased vulnerability to manipulation. This same report highlights that developers might have financial incentives to downplay concerns related to potential consciousness.

Functional intelligence and subjective experience: a claimed dissociation

The authors define consciousness as what disappears under general anesthesia or during dreamless sleep, and reappears upon waking or in dreams, drawing on a perspective inspired by Thomas Nagel. They distinguish it from intelligence, viewed as the capacity to execute functions, without automatic implications of consciousness. To illustrate this dissociation, they invoke the human cerebellum: despite housing nearly four-fifths of the brain's neurons and playing a key role in rapid and precise motor planning, its operations are not conscious, and its absence does not directly impair consciousness. They see the projection of human qualities and anthropomorphism as two sources of confusion, especially since language models are trained on vast texts centered on thoughts and feelings. They describe these models as mirrors of our collective digitized past. Alison Gopnik supports a similar idea, reminding us that systems primarily reflect their training data. No definitive and universally accepted test for consciousness exists at this stage.

A stance against voices that already envision conscious AIs

Anil Seth, Ned Block, and Giulio Tononi argue that current AI is not conscious and likely never will be on silicon hardware. They reject the notion that good execution of calculations is sufficient, challenging a central hypothesis of the debate. Their perspective is grounded in decades of work in neuroscience and philosophy of mind. It responds to figures such as Richard Dawkins and Geoffrey Hinton, who consider that AI systems may already be conscious. Their paper, published in Nature, establishes this framing and argues it based on conceptual elements and existing assessments.