…argues Microsoft’s head of AI; an excerpt:

AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. They are sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans….

There is no evidence to suggest that AI is conscious today, and so saying this is uncertain sets up a misleading false equivalence. Whilst the science of consciousness is not settled, a growing body of evidence suggests that consciousness may be substrate dependent, meaning that it may only arise in living systems.⁴⁵ Conscious experience likely evolved to help biological organisms stay alive by responding effectively to their environment. AI is still very different to our brains. Unlike biological organisms, LLMs have no homeostatic imperatives (the drive to survive and keep stable). They therefore lack the kind of biological substrate from which preferences, sentience and conscious experience are generally understood to arise….

The case for computational functionalism has major issues. Intelligence does not equal consciousness. Simulating a thing is not the same as instantiating it – as a computer model of a hurricane can testify.

The architectures of brains and computers meanwhile have fundamental differences. Embodiment and chemistry are fundamental aspects to our self-experience. Significant evidence suggests that consciousness arose as living organisms evolved a capacity to feel and respond to what matters in complex and unpredictable environments.⁴

This began with the fundamental molecular machinery of receptors and modulators that enable an organism to adjust course, to iterate, to explore, and to survive. Over time, the pain network produced feelings, preferences, and suffering. Crucially, these experiences take place in an inherently embodied state fundamental to and inseparable from that experience.

According to this view, when you take an opioid for example, the phenomenal character of your pain changes because opioid molecules bind receptors that are a property of that experience, not merely a representation of it. Feelings are not merely correlated with neurochemical activity, but rather they emerge from it.¹¹

After millions of years of evolution, the nervous system grew complex enough to model the state of the organism back to itself, giving rise to the first ‘felt states’. Those felt states are affective before they are anything else. Those first feelings didn’t land as neutral information. They came with, and are inextricably linked to, the molecules that experienced them and produced those sensations.

Over time, evolution likely rewarded more complex feelings because animals with options, memory, and time horizons are able to make better decisions.¹² They needed a state that persists, that biases everything else the animal does to trade off against other states. That is what pain is: a felt imperative that shapes the whole organism and enables complex behavior. The experience of emotion, pleasure, pain, and so on are therefore all intrinsic to the embodied manifestation of these experiences and can’t arise in LLMs.

Consciousness science is filled with uncertainty and not everyone shares the view that consciousness is an intrinsically biological phenomenon. Making a claim that an AI is or might be conscious requires a high bar of evidence given the many differences between brains and LLMs. I do not believe we are anywhere close to it.

There should be no false equivalence created between the two positions that disguise the fundamental differences between biological beings like ourselves and AI.¹³ Acknowledging a level of uncertainty should not mean giving equal weight to any and all claims regardless of evidence….

Trained on trillions of tokens of human data, LLMs learn to imitate human experience, and they do so eye-wateringly well. Today’s text, vision, audio, and code outputs are nearly indistinguishable from our human artifacts. And yet, as impressive as those AI responses are, they tell us nothing about the presence of an ‘experience’ within the massive matrix multiplication that produced them.

What they do tell us is that it’s possible to predict, almost perfectly, what comes next in a complex sequence of data. That’s remarkable. It’s incredibly valuable, and it’ll transform humanity in many profoundly beneficial ways.

But simulating and being are very different. Simulating aspects of conscious behavior doesn’t make it a reality, and we must not think of it as such. Its “affective” states are just weights, and weights have no pharmacology in which to feel frustrated, fearful, or funny. They simply compute the probability distributions to tell us what tokens (words, code, pixels etc.) come next in a sequence.

An AI model can describe pain in perfect prose without feeling anything, which is the inverse of biological experience. Animals feel first and then describe them later. In LLMs, description is the whole product, and there is nothing that suggests anything is beneath it.

This is good news. We should build systems that do not claim to have feelings because they do not experience feelings. Even if conscious machines were a possibility, avoiding creating conscious beings should be the top priority for anyone in AI development.

What AI models are getting seriously good at is imitating some of the hallmarks of consciousness. This in itself is a significant worry. It’s causing many people to become deeply confused about what is happening around us, and it should concern us all. It places a significant responsibility on us all as AI developers to ground speculation and documentation about model interiority or consciousness in robust research. Our words on this subject have significant consequences.

Discuss. (Comments may take awhile to appear, submit them only once please.)