Simulacra as conscious exotica

The advent of conversational agents with increasingly human-like behaviour throws old philosophical questions into new light. Does it, or could it, ever make sense to speak of AI agents built out of generative language models in terms of consciousness, given that they are “mere” simulacra of human behaviour, and that what they do can be seen as “merely” role play? Drawing on the later writings of Wittgenstein, this paper attempts to tackle this question while avoiding the pitfalls of dualistic thinking.

As such, LLM-based conversational agents can be considered as simulacra of human language users, and their linguistic behaviour can be understood as a kind of role play (Andreas, 2022; Janus, 2022; Shanahan et al., 2023).

Today’s conversational agents are hardly with- out their limitations. They have a tendency to generate inaccurate, made-up information (a phenomenon often (mis-)termed “hallucination”), and their reasoning skills are poor. Nevertheless, the experience of interacting with them is sufficiently compelling, and their conversational capabilities are sufficiently close to human level, that the urge to speak of them in anthropomorphic terms is almost overwhelming.

Anthropomorphising AI systems is sometimes harmless fun, if the user is in the know, and sometimes useful for explaining and predicting the a system’s behaviour, as when we adopt what Dennett calls the intentional stance, the strategy of “interpreting the behavior of an entity ... as if it were a rational agent” (Dennett, 2009). Anthropomorphism is problematic when it involves the misleading attribution of human properties to systems that lack those properties, giving rise to false expectations for how the system will be- have.

By contrast, when I say that “ChatGPT thinks the current Wimbledon men’s champion is Carlos Alcaraz” this does come with the expectation that I could have a conversation with ChatGPT about tennis. Accordingly, the question of whether or not LLMs “really” have beliefs becomes a matter of philosophical debate.

LLMs encode a great deal of human knowledge, and a suitably fine-tuned and prompted base model will effectively play the part of a helpful assistant in a turn-taking set- ting, answering factual questions (more or less accurately) as if it believed, and had good rea- sons to believe, its own answers. In general, the role play framing allows us to use familiar folk- psychological terms to describe, explain, and predict the behaviour of LLM-based systems with- out falling into the trap of anthropomorphism.

The upshot is that we will increasingly be able to speak of the “beliefs” of a conversational agent without implied scare quotes and with fewer philosophical caveats.

Now, to what extent can we extend this treatment of belief to other mental attributes, such as desires, goals, intentions, and, most pertinently, to consciousness? To a degree, the same trends that legitimise talk of belief in enhanced conversational agents apply to the concepts of goals and intentions. A conversational agent capable of deliberation and tool-use can play the part of an assistant that forms plans on behalf of the user, and sets about executing them by carrying out real-world actions such as making purchases,

However, we can imagine further enhancements that might legitimise talk of needs, desires, and selfhood, narrowing the gap between role play and authenticity for these attributes, such as extending an agent’s lifetime and endowing it with a persistent memory.

Rather, thanks to the stochastic nature of the sampling process behind the generation of text, they are better thought of as simultaneously role-playing a set of possible characters consistent with the conversation so far (Janus, 2022; Shanahan et al., 2023). If we view the underlying language model as a simulator, then it generates a set of simulacra in superposition.

The simulator, the superposed set of simulacra it generates, and the multiverse of narrative possibility thus induced, collectively produce behaviour that is very human-like on the surface.

Rather, the point is that when we speak of consciousness, our words have meaning only insofar as they relate to what is public, what is manifest in the world we share, notably our bodies (and brains) and our behaviour.

According to the stance of the present paper, the key to dealing with more exotic entities is the ability, at least in principle, to engineer an encounter with them (Shanahan, 2016). The reason for this is that, in everyday speech, when we speak of conscious- ness, we do so against a backdrop of purposeful behaviour, in a sense of “purposeful” that only applies to entities that inhabit a world like our own, in the broadest sense. To engineer an encounter is to put ourselves in a position to meaningfully interact with an entity given the purpose we discern in it.

The ability to engineer an encounter, even if only in principle, establishes an exotic entity’s candidature for the fellowship of conscious beings. If encounters can be made to happen in reality, not just hypothetically, then the human participants may (or may not) begin to see it as a fellow conscious being, and may (or may not) start to speak of it using the vocabulary of conscious- ness.

In this way, the new entity would be absorbed into the conceptual repertoire of our language, while our language and its conceptual repertoire would adapt and extend to accommodate it.

Perhaps we will decide, collectively, that the language of consciousness is not the right one after all. Perhaps a little more nuance will be required. Perhaps a whole new vocabulary will emerge.

Put differently, if we were to try (foolishly) to visualise the space of possible minds by plotting human- likeness against capacity for consciousness, we would find no data points in the region where human-likeness is very low but capacity for consciousness is above zero. This is the void of inscrutability (Shanahan, 2016).

Instead, we can ask whether it would be possible to engineer an encounter with the entity, and how our consciousness language would adapt to the arrival of such an entity within our shared world if such encounters took place.

it is not possible to engineer an encounter with a simple conversational agent, even in principle. This is because simple LLM-based conversational agents are not embodied; we cannot be with them in a shared world.

The basis for treating other humans as fellow conscious beings is our being together in the world, and this is the original home of the language of consciousness. We can hear, look at, point to, or touch the same things; we can triangulate on them, so to speak.

Is there any way to extend an LLM-based conversational agent into an AI system with which we could engineer an encounter, and that would thereby qualify as a candidate for consciousness? Indeed there is. We could embody it.

a robot controlled by an LLM that exhibited human-like behaviour would be an especially exotic artefact.

pioneering work of Owen and others showed that it is sometimes possible to communicate with locked- in patients using an fMRI scanner (Monti et al., 2010). Should we view these episodes of communication as “encounters”, in the relevant sense, given that they do not involve the patient physically interacting with the world they share with the clinician?

Suppose an LLM- based conversational agent is embedded in a mobile device equipped with visual and audio in- put, such as phone or a mixed reality headset, and carried around by the user.12 Though not capable of self-initiated motion, or of interacting directly with physical objects, it could be argued that the agent and the user share a world. They can converse about objects in the world to which they are jointly attending, for example. Should this count as an ongoing encounter, in the pertinent sense?

We are now in a position to entertain the philosophically provocative prospect of (virtu- ally) embodied simulacra. These entities are, as we shall see, doubly philosophically provocative. First, as “mere” mimics of human behaviour, mimics that nevertheless might persuade whole communities to speak and think of them as fellow conscious beings, they present a challenge to our non-dualistic treatment of (the language of) consciousness. Second, because they inhabit a multiverse of narrative possibility, to speak of them in terms of consciousness at all is to teeter on the edge of the void of inscrutability.

… let’s suppose that a virtually embodied agent is built that fulfils the criteria for discernibly purposeful behaviour.  Though reluctant to speak directly of consciousness or sentience, these researchers routinely characterise the behaviour of their new subjects in terms of attention, awareness, motivation, goal- directedness, intention, orientation, a cloud of concepts closely associated with consciousness. In short, the scientists are broadly in accord with the ordinary users.

changing attitudes towards AI agents in this (moderately) speculative scenario are obviously analogous to the real-life example of the octopus

Perhaps it would stop seeing the AI agents as fellow conscious beings, much as a person who heard a scream and then discovered it was merely a recording would stop feeling concerned.

Alternatively, further investigation might re- veal emergent mechanisms underlying the AI agent’s mimicry that were functionally equivalent to the neural mechanisms underlying human behaviour (Wei et al., 2022). In that case, the gap between role play and authenticity would have closed. Or perhaps the community would begin thinking and speaking of the AI agents in an altogether different way, developing a whole new conceptual framework,

Humans change over time, from childhood to adulthood to old age, and take on different personas in different social situations (Goffman, 1959). Nevertheless, we take it for granted that there is some kind of stable self at the core of each of us. For every human actor, for every social chameleon, we can always meaningfully speak of the “person behind the mask”. This is not so with generative agents, which lack even the bio- logical needs common to all animals. With generative agents, it’s “role play all the way down” (Shanahan et al., 2023).

As in less exotic cases, through the experience of being with them, and by getting to know more about how they work, we will settle on a certain way of talking about them. Perhaps we will invent a whole new vocabulary to do so, a vocabulary that is “consciousness adjacent”

questions about consciousness should be approached with the imagination of a science fiction writer and the detachment of an anthropologist

Just as it is inherent in the language game of truth to say that truth is more than just a language game, it is inherent in the language game of morality to say that morality is more than just a language game.

In a mixed reality future, we might find a cast of such characters – assistants, guides, friends, jesters, pets, ancestors, romantic partners – increasingly accompanying people in their everyday lives. Optimistically (and fantastically), this could be thought of as reenchanting our spiritually denuded world by populating it with new forms of “magical” being. Pessimistically (and perhaps more realistically), the upshot could be a world in which authentic human relations are degraded beyond recognition, where users prefer the company of AI agents to that of other humans.