“Understanding AI”: Semantic Grounding in Large Language Models

This motivates another method: looking under the hood of systems and exploring their internal mechanisms and functions. But in the case of deep learning neural networks, the notorious black box problem looms, i.e. the fact that sophisticated AI systems are notoriously opaque. This is indeed a problem that should not be underestimated. I would like to give it the following twist here: By bringing their rather simplistic and premature analogies of octopuses and parrots to LLMs, the semantic skeptics demonstrate their perhaps overly hasty assumptions about the inner workings of transformers – an instance of the black box problem.

To avoid falling into the same trap, I propose a method for assessing the question of grounding that consists in the top-down application of theories of meaning from the analytic philosophy of mind and language. This goes hand in hand with important insights into the nature of semantic grounding. First, it is by no means a simple yes-no question. Second, it is multi-criterial, and I will distinguish three basic types of grounding under the titles functional, social and causal grounding. Third, grounding is also gradual, in other words, it comes in degrees. My analysis aims to show that modern LLMs possess a decent functional, a weak social, and an indirect causal grounding. The strongest argument for grounding is that LLMs develop world models.

They can be characterized as top-down and bottom-up. The top-down method consists of applying theories, models, and explanations of semantics. This includes the common canon of theories of semantics and meaning as well as programs for naturalizing mental representation in analytic philosophy of mind and philosophy of language. In contrast, the bottom-up method focuses on the AI system’s internal realization by looking under the hood of its inner architecture, functioning, and mechanisms for producing semantic capabilities.

While the extrospective approach comprises three methodological options, only two introspective methods can be distinguished. The introspective-behavioral method aims at self-reports. In some sense, they are the counterpart of Turing tests. The counterpart of the theoretical and explanatory top-down method are self-explanations and self-reports where a system reflects or justifies its own performance.

Hence, from a systematic point of view, we can distinguish five methods by which semantics can in principle be tested:

E. Extrospective methods

(1) Behavioral: general performance

(2) Top-down: application of theories of semantics

(3) Bottom-up: mechanistic realization

I. Introspective methods

(1) Self-reports

(2) Self-explanation

As a first observation: ChatGPT's answers not only contain self-reports but also elements of self-explanations. In this respect, LLMs perform even better than most people. ChatGPT does, however, not separate phenomenality from intentionality. The overall response behavior suggests that the fine-tuning of OpenAI is designed to keep the system away from any suspicion of consciousness or personhood. Recent findings also indicate that prompting is crucial when interacting with LLMs. Prompt engineering has even become a professional skill. Terrence Sejnowski (2023) has argued that LLMs reflect the intelligence of their users. They tune in to their interlocutors. In Sejnowski's words: "In mirroring the interviewer, LLMs may effectively be carrying out a much more sophisticated reverse Turing test, one that tests the intelligence of our prompts and dialog by mirroring it back to us." This analysis is consistent with the Theory-of-Mind-capabilities of GPT-4 already highlighted in the foregoing section 2.1.

Taken together, this shows that introspective methods are not reliable and are subject to at least as much suspicion as E1 methods. On the other hand, the situation in the field of AI does not differ significantly from that in humans. Introspection can also be criticized in psychology with good reasons (cf. Schwitzgebel 2019). With regard to self-explanations, AI systems perform even better than humans, who can generally provide little, if any,information about the cognitive background to their behavior.

Functional grounding is a purely system-internal matter, no direct connection between LLMs and the world has yet been taken into account. Social and causal grounding are two principled ways of doing so. And, interestingly enough, social grounding can be viewed as a natural extension of functional grounding. The idea is to extend the concept of functional roles beyond the system boundary. This leads to the functional roles that cognitive agents and systems play in the world through their behavior. Block (1998) has dubbed them “longarm” functional roles as opposed to the traditional short-arm roles that stop at the system boundary. Long-arm functional roles of linguistic behavior are reminiscent of Wittgensteinian use-theoretic conceptions of meaning, where linguistic meaning is constituted by the functional roles of the speaker’s utterances or expressions in the language game. Thus, meaning manifests itself in the use of language and is bound to the existence of a linguistic community. Accordingly, rule following (whether in the sense of grammar or otherwise) is considered by Wittgenstein (1953) as a social practice. We humans have a social grounding precisely to the extent that we are participants in the language games of our language communities.

Such ideas can directly be applied to LLMs. They become participants in our language games precisely to the extent that we increasingly include them in our linguistic practices. This process makes it particularly clear that social grounding, and indeed semantic grounding in general, is gradual in nature. The more useful LLMs are, the more they are integrated into our linguistic practices, and the more they become established as communicative partners, the more they acquire a social grounding. This happens in full analogy to children who gradually grow into their language communities. It is straightforward to assume that the strongest LLMs have already acquired an elementary social grounding comparable to what we find in young children.

Of course, the weak and elementary social grounding of LLMs comprises linguistic behavior only. Lacking any embodiment and interventional skills, the behavior of LLMs is limited to language-based behavior and cannot (yet) possess the full grounding open to causally interactive agents or systems. At this point social and causal grounding overlap. The example of chess from the previous section is illustrative and tricky in this respect. Wittgenstein made it vehemently clear that “playing games” is a practice (and he used the example in a variety of ways to discuss, for instance, family resemblances, language games, or rule following). Following Wittgenstein, MuZero doesn’t "play" chess (nor does any other chess program), since current AI systems still lack the social context: the shared and public exercise of games as a behavioral practice. Hence, LLMs possess a functional grounding as well as a weak social grounding limited to language-based behavior.

Current LLMs have not yet reached a direct causal grounding (modulo first multimodal approaches and approaches in robotics). However, the models are trained with large amounts of data made available on the Internet. This data has mostly been produced by us humans, i.e. causally grounded beings. This opens the door to a sort of indirect causal grounding. Such indirect grounding is functionally established by the stunning fact that modern LLMs develop world models, i.e. representations that are structurally similar to (parts of) the world. Put differently: the totality of text and language data is like a huge mirror of the world created by us humans, and modern LLMs are capable of extracting lawlike world structures and regularities from the huge amounts of text data from which they learn. Indeed, the breathtaking successes of recent LLMs would be downright mysterious if we did not make the assumption that these systems form grounded world models, even if only in an indirect causal manner.

In fact, an increasing number of recent studies suggest that LLMs possess world knowledge (see examples below).