Theory of Mind abilities of Large Language Models in Human-Robot Interaction : An Illusion?

we study a special application of ToM abilities that has higher stakes and possibly irreversible consequences : Human Robot Interaction. In this work, we explore the task of Perceived Behavior Recognition, where a robot employs an LLM to assess the robot’s generated behavior in a manner similar to human observer. We focus on four behavior types, namely - explicable, legible, predictable, and obfuscatory behavior which have been extensively used to synthesize interpretable robot behaviors. The LLMs goal is, therefore to be a human proxy to the agent, and to answer how a certain agent behavior would be perceived by the human in the loop, for example "Given a robot’s behavior X, would the human observer find it explicable?".

A first analysis of the belief test yields extremely positive results inflating ones expectations of LLMs possessing ToM abilities.

A vital component of such interaction is the Theory of Mind (ToM), which involves attributing mental states – such as beliefs, intentions, desires, and emotions – to oneself and others, and to understand that these mental states may differ from one’s own. ToM has extensive roots in Human-Human interaction and has motivated several critical studies in pursuit of social intelligence [2, 3, 15, 60]. Moreover, ToM is the corner stone enabling effective communication, collaboration, or deception and becomes a pre-requisite for most of human-agent interaction [12, 20, 29, 43, 57]. Everyday interactions which are second nature to human conversations like our ability to empathize with a character in a movie or understand social humor is in part due to our ability to perform theory of mind and comes naturally to humans [41].

Theory of Mind becomes all the more important in Human-Robot Interaction to facilitate improved behavior synthesis [8, 9, 11, 26– 28, 47, 48, 69] and engender Human-Robot trust [14, 65–68]. Prior works have either assumed a human mental model [9] or learned it through interaction [56, 69]. Recent developments in generative AI, specifically LLMs may have opened a new way to access approximate human mental model to enable robots achieve their HRI objectives in better ways through symbolic [49] or other modalities [16, 55, 56]. Initial research hinted at emergent LLM’s Reasoning, Planning & Theory of Mind abilities [6, 21, 30, 46], however, recent works have refuted such claims [44, 52, 54]. While LLMs may not be sound reasoners or perform ToM as humans do, their performance on existing benchmarks for such tasks certainly exceeds chance behavior and it may find significant use as a Human-Proxy available to the robot.