DR-HAI: Argumentation-based Dialectical Reconciliation in Human-AI Interactions

“The significance of creating AI agents that can establish trust and accountability by interacting with human users is constantly increasing. This is the central idea behind the field of explainable AI planning (XAIP), which aims to design agents that are transparent and explainable by considering the human user’s knowledge, preferences, and values [32]. A crucial aspect in XAIP is the human-AI interaction, where the AI agent is expected to communicate effectively with humans to enhance their understanding and address their concerns. Most research efforts within XAIP have been placed on (sequential) decision-making tasks involving an agent and a human user, and the goal is to make the agent’s decisions explainable and transparent to the human user when those decisions appear inexplicable to them. While there have been a plethora of approaches to solve XAIP from different perspectives [6], a process called model reconciliation has garnered growing interest [7, 28, 30, 31, 35, 36]. In model reconciliation, it is assumed that the agent and the human user each have their own (mental) models of the task, and the need for explainability arises due to some knowledge asymmetry between these two models that makes the agent’s decisions inexplicable with respect to the human user’s model. A solution in model reconciliation is then an explanation (technically, a minimal set of model updates) from the agent to the human user such that the agent’s decisions become explicable to the human user.

Despite the growing popularity of model reconciliation approaches, we identify two limitations: (1) It is commonly assumed that the agent possesses the human user’s model a-priori in order to anticipate their goals and predict how its decision will be perceived by them. This may lead to incorrect assumptions about the human user’s knowledge and preferences, and consequently to unsatisfactory explanations. (2) Most model reconciliation approaches are formulated around a single-shot reconciliation paradigm, that is, they focus on generating a single, albeit comprehensive, explanation that is presented all at once. While this type of reconciliation can be useful when the human user needs to quickly understand a decision or when the underlying task is relatively simple, it may fail to work for more complex decisions and tasks that require a more detailed understanding from the human user, especially when there is substantial knowledge discrepancy between the agent and user models.”

Indeed, with DR-HAI, we are introducing a new dialogue type: dialectical reconciliation. Related dialogue types include the following: Persuasion [13,26], where an agent attempts to persuade another agent to accept a proposition that they do not initially hold; information-seeking [12,24], where an agent obtains information from another agent who is believed to possess it; and inquiry [3,16], where two agents collaborate to find a joint proof to a query that neither could individually. Although many dialogue systems have been proposed for the aforementioned dialogue types, to the best of our knowledge there are no existing dialogue frameworks that consider the use of dialectical reconciliation that is aimed at enhancing an explainee’s understanding. Table 4 shows a logical description and comparison of the four dialogue types.

Limitations and Future Directions: Despite the promising aspects of DR-HAI, it is important to acknowledge its limitations and potential areas for improvement. DR-HAI follows a fixed structure in presenting arguments and does not consider the effectiveness of personalizing the interactions according to each user’s beliefs and preferences. In addition, the current model assumes that both agents communicate through well-defined dialogue moves and that their communication is seamless. In reality, however, communication might be affected by factors such as miscommunication or uncertainty. Finally, the current framework is limited to propositional logic, which may not be sufficient to express complex relationships and dependencies in real-world domains.