Turiya at DialAM-2024: Inference Anchoring Theory Based LLM Parsers

Representing discourse as argument graphs facilitates robust analysis. Although computational frameworks for constructing graphs from monologues exist, there is a lack of frameworks for parsing dialogue. Inference Anchoring Theory (IAT) is a theoretical framework for extracting graphical argument structures and relationships from dialogues. Here, we introduce computational models for implementing the IAT framework for parsing dialogues. We experiment with a classification-based biaffine parser and Large Language Model (LLM)-based generative methods and compare them. Our results demonstrate the utility of finetuning LLMs for constructing IAT-based argument graphs from dialogues, which is a nuanced task.

where argumentation and dialogue information are modeled jointly in the domainin-dependent IAT framework (Budzynska et al., 2014, 2016; Janier et al., 2014). The framework represents dialogues as a graph where the nodes comprise (i) Locutions (L-nodes)-the Argumentative Discourse Units (ADUs) from each speaker turn. (ii) Propositions (I-nodes)-reconstructed L-nodes with resolved anaphora, pronouns, and deixis, making them independently coherent. The edges comprise (i) Default Transitions (TAs) between L-nodes. (ii) S-nodes that connect propositions (I-nodes) and can be of types RA (default inference), MA (default rephrase), or CA (default conflict). (iii) YA-nodes that connect L-nodes with I-nodes, TAs with S-nodes, or TAs with I-nodes.

We ask the following research questions: (i) Can LLMs be used for parsing dialogues in the IAT framework? We experiment with Mistral-7B-Instructv0.2 (Jiang et al., 2023) and present dialAM as a generative task where the L-nodes, I-nodes, and TA-nodes are the context of the LLM, and the task comprises determining the propositional (Task A) and illocutionary (Task B) relations. (ii) How do LLMs compare against simpler classification-based dialogue parsers? We compare the LLM parser against a biaffine-parsing-based parser that predicts the relationship and type between nodes.

Here, we computationally implement the theoretical IAT framework using classification and LLM-based models. We question the viability of leveraging LLMs, which are generative models, for such a nuanced task and compare them against simpler classifiers (non-generative) such as biaffine parsers. Our results indicate that posing the graph construction problem as a generative task and finetuning LLMs outperforms biaffine classifiers. Furthermore, ensembling the generative and classification-based approaches yields the best results.