Can Large Language Models perform Relation-based Argument Mining?

The general AM problem can be split into three main tasks: 1) argument identification, involving segmenting text into units and determining which are argumentative; 2) identification of argumentative components, typically involving classifying claims and/or premises of argumentative text; and 3) identification of argumentative relations, aiming at determining how different texts are related within argumentative discourse.

given a pair (𝐴, 𝐵) of texts𝐴 and 𝐵, determine whether𝐴 attacks or supports 𝐵. For example, take the three arguments, drawn from the Debatepedia/ Procon dataset Cabrio and Villata [2014], 𝑎1=‘Abortion should be legal’, 𝑎2=‘A baby should not come into the world unwanted’, and 𝑎3=‘Abortion increases the likelihood that women will develop breast cancer’. Here, 𝑎2 can be deemed to support 𝑎1 and 𝑎3 to attack 𝑎1.

RbAM can be used to support several downstream tasks, for example, to gather evidence Carstens and Toni [2015], to determine which online arguments are acceptable Bosc et al. [2016], and to analyse divisive issues about new regulations Konat et al. [2016]. However, it is a challenging task, with different BERT-based models performing reasonably well on some datasets but individual baselines failing to perform well across datasets Cocarascu et al. [2020]; Ruiz-Dolz et al. [2021].

consists of few-shot priming, which has shown to perform well with LLMs without the need for fine-tuning Brown et al. [2020], followed by prompting. The primer uses four labelled examples of attack and support relations between arguments, before we provide an example in the prompt for the LLM to classify as attack or support. The four examples in the primer are fixed text comprising a parent argument (Arg1), a child argument (Arg2) and the classification of the relation from the child to the parent argument, as shown in the top, pink part of the box in Figure 1. Then, the prompt amounts to a pair of arguments presented as the four in the primer, but without indicating the relation,