Computational Modelling of Undercuts in Real-world Arguments

Argument Mining (AM) is the task of automatically analysing arguments, such that the unstructured information contained in them is converted into structured representations. Undercut is a unique structure in arguments, as it challenges the relationship between a premise and a claim, unlike direct attacks which challenge the claim or the premise itself. Undercut is also an important counterargument device as it often reflects the value of arguers. However, undercuts have not received the attention in the filed of AM they should have — there is neither much corpus data about undercuts, nor an existing AM model that can automatically recognise them. In this paper, we present a real-world dataset of arguments with explicitly annotated undercuts, and the first computational model that is able to recognise them. The dataset consists of 400 arguments, containing 326 undercuts.

Social media allows people to express divergent opinions on the same subject and to reach many more people than was possible in earlier times. However, the ubiquity of the internet and social media also has some negative consequences. One of these is the growing polarisation between individuals holding different beliefs and opinions. It is thus increasingly important to promote productive communication and understanding among people with opposing perspectives. This is where Argument Mining (AM) comes into play. AM aims to automatically identify and extract arguments from natural language texts (Peldszus and Stede, 2013; Green et al., 2014). It can convert unstructured textual information into structured argument data, which not only identifies the argumentative text segments in the text but also the relations between them (Prakken and Vreeswijk, 2002; Lawrence and Reed, 2020).

A critical aspect of AM is recognising and understanding various argumentative structures, including undercuts. An undercut challenges the relationship between a premise and a claim (Pollock, 1987), unlike direct attacks that challenge the claim or the premise itself. Due to its complex structure, it is difficult to annotate undercuts or to computationally model them. There exist some AM datasets with annotation of undercuts (Peldszus and Stede, 2015a; Visser et al., 2020), but they are often limited in the size, the quality of source text, or the annotation scheme. To our best knowledge, there is no existing AM models that can automatically recognise undercuts. To addresses this gap, in this paper we present a novel dataset of real-world arguments from Quora1, a popular question-answering platform.

Twitter, unlike those previous platforms, is a more open and informal platform, where arguments can take on a more combative tone. As a result, arguments there may not always be rational and often lack the depth and nuance seen on the other platforms. Additionally, due to Twitter’s character limit, arguments in tweets tend to be very short, with structures that may be too simplistic to warrant a detailed analysis. Therefore, we chose Quora over other online discussion platforms.

Our scheme only has two relation categories, namely SUPPORT (e.g. component 2 supporting component 1) and ATTACK (e.g. component 7 attacking component 3). In our scheme, the representation of undercuts does not rely on relation labels, but on the target of an attacking relation: if the target is a relation, an undercut occurs (e.g. component 4 undercutting the relation between component 5 and component 2); otherwise it is just a typical direct SUPPORT.

We define four component categories in our annotation scheme, including PROPOSITION, STAKE, ANECDOTE, and ANALOGY. These categories are decided based on our manual observation of arguments on Quora and the argumentation schemes by Walton et al. (2008).

We manually examined all undercuts in the QuoraAM dataset, classifying them into three categories:

• Rejection: Rejecting the relation by denying the relevance between the source component

and the target component.

• Low importance: Questioning the importance of the relation, or providing more important

reasons.

• Alternative option: Stating that the current solution is not the only option, or providing alternative options. This kind of undercuts often appears in arguments about policies.

 Figure 2 shows the distribution of undercuts in the QuoraAM dataset. “Low importance” (41%) is the most frequent, followed by “Alternative option” (36%), “Rejection” (15%), and others (8%). This indicates that Quora authors prefer less direct methods of undercutting relations, often pointing out weaknesses or suggesting alternatives rather than outright rejection.