Can large language models classify argument schemes reliably?

Classifying an argument under Walton's taxonomy of 60+ schemes is a harder task than it looks. It requires recognizing the form of presumptive inference (argument from expert opinion, argument from cause to effect, argument from analogy) rather than the surface lexicon. The systematic evaluation across seven LLMs finds that zero-shot prompting fails almost uniformly; few-shot with examples helps; but the reliable lift comes from adding descriptions of the schemes — and even then, only larger models clear F1 ~0.55, with Claude topping out at 0.65.

The size-dependence is the most informative finding. Smaller LLMs and pre-trained language models like BERT (F1 0.53) plateau in roughly the same range. This is not a "scale solves it" curve — it is a step function: the task seems to require enough representational capacity to hold an abstract scheme template in working memory while comparing it against a candidate argument. Below that capacity, models pattern-match on surface lexical features and miss the inferential structure that defines a scheme.

The cognitive-load framing the authors invoke is consistent with this: scheme classification is harder than component identification (claim, premise, warrant) or stance detection because the unit of recognition is a pattern of reasoning, not a piece of text. A premise is recognizable from its position; a scheme is recognizable only by integrating premises, conclusion, and the inferential move connecting them.

The practical consequence for argumentation systems: zero-shot scheme tagging is not yet a viable component. Pipelines that need scheme labels — for argument generation, legal/medical reasoning, dialectical evaluation — need at minimum few-shot with descriptions and larger models. The cheaper alternative is to use scheme critical questions as a prompting structure instead of trying to classify into schemes after the fact.