What role do multi-dimensional quality frameworks play in assessing arguments versus single-metric approaches?

This explores whether assessing an argument (or a prompt, or a reasoning trace) as a structured space of distinct qualities works better than collapsing it to one score — and the collection keeps landing on the same answer from different angles: single metrics hide the thing you actually care about.

The clearest case for frameworks comes from argument assessment itself. Models fine-tuned on labeled 'good vs. bad' examples learn surface patterns and fail to transfer to new argument types — they need an explicit theoretical scaffold (criteria like RATIO or QOAM) to learn principled quality rather than mimicry Can models learn argument quality from labeled examples alone?. Prompt quality tells the same story from the other side: it decomposes into six measurable dimensions grounded in communication theory, and improving one cascades into others — meaning quality is a connected structure, not a flat checklist you can average away Can we measure prompt quality independent of model outputs?. The lesson that recurs is that a single number is a lossy compression of something with internal shape.

What's striking is how this shows up wherever evaluation happens, even far from 'arguments.' A model can post perfect accuracy while its internal representations are fractured and brittle — the headline metric is blind to the disorganization underneath Can models be smart without organized internal structure?. Reasoning traces show the same pattern: a global confidence average smooths over the exact step where reasoning breaks, while step-level scoring catches the local collapse the average hides Does step-level confidence outperform global averaging for trace filtering?. And human annotations — the raw material of 'quality' labels — turn out to contain three distinct signal types (genuine preferences, non-attitudes, constructed preferences); treating them as one uniform measure quietly contaminates everything trained on them Do all annotation responses measure the same underlying thing?. Decomposition isn't a stylistic preference here; it's what makes the failure visible.

But the corpus also pushes back, which is the interesting part. Decomposition only helps when the pieces are real. Structured novelty assessment that splits into extract-claims / retrieve / compare reaches 86% alignment with human reviewers, beating holistic scoring Can structured pipelines make LLM novelty assessment reliable? — and an eight-module agentic judge cuts evaluation error by 100x over a single LLM-as-judge Can agents evaluate AI outputs more reliably than language models?. Yet that same agentic judge had a memory module that cascaded errors, and a separate analysis of reasoning frameworks found that once you control for total compute, elaborate multi-step machinery converges with simple methods — the framework mattered less than the budget and the reliability of the underlying reward signal Does the choice of reasoning framework actually matter for test-time performance?. More dimensions can mean more places to break.

The thing you might not have expected to learn: the hardest limit on argument assessment isn't the number of dimensions at all. An argument's force partly comes from who makes it — reputation, track record, standing in a field — and a text-only model loses that social context entirely, scoring expert claims and common assumptions as equally weighted prose Can language models distinguish expert arguments from common assumptions?. No quality rubric, however multi-dimensional, recovers a signal that was never in the text. Multi-dimensional frameworks beat single metrics because they refuse to average away what matters — but they can only measure the dimensions that survived the act of writing things down.