Does sparse attention trade off quality for speed?

Sparse attention has been treated as a cost-quality trade-off: it reduces computation, but at the price of some accuracy. The empirical analysis in The Sparse Frontier — the largest-scale evaluation of training-free sparse attention to date, across six methods, multiple model families, sequences up to 128K tokens, and sparsity levels up to 0.95 — argues that this framing is wrong at the right comparison point.

The key result: at equivalent compute cost, larger sparse-attention models outperform smaller dense models. The relevant comparison is not "dense model vs sparse-attention version of the same model" but "dense model vs larger sparse model at the same dollar cost." Under the latter comparison, sparse attention is Pareto-improving — it expands the cost-performance frontier rather than moving along it.

The mechanism is straightforward in retrospect. A sparse-attention model spends less compute per token, so for the same compute budget you can train (or run) a larger model. That larger model has more parameters, captures more knowledge, and on long-context tasks where attention is the bottleneck, the sparse version of it outperforms a smaller dense baseline despite using only a fraction of the attention budget. Sparsity is a way to spend the saved compute on capacity rather than to keep capacity fixed.

This reframes the deployment decision. The default question — "should we use sparse attention?" — implicitly assumes a fixed model. The better question is "given our compute budget, should we run a smaller dense model or a larger sparse one?" The Sparse Frontier evidence answers: a larger sparse model in most long-context settings.

The finding is bounded. It holds across the tasks evaluated and across the sparsity levels tested. It does not say sparse attention is universally Pareto-improving — task-dependence and sparsity-tolerance variation matter, and the paper documents these. But the headline claim — that sparse attention expands the frontier rather than trading along it — is robust enough to change how compute-budgeted deployments should think about architecture choice.