How does instruction density affect model performance?

Production LLM systems routinely require adherence to dozens or hundreds of simultaneous instructions — style guidelines, business rules, compliance standards, tool usage protocols. IFScale measures how performance degrades as instruction density increases using 500 keyword-inclusion instructions for a business report writing task.

Key findings across 20 SOTA models from 7 providers:

Three degradation patterns correlate with model size and reasoning capability:

1. Linear decay — steady degradation from the start (smaller models)
2. Exponential decay — accelerating degradation as density increases (mid-range models)
3. Threshold decay — near-perfect performance maintained until a threshold, then steep decline (reasoning models: gemini-2.5-pro, o3 maintain through ~150 instructions)

Primacy effects follow a non-obvious pattern: minimal bias at low density, peak at 150-200 instructions (where models begin to struggle), then converge toward 1.0 at extreme density (300+). The convergence indicates a shift from selective instruction satisfaction to uniform failure — an "instruction saturation point" where the model is completely overwhelmed.

Two error types: omission errors (complete failure to include required terms) and modification errors (morphological variants like "accountable" when "accountability" was required). The distinction has practical implications for prompt design — models may recognize the concept but fail at exact specification.

Even the best frontier models achieve only 68% accuracy at maximum density. Deliberative processing architectures (reasoning models) provide robust tracking up to critical thresholds, extending the useful range significantly but not eliminating the ceiling.

Source: Flaws