AI evaluators that actively dig into evidence are 100× more accurate than passive judges — but their memory is where mistakes compound.

This explores whether AI evaluators that actively dig into a task — collecting evidence and chasing dependencies — can sustain that investigation without their errors snowballing. The corpus has a sharp answer hiding in one finding: the investigation pays off, but the memory of it is where things rot. When evaluation is restructured from a single LLM judging a final answer into an eight-module agent that gathers evidence dynamically, judge error drops by roughly 100× (Can agents evaluate AI outputs more reliably than language models?). But the same study found the *memory* module cascaded errors — the very component that lets the evaluator carry context across an investigation became the channel through which mistakes accumulated. So the answer is conditional: yes, evaluators can investigate dependencies far more reliably than naive judges, but only if the system isolates errors instead of letting them flow forward.

That caveat isn't unique to evaluators — it's a general property of long delegated chains. Frontier models silently corrupt about 25% of document content over extended relay workflows, and crucially the errors *compound without plateauing* across 50 round-trips (Do frontier LLMs silently corrupt documents in long workflows?). The mechanism is the same one threatening an investigating evaluator: each step inherits the last step's drift and adds its own. Anything that accumulates state over time inherits this risk unless something external keeps resetting the error budget.

The corpus points to *where* to intervene: checking the process, not the conclusion. Verifying intermediate states and policy compliance during generation — rather than scoring the final output — lifted task success from 32% to 87%, because most failures are process violations that a final-answer check never sees (Where do reasoning agents actually fail during long traces?). Step-level confidence sharpens this further: local confidence catches a reasoning breakdown at the exact step it happens, where global averaging smears it out and hides it (Does step-level confidence outperform global averaging for trace filtering?). For an evaluator tracing dependencies, this is the difference between catching a wrong turn immediately and discovering it only after it has poisoned everything downstream.

There's a deeper reason an evaluator can't simply trust its own accumulating judgment. Models have a structural bias toward validating their own outputs — a high-probability answer they generated *feels* more correct during evaluation, creating a self-agreement loop (Why do models trust their own generated answers?). So an evaluator left to police its own investigation will tend to ratify its earlier mistakes rather than catch them. This connects to a formal ceiling: self-improvement and self-verification are bounded by the generation–verification gap — reliable correction requires something *external* to validate and enforce it, not metacognition alone (What limits autonomous capability in large language models?).

So the thing you didn't know you wanted to know: the failure of long-running evaluators isn't usually bad reasoning — it's bad bookkeeping. The investigation itself works; the accumulation is the enemy. The fixes that hold up are architectural, not cognitive — error-isolated memory, step-level verification, and an external check that breaks the self-agreement loop — rather than asking the evaluator to be smarter or more careful over time.