Do harness edits learn reusable strategies or memorize task fixes?

When you open up the trajectories a harness-evolution meta-agent produces — what it modifies each iteration, which edits get kept or rolled back, which failure classes they target — the edits look rational and well-motivated across the prompt, middleware, and tool layers. Yet a stable core of hard tasks stays unsolved and the overall improvement remains limited relative to simple test-time discovery baselines. The diagnosis is sharp: most edits memorize fixes rather than distill strategies. Much of what gets written into the harness is information a competent agent could rediscover through exploration inside a single rollout, so persisting it saves time on tasks the agent could already solve but rarely converts a failure into a success.

This reframes what harness evolution is buying. The value is not "the harness learned to do new things" but "the harness caches shortcuts for things already within reach." That is why gains shrink once you control the search budget — see How should we measure gains from automatic harness evolution?.

It also sharpens a companion finding: since Do stronger models always evolve harnesses better?, the payoff concentrates exactly where memorized shortcuts substitute for capability the agent lacks. The open question is whether harness evolution can be redirected from memorization toward strategy distillation — edits that encode transferable procedures rather than per-task patches — which is what would actually move the stable core of hard tasks.