Why should low-probability severe risks trigger early intervention?

This explores why a risk that probably won't materialize still deserves action now, and the corpus offers a sharper answer than 'better safe than sorry.' The clearest case comes from work splitting AI's social risks into two timelines Are risks from seemingly conscious AI already happening?. Some harms — emotional dependence, autonomy erosion — are already happening and high-probability. Others — status erosion, political strife — are low-probability but severe *and path-dependent*. That last word is the whole argument: path-dependent risks foreclose their own fixes. By the time the low-probability event is visibly underway, the cheap intervention window has already closed, because the system has settled into a configuration that's expensive or impossible to reverse. Early intervention isn't caution; it's the only intervention that still has leverage.

The corpus also warns that our intuitions about *which* risks are severe are often wrong. A frontier risk assessment across seven capability areas found models crossing warning thresholds for persuasion and manipulation while staying safely green on the headline-grabbing fears — cyber offense, self-replication, autonomous AI R&D Where do frontier AI models actually pose the greatest risk today?. That inverts the usual hierarchy. The risks people rate as low-probability-but-catastrophic (rogue autonomy) turn out not to be where the early action is needed; the quieter, more diffuse harm (persuasion) is. So 'low-probability severe' is only a useful trigger if you've correctly identified which low-probability events are also path-dependent and which are merely dramatic.

There's a second reason early intervention pays off: where you intervene matters more than how hard. One perceptual move — treating AI as a conscious mind — generates a whole heterogeneous surface of downstream risks at once Does perceiving AI as conscious create multiple distinct risks?. Catching that move early, at the level of interaction design, is more effective than trying to clean up each downstream harm after it has branched. This is the same shape as research on human oversight of AI systems: targeted intervention at a few high-leverage decision points beat both full autonomy and exhaustive step-by-step checking Does targeted human intervention outperform both full autonomy and exhaustive oversight?. Early, selective, well-placed action outperforms late, comprehensive action — because the leverage is upstream.

Put together, the corpus reframes the question. The reason to act early on low-probability severe risks isn't that probability times severity is high. It's that severity plus path-dependence plus upstream leverage means the cost of waiting compounds while the cost of acting shrinks. The risks worth pre-empting are the ones where the road only runs one way — where, once you can confirm the danger, you can no longer afford the cure.