Is AI's real danger superintelligence or loss of oversight?

Sacasas, writing in The Convivial Society, contends that "the traditional danger of AI is usually thought to be superintelligence acting as an existential threat," but that this framing "may miss the true and more subtle danger": AI as "a mechanism for transferring the processes of our civilization from under the supervision of consciousness to unconsciousness." He builds the case by setting two thinkers against each other. Whitehead held that "civilization advances by extending the number of important operations which we can perform without thinking about them." Arendt, writing in 1958 amid fears that technical capacity was outstripping ordinary-language comprehension, insisted on the opposite: "to think what we are doing" — warning that otherwise "we would indeed need artificial machines to do our thinking and speaking," which she called "a political problem of the first order" because self-governance depends on being able to say in speech what we are doing.

Sacasas's own move is the analogy to the unconscious. As "agentic AI" is delegated tasks "at the personal, organizational, and institution levels," it generates "a layer of activity in the world that is functionally sundered from active human judgment and oversight," so that "the ratio of conscious to unconscious human action shrinks." He distinguishes this from earlier opaque industrial systems, which were "relatively sequestered from the course of ordinary human activity" — AI, by contrast, is "woven far more intimately into our experience," acting both on us and for us, a "proximate intermingling of human and machine" that is what makes the unconscious analogy apt rather than merely rhetorical. The consequence he draws is epistemic rather than existential: a world run by action nobody is consciously tracking becomes progressively "uninterpretable, ever more strange and unintelligible," a condition he links to "the great ensloppification of the commons."

This names a different mechanism for the same destination as Does incremental AI replacement erode human influence over society?: that note locates lost oversight in AI replacing the human labor that implicitly kept systems aligned, while Sacasas locates it in delegating judgment and oversight themselves to an unsupervised layer of action — both describe systems drifting from human control without any single decision to allow it. It also scales up Does AI augmentation protect workers from skill erosion? from the workplace to the civilizational register: Sacasas's "unconscious" is what that erosion looks like once aggregated across "personal, organizational, and institution levels." It restates, outside the Adorno/Horkheimer frame, the same reversal traced in Does AI repeat the Enlightenment's reversal into its opposite? — automation offered as liberation from conscious labor, delivered as a loss of the oversight that liberation was supposed to free up. And where Are we underestimating human minds while debating machine minds? redirects attention from mind wrongly attributed to machines toward mind wrongly denied to humans, Sacasas redirects attention from existential risk toward epistemic risk — both essays share the move of naming what mainstream AI-risk debate is missing, but locate the missing piece differently.

The excerpt does not quantify how large this unconscious layer already is, nor name specific deployments beyond the general claim that delegation is occurring "at the personal, organizational, and institution levels" — Sacasas offers the unconscious analogy explicitly as "a useful analogy," not as a mechanism he claims to have demonstrated. It also does not address whether design responses — audit trails, human-in-the-loop checkpoints — could keep delegated action within conscious oversight rather than outside it; the essay reads the current trend as running toward unconsciousness, not as fixed by definition. If the analogy holds even loosely, the implication is that existing AI-safety vocabulary, split between existential and mundane risk, is missing a third category: epistemic risk from action that nobody, human or machine, is tracking consciously.