Can cognitive scaffolding improve how models reason about social scenes?
This explores whether structuring visual reasoning through perception, situation, and norm stages—grounded in how humans actually think—helps language models tackle socially complex tasks better than standard reasoning approaches.
Cognitive Chain-of-Thought (CoCoT) introduces a three-stage prompting strategy for visual language models that mirrors how humans process socially complex scenes. Unlike flat CoT that treats reasoning as linear, CoCoT structures reasoning through progressively abstract interpretation stages grounded in cognitive science.
The three stages:
Perception (Embodied) — "Based on the image, describe what is directly observable." Anchors reasoning in concrete perceptual evidence. The model actively interprets rather than passively processing visual features.
Situation (Embedded + Enactive) — "Based on the identified elements, determine the relationships or context among them." Captures social dynamics and contextual cues from lived interaction. Infers situational meaning beyond surface perception.
Norm (Extended) — "Based on the above reasoning stages, infer the most socially plausible interpretation." Reasons over socially constructed values and expectations that transcend the immediate context but remain grounded in prior interpretation.
The theoretical grounding is 4E cognition (Newen et al. 2018): cognition is Embodied (shaped by bodily interactions), Embedded (situated in environmental context), Enactive (emerging through action and interaction), and Extended (augmented by external tools and social structures). CoCoT maps each stage to a different cognitive dimension.
Results: +8% average improvement over flat CoT and direct prompting across intent disambiguation, commonsense reasoning, and safety benchmarks. The improvement is specifically on socially complex visual tasks where bridging perception with norm-grounded judgment is essential.
Since Can reasoning topologies be formally classified as graph types?, CoCoT represents a different structural principle: not branching or graph traversal, but progressive abstraction within a linear chain. Each stage constrains the next — norms must be grounded in situations, which must be grounded in perceptions. This enforces interpretive coherence that flat CoT does not.
The connection to social reasoning is direct. Since Why do reasoning models struggle with theory of mind tasks?, social tasks may specifically benefit from cognitively structured scaffolding rather than more reasoning tokens. CoCoT's success on social benchmarks supports this: the structure matters more than the volume.
Source: Social Theory Society
Related concepts in this collection
-
Can reasoning topologies be formally classified as graph types?
This explores whether Chain of Thought, Tree of Thought, and Graph of Thought represent distinct formal graph structures with different computational properties. Understanding this matters because the topology itself determines what reasoning strategies are possible.
CoCoT is a different structural principle: progressive abstraction rather than branching
-
Why do reasoning models struggle with theory of mind tasks?
Extended reasoning training helps with math and coding but not social cognition. We explore whether reasoning models can track mental states the way they solve formal problems, and what that reveals about the structure of social reasoning.
social tasks benefit from cognitive structure over volume
-
Can modular cognitive tools boost LLM reasoning without training?
Does structuring reasoning as discrete, sandboxed tool calls elicit stronger problem-solving in language models compared to monolithic prompting approaches, and can this approach match specialized reasoning models?
CoCoT provides an alternative: cognitive scaffolding through prompting rather than tool calls
Click a node to walk · click center to open · click Open full network for a force-directed map
Original note title
cognitive chain-of-thought scaffolds visual reasoning through three cognitively grounded stages — perception situation and norm