Do reasoning scaffolds reshape which empathy skills models develop?
When language models receive identical empathy rewards, does adding explicit reasoning blocks before responses change which capabilities they actually improve? This matters for understanding how training structure, not just training signal, shapes model development.
Under RLVER training with identical verifiable emotion rewards, models with and without explicit reasoning scaffolds develop along different axes:
- Thinking models (with
<think>...</think>blocks before each response) enhance empathy and insight — understanding the user's emotional state, anticipating the impact of words, formulating multi-step conversational plans - Non-thinking models focus on action-oriented capabilities — providing helpful solutions, directing toward resources, taking practical steps
This divergence under the same training signal is the key finding. The explicit reasoning scaffold doesn't just improve the model — it redirects what the model improves at. The think-then-say template forces the model to "access and refine higher-order empathetic skills" by externalizing its reasoning about the user's emotional state before responding.
This connects to the broader reasoning literature in two ways:
First, it parallels Does RL teach reasoning or just when to use it? — the thinking scaffold provides a pre-existing mechanism (extended deliberation), and RL teaches the model when and how to apply that mechanism to empathetic dialogue. The capability was latent; RL surfaces it through the scaffold.
Second, it complicates When does explicit reasoning actually help model performance?. Empathy is arguably a "continuous nuanced judgment" task, yet the thinking scaffold helps. The resolution may be that the scaffold here works not by imposing logical structure on empathy, but by creating space for the model to deliberate about social context before committing to a response.
Source: Psychology Empathy
Related concepts in this collection
-
Does RL teach reasoning or just when to use it?
Does reinforcement learning in thinking models actually create new reasoning abilities, or does it simply teach existing capabilities when to activate? This matters for understanding where reasoning truly emerges.
parallel mechanism: scaffold provides capability, RL teaches deployment
-
When does explicit reasoning actually help model performance?
Explicit reasoning improves some tasks but hurts others. What determines whether step-by-step reasoning chains are beneficial or harmful for a given problem?
apparent counter-example: reasoning scaffold helps with empathy (nuanced judgment), but may work via social deliberation not logical derivation
-
Why do reasoning models struggle with theory of mind tasks?
Extended reasoning training helps with math and coding but not social cognition. We explore whether reasoning models can track mental states the way they solve formal problems, and what that reveals about the structure of social reasoning.
the thinking scaffold may work for empathy precisely because it enables social deliberation rather than formal reasoning
Click a node to walk · click center to open · click Open full network for a force-directed map
Original note title
Thinking and non-thinking models develop distinct empathy profiles under RL training — thinking models enhance empathy and insight while non-thinking models focus on action-oriented capabilities