Can AI agents learn when they have something worth saying?

The Inner Thoughts framework reverses the standard approach to AI proactivity. Instead of asking "who speaks next?" (next-speaker prediction, which fails to beat the "repeat last" baseline in social contexts), it asks "does this agent have something worth saying right now?"

The framework is inspired by cognitive psychology's distinction between covert responses (internal thoughts, feelings) and overt responses (verbal utterances, gestures). During human conversation, we process others' words, reflect on our experiences, and develop an internal train of thought. At some point we feel a strong urge to contribute — when we seek clarification, when someone mentions something we've experienced, when we detect a gap. The Inner Thoughts framework equips AI with this parallel covert stream.

Five stages structure the framework:

1. Trigger — detecting a conversation moment worth processing
2. Retrieval — accessing long-term and working memory for relevant knowledge
3. Thought formation — generating a covert thought about the current exchange
4. Evaluation — scoring intrinsic motivation to express this thought
5. Participation — deciding whether to contribute based on the evaluation

The intrinsic motivation model draws from a think-aloud study with 24 participants across four group chats. Ten high-level themes emerged for how people decide to engage: relevance, information gap, emotional resonance, social obligation, etc. These are formalized into automatic evaluation criteria.

Technical evaluation shows agents driven by Inner Thoughts significantly outperform next-speaker prediction plus persona baselines across seven metrics: turn appropriateness, coherence, anthropomorphism, perceived engagement, intelligence, initiative, and adaptability. Participants preferred Inner Thoughts 82% of the time.

The distinction from CoT/ToT/o1 reasoning is important: those externalize intermediate steps for task reasoning. Inner Thoughts generate a parallel covert stream that models social motivation to participate — not task decomposition but interaction participation. Since Why can't conversational AI agents take the initiative?, this framework provides a concrete architecture for the missing proactivity.

Curiosity reward for personalization is a specific application of intrinsic motivation. While Inner Thoughts uses 10 general social motivation heuristics (from cognitive psychology think-aloud studies), the curiosity reward approach targets a specific type of intrinsic motivation: reducing uncertainty about the user's type. The agent is rewarded for each turn that improves its belief about who it's talking to — encouraging strategic questions and context-sensitive probes aimed at uncovering user preferences, personality, or attributes. This is personalization-specific proactivity rather than general social proactivity. The two approaches may be complementary: Inner Thoughts determines when to speak based on social motivation; curiosity reward determines what to say to learn about the user. See Can conversations themselves personalize without user profiles?.

Source: Conversation Topics Dialog