When should AI agents ask users instead of just searching?

Tool-enabled LLMs have a structural problem: when they can't immediately answer a query, they chain tool calls (search, calculation, code execution) and each intermediate step is conditioned on the output of the previous step. The result is progressive divergence from the user's original intent. The more tools the model uses, the further it drifts.

Conversation Analysis (Schegloff, 2007) offers a formal alternative from human talk-in-interaction. When human speakers can't immediately provide the expected response, they don't silently think harder — they insert a new pair of utterances to bridge the gap. These "insert-expansions" serve three functions: clarifying intent ("Do you mean the downtown location?"), scoping responses ("Are you looking for something under $50?"), and enhancing appeal ("I should mention it also comes in blue").

The key move is the "user-as-a-tool" paradigm: instead of the model consulting external tools and accumulating drift, it consults the user. The user provides necessary details and refines their request. This replicates exactly the structure of human insert-expansions — post-first inserts recover from misunderstandings, pre-second inserts gather information needed to choose the right response.

The empirical evidence from recommendation tasks shows benefits from this approach. But the deeper point is architectural: since Why can't conversational AI agents take the initiative?, the insert-expansion framework gives a principled answer to WHEN agents should break passivity — not by adding unsolicited content, but by asking structured questions when their internal processing would otherwise diverge.

This connects to the distinction between formal and functional linguistic competence: LLMs have formal competence (handling language in itself) but lack functional competence (doing things WITH language — reasoning, using world knowledge, establishing common ground). Insert-expansions are a functional linguistic capability. The paper argues that natural speech patterns may emerge as a side-effect of more closely imitated reasoning paths — if agents reason through dialogue rather than through silent chains.

Since Does preference optimization harm conversational understanding?, insert-expansions are precisely the kind of conversational work that RLHF training discourages — they slow things down, ask questions instead of answering, and score lower on single-turn helpfulness ratings, despite being more effective for multi-turn interaction. Insert-expansions are the PRE-EMPTIVE half of the repair space; since Can AI systems detect and correct misunderstandings after responding?, TPR provides the REACTIVE half -- correcting misunderstanding after it has already been acted on. Together they cover the full repair lifecycle: insert-expansions prevent, TPR recovers.

The insert-expansion framework connects to a trainable capability. Since Can models learn to ask clarifying questions instead of guessing?, RL training can bring proactive questioning from 0.15% to 73.98% accuracy — but the insert-expansion framework provides the conversational-analytic structure for WHEN and HOW to deploy that capability in dialogue, not just whether the model can detect missing information.

Source: Conversation Architecture Structure