Can unified policy learning improve conversational recommender systems?

A CRS makes three decisions per turn: which attribute to ask about, which items to recommend if recommending, and whether this turn should ask or recommend. Existing methods typically solve one or two of these in isolation, with separated conversation and recommendation components glued together at the end. This restricts scalability and undermines training stability — gradient signals from one decision cannot inform another, and the joint trajectory of decisions across the conversation isn't optimized as a whole.

The proposal is to formulate all three decisions as a single policy learning task. A dynamic weighted graph captures the state of the conversation and reinforcement learning learns what action to take at each turn — either asking an attribute or recommending items. The graph weighting evolves as the conversation progresses, integrating evidence about the user's preferences from past turns.

The unification matters because the three decisions are tightly coupled in practice. Whether to ask depends on how confident the system is about its candidates, which depends on which attributes have been clarified, which depends on which items are still in the candidate set. Solving them separately means each component must guess at the others' state, leading to suboptimal joint behavior. A single policy can learn the trade-offs directly. The mechanism integrates conversation and recommendation components systematically rather than treating them as separate modules with brittle handoffs.

Source: Recommenders Conversational