Incorporating External Knowledge and Goal Guidance for LLM-based Conversational Recommender Systems

In light of this finding, we propose a novel ChatCRS framework to decompose the complex CRS task into several subtasks through the implementation of 1) a knowledge retrieval agent using a tool-augmented approach to reason over external Knowledge Bases and 2) a goal-planning agent for dialogue goal prediction. Experimental results on two multi-goal CRS datasets reveal that ChatCRS sets new state-of-the-art benchmarks, improving language quality of informativeness by 17% and proactivity by 27%, and achieving a tenfold enhancement in recommendation accuracy1.

Motivated by the empirical evidence that external inputs can significantly boost LLM performance on both CRS tasks, we propose a novel ChatCRS framework. It decomposes the overall CRS problem into sub-components handled by specialized agents for knowledge retrieval and goal planning, all managed by a core LLM-based conversational agent. This design enhances the framework’s flexibility, allowing it to work with different LLM models without additional fine-tuning while capturing the benefits of external inputs (Figure 2b).

Training-based methods, which train LMs to memorize or interpret knowledge representations through techniques like graph propagation, have been widely adopted in prior CRS research (Wei et al., 2021; Zhang et al., 2023). However, such approaches are computationally infeasible for LLMs due to their input length constraints and training costs. RA methods, which first collect evidence and then generate responses, face two key limitations in CRS (Manzoor and Jannach, 2021; Gao et al., 2023). First, without a clear query formulation in CRS, RA methods can only approximate results rather than retrieve the exact relevant knowledge (Zhao et al., 2024; Barnett et al., 2024). Especially when multiple similar entries exist in the knowledge base (KB), precisely locating the accurate knowledge for CRS becomes challenging. Second, RA methods retrieve knowledge relevant only to the current dialogue turn, whereas CRS requires planning for potential knowledge needs in future turns, differing from knowledge-based QA systems (Mao et al., 2020; Jiang et al., 2023). For instance, when discussing a celebrity without a clear query (e.g., “I love Cecilia...”), the system should anticipate retrieving relevant factual knowledge (e.g., “birth date” or “star sign”) or item-based knowledge (e.g., “acting movies”) for subsequent response generation or recommendations, based on the user’s likely interests.