Enhancing Pipeline-Based Conversational Agents with Large Language Model

“This paper proposes a hybrid approach that leverages LLMs, in particular GPT-4, to enhance pipeline-based CAs. Using this approach, maintainers of existing CAs can adopt new domains and overcome the limitations in conversations with users while ensuring seamless integration with the existing ecosystem. This approach accelerates the CA delivery process through the assistance of LLMs in generating intents, entities, synonyms, respective training data, and agent personality traits. During deployment, LLMs can boost pipeline-based CAs’ performance by utilizing autocorrect, context-switching capabilities, answering out-of-scope questions, creating diverse and stylistically richer responses, and incorporating Closed Q&A and summarization. This paper presented experiments to showcase the scenarios mentioned above.”

Despite the various frameworks for building pipeline-based CAs, it still requires substantial time and expertise to design and develop successful CAs. Related tasks concern the design of high-quality training utterances, the definition of intents and consistent and accurate named entities, the selection of domain-specific synonyms, and the localization of training data and responses. Besides, designers must modulate, for instance, training data, dialog management rules, and pre-defined responses to represent desired assistant traits (e.g., client orientation) or personas.

A second area for improvement is the robustness of a CA at run-time, i.e., when it interacts with a user. Often, pipeline-based CAs produce repetitive responses (robust but less attractive) or experience conversational breakdown because users switched contexts (not robust). In addition, pipeline-based CAs’ narrow domain knowledge provokes out-of-scope answers due to smaller training data and limited responses. All of the situations above lower the user’s satisfaction and could encourage them to give up on the agent. We assume that LLMs’ extensive general and domain knowledge, coupled with their capability of generating attractive and diverse natural language texts, has the potential to achieve more robust and attractive CAs.

LLM, as a real-time booster, involves improving the agent’s ability to understand and respond to user input in real-time conversations. This includes contextualizing the conversation to provide more relevant and personalized responses, intent classification and response generation to handle fallback and unhappy paths when the agent is unable to understand or fulfill the user’s request, and disambiguating ambiguous requests by asking for more information. Additionally, generative models can assist with integrating closed Q&A and providing quick and accurate answers to common questions, among others.