Dynamic Task-Oriented Dialogue: A Comparative Study of Llama-2 and Bert in Slot Value Generation

One form of these are Task-Oriented Dialogue (TOD) systems that allow the user to fulfill a task, such as booking a hotel or making a reservation at a restaurant, by the usage of external services. To achieve this, the TOD system has to i) understand the user (dialogue Understanding), ii) plan its next action, for example to provide information or request more information from the user (Policy Planning) and iii) generate a response that fits the dialogue, policy, and eventual responses from the external service (dialogue Generation) [6].

The system configuration text explains to the model the general set-up and contains various information. This element turned out to be heavily responsible for generating good results. Therefore, we tried Prompt Engineering, i.e., tested multiple variations of the system configuration text to better communicate the task to the model [5].

The final system text we used contained information about:

– The general task: generate fictional dialogues with labels, – basic rules for the dialogues – all possible domains, their slots, and a template on how to name them, – a list of possible dialogue acts, – a template on how to mark annotations, – how to proceed when no value is given for a slot, – and finally an annotated example dialogue.

We retrieved all responses with the default model parameters and saved them for further preprocessing, since the final format of the data to train the TOD system will be a specific JSON format in which all information about the text and annotation is structured. We have tried to generate this directly, but were unable to achieve usable responses.