Alternating Recurrent Dialog Model with Large-scale Pre-trained Language Models

Previous sequence-to-sequence models are used to tackle documents with only one narrator. However, in dialogs, two speakers have different roles; therefore, their language model distributions are very different from each other. To address this issue, we propose ARDM, a dialog model that encodes and decodes different speaker utterances in alternating order. This structure makes the model more flexible and efficient than traditional sequence-to-sequence models in processing various dialogs.