Honors Thesis 2020 - Ruixiang Qi

Analysis of a State Machine-based Interactive Dialogue Management System

Ruixiang Qi

Highest Honor in Computer Science

Abstract

The sports topic handler has been one of the key components in `Emora', an open-domain chatbot system that competes for the Alexa Prize 2020, which is a university challenge for creating the best socialbots. This thesis first gives a comprehensive description of Emora’s sports topic handler, including its architecture and innovations. These innovations involve effective approaches to opinion-based state transition dialogue management that uses a daily updated database as well as derivation of engaging conversations on flashing events in sports. Given this topic handler, Emora is capable of making multi-turn dialogues on any game, player, and team upon request by inferring statistical facts from the database and sharing its own opinions about the latest topics. These unique features help the sports topic handler to become the highest rated components in Emora in February and one of the highest rated components of all time. This thesis also presents the result of a user study on the sports topic handler which evaluates the impact of various modular improvements on the overall ratings provided by random users interacting with the chatbot. The impact of each update is evaluated extrinsically through the overall ratings. Our analysis finds a strong positive correlation between these updates and the user ratings, while also finds a negative correlation associated with uncovered topics and ignorance to the user input.

Department / School

Computer Science / Emory University

Degree / Year

BS / Spring 2020

Committee

Jinho D. Choi, Computer Science and QTM, Emory University (Chair)
Davide Fossati, Computer Science, Emory University
James Nagy, Mathematics, Emory University

Links

Anthology | Paper | Presentation