RI: Small: Using Automatically Generated Paraphrases and Discriminative ASR Training to Author Robust Question-Answering Dialogue Systems

Ohio State University, The, Columbus OH

Investigators

Abstract

Question-answering (QA) dialogue systems are useful in a broad range of contexts where the primary role of a virtual character is to answer questions posed by the human user. In QA-based dialogue systems, the primary interpretation task can be framed as matching a user's question against a set of questions anticipated by the content author. This project investigates for the first time how methods for automatically paraphrasing anticipated questions can aid authors in establishing a large set of expected question variants, making it possible to dramatically enhance interpretation robustness for both chatted and spoken language. By evaluating the project with an existing virtual patient dialogue system and new virtual guide for Columbus, Ohio's COSI (Center of Science and Industry) science museum, the project will enhance medical education and provide an inspirational example of science in action to the children who attend the museum. It also promises to enhance the effectiveness of short answer scoring in educational software and commercial QA systems for frequently asked questions. The proposed approach is the first to explore the potential of advanced automatic paraphrasing techniques to enhance the robustness of interpretation in an easy-to-author QA dialogue system. By employing paraphrasing at content authoring time, it becomes possible to take advantage of and make explicit the author's knowledge of the space of functionally equivalent questions, potentially leading to dramatic improvements in interpretation accuracy; furthermore, doing so makes it possible to set up an effective, task-relevant discrimination space for Automatic Speech Recognition (ASR) training. To generate paraphrases, the project uses the grammar-based surface realizer OpenCCG for lexico-syntactic alternations together with broad coverage resources, vector space models of word meaning and multiword alignments. To train discriminative ASR models that make a difference in question interpretation, generated paraphrases are incorporated into the semantic error rate estimation. Using data collected from medical students and museum visitors, the project assesses the approach via its impact on interpretation accuracy and qualitative measures of usability.