RI: Small: Applying discrete reasoning steps in solving natural language processing tasks

University Of Texas At Austin, Austin TX

Investigators

Abstract

Modern natural language processing systems are effective at shallow analysis of unstructured text data, performing tasks such as discovering events, identifying the actors of those events, and grouping events with the same actors. Neural networks help make these systems robust to effects like paraphrasing, but still capture mostly superficial text patterns. To answer deeper questions about things like causal relationships between the events in a text, a system might need to combine several pieces of information, abstract away irrelevant details, and incorporate prior world knowledge to arrive at an answer. This project aims to develop systems that can address these challenges: these systems explicitly model reasoning over text and draw on the power of neural networks to do this reasoning in a nuanced way. Such reasoning is explicitly taught to the systems via "handholding" supervision, which encourages the systems to mimic how humans solve a problem and helps them generalize better to new problem instances. This alignment with what humans do also serves to expose the systems' decision-making processes; it provides a form of explanation of their behavior so that one may evaluate them against desired criteria such as equitability. This proposal's technical innovation is focused on two fronts: designing latent variable models and exploiting new types of handholding supervision during model training. These techniques are explored in the context of three challenging problems requiring complex reasoning: (1) solving mathematical word problems; (2) resolving coreference using world knowledge; (3) answering questions from documents. For each problem, new models are proposed centering around discrete derivations of answers, which draw on state-of-the-art tools like attention-based recurrent neural networks to capture the larger context of the reasoning process. The discreteness of the models' decisions provides an anchor to incorporate auxiliary supervision, which is hard to do in fully end-to-end neural models. The nature of the handholding supervision depends on the task and is a combination of incidental supervision, heuristically identified derivations, and targeted human annotation. Each of the addressed problems tests different aspects of the approach, such as handling complex derivations and incorporating world knowledge, and these problems yield concrete evaluation frameworks to understand the efficacy of the proposed techniques. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.