RI: Small: Fast and Accurate Natural Language Parsing and Generation by Marrying Deep Learning with Dynamic Programming

Oregon State University, Corvallis OR

Investigators

Abstract

This grant aims to improve automatic understanding, generation, and translation of natural language by machines. Automated natural language processing has already changed the way we interact with digital assistants such as Amazon Echo and Apple Siri on smartphones and other devices. Automated machine translation reduces information barriers on the Web, where most information is inaccessible to most users because it is in a language they do not understand. Today's natural language systems, however, are limited to short exchanges and often make errors. These limitations are due to need for the systems to respond very quickly: current methods for more accurate understanding and generation take too long. This project will overcome this problem by developing new, fast, principled algorithms for these tasks. This project also supports STEM education of underrepresented minorities (who do not speak English natively) by recruiting them in machine translation studies. This grant aims to construct fast (linear-time) and accurate natural language parsers and generators (including translators) that utilize the power of both deep learning (for accurate and automatic feature engineering) and dynamic programming (to speed up the search). This project focuses on Recurrent Neural Network-based models (RNNs) such as Long Short Term Memory (LSTMs). In particular, this project aims to (1) Develop linear-time dynamic programming-based neural parsers by using RNNs to summarize the input text and extend them to joint syntactic-discourse parsing and predictive parsing. (2) Develop approximate dynamic programming algorithms and principled beam search methods for text generation and machine translation systems that use RNN-based decoders to model output text. (3) Combine the above two directions with an innovative application of simultaneous translation using predictive parsing. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.