RI: Small: Developing Large Scale Distributed Syntactic, Semantic and Lexical Language Models for Machine Translation and Speech Recognition

Wright State University, Dayton OH

Investigators

Abstract

This project aims to build large scale distributed syntactic, semantic, and lexical language models that are trained by corpora with up to Web-scale data on a supercomputer in order to substantially improve the performance of machine translation and speech recognition systems. It is conducted under the directed Markov random field paradigm to integrate both topics and syntax to form complex distributions for natural language, and uses hierarchical Pitman-Yor processes to model long-tail properties of natural language. By exploiting this particular structure, the complex statistical estimation and inference algorithms are decomposed and performed in a distributed environment. The language models are put into one-pass decoders of machine translation systems, and the lattice rescoring decoder into a speech recognition system. In addition, a principled solution to a long-standing open problem, smoothing fractional counts due to latent variables in Kneser-Ney's sense, might be found. This project fits into the NSF's strategic long term vision of a Cyber-infrastructure Framework for 21st Century Science and Engineering (CIF21). The project integrates various kinds of known language models and provides a way to overcome the limitations of existing combination methods for language models and to deploy algorithmically interesting methodologies that are scalable to data sets available on the Web. The project provides an environment for interdisciplinary education in information technology that bridges areas of language and speech processing, machine learning, and data-intensive science and engineering to benefit students at several levels.