CNS Core: Small: Ultra-Efficient Neural Network and LSTM Architectures

Princeton University, Princeton NJ

Investigators

Abstract

Neural networks (NNs) have begun to have widespread impact on various important applications, such as image recognition, speech recognition, and machine translation. The spurt of interest in machine learning and artificial intelligence in this decade can be traced back to the increase in accuracy that NNs have enabled. Yet, how to come up with the best NN architecture has remained an open problem. Hence, it is attracting a lot of attention from the academia and industry. This work will address this problem. NN synthesis has largely been limited to big-data applications and the NN models are typically expected to run in the cloud. However, there is recent interest from the industry to have edge-level (e.g., in smartphone or smartwatch) NN models. The current edge-level NNs sacrifice accuracy (by 4-5%) for energy and latency efficiency. NNs are also often not competitive with other models for medium-data and small-data applications. Finally, sequence-to-sequence modeling (e.g., for language translation) also needs to be made much more accurate, fast, and compact enough for edge devices. All these problems will be tackled in this work through new NN synthesis techniques and tools. This research has the potential to enable transformative advances in overcoming the deficiencies of current NN synthesis methodologies. Due to the explosion in machine learning applications, this research has the promise to provide a significant boost to U.S. companies and economy. Thus, it will involve significant industrial engagements. Several underrepresented (minority/female) will be involved in the research. The research outcomes will be included in two undergraduate courses on Machine Learning and Embedded Computing. Broad dissemination to the academic and industrial communities will be achieved through published papers, posters, and seminars. Additionally, various tools and models will be distributed online. The list of publications/students and tools/data with appropriate documentation will be made available at https://www.princeton.edu/~jha/. Free use of data and artifacts will be permitted for research and educational purposes. The data will be available online for at least five years following the completion of the project. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.