SHF: Small: Practical Analyses and Safe Transformations for Imperative Deep Learning Programs

Cuny Hunter College, New York NY

Investigators

Abstract

Learning often occurs by pattern recognition. Software systems learn by using algorithms to recognize patterns and draw inferences from existing data and apply the inferences to previously unseen data. Deep Learning (DL) is a kind of machine-learning algorithm inspired by neural networks of human brains. A DL model learns decision logic from a large set of examples. A classic application is image processing, where a model may learn to recognize particular images through training with many sample images. While software systems that incorporate DL models involve large amounts of data, they still have to be efficient and responsive. This project is expected to increase DL system robustness, reliability, and scalability, positively impacting computer vision, autonomous driving, medicine, and extremism identification. Tools developed as a result of this project are also expected to democratize the Artificial Intelligence workforce, as they will assist data scientists and software engineers of varying proficiencies in writing quality DL code. Such tools can potentially contribute to a diverse, globally competitive STEM workforce and increase US economic competitiveness. This project will also promote software engineering concepts in machine learning by augmenting and creating several undergraduate and graduate courses. Dissemination will occur through publicly distributing datasets, papers, open-source software, and Open Educational Resources. DL frameworks increasingly make various tradeoffs to balance the often competing requirements of reliability, usability, and generality. Popular DL frameworks have historically embraced graph-based, deferred execution-style (low-level) Application Programming Interfaces (APIs). While efficient, (legacy) systems using such interfaces are cumbersome, error-prone, and difficult to debug, maintain, and port. Contrarily, (modern) eager execution-style DL APIs facilitate higher-level, imperative, and Object-Oriented (Python) programs that are easier to debug, less error-prone, and more extensible have consequently emerged at the expense of run-time performance. Though hybrid approaches aim to bridge the two paradigms, they necessitate a non-trivial amount of technical metadata and exhibit several limitations and known issues on the use of native program constructs. This project is expected to contribute practical analyses and safe transformations for modern imperative and Object-Oriented DL programs that markedly improve their reliability and scalability. First, various software engineering artifacts will be mined for bug fixes, (manual) refactorings (semantics-preserving source-to-source program transformations), and missed opportunities in efficiently executing imperative DL code. Then, novel analyses and refactorings for automatically (i) migrating legacy, deferred execution-style DL code to more robust imperative DL code and (ii) specifying how their otherwise eagerly-executed imperative DL code should be reliably and efficiently executed as graphs at run-time will be formulated. Finally, novel analyses for detecting performance bottlenecks and semantic errors associated with graph-based execution of imperative, otherwise eagerly-executed DL code will be designed. This contribution is significant because it fills the void of techniques, methodologies, and tools for effectively developing---and evolving long-lived---trustworthy and efficient DL systems that pervasively use imperative and Object-Oriented DL programming. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.