RI: Small: Local and Forward-Oriented Deep Learning for Decentralized and Dynamic Environments

Purdue University, West Lafayette IN

Investigators

Abstract

The field of deep learning enables computers to create complex models to perform challenging tasks previously limited to humans. It has been the key to modern successes in artificial intelligence (AI) including automatic object identification in images, machine translation, and autonomous cars. However, current deep learning models require data to be collected in one centralized location and globally optimized by powerful computers. This significantly limits deep learning in the decentralized and dynamic environments common to future AI applications critical for national defense and economic pre-eminence. These future AI applications will be deployed over multiple heterogeneous wireless devices that have limited computational and communication capabilities (e.g., a surveillance camera, a weather sensor, or an aerial drone). This work will develop novel deep learning methods that execute in these decentralized environments, adapt to changing conditions, and recover from communication failures. To broaden STEM participation, this project will also develop virtual machine learning labs to engage high school students. Ultimately, this work will take a key step towards the next generation of decentralized and dynamic AI systems. This project will develop approaches to optimize a sequence of invertible functions that iteratively deconstruct data patterns, an approach called destructive learning. Specifically, this work will advance knowledge on both the foundational and practical aspects of local and forward-oriented destructive learning. The first objective will generalize the iterative algorithms from the investigator’s prior work and provide the foundation for forward-only algorithms. These forward-only algorithms do not require a centralized computational environment and can locally optimize AI components even under imperfect communication between devices. The second objective will develop more practical destructive learning algorithms that combine the strengths of forward-only and global centralized learning. The final objective will implement and evaluate forward-oriented algorithms on simulated and real device networks to demonstrate the feasibility of this new deep learning approach. These objectives work together in creating a foundation for a novel alternative to end-to-end learning across devices that does not require global synchronization. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.