CAREER: From Federated to Fog Learning: Expanding the Frontier of Model Training in Heterogeneous Networks

Purdue University, West Lafayette IN

Investigators

Abstract

Billions of Internet-connected devices are gathering data to enable machine learning (ML) capabilities for everyday use. Recent networking research has focused on the potential of distributing ML tasks across these increasingly powerful devices. For example, consider smartphones aiming to learn an image recognition capability: rather than centralizing the images in the cloud, the smartphones may each learn local versions of the ML model on their individual images, and the cloud can periodically synchronize these models. However, devices at the network edge often exhibit considerable heterogeneity in their communication and computation capabilities, as well as in the statistical properties across their local datasets, which can lead to significant variations in decision-making quality. The goal of this project is to establish fog learning, a new paradigm that will enable efficient model learning at scale by integrating ML with the orchestration of “fog” networking resources from the edge to cloud. The findings of this project will be incorporated into an education plan emphasizing the role of ML in shaping future networks, including new undergraduate, graduate, and open online courses augmented with innovative educational technologies promoting student engagement and pathways to research. The investigator also pursues collaborations with industry and interdisciplinary researchers. The proposed research has two main objectives: (1) establishing an understanding of how different heterogeneous network configurations affect ML performance, and (2) developing methodologies that orchestrate fog networking resources for jointly optimizing model learning and resource efficiency. Investigations are divided into three thrusts. Thrust 1 focuses on optimizing distributed learning across edge networks, by integrating federated model training with intelligent device sampling and data offloading. This will characterize the impact of partial device participation on learning convergence and develop the notion of local dataset diversification. Thrust 2 considers the design of model aggregation stages throughout the fog hierarchy to facilitate device cooperation at different timescales. This will codify the tradeoffs between different architectures for local synchronization, including those in Thrust 1, and lead to adaptive orchestration methodologies. Thrust 3 will investigate performance enhancements to Thrusts 1 and 2 from control over the wireless substrate. This includes cross-layer techniques that unify signal design with model training, and learning architectures for edge subnetwork partitioning. Each thrust will develop new theories and algorithms for optimizing ML over networks and consider innovative ML techniques for solving and enhancing these optimizations. Evaluations are based on large-scale wireless emulators and testbeds with commercial-grade network equipment. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.