CAREER: Towards Efficient and Fast Hierarchical Federated Learning in Heterogeneous Wireless Edge Networks

Auburn University, Auburn AL

Investigators

Abstract

The accelerating penetration of machine learning (ML) based artificial intelligence (AI) in a variety of domains and the explosive growth of wireless applications spur wireless federated learning (WFL), which can achieve collaborative intelligence via federated learning (FL) in wireless edge networks. This project explores wireless hierarchical federated learning (WHFL), which leverages a hierarchical communication structure to substantially reduce the communication costs of WFL. It develops fundamental understandings as well as adaptive and efficient algorithms and schemes for WHFL while addressing several unique challenges that have been predominantly unexplored before: 1) inflexibility of homogeneous computation configurations (including local iteration number, mini-batch size, step size) for participating devices; 2) inefficiency of bandwidth-sharing based communication resource allocation; 3) complex impacts of the hierarchical communication structure for heterogeneous devices. The project explores innovative cross-disciplinary research of wireless networking and machine learning, and aims to provide useful insights of networking research for future intelligent networked computational systems based on data analytics. The research outcomes of this project have the potential to enable intelligent control and management of wireless networks, and also support various emerging AI applications over wireless networked systems, such as connected and autonomous vehicles, and collaborative robots. Various substantial education programs are integrated with the proposed research, including hands-on wireless and ML/AI projects for college students, and outreach activities on robotics for K-12 students. This project studies hierarchical FL in wireless edge networks for devices with heterogeneous computation and communication capabilities. The proposed research is motivated by some key insights obtained from our preliminary work: 1) heterogeneous computation configurations, particularly heterogeneous local iteration numbers, have non-trivial impacts on the learning accuracy and learning cost of FL; 2) time-sharing based communication resource allocation is more efficient than bandwidth-sharing, while it results in non-trivial coupling between computation configuration and communication scheduling; 3) the hierarchical model communication and aggregation have non-trivial impacts when devices have heterogeneous computation configurations. With these insights, the proposed research is organized into the following three interdependent thrusts: i) adaptive cost-aware device selection and computation configuration in a local cluster; ii) co-design of computation configuration and communication scheduling for fast convergence in a local cluster; iii) global model communication and aggregation for accurate learning across local clusters. The proposed schemes and algorithms for WHFL will be implemented to evaluate their practical performance. This project is jointly funded by CNS and the Established Program to Stimulate Competitive Research (EPSCoR). This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.