CAREER: Federated Learning: Statistical Optimality and Provable Security

Duke University, Durham NC

Investigators

Abstract

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2). Rapid developments in machine learning and data science have compelled organizations and individuals to rely more and more on data to solve inference and decision problems. To ease the privacy concerns of data owners, researchers and practitioners have been advocating a new learning paradigm – federated learning. Under this framework, the central learner trains a model by communicating with distributed users and keeping the training data stored locally at the users. While opening up a world of new opportunities for training machine-learning models without compromising data privacy, federated learning faces significant challenges in maintaining statistical efficiency and security due to the heterogeneity and unreliability of the distributed users. Successful completion of the project provides key enabling technologies for efficient and secure federated learning and accelerates its adoption in security- and safety-critical systems such as self-driving cars and personalized medicine. The proposed education activities include teaching and mentoring graduate and undergraduate students and community outreach aiming at raising public privacy and security awareness. This research develops an interdisciplinary program to investigate the fundamental and algorithmic aspects of federated learning ranging from statistical efficiency to security and privacy. The statistical efficiency of the widely-adopted algorithms is analyzed beyond their failures of reaching stationary points. New algorithms are developed based on meta-learning and clustering, and shown to be statistically optimal even in the presence of model and data heterogeneity. Moreover, this research conducts a comprehensive study of decentralized learning under Byzantine attacks. By borrowing insights from robust statistics, byzantine-resilient gradient descent algorithms with exponential convergence to the optimal error rates are devised. Finally, to protect the learner's privacy against eavesdropping attacks, the investigator aims to design optimal private learning strategies by innovating ideas from information theory and duality theory between the learner and adversary. Complementing the theoretical investigation, the new learning algorithms are made available as computational packages for the federated learning systems and real-data applications. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.