CAREER: Decentralized Federated Compositional Learning: Algorithm and Theory

Temple University, Philadelphia PA

Investigators

Abstract

Federated Learning is an emerging and promising collaborative learning paradigm that allows data to stay secure to enhance privacy and help address regulations for handling and storing private data. As computing moves to edge devices such as cell phones, federated learning becomes more important. Unlike traditional artificial intelligence (AI) applications that train on data gathered and processed at a central location, federated AI applications learn collaboratively at the edge. With federated learning, multiple users remotely provide their data to train an AI learning algorithm in a collaborative fashion. The process runs in an iterative fashion with local updates at the edge using private data, while benefiting from global collaboration from several remote/edge users with their own private data. It enables training machine learning models on large-scale distributed data without sharing the raw data. Conventional federated learning focus on the standard minimization problem poses challenges to new learning paradigms such as the compositional learning paradigm. This learning paradigm has been attracting a surge of attention recently because it covers many new machine learning models, such as model-agnostic meta-learning, imbalanced data classification, and contrastive self-supervised learning. Traditional federated learning concentrates on centralized setting and suffers from single-point failure and communication bottleneck. This project focuses on enabling decentralized federated learning for the compositional learning paradigm. It pushes the boundary of federated learning, advancing the development of both federated learning itself and the emerging learning paradigms. The developed federated learning algorithms can be applied to a wide range of application domains, such as healthcare and social science. This research provides education and research opportunities for undergraduate students and K-12 students. The objective of this project is to develop a new decentralized federated compositional learning (DFCL) framework to enable federated learning for the compositional learning paradigm. To address the fundamental challenges caused by the compositional structure and decentralized communication, this project will systematically investigate decentralized federated compositional optimization problems from the perspective of algorithmic design and theoretical foundation. Thrust 1 focuses on the computational foundations for the multi-level compositional minimization problem of DFCL, including developing converging-fast and generalizing-well algorithms. Thrust 2 concentrates on the communication foundations for DFCL, aiming to develop robust communication topologies and efficient communication strategies. Thrust 3 investigates the computational foundations for the compositional minimax problems of DFCL, which aims to propose efficient algorithms for multi-level stochastic compositional minimax problems. Finally, DFCL will be evaluated on biomedical applications and generic machine learning applications to address the challenges in real-world learning tasks. The developed research will provide abundant topics and opportunities in education and research for both undergraduate and graduate students. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.