III: Small: Advances in Federated Graph Machine Learning: A Data Perspective

University Of Virginia Main Campus, Charlottesville VA

Investigators

Abstract

With the advent of federated systems, it is now common to have devices that operate at the edge of the networks (example: mobile phones, sensors on bridges). While federated learning is an advance over moving data to one place for analysis, machine learning in federated systems creates challenges to privacy consideration at the edges. New machine learning techniques are emerging that can address these privacy concerns at the edges. Federated learning is a prominent distributed learning approach to address the privacy issue through collaborative training. It enables data owners (clients) to jointly train a machine learning model without sharing their private data, orchestrated by a central server. In the real world, data samples within a client often exhibit strong relational dependencies and naturally form a local graph. In this project, we consider learning over such distributed graph data in a federated manner, termed Federated Graph Machine Learning (FGML). Typically, Graph Neural Networks (GNNs) are widely adopted for collaborative training over graph data in FGML because they excel at modeling relational information. However, federated training on graphs faces several unique challenges, including (1) data heterogeneity—significant graph data heterogeneity across clients severely degrades the performance of GNN models in FGML; (2) label deficiency—the scarcity of labeled data for each client, when combined with complex graph structures, complicates training GNNs in the federated setting; and (3) data privacy protection—privacy leakage may arise for both attributes and structures, especially when cross-client relations exist. Our proposed framework systematically investigates these challenges and develops innovative algorithms to enhance model utility and strengthen data privacy protection in FGML. This project will develop novel and significant advances in the scientific fields of graph mining, federated learning, and distributed machine learning. Each advance will dramatically advance current machine learning research in addressing the multifaceted challenges presented in the emerging collaborative training paradigm over graph data. First, this project aims to thoroughly study data heterogeneity in FGML across three different modalities (i.e., label, structure, and feature) and develops novel solutions to address these issues in FGML. Second, this project designs novel algorithms to tackle the label deficiency issue across three scenarios (i.e., few labels, noisy labels, and no labels) to enhance the generalizability, robustness, and effectiveness of GNN models in FGML. Third, this project aims to strengthen data privacy protection in FGML concerning node attributes and graph structures and proposes two effective techniques based on attribute perturbation and adversarial training. This project has the potential to impact a wide range of industries and applications, including social networks, e-commerce, healthcare, and more. Finally, this project will play an integral part in educating and training students. The research will be tightly integrated with existing and new courses related to data mining and machine learning. The investigators will actively encourage undergraduate participation in the project, host more REU students at their institutions, and continue ongoing efforts to advise female and underrepresented students. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.