III: Small: Towards Scalable and Efficient Graph Representation Learning With Modern Data Lakes

Louisiana State University, Baton Rouge LA

Investigators

Abstract

This project seeks to address a critical challenge in modern artificial intelligence (AI): efficiently analyzing large-scale graph data. Graphs are data structures used to represent interconnected information, such as social networks, molecular interactions, and recommendation systems. They are essential components in a diverse array of applications across various industries, including healthcare, cybersecurity, and financial services. However, as graph data continues to grow in size and complexity, it becomes increasingly difficult to analyze using existing AI models. This project aims to develop new techniques that make working with these massive datasets easier and more efficient, particularly when they are stored in modern data lakes, which are large, scalable storage systems used by organizations to handle vast amounts of data. By improving how we process and learn from graph data, this research holds the potential to benefit not only fields like AI and data management but also disciplines that rely on graph data for gaining valuable insights, such as biology, sociology, and cybersecurity. The project will also support education by providing students with opportunities to engage in cutting-edge research and contribute to the field. This project aims to design and develop a set of scalable and efficient techniques for graph representation learning (GRL), particularly tailored for graph data stored in modern data lakes. The project will focus on three primary objectives. First, it will create a partitioning-based framework that enhances the scalability of GRL by enabling various models to process large graphs without requiring significant code modifications. Second, it will develop methods to optimize the reading and partitioning of graph data from data lakes to improve the computational efficiency of GRL. Third, it will implement predictive optimization techniques to automatically select the most suitable GRL models, computational resources, and data lake configurations based on specific workloads. These advancements will be evaluated through practical case studies and benchmarks, which will help establish a methodology for running large-scale GRL pipelines considering compute and storage layers. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.