Distributed Synchronous and Asynchronous Stochastic Optimization Algorithms over Networks

Michigan State University, East Lansing MI

Investigators

Abstract

The number of installed internet of things (IoT) devices reached 26.66 billion in 2019, and this number will reach 75 billion by 2025. These devices collect massive volumes of datasets, and the analysis of these datasets can significantly improve our daily lives. However, how to process these datasets efficiently is still challenging. First, it is impossible to transfer all the data to a location because of the large volume and privacy concerns. Second, the networks formed by these devices are complex. Thus, existing distributed methods can not be directly applied to this scenario, and novel algorithms based on the communication between these devices have to be developed to make sense of these large-scale datasets. In this project, the PI will tackle the major drawbacks of existing decentralized consensus algorithms and greatly promote the efficiency and scalability of large-scale decentralized algorithms. To achieve this goal, the PI will systematically investigate the theoretical understanding of decentralized algorithms and two major challenges, i.e., large-scale data and large-scale networks. There are three objectives. The first objective is a better convergence rate for existing, and new, decentralized deterministic algorithms. The success of this objective will be the first step that will form the foundation of the next two objectives. The second objective is to develop decentralized stochastic algorithms with variance reduction for large-scale data. The last objective is asynchronous decentralized algorithms. This project will pave the way for new research endeavors to deal with large-scale distributed datasets and largely push the research boundaries of decentralized optimization in various application domains. This research will impact the use of decentralized algorithms in fields including wireless sensor networks, machine learning, the internet of things, and healthcare. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.