CSR: Small: Data Parallel Frameworks for Large-scale Machine Learning through Sync-on-the-Fly

University Of Massachusetts Amherst, Amherst MA

Investigators

Abstract

The advances in sensing, storage, and networking technologies have led to the collections of high-volume, high-dimensional data. Making sense of these data is critical for companies and organizations to make better business decisions, to bring convenience to our daily life, and even enable better health through biological information and drug discovery. Recent advances in machine learning have led to a flurry of data analytic techniques that typically require an iterative refinement process. However, the massive amount of data involved and potentially numerous iterations required make performing data analytics in a timely manner challenging. This project aims to design and implement a data parallel programming framework called Sync-on-the-fly. The framework enables machine learning computations within cloud computing environments to establish synchronization barriers during the execution of the computations. Since synchronization barriers are established for building consistent model parameters, they do not need to be predefined. The barriers can be established during the computation based on the progress of the computation. This data parallel programming model preserves the semantics of machine learning algorithms. The goals are to build theoretical foundations and create efficient distributed frameworks for a series of well-known machine learning algorithms, and establish the programming models for these computations. The technologies developed from this project will have immediate important applications on road traffic prediction, biological information discovery, online marketing, and computer forensic analysis. This project will bring fast, accurate, and cost-effective processing of massive data to users. This project will also train new graduate engineers in distributed framework design, machine learning algorithms, and big data analytics. All of these skillsets are in broad demand in US industry. The data and software codes produced for the distributed framework and distributed machine learning algorithms will be made publicly available at the research website http://rio.ecs.umass.edu/html/research/index.html. The website will be retained for at least three years after the conclusion of the project. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.