CAREER: Hashing and Sketching Algorithms for Resource-Frugal Machine Learning

William Marsh Rice University, Houston TX

Investigators

Abstract

Modern applications are constantly dealing with datasets at terabyte scale, and the anticipation is that very soon it will reach petabyte levels. The size and dimensionality of current datasets have made machine learning (ML) models significantly large and complex, which adds to the existing problems. Classical approaches to learning and inference fail to address new concerns of computational resources, storage limitations, network communication constraints, energy efficiency, real-time latency, etc. This project focuses on basic design and implementation of (exponentially) resource-frugal and scalable machine learning algorithms which are ideally suited for current big-data constraints. This project leverages probabilistic hashing techniques for advancing the state-of-the-art machine learning algorithms. The focus is on redesigning existing machine learning pipelines to make them amenable to the hashing speedup. Apart from being exponentially cheap, the designed algorithms are also massively parallelizable. The three primary objectives are: 1) Computationally Efficient Deep-Learning and Kernel-Based Learning via Hashing, 2) Sketching Algorithms for (Exponentially) Compressing Machine Learning Models, and 3) Improving Efficiency of Hash Functions. This project capitalizes on several recent ideas, including asymmetric hashing, hash-based kernels, densified hashing schemes, sub-linear adaptive sampling, and adaptive sketching, to push learning algorithms to the extreme-scale. By creating a unique bridge between probabilistic hashing and machine learning, this project further enhances the current understanding of tradeoffs involving computations, space, and accuracy.