EAGER: Algorithm-Hardware Co-Design for Multivariate Data Analysis

Stanford University, Stanford CA

Investigators

Abstract

The goal of this project is to develop new methods for unsupervised learning from multivariate data based on counting and comparing frequencies of data patterns. A recursive testing approach will be used to infer the multivariate distribution. The investigator will use hardware-algorithm co-design to achieve qualitative improvement over existing methods in computational time as well as in the maximum data dimension and sample size that can be handled. The economic feasibility of making this methodology widely available will also be investigated. This research is motivated by the challenge of "Big Data" analysis where the high dimensionality and extremely large sample size had made it infeasible to apply traditional statistical methods. The new methods developed in this project will be applied to several "big data" applications such as the analysis of videos, next generation sequencing data and microblogs. By developing the statistical methods for such analyses as well as customized computing resources to make these methods scalable to extremely large data sets, this research will enable more effective use of the rich information embedded in these data. Finally, the multidisciplinary approach integrating statistical, computational and hardware expertise is well suited for the training of next generation data scientists.