Canonical Linear Methods and Hierarchical Non-Linear Methods in High-Dimensional Statistics

University Of California-Berkeley, Berkeley CA

Investigators

Abstract

Statistics is at the heart of extracting meaningful information from big data. Its primary tasks include estimation and uncertainty assessment. The latter is crucial in big data analysis for sound decision making. For the former, the methods employed in deep learning machines, such as those behind Google's Brain and AlphaGo and Microsoft's Cortana, beg understanding. This research project is intended to bridge practice and theory of statistics in these areas. It aims to provide accessible uncertainty measures for linear modeling of big data and to derive insights into how deep learning works, based on mathematical analysis. This research project develops and analyzes linear and non-linear high-dimensional statistical inferential methods that are easily accessible by practitioners in data science. In the linear case, it develops and analyzes inferential methods based on well-established bootstrap, lasso, partial ridge, and random projection methods. In the non-linear case, it takes the first steps to explain in a principled manner the impressive success of deep learning in practical problems such as image classification and speech recognition. In particular, statistical properties of these methods will be studied under linear and Neyman-Rubin high dimensional models, and via analytical and simulation means. A generative model of a two-layer neural network (or hierarchical non-linear model) will be explored to understand and compare deep learning with other methods, analytically and through simulation studies. Improvements over deep learning as a general supervised learning method are sought by enforcing biologically meaningful constraints from brain connectivity research.