CAREER: Locally Adaptive Nonparametric Estimation for the Modern Age - New Insights, Extensions, and Inference Tools

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

Nonparametric modeling---which means, roughly, flexible modeling of smooth trends without specific assumptions about their form or shape---finds diverse applications in many areas such as epidemiology, astrophysics, finance, and artificial intelligence. It is also a field ripe for modern statistical development, since nonparametric models are in a sense even more appealing in the "big data" era, as it is precisely in data-rich settings that the increased flexibility of these models will begin to show real rewards in terms of statistical accuracy. The proposed work will develop nonparametric methods (and affiliated software) that will be useful to data scientists who model smooth, nonlinear trends in areas like those mentioned above, as well as many others. A specific scientific emphasis will be the forecasting of influenza and dengue fever. Such forecasts will help policy makers design and implement more effective countermeasures towards these diseases. The proposal puts forward two main ideas for educational training, closely related to the research aims to be pursued. The first is a set of short videos on nonparametric smoothing, intended as supplements to an undergraduate level course called Advanced Methods for Data Analysis. They will be integrated with an interactive quiz system, and will be made freely available (on YouTube) so that others outside the class may watch too. The second idea is a statistical computation training group, for PhD students from Statistics and Computer Science. "Locally adaptive" nonparametric methods offer more fine-grained flexibility than traditional nonparametric methods, in that they can simultaneously represent different amounts of smoothness at different parts of the function domain. Currently, locally adaptive nonparametric methods are not often used in big, modern data sets, likely because of their computational inefficiency, and the general inavailability of locally adaptive methods in many modern problem settings. The proposed work seeks to change this, and to push the state of the art in modern locally adaptive nonparametric estimation. The research aims are to: deepen the theoretical understanding of existing locally adaptive methods for univariate problems; efficiently scale these methods and extend these theories to problems where data are collected in high dimensions and over graphs; and develop inferential tools for all of these locally adaptive procedures. The specific contributions will be balanced between the theoretical (statistical theories that describe the underpinnings of the methods in question) and computational (practical algorithms that describe implementation of these methods at scale) perspectives. A final more applied research aim is to use the proposed methods to improve and extend a forecasting system for major epidemics such as influenza and dengue fever.