Optimization in the Small-Data Regime

University Of Southern California, Los Angeles CA

Investigators

Abstract

Modern decision making under uncertainty frequently involves the need to make many simultaneous decisions at a highly granular level, often in a time varying environment. As a result, the amount of relevant data per decision is often quite small. Conventional techniques in data-driven optimization have provably poor performance under these conditions. This project aims to develop a new class of data-driven methods specifically tailored for the "small-data" regime, offering a new perspective on data-driven methods. The prevalence of the small-data regime in applications ranging from epidemiology to inventory management to new product launches underscores the potential of a successful research program to have cross-disciplinary impact. On the educational side, the project will create web-based educational tools that highlight the unique challenges of the small-data regime, and foster project collaborations between graduate students and local government. The project's approach will blend large-scale linear programming, robust optimization and empirical Bayes estimation. These key ideas exploit the large-scale structure of these optimization problems to attempt to overcome the challenges of insufficient data. The research will focus on: 1) formulating a general framework for the "small-data" decision regime, 2) developing methods that are provably best possible as the size of the optimization problem grows large for a fixed amount of data, and 3) illustrating the techniques through case-studies of high-impact applications. The award will support the PI's ongoing collaborations with decision makers in both the public and private sectors who will make use of these decision tools.