A Unified Multi-Stage Approach to Generalized Sequential Decision Making Problems with Covariates

University Of Delaware, Newark DE

Investigators

Abstract

Sequential decision making problems are commonly encountered optimization tasks with important modern applications. With rapid advances in data-driven technology, the diverse application examples include online service recommendation for smart phone users, intelligent implementation of intervention plans for medical service, automated financial service processing, and many others. Generally, faced with multiple decision arms, a service provider needs to choose one to be delivered for each upcoming service user, and targets to maximize the overall reward and benefits for all these service users. Furthermore, in this Big Data era, individual user covariates and metrics are often accessible to service providers, which holds great promise in personalized (mobile, medical, or business) service decision making to enhance user reward outcomes. This project will significantly advance the methods and theory for the sequential decision making problems with covariates, and address important questions that are also of interests to multiple statistics-related fields such as computer science, operations research, business analytics, health sciences, and broader machine learning communities. The promising use of personalized service will be promoted through close interdisciplinary collaborations with business and medical research communities. The graduate student supported by this grant will help with statistical theory, programming, and data analytics. Under both parametric and nonparametric frameworks, a unified multi-stage approach will be developed to optimally solve a series of generalized sequential decision making problem settings formulated as multi-armed stochastic bandit problems with covariates. In particular, the investigators aim to (1) develop a new algorithm to handle high-dimensional user covariates under assumptions much relaxed from existing work while improving performance; with integration of a class of high-dimensional regression methods and new technical tools for non-i.i.d. samples inherited from the algorithm, establish rigorous finite-time regret analysis and useful statistical properties; (2) propose a new nonparametric framework as the censored bandit problem with covariates and show optimal cumulative regret and flexible use with possibly censored reward response and non-linear decision boundary; (3) study a class of high-dimensional dimension reduction methods to mitigate curse of dimensionality issues in the nonparametric regression settings and significantly extend the use of classical nonparametric methods in high dimensional problems. Both theoretical and empirical studies to incorporate complex covariate structures inspired from business and medical research questions for decision making will create valuable training and research opportunities for graduate and undergraduate students. The graduate student supported by this grant will help with statistical theory, programming and data analytics. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.