CRII: CIF: Sequential Decision-Making Algorithms for Efficient Subset Selection in Multi-Armed Bandits and Optimization of Black-Box Functions

Missouri University Of Science And Technology, Rolla MO

Investigators

Abstract

Unlike fixed decision-making algorithms, sequential decision-making (SDM) algorithms are often more efficient in the number of samples needed to solve a problem. Obtaining additional samples can be expensive and time-consuming. Consequently, SDM algorithms have been used in a wide range of applications ranging from recommender systems and automated drug discovery to the tuning of parameters in engineering design models. This project will develop new algorithms that enhance the applicability of SDM and improve its efficiency. Performance guarantees for the algorithms will be obtained and they will be tested in real-world data experiments. The project outcomes can lead to increased use of SDM in industry sectors such as e-commerce and drug design. The project will provide research opportunities for a diverse group of students and the knowledge gained will be integrated into the university’s graduate curriculum and will broaden the K-12 outreach activities undertaken by the investigator. The project is organized under two problem frameworks: subset selection and black-box optimization. In the first problem of subset selection, the objective is to select a good subset among many alternatives. This objective is formulated as a subset selection problem in multi-armed bandit models. The project will develop SDM algorithms that obtain a good subset using the gaps between consecutive arm means. This removes the need to specify a parameter that defines the good subset. The SDM algorithms will exploit arm features in linear multi-armed bandit models and the theoretical analysis will obtain sample complexity bounds for the proposed algorithms. In the second problem of black-box optimization, such arm features are not available, and partition-based optimization methods are often used. This project will develop SDM algorithms that adapt the partitioning scheme to the black-box function being optimized. Convergence to the optimal value will be ensured and their practical performance on benchmark simulations will be measured. The outcomes of the project will be communicated in papers and the developed code will be made publicly available. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.