ITR: Decision Making, Optimization, Learning, and Adaptation in Uncertain and Dynamic Environments

Massachusetts Institute Of Technology, Cambridge MA

Investigators

Abstract

PROJECT SUMMARY This project deals with decision making,optimization,learning,and adaptation in uncertain and dynamic environments.The main objectives are: (a)To develop e .cient learning methods (either simulation-based or on-line)for construct- ing near-optimal policies for dynamic decision making in uncertain environments. (b)To develop new methods and to enhance the understanding of certain existing methods for addressing large scale dynamic decision-making problems. (c)To develop computational methods and learning algorithms that pertain to risk-sensitive performance criteria,as well as fundamental limitations in the form of computational complexity results. (d)To establish the fundamental limitations of learning in multi-armed bandit problems and Markov decision processes (MDPs),in the form of lower bounds on the amount of learning that is required,and simultaneously derive optimal algorithms whose require- ments match the lower bounds. (e)To develop new formulations,models,and algorithms for multi-agent learning and adaptation in dynamic environments. The proposed research involves problems that are both practically relevant and intellec- tually deep. On the application side,the range of relevant contexts is vast.It includes logistics (e.g.,supply chain management,inventory control,.eet assignment),manufacturing systems (e.g.,sequencing and scheduling),communications (e.g.,frequency allocation and hando . management in wireless systems,routing and congestion control),.nance,robotics,and economic systems (e.g.,auctions and real-time markets). The intellectual merit arises because some of the problems present signi .cant mathe- matical challenges,that require new approaches as well as new problem formulations.This project will lead to advances in the scienti .c knowledge-base and the state of the art in the .eld of decision making.In addition,this research combines methods from operations research and control theory (dynamic programming),applied probability (stochastic approx- imation and large deviations),arti .cial intelligence (learning),and economics (game theory). As such,a broader impact of this work will be the advancement of the cross-fertilization of these disciplines.In more concrete terms,this cross-fertilization will occur not only through publications,but also through presentations at audiences from disparate communities (e.g., at machine learning as well as control theory conferences),and also through the development of new courses.Finally,at the human resource and training level,the most direct impact will occur through the mentoring of doctoral students,who will be expected to populate leading academic institutions. 1