CIF: Small: Inverse Reinforcement Learning for Cognitive Sensing

Cornell University, Ithaca NY

Investigators

Abstract

Cognitive sensing systems such as adaptive radars choose their decisions by maximizing a utility function subject to sensing constraints. This project studies the question: By observing the decisions of a cognitive sensing system, how can an adversary estimate the sensor's utility function and therefore predict its future decisions? Such inverse reinforcement learning problems arise in numerous defense and civilian sensing systems. Using ideas from microeconomics, machine learning and optimization, this project investigates how to design algorithms to achieve inverse reinforcement learning. The project also investigates how to design a covert sensing system, such as a meta-cognitive radar that hides its utility and, therefore, its plan from an adversary. The research will lead to new methods for inverse reinforcement learning with performance guarantees for sensing and the design of covert sensing systems that hide their cognition. The project will support the cross-disciplinary development of a diverse cohort of PhD and undergraduate students at Cornell University and also contribute to the STEM education of high school students from rural New York state. This project draws from micro-economics, machine learning and stochastic optimization to study adversarial signal processing problems in cognitive sensing. The technical aims of this project fall under three interrelated themes. The first theme investigates revealed preference methods to detect the presence of cognitive sensors. The research studies how to detect statistically the presence of a cognitive sensing system and how to interrogate a sensor to detect if it is cognitive. The second theme investigates inverse Bayesian sequential detection: given the decisions of an optimal sequential detector, how to estimate its parameters such as misclassification costs and false alarm penalty? The third theme investigates inverse stochastic gradient algorithms: given real-time noisy estimates from a stochastic gradient algorithm, how to estimate the expected utility function that it is optimizing? The research in this theme studies adaptive inverse reinforcement learning while the cognitive sensor is learning to optimize its strategy. This project transcends classical statistical signal processing (estimation/detection) to address the deeper issue of how to infer strategy from sensing. The outcome of this research will be novel algorithms for strategy identification with provable performance guarantees that are broadly applicable to detect complex sensing systems. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.