Reinforcement learning in the human brain: Dimensions, features, and contexts

University Of Delaware, Newark DE

Investigators

Abstract

Human decision-making depends not only upon logical reasoning about the world, but also trial-and-error learning to associate features and actions with rewarding and punishing outcomes. These reinforcement-learning mechanisms have well-known neural correlates, and theories of such learning provide excellent accounts for human behavior in limited contexts. However, trial-and-error learning has typically been studied in laboratory tasks in which value-associated features are known and are the only stimulus aspects presented to subjects. Thus, theories of how this type of learning is accomplished in the real world face a key problem: there are a multitude of potentially relevant aspects of experience that co-occur with any given decision or outcome, so which associations should be learned and guide future behavior? This project explores how humans cope with a multidimensional world when making experience-guided decisions. A better understanding of human decision-making behavior will facilitate more complete theories of learning, enable us to better predict and enhance real-world decision-making, and improve understanding of how such decision-making might break down due to mental disorders. This project will provide research opportunities for undergraduate, graduate, and postdoctoral students, and include broad public outreach in the form of online demonstrations and explanations of models of learning and decision-making. Further, the proposed activity includes a yearly workshop on MRI methods, which will provide training opportunities in new methods and outreach to students and faculty across fields. The proposed project will use computational modeling of behavior and model-based fMRI to assess how humans learn about relevant and irrelevant features of the world during reward-guided decision-making. The first study will address whether irrelevant stimulus dimensions are tracked with respect to value, both in terms of choice behavior and in terms of neural representations of value. Specifically, the project will ask whether reward prediction error signals in ventral striatum and elsewhere in the brain are explained solely by relevant feature-value associations, or whether irrelevant feature-value associations are also tracked neurally and influence behavior. The second study will examine whether learned statistical contingencies in one domain, visual perception, arbitrarily influence value learning both in terms of behavior and brain activity. Findings from this study will illuminate how distinct associative learning mechanisms interact to guide behavior. Finally, a third study will examine the role of context in reward-guided decision-making, examining how well contextual features can be incorporated into decision-making. The results of this work will guide development of human reinforcement learning theories towards accommodating the complexity of real-world decision-making environments. This project will illuminate the degree to which control over which particular associations guide behavior is exerted during learning or at the time of choice.