Neural Mechanisms Of Motivation And Reward

National Institute Of Mental Health

Investigators

Linked publications & trials

Abstract

The intensity or vigor of goal-directed behavior is a correlate of the motivation underlying it, and, therefore, motivation can be inferred by monitoring the performance of goal-directed behavior. To study motivated behavior in monkeys, we use a task in which monkeys must perform some work, in this case detecting when a target spot turns from red-to-green, to obtain a drop of juice. In one set of tasks, a visual stimulus, a cue, indicates how much discomfort must be endured, e.g., the number of trials to be worked, to obtain the reward. The monkeys learn about the cues quickly, often after just a few trials. [unreadable] [unreadable] The number of errors in detecting the red changing to green becomes proportional to amount of work remaining before reward, with the monkeys working faster and with fewer errors when a visual cue indicates that the reward will be delivered immediately after the next correct response than when the cue indicates that additional red-to-green detections will be needed. This is a behavior in which the monkeys decrease the accuracy of their performance in response to an increased predicted workload. This achieves our goal of manipulating motivation.[unreadable] [unreadable] Using a reward postponement version of this task we examined the effect of relationship among two external factors, reward size and delay, and one internal factor, thirst, which we modeled as accumulated reward/total reward, on motivation using the reward postponement task.[unreadable] Reward size and delay were independently manipulated and informed by the incentive cue. A simple mathematical relation describes the relations among reward size, delay and satiation level on motivation, errors = (1+kD)/ ((aR) F(S)). k is a constant, D is the delay, a is another constant, R is reward size (in drops), and F(S) is a sigmoidal funtion of satiation, which is 1-thirst. These results are consistent with temporal discounting of reward with time. In this form there is no interactive effect between satiation level and incentive value on motivation. Thirst simply enhances the incentive value, and reward is discounted as a hyperbolic function of delay duration as shown before in choice tasks. This model gives us an extended view of how incentive and motivational values are calculated and represented in the brain.[unreadable] [unreadable] [unreadable] Motivation depends on the current perceived value of action outcome. The intrinsic value of the outcome is modified by many factors, including delay to reward, accumulated reward, and task difficulty, all of which lead to changes in motivational intensity. We investigated the role of motivation on the behavior of 24 monkeys tested in visually-cued reinforcement schedules. Because the monkeys make the largest number of errors near the beginning of the schedules, we infer that their motivation to perform increases when the reward is more proximal. Error rates are typically smaller in trials equally distant from reward, but belonging to longer schedules, despite the equal number of trials remaining before the reward. Since only the proximity to reward matters in the reward schedule, the monkeys are differentially motivated in trials that are equivalent in terms of reward. This is like the framing effect observed in humans, wherein equivalent options are treated as if they were different based on the context in which they are presented.[unreadable] [unreadable] Reinforcement Learning (RL) theory provides a set of methods for learning to predict delayed reward contingencies and to exploit these predictions to generate effective behavioral policies. These policies should comply with rational principles such as optimality and invariance. According to the former, subjects should minimize the cost (number of trials) needed to obtain a reward; according to the latter, subjects should respond with equal motivation in trials equally far from the reward. The monkeys violate both of these principles. We modeled the behavior by noting that performance must reflect the learned motivational values (which are larger in trials more proximal to reward), and the influence of the current trial to the next trial to be performed. Both the proximity to the reward and the number of trials already completed emerge spontaneously emerge as properties of the model.[unreadable] [unreadable] We recorded from single neurons in cingulate cortex, an area widely accepted as playing an important role in organizing motivated behavior, of a monkey performing the reward schedule task. We found strong effects on the cue response related to predicted delay duration and only a small effect of predicted reward size. These results suggest that the anterior cingulate cortex plays a role in anticipating and monitoring the cost expended to obtain a reward, such as the burden of waiting.[unreadable] [unreadable] Because cingulate cortex appears to be involved in aspects of reward representation and decision-making processes, we studied how a bilateral ablation of the cingulate cortex influences reward evaluation, performance monitoring, and cost-benefit balance in monkeys. In the first experiment, the performance of the monkeys was measured before and after cingulate ablation in a task variant with 2 levels of reward. There were visual cues for each reward amount. The behavior after the ablations was indistinguishable from the behavior before the ablations. When new cue sets were introduced the pre- and post-ablation performances were also indistinguishable. When the task was changed so the amount of reward would increase (5 drops max.) every time 3 successive schedules were completed without error, and would decrease by a drop (1 drop min.) after an error, the performance was indistinguishable from intact monkeys, where the monkeys made almost no errors early in the testing session, and then worked for a considerable period of time, i.e. making errors at the lowest reward level. In this task in addition to learning associations between stimuli and rewards the monkeys had to take into account their errors to adjust their performance to maximize reward outcome. In a third experiment, there was only one trial, and the cue indicated the time after trial completion and reward delivery (reward postponement), allowing us to test the ability of the monkeys to evaluate the balance between reward and delay cost. The performance of the lesioned monkeys was indistinguishable from intact animals. Thus, cingulate cortex ablation does not impair the ability of the monkeys to evaluate the reward value and adjust their behavior accordingly. The cingulate cortex does not seem necessary in reward or for cost-benefit evaluation in tasks in which there is no need to select among several actions. [unreadable] [unreadable] We examined neural activity in the caudate nucleus, a brain structure related to reward expectation, using the reward postponement task. Error rates were explained well by hyperbolic discounting with both reward size and delay duration. Of 31 phasically active neurons recorded in the caudate nucleus of a monkey performing this task, 27 showed significant responses to at least one task event (cue, GO, release or reward). For the 18 cue-responsive cells, normalized regression coefficients for reward size were negatively correlated with those for delay, suggesting that this population of neurons encodes subjective value computed from reward size and postponement. The responses of 4 of these 18 were related to both reward and delay. These results suggest that neurons in the caudate nucleus combine temporal discounting and intrinsic reward value in different proportions, giving rise to an estimate of overall subjective value within the population.