CiteULike: nelmor's value

Neuronal Ensemble Bursting in the Basal Forebrain Encodes Salience Irrespective of Valence

Shih-Chieh Lin — 2008-07-11T08:26:00-00:00

Neuron, Vol. 59, No. 1. (10 July 2008), pp. 138-149.

Summary Both reward- and punishment-related stimuli are motivationally salient and attract the attention of animals. However, it remains unclear how motivational salience is processed in the brain. Here, we show that both reward- and punishment-predicting stimuli elicited robust bursting of many noncholinergic basal forebrain (BF) neurons in behaving rats. The same BF neurons also responded with similar bursting to primary reinforcement of both valences. Reinforcement responses were modulated by expectation, with surprising reinforcement eliciting stronger BF bursting. We further demonstrate that BF burst firing predicted successful detection of near-threshold stimuli. Together, our results point to the existence of a salience-encoding system independent of stimulus valence. We propose that the encoding of motivational salience by ensemble bursting of noncholinergic BF neurons may improve behavioral performance by affecting the activity of widespread cortical circuits and therefore represents a novel candidate mechanism for top-down attention.

Noncholinergic Neurons in the Basal Forebrain: Often Neglected but Motivationally Salient

Brian Lau — 2008-07-11T08:24:44-00:00

Neuron, Vol. 59, No. 1. (10 July 2008), pp. 6-8.

Although noncholinergic neurons in the basal forebrain are known to contribute to cognition, their response properties in behaving animals is unclear. In this issue of Neuron, Lin and Nicolelis demonstrate that these neurons represent the motivational salience of sensory stimuli and may modulate cortical processing to direct top-down attention.

Dissociating the Role of the Orbitofrontal Cortex and the Striatum in the Computation of Goal Values and Prediction Errors

Todd Hare — 2008-05-29T14:39:09-00:00

J. Neurosci., Vol. 28, No. 22. (28 May 2008), pp. 5623-5630.

To make sound economic decisions, the brain needs to compute several different value-related signals. These include goal values that measure the predicted reward that results from the outcome generated by each of the actions under consideration, decision values that measure the net value of taking the different actions, and prediction errors that measure deviations from individuals' previous reward expectations. We used functional magnetic resonance imaging and a novel decision-making paradigm to dissociate the neural basis of these three computations. Our results show that they are supported by different neural substrates: goal values are correlated with activity in the medial orbitofrontal cortex, decision values are correlated with activity in the central orbitofrontal cortex, and prediction errors are correlated with activity in the ventral striatum. 10.1523/JNEUROSCI.1309-08.2008

Value Representations in the Primate Striatum during Matching Behavior

Brian Lau — 2008-05-08T09:40:23-00:00

Neuron, Vol. 58, No. 3. (8 May 2008), pp. 451-463.

Summary Choosing the most valuable course of action requires knowing the outcomes associated with the available alternatives. The striatum may be important for representing the values of actions. We examined this in monkeys performing an oculomotor choice task. The activity of phasically active neurons (PANs) in the striatum covaried with two classes of information: action-values and chosen-values. Action-value PANs were correlated with value estimates for one of the available actions, and these signals were frequently observed before movement execution. Chosen-value PANs were correlated with the value of the action that had been chosen, and these signals were primarily observed later in the task, immediately before or persistently after movement execution. These populations may serve distinct functions mediated by the striatum: some PANs may participate in choice by encoding the values of the available actions, while other PANs may participate in evaluative updating by encoding the reward value of chosen actions.

Action and Outcome Encoding in the Primate Caudate Nucleus

Brian Lau — 2007-12-27T13:19:51-00:00

J. Neurosci., Vol. 27, No. 52. (26 December 2007), pp. 14502-14514.

The basal ganglia appear to have a central role in reinforcement learning. Previous experiments, focusing on activity preceding movement execution, support the idea that dorsal striatal neurons bias action selection according to the expected values of actions. However, many phasically active striatal neurons respond at a time too late to initiate or select movements. Given the data suggesting a role for the basal ganglia in reinforcement learning, postmovement activity may therefore reflect evaluative processing important for learning the values of actions. To better understand these postmovement neurons, we determined whether individual striatal neurons encode information about saccade direction, whether a reward had been received, or both. We recorded from phasically active neurons in the caudate nucleus while monkeys performed a probabilistically rewarded delayed saccade task. Many neurons exhibited peak responses after saccade execution (77 of 149) that were often tuned for the direction of the preceding saccade (61 of 77). Of those neurons responding during the reward epoch, one subset showed direction tuning for the immediately preceding saccade (43 of 60), whereas another subset responded differentially on rewarded versus unrewarded trials (35 of 60). We found that there was relatively little overlap of these properties in individual neurons. The encoding of action and outcome was performed by largely separate populations of caudate neurons that were active after movement execution. Thus, striatal neurons active primarily after a movement appear to be segregated into two distinct groups that provide complimentary information about the outcomes of actions. 10.1523/JNEUROSCI.3060-07.2007

The neural correlates of subjective value during intertemporal choice.

Joseph W Kable — 2007-11-14T23:00:04-00:00

Nat Neurosci (4 November 2007)

Neuroimaging studies of decision-making have generally related neural activity to objective measures (such as reward magnitude, probability or delay), despite choice preferences being subjective. However, economic theories posit that decision-makers behave as though different options have different subjective values. Here we use functional magnetic resonance imaging to show that neural activity in several brain regions-particularly the ventral striatum, medial prefrontal cortex and posterior cingulate cortex-tracks the revealed subjective value of delayed monetary rewards. This similarity provides unambiguous evidence that the subjective value of potential rewards is explicitly represented in the human brain.

Encoding Predicted Outcome and Acquired Value in Orbitofrontal Cortex during Cue Sampling Depends upon Input from Basolateral Amygdala

Geoffrey Schoenbaum — 2007-10-18T17:07:56-00:00

Neuron, Vol. 39, No. 5. (28 August 2003), pp. 855-867.

Certain goal-directed behaviors depend critically upon interactions between orbitofrontal cortex (OFC) and basolateral amygdala (ABL). Here we describe direct neurophysiological evidence of this cooperative function. We recorded from OFC in intact and ABL-lesioned rats learning odor discrimination problems. As rats learned these problems, we found that lesioned rats exhibited marked changes in the information represented in OFC during odor cue sampling. Lesioned rats had fewer cue-selective neurons in OFC after learning; the cue-selective population in lesioned rats did not include neurons that were also responsive in anticipation of the predicted outcome; and the cue-activated representations that remained in lesioned rats were less associative and more often bound to cue identity. The results provide a neural substrate for representing acquired value and features of the predicted outcome during cue sampling, disruption of which could account for deficits in goal-directed behavior after damage to this system.

The influence of expected value on saccadic preparation.

DM Milstein — 2007-05-05T00:34:41-00:00

J Neurosci, Vol. 27, No. 18. (2 May 2007), pp. 4810-4818.

Basing higher-order decisions on expected value (reward probability x reward magnitude) maximizes an agent's accruement of reward over time. The goal of this study was to determine whether the advanced preparation of simple actions reflected the expected value of the potential outcomes. Human subjects were required to direct a saccadic eye movement to a visual target that was presented either to the left or right of a central fixation point on each trial. Expected value was manipulated by adjusting the probability of presenting each target and their associated magnitude of monetary reward across 15 blocks of trials. We found that saccadic reaction times (SRTs) were negatively correlated to the relative expected value of the targets. Occasionally, an irrelevant visual distractor was presented before the target to probe the spatial allocation of saccadic preparation. Distractor-directed errors (oculomotor captures) varied as a function of the relative expected value of, and the distance of distractors from, the potential valued targets. SRTs and oculomotor captures were better correlated to the relative expected value of actions than to reward probability, reward magnitude, or overall motivation. Together, our results suggest that the level and spatial distribution of competitive dynamic neural fields representing saccadic preparation reflect the relative expected value of the potential actions.

Representation of action-specific reward values in the striatum.

K Samejima — 2005-11-30T17:36:20-00:00

Science, Vol. 310, No. 5752. (25 November 2005), pp. 1337-1340.

The estimation of the reward an action will yield is critical in decision-making. To elucidate the role of the basal ganglia in this process, we recorded striatal neurons of monkeys who chose between left and right handle turns, based on the estimated reward probabilities of the actions. During a delay period before the choices, the activity of more than one-third of striatal projection neurons was selective to the values of one of the two actions. Fewer neurons were tuned to relative values or action choice. These results suggest representation of action values in the striatum, which can guide action selection in the basal ganglia circuit.

Beetles, boxes and brain cells: neural mechanisms underlying valuation and learning

Daniel Salzman — 2006-05-30T15:53:13-00:00

Current Opinion in Neurobiology, Vol. 15, No. 6. (December 2005), pp. 721-729.

Sensory cues in the environment can predict the availability of reward. Through experience, humans and animals learn these predictions and use them to guide their actions. For example, we can learn to discriminate chanterelles from ordinary champignons through experience. Assuming the development of a taste for the complex and lingering flavors of chanterelles, we therefore learn to value the same action -- picking mushrooms -- differentially depending upon the appearance of a mushroom. One major goal of cognitive neuroscience is to understand the neural mechanisms that underlie this sort of learning. Because the acquisition of rewards motivates much behavior, recent efforts have focused on describing the neural signals related to learning the value of stimuli and actions. Neurons in the basal ganglia, in midbrain dopamine areas, in frontal and parietal cortices and in other brain areas, all modulate their activity in relation to aspects of learning. By training monkeys on various behavioral tasks, recent studies have begun to characterize how neural signals represent distinct processes, such as the timing of events, motivation, absolute (objective) and relative (subjective) valuation, and the formation of associative links between stimuli and potential actions. In addition, a number of studies have either further characterized dopamine signals or sought to determine how such signaling might interact with target structures, such as the striatum and rhinal cortex, to underlie learning.

Orbitofrontal cortex mediates outcome encoding in pavlovian but not instrumental conditioning.

SB Ostlund — 2007-05-08T16:49:25-00:00

J Neurosci, Vol. 27, No. 18. (2 May 2007), pp. 4819-4825.

Previous studies have implicated the orbitofrontal cortex (OFC) in outcome encoding. However, it remains unknown whether the OFC is selectively involved in pavlovian stimulus-outcome learning or whether it also contributes to instrumental action-outcome learning. In experiment 1, we investigated this issue by assessing the effects of bilateral lesions of the OFC on the sensitivity of instrumental lever press performance to a reduction in the incentive value of the training outcome (a test of action-outcome encoding) and to outcome-specific pavlovian-instrumental transfer (a test of stimulus-outcome encoding). We found that post-training lesions of the OFC did not affect instrumental outcome devaluation, but abolished the transfer effect. Interestingly, lesions made before training had no effect on either task. In experiment 2, we explored the involvement of the OFC in updating stimulus-outcome associations after the underlying contingency, or predictive relationship, between these two events has been degraded. Shams displayed clear contingency learning, withholding conditioned responding to a stimulus that no longer reliably predicted its outcome while continuing to respond to a control stimulus that remained a good predictor of a different outcome. In contrast, OFC-lesioned rats stopped responding to both stimuli, regardless of their predictive status. Together, these findings suggest that the OFC supports outcome encoding in pavlovian, but not instrumental conditioning.

Medial prefrontal cell activity signaling prediction errors of action values

Madoka Matsumoto — 2007-04-24T23:38:33-00:00

Nature Neuroscience, Vol. 10, No. 5. (22 April 2007), pp. 647-656.

Encoding of time-discounted rewards in orbitofrontal cortex is independent of value representation.

MR Roesch — 2006-08-24T22:34:27-00:00

Neuron, Vol. 51, No. 4. (17 August 2006), pp. 509-520.

We monitored single-neuron activity in the orbitofrontal cortex of rats performing a time-discounting task in which the spatial location of the reward predicted whether the delay preceding reward delivery would be short or long. We found that rewards delivered after a short delay elicited a stronger neuronal response than those delivered after a long delay in most neurons. Activity in these neurons was not influenced by reward size when delays were held constant. This was also true for a minority of neurons that exhibited sustained increases in firing in anticipation of delayed reward. Thus, encoding of time-discounted rewards in orbitofrontal cortex is independent of the encoding of absolute reward value. These results are contrary to the proposal that orbitofrontal neurons signal the value of delayed rewards in a common currency and instead suggest alternative proposals for the role this region plays in guiding responses for delayed versus immediate rewards.

Distributed neural representation of expected value.

B Knutson — 2005-05-25T14:27:33-00:00

J Neurosci, Vol. 25, No. 19. (11 May 2005), pp. 4806-4812.

Anticipated reward magnitude and probability comprise dual components of expected value (EV), a cornerstone of economic and psychological theory. However, the neural mechanisms that compute EV have not been characterized. Using event-related functional magnetic resonance imaging, we examined neural activation as subjects anticipated monetary gains and losses that varied in magnitude and probability. Group analyses indicated that, although the subcortical nucleus accumbens (NAcc) activated proportional to anticipated gain magnitude, the cortical mesial prefrontal cortex (MPFC) additionally activated according to anticipated gain probability. Individual difference analyses indicated that, although NAcc activation correlated with self-reported positive arousal, MPFC activation correlated with probability estimates. These findings suggest that mesolimbic brain regions support the computation of EV in an ascending and distributed manner: whereas subcortical regions represent an affective component, cortical regions also represent a probabilistic component, and, furthermore, may integrate the two.