CiteULike: nelmor's Glimcher

Value Representations in the Primate Striatum during Matching Behavior

Brian Lau — 2008-05-08T09:40:23-00:00

Neuron, Vol. 58, No. 3. (8 May 2008), pp. 451-463.

Summary Choosing the most valuable course of action requires knowing the outcomes associated with the available alternatives. The striatum may be important for representing the values of actions. We examined this in monkeys performing an oculomotor choice task. The activity of phasically active neurons (PANs) in the striatum covaried with two classes of information: action-values and chosen-values. Action-value PANs were correlated with value estimates for one of the available actions, and these signals were frequently observed before movement execution. Chosen-value PANs were correlated with the value of the action that had been chosen, and these signals were primarily observed later in the task, immediately before or persistently after movement execution. These populations may serve distinct functions mediated by the striatum: some PANs may participate in choice by encoding the values of the available actions, while other PANs may participate in evaluative updating by encoding the reward value of chosen actions.

Action and Outcome Encoding in the Primate Caudate Nucleus

Brian Lau — 2007-12-27T13:19:51-00:00

J. Neurosci., Vol. 27, No. 52. (26 December 2007), pp. 14502-14514.

The basal ganglia appear to have a central role in reinforcement learning. Previous experiments, focusing on activity preceding movement execution, support the idea that dorsal striatal neurons bias action selection according to the expected values of actions. However, many phasically active striatal neurons respond at a time too late to initiate or select movements. Given the data suggesting a role for the basal ganglia in reinforcement learning, postmovement activity may therefore reflect evaluative processing important for learning the values of actions. To better understand these postmovement neurons, we determined whether individual striatal neurons encode information about saccade direction, whether a reward had been received, or both. We recorded from phasically active neurons in the caudate nucleus while monkeys performed a probabilistically rewarded delayed saccade task. Many neurons exhibited peak responses after saccade execution (77 of 149) that were often tuned for the direction of the preceding saccade (61 of 77). Of those neurons responding during the reward epoch, one subset showed direction tuning for the immediately preceding saccade (43 of 60), whereas another subset responded differentially on rewarded versus unrewarded trials (35 of 60). We found that there was relatively little overlap of these properties in individual neurons. The encoding of action and outcome was performed by largely separate populations of caudate neurons that were active after movement execution. Thus, striatal neurons active primarily after a movement appear to be segregated into two distinct groups that provide complimentary information about the outcomes of actions. 10.1523/JNEUROSCI.3060-07.2007

The neural correlates of subjective value during intertemporal choice.

Joseph W Kable — 2007-11-14T23:00:04-00:00

Nat Neurosci (4 November 2007)

Neuroimaging studies of decision-making have generally related neural activity to objective measures (such as reward magnitude, probability or delay), despite choice preferences being subjective. However, economic theories posit that decision-makers behave as though different options have different subjective values. Here we use functional magnetic resonance imaging to show that neural activity in several brain regions-particularly the ventral striatum, medial prefrontal cortex and posterior cingulate cortex-tracks the revealed subjective value of delayed monetary rewards. This similarity provides unambiguous evidence that the subjective value of potential rewards is explicitly represented in the human brain.

Statistics of Midbrain Dopamine Neuron Spike Trains in the Awake Primate

Hannah Bayer — 2007-07-19T11:11:14-00:00

J Neurophysiol (5 July 2007), 01140.2006.

Work in behaving primates indicates that midbrain dopamine neurons encode a prediction error, the difference between an obtained reward and the reward expected. Studies of dopamine action potential timing in the alert and anaesthetized rat indicate that dopamine neurons respond in tonic and phasic modes, a distinction that has been less well characterized in the primates. We used spike train models to examine the relationship between the tonic and burst modes of activity in dopamine neurons while monkeys were performing a reinforced visuo-saccadic movement task. We studied spiking activity during four task-related intervals; two of these were intervals during which no task-related events occurred, while two were periods marked by task-related phasic activity. We found that dopamine neuron spike trains during the intervals when no events occurred were well described as tonic. Action potentials appeared to be independent, to occur at low frequency, and to be almost equally well described by Gaussian and Poisson-like (Gamma) processes. Unlike in the rat, interspike intervals as low as 20 ms were often observed during these presumptively tonic epochs. Having identified these periods of presumptively tonic activity we were able to quantitatively define phasic modulations (both increases and decreases in activity) during the intervals in which task-related events occurred. This analysis revealed that the phasic modulations of these neurons include both bursting, as has been described previously, and pausing. Together bursts and pauses seemed to provide a continuous, although non-linear, representation of the theoretically defined reward prediction error of reinforcement learning. 10.1152/jn.01140.2006

Midbrain dopamine neurons encode a quantitative reward prediction error signal.

HM Bayer — 2005-07-20T16:11:35-00:00

Neuron, Vol. 47, No. 1. (7 July 2005), pp. 129-141.

The midbrain dopamine neurons are hypothesized to provide a physiological correlate of the reward prediction error signal required by current models of reinforcement learning. We examined the activity of single dopamine neurons during a task in which subjects learned by trial and error when to make an eye movement for a juice reward. We found that these neurons encoded the difference between the current reward and a weighted average of previous rewards, a reward prediction error, but only for outcomes that were better than expected. Thus, the firing rate of midbrain dopamine neurons is quantitatively predicted by theoretical descriptions of the reward prediction error signal used in reinforcement learning models for circumstances in which this signal has a positive value. We also found that the dopamine system continued to compute the reward prediction error even when the behavioral policy of the animal was only weakly influenced by this computation.

Neural correlates of decision variables in parietal cortex.

ML Platt — 2005-02-01T19:51:55-00:00

Nature, Vol. 400, No. 6741. (15 July 1999), pp. 233-238.

Decision theory proposes that humans and animals decide what to do in a given situation by assessing the relative value of each possible response. This assessment can be computed, in part, from the probability that each action will result in a gain and the magnitude of the gain expected. Here we show that the gain (or reward) a monkey can expect to realize from an eye-movement response modulates the activity of neurons in the lateral intraparietal area, an area of primate cortex that is thought to transform visual signals into eye-movement commands. We also show that the activity of these neurons is sensitive to the probability that a particular response will result in a gain. When animals can choose freely between two alternative responses, the choices subjects make and neuronal activation in this area are both correlated with the relative amount of gain that the animal can expect from each response. Our data indicate that a decision-theoretic model may provide a powerful new framework for studying the neural processes that intervene between sensation and action.