CiteULike: nelmor's monkeys

Reward-Dependent Modulation of Neuronal Activity in the Primate Dorsal Raphe Nucleus

Kae Nakamura — 2008-05-14T17:09:11-00:00

J. Neurosci., Vol. 28, No. 20. (14 May 2008), pp. 5331-5343.

The dopamine system has been thought to play a central role in guiding behavior based on rewards. Recent pharmacological studies suggest that another monoamine neurotransmitter, serotonin, is also involved in reward processing. To elucidate the functional relationship between serotonin neurons and dopamine neurons, we performed single-unit recording in the dorsal raphe nucleus (DRN), a major source of serotonin, and the substantia nigra pars compacta, a major source of dopamine, while monkeys performed saccade tasks in which the position of the target indicated the size of an upcoming reward. After target onset, but before reward delivery, the activity of many DRN neurons was modulated tonically by the expected reward size with either large- or small-reward preference, whereas putative dopamine neurons had phasic responses and only preferred large rewards. After reward delivery, the activity of DRN neurons was modulated tonically by the received reward size with either large- or small-reward preference, whereas the activity of dopamine neurons was not modulated except after the unexpected reversal of the position-reward contingency. Thus, DRN neurons encode the expected and received rewards, whereas dopamine neurons encode the difference between the expected and received rewards. These results suggest that the DRN, probably including serotonin neurons, signals the reward value associated with the current behavior. 10.1523/JNEUROSCI.0021-08.2008

Value Representations in the Primate Striatum during Matching Behavior

Brian Lau — 2008-05-08T09:40:23-00:00

Neuron, Vol. 58, No. 3. (8 May 2008), pp. 451-463.

Summary Choosing the most valuable course of action requires knowing the outcomes associated with the available alternatives. The striatum may be important for representing the values of actions. We examined this in monkeys performing an oculomotor choice task. The activity of phasically active neurons (PANs) in the striatum covaried with two classes of information: action-values and chosen-values. Action-value PANs were correlated with value estimates for one of the available actions, and these signals were frequently observed before movement execution. Chosen-value PANs were correlated with the value of the action that had been chosen, and these signals were primarily observed later in the task, immediately before or persistently after movement execution. These populations may serve distinct functions mediated by the striatum: some PANs may participate in choice by encoding the values of the available actions, while other PANs may participate in evaluative updating by encoding the reward value of chosen actions.

Single neurons in posterior parietal cortex of monkeys encode cognitive set.

G Stoet — 2008-01-04T12:25:09-00:00

Neuron, Vol. 42, No. 6. (24 June 2004), pp. 1003-1012.

The primate posterior parietal cortex (PPC), part of the dorsal visual pathway, is best known for its role in encoding salient spatial information. Yet there are indications that neural activity in the PPC can also be modulated by nonspatial task-related information. In this study, we tested whether neurons in the PPC encode signals related to cognitive set, that is, the preparation to perform a particular task. Cognitive set has previously been associated with the frontal cortex but not the PPC. In this study, monkeys performed a cognitive set shifting paradigm in which they were cued in advance to apply one of two different task rules to the subsequent stimulus on every trial. Here we show that a subset of neurons in the PPC, concentrated in the lateral bank of the intraparietal sulcus and on the angular gyrus, responds selectively to cues for different task rules.

Single neurons in prefrontal cortex encode abstract rules

Jonathan Wallis — 2008-01-04T12:19:07-00:00

Nature, Vol. 411, No. 6840. (21 June 2001), pp. 953-956.

Action and Outcome Encoding in the Primate Caudate Nucleus

Brian Lau — 2007-12-27T13:19:51-00:00

J. Neurosci., Vol. 27, No. 52. (26 December 2007), pp. 14502-14514.

The basal ganglia appear to have a central role in reinforcement learning. Previous experiments, focusing on activity preceding movement execution, support the idea that dorsal striatal neurons bias action selection according to the expected values of actions. However, many phasically active striatal neurons respond at a time too late to initiate or select movements. Given the data suggesting a role for the basal ganglia in reinforcement learning, postmovement activity may therefore reflect evaluative processing important for learning the values of actions. To better understand these postmovement neurons, we determined whether individual striatal neurons encode information about saccade direction, whether a reward had been received, or both. We recorded from phasically active neurons in the caudate nucleus while monkeys performed a probabilistically rewarded delayed saccade task. Many neurons exhibited peak responses after saccade execution (77 of 149) that were often tuned for the direction of the preceding saccade (61 of 77). Of those neurons responding during the reward epoch, one subset showed direction tuning for the immediately preceding saccade (43 of 60), whereas another subset responded differentially on rewarded versus unrewarded trials (35 of 60). We found that there was relatively little overlap of these properties in individual neurons. The encoding of action and outcome was performed by largely separate populations of caudate neurons that were active after movement execution. Thus, striatal neurons active primarily after a movement appear to be segregated into two distinct groups that provide complimentary information about the outcomes of actions. 10.1523/JNEUROSCI.3060-07.2007

Neural correlates of a postponed decision report

Luis Lemus — 2007-10-23T09:32:44-00:00

Proceedings of the National Academy of Sciences (16 October 2007), 0707961104.

Depending on environmental demands, a decision based on a sensory evaluation may be either immediately reported or postponed for later report. If postponed, the decision must be held in memory. But what exactly is stored by the underlying memory circuits, the final decision itself or the sensory information that led to it? Here, we report that, during a postponed decision report period, the activity of medial premotor cortex neurons encodes both the result of the sensory evaluation that corresponds to the monkey's possible choices and past sensory information on which the decision is based. These responses could switch back and forth with remarkable flexibility across the postponed decision report period. Moreover, these responses covaried with the animal's decision report. We propose that maintaining in working memory the original stimulus information on which the decision is based could serve to continuously update the postponed decision report in this task. 10.1073/pnas.0707961104

Functional Specialization of the Primate Frontal Cortex during Decision Making

Daeyeol Lee — 2007-08-03T09:16:35-00:00

J. Neurosci., Vol. 27, No. 31. (1 August 2007), pp. 8170-8173.

Economic theories of decision making are based on the principle of utility maximization, and reinforcement-learning theory provides computational algorithms that can be used to estimate the overall reward expected from alternative choices. These formal models not only account for a large range of behavioral observations in human and animal decision makers, but also provide useful tools for investigating the neural basis of decision making. Nevertheless, in reality, decision makers must combine different types of information about the costs and benefits associated with each available option, such as the quality and quantity of expected reward and required work. In this article, we put forward the hypothesis that different subdivisions of the primate frontal cortex may be specialized to focus on different aspects of dynamic decision-making processes. In this hypothesis, the lateral prefrontal cortex is primarily involved in maintaining the state representation necessary to identify optimal actions in a given environment. In contrast, the orbitofrontal cortex and the anterior cingulate cortex might be primarily involved in encoding and updating the utilities associated with different sensory stimuli and alternative actions, respectively. These cortical areas are also likely to contribute to decision making in a social context. 10.1523/JNEUROSCI.1561-07.2007

Representation of action-specific reward values in the striatum.

K Samejima — 2005-11-30T17:36:20-00:00

Science, Vol. 310, No. 5752. (25 November 2005), pp. 1337-1340.

The estimation of the reward an action will yield is critical in decision-making. To elucidate the role of the basal ganglia in this process, we recorded striatal neurons of monkeys who chose between left and right handle turns, based on the estimated reward probabilities of the actions. During a delay period before the choices, the activity of more than one-third of striatal projection neurons was selective to the values of one of the two actions. Fewer neurons were tuned to relative values or action choice. These results suggest representation of action values in the striatum, which can guide action selection in the basal ganglia circuit.

Statistics of Midbrain Dopamine Neuron Spike Trains in the Awake Primate

Hannah Bayer — 2007-07-19T11:11:14-00:00

J Neurophysiol (5 July 2007), 01140.2006.

Work in behaving primates indicates that midbrain dopamine neurons encode a prediction error, the difference between an obtained reward and the reward expected. Studies of dopamine action potential timing in the alert and anaesthetized rat indicate that dopamine neurons respond in tonic and phasic modes, a distinction that has been less well characterized in the primates. We used spike train models to examine the relationship between the tonic and burst modes of activity in dopamine neurons while monkeys were performing a reinforced visuo-saccadic movement task. We studied spiking activity during four task-related intervals; two of these were intervals during which no task-related events occurred, while two were periods marked by task-related phasic activity. We found that dopamine neuron spike trains during the intervals when no events occurred were well described as tonic. Action potentials appeared to be independent, to occur at low frequency, and to be almost equally well described by Gaussian and Poisson-like (Gamma) processes. Unlike in the rat, interspike intervals as low as 20 ms were often observed during these presumptively tonic epochs. Having identified these periods of presumptively tonic activity we were able to quantitatively define phasic modulations (both increases and decreases in activity) during the intervals in which task-related events occurred. This analysis revealed that the phasic modulations of these neurons include both bursting, as has been described previously, and pausing. Together bursts and pauses seemed to provide a continuous, although non-linear, representation of the theoretically defined reward prediction error of reinforcement learning. 10.1152/jn.01140.2006

Neural Mechanisms of Visual Attention: How Top-Down Feedback Highlights Relevant Locations

Yuri Saalmann — 2007-06-15T08:59:06-00:00

Science, Vol. 316, No. 5831. (15 June 2007), pp. 1612-1615.

Attention helps us process potentially important objects by selectively increasing the activity of sensory neurons that represent the relevant locations and features of our environment. This selection process requires top-down feedback about what is important in our environment. We investigated how parietal cortical output influences neural activity in early sensory areas. Neural recordings were made simultaneously from the posterior parietal cortex and an earlier area in the visual pathway, the medial temporal area, of macaques performing a visual matching task. When the monkey selectively attended to a location, the timing of activities in the two regions became synchronized, with the parietal cortex leading the medial temporal area. Parietal neurons may thus selectively increase activity in earlier sensory areas to enable focused spatial attention. 10.1126/science.1139140

Direct and indirect activation of cortical neurons by electrical microstimulation.

EJ Tehovnik — 2007-06-05T22:02:38-00:00

J Neurophysiol, Vol. 96, No. 2. (August 2006), pp. 512-521.

Electrical microstimulation has been used to elucidate cortical function. This review discusses neuronal excitability and effective current spread estimated by using three different methods: 1) single-cell recording, 2) behavioral methods, and 3) functional magnetic resonance imaging (fMRI). The excitability properties of the stimulated elements in neocortex obtained using these methods were found to be comparable. These properties suggested that microstimulation activates the most excitable elements in cortex, that is, by and large the fibers of the pyramidal cells. Effective current spread within neocortex was found to be greater when measured with fMRI compared with measures based on single-cell recording or behavioral methods. The spread of activity based on behavioral methods is in close agreement with the spread based on the direct activation of neurons (as opposed to those activated synaptically). We argue that the greater activation with imaging is attributed to transynaptic spread, which includes subthreshold activation of sites connected to the site of stimulation. The definition of effective current spread therefore depends on the neural event being measured.

Activity in prefrontal cortex during dynamic selection of action sequences

Bruno Averbeck — 2006-01-27T00:56:57-00:00

Nature Neuroscience, Vol. 9, No. 2. (22 January 2006), pp. 276-282.

Leading tonically active neurons of the striatum from reward detection to context recognition

Paul Apicella — 2007-04-12T13:30:26-00:00

Trends in Neurosciences, Vol. In Press, Corrected Proof

Tonically active neurons (TANs) in the primate striatum, which are presumed to be cholinergic interneurons, carry signals that are traditionally considered to be important for reward-related learning. Recent studies investigating the functional properties of TANs in behaving monkeys have shown that other factors beyond motivation can affect their responsiveness. There is now evidence that TAN responses reflect stimulus detection, movement control and recognition of a specific context, suggesting that these local circuit neurons contribute to different computations used in learning and action functions of the striatum. This is consistent with the view that TAN responses could represent an important component of the processes that are responsible for the ability to select the appropriate behavioral response to environmental events.

Midbrain dopamine neurons encode a quantitative reward prediction error signal.

HM Bayer — 2005-07-20T16:11:35-00:00

Neuron, Vol. 47, No. 1. (7 July 2005), pp. 129-141.

The midbrain dopamine neurons are hypothesized to provide a physiological correlate of the reward prediction error signal required by current models of reinforcement learning. We examined the activity of single dopamine neurons during a task in which subjects learned by trial and error when to make an eye movement for a juice reward. We found that these neurons encoded the difference between the current reward and a weighted average of previous rewards, a reward prediction error, but only for outcomes that were better than expected. Thus, the firing rate of midbrain dopamine neurons is quantitatively predicted by theoretical descriptions of the reward prediction error signal used in reinforcement learning models for circumstances in which this signal has a positive value. We also found that the dopamine system continued to compute the reward prediction error even when the behavioral policy of the animal was only weakly influenced by this computation.

Medial prefrontal cell activity signaling prediction errors of action values

Madoka Matsumoto — 2007-04-24T23:38:33-00:00

Nature Neuroscience, Vol. 10, No. 5. (22 April 2007), pp. 647-656.

Neuronal activity related to reward value and motivation in primate frontal cortex.

MR Roesch — 2005-02-07T23:12:13-00:00

Science, Vol. 304, No. 5668. (9 April 2004), pp. 307-310.

In several areas of the macaque brain, neurons fire during delayed-response tasks at a rate determined by the value of the reward expected at the end of the trial. The activity of these neurons might be related to the value of the expected reward or to the degree of motivation induced by expectation of the reward. We describe results indicating that the nature of reward-dependent activity varies across areas. Neuronal activity in orbitofrontal cortex represents the value of the expected reward, whereas neuronal activity in premotor cortex reflects the degree of motivation.

Neural correlates of decision variables in parietal cortex.

ML Platt — 2005-02-01T19:51:55-00:00

Nature, Vol. 400, No. 6741. (15 July 1999), pp. 233-238.

Decision theory proposes that humans and animals decide what to do in a given situation by assessing the relative value of each possible response. This assessment can be computed, in part, from the probability that each action will result in a gain and the magnitude of the gain expected. Here we show that the gain (or reward) a monkey can expect to realize from an eye-movement response modulates the activity of neurons in the lateral intraparietal area, an area of primate cortex that is thought to transform visual signals into eye-movement commands. We also show that the activity of these neurons is sensitive to the probability that a particular response will result in a gain. When animals can choose freely between two alternative responses, the choices subjects make and neuronal activation in this area are both correlated with the relative amount of gain that the animal can expect from each response. Our data indicate that a decision-theoretic model may provide a powerful new framework for studying the neural processes that intervene between sensation and action.