An exploration-exploitation model based on norepinepherine and dopamine activity Export

@article{citeulike:1142278, abstract = {We propose a model by which dopamine (DA) and norepinepherine (NE) combine to alternate behavior between relatively exploratory and exploitative modes. The model is developed for a target detection task for which there is extant single neuron recording data available from locus coeruleus (LC) NE neurons. An exploration-exploitation trade-off is elicited by regularly switching which of the two stimuli are rewarded. DA functions within the model to change synaptic weights according to a reinforcement learning algorithm. Exploration is mediated by the state of LC firing, with higher tonic and lower phasic activity producing greater response variability. The opposite state of LC function, with lower baseline firing rate and greater phasic responses, favors exploitative behavior. Changes in LC firing mode result from combined measures of response conflict and reward rate, where response conflict is monitored using models of anterior cingulate cortex (ACC). Increased long-term response conflict and decreased reward rate, which occurs following reward contingency switch, favors the higher tonic state of LC function and NE release. This increases exploration, and facilitates discovery of the new target.}, author = {Mcclure, Samuel M. and Gilzenrat, Mark S. and Cohen, Jonathan D.}, citeulike-article-id = {1142278}, comment = {see summary here: http://scienceblogs.com/developingintelligence/2007/02/modulation\_of\_exploration\_expl.php}, journal = {Advances in Neural Information Processing Systems}, keywords = {acc, da, exploration, lc, model, monitoring, na, rt\_variability}, posted-at = {2007-03-05 21:02:45}, priority = {2}, title = {An exploration-exploitation model based on norepinepherine and dopamine activity}, year = {2007} }

TY - JOUR ID - citeulike:1142278 L3 - citeulike-article-id:1142278 N2 - We propose a model by which dopamine (DA) and norepinepherine (NE) combine to alternate behavior between relatively exploratory and exploitative modes. The model is developed for a target detection task for which there is extant single neuron recording data available from locus coeruleus (LC) NE neurons. An exploration-exploitation trade-off is elicited by regularly switching which of the two stimuli are rewarded. DA functions within the model to change synaptic weights according to a reinforcement learning algorithm. Exploration is mediated by the state of LC firing, with higher tonic and lower phasic activity producing greater response variability. The opposite state of LC function, with lower baseline firing rate and greater phasic responses, favors exploitative behavior. Changes in LC firing mode result from combined measures of response conflict and reward rate, where response conflict is monitored using models of anterior cingulate cortex (ACC). Increased long-term response conflict and decreased reward rate, which occurs following reward contingency switch, favors the higher tonic state of LC function and NE release. This increases exploration, and facilitates discovery of the new target. JF - Advances in Neural Information Processing Systems TI - An exploration-exploitation model based on norepinepherine and dopamine activity KW - acc KW - da KW - exploration KW - lc KW - model KW - monitoring KW - na KW - rt_variability N1 - see summary here: http://scienceblogs.com/developingintelligence/2007/02/modulation_of_exploration_expl.php AU - Mcclure, Samuel AU - Gilzenrat, Mark AU - Cohen, Jonathan PY - 2007/// ER -