An adaptive optimal controller for discrete-time Markov environments

@article{witten1977, abstract = {This paper describes an adaptive controller for discrete-time stochastic environments. The controller receives the environment's current state and a reward signal which indicates the desirability of that state. In response, it selects an appropriate control action and notes its effect. The cycle repeats indefinitely. The control environments to be tackled include the well-known n-armed bandit problem, and the adaptive controller comprises an ensemble of n-armed bandit controllers, suitably interconnected. The design of these constituent elements is not discussed. It is shown that, under certain conditions, the controller's actions eventually become optimal for the particular control task with which it is faced, in the sense that they maximize the expected reward obtained in the future.}, author = {Witten, Ian H.}, citeulike-article-id = {436261}, doi = {10.1016/S0019-9958(77)90354-0}, journal = {Information and Control}, keywords = {adaptive, controltheory, markov}, month = {August}, number = {4}, pages = {286--295}, posted-at = {2005-12-12 14:07:53}, priority = {2}, title = {An adaptive optimal controller for discrete-time Markov environments}, url = {http://dx.doi.org/10.1016/S0019-9958(77)90354-0}, volume = {34}, year = {1977} }

TY - JOUR ID - witten1977 L3 - citeulike-article-id:436261 N2 - This paper describes an adaptive controller for discrete-time stochastic environments. The controller receives the environment's current state and a reward signal which indicates the desirability of that state. In response, it selects an appropriate control action and notes its effect. The cycle repeats indefinitely. The control environments to be tackled include the well-known n-armed bandit problem, and the adaptive controller comprises an ensemble of n-armed bandit controllers, suitably interconnected. The design of these constituent elements is not discussed. It is shown that, under certain conditions, the controller's actions eventually become optimal for the particular control task with which it is faced, in the sense that they maximize the expected reward obtained in the future. IS - 4 JF - Information and Control EP - 295 TI - An adaptive optimal controller for discrete-time Markov environments VL - 34 SP - 286 KW - adaptive KW - controltheory KW - markov AU - Witten, Ian PY - 1977/08// UR - http://dx.doi.org/10.1016/S0019-9958(77)90354-0 ER -