Online resource allocation using decompositional reinforcement learning Export

@inproceedings{citeulike:5939614, abstract = {This paper considers a novel application domain for reinforcement learning: that of "autonomic computing," i.e. selfmanaging computing systems. RL is applied to an online resource allocation task in a distributed multi-application computing environment with independent time-varying load in each application. The task is to allocate servers in real time so as to maximize the sum of performance-based expected utility in each application. This task may be treated as a composite MDP, and to exploit the problem structure, a simple localized RL approach is proposed, with better scalability than previous approaches. The RL approach is tested in a realistic prototype data center comprising real servers, real HTTP requests, and realistic time-varying demand. This domain poses a number of major challenges associated with live training in a real system, including: the need for rapid training, exploration that avoids excessive penalties, and handling complex, potentially non-Markovian system effects. The early results are encouraging: in overnight training, RL performs as well as or slightly better than heavily researched model-based approaches derived from queuing theory.}, author = {Tesauro, Gerald}, booktitle = {AAAI'05: Proceedings of the 20th national conference on Artificial intelligence}, citeulike-article-id = {5939614}, citeulike-linkout-0 = {http://portal.acm.org/citation.cfm?id=1619410.1619475}, isbn = {1-57735-236-x}, keywords = {2005, decentralised, mdp, reinforcement-learning, resource-allocation}, location = {Pittsburgh, Pennsylvania}, pages = {886--891}, posted-at = {2009-10-14 16:02:57}, priority = {0}, publisher = {AAAI Press}, title = {Online resource allocation using decompositional reinforcement learning}, url = {http://portal.acm.org/citation.cfm?id=1619410.1619475}, year = {2005} }

TY - CONF ID - citeulike:5939614 L3 - citeulike-article-id:5939614 N2 - This paper considers a novel application domain for reinforcement learning: that of "autonomic computing," i.e. selfmanaging computing systems. RL is applied to an online resource allocation task in a distributed multi-application computing environment with independent time-varying load in each application. The task is to allocate servers in real time so as to maximize the sum of performance-based expected utility in each application. This task may be treated as a composite MDP, and to exploit the problem structure, a simple localized RL approach is proposed, with better scalability than previous approaches. The RL approach is tested in a realistic prototype data center comprising real servers, real HTTP requests, and realistic time-varying demand. This domain poses a number of major challenges associated with live training in a real system, including: the need for rapid training, exploration that avoids excessive penalties, and handling complex, potentially non-Markovian system effects. The early results are encouraging: in overnight training, RL performs as well as or slightly better than heavily researched model-based approaches derived from queuing theory. CY - Pittsburgh, Pennsylvania SN - 1-57735-236-x EP - 891 TI - Online resource allocation using decompositional reinforcement learning PB - AAAI Press T2 - AAAI'05: Proceedings of the 20th national conference on Artificial intelligence SP - 886 KW - 2005 KW - decentralised KW - mdp KW - reinforcement-learning KW - resource-allocation AU - Tesauro, Gerald PY - 2005/// UR - http://portal.acm.org/citation.cfm?id=1619410.1619475 ER -