American Statistical Association
Reinforcement learners seek to minimize sample complexity, the amount of experience needed to achieve adequate behavior, and computational complexity, the amount of computation needed per experience. Q-learning is a baseline algorithm with minimal computational complexity, but potentially unbounded sample complexity. Variants of Q-learning that use eligibility traces, value function approximation, or hierarchical task representations, have shown promise in decreasing sample complexity. I will compare these results to what we can obtain by using model-based learning -- using experience to model the contingencies in the environment. In terms of sample complexity, we find that model-based learning without these variants dominate Q-learning with these variants, but also that the modifications prove useful in other ways in the model-based setting.
Michael L. Littman is professor and chair of the Department of Computer Science at Rutgers University and directs the Rutgers Laboratory for Real-Life Reinforcement Learning (RL3). His research in machine learning examines algorithms for decision making under uncertainty. Littman has earned multiple awards for teaching and his research has been recognized with three best-paper awards on the topics of meta-learning for computer crossword solving, complexity analysis of planning under uncertainty, and algorithms for efficient reinforcement learning. He has served on the editorial boards for several machine-learning journals and was Programme Co-chair of the International Conference of Machine Learning in 2009.
|Date:||Wednesday, February 9, 2011|
|Time:||4:00 - 5:00 P.M.|
Mailman School of Public Health
Department of Biostatistics
722 West 168th Street
Biostatistics Computer Lab
6th Floor - Room 656
New York, New York