American Statistical Association
New York City
Metropolitan Area Chapter

Mailman School of Public Health
Columbia University
Department of Biostatistics Colloquium



ADVANCES IN MODEL-BASED REINFORCEMENT LEARNING
OR Q-LEARNING CONSIDERED HARMFUL

by

Michael Littman, Ph.D.
Professor & Chair
Department of Computer Science
Rutgers University
www.cs.rutgers.edu/~mlittman/


Abstract

Reinforcement learners seek to minimize sample complexity, the amount of experience needed to achieve adequate behavior, and computational complexity, the amount of computation needed per experience. Q-learning is a baseline algorithm with minimal computational complexity, but potentially unbounded sample complexity. Variants of Q-learning that use eligibility traces, value function approximation, or hierarchical task representations, have shown promise in decreasing sample complexity. I will compare these results to what we can obtain by using model-based learning -- using experience to model the contingencies in the environment. In terms of sample complexity, we find that model-based learning without these variants dominate Q-learning with these variants, but also that the modifications prove useful in other ways in the model-based setting.

Biographical Note

Michael L. Littman is professor and chair of the Department of Computer Science at Rutgers University and directs the Rutgers Laboratory for Real-Life Reinforcement Learning (RL3). His research in machine learning examines algorithms for decision making under uncertainty. Littman has earned multiple awards for teaching and his research has been recognized with three best-paper awards on the topics of meta-learning for computer crossword solving, complexity analysis of planning under uncertainty, and algorithms for efficient reinforcement learning. He has served on the editorial boards for several machine-learning journals and was Programme Co-chair of the International Conference of Machine Learning in 2009.


Date: Wednesday, February 9, 2011
Time: 4:00 - 5:00 P.M.
Location: Mailman School of Public Health
Department of Biostatistics
722 West 168th Street
Biostatistics Computer Lab
6th Floor - Room 656
New York, New York

RESERVATIONS ARE NOT REQUIRED

Informal tea at 3:40 P.M.


Home Page | Chapter News | Chapter Officers | Chapter Events
Other Metro Area Events | ASA National Home Page | Links To Other Websites
NYC ASA Chapter Constitution | NYC ASA Chapter By-Laws

Page last modified on February 3, 2011

Copyright © 1998-2011 by New York City Metropolitan Area Chapter of the ASA
Designed and maintained by Cynthia Scherer
Send questions or comments to nycasa@mindspring.com