Adaptive Optimization Techniques for Large-Scale Stochastic Planning
Abstract
We developed new optimization-based methods for stochastic planning that offer better performance and better convergence guarantees compared to the state-of-the-art AI methods. In AI, reinforcement learning algorithms have proved useful in many complex domains, such as resource management and planning under uncertainty. These algorithms are often iterative-they successively approximate the solution based on a set of samples and features. Although these iterative algorithms can achieve impressive results in some domains, they have substantial drawbacks: they often require extensive parameter tweaking to work well and provide only weak guarantees of solution quality. Some of the most interesting reinforcement learning algorithms are based on approximate dynamic programming (ADP). ADP, also known as value function approximation, approximates the value of being in each state. This project produced new reliable algorithms for ADP that use optimization instead of iterative improvement. Because these optimization-based algorithms explicitly seek solutions with favorable properties, they are easy to analyze, offer much stronger guarantees than iterative algorithms, and have few or no parameters to tweak. In particular, we derive approximate bilinear programming|a new robust approximate method.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jun 28, 2011
- Accession Number
- ADA563724
Entities
People
- Shlomo Zilberstein
Organizations
- University of Massachusetts Amherst