The Essential Dynamics Algorithm: Essential Results

Abstract

This paper presents a novel algorithm for learning in a class of stochastic Markov decision processes (MDPs) with continuous state and action spaces that trades speed for accuracy. A transform of the stochastic MDP into a deterministic one is presented which captures the essence of the original dynamics, in a sense made precise. In this transformed MDP, the calculation of values is greatly simplified. the online algorithm estimates the model of the transformed MDP and simultaneously does policy search against it. Bounds on the error of this approximation are proven, and experimental results in a bicycle riding domain are presented. The algorithm learns near optimal policies in orders of magnitude fewer interactions with the stochastic MDP, using less domain knowledge.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
May 01, 2003
Accession Number
ADA434755

Entities

People

  • Martin C. Martin

Organizations

  • Massachusetts Institute of Technology

Tags

Communities of Interest

  • Autonomy
  • Energy and Power Technologies

DTIC Thesaurus Topics

  • Algorithms
  • Artificial Intelligence
  • Covariance
  • Distance Learning
  • Dynamics
  • Four Dimensional
  • Learning
  • Numbers
  • Probability
  • Random Variables
  • Real Numbers
  • Simulations
  • Simulators
  • Square Roots
  • Theorems
  • Three Dimensional
  • Transitions

Readers

  • Agent-Based Social Robotics and Mobile-Assisted Learning in Virtual Environments.
  • Computational Modeling and Simulation
  • Mathematical Modeling and Probability Theory.

Technology Areas

  • Space
  • Space - Spacecraft Maneuvers