Learning How to Make Decisions from Large Datasets
Abstract
Reinforcement learning (RL) algorithms have been shown to be capable of learning remarkable skills in recent years, from beating the world champion at Go, to learning robotic manipulation skills, to piloting an F-16 in a dogfight. Currently, such RL systems requir e active interaction with the system that they are learning to control. This means that either the system must be simulated, or a ph ysical system must be available with which it is possible to perform trial-and-error learning. This is often impossible: many system s are hard to simulate (e.g., systems that require interacting with humans, especially real-world networks of human agents), and rea l-world interaction can be dangerous (as in the case of aircraft or cars), expensive (as in the case of human-in-the-loop systems), or simply impractical (as in the case of large-scale decision making for inventory management, logistics, etc.). In contrast, superv ised deep learning methods readily generalize in open-world settings in vision, speech recognition, and NLP. Their ability to learn from large amounts of real world data is what makes this generalization possible. We will aim to enable reinforcement learning algor ithms to benefit from large real-world datasets, without simulation or active interaction. By developing a new generation of data-dr iven offline RL methods, we will aim to enable learning of optimal goal-directed skills directly from prior datasets. Such data-driv en reinforcement learning methods would not require interaction with a physical system, only data of past interactions from a human user or operator, or even a hand-designed controller.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Sep 07, 2021
- Source ID
- N000142112838
Entities
People
- Sergey Levine
Organizations
- Office of Naval Research
- United States Navy
- University of California Regents