Robotic Reinforcement Learning with Diverse Data Sources
Abstract
Robotic control requires close integration between perception and action: advanced modern robotic systems, from mobile robots to robotic manipulators, must not only perceive the world and plan complex trajectories, but they must do so in closed loop, rapidly responding to unpredictable and changing environments while integrating rich sensory information from a wide variety of inputs, including vision, touch, sound, and a myriad of other sources. The classic paradigm based on model-based control requires accurate characterization of the environment and the robot, which quickly becomes prohibitive in complex open-world settings. Machine learning offers an alternative, tightly closing the loop between perception and control directly using data and experience. However, the standard machine learning approach based on reinforcement learning suffers from major limitations when applied naively in robotics: data-hungry methods that are designed to work with large-scale manually annotated datasets are difficult to apply to robotic problems where such datasets do not exist. Therefore, effective robotic learning requires overcoming the challenge of data procurement. In this project, we propose to study this through automating direct real-world robotic interaction so as to make it cheap to acquire data in the real world, utilizing simulated data in combination with real-world experience, and utilizing auxiliary non-robotic data sources to complement learning from the robot#s own experience. We anticipate that these directions will make it possible to apply modern machine learning methods such as deep learning, which traditionally require large labeled data sources, to complex problems in robotics where generating sufficiently large datasets is currently challenging. Since the focus of this research revolves around methods that make it possible to integrate plentiful data sources into robotic learning, the computational costs of developing such algorithms are considerable, and therefore our proposal is to acquire sufficient GPU computing infrastructure to accelerate such research. This computing infrastructure will be used to facilitate eight existing DoD-supported projects, including six ONR-funded projects which include a MURI where both Levine and Abbeel are PIs.[Publicly Releasable]
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jan 12, 2023
- Source ID
- N000142312178
Entities
People
- Sergey Levine
Organizations
- Office of Naval Research
- United States Navy
- University of California Regents