Agile, Dynamic, and Dexterous Robot Learning in the Real World
Abstract
Program Officer: Dr. Behzad Kamgar-Parsi, behzad.kamgarparsi.civ@us.navy.milThis abstract is publicly releasable.ABSTRACTThere have been significant advances in robot manipulation in recent years: from grasping to pushing and pick-n-place tasks; from manipulating a rubik s cube to opening cabinet doors or makeshift doors. While there has been substantial progress, most experiments in this area have still been restricted to simulation or table-top experiments in the lab. This work delves into the question of how we could move from lab experiments to more in-the-wild setups. If the robots were to work in realistic scenarios in the wild, we argue that they must be trained in the real world itself. But how to do so safely and efficiently remains an open question.Biological evolution,on the other hand, concentrated most of the time on developing fundamental low-level sensorimotor skills, such as locomotion, manipulation, etc., that are just sufficient enough for survival. But once that infrastructure was in place, so-called "intelligent behavior", as we define it today, evolved relatively quickly. We plan to follow a similar analogy where we start with robust and agile locomotion followed by dexterous manipulation with multi-fingered hands the skills of which are then combined into a general-purpose mobile manipulation platform with both legs and wheels to perform complex dynamic tasks. To ensure the efficiency of the real-world robot learning, we initialize the process by pretrain the agent for discovering both low-level priors related to sensorimotor controland mid-level priors about what/where to interact in the real world using simulation and diverse video data on the web respectively. We use the above mechanisms to first develop agile locomotion and dexterous manipulation skills in isolation, and then put them together jointly in a general-purpose mobile manipulation setup. We evaluate diverse and dynamic real-world tasks with both wheeled aswell as legged mobile manipulators.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Aug 11, 2023
- Source ID
- N000142312782
Entities
People
- Deepak Pathak
Organizations
- Carnegie Mellon University
- Office of Naval Research
- United States Navy