Large-Scale Unsupervised Learning For Robots
Abstract
RESEARCH PROBLEM: Recent advances in AI have given rise to seemingly unprecedented capabilities in robotics. Indeed, the same kinds of advances that have powered mastering the game of Go and excelling at a wide range of video games, have also powered simulated an d real robots learning locomotion, robotic grasping and assembly, and even some forms of dexterous manipulation such as solving a Ru biks cube. While impressive, a closer look at these breakthroughs reveals some fundamental limitations: each of these pertained to robotics in simulation or lab environments, which are much more structured and controlled compared to what robots would encounter i n real-world operations. In contrast, todays computer vision systems perform well on previously unseen real-world images. Similarly , todays natural language processing systems perform well on previously unseen real-world text.RESEARCH OBJECTIVES AND TECHNICAL AP PROACHES: We propose to investigate the following core research question: Can we leverage the large amount of videos of humans to en able robots to acquire new visuomotor skills in highly varied real-world settings?Our agenda will consist of three research objectiv es: [O1] Representation Learning from Unstructured Videos: Under O1 we will investigate representation learning from unstructured vi deos. Representation learning from videos is still in its early days, so in addition to investigating the effectiveness of pre-trai ned representations on imitation and reinforcement learning, we will also investigate their effectiveness on other downstream tasks of general interest, including video classification, image and video segmentation, depth prediction. [O2] Few-Shot Imitation and Emb odiment Transfer: While fine-tuning on imitation of robot demonstrations (as investigated under O1) is a natural first step, ultimat ely we would want to only require a few human demonstrations. Past work on few-shot learning has required many demonstrations on ta sks very similar tothe final downstream task. We will investigate to what extent this can be alleviated, while also addressing the h uman-to-robot embodiment transfer.[O3] RL Finetuning and Reward Learning: For more complex skills, demonstrations might alleviate hi gh-level exploration by providing the gist of how the task can be completed, but a robot might still need to practice to truly maste r the task. Such practice can naturally be done through Reinforcement Learning --- assuming a reward function is available. Under O3 we will investigate few-shot acquisition of reward functions enabling RL for fine-tuning a skill.ANTICIPATED OUTCOME AND IMPACT O N DOD: Ultimately, we envision a system where, given a couple of human demonstrations of a new skill, robots can quickly acquire the skill in a real-world unstructured environment thanks to large-scale pre-training. Logistical operations, from shipping, warehousi ng, stock-keeping to kit making and meal preparation, as well as equipment maintenance and repair are all tasks currently done by ha nd. The proposed research will open up the potential to start automating a large fraction (if not all) of these tasks. This will a llow for more efficient and reliable operations. It will also prevent requiring people to do those operations in possibly dangerous environments.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Aug 20, 2021
- Source ID
- N000142112769
Entities
People
- Pieter Abbeel
Organizations
- Office of Naval Research
- United States Navy
- University of California Regents