Lifelong Robot Learning With Sensorimotor Skills
Abstract
This project focuses on developing algorithms, principles, and tools for lifelong learning of intelligent robotic agents. A longstanding goal of the science of autonomy is to develop a generalist agent capable of mastering a diverse and potentially expanding set of tasks that it encounters over the course of its autonomous existence. For example, consider a naval robot for coastal navigation,where the environments and currents change monthly. In this dynamic context, it would be impractical for the robot to foresee all potential circumstances and uncertainties a priority. Instead, the robot must continually adapt and learn new behaviors as it undertakes missions in the field. To endow robots with the adaptability and resilienceneeded for long-term autonomy, lifelong learning aimsto empower the robot to re-use past knowledge to learn faster while retaining proficiency at previous tasks. While lifelong learning algorithms have made great strides in fields like visual recognition and natural language understanding, building lifelong learning agents for realistic decision-making domains, such as long-horizon mobile manipulation, remains a daunting challenge. The challenge is three-fold: 1) The research community lacks a systematic benchmark and standardized metrics for studying lifelong learning agents, hindering researchers from investigating the aspects of continual learning and adaptation required for an autonomous robot#s long lifespan; 2) The high sample complexity and limited generalization of conventional policy learning algorithms render it difficult to extend them to lifelong learning settings; and 3) Prior approaches have primarily focused on lifelong learning of semantic concepts, while the continual adaptation and transfer of robotic behaviors remains an open challenge. The overarching goal of this projectis to build autonomous agents that continually learn new tasks in new environments. Specifically, we focus on the autonomous robot manipulation domain,including tabletop and mobile manipulation tasks in real-world unstructured environments, which will broaden theset of autonomous capabilities available in naval settings. This domain requires the robotic agent not only to understand various concepts through the lens of its visual perception (e.g., declarative knowledge about where and what an object is) but also to reasonabout how its actions influence the environment (e.g., procedural knowledge about how to accomplish the task). We aim to tackle thefundamental challenges of lifelong learning in decision-making through three intimately connected research efforts: 1) We will construct a new benchmark for systematically studying the problem of lifelong learning for robotic decision-making; 2) We will develop aversatile model architecture for data-efficient learning of generalizable sensorimotor skills; and 3) We will build a skill-based lifelong learning algorithm to facilitate the transfer of both declarative and procedural knowledge during long-term robot autonomy. This research will innovate general methods and principles for lifelong learning in sequential decision-making and establish standard benchmarks for accelerating future research in this direction. Methods developed in this research will catalyze the deployment of intelligent manipulation robots in the real world, including complex missions in naval environments. Methods and principles developed in this research will reduce the technical barriers of deploying autonomous robots to perform complex tasks, making them more accessible to sailors, military personnel, and civilians. In addition to military advantages, this research includes outreach activitiesthrough UT Austin#s Freshman Research Initiative (FRI), mentoring graduate students and undergraduate researchers from underrepresented groups, and dissemination of research outcomes through publications and open-source initiatives.Approved for Public Release.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Nov 08, 2024
- Source ID
- N000142412550
Entities
People
- Yuke Zhu
Organizations
- Office of Naval Research
- United States Navy
- University of Texas at Austin