Online Meta-Learning for Continuous Adaptation in Multi-Task Settings
Abstract
Current machine learning methods, especially deep neural networks, excel in the regime where large, statistically unbiased datasets are available. However, autonomous systems that operate in continual open-world settings must be able to adapt to changes in the environment, as well as changes in the task. In these settings, it is impractical to gather large datasets for every situation and task that the system may be confronted with, and the model must instead adapt to these changes while leveraging its prior knowledge. Unfortunately, precisely those things that make deep networks well-suited for the large data regime make them poorly suited for online adaptation: their high capacity and comparatively slow learning speed means that conventional supervised learning and reinforcement learning methods adapt relatively slowly. The objective of this project would be to investigate how meta-learning and multi-task learning methods can bridge this gap, developing a new generation of online meta-learning methods that can offer efficient continual adaptation by leveraging the experience of adapting to changes in the environment and task in the past, and by distilling this past experience into a compositional and modular multi-task representation. We will aim to develop a framework for online meta-learning in the multi-task setting, studying algorithms, representations, and applications. The algorithms portion of the project will aim to develop an algorithmic and theoretical framework for online meta-learning, so as to provide a general recipe and theoretical analysis for this new class of learning methods in a general, abstract sense. The representations portion of the project will extend this online meta-learning framework with compositional and modular representations, so as to make it suitable for the multi-task setting. The applications portion of the project will then instantiate the complete method in the setting of several applications, including predictive modeling and reinforcement learning of robotic skills.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jun 25, 2021
- Source ID
- W911NF2110097
Entities
People
- Sergey Levine
Organizations
- Army Contracting Command
- United States Army
- University of California, Berkeley