Uncertainty-based Active Self-Learning for Perception
Abstract
State of the art in perception has been dominated by labeled datasets with millions of images. Even approaches using self-supervised objective function during learning without labels need millions of examples or episodes in reinforcement learning without applying any active strategy in acquiring new samples. When encountered with examples outside of the original distribution, they have difficulty recognizing them as such and adapting their model to the new encountered environments. We propose to study how a self-learning agent can introspect about its performance and identify when more informative data are needed. We model world knowledge in terms of uncertainty either due to lack of acquired knowledge about the world (epistemic, model) or due to ambiguity and noise in the measurements (aleatoric). Uncertainty can be computed using the disagreement in ensembles of neural networks and it can guide the self learning process by acquiring new samples that would decrease the model uncertainty. Self-learning has to be continual, meaning that models have to be adapted when introspection through uncertainty mandates it. Still, it is rarely the case that the entire model must be modified. We propose to develop new methods of learning factored and composable models that allow data-efficient learning of perceptual and action models in new environments and tasks. While active perception seeks to gain information about the world, purposeful, active perception explores in a task-relevant way. Instead of maximizing information gain about the general state of the world, we propose to reward actions that maximize the model uncertainty of task value predictions based on a learned dynamics model. As an example of incorporating the concepts above we will study an agent that learns to map and is thus capable of navigating easily in unseen environments. We propose a framework that actively learns the distribution of environmental layouts in terms of occupancy and locations of objects by exploring locations of maximal epistemic uncertainty. When an agent encounters a new environment and cannot explore it exhaustively, the agent follows an active strategy of simultaneous mapping and predicting where the object to search for might be located. Outcomes of this research will enable new DoD capabilities in environments that are unseen by allowing an efficient training strategy through active, continual self-learning.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Sep 08, 2022
- Source ID
- N000142212677
Entities
People
- Kostas Daniilidis
Organizations
- Office of Naval Research
- United States Navy
- University of Pennsylvania