Learning to Generalize Beyond Training
Abstract
"APPROVED FOR PUBLIC RELEASE": We, as humans, are naturally able to re-use our past experiences to adapt to novel scenarios. This ab,ility to generalize is the hallmark of all biological intelligence. While we have systems that excel at recognizing objects, cleanin,g floors, playing complex games, and occasionally beating humans, they are incredibly specific in that they only perform the tasks t,hey are trained for and are miserable at generalization. Why is that the case? The reason is rather unsurprising, but questions a fu,ndamental assumption in machine learning: most of the ML approaches assume that the distribution at test time will be the same as se,en during training. We cannot hope to capture all the possibilities, which one could ever encounter in the future, in a training dat,aset collected once ahead of time. It is a problem because our world is complex and continuously changing which provides us with com,plex novel tasks and challenges, hence, just scaling data and compute is implausible as a solution.To scale our AI systems beyond th,e lab environments to real-world setups, our models must learn to generalize across changing data as well as task distributions. Thi,s suggests, instead of learning on pre-defined specific tasks and data, agents should rather learn how to continually adapt themselv,es when faced with new scenarios. In this proposal, we expand upon the line of research in endowing artificial agents with a human-l,ike ability to generalize in diverse scenarios.The main insight is to move away from the current paradigm of having a fixed training, and testing datasets and have the agent learn continually in a lifelong manner. Although continual adaptation can aid generalizatio,n to out-of-distribution scenarios, it raises several questions regarding how to generate continuous supervision, how to reuse knowl,edge, how to learn efficiently in the real world, and how not to forget while accumulating knowledge in an easily accessible fashion,. We propose a unified framework with concrete steps to tackle these considerations and build continually evolving agents which woul,d then be able to show generalization across tasks, environments, and robots.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jan 14, 2022
- Source ID
- N000142212096
Entities
People
- Deepak Pathak
Organizations
- Carnegie Mellon University
- Office of Naval Research
- United States Navy