Learning Dynamics, Task Technology and Information Plasticity for Weakly- Supervised Representation Learning
Abstract
This project is intended to explore (Task 1) defining a topology in the space of tasks, and computing ""static"" distances, or diverg"ence measures, between tasks, which help determine what resources (time and capacity) are needed to transfer a trained model to a f"ine-tuned task; (Task 2) studying the role of the transient dynamics of learning, and define a ""dynamic"" distance, or discrepancy ," between tasks, to enable analyzing critical learning periods in biological and artificial systems. and (Task 3) exploit information plasticity to overcome critical learning period and facilitate learning with few or no supervision (semi supervised learning). We have discovered that artificial systems exhibit critical learning periods, much like biological systems, which prompts to the critical role of the transient dynamics in learning, whereas almost the entirety of the analysis and theoretical results in machine learning and optimization are asymptotic. Since the transient is, literally, critical, we believe it is urgent to investigate the fundamental issues underlying critical periods, and we have preliminary evidence that such understanding can be leveraged for improving semi-supervised and transfer learning algorithms. The fact that critical period phenomena involve information plasticity, rather than (biological) neural plasticity, suggests that we can conduct Artificial Neuroscience experiments, far simpler and cheaper than psychophysical experiments, before validating the observations in biology.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Apr 25, 2019
- Source ID
- N000141912229
Entities
People
- Stefano Soatto
Organizations
- Office of Naval Research
- United States Navy
- University of California, Los Angeles