Deep Predictive Learning in Vision
Abstract
Research problem: Our existing research has identified powerful predictive learning mechanisms that enable our biologically-based computational models to learn invariant object representations in a purely self-organizing manner, without requiring the massive sets of hand-labeled images that standard deep neural networks are critically dependent upon. These mechanisms are based directly upon the biology of the primate visual system, and make sense of a wide range of data at many different levels of analysis. However, our existing models were developed using simplified visual inputs, and we propose here to scale them up to handle large numbers of realistic 3D objects, ultimately with multiple objects at a time interacting with each other.Technical approach: Predictive learning requires movie-like animated scenes, where the model is constantly predicting what will happen next, and learning from the difference between what actually does happen and the prediction. According to our theory, this prediction-outcome learning cycle occurs every 100 msec (10 Hz, i.e, the alpha rhythm). Thus, massive amounts of predictive learning can occur every day, providing the constraints to shape powerful internal generative, predictive models of the world. We propose to render large numbers of systematically generated scenes involving 3D objects moving with realistic physics, to train our models on increasingly complex and realistic environments. Ultimately, our models should be able to learn from real movies and video footage, and apply their knowledge to the interpretation of real-world images and scenes, in a semantically rich and meaningful manner.Anticipated outcome and impacts: It is hard to overstate the potential impact that this project could have on DoD capabilities, by enabling multiple different mission critical technologies that all depend, as people do, on the ability to effectively extract physical semantics from visual scenes. For example, this could greatly facilitate automated processing of large volumes of imagery and monitoring data. Likewise, autonomous robotic systems, which currently depend to a large extent on human operators for their perceptual abilities, could benefit substantially. In addition, we anticipate learning much more about advanced visual processing in the brain.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Aug 20, 2019
- Source ID
- N000141912684
Entities
People
- Randall O Reilly
Organizations
- Office of Naval Research
- United States Navy
- University of California, Davis