Developing next generation AI vision systems by characterizing and exploiting untapped primate visual processing circuit motifs
Abstract
We propose that a deeper understanding of the functional role of primate brain circuits that are not yet implemented in current AI vision systems (e.g. massive bi-directionality, top-down flexibility) will when properly implemented in novel neural network algorithms result in highly-robust AI perceptual systems of the future. We have recently demonstrated first payoffs of this strategy for AI vision systems (19).Research problem: Current state-of-the-art AI systems (particular artificial neural network algorithms) for visual inference tasks such as object recognition only partially incorporate primate brain image processing circuit motifs and strategies. Thus, such algorithms fail to capture the complex and deeply behaviorally-relevant dynamics of the underlying neural responses. Not surprisingly, these current AI vision systems, while much improved for tasks on static images, are still highly brittle relative to human vision. The performance gap is substantial with even simple changes like novel backgrounds or novel viewpoints.Objectives and technical approaches: Our overall technical objective is to expose the functions of currently untapped brain circuit motifs that likely enable the robust visual inference abilities of primates and implement those in novel neural network algorithms for AI vision tasks.Objective 1. Using the non-human primate experimental model system, we will deploy neural circuit silencing, large-scale neural population recording at high temporal resolution, and high-throughput behavioral testing to expose the importance of a set of circuit motifs in the primate visual system (esp. recurrence and feedback) that have not yet been incorporated into AI vision systems.Objective 2. Incorporating fine grained experimental results from Objective 1, as well as results from other neuroscience and psychophysical experiments, we will create a comprehensive set of quantitative benchmarks (the "BrainScore" framework) for model-to-brain comparison against which any candidate neural network visual inference algorithm can be evaluated.Objective 3. Using scientific insights from Objective 1 experiments, as well as knowledge from other recent neuroscience and cognitive science results, we will engineer a suite of new neural network models of the primate visual system, and select among them using the quantitative metrics assembled in Objective 2.Objective 4. We will evaluate the capabilities of the biologically-validated neural network algorithms from Objective 3, on a variety of cutting-edge computer vision benchmarks, especially on metrics that assess human-like robustness in situations where current AI vision systems are very brittle.Objective 5. We will develop new theory to bridge between recurrent neural network algorithms, and compositional models.Expected outcomes and DoD impact if successful:1) Entirely new scientific understanding of which brain circuits are important for which kind of visual object inference tasks and image challenges (occlusion, uncommon object pose, etc.).2) New brain benchmarks that will provide for the first time fine-grained brain guidance for AI vision system develhe performance, learning efficiency, and robustness of those networks beyond current state-of-the-art. Each has high DoD relevance for autonomous AI systems.4) New human-behavior based system performance evaluation measures and our ability to develop new algorithen recurrent neural networks and compositional models will show how to extend the engineered network algorithms above to other DoD-relevant AI problems beyond those of visual inference, e.g., to robotic models that take actions.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Aug 31, 2020
- Source ID
- N000142012589
Entities
People
- James J. DiCarlo
Organizations
- Massachusetts Institute of Technology
- Office of Naval Research
- United States Navy