Brain inspired neural computation of structured knowledge
Abstract
Approved for Public ReleaseDeep neural networks have revolutionized Artificial Intelligence (AI) and Machine Learning (ML) with unpr,ecedented applications in vision, signal processing, medical diagnostics, and Natural Language Processing. Nevertheless, current AI,and ML systems lag far behind cognition in humans and other animals. This motivates the exploration of new brain inspired architectu,res and dynamic motifs as candidates for novel AI systems with substantially enhanced capabilities. In this project, we focus on und,erstanding neural computations that are rich in contextual, relational, and compositional structures, which underlie reasoning, robu,st generalization, multi-tasking, and life-long knowledge acquisition. How can distributed neural representations and data-driven le,arning accomplish such feats without explicit manipulations of symbols and rules? What insights can we gain from the anatomy and bio,physics of biological networks towards building robust and flexible artificial cognitive systems? This project will focus on two key, circuit motifs: gating and binding. Signals from multi-modal task, attention, or long-term memory buffers, can drive inhibitory neu,rons to switch on or off (gate) the processing of ongoing stimuli elsewhere in the brain. Gating neurons also can enable continual l,earning and alleviate cross-task interference. Binding operations via multiplication and aggregation can compress complex mental ma,ps or whole sentences into a fixed group of neurons.Which gating or binding architectures are best suited for processing struc,heoretical studies by the group of PI Haim Sompolinsky (Harvard University) on the role of gating in generalization, multi-tasking,,and continual learning, to build candidate models of deep gating networks (Objective 1). Recent theoretical work by the same group,on binding networkswill be expanded to quantify the capacity of binding networks to store and retrieve structured knowledge in worki,ng and long-term memory (Objective 2). Recent years have witnessed an explosive expansion of high-quality data on the anatomy and bi,ophysics of neurons and circuits in cortex, the part of the brain that is the site of high cognitive functions in mammals and humans,. Detailed biophysical models of cortical neurons and circuits of both rodents and humans have been developed over the years by the,group of Co-PI Idan Segev (Hebrew University). These studies highlighted the nonlinear interactions between synaptic inputs, includi,els to incorporate the recently discovered rich subnetworks of different types of inhibitory neurons, targeting different parts of t,he dendritic tree of pyramidal neurons; these neurons are the main outputs of cortical circuits (Objective 3). We will use this know,ledge to build novel circuit motifs with hierarchical gating motifs. Furthermore, inhibitory networks are recurrently connected, giv,ing rise to rich gating dynamics; this is yet another novel gating motif that we will embed in our deep gating networks. Other types, of nonlinearities at the single cell level will be explored to build biophysically inspired large-scale binding networks. Finally,,we will apply our theoretically and biologically informed models to contemporary large-scale benchmark AI challenges, particularly i,n the domains of multi-tasking and continual learning (Objective 4). By elucidating biological motifs in cortical circuits and build,ing theoretical foundations of complex deep networks, this ambitious project will narrow the gap between AI functionality and robust, and flexible human reasoning and learning capabilities.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Dec 06, 2022
- Source ID
- N000142312051
Entities
People
- Haim Sompolinsky
Organizations
- Office of Naval Research
- President and Fellows of Harvard College
- United States Navy