Sparse, recurrent, lateral-inhibition networks build rich structured representations
Abstract
Several widely accepted properties of brain circuitry are notably absent from artificial neural network (ANN) systems. Either theyare simply biological characteristics with no important computational consequences, or else, adding them into ANNs will add computational powers. We introduce a model (SPRALL) incorporating four such biological properties: i) very sparse connections among cells;ii) low-precision synaptic connections; iii) recurrent and feedforward-feedback circuitry; iv) prominent local-circuit lateral inhibition. We propose that these result in a set of unanticipated mathematical outcomes that enable performing massively parallel operations with notably distinct characteristics from current ANNs. We propose that the networks are interpretable in terms of logic gates whose operations are carried out via Hamiltonians, with minimum-energy states achievable via adiabatic annealing. The resultingnetworks construct rich internal representations, shown to be equivalent toindexed grammars. It is proposed that empirically, theywill compete with current transformer architectures (e.g., GPT-3), with orders of magnitude less computational cost, scaling linearly to very large data. Initial results indicate these proposed properties. In addition to these pragmatic applications, analyses are proposed to demonstrate that these high-level logic operations are all cast in terms of physiological states of the biologically-based (SPRALL) network. The work constitutes a candidate approach that directly bridges between relatively low level detailed braincircuit operations, and high level symbolic operations, as a new method for addressing the disconnect between perceptual level and cognitive level functions. - Approved for Public Release
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jun 09, 2021
- Source ID
- N000142112290
Entities
People
- Richard Granger
Organizations
- Board of Trustees of Dartmouth College
- Office of Naval Research
- United States Navy