Sparse Distributed Representation and Hierarchy: Keys to Scalable Machine Intelligence
Abstract
We developed and tested a cortically-inspired model of spatiotemporal pattern learning and recognition called Sparsey. Sparsey is a hierarchical model allowing an arbitrary number of levels consisting of coding modules that code information, specifically particular spatiotemporal input moments using sparse distributed representations (SDRs). The modules are called "macs" as they are proposed as analogs of the canonical cortical processing module known as macrocolumns. Sparsey differs from mainstream neural models, e.g., Deep Learning, in many ways including: a) it uses single-trial, Hebbian learning rather than incremental, many-trial, gradient-based learning; and b) it multiplicatively combines bottom-up, top-down, and horizontal, evidence at every unit (neuron) in every mac at every level on every time step during learning and inference (retrieval). However, Sparseys greatest distinguishing characteristic is that it does both learning (storage) and retrieval of the best matching stored input in time that remains constant regardless of how many patterns (how much information) has been stored. Thus, it has excellent scaling potential to "Big Data"-sized problems. We conducted numerous studies establishing basic properties and capacities, culminating in demonstration of 67% classification accuracy on the Weizmann data set, accomplished with 3.5 minutes training time, with no machine parallelism and almost no software optimization.
Document Details
- Document Type
- Technical Report
- Publication Date
- Apr 01, 2016
- Accession Number
- AD1006958
Entities
People
- Gerard Rinkus
- Greg Lesher
- Jasmin Leveille
- Oliver W Layton