Information Storage Capacity of Connectionist Systems: The Linear Associator

Abstract

The information-storage capacity of hetero-associative memory systems is addressed. The associator can be treated as an M-ary symmetric channel when M associations are stored. The maximum number of associations storable is bounded asymptotically by N/2 where N is the number of connection weights. Storage efficiency is bounded by M/N so that it never exceeds 1/2. Information capacity degrades as inter-vector correlations increase and also when classification tasks are performed. The correlation effect is most pronounced in high- dimensional systems storing a large number of associations. Keywords: Artificial intelligence, Connectionism, Information theory.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Sep 29, 1987
Accession Number
ADA219001

Entities

People

  • Dean C. Mumme
  • Walter Schneider

Organizations

  • Carnegie Mellon University

Tags

Communities of Interest

  • Air Platforms

DTIC Thesaurus Topics

  • Algorithms
  • Artificial Intelligence
  • Cognitive Science
  • Computations
  • Computer Programming
  • Computer Science
  • Computers
  • Content Addressable Memory
  • Information Processing
  • Information Science
  • Information Theory
  • Mathematical Analysis
  • Neural Networks
  • Numerical Analysis
  • Probability
  • Psychology
  • Random Variables

Readers

  • Graph Algorithms and Convex Optimization.
  • Neural Network Machine Learning.
  • Radio communications and signal processing.

Technology Areas

  • AI & ML
  • AI & ML - DoD AI Strategy
  • AI & ML - Machine Learning Algorithms