Learning continuous chaotic attractors with a reservoir computer
Abstract
Neural systems are well known for their ability to learn and store information as memories. Even more impressive is their ability to abstract these memories to create complex internal representations, enabling advanced functions such as the spatial manipulation of mental representations. While recurrent neural networks (RNNs) are capable of representing complex information, the exact mechanisms of how dynamical neural systems perform abstraction are still not well-understood, thereby hindering the development of more advanced functions. Here, we train a 1000-neuron RNN—a reservoir computer (RC)—to abstract a continuous dynamical attractor memory from isolated examples of dynamical attractor memories. Furthermore, we explain the abstraction mechanism with a new theory. By training the RC on isolated and shifted examples of either stable limit cycles or chaotic Lorenz attractors, the RC learns a continuum of attractors as quantified by an extra Lyapunov exponent equal to zero. We propose a theoretical mechanism of this abstraction by combining ideas from differentiable generalized synchronization and feedback dynamics. Our results quantify abstraction in simple neural systems, enabling us to design artificial RNNs for abstraction and leading us toward a neural basis of abstraction.
Document Details
- Document Type
- Pub Defense Publication
- Publication Date
- Jan 01, 2022
- Source ID
- 10.1063/5.0075572
Entities
People
- Danielle Bassett
- Jason Z Kim
- Lindsay M. Smith
- Zhixin Lu
Organizations
- Army Research Office
- National Science Foundation
- Office of Extramural Research
- Paul G. Allen Family Foundation
- Santa Fe Institute
- University of Pennsylvania