Training biophysical thalamocortical models to play games through biologically realistic reinforcement learning rules
Abstract
Recent technological breakthroughs enable neuroscientists to obtain abundant recordings of multiscale brain activity. Although the new techniques produce vast amounts of data, theoretical breakthroughs in understanding decision-making and behavior are lagging due to the lack of a coherent framework that addresses how adaptive dynamics in complex neuronal circuits contribute to perception, behavior, and cognition. Two fields of research -- artificial intelligence (AI) and computational neuroscience -- that in recent history have only marginally influenced each other, could both be revolutionized by being brought together to understand the brainÕs control of behavior. The recent AI developments in deep learning enable abstract neural networks to perform well in game playing and autonomous vehicle control, mimicking certain aspects of human decision making and behavior. However, due to the abstracting away of biological detail, the AI approaches provide little insight into actual brain mechanisms underlying the choices made. We propose that integrating bottom-up data-constrained biophysically realistic neural modeling and top-down AI-style modeling can deepen our understanding of the brainÕs mechanisms of decision making and its dynamic control of behavior. Our research goals are to interface our biologically-detailed thalamocortical models (thalamus, visual cortex, motor cortex) in a closed loop design with different video game environments. The integrated models will be trained to both sense and learn to appropriately respond to time-varying, behaviorally-relevant simulated visual stimuli. We will train our models to play video games through known biologically-realistic learning algorithms based on the dopaminergic reward system. We will train our models through a two-stage sequence consisting of 1) unsupervised learning through spike-timing-dependent plasticity (STDP) of visual areas followed by 2) STDP-dependent reinforcement learning within motor areas. This sequence will allow our models to first learn visual representations and then to adjust synaptic weights in order to maximize reward-seeking behavior. We will compare our modelÕs performance (training duration and efficacy) against commonly used deep reinforcement learning algorithms. Our biologically-detailed model will allow us to test the critical neural components and circuit architectures contributing to effective behavior, through modulating the level of biological detail, and architectural complexity employed. Our neurobiologically-derived models will generate data directly comparable to in vivo experiment data (e.g. EEG), enabling us to ultimately test model predictions that could uncover the mechanistic origins of the human brainÕs complex dynamics underlying perception, decision making, and behavior. Our integrated model represents a foundational step in linking AI approaches and realistic brain dynamics, and could enable understanding pathological dynamics in neurological and psychiatric decision making disorders such as ADD/ADHD, OCD, and schizophrenia.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jul 24, 2019
- Source ID
- W911NF1910402
Entities
People
- Samuel A. Neymotin
Organizations
- Army Contracting Command
- Research Foundation For Mental Hygiene
- United States Army