Brain mechanism of sequentially organized behavior: predictive perception and compositional action p
Abstract
How does our brain predict upcoming sensory events (e.g., what will your interlocutor say next in a conversation?) and plan for futu,re actions (e.g., how to respond in return)? Such internal and proactive processes cannot be accomplished by purely feedforward neur,al networks (a commonly used architecture of today?s AI systems); they depend on recurrent neural circuit dynamics and top-down sign,aling in the brain. In this grant application, we propose to undertake research with the aim to elucidate the neural circuit mechan,isms and computational principles of predictive and temporally extended behaviors, in both the sensory domain (predictive coding in,perception of a continuous input stream) and the motor domain (planning and execution of action sequences). Our computational work,will be carried out in close collaborations with the monkey neurophysiological laboratories of David Freedman and Winrich Freiwald (,see their support letters). Our collaborators train monkeys to perform novel temporally extended tasks; and they use the start-of-th,e-art technology for simultaneously recording from many neurons in multiple cortical areas, providing data for our computational mod,eling and testing model predictions. The Freedman lab investigates how the brain proactively makes inference about the sensory world,eiwald lab focuses on the brain s process of internally generating actions that are not merely dictated by the external inputs, whic,h in humans is exemplified by language: we can give a speech or write a story on the basis of semantic and syntactic knowledge as we,ll as creativity. Specifically, Aims 1-2 are dedicated to developed a detailed neural circuit model for predictive coding and its p,hysiological marker called mismatch negativity (MMN), in response to a series of visual motion stimuli with different directions. In, both monkey experiments in the Freedman lab and model simulations, we will measure MMN at the time when an unexpected stimulus is p,resented, dissect the underlying circuit mechanism. Crucially, we will identify specific ways to identify top-down prediction signal,, elucidate functional benefits of predictive coding, and extend the ,ortex for hierarchical predictive coding. Importantly, our work will probe predictions based on categories (rather than sensory feat,erform a drawing task used in the monkey experiments in the Freiwald lab. In this task, a subject learns categories of action (e.g.,, a vertical line, a circle), produces a motor sequence according to syntactic rules for combining and ordering symbols (e.g., drawin,g two lines, then a circle). We are particularly interested in the emergence and neural circuitmechanism of symbols (action categori,es), compositionality (for combining motor primitives to generate complex sequences) and syntax (?grammar of action?). Methodologica,lly, the proposed theoretical research is built on our advances during the last grant cycle in connectome-based modeling of the mult,iregional primate neurocortex, and using recurrent neural networks (RNNs) trained by machine learning tools to help us design neural, circuit models for performing increasingly complex cognitive tasks.Advances in the proposed research will identify new computationa,l principles for the development of deep recurrent neural network models that are capable of active prediction of the sensory world,, planning and generation of compositional action sequences. Insights gained from our neuroscientific basic research will potentially, inspire the design of new types of intelligent machines, of wide naval relevance.Approved for Public Release.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Dec 06, 2022
- Source ID
- N000142312040
Entities
People
- Xiao-Jing Wang
Organizations
- New York University
- Office of Naval Research
- United States Navy