Decoding multisensory information at different spatiotemporal scales of neuronal resolution
Abstract
Despite a considerable literature that have investigated these issues, they have yet to be fully resolved. We are uniquely poised to make significant inroads into these issues through three conceptual advantages. First, we will decode neuronal signals that arise from behavioral tasks that test multisensory perception. Second, we record mesoscopic-level spiking activity (and, hence, multi-unit activity and local-field potentials) from multiple brain areas, while simultaneously recording EEG activity from multiple locations on the skull surface. Third, we decode neuronal signals using modern, state-of-the-art machine-learning techniques. The overall Aim of this project is to identify which combinations of neuronal signals at which spatiotemporal resolution optimize decoding performance. We address this Aim by simultaneously collecting large-scale neuronal recordings at different spatiotemporal scales of neuronal activity (e.g., spiking activity, local-field potentials, and surface EEG signals) while monkeys participate in a multisensory task and then decoding task-related information using state- of-the-art machine-learning techniques. In particular, we have 4 deliverables. (1) We will identify which combinations of neuronal signals are synergistic, resulting in improved decoding performance In particular, we will identify how incorporating neuronal information at smaller spatiotemporal scales of resolution (e.g., single-neuronal spiking activity) improves the decoding of larger spatiotemporal scale of neuronal signaling (i.e., EEG). Conversely, we will identify how the availability of larger spatiotemporal signals improves decoding at smaller spatiotemporal scales of neuronal information processing. (2) As behavioral trials evolve over time, we determine whether certain neuronal signals allow us to decode information earlier than other types of neuronal signals. Fast responses matter for most real-world tasks. (3) We identify the combination of brain regions and the locations of surface EEG potentials that are the most effective in the decoding of components of a multisensory task. (4) Lastly, we will analyze how all these factors affect latency, a largely ignored dimension of high importance. Individually and collectively, the questions addressed in this proposal provide quantitative insights into the degree to which modern machine-learning techniques can decode complex multisensory-related behavior from neuronal signals at different spatiotemporal scales.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Oct 01, 2019
- Source ID
- W911NF1910503
Entities
People
- Yale Cohen
Organizations
- Army Contracting Command
- United States Army
- University of Pennsylvania