Multimodal Neural Decoding: Data-Intensive Approaches to Understanding Long-Term, Unlabeled Human Brain Data

Abstract

Fully automated decoding of human actions and intentions from neural signals is a major challenge in human-computer interactions. The current success of brain-computer interfaces (BCI) has hinged on labeled training data in laboratory conditions. To deploy BCIs in real-life, one must develop robust strategies to handle naturalistic disturbances. This research effort focused on decoding movements from large-scale human intracranial brain recordings, video, and audio, all continuously acquired over one week. Importantly, here the participants are simply behaving as they wish. This research demonstrated the ability to decode movements from neural recordings as well as that this decoder generalizes remarkably well on application to unseen participants.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
May 25, 2021
Accession Number
AD1134034

Entities

People

  • Bing W. Brunton

Organizations

  • University of Washington

Tags

Communities of Interest

  • Energy and Power Technologies
  • Materials and Manufacturing Processes

DTIC Thesaurus Topics

  • Air Force
  • Air Force Research Laboratories
  • Artificial Intelligence Software
  • Brain-Computer Interfaces
  • Cognitive Systems Engineering
  • Computational Science
  • Computer Programs
  • Computer Vision
  • Computers
  • Convolutional Neural Networks
  • Data Mining
  • Data Processing
  • Data Sets
  • Decoders
  • Decoding
  • Detection
  • Engineering
  • Experimental Design
  • Frequency
  • Health Services
  • Information Science
  • Machine Learning
  • Magnetic Resonance
  • Neural Engineering
  • Neural Networks

Readers

  • Agent-Based Social Robotics and Mobile-Assisted Learning in Virtual Environments.
  • Neural Network Machine Learning.
  • Neuroscience