Building a Multimodal Imaging System to Support Multimodal Data Fusion via Deep learning
Abstract
As sensing and computing technologies rapidly advance, we are facing two unprecedented opportunities for solving many long-standing open problems in machine vision and pattern recognition: big data and data-driven. Big data refers to the acquisition of multimodal (a.k.a. heterogeneous) data sources from a variety of sensors (e.g., multispectral and 3D LIDAR); data-driven refers to the use of deep neural networks for various visual inference tasks from detection to recognition. Connecting these two latest trends has the potential of revolutionizing the practice of reconnaissance and surveillance. In this project, we propose to build a multimodal imaging system to collect both multispectral and 3D depth information of a physical scene. A novel deep learning based approach will be developed to support the fusion of multimodal data. The proposed instrumentation will be used to collect the first comprehensive multimodal data set. Such data set will be complementary to the existing KITTI data set emphasizing on moving objects in outdoor environments and expected to better support deep learning based approach toward multimodal data fusion. The learning-based approach toward multimodal data fusion has a wide of applications in both military and civilian domains (e.g., night vision, persistent surveillance and intelligent vehicles).
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Feb 14, 2019
- Source ID
- W911NF1810219
Entities
People
- Xin Li
Organizations
- Army Contracting Command
- United States Army
- West Virginia University