Target Detection/Tracking and Activity Recognition from Multimodal Data
Abstract
Research Problem and Technical Approaches: Our primary objectives in this proposal are to develop operational target detection/tracking techniques and activity recognition/understanding methods that possess the capability to leverage multimodal data in a fusion framework using deep learning algorithms and coupled tensor techniques while providing accurate and near real-time performance. The proposed methodology is divided into four major stages: 1) pre-processing, 2) co-registration and fusion, 3) target detection and tracking, and 4) activity recognition and scene understanding. The aforementioned algorithms will be benchmarked against existing state of the art techniques to highlight their advantages and distinguish their abilities. The above is intended to assist analysts to effectively and efficiently ingest large volumes of data, or perform object detection and tracking in real-time under dynamic settings. Anticipated Outcomes: We anticipate that the above techniques would yield the following results: (1) accurately ingest multimodal data sets acquired through various instruments and sensors, (2) provide the proper pre-processing (segmentation, registration, enhancement, motion estimation, noise filtering, etc.) to correctly register the acquired information indicating areas of potential errors that may need to be addressed by human intervention, (3) deliver the proper mechanisms for fusion of all acquired data, (4) perform target detection, identification, and tracking on the imagery yielding labeled tracks for various objects of interest, and 5) yield scene understanding by predicting potential activities in a prioritized probabilistic framework. The above outcomes are intended to enhance the analyst’s ability to evaluate large quantities of streaming content. Impact on NGA Capabilities: This proposal advances new methodologies for target tracking and activity recognition that possess the capability to leverage multimodal data in a fusion framework using deep learning algorithms. The outcomes will be significant to NGA as follows: 1) target detection and tracking, 2) activity recognition and understanding, 3) ability to selectively fuse modalities as needed to enhance the outcome, and 4) widespread military and geospatial intelligence applications.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Oct 06, 2020
- Source ID
- HM04761912014
Entities
People
- Eli S. Saber
Organizations
- National Geospatial-Intelligence Agency
- Rochester Institute of Technology