Learning and Explaining Information Dynamics for Overhead Imagery
Abstract
While image and video understanding from consumer type of cameras and ordinary scenes such as indoor and street views has seen incredible progress in recent years, this is not yet the case for overhead imagery. The goal of this project is to develop a computational framework for information extraction from overhead images, going from drones to satellites and their combination, including the integration with ground images such as those coming from street cameras for example. In particular, a major goal of this project is to go from individual object detection to objects relationships to change and outlier detection and description with natural language, being these last challenges also in very early development stages for consumer video. In consumer type image and video understanding, progress had to be made in all the steps of the pipeline in order to be able to go from the input image/video to outputs common today such as activity recognition and preliminary natural language scene description (summarization, captioning, etc). These include image/video pre-processing (denoising, enhancement, registration, etc), object recognition and localization, objects interactions, change detections, etc. While the job is not yet completed for consumer images, significant advances have been made, but all these steps are still in their infancy when considering overhead imagery. The goal of this project is to adapt and extend tools we and others have been developing to bridge the performance gap between consumer and overhead imagery analysis, while at the same time advancing the state-of-the-art in consumer imaging itself. In order to achieve these goals we have to bring together expertise at all layers of the pipeline, from low-level overhead imagery to machine learning. The project is further enhanced by the extensive ongoing collaboration with NGA.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Oct 06, 2020
- Source ID
- HM04761912010
Entities
People
- Guillermo Sapiro
Organizations
- Duke University
- National Geospatial-Intelligence Agency