A Sparse Coding Approach to Spatiotemporal Saliency Detection in Image and Video Databases
Abstract
The wealth, ubiquity and volume of image and video data collected by the U.S. DoD demand new approaches to searching and analyzing image and video databases. In contrast to ineffective global image and video methods, the proposed project uses object-based features of super-pixels from a crude but fast over-segmentation of the images/videos. Then, a sparse code is learned from the feature sets, which enables rapid and robust similarity computation. The resultant similarity measures can, of course, be used for classification and recognition. Moreover, the sparsity based similarity measures can be employed to detect salient objects and events in both space and time. This salience approach serves to triage important objects and events for higher level processing in tasks such as the segmentation of salient objects, scene change detection, object classification and content-based retrieval from a database. The project posits that segmentation based on saliency brings semantic value over conventional generalized segmentation approaches. Such extracted salient objects can be then used in important analysis tasks such as content-based image/video retrieval and event detection. The core ideas presented in this proposal build on recent developments in compression based similarity measurement, salience detection, dictionary learning and feature self-nomination. This research has three specific aims. First, sparse coding similarity assessment for images and image regions will be developed by combining a multi-scale super-pixel representation with dictionary learning. Second, the project team will introduce a saliency detection framework based on sparse coding. Third, the theoretical framework will be applied to perform image segmentation, image/video retrieval and salient event detection in video. The theoretical framework and corresponding software will be developed in Year One, along with demonstration of saliency in content-based image/video retrieval. Spatiotemporal salience detection and salience based segmentation will be pursued and delivered in Year Two. In Year Three, the focus will be on video databases and on validation of the entire approach using real data provided by U.S. Army collaborators.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Apr 22, 2019
- Source ID
- W911NF1510275
Entities
People
- Scott T. Acton
Organizations
- Army Contracting Command
- United States Army
- University of Virginia