1.c.i.2: Learning Deep AND-OR Grammar Networks for Object Tracking, Detection and Parsing: a Unified Framework
Abstract
The objective of this project is to develop a unified framework of learning deep AND-OR Grammar networks (AOGNets) for object Tracking, Detection and Parsing (TDP), which has the following capabilities: (i) Harness both the explainable rigor of top-down AND-OR grammar and the discriminative power of bottom-up deep feature extractors in an end-to-end learning framework, thus learn actionable information for object tracking-by-detection-and-parsing and object detection-and-parsing-by-tracking. (ii) Address the inherent data-scarce challenge rigorously in online object TDP (e.g., only one shot with ground-truth label available in learning). (iii) Tackle learning-to-learn to scale up both object tracking-by-detection-and-parsing and object detection-and-parsing-by-tracking in an effective and efficient way, thus showcase continual learning without catastrophic forgetting. The proposed work can be divided in three main components under a unified framework. First, we study deep AOGNets at both deep feature extraction level and interpretable prediction level. Second, we study robust continual learning framework which schedules models learned in individual videos (i.e., specific AOGNets for object instances) for learning generic AOGNets for object detection and fine-grained parsing of object-part-subpart at category level and vice verse (i.e., bi-directional scheduling-for-learning in TDP). Third, we study decision policies of adaptively optimizing the structure of deep AOGNets to handle both TDP of unknown objects in individual streaming videos (data-scarce tasks) and TDP in large scale offline video datasets (data rich tasks) and to balance accuracy and efficiency. The proposed work has immediate impacts on a range of DoD missions, including, but not limited to, persistent security video surveillance, and actionable information gathering and retrieval.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Feb 25, 2019
- Source ID
- W911NF1810295
Entities
People
- Tianfu Wu
Organizations
- Army Contracting Command
- North Carolina State University
- United States Army