THIS IS A CONTINUATION OF N00014-14-1-0679 Feedback-Enabled Joint Reasoning over Uncertain Sub-components of Perc
Abstract
Statement of Work:Develop reliable visual perception by integrating ambiguous information from visual sub-components and evaluate these methods on realistic datasets for image/video understanding.Objective:Develop methods for efficient integration of ambiguous results from visual sub-components for building reliable visual perception.Approach:A key difficulty in visual perception (image/video understanding) is ambiguity because images are underspecified. This may be because perception sub-components are ambiguous, namely, the scene context is unknown, segmentations are incorrect, object boundaries are unclear, etc. The PI proposes to build a perception system that jointly reasons about different sub-components or modules of perception such as 3D scene layout, object layout, language parsing,etc., to extract a holistic view of the world. The expectation is that such systems can improve performance by avoiding nonsensical mistakes, and also establish operator trust by providing an interpretable summary of what they believe in terms of different modules. A key challenge to joint reasoning is state-space explosion. It is infeasible to search overthe joint space of {all-3D-layouts} ? {all-segmentations} ? . . . ? {all-sentence parses}. To overcome this challenge, they propose to extract and leverage a small set of diverse plausible hypotheses from perception modules to use for jointreasoning. The main thesis of this research program is that a small set of plausible hypotheses can serve as a concise interpretable summary of uncertainty in perception modules (What does this module believe about the world?) and form the basis for tractable joint reasoning (How do we reconcile what module A believes about the world with what module B believes?). This work is expected to develop novel and efficient methods for learning plausible hypotheses, efficient methods for joint learning and reasoning in the CRF framework, among other contributions.Overall Merit and ONR Mission/Relevance:This research is expected to develop novel methods that incorporate feedback among different visual/perceptual subcomponents (modules) to reason about ambiguities and overcome them for better scene understanding.This research addresses Information Dominance and Autonomy focus areas. This work is expected to develop robust perception for autonomous agents and for image/video analysis.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Sep 23, 2016
- Source ID
- N000141612793
Entities
People
- Dhruv Batra
Organizations
- Office of Naval Research
- United States Navy
- Virginia Tech