Spatial Semantic Representations for Interaction and Explanation
Abstract
Approved for Public ReleaseThe project focuses on representations of sensory data in support of spatial interaction tasks. Such repr,esentations can be obtained via inductive learning, by processing a fixed dataset of past data and performing feed-forward computati,on at inference time. However, where the error signal is available at inference time, one could instead solve an optimization proble,m and forgo learning altogether. Much of the recent literature in deep learning has focused in inductive inference, where deep netwo,rks have long been the standard for supervised classifications. However, such methods have failed on spatial inference, where most c,ritical inference problems such as localization are still performed using conventional processing pipelines. Furthermore, inductive,learning presents fundamental limitations in assessing confidence, or uncertainty, of the outcome of inference.This proposal aims to, make progress by first developing methods to define and compute uncertainty in an inductively trained state-of-the-art deep neural,network model. Existing methods either focus on uncertainty of the model, as opposed to the outcome of inference, which is of limite,d value in practice, or rely on training methods or inference procedures that are not usable in current large-scale applications. In,deed, even defining uncertainty is non-trivial, and involves counterfactual hypotheses with connections to causal analysis.Where inf,erence-time optimization is possible, learning is not necessary but may be beneficial. This proposal aims to bridge the gap between,purely optimization-based inference and learning. The former are employed in the traditional spatial processing pipeline: sparse fea,ture detection, invariant description, putative correspondence, combinatorial matching, robust data association, epipolar geoemtry e,stimation, bundle adjustment, densification. The latter are employed in semantic inference customary in supervised classification. I,n between, we explore transductive learning where some of the process required for test-time optimization is performed at training t,ime, thus reducing latency and test-time complexity, but some residual optimization is possible at test time, enabling uncertainty q,uantification and adaptivity to distributional shifts in a lifelong setting.The program is articulated into 4 goals, with milestones, distributed across three years.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Apr 01, 2022
- Source ID
- N000142212252
Entities
People
- Stefano Soatto
Organizations
- Office of Naval Research
- United States Navy
- University of California, Los Angeles