Calibrated uncertainty quantification in statistical learning
Abstract
The proposed research focuses on developing an extremely broad class of fundamentally novel methods for characterizing uncertainty in conducting inferences from data. It is routine to use statistical learning tools to infer low-dimensional structure and to estimate parameters governing the data generating process without providing any characterization of uncertainty. In many applications, including those of direct relevance to the Navy, there is substantial uncertainty and it is critical to accurately characterize this uncertainty in interpreting results of statistical learning algorithms and in making decisions based on these results. Although there are existing tools for uncertainty quantification (UQ), broad classes of algorithms used routinely in practice lack approaches for UQ, while existing methods for UQ tend to be highly sensitive to modelingassumptions, computationally very expensive and/or provide unrealistic characterizations of uncertainty. With this motivation, this proposal focuses on generalizing Bayesian inferences to bypass the above problems and provide a broad framework for characterizing uncertainty in statistical learning problems. This will be accomplished through developing two alternative generalized Bayes (G-Bayes) frameworks for computationally efficient, robust and calibrated UQ: (1) Conformal Bayes methods that calibrate an initial posterior to obtain accurate out-of-sample predictive distributions; (2) Gibbs posteriors for loss-based updating of beliefs bypassing the need to specify likelihood functions and prior distributions for parameters in these likelihoods. In eachof these objectives, fundamentally new classes of algorithms are proposed for UQ in extremely broad settings. Strong theoretical guarantees will be provided for these algorithms, and a general toolbox will be developed that can be implemented routinely by practitioners in broad fields to add UQ to canonical statistical learning algorithms ranging from k-means to PCA. This toolbox provides a significant enhancement of algorithms already used routinely by the Navy and in broad fields of the government, academics and industry.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jun 09, 2021
- Source ID
- N000142112510
Entities
People
- David B. Dunson
Organizations
- Duke University
- Office of Naval Research
- United States Navy