Optimal Bayesian Approximations for Targeted Prediction
Abstract
There are two fundamental challenges associated with prediction problems: predictions should be accurate and interpretable. Predictive models that are both precise and understandable can provide new insights into scientific processes and deliver valuable information to decision makers. Bayesian models are particularly powerful tools for this endeavor: these models can aggregate multiple data sources, accommodate a variety of dependence structures, and provide posterior predictive uncertainty quantification. However, Bayesian models are often highly complex and computationally intensive, which limits their interpretability and capability for real-time decision making under uncertainty. Scientific objectives: The goal of this proposal is to design a framework for constructing, computing, and evaluating simple yet accurate predictive approximations to Bayesian models. Most importantly, these approximations will be calibrated and optimized for specific decision tasks. Decision analysis commonly features functionals of the data, which fundamentally impact the overarching prediction problem. For example, modern monitoring systems, including wearable devices, record massive quantities of data at near-continuous resolutions. Scientific analysis and decision making typically requires filtering and extracting specific features, such as rates of change or peak activity. By targeting the predictive approximations, there is an opportunity to (1) obtain predictive models that substantially reduce complexity, (2) select models and variables customized to the task at hand, and (3) discard extraneous information to reduce storage costs and streamline the analysis. An accompanying objective is to provide uncertainty quantification for all predictions, comparisons, and parameters, which lends interpretability to each stage of the analysis. Methods: For any predictive Bayesian model, approximations will be obtained by minimizing an expected loss function that is designed to balance predictive accuracy with model simplicity. Point and interval approximations will target predictive functionals of interest while leveraging the full posterior distribution conditional on all the observed data. Computational simplifications of this objective will be derived, which allows for rapid and parallelizable computation of optimal approximations for a broad class of models. Accompanying metrics for out-of-sample evaluations and comparisons will be developed, and importantly, will include posterior uncertainty quantification. Building upon these out-of-sample comparisons, new techniques for model and variable selection will be designed to identify approximations that match or exceed the predictive accuracy of the full model. For interpretability, uncertainty quantification will be provided for all approximating model parameters based on the original Bayesian model posterior, which notably avoids the need for intensive model re-fitting. Significance: The core focus of this proposal is targeted prediction, where point and interval predictions are optimized for the functionals needed for scientific analysis and decision making. Preliminary results demonstrate that these approximations not only match the predictive accuracy of state-of-the-art Bayesian models, but also offer clear improvements. The ability to achieve accurate predictions from simple models---while improving computational scalability and retaining full uncertainty quantification---would be an essential tool for Bayesian analysis of scientific data. The synthesis of (1) fast approximations and (2) out-of-sample comparisons for point and interval predictions provides enhanced capabilities for model evaluation and selection. By focusing the approximations on key decision tasks, the proposed approach taps into an unrealized potential of Bayesian predictive modeling and establishes new mechanisms for extracting meaningful information from the posterior distribution.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jul 09, 2020
- Source ID
- W911NF2010184
Entities
People
- Daniel R Kowal
Organizations
- Army Contracting Command
- Rice University
- United States Army