Informative priors for Bayesian inference and regularization

Abstract

Informative priors for Bayesian inference and regularization (Proposal for ONR Mathematical Data Science Program) Andrew Gelman, Columbia University Abstract We propose to develop practical tools for data analysis using Bayesian inference by providing guidelines for the use of informative priors. At the theoretical level, we propose to unify the choice of prior distribution by placing noninformative and weakly informative priors into a common framework of hierarchical modeling. We propose to {em implement} these models in Stan, our open-source C++ program for Bayesian inference which uses the no-U-turn sampler and Hamiltonian Monte Carlo to efficiently explore high-dimensional posterior distributions (or, alternatively, uses an efficient optimization algorithm to find posterior modes or penalized maximum likelihood estimates). For applications, we propose to fit our models} in ongoing research efforts in survey sampling, social networks, and public health, among others. This work is relevant to several focus areas of the ONR Mathematical Data Science Program, including adjustment for big data, regularization for small data, complex networks, and multi-modal information aggregation. Another important application for this work involves the current crisis in science, which is sometimes called the replication crisis in psychology, that various well-publicized claims, supported by published papers with statistically significant p-values, to not stand up under attempted replication. One way to understand this is by considering these studies as weak-data, strong-prior scenarios.

Document Details

Document Type
DoD Grant Award
Publication Date
Aug 12, 2016
Source ID
N000141512541

Entities

People

  • Andrew Gelman

Organizations

  • Office of Naval Research
  • Trustees of Columbia University in the City of New York
  • United States Navy

Tags

Fields of Study

  • Computer science

Readers

  • Distributed Systems and Data Platform Development
  • Neural Network Machine Learning.
  • Statistical inference.

Technology Areas

  • AI & ML
  • AI & ML - Bayesian Inference
  • AI & ML - Machine Learning Algorithms