Gibbs Sampling for the Uninitiated

Abstract

This document is intended for computer scientists who would like to try out a Markov Chain Monte Carlo (MCMC) technique, particularly to do inference with Bayesian models on problems related to text processing. We try to keep theory to the absolute minimum needed, though we work through the details much more explicitly than you usually see even in "introductory" explanations. That means we've attempted to be ridiculously explicit in our exposition and notation. After providing the reasons and reasoning behind Gibbs sampling (and at least nodding our heads in the direction of theory), we work through an example application in detail -- the derivation of a Gibbs sampler for a Naive Bayes model. Along with the example, we discuss some practical implementation issues, including the integrating out of continuous parameters when possible. We conclude with some pointers to literature that we've found to be somewhat more friendly to uninitiated readers.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Apr 01, 2010
Accession Number
ADA523027

Entities

People

  • Eric Hardisty
  • Philip Resnik

Organizations

  • University of Maryland

Tags

Communities of Interest

  • Autonomy
  • Energy and Power Technologies
  • Materials and Manufacturing Processes

DTIC Thesaurus Topics

  • Algorithms
  • Automated Speech Recognition
  • Computational Science
  • Computer Science
  • Data Science
  • Information Science
  • Language
  • Linguistics
  • Markov Chains
  • Models
  • Monte Carlo Method
  • Natural Language Processing
  • Probabilistic Models
  • Probability
  • Probability Distributions
  • Sampling
  • Two Dimensional

Fields of Study

  • Mathematics

Readers

  • Military History of the United States in the 20th Century.
  • Statistical inference.
  • Systems Analysis and Design

Technology Areas

  • AI & ML
  • AI & ML - Bayesian Inference