Factorial Hidden Markov Models.

Abstract

We present a framework for learning in hidden Markov models with distributed state representations. Within this framework, we derive a learning algorithm based on the Expectation-Maximization (EM) procedure for maximum likelihood estimation. Analogous to the standard Baum-Welch update rules, the M-step of our algorithm is exact and can be solved analytically. However, due to the combinatorial nature of the hidden state representation, the exact E-step is intractable. A simple and tractable mean field approximation is derived. Empirical results on a set of problems suggest that both the mean field approximation and Gibbs sampling are viable alternatives to the computationally expensive exact algorithm.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jan 01, 1996
Accession Number
ADA307097

Entities

People

  • Michael I. Jordan
  • Zoubin Ghahramani

Organizations

  • Massachusetts Institute of Technology

Tags

Communities of Interest

  • Autonomy
  • Energy and Power Technologies

DTIC Thesaurus Topics

  • Algorithms
  • Artificial Intelligence
  • Cognitive Science
  • Data Science
  • Hidden Markov Models
  • Information Processing
  • Information Science
  • Information Systems
  • Markov Models
  • Models
  • Monte Carlo Method
  • Neural Networks
  • Probability
  • Probability Distributions
  • Random Variables
  • Sampling
  • Signal Processing

Fields of Study

  • Computer science

Readers

  • Computational Modeling and Simulation
  • Mathematical Modeling and Probability Theory.
  • Statistical inference.