Convergence Results for the EM Approach to Mixtures of Experts Architectures

Abstract

The Expectation-Maximization (EM) algorithm is an iterative approach to maximum likelihood parameter estimation. Jordan and Jacobs (1993) recently proposed an EM algorithm for the mixture of experts architecture of Jacobs, Jordan, Nowlan and Hinton (1991) and the hierarchical mixture of experts architecture of Jordan and Jacobs (1992). They showed empirically that the EM algorithm for these architectures yields significantly faster convergence than gradient ascent. In the current paper we provide a theoretical analysis of this algorithm. We show that the algorithm can be regarded as a variable metric algorithm with its searching direction having a positive projection on the gradient of the log likelihood. We also analyze the convergence of the algorithm and provide an explicit expression for the convergence rate. In addition, we describe an acceleration technique that yields a significant speedup in simulation experiments.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Nov 01, 1993
Accession Number
ADA276801

Entities

People

  • Lei Xu
  • Michael I. Jordan

Organizations

  • Massachusetts Institute of Technology

Tags

Communities of Interest

  • Counter IED
  • Materials and Manufacturing Processes

DTIC Thesaurus Topics

  • Algorithms
  • Applied Mathematics
  • Artificial Intelligence
  • Cognitive Science
  • Computational Science
  • Convergence
  • Equations
  • Information Processing
  • Information Science
  • Mathematics
  • Models
  • Neural Networks
  • Probabilistic Models
  • Probability
  • Simulations
  • Statistical Analysis
  • Statistics

Fields of Study

  • Computer science

Readers

  • Military History
  • Operations Research
  • Statistical inference.