Efficient Computation of Entropy Gradient for Semi-Supervised Conditional Random Fields

Abstract

Entropy regularization is a straightforward and successful method of semi-supervised learning that augments the traditional conditional likelihood objective function with an additional term that aims to minimize the predicted label entropy on unlabeled data. It has previously been demonstrated to provide positive results in linear-chain CRFs, but the published method for calculating the entropy gradient requires significantly more computation than supervised CRF training. This paper presents a new derivation and dynamic program for calculating the entropy gradient that is significantly more efficient having the same asymptotic time complexity as supervised CRF training. We also present efficient generalizations of this method for calculating the label entropy of all sub-sequences, which is useful for active learning, among other applications.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jan 01, 2007
Accession Number
ADA464763

Entities

People

  • Andrew McCallum
  • Gideon S. Mann

Organizations

  • University of Massachusetts Amherst

Tags

DTIC Thesaurus Topics

  • Acquisition
  • Algorithms
  • Computational Complexity
  • Computations
  • Computer Science
  • Data Sets
  • Gaussian Distributions
  • Information Science
  • Learning
  • Machine Learning
  • Models
  • Probabilistic Models
  • Probability
  • Semi-Supervised Learning
  • Sequences
  • Supervised Machine Learning
  • Training

Fields of Study

  • Computer science

Readers

  • Neural Network Machine Learning.
  • Statistical inference.

Technology Areas

  • AI & ML
  • AI & ML - Bayesian Inference
  • AI & ML - Machine Learning Algorithms
  • AI & ML - Neural Networks