Simple, Robust, Scalable Semi-supervised Learning via Expectation Regularization

Abstract

Although semi-supervised learning has been an active area of research, its use in deployed applications is still relatively rare because the methods are often difficult to implement, fragile in tuning, or lacking in scalability. This paper presents expectation regularization, a semi-supervised learning method for exponential family parametric models that augments the traditional conditional label-likelihood objective function with an additional term that encourages model predictions on unlabeled data to match certain expectations - such as label priors. The method is extremely easy to implement, scales as well as logistic regression and can handle non-independent features. We present experiments on five different data sets, showing accuracy improvements over other semi-supervised methods.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jan 01, 2007
Accession Number
ADA534509

Entities

People

  • Andrew McCallum
  • Gideon S. Mann

Organizations

  • University of Massachusetts Amherst

Tags

DTIC Thesaurus Topics

  • Accuracy
  • Artificial Intelligence Software
  • Automata Theory
  • Computational Science
  • Computer Languages
  • Computer Science
  • Data Sets
  • Information Science
  • Language
  • Learning
  • Machine Learning
  • Models
  • Probabilistic Models
  • Probability
  • Semi-Supervised Learning
  • Supervised Machine Learning

Fields of Study

  • Computer science

Readers

  • Computational Modeling and Simulation
  • Neural Network Machine Learning.

Technology Areas

  • AI & ML
  • AI & ML - Bayesian Inference
  • AI & ML - Machine Learning Algorithms
  • AI & ML - Neural Networks