Bagging Regularizes

Abstract

Intuitively, we expect that averaging - or bagging - different regressors with low correlation should smooth their behavior and be somewhat similar to regularization. In this note we make this intuition precise. Using an almost classical definition of stability, we prove that a certain form of averaging provides generalization bounds with a rate of convergence of the same order as Tikhonov regularization - similar to fashionable RKHS-based learning algorithms,

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Mar 01, 2002
Accession Number
ADA459843

Entities

People

  • Alex Rakhlin
  • Ryan Rifkin
  • Sayan Mukherjee
  • Tomaso Poggio

Organizations

  • Massachusetts Institute of Technology

Tags

DTIC Thesaurus Topics

  • Abstracts
  • Algorithms
  • Artificial Intelligence
  • Classification
  • Convergence
  • Errors
  • Hilbert Space
  • Information Operations
  • Learning
  • Machine Learning
  • Perturbations
  • Probability
  • Probability Distributions
  • Standards
  • Training

Fields of Study

  • Computer science

Readers

  • Approximation Theory.
  • Mathematical Modeling and Probability Theory.
  • Neural Network Machine Learning.