Dimensionality Reduction for Supervised Learning With Reproducing Kernel Hilbert Spaces
Abstract
We propose a novel method of dimensionality reduction for supervised learning problems. Given a regression or classifcation problem in which we wish to predict a response variable Y from an explanatory variable X, we treat the problem of dimensionality reduction as that of finding a low-dimensional "effective subspace" of X which retains the statistical relationship between X and Y . We show that this problem can be formulated in terms of conditional independence. To turn this formulation into an optimization problem we establish a general nonparametric characterization of conditional independence using covariance operators on a reproducing kernel Hilbert space. This characterization allows us to derive a contrast function for estimation of the effective subspace. Unlike many conventional methods for dimensionality reduction in supervised learning, the proposed method requires neither assumptions on the marginal distribution of X, nor a parametric model of the conditional distribution of Y . We present experiments that compare the performance of the method with conventional methods.
Document Details
- Document Type
- Technical Report
- Publication Date
- May 25, 2003
- Accession Number
- ADA446572
Entities
People
- Francis R. Bach
- Kenji Fukumizu
- Michael I. Jordan
Organizations
- University of California, Berkeley