ESTIMATION BY THE NEAREST-NEIGHBOR RULE.

Abstract

The nearest-neighbor estimate of the random parameter associated with a given observation is defined to be the parameter associated with the nearest observation in some training set. This paper is concerned with the infinite parameter problem (estimation) as opposed to the finite parameter problem (classification). Because of the unboundedness of the loss function in the general estimation problem, certain new considerations are required. For a wide range of probability distributions, the large-sample risk of the nearest-neighbor estimate is shown here to be less than twice the Bayes risk for metric loss functions and equal to twice the Bayes risk for squared-error loss functions. In this sense, at least half the information in the training set is contained in the nearest neighbor. (Author)

Document Details

Document Type
Technical Report
Publication Date
Sep 01, 1966
Accession Number
AD0647045

Entities

People

  • Thomas M. Cover

Organizations

  • Stanford University

Tags

DTIC Thesaurus Topics

  • Classification
  • Mathematics
  • Observation
  • Probability
  • Probability Distributions
  • Training

Readers

  • Regression Analysis.
  • Statistical inference.

Technology Areas

  • AI & ML
  • AI & ML - Bayesian Inference
  • AI & ML - Machine Learning Algorithms