A Note on the Effect of Ignoring Small Measurement Errors in Precision Instrument Calibration.

Abstract

The authors' focus is the simple linear regression model with measurement errors in both variables. It is often stated that if the measurement error in x is small, then we can ignore this error and fit the model to data using ordinary least squares. There is some ambiguity in the statistical literature concerning the exact meaning of a small error. For example Draper and Smith (1981) state that if the measurement error variance in x is small relative to the variability of the true x's, then errors in the x's can be effectively ignored, see Montgomery & Peck (1983) for a similar statement. Scheffe (1983) and Mandel (1984) argue for a second criterion, which may be informally summarized that the error in x should be small relative to (the standard deviation of the observed Y about the line)/(slope of the line). We argue that for calibration experiments both criteria are useful and important, the former for estimation of x given Y and the latter for confidence intervals for x given Y. Keywords: Confidence intervals. (Author)

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jun 01, 1985
Accession Number
ADA159104

Entities

People

  • C. H. Spiegelman
  • Raymond J. Carroll

Organizations

  • University of North Carolina at Chapel Hill

Tags

DTIC Thesaurus Topics

  • Air Force
  • Air Force Facilities
  • Calibration
  • Data Science
  • Data Sets
  • Equations
  • Estimators
  • Information Science
  • Intervals
  • Linear Regression Analysis
  • Literature
  • Measurement
  • Military Research
  • North Carolina
  • Regression Analysis
  • Standards
  • Statistics

Fields of Study

  • Mathematics

Readers

  • Military History of the United States in the 20th Century.
  • Regression Analysis.