The Effect of Ignoring Small Measurement Errors in Precision Instrument Calibration

Abstract

This paper discusses the effect of measurement errors in both variables when using the simple linear regression model. It is often stated that if the measurement error in x is small, then we can ignore this error and fit the model to data using ordinary least squares. There is some ambiguity in the statistical literature concerning the exact meaning of a small error. For example. Draper and Smith (1981) state that if the measurement error variance in x is small relative to the variability of the true x's, then “errors in the x's can be effectively ignored.” See Montgomery and Peck (1983) for a similar statement. Scheffé (1973) and Mandel (1984) argue for a second criterion, which may be informally summarized that the error in x should be small relative to (the standard deviation of the observed Y about the line)/(slope of the line). We argue that for calibration experiments, both criteria are useful and important; the former for estimation of x given Y, and the latter for the lengths of confidence intervals for x given Y.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 22 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.