Ordinary least squares with laboratory calibrations: a practical way to show students that this fitting model may easily yield biased results when used indiscriminately

Abstract

Analytical calibration using ordinary least squares (OLS) is the most widely applied response function for calibration in all type of laboratories. However, this calibration function is not always the most adequate and its indiscriminant use can lead to obtain biased estimates of unknowns. Students need to be taught about the practical requirements needed to obtain good results with OLS and when this fitting method is not accurate. Different experimental calibration curves were obtained in laboratory sessions using two common instrumental techniques: chromatography and atomic absorption spectrometry. After discussion seminars evaluating the data obtained by students, they were able to understand that linear fitting was not the most accurate model using atomic absorption spectrometry and a quadratic fitting provided most accurate estimates. Linearity was confirmed in chromatographic calibrations, but data presented heteroscedasticity, which is very common in calibrations done in chemical and biological analyses. A simple experiment was applied to show students how the use of the regression coefficients obtained by OLS with heteroscedastic data lead to highly biased estimates near the quantification limits of the calibration curve. The results obtained allowed to show students that, despite being widely used, OLS is not the most adequate fitting model to obtain accurate and precise results with many calibration methods routinely used in chemical and biological laboratories