The inadequate use of the determination coefficient in analytical calibrations: How other parameters can assess the goodness-of-fit more adequately

Abstract

Simple linear regression using ordinary least-squares is the most common function applied in laboratories for analytical calibrations. The determination and/or the correlation coefficients are usually the parameters applied for assessing the goodness-of-fit of a simple linear calibration. However, these parameters are unable to detect the highly biased results at low calibration levels that are obtained with ordinary least-squares. In this study, the use of other parameters based on the relative standard errors of the calculated contents is evaluated. It has been found that these alternative parameters can detect the biased results obtained at low calibration levels with ordinary least-squares, being the relative standard error the one that seems to provide the most adequate results. Ordinary least-squares should only be applied if the lower limit of quantification is set to at least five times above the conventional limit of quantification. For trace analysis, where the lowest possible limit of quantification is required, weighted least-squares should be applied to obtain accurate estimates, especially at low concentrations. One of the greatest advantages of the relative standard error is that this parameter can be determined for all types of regression functions and is not limited to calibrations with linear relationships between the variables