ISO/TC 69/SC 6 - Measurement methods and results

This document provides guidance for implementing the theories of the ISO 11843 series in various practical situation. As defined in this series, the term minimum detectable value corresponds to the limit of detection or detection limit defined by the IUPAC. The focus of interest is placed on the practical applications of statistics to quantitative analyses.

This document introduces five statistical methods for evaluating the precision of binary measurement methods and their results. The five methods can be divided into two types. Both types are based on measured values provided by each laboratory participating in a collaborative study. In the first type, each laboratory repeatedly measures a single sample. The samples measured by the laboratories are nominally identical. The second type is an extension of the first type, where there are several levels of samples. For each statistical method, this document briefly summarizes its theory and explains how to estimate the proposed precision measures. Some real cases are illustrated to help the readers understand the evaluation procedures involved. For the first and second types of methods, five and three cases are presented, respectively. Finally, this document compares the five statistical methods.

1.1 This document — specifies basic methods for estimating the bias of a measurement method and the laboratory bias when a measurement method is applied; — provides a practical approach of a basic method for routine use in estimating the bias of measurement methods and laboratory bias; — provides a brief guidance to all personnel concerned with designing, performing or analysing the results of the measurements for estimating bias. 1.2 It is concerned exclusively with measurement methods which yield measurements on a continuous scale and give a single value as the measurement result, although the single value can be the outcome of a calculation from a set of observations. 1.3 This document applies when the measurement method has been standardized and all measurements are carried out according to that measurement method. NOTE In ISO/IEC Guide 99:2007(VIM), "measurement procedure" (2.6) is an analogous term related to the term "measurement method" used in this document. 1.4 This document applies only if an accepted reference value can be established to substitute the true value by using the value, for example: — of a suitable reference material; — of a suitable measurement standard; — referring to a suitable measurement method; — of a suitable prepared known sample. 1.5 This document applies only to the cases where it is sufficient to estimate bias on one property at a time. It is not applicable if the bias in the measurement of one property is affected by the level of any other property (i.e. it does not consider interferences by any influencing quantity). Comparison of the trueness of two-measurement methods is considered in ISO 5725-6.

1.1 This document — amplifies the general principles for designing experiments for the numerical estimation of the precision of measurement methods by means of a collaborative interlaboratory experiment; — provides a detailed practical description of the basic method for routine use in estimating the precision of measurement methods; — provides guidance to all personnel concerned with designing, performing or analysing the results of the tests for estimating precision. NOTE Modifications to this basic method for particular purposes are given in other parts of ISO 5725. 1.2 It is concerned exclusively with measurement methods which yield measurements on a continuous scale and give a single value as the test result, although this single value can be the outcome of a calculation from a set of observations. 1.3 It assumes that in the design and performance of the precision experiment, all the principles as laid down in ISO 5725-1 are observed. The basic method uses the same number of test results in each laboratory, with each laboratory analysing the same levels of test sample; i.e. a balanced uniform-level experiment. The basic method applies to procedures that have been standardized and are in regular use in a number of laboratories. 1.4 The statistical model of ISO 5725-1:1994, Clause 5, is accepted as a suitable basis for the interpretation and analysis of the test results, the distribution of which is approximately normal. 1.5 The basic method, as described in this document, (usually) estimates the precision of a measurement method: a) when it is required to determine the repeatability and reproducibility standard deviations as defined in ISO 5725-1; b) when the materials to be used are homogeneous, or when the effects of heterogeneity can be included in the precision values; and c) when the use of a balanced uniform-level layout is acceptable. 1.6 The same approach can be used to make a preliminary estimate of precision for measurement methods which have not reached standardization or are not in routine use.

This document presents methods for determining the critical value of the response variable and the minimum detectable value in Poisson distribution measurements. It is applicable when variations in both the background noise and the signal are describable by the Poisson distribution. The conventional approximation is used to approximate the Poisson distribution by the normal distribution consistent with ISO 11843‑3 and ISO 11843‑4. The accuracy of the normal approximation as compared to the exact Poisson distribution is discussed in Annex C.

1.1 This document is concerned with polynomial calibration functions that describe the relationship between a stimulus variable and a response variable. These functions contain parameters estimated from calibration data consisting of a set of pairs of stimulus value and response value. Various cases are considered relating to the nature of any uncertainties associated with the data. 1.2 Estimates of the polynomial function parameters are determined using least‐squares methods, taking account of the specified uncertainty information. It is assumed that the calibration data are fit for purpose and thus the treatment of outliers is not considered. It is also assumed that the calibration data errors are regarded as drawn from normal distributions. An emphasis of this document is on choosing the least‐squares method appropriate for the nature of the data uncertainties in any particular case. Since these methods are well documented in the technical literature and software that implements them is freely available, they are not described in this document. 1.3 Commonly occurring types of covariance matrix associated with the calibration data are considered covering (a) response data uncertainties, (b) response data uncertainties and covariances, (c) stimulus and response data uncertainties, and (d) stimulus data uncertainties and covariances, and response data uncertainties and covariances. The case where the data uncertainties are unknown is also treated. 1.4 Methods for selecting the degree of the polynomial calibration function according to prescribed criteria are given. The covariance matrix associated with the estimates of the parameters in the selected polynomial function is available as a by‐product of the least‐squares methods used. 1.5 For the chosen polynomial function this document describes the use of the parameter estimates and their associated covariance matrix for inverse and direct evaluation. It also describes how the provisions of ISO/IEC Guide 98‐3:2008 (GUM) can be used to provide the associated standard uncertainties. 1.6 Consideration is given to accounting for certain constraints (such as the polynomial passing through the origin) that may need to be imposed and also to the use of transformations of the variables that may render the behaviour of the calibration function more polynomial‐like. Interchanging the roles of the variables is also considered. 1.7 Examples from several areas of measurement science illustrate the use of this document.

Background noise exists ubiquitously in analytical instruments, whether or not a sample is applied to the instrument. This document is concerned with mathematical methodologies for estimating the minimum detectable value in case that the most predominant source of measurement uncertainty is background noise. The minimum detectable value can directly and mathematically be derived from the stochastic characteristics of the background noise. This document specifies basic methods to — extract the stochastic properties of the background noise, — use the stochastic properties to estimate the standard deviation (SD) or coefficient of variation (CV) of the response variable, and — calculate the minimum detectable value based on the SD or CV obtained above. The methods described in this document are useful for checking the detection of a certain substance by various types of measurement equipment in which the background noise of the instrumental output predominates over the other sources of measurement uncertainty. Feasible choices are visible and ultraviolet absorption spectrometry, atomic absorption spectrometry, atomic fluorescence spectrometry, luminescence spectrometry, liquid chromatography and gas chromatography.

ISO 21748:2017 gives guidance for - evaluation of measurement uncertainties using data obtained from studies conducted in accordance with ISO 5725‑2, and - comparison of collaborative study results with measurement uncertainty (MU) obtained using formal principles of uncertainty propagation (see Clause 14). ISO 5725‑3 provides additional models for studies of intermediate precision. However, while the same general approach may be applied to the use of such extended models, uncertainty evaluation using these models is not incorporated in this document. ISO 21748:2017 is applicable to all measurement and test fields where an uncertainty associated with a result has to be determined. ISO 21748:2017 does not describe the application of repeatability data in the absence of reproducibility data. ISO 21748:2017 assumes that recognized, non-negligible systematic effects are corrected, either by applying a numerical correction as part of the method of measurement, or by investigation and removal of the cause of the effect. The recommendations in this document are primarily for guidance. It is recognized that while the recommendations presented do form a valid approach to the evaluation of uncertainty for many purposes, it is also possible to adopt other suitable approaches. In general, references to measurement results, methods and processes in this document are normally understood to apply also to testing results, methods and processes.

ISO/TS 17503:2015 describes the estimation of uncertainties on the mean value in experiments conducted as crossed designs, and the use of variances extracted from such experiments and applied to the results of other measurements (for example, single observations). ISO/TS 17503:2015 covers balanced two-factor designs with any number of levels. The basic designs covered include the two-way design without replication and the two-way design with replication, with one or both factors considered as random. Calculations of variance components from ANOVA tables and their use in uncertainty estimation are given. In addition, brief guidance is given on the use of restricted maximum likelihood estimates from software, and on the treatment of experiments with small numbers of missing data points. Methods for review of the data for outliers and approximate normality are provided. The use of data obtained from the treatment of relative observations (for example, apparent recovery in analytical chemistry) is included.

ISO 13528:2015 provides detailed descriptions of statistical methods for proficiency testing providers to use to design proficiency testing schemes and to analyse the data obtained from those schemes. It provides recommendations on the interpretation of proficiency testing data by participants in such schemes and by accreditation bodies. The procedures in ISO 13528:2015 can be applied to demonstrate that the measurement results obtained by laboratories, inspection bodies, and individuals meet specified criteria for acceptable performance. ISO 13528:2015 is applicable to proficiency testing where the results reported are either quantitative measurements or qualitative observations on test items.

ISO/TR 13587:2012 is concerned with three basic statistical approaches for the evaluation and interpretation of measurement uncertainty: the frequentist approach including bootstrap uncertainty intervals, the Bayesian approach, and fiducial inference. The common feature of these approaches is a clearly delineated probabilistic interpretation or justification for the resulting uncertainty intervals. For each approach, the basic method is described and the fundamental underlying assumptions and the probabilistic interpretation of the resulting uncertainty are discussed. Each of the approaches is illustrated using two examples including an example from the ISO/IEC Guide 98-3 (Uncertainty of measurement ? Part 3: Guide to the expression of uncertainty in measurement (GUM:1995)). This document also includes a discussion of the relationship between the methods proposed in GUM Supplement 1 and these three statistical approaches.

ISO/TS 28037:2010 is concerned with linear, that is, straight-line, calibration functions that describe the relationship between two variables X and Y, namely, functions of the form Y = A + BX. Although many of the principles apply to more general types of calibration function, the approaches described exploit the simple form of the straight-line calibration function wherever possible. Values of the parameters A and B are determined on the basis of measured data points (xi, yi), i = 1, ... , m. Various cases are considered relating to the nature of the uncertainties associated with these data. No assumption is made that the errors relating to the yi are homoscedastic (having equal variance), and similarly for the xi when the errors are not negligible. Estimates of the parameters A and B are determined using least squares methods. The emphasis is on choosing the least squares method most appropriate for the type of measurement data, in particular methods that reflect the associated uncertainties. The most general type of covariance matrix associated with the measurement data is treated, but important special cases that lead to simpler calculations are described in detail. For all cases considered, methods for validating the use of the straight-line calibration functions and for evaluating the uncertainties and covariance associated with the parameter estimates are given. ISO/TS 28037:2010 also describes the use of the calibration function parameter estimates and their associated uncertainties and covariance to predict a value of X and its associated standard uncertainty given a measured value of Y and its associated standard uncertainty.

ISO 11843-5:2008 is concerned with calibration functions that are either linear or non-linear. It specifies basic methods to construct a precision profile for the response variable, namely a description of the standard deviation or coefficient of variation of the response variable as a function of the net state variable, transform this precision profile into a precision profile for the net state variable in conjunction with the calibration function, and use the latter precision profile to estimate the critical value and minimum detectable value of the net state variable. The methods described ISO 11843-5:2008 are useful for checking the detection of a certain substance by various types of measurement equipment to which ISO 11843-2 cannot be applied. Included are assays of persistent organic pollutants (POPs) in the environment, such as dioxins, pesticides and hormone-like chemicals, by competitive ELISA (enzyme-linked immunosorbent assay), and tests of bacterial endotoxins that induce hyperthermia in humans. The definition and applicability of the critical value and minimum detectable value of the net state variable are described in ISO 11843-1 and ISO 11843-2. ISO 11843-5:2008 extends the concepts in ISO 11843-2 to the cases of non-linear calibration. Examples are provided.

ISO 9169:2006 provides definitions and specifies methods to determine performance characteristics of an identified automatic air quality measuring system. Tests are carried out under stable laboratory conditions or field conditions. The automatic measuring system is considered as a black box operated according to specified procedures. ISO 9169:2006 applies to measuring systems for which the following information is available: a description of the automatic measuring system providing the result of measurement in the physical unit of the measurand; operating procedures of the automatic measuring system including, where appropriate, the procedures of routine adjustment, routine verification and calibration; terms of reference for the test program specifying the client requirements and test conditions. ISO 9169:2006 applies to measuring systems for which it is possible to apply several reference materials with accepted values with known uncertainty for the measurand, within the range of application. ISO 9169:2006 does not specify the number of automatic measuring systems to be tested.

ISO/TS 21749:2005 follows the approach taken in the Guide to the expression of the uncertainty of measurement (GUM) and establishes the basic structure for stating and combining components of uncertainty. To this basic structure, it adds a statistical framework using the analysis of variance (ANOVA) for estimating individual components, particularly those classified as Type A evaluations of uncertainty, i.e. based on the use of statistical methods. A short description of Type B evaluations of uncertainty (non-statistical) is included for completeness. ISO/TS 21749:2005 covers experimental situations where the components of uncertainty can be estimated from statistical analysis of repeated measurements, instruments, test items or check standards. It provides methods for obtaining uncertainties from single-, two- and three-level nested designs only. More complicated experimental situations where, for example, there is interaction between operator effects and instrument effects or a cross effect, are not covered. ISO/TS 21749:2005 is not applicable to measurements that cannot be replicated, such as destructive measurements or measurements on dynamically varying systems (such as fluid flow, electronic currents or telecommunications systems). It is not particularly directed to the certification of reference materials (particularly chemical substances) and to calibrations where artefacts are compared using a scheme known as a "weighing design". For certification of reference materials, see ISO Guide 35. When results from interlaboratory studies can be used, techniques are presented in the companion guide ISO/TS 21748. The main difference between ISO/TS 21748 and this Technical Specification is that the ISO/TS 21748 is concerned with reproducibility data (with the inevitable repeatability effects), whereas this Technical Specification concentrates on repeatability data and the use of the analysis of variance for its treatment. ISO/TS 21749:2005 is applicable to a wide variety of measurements, for example, lengths, angles, voltages, resistances, masses and densities.

ISO 11843-4:2003 deals with the assessment of the capability of detection of a measurement method without the assumptions in ISO 11843-2 of a linear calibration curve and certain relationships between the residual standard deviation and the value of the net state variable Instead of estimating the minimum detectable value, ISO 11843-4:2003 provides a criterion for judging whether the minimum detectable value is less than a given level of the net state variable, and the basic experimental design for testing the conformity of this criterion.

ISO 11843-3:2003 gives a method of estimating the critical value of the response variable from the mean and standard deviation of repeated measurements of the reference state in certain situations in which the value of the net state variable is zero, for all reasonable and foreseeable purposes. Hence, it can be decided whether values of the response variable in an actual state (or test sample) are above the range of values attributable to the reference state. General procedures for determination of critical values of the response variable and the net state variable and of the minimum detectable value have been given in ISO 11843-2. Those procedures are applicable in situations in which there is relevant straight-line calibration and the residual standard deviation of the measured responses is either constant or is a linear function of the net state variable. The procedure given in this part of ISO 11843 for the determination of the critical value of the response variable only is recommended for situations in which no calibration data are used. The distribution of data is assumed to be normal or near-normal. The procedure given in this part of ISO 11843 is recommended for situations in which it is difficult to obtain a large amount of the actual states although a large amount of the basic state can be prepared.

ISO 10576-1:2002 sets out guidelines for drafting requirements that may be formulated as limiting values for a quantifiable characteristic and for checking conformity to such requirements when the test or measurement result is subject to uncertainty. It is applicable whenever the uncertainty may be quantified according to the principles laid down in the Guide to the expression of uncertainty in measurement. The term uncertainty is thus a descriptor for all elements of variation in the measurement result, including uncertainty due to sampling. It is outside the scope of ISO 10576-1:2002 to give rules for how to act when an inconclusive result of a conformity test has been obtained.

Outlines the general principles needed to calibrate a measurement system and to maintain that system in a state of statistical control. Provides a basic method for estimating a linear calibration function, a control method for extended use of a calibration function and two alternative methods to the basic method.

Specifies four intermediate measures due to changes in observation conditions (time, calibration, operator and equipment) within a laboratory. These intermediate measures can be established by an experiment within a specific laboratory or by an interlaboratory experiment. Furthermore, discusses the implications of the definitions of intermediate precision measures, presents guidance on the interpretation and application of the estimates of intermediate precision measures in practical situations, discusses the connections between trueness and measurement conditions.

The purpose is to outline the general principles to be understood when assessing accuracy (trueness and precision) of measurement methods and results, and in applications, and to establish practical estimations of the various measures by experiment. Is concerned exclusively with measurement methods which yield measurements on a continuous scale and give a single value as the test result. May be applied to a very wide range of materials, including liquids, powders and solid objects, manufactured or naturally occurring, provided that due consideration is given to any heterogeneity of the material.

The purpose is to give some indications of the way in which accuracy data can be used in various practical situations by: giving a standard method of calculating the repeatability limit, the reproducibility limit and other limits, providing a way of checking the acceptability of test results obtained under repeatability or reproducibility conditions, describing how to assess the stability of results within a laboratory over a period of time, describing how to assess whether a given laboratory is able to use a given standard measurement method in a satisfactory way, describing how to compare alternative measurement methods.

ISO 11843-6:2013 presents methods for determining the critical value of the response variable and the minimum detectable value in Poisson distribution measurements. It is applicable when variations in both the background noise and the signal are describable by the Poisson distribution. The conventional approximation is used to approximate the Poisson distribution by the normal distribution consistent with ISO 11843-3 and ISO 11843-4. The accuracy of the normal approximation as compared to the exact Poisson distribution is discussed.

Background noise ubiquitously exists in analytical instruments whether or not a sample is applied to the instrument. ISO 11843-7:2012 is concerned with mathematical methodologies for estimating the minimum detectable value in case that the most predominant source of measurement uncertainty is background noise. The minimum detectable value can directly and mathematically be derived from the stochastic characteristics of the background noise. It specifies basic methods to ? extract the stochastic properties of the background noise, ? use the stochastic properties to estimate the standard deviation (SD) or coefficient of variation (CV) of the response variable, and ? calculate the minimum detectable value based on the SD or CV obtained above. The methods described in ISO 11843-7:2012 are useful for checking the detection of a certain substance by various types of measurement equipment in which the background noise of the instrumental output predominates over the other sources of measurement uncertainty. Feasible choices are visible and ultraviolet absorption spectrometry, atomic absorption spectrometry, atomic fluorescence spectrometry, luminescence spectrometry, liquid chromatography and gas chromatography.

ISO 21748:2010 gives guidance for evaluation of measurement uncertainties using data obtained from studies conducted in accordance with ISO 5725-2:1994; comparison of collaborative study results with measurement uncertainty (MU) obtained using formal principles of uncertainty propagation. (ISO 5725-3:1994 provides additional models for studies of intermediate precision. However, while the same general approach may be applied to the use of such extended models, uncertainty evaluation using these models is not incorporated in ISO 21748:2010.) ISO 21748:2010 is applicable in all measurement and test fields where an uncertainty associated with a result has to be determined. It does not describe the application of repeatability data in the absence of reproducibility data. ISO 21748:2010 assumes that recognized, non-negligible systematic effects are corrected, either by applying a numerical correction as part of the method of measurement, or by investigation and removal of the cause of the effect. The recommendations in ISO 21748:2010 are primarily for guidance. It is recognized that while the recommendations presented do form a valid approach to the evaluation of uncertainty for many purposes, it is also possible to adopt other suitable approaches. In general, references to measurement results, methods and processes in ISO 21748:2010 are normally understood to apply also to testing results, methods and processes.

ISO 13528:2005 complements ISO Guide 43 (all parts) by providing detailed descriptions of sound statistical methods for organizers to use to analyse the data obtained from proficiency testing schemes, and by giving recommendations on their use in practice by participants in such schemes and by accreditation bodies. ISO 13528:2005 can be applied to demonstrate that the measurement results obtained by laboratories do not exhibit evidence of an unacceptable level of bias. It is applicable to quantitative data but not qualitative data.

ISO/TR 22971:2005 provides users with practical guidance to the use of ISO 5725-2:1994 and presents simplified step-by-step procedures for the design, implementation, and statistical analysis of inter-laboratory studies for assessing the variability of a standard measurement method and on the determination of repeatability and reproducibility of data obtained in inter-laboratory testing.

TS 21748:2004 gives guidance for the evaluation of measurement uncertainties using data obtained from studies conducted in accordance with ISO 5725-2, and for the comparison of collaborative study results with measurement uncertainty obtained using formal principles of uncertainty propagation. TS 21748:2003 is applicable in all measurement and test fields where an uncertainty associated with a result has to be determined, but does not describe the application of repeatability data in the absence of reproducibility data. The recommendations in this document are primarily for guidance.

Provides basic methods for estimating the bias of a measurement method and the laboratory bias when a measurement method is applied. In order that the measurements are made in the same way, it is important that the measurement method has been standardized. Can be applied only if the accepted reference value can be established as a conventional true value, e.g. by measurement standards or suitable reference materials or by referring to a reference measurement method or by preparation of a known sample.

Amplifies the general principles to be observed in designing experiments for the numerical estimation of the precision of measurement methods by means of a collaborative interlaboratory experiment, provides a detailed practical description of the basic method for routine use in estimating the precision of measurement methods, provides guidance to all personnel concerned with designing, performing or analysing the results of the tests for estimating precision. Annex B provides practical examples of estimating the precision of measurement methods by experiment.