EXPRESSION OF UNCERTAINTY OF MEASUREMENTS

In general, the result of a measurement is only an approximation or estimate of the true value of the quantity subject to measurement, and thus the result is of limited value unless accompanied by a statement of its uncertainty. Much (but not all) of the scientific data appearing in the literature does include some indication of the uncertainty, but this may be stated in many different ways and is often explained poorly. In an effort to encourage consistency in uncertainty statements, the International Committee for Weights and Measures (CIPM) of the Bureau International des Poids et Mesures initiated a project, in collaboration with several other international organizations, to prepare a set of guidelines expressing international consensus on the recommended method of stating uncertainties. This project resulted in the publication of the Guide to the Expression of Uncertainty in Measurement (Refs. 1 and 2), which is often referred to as GUM. The recommendations of GUM have been summarized by the National Institute of Standards and Technology in NIST Technical Note 1297, Guidelines for Evaluating the Uncertainty of NIST Measurement Results (Ref. 3).

In the notation of GUM, we are concerned with the measurand, i.e., the quantity that is being measured. In physics and chemistry this is usually called a physical quantity and represents some inherent characteristic of a material, system, or process that can be expressed in numerical terms — specifically as the product of a number and a reference, commonly called a unit. Thus, the density of water at room temperature is (approximately) 0.998 g/mL (grams per milliliter) or, alternatively 998 kg m^–3 (kilograms per meter cubed). This statement gives the most likely value of the measurand, to this level of precision, but gives no information on how much the stated value might differ from the true value. A more detailed discussion of measurement terminology is given in the International Vocabulary of Metrology (VIM) (Ref. 4).

It is important to differentiate between the terms error and uncertainty. The error in a measurement is the difference between the measured value and the true value; the error can be stated if the true value is known (to some level of accuracy). The uncertainty is an estimate of the maximum reasonable extent to which the measured value is believed to deviate from the true value, in a situation where the true value is not known (most often the case). The result of a measurement can unknowably be very close to the true value, and thus have negligible error, even though its uncertainty is large.

The uncertainty of the result of a measurement generally consists of several components, which may be grouped in two types according to the method used to estimate their numerical values:

• Type A. Those which are evaluated by statistical methods

•           Type B. Those which are evaluated by other means

The terms “random uncertainty” and “systematic uncertainty” are often used, but these terms do not always correspond in a simple way to the A and B categories. This is because the nature of an uncertainty component is conditioned by how the quantity appears in the mathematical model that describes the current measurement process. An uncertainty component arising from a systematic effect may in some cases be evaluated by methods of Type A while in other cases by methods of Type B.

In the GUM formulation, each component of uncertainty, whether in the A or B category, is represented by an estimated standard deviation, termed standard uncertainty, symbol u_i , and equal to the positive square root of the estimated variance u_i².

For an uncertainty component of Type A, u_i  = s_i , where s_i is the statistically estimated standard deviation, as determined from a series of observations by appropriate statistical analysis. Any valid statistical method may be used. Examples are calculating the standard deviation of the mean of a series of independent observations; using the method of least squares to fit a curve to data in order to estimate parameters of the curve and their standard deviations; and carrying out an analysis of variance (ANOVA) in order to identify and quantify random effects in certain types of measurements. Details of statistical analysis are given in Refs. 5–9 and many other places.

In a similar manner, each uncertainty component of Type B is represented by a quantity u_j , which is obtained from an assumed probability distribution based on all the available information about the measurement process. Because u_j is treated like a standard deviation,the standard uncertainty in each Type B component is simply u_j . The evaluation of u_j is usually based on scientific judgment using all the relevant information available, which may include

• Previous measurement data
• Experience with, or general knowledge of, the behavior and properties of relevant materials and instruments
• Manufacturer’s specifications
• Data provided in calibrations and other reports
• Uncertainties assigned to reference data taken from handbooks

The specific approach to evaluating the standard uncertainty u_j of a Type B uncertainty will depend on the detailed model of the measurement process. The following are examples of steps that may be used:

1. Convert a quoted uncertainty (for example, in a calibration factor) that is a stated multiple of an estimated standard deviation to a standard uncertainty by dividing the quoted uncertainty by the multiplier.
2. Convert a quoted uncertainty that defines a “confidence interval” having a stated level of confidence, such as 95% or 99%, to a standard uncertainty by treating the quoted uncertainty as if a normal distribution had been used to calculate it (unless otherwise indicated) and dividing it by the appropriate factor for such a distribution. These factors are 1.960 and 2.576 for the two levels of confidence given.
3. Model knowledge of the quantity in question by a normal distribution and estimate lower and upper limits a_– and a₊ such that the best estimated value of the quantity is (a₊ + a_–)/2 (i.e., the midpoint of the limits) and there is 1 chance out of 2 (i.e., a 50% probability) that the value of the quantity lies in the interval a_– to a₊. Then u_j ≈ 1.48 a, where a = (a₊ – a_–)/2 is the half-width of the interval.
4. Model knowledge of the quantity in question by a normal distribution and estimate lower and upper limits a_– and a₊ such that the best estimated value of the quantity is (a₊ + a_–)/2 and there is about a 2 out of 3 chance (i.e., a 67% probability) that the value of the quantity lies in the interval a_– to a₊. Then u_j ≈ a, where a = (a₊ – a_– )/2.
5. Estimate lower and upper limits a_– and a₊ for the value of the quantity in question such that the probability that the value lies in the interval a_– to a₊ is, for all practical purposes, 100%. Provided that there is no contradictory information, treat the quantity as if it is equally probable for its value to lie anywhere within the interval a_– to a₊; that is, model it by a uniform or rectangular probability distribution. The best estimate of the value of the quantity is then (a₊ + a_–)/2 with u_j = a/√3 where a = (a₊ – a_–)/2. If the distribution used to model the quantity is triangular rather than rectangular, then u_j = a/√6. The rectangular distribution is a reasonable default model in the absence of any other information. But if it is known that values of the quantity in question near the center of the limits are more likely than values close to the limits, a triangular or a normal distribution may be a better model.

When all the standard uncertainties of Type A and Type B have been determined in this way, they should be combined to produce the combined standard uncertainty (suggested symbol u_c), which may be regarded as the estimated standard deviation of the measurement result. This process, often called the law of propagation of uncertainty or “root-sum-of-squares,” involves taking the square root of the sum of the squares of all the u_i . In many practical measurement situations, the probability distribution characterized by the measurement result y and its combined standard uncertainty u_c(y) is approximately normal (Gaussian). When this is the case, u_c(y) defines an interval y – u_c(y) to y + u_c(y) about the measurement result y within which the value of the measurand Y estimated by y is believed to lie with a level of confidence of approximately 68%. That is, it is believed with an approximate level of confidence of 68% that y – u_c(y) ≤ Y ≤ y + u_c(y), which is commonly written as Y = y ± u_c(y).

In fundamental metrological research (involving physical constants, calibration standards, and the like) the combined standard uncertainty u_c is normally used as the statement of uncertainty in a measurement. In most cases, however, it is desirable to use a measure of uncertainty that defines an interval about the measurement result y within which the value of the measurand Y is confidently believed to lie. The measure of uncertainty intended to meet this requirement is termed expanded uncertainty, suggested symbol U, and is obtained by multiplying u_c (y) by a coverage factor, suggested symbol k. Thus U = ku_c(y) and it is believed with high confidence that y – U ≤ Y ≤ y + U, which is commonly written as Y = y ± U. The value of the coverage factor k is chosen on the basis of the desired level of confidence to be associated with the interval defined by U = ku_c. Typically, k is in the range 2 to 3. When the normal distribution applies, U = 2u_c (i.e., k = 2) defines an interval having a level of confidence of approximately 95%, and U = 3u_c defines an interval having a confidence level greater than 99%. In current international practice it is most common to use k = 2, corresponding to about 95% confidence, but the value of k should be stated in each case to avoid confusion. See Refs. 1 and 3 for methods of calculating k when a value other than k = 2 is needed for a specific requirement.

It should be noted that the International Union of Pure and Applied Chemistry (IUPAC) is reviewing recommendations on metrological and quality concepts in analytical chemistry (Ref. 10). 

Summary of Key Steps

• Group the uncertainty components into Type A (can be evaluated by statistical methods) and Type B (must be evaluated by other means).
• Determine the standard uncertainty for each component of Type A by statistical methods and for each component of Type B by other suitable methods, based on modeling the measurement process.
• Take the square root of the sum of the squares of all the standard uncertainties to get the combined standard uncertainty u_c.
• Specify a coverage factor k which, when multiplied by u_c, gives the expanded uncertainty U. In fundamental metrological research k = 1 is usually chosen; in other cases, k = 2 (corresponding to a confidence level of about 95%) is the most common choice.

References

1. Evaluation of Measurement Data - Guide to the Uncertainty in Measurement, JCGM 100:2008, BIPM, Sevres, 2008, <bipm.org/utils/common/documents/jcgm/JCGM_1002008_E.pdf>
2. ISO, Guide to the Expression of Uncertainty in Measurement, International Organization for Standardization, Geneva, Switzerland, 1993. Several supplements have been published; see Bich, W., Cox, M. C., and Harris, P. M., “Evolution of the Guide to the Expression of Uncertainty in Measurement,” Metrologia 43, S161, 2006. [https://doi.org/10.1088/0026-1394/43/4/S01]
3. Taylor, B. N., and Kuyatt, C. E., Guidelines for Evaluating and Expressing the Uncertainty of NIST Measurement Results, NIST Technical Note 1297, National Institute of Standards and Technology, Gaithersburg, MD, 1994; available for free download at <physics.nist.gov/cuu/Uncertainty/bibliography.html>.
4. International Vocabulary of Metrology - Basic and General Concepts and Associated Terms, JCGM 200:2012, Third Edition, BIPM, Sevres, 2012.
5. Bell, S., A Beginner’s Guide to Uncertainty of Measurement, National Physical Laboratory, Teddington, Middlesex, UK, 2001; available on the Internet at <www.npl.co.uk/server.php?show=ConWebDoc.1785>.
6. Eisenhart, C., “Realistic Evaluation of the Precision and Accuracy of Instrument Calibration Systems,” J. Res. Natl. Bur. Stand. (U.S.) 67C, 161, 1963. [https://doi.org/10.6028/jres.067C.015]
7. Mandel, J., The Statistical Analysis of Experimental Data, Dover Publishers, New York, 1984.
8. Nantrella, M. G., Experimental Statistics, NBS Handbook 91, U.S. Government Printing Office, Washington, DC, 1966.
9. Box, G. E. P., Hunter, J. S., and Hunter, W. G., Statistics for Experimenters: Design, Innovation, and Discovery, Second Edition, John Wiley & Sons, Hoboken, NJ, 2005.
10. https://iupac.org/recommendation/metrological-and-quality-concepts-in-analytical-chemistry/>