Measurement Uncertainty

Working Group 8.42

Content

• Overview
• GUM
• Workshops
• Research
• Software
• Publications

The reliability of measurement results is a crucial prerequisite for today’s world-wide economy, e.g. when checking the conformity of products with an agreed quality standard. A complete measurement result hence requires a quantitative statement describing its associated uncertainty. Measurement uncertainty is particularly relevant in metrology, for example when the result of a measurement is traced back to an SI unit.

The uncertainty about a measurement result is often due to random variations in measured data. Systematic deviations which remain unchanged in repeated measurements are another source of uncertainty. In order to evaluate measurement uncertainties, statistical methods are employed. Classical statistics, for example, can be used to determine confidence intervals for the sought quantities that account for observed random variations in the measured data. Bayesian statistics provides an alternative that also allows further information to be taken into account. This approach treats random variability and systematic deviations in a consistent way, and it results in probability statements about the quantities of interest. In particular, the Bayesian approach enables to be taken into account prior knowledge, e.g., when negative results can be ruled out for physical reasons. As a result probability statements can then be made about the sought measureand.

In metrology the uncertainty of a measurement result is often dominated by systematic deviations. The “Guide to the Expression of Uncertainty in Measurement“ (GUM) enables systematic deviations and random variations to be coherently taken into account. The GUM can be seen as the de facto standard for the evaluation of measurement uncertainty in metrology. An important concept of the GUM methodology is that of a model which relates the measurand to so-called input quantities. In using information about the input quantities, this model is utilized to determine an estimate and its associated uncertainty for the measurand according to the rule of “propagation of uncertainties”.

Supplement 1 to the GUM (GUM S1) proposes a Monte Carlo Method (MCM) for the calculation of uncertainties. Similarl to the GUM, a model relating the measurand and input quantities is taken as the basis. By using probability density functions (PDFs) expressing the knowledge about the input quantities, MCM is then used to determine the PDF associated with the measurand by “propagation of distributions”. There is a certain relationship between GUM S1 and a Bayesian uncertainty analysis. A further supplement to the GUM deals with uncertainty evaluation for multivariate (e.g., complex valued) quantities (GUM S2).

In order to facilitate the application of the methods developed in the working group, the following software implementations are made available free of charge.

MCMC implementation for the analysis of magnetic field fluctuation thermometry

Bayesian approaches to performing regression often require numerical methods such as Markov Chain Monte Carlo (MCMC) sampling. For the analysis in magnetic field fluctuation thermometry, PTB Working Group 8.42 has developed a MATLAB software package to perform MCMC sampling from the posterior distribution of the calibration parameters and to subsequently estimate temperatures.
This software is available in the electronic supplement to the related publication.

Related publication

G. Wübbeler, F. Schmähling, J. Beyer, J. Engert, and C. Elster (2012). Analysis of magnetic field fluctuation thermometry using Bayesian inference. Meas. Sci. Technol. 23, 125004 (9pp), [DOI: 1018088/0957-0233/23/12/125004].

 

WinBUGS software for the analysis of immunoassay data

The Bayesian approach enables the inclusion of additional prior knowledge in regression problems, but often requires numerical methods such as Markov Chain Monte Carlo (MCMC) sampling. For the analysis of immunoassay data, PTB Working Group 8.42 has developed WinBUGS software code to perform MCMC sampling from the posterior distribution for the calibration parameters and the unknown concentration.
This software is available in A Guide to Bayesian Inference for Regression Problems.

Related publications

K. Klauenberg, M. Walzel, B. Ebert, and C. Elster (2015). Informative prior distributions for ELISA analyses. Biostatistics 16, 454-464, [DOI: 10.1093/biostatistics/kxu057].

C. Elster, K. Klauenberg, M. Walzel, G. Wübbeler, P. Harris, M. Cox, C. Matthews, I. Smith, L. Wright, A. Allard, N. Fischer, S. Cowen, S. Ellison, P. Wilson, F. Pennecchi, G. Kok, A. van der Veen, and L. Pendrill (2015). A Guide to Bayesian Inference for Regression Problems Deliverable of EMRP project NEW04 “Novel mathematical and statistical approaches to uncertainty evaluation”, [download (pdf)].

 

Rejection sampling for the flow meter calibration problem

Bayesian approaches to Normal linear regression problems yield analytical solutions under certain circumstances. Nevertheless, accounting for constraints on the values of the regression curve when calibrating flow meters requires a Monte Carlo procedure combined with an accept/reject algorithm to obtain samples from the posterior distribution.
MATLAB source code implementing this algorithm is available in A Guide to Bayesian Inference for Regression Problems

Related publications

G. J. P. Kok, A. M. H. van der Veen, P. M. Harris, I.M. Smith, C. Elster (2015). Bayesian analysis of a flow meter calibration problem. Metrologia 52, 392-399,  [DOI: 10.1088/0026-1394/52/2/392].

C. Elster, K. Klauenberg, M. Walzel, G. Wübbeler, P. Harris, M. Cox, C. Matthews, I. Smith, L. Wright, A. Allard, N. Fischer, S. Cowen, S. Ellison, P. Wilson, F. Pennecchi, G. Kok, A. van der Veen, and L. Pendrill (2015). A Guide to Bayesian Inference for Regression Problems Deliverable of EMRP project NEW04 “Novel mathematical and statistical approaches to uncertainty evaluation”, [download (pdf)].

An introductory example for Markov chain Monte Carlo (MCMC)

When the Guide to the Expression of Uncertainty in Measurement (GUM) and methods from its supplements are not applicable, the Bayesian approach may be a valid and welcome alternative. Evaluating the posterior distribution, estimates or uncertainties involved in Bayesian inferences often requires numerical methods to avoid high-dimensional integrations. Markov chain Monte Carlo (MCMC) sampling is such a method—powerful, flexible and widely applied. PTB Working Group 8.42 has developed a concise introduction, illustrated by a simple, typical example from metrology. Accompanied with few lines of software code to implement the most basic and yet flexible MCMC method, interested readers are invited to get started. MATLAB as well as R source code are available in the related publication.

Related Publication

K. Klauenberg und C. Elster Markov chain Monte Carlo methods: an introductory example. Metrologia, 53(1), S32, 2016. [DOI: 10.1088/0026-1394/53/1/S32] 

While the available guidelines are appropriate in many metrological applications, the development of procedures for the evaluation of measurement uncertainty in more involved applications is a topic of ongoing research. Our current research interests are