 # Regression

Working Group 8.42

# Description

Regression problems occur in many metrological applications, e.g. in everyday calibration tasks (as illustrated in Annex H.3 of the GUM), in the evaluation of interlaboratory comparisons, the characterization of sensors [ Matthews et al., 2014], determination of fundamental constants [ Bodnar et al., 2014], interpolation or prediction tasks [ Wübbeler et al., 2012], and many more. Such problems arise when the quantity of interest cannot be measured directly, but has to be inferred from measurement data (and their uncertainties) using a mathematical model that relates the quantity of interest to the data. For example, regressions may serve to evaluate the functional relation between variables. Fig. 1: Illustration of a typical straight line regression problem with normally distributed measurement errors. Displayed is the fitted mean regression curve (solid line) and its pointwise 95% credible intervals (dashed lines). The thin vertical line shows a prediction at a new value $x$ and its 95% credible interval. The small circles represent the measurements data.

#### Definition and Examples

Regression problems often take the form
$$\begin{equation*} y_i = f_{\boldsymbol{\theta}}(x_i) + \varepsilon_i , \quad i=1, \ldots, n \,, \end{equation*}$$
where the measurements $\boldsymbol{y}=(y_1, \ldots, y_n)^\top$ are explained by a function $f_{\boldsymbol{\theta}}$ evaluated at values $\boldsymbol{x}=(x_1, \ldots, x_n)^\top$ and depending on unknown parameters $\boldsymbol{\theta}=(\theta_1, \ldots, \theta_p)^\top$. The measurement error $\pmb{\varepsilon}=(\varepsilon_1, \ldots, \varepsilon_n)^\top$ follows a specified distribution $p(\pmb{\varepsilon} | \boldsymbol{\theta}, \boldsymbol{\sigma}).$
Regressions may be used to describe the relationship between a traceable, highly accurate reference device with values denoted by $x$ and a device to be calibrated with values denoted by $y$. The pairs $(x_i,y_i)$ then denote simultaneous measurements made by the two devices of the same measurand such as, for example, temperature.
A simple example is the Normal straight line regression model (as illustrated in Figure 1):
$$\begin{equation} \label{int_reg_eq1} y_i = \theta_1 + \theta_2 x_i + \varepsilon_i , \quad \varepsilon_i \stackrel{iid}{\sim} \text{N}(0, \sigma^2), \quad i=1, \ldots, n \,. \end{equation}$$
The basic goal of regression tasks is to estimate the unknown parameters $\pmb{\theta}$ of the regression function and possibly also the unknown parameters of the error distribution $\pmb{\sigma}$. The estimated regression model may then be used to evaluate the shape of the regression function, predictions or interpolations of intermediate or extrapolated $x$-values, or to invert the regression function to predict $x$-values for new measurements.

# Research

Decisions based on regression analyses require a reliable evaluation of measurement uncertainty. The current state of the art in uncertainty evaluation in metrology (i.e. the GUM and its supplements) provides little guidance for regression, however. One reason is that the GUM guidelines are based on a model that relates the quantity of interest (the measurand) to the input quantities. Yet, regression models cannot be uniquely formulated as such a measurement function. By way of example, Annex H.3 of the GUM nevertheless suggests a possibility for analyzing regression problems. However, this analysis contains elements from both classical (least squares) and Bayesian statistics such that the results are not deduced from state-of-knowledge distributions and usually differ from a purely classical or Bayesian approach which was shown in [ Elster et al., 2011].

Consequently, there is a need for guidance and research in metrology for uncertainty evaluation in regression problems. The Joint Committee for Guides in Metrology (JCGM) has recognized this need. PTB Working Group 8.42 lead the development of guidance for Bayesian inference of regression problems within the EMRP project NEW04, which is summarized in a Guide [ Elster et al., 2015]. This Guide also contains template solutions for specific regression problems with known values $\boldsymbol{x}$ and is available free of charge at the NEW04 project web page. For regression problems with Gaussian measurement errors and linear regression functions (such as in formula (1)), [ Klauenberg et al., 2015_2] provide guidance when extensive numerical calculations (such as Markov Chain Monte Carlo methods) are to be avoided in a Bayesian inference.

Regression problems often involve uncertainty in the x-values as well. Within the EMPIR project 17NRM05 EMUE three adaptable examples were developed, which illustrate different aspects of fitting a straight-line:

In addition, PTB Working Group 8.42 carries out research emerging from metrological applications involving regression. For example,

# Publications

 • B. Kästner, M. Marschall, A. Hornemann, S. Metzner, P. Patoka, S. Cortes, G. Wübbeler, A. Hoehl, E. Rühl;C. Elster Measurement Science and Technology, 2023. [DOI: 10.1088/1361-6501/acfc27] • G. Wübbeler, M. Marschall, E. Rühl, B. Kästner;C. Elster Measurement Science and Technology, 33 035402, 2021. [DOI: 10.1088/1361-6501/ac407a] • S. Metzner PhD Thesis 2021. • K. Klauenberg, S. Martens, A. Bošnjaković, M. Cox, A. M. van der Veen;C. Elster Measurement, 187 110340, 2021. • S. Metzner, G. Wübbeler, S. Flassbeck, C. Gatefait, C. Kolbitsch;C. Elster Physics in Medicine & Biology, 66(7), 075006, 2021. [DOI: 10.1088/1361-6560/abeae7] • G. Wübbeler;C. Elster Metrologia, 58(1), 014002, 2021. [DOI: 10.1088/1681-7575/abc97b] (online) • M. Marschall, A. Hornemann, G. Wübbeler, A. Hoehl, E. Rühl, B. Kästner;C. Elster Opt. Express, 26(28), 38762--38772, 2020. [DOI: 10.1364/OE.404959] • S. Martens, K. Klauenberg;C. Elster Teddington, United Kingdom 2020. • S. Martens, K. Klauenberg, J. Neukammer, S. Cowen, S. L. R. Ellison;C. Elster Teddington, United Kingdom 2020. • S. Martens, K. Klauenberg, B. Mickan, C. Yardin, N. Fischer;C. Elster Teddington, United Kingdom 2020. • G. Bartl, C. Elster, J. Martin, R. Schödel, M. Voigt;A. Walkov Measurement Science and Technology, 31(6), 2020. [DOI: 10.1088/1361-6501/ab7359] • J. Lehnert, C. Kolbitsch, G. Wübbeler, A. Chiribiri, T. Schäffter;C. Elster SIAM J. Imaging Sci., 12(4), 2035--2062, 2019. [DOI: 10.1137/19M1246274] • S. Metzner, G. Wübbeler;C. Elster AStA Adv Stat Anal, 103(3), 333--355, 2019. • J. Martin, G. Bartl;C. Elster Measurement Science and Technology, 30 045012, 2019. [DOI: 10.1088/1361-6501/ab094b] • J. Lehnert, G. Wübbeler, C. Kolbitsch, A. Chiribiri, L. Coquelin, G. Ebrard, N. Smith, T. Schäffter;C. Elster Physics in Medicine & Biology, 63 215017, 2018. [DOI: 10.1088/1361-6560/aae758] • G. Wübbeler, O. Bodnar;C. Elster Metrologia, 55(1), 20, 2018. [DOI: 10.1088/1681-7575/aa98aa] • C. Elster;G. Wübbeler Comput. Stat., 32(1), 51--69, 2017. • M. Dierl, T. Eckhard, B. Frei, M. Klammer, S. Eichstädt;C. Elster Journal of the Optical Society of America A, 33(7), 1370--1376, 2016. [DOI: 10.1364/JOSAA.33.001370] • C. Elster;G. Wübbeler Metrologia, 53(1), S10, 2016. • K. Klauenberg;C. Elster Metrologia, 53(1), S32, 2016. • C. Elster, K. Klauenberg, M. Walzel, P. M. Harris, M. G. Cox, C. Matthews, L. Wright, A. Allard, N. Fischer, S. Ellison, P. Wilson, F. Pennecchi, G. J. P. Kok, A. Van der Veen;L. Pendrill EMRP NEW04, 2015. • K. Klauenberg, M. Walzel, B. Ebert;C. Elster Biostatistics, 16(3), 454--64, 2015. • G. J. P. Kok, A. M. H. van der Veen, P. M. Harris, I. M. Smith;C. Elster Metrologia, 52(2), 392-399, 2015. • K. Klauenberg, G. Wübbeler, B. Mickan, P. Harris;C. Elster Metrologia, 52(6), 878--892, 2015. • O. Bodnar;C. Elster Metrologia, 51(5), 516--521, 2014. • S. Eichstädt;C. Elster Journal of Physics: Conference Series, 490(1), 012230, 2014. • S. Heidenreich, H. Gross, M.-A. Henn, C. Elster;M. Bär J. Phys. Conf. Ser., 490(1), 012007, 2014. • C. Matthews, F. Pennecchi, S. Eichstädt, A. Malengo, T. Esward, I. M. Smith, C. Elster, A. Knott, F. Arrhén;A. Lakka Metrologia, 51(3), 326-338, 2014. • G. Wübbeler;C. Elster Measurement Science and Technology, 24(11), 115004, 2013. • G. Wübbeler, F. Schmähling, J. Beyer, J. Engert;C. Elster Measurement Science and Technology, 23(12), 125004, 2012. • C. Elster;B. Toman Metrologia, 48(5), 233--240, 2011. • I. Lira, C. Elster;W. Wöger Metrologia, 44(5), 379--384, 2007.
Export as:
BibTeX, XML 