Uncertainty Representation & Propagation

José Martín-García
Algorithms R&D
Wolfram Research

Join the Conversation #WolframTechConf

Uncertainty in measurements

• Broad applicability in science and engineering, in research and industry.

• There are many potential (possibly dependent) sources of uncertainty in measurements:
	a) incomplete definition of the quantity to measure;
	b) imperfect reaIization of that definition;
	c) nonrepresentative sampling;
	d) inadequate knowledge of environment;
	e) personal bias in reading instruments;
	f) finite resolution of instruments;
	g) inexact values of standards and reference materials;
	h) inexact values of constants and parameters;
	i) approximations and assumptions incorporated in the procedure;
	j) variations in repeated observations.

• Important for uncertainty representation in WolframAlpha.

GUM

General

• Guide to the expression of Uncertainty in Measurement [web].

• Document produced by Working Group 1 of the Joint Committee for Guides in Metrology (JCGM/WG 1), following a recommendation from 1980.

• Current edition of 2010. Minor correction of the 2008, 1995 versions.

• Objectives:
	⎯ To promote full information on how uncertainty statements are arrived at.
	⎯ To provide a basis for the international comparison of measurement results.

• We need agreement in uncertainty, like the SI units brought agreement in measurement.

Main ideas

• ‘Error’ is the difference with the ‘true value’, which cannot be known. Errors can only be estimated.

• ‘Uncertainty’ is a more general concept. GUM: “Uncertainty of measurement” is the parameter, associated with the result of a measurement, that characterizes the dispersion of the values that could reasonably be attributed to the measurand.

• Types of evaluation of uncertainty: 
	- Type A: method of evaluation of uncertainty by the statistical analysis of series of observations.
	- Type B: method of evaluation of uncertainty by means other than the statistical analysis of series of observations.

• ‘Combined uncertainty’: traditional ‘error propagation’.

• ‘Extended uncertainty’: coverage factors and confidence levels.

Around

• (This is Experimental design, and may change. In particular, the name Around will probably change.)

• Basic numeric case (no distinction between type A and type B):

Around[12.34567,0.371]

FullForm[%]

Sqrt[%]

FullForm[%]

• Symbolic input:

Around[μ,σ]

Sqrt[%]

%/.{μ->12.34567,σ->0.371}

Significant figures

• The numbers of digits after decimal point of value and uncertainty must always coincide:

Around[12345.56789,0.001234]

• Following the Particle Data Group [web], we follow a ‘rule of 355’.

- If the first 3 nonzero digits of the uncertainty are above 355, round to one digit:

Around[12.345,0.356]

- If they are equal or below 355, round to two digits.

Around[12.345,0.355]

- On the upper limit, 0.95 and above is rounded to 1.0:

Around[12.345,0.949]

Around[12.345,0.950]

• Hence the maximal ‘change’ in uncertainty is 0.355... -> 0.4, which is a 13% increase. A ‘rule of 155’ would produce a 29% increase.

• The value can be smaller than its uncertainty:

Around[-0.123,0.246]

Around[0,0.246]

Around & Quantity

• Quantity of Around:

Quantity[Around[12,0.14],"Meters"]

%^2

• Around of Quantities:

Around[Quantity[12,"Meters"],Quantity[14,"Centimeters"]]

%^2

• Numbers of significant digits are still consistent, for any pair of units:

Around[Quantity[4,"Feet"],Quantity[3,"Inches"]]

UnitConvert[%,"Feet"]

UnitConvert[%%,"Inches"]

• Usual compatibility checks:

Around[Quantity[12,"Meters"],Quantity[3,"Seconds"]]

Absolute and relative uncertainty

• This is absolute uncertainty:

Around[Quantity[4,"Feet"],Quantity[3,"Inches"]]

• This is relative uncertainty, which is converted into absolute uncertainty:

Around[Quantity[4,"Feet"],Scaled[0.05]]

Uncertainty propagation (1)

• Around[μ, σ] represents dispersion distributed as a normal distribution with mean μ and standard deviation σ.

• Standard formulas for Gaussian propagation of uncertainty:

Sin[Around[μ,σ]]

• Any numeric function:

f[Around[μ,σ]]

AroundReplace[f[x],x->Around[μ,σ]]

• Addition and subtraction:

Around[2,0.3]+Around[3,0.2]

Around[2,0.3]-Around[3,0.2]

Around[(Subscript[μ, 1]),(Subscript[σ, 1])]-Around[(Subscript[μ, 2]),(Subscript[σ, 2])]

In particular:

Around[μ,σ]-Around[μ,σ]

• Products:

Around[(Subscript[μ, 1]),(Subscript[σ, 1])]Around[(Subscript[μ, 2]),(Subscript[σ, 2])]

Around[(Subscript[μ, 1]),(Subscript[σ, 1])]/Around[(Subscript[μ, 2]),(Subscript[σ, 2])]

• Powers:

Around[μ,σ]^Around[n,α]

Uncertainty propagation (2)

• Why this difference?

With[{x=Around[μ,σ]},
{x+x,2x}
]

• We need to take into account variable correlation with a covariance matrix:

AroundReplace[(Subscript[x, 1])+(Subscript[x, 2]),{(Subscript[x, 1]),(Subscript[x, 2])}->Around[{(Subscript[μ, 1]),(Subscript[μ, 2])},{(Subscript[σ, 1]),(Subscript[σ, 2])}]]

AroundReplace[(Subscript[x, 1])+(Subscript[x, 2]),{(Subscript[x, 1]),(Subscript[x, 2])}->Around[{(Subscript[μ, 1]),(Subscript[μ, 2])},{{((Subscript[σ, 1])^2),0},{0,((Subscript[σ, 2])^2)}}]]

AroundReplace[(Subscript[x, 1])+(Subscript[x, 2]),{(Subscript[x, 1]),(Subscript[x, 2])}->Around[{(Subscript[μ, 1]),(Subscript[μ, 2])},{{((Subscript[σ, 1])^2),(Subscript[σ, 1])(Subscript[σ, 2])},{(Subscript[σ, 1])(Subscript[σ, 2]),((Subscript[σ, 2])^2)}}]]

%/.{(Subscript[μ, _])->μ,(Subscript[σ, _])->σ}

• The case of two variables is specially important and has special notation: the ‘correlation factor’ -1 ≤ ρ ≤ 1 :

AroundReplace[(Subscript[x, 1])+(Subscript[x, 2]),{(Subscript[x, 1]),(Subscript[x, 2])}->Around[{(Subscript[μ, 1]),(Subscript[μ, 2])},{{(Subscript[σ, 1]),(Subscript[σ, 2])},ρ}]]

Uncertainty propagation (3)

Example [Taylor page 91, exercise 3.49]:

Take these replacement rules:

rules={p->Around[p0,δp],q->Around[q0,δq]}

simp=Simplify[#,(p0|δp|q0|δq)∈Reals]&;

This is correct:

AroundReplace[((p q)/(p+q)),rules]//simp

This is not correct. We cannot replace AroundReplace by ReplaceAll:

ReplaceAll[((p q)/(p+q)),rules]//simp

We can use ReplaceAll only if each variable appears once in the expression. We can rewrite the original as:

ReplaceAll[(1/((1/p)+(1/q))),rules]//simp

Example H.1 from GUM: End-gauge calibration

• Formula of six variables (Subscript[l, s]), d, (Subscript[α, s]), (Subscript[θ, s]), δα, δθ:

length=((Subscript[l, S])(1+(Subscript[α, S])(Subscript[θ, S]))+d)/(1+((Subscript[α, S])+δα)((Subscript[θ, S])+δθ))

• Replacement rules, following the information provided by the example:

rule[(Subscript[l, S])]=(Subscript[l, S])->Around[(Quantity[50.000623, "mm", millimeters, "Millimeters"]),(Quantity[25, "nm", nanometers, "Nanometers"])]

Combine three different sources of uncertainty for d:

Around[(Quantity[215, "nm", nanometers, "Nanometers"]),((Quantity[13, "nm", nanometers, "Nanometers"])/(Sqrt[5]))]

Around[%,((Quantity[0.01, "µm", micrometers, "Micrometers"])/2.57)]

rule[d]=d->Around[%,((Quantity[0.02, "µm", micrometers, "Micrometers"])/3)]

rule[(Subscript[α, S])]=(Subscript[α, S])->10^-6Around[(QuantityPostfix[11.5, "/""°C", reciprocal degrees Celsius, (1/("DegreesCelsius"))]),((QuantityPostfix[2, "/""°C", reciprocal degrees Celsius, (1/("DegreesCelsius"))])/(Sqrt[3]))]

rule[(Subscript[θ, S])]=(Subscript[θ, S])->Around[Around[(Quantity[-0.1, "°C", degrees Celsius, "DegreesCelsius"]),(Quantity[0.2, "°C", degrees Celsius, "DegreesCelsius"])],((Quantity[0.5, "°C", degrees Celsius, "DegreesCelsius"])/(Sqrt[2]))]

rule[δα]=δα->(10^(-6))Around[(QuantityPostfix[0, "/""°C", reciprocal degrees Celsius, (1/("DegreesCelsius"))]),((QuantityPostfix[1, "/""°C", reciprocal degrees Celsius, (1/("DegreesCelsius"))])/(Sqrt[3]))]

rule[δθ]=δθ->Around[(Quantity[0, "°C", degrees Celsius, "DegreesCelsius"]),((Quantity[0.05, "°C", degrees Celsius, "DegreesCelsius"])/(Sqrt[3]))]

• Result:

AroundReplace[length,rule/@{(Subscript[l, S]),d,(Subscript[α, S]),(Subscript[θ, S]),δα,δθ}]

The 32 nm error coincides with GUM eq. (H.6c). Copied from GUM sec. H.1.5:

Higher-order moments (actually cumulants)

• Standard formula for the propagation of uncertainty, including only first derivatives of f:

AroundReplace[f[x],x->Around[μ,σ]]

• Include second order derivatives:

AroundReplace[f[x],x->Around[μ,σ],2]//Simplify

Example showing the approximation:

AroundReplace[x^2,x->Around[μ,σ]]

AroundReplace[x^2,x->Around[μ,σ],2]

exact=With[{dist=TransformedDistribution[x^2,x\[Distributed]NormalDistribution[μ,σ]]},
{Mean[dist],StandardDeviation[dist],Skewness[dist]}]

• Specify explicitly zero skewness in input. The result has skewness as well:

around=AroundReplace[x^2,x->Around[μ,σ,0],2]

exact-List@@around//Simplify

• Finally specify explicit skewness in the input:

around=AroundReplace[x^2,x->Around[μ,σ,s],2]

• A fourth argument can also handle the fourth cumulant (“excess kurtosis”). Recall that for the normal distribution all cumulants vanish except for the first two (mean and variance).

• Currently, fourth-order derivatives and four-cumulant computations are also available for vectorial uncertainty via series expansions.

• A second order case with correlation:

AroundReplace[(Subscript[x, 1])/(Subscript[x, 2]),{(Subscript[x, 1]),(Subscript[x, 2])}->Around[{(Subscript[μ, 1]),(Subscript[μ, 2])},{{(Subscript[σ, 11]),(Subscript[σ, 12])},{(Subscript[σ, 12]),(Subscript[σ, 22])}}],2]//Simplify

• The GUM example above mentions that in such case the 2nd order terms are relevant:

Copied from GUM sec. H.1.7:

Their conclusion is:

Which we can easily reproduce just by requesting order 2:

AroundReplace[length,rule/@{(Subscript[l, S]),d,(Subscript[α, S]),(Subscript[θ, S]),δα,δθ},2]

Asymmetric uncertainty

• Different uncertainties on left and right:

Around[4.12,{0.3,0.6}]

• Mix symmetric and asymmetric uncertainties:

2+Around[3.,0.5]+Around[4.,{0.3,0.6}]

Around[3.,0.5]Around[4.,{0.3,0.6}]

• Left and right computations are independent. Compare:

{Around[2.,0.4]^2,Around[2.,0.7]^2}

Around[2.,{0.4,0.7}]^2

Visualization

• Some visualization functions already have support for representation of the error bars associated to Around. Many more things in the future...

random:=Around[RandomReal[20],RandomReal[{0.5,1}]]

ListPlot[Table[random,10]]

ListLinePlot[Table[random,10]]

BarChart[Table[Around[RandomReal[20],1],10]]

Future

• Native understanding of coverage factors (“3 sigma”) and confidence levels (“95% confidence level”).

• Treatment of complex numbers and possible correlation between their real and imaginary parts.

• Extension to dates.

• Relation with fitting functionality.

If you have criticisms, comments or suggestions, please let us know!

Thanks!