Precision Engineering 63 (2020) 1–9

Contents lists available at ScienceDirect

Precision Engineering
journal homepage: [Link]/locate/precision

Uncertainty model for a traceable stereo-photogrammetry system

Danny Sims-Waterhouse ∗, Mohammed Isa, Samanta Piano, Richard Leach
Manufacturing Metrology Team, University of Nottingham, Nottingham, United Kingdom

ARTICLE INFO ABSTRACT

Keywords: Through the computational modelling and experimental verification of a stereo-photogrammetry system, the
Photogrammetry expanded uncertainty on form measurement was estimated and was found to be 32 μm, 12 μm and 29 μm
Error modelling for a 95% confidence interval (coverage factors of 𝑘 = 3.2, 2.0 and 2.0 respectively) in the x, y and
Calibration
z axis respectively. The contribution of systematic offsets in the system properties was also investigated,
Traceability
demonstrating the complex distortions of the measurement volume that result from these systematic errors.
Additionally, a traceable method of applying a scale factor to the reconstruction was demonstrated using
a laser interferometer and gauge block. The relative standard uncertainty on the size of the measurements
was estimated to be 0.007% corresponding to length measurement uncertainties of around 7 μm over a
100 mm range. Finally, the residuals from a linear fit of the scale factor were found to exhibit behaviour
that would be expected to result from small offsets in the system properties. The outcome of this work is a
better understanding of the propagation of uncertainty through the stereo-photogrammetry system as well as
highlighting key influence factors that must be addressed in future work.

1. Introduction of the photogrammetry system allowing the correct scale factor to be

applied after the measurement is taken. Although this approach is easily
Photogrammetry is a passive form measurement technique based applied, it greatly limits the working volume of the system and requires
on the triangulation of common features in multiple images [1]. The a calibrated artefact to be available for all measurements.
strength of photogrammetry lies in its capacity to achieve reconstruc- Another common approach, in the case of a multi-cameras sys-
tion without relying on any additional information other than the tem, is to fix the physical distance between the multiple cameras
images of the object to be measured. Despite the advantages of re- within the system and apply that physical property to the measure-
constructing geometries from images, this can only be achieved at the ment reconstruction [5]. This is a typical approach in many optical
loss of scale information [2]. Additionally, the complex and non-linear form measurement techniques, particularly in structured light-based
process of characterising the system properties and triangulation of the measurement systems [6]. The calibrated artefact must be measurable
sample form makes the propagation of error through the system diffi-
by the system, as well as traceably calibrated to a sufficient level
cult to calculate [3]. The lack of a standardised methodology for scaling
of accuracy for the system under consideration. For contact-based
and consequent lack of traceability, plus a poor understanding of the
systems, gauge blocks are easily used and widely available whilst being
uncertainty of photogrammetry measurements, are significant hurdles
traceably calibrated to very high accuracies (<0.06 μm for grade K
in the widespread use of photogrammetry as a form measurement
gauge blocks) [7]. However, optical form measurement systems are
system.
typically not able to measure steel gauge blocks due to their highly
specular surfaces [8]. Instead, two dimensional patterned artefacts,
1.1. Arbritrary scale
such as chequerboards or dot patterns, are typically used in order to
calibrate optical form measurement systems [9]. While chequerboard
The solution to the arbitrary scale of photogrammetry reconstruc-
artefacts [9] may be effective for the calibration of optical form mea-
tions is a key area of research into the metrological applications of
photogrammetry methods. Typically, some physical property of the surement systems, they are difficult to calibrate to high accuracies (<1
measurement system and/or the sample must be fixed and calibrated μm) and can be difficult to use for non-experts. Ball bars are also a
in order for measurements to be appropriately scaled [4]. For example, widely used approach for the calibration of optical form measurement
a calibrated artefact can be placed within the measurement volume systems [10], with the sphere-to-sphere distances providing the scaled

∗ Corresponding author.
E-mail address: [Link]-waterhouse@[Link] (D. Sims-Waterhouse).

[Link]
Received 12 September 2019; Received in revised form 12 December 2019; Accepted 16 December 2019
Available online 18 December 2019
0141-6359/© 2020 Elsevier Inc. All rights reserved.
D. Sims-Waterhouse et al. Precision Engineering 63 (2020) 1–9

length. Similar to two-dimensional patterned artefacts, ball bars are Table 1

Contributions to the measurement uncertainty.
difficult to use for calibration as they exhibit very little observable
surface texture for photogrammetry measurements, although texture Influence factor Evaluation Distribution Designation
method
projection methods have been shown to make this feasible [10].
Focal length Type A Rectangular 𝑢(𝑓𝑢 ), 𝑢(𝑓𝑣 )
An alternative method to effectively calibrate optical form mea-
Principle point Type A Rectangular 𝑢(𝑢0 ), 𝑢(𝑣0 )
surement systems is to use a combination of self-calibration and direct Radial distortion Type A Rectangular 𝑢(𝑘1 ), 𝑢(𝑘2 ), 𝑢(𝑘3 )
measurement of a single calibrated length [10]. Self-calibration does Tangential distortion Type A Rectangular 𝑢(𝑠1 ), 𝑢(𝑠2 )
not require a calibrated object to effectively characterise the optical Camera rotation Type A Rectangular 𝑢(𝜃), 𝑢(𝜙), 𝑢(𝜌)
system up to some arbitrary scale factor. By then measuring some Camera translation Type A Rectangular 𝑢(𝑇𝑥 ), 𝑢(𝑇𝑦 ), 𝑢(𝑇𝑧 )
Feature location Type B Normal 𝑢(𝑢𝑝 ), 𝑢(𝑣𝑝 )
single calibrated length, such as a gauge block, the arbitrary scale
Temperature of system Type A Rectangular 𝑢(𝑇 𝑒𝑚𝑝)
factor of the system can be fixed. Although gauge blocks are typically Measurement Type B Rectangular 𝑢(𝑐𝑎𝑙)
difficult to measure with optical form measurement systems, this issue uncertainty of the laser
is simplified as only a single measurement is required: in fact, limiting interferometer
the required number of measurements ensures that the gauge block can
be measured in the most optimal position, reducing specular reflection
and maximising the measurement accuracy. This methodology is more factors: geometric uncertainty and global scale uncertainty factors.
easily implemented as an intermittent calibration procedure for optical The outcome of this work is an understanding of the measurement
measurement systems whilst only requiring widely available calibration uncertainty of the system in question, as well as the identification of
artefacts. key areas for future work in the improvement of the technology.

1.2. Existing standards and methods 2. Methodology

The understanding of error propagation through contact-based coor- The methodology of this work falls into four main stages: 1. evalu-
dinate measurement machines (CCMs) is already well established [11, ation of the measurement influence factors, 2. Monte Carlo simulation
12]. However, the propagation of errors through optical form mea- of the stereo-camera system, 3. evaluation of the global scale factor and
surements is still a poorly understood area of research [3,13–16]. 4. verification through gauge block measurement.
The non-linear nature of optical form measurement systems makes the
analytical approach for the evaluation of the combined standard uncer- 2.1. Evaluation of the measurement influence factors
tainty highly complex. The non-linearity of optical methods arises for
several reasons, such as choice of triangulation method [17] and optical The influence factors were evaluated through two methods, type A
distortion [4]. Various triangulation methods do not rely on a linear and type B, as defined by the Guide to the Expression of Uncertainty
analytical solution, but instead use iterative approaches to minimise in Measurement [19]. A list of the influence factors, their evaluation
reprojection errors. Optical distortion is also particularly problematic method, their distribution and their designation can be seen in Table 1.
as there is no analytical solution to the inverse mapping of distorted The influence factors to be evaluated through the type A method
camera points to their undistorted position [17]. correspond to the intrinsic and extrinsic properties of the stereo-camera
A common approach to the assessment of measurement uncertainty system and are evaluated through a repeated calibration procedure. The
in photogrammetry systems is to use comparison with other, more ac- uncertainties of the intrinsic and extrinsic properties are given by the
curate, measurement technologies. For example, Lavecchia et al. used a characterisation algorithm [9]. As the distribution of the stereo-camera
white light interferometer to compare photogrammetry measurements parameters is unknown, a rectangular distribution has been selected
to a more accurate measurement of a step height [3]. Although com- for these influence factors in order to provide a conservative estimate
parison with more accurate measurements provides some insight into of their contribution. However, through the characterisation process of
the accuracy of the system, it is limited in the information that can be the system, it can be seen that the distribution of pixel re-projection
extracted about how errors propagate through the system. Comparison errors follows a normal distribution. Therefore, a normal distribution
with other measurements only allows the user to observe the relative for the feature location can be used within the Monte Carlo simulation.
effects of changing certain aspects of the system, such as camera angles Unlike type A evaluated uncertainties, type B uncertainties are
and reconstruction methods. However, measurement comparisons do instead evaluated using prior results, manufacturer specifications and
not provide any insight into the effect of the uncertainties of the system experience of using the system. The key parameters to be evaluated
properties that cannot be as easily altered. For example, the require- using the type B method are the uncertainty in the laser interferometer
ments on the mechanical stability of the system and the performance of measurements and environmental factors. However, the most important
different cameras can be tested prior to the system being manufactured. factor for the type B evaluation is the uncertainty on the feature loca-
Monte Carlo (MC) based methods provide an ideal solution for tion. The uncertainty on the feature location is in fact a combination
the evaluation of the combined standard uncertainty for such complex of a wide range of factors; such as, focus, feature detection method,
systems [18]. Essentially, the Monte Carlo approach involves the simu- feature size, depth-of-field, chromatic aberrations, sensor resolution
lation of many measurements for which the influence factors are varied and surface properties. Whereas all the other influence factors can
according to their predetermined distribution. The distribution of the be explicitly defined, the feature location uncertainty will be consid-
simulated measurand can then be used to evaluate the combined stan- ered for a range of values typically expected in order to provide an
dard uncertainty as whole, as well as the individual contributions from indication of the measurement uncertainty in a variety of scenarios.
each influence factor. Di Leo et al. show that through MC simulations,
it is possible to investigate the effect of varying system properties and 2.2. Monte Carlo simulation of the stereo-camera system
isolating the effect of various parameters [13].
The aim of the work presented in this paper is to model a stereo- The stereo-photogrammetry system consists of two cameras for
camera based photogrammetry system in order to assess the expected which a three-dimensional grid of points is imaged using the ideal
measurement uncertainties as well as provide a traceable method of system and then triangulated using the perturbed system. The per-
scaling subsequent measurements. The scaling will be performed using turbed system is produced by applying an offset to influence factors
a laser interferometer and verified using a calibrated gauge block. The under investigation according to the probability distribution of that
measurement uncertainty is considered as made up of two separate factor. The spatial deviation of the triangulated points can then be

2
D. Sims-Waterhouse et al. Precision Engineering 63 (2020) 1–9

Fig. 1. Schema of the projection of a world point onto two images.

used to evaluate the accuracy of the measurement. The projection of 2.2.1. Intrinsic parameters
three-dimensional points onto the camera sensor is described by [4] The intrinsic properties of the stereo-photogrammetry system in-
clude the information about the internal properties of the camera, such
⎡𝑋 ⎤ ⎡𝑋𝑚 ⎤
⎡𝑢𝑛,𝑚 ⎤ [ ] ⎢ 𝑚⎥ ⎢ ⎥ as focal length, sensor size, sensor position and the distortion effect of
𝑅𝑛 𝑇𝑛 ⎢ 𝑌𝑚 ⎥ 𝑌
𝑠𝑚 ⎢𝑣𝑛,𝑚 ⎥ = 𝐾𝑛 = 𝑃𝑛 ⎢ 𝑚 ⎥ (1) the imaging lens. The application of the intrinsic parameters is shown
⎢ ⎥ 0 0 0 1 ⎢ 𝑍𝑚 ⎥ ⎢ 𝑍𝑚 ⎥
⎣ 1 ⎦ ⎢ 1 ⎥ ⎢ 1 ⎥ in Eq. (1) in terms of 𝐾 given by
⎣ ⎦ ⎣ ⎦
⎡𝑓𝑢 0 𝑢0 ⎤
where (𝑢(𝑚,𝑛) , 𝑣(𝑚,𝑛) ) are the pixel coordinates of the point 𝑚 in camera
𝐾 =⎢0 𝑓𝑣 𝑣0 ⎥ , (3)
𝑛, 𝑠𝑚 is a normalisation factor, 𝐾𝑛 is the intrinsic matrix for camera 𝑛, ⎢ ⎥
⎣0 0 1⎦
𝑅𝑛 and 𝑇𝑛 are the rotation and translation matrices of camera 𝑛, 𝑃𝑛 is
the camera projection matrix and (𝑋𝑚 , 𝑌𝑚 , 𝑍𝑚 ) are the coordinates of where 𝑓𝑢 and 𝑓𝑣 describe the focal length of the lens in the u and v
point 𝑚. A schema of the projection process is shown in Fig. 1, where direction respectively and 𝑢0 and 𝑣0 are the pixel coordinates of the
the cube represents the measurement volume. intersection of the optical axis with the imaging plane. The inclusion of
In addition to the linear projection process, there is an additional uncertainty within the intrinsic factors is straightforward as the offset
stage in which distortion parameters are applied in order to account value can be generated according to a rectangular distribution and
for the non-linear distortion of the camera optics. As only the image added to each term. However, as the principle point is mainly a term
pixel coordinates and camera intrinsic and extrinsic properties will be for the characterisation of the centre of distortion, the uncertainty term
known, a minimum of two cameras is required in order to triangulate is only added to the principle point when calculating the distortion
the position of the point in three-dimensional space. The linear solution correction.
to the triangulation is described by
⎡𝑢𝑚, 1𝑝31 − 𝑝11 ⎤ 2.2.2. Distortion
⎡𝑋𝑚 ⎤ ⎢𝑣𝑚, 1𝑝3 − 𝑝2 ⎥ ⎡𝑋𝑚 ⎤ The distortion of the imaged object is a result of several factors,
⎢ 1⎥ ⎢
𝐴 ⎢ 𝑌𝑚 ⎥ = ⎢ 1
𝑌 ⎥, (2) including the optical effects of the lenses and misalignment of the
⎢ ⎥ ⎢𝑢𝑚, 2𝑝3 − 𝑝1 ⎥⎥ ⎢ 𝑚 ⎥ optical components. The two key forms of distortion are the radial and
⎣𝑍𝑚 ⎦ 2 2 ⎣ 𝑍𝑚 ⎦
⎢ 3 ⎥ tangential components [4]. The influence of the tangential distortion
⎣𝑣𝑚, 2𝑝2 − 𝑝22 ⎦
was found to be lower than 0.1 pixels and substantially increased the
where 𝑝𝑘𝑛 is row 𝑘 of the 𝑛 camera projection matrix. The triangulated calculation times for the Monte Carlo simulations. For the scope of this
point can then be calculated as a solution to Eq. (2) through the work, only the radial component of the distortion will be considered in
calculation of the singular value decomposition of A. The triangulated order to simplify and accelerate the simulations. The radial distortion
point is then given as the smallest singular value of 𝐴. of image points is described by
Once the measured points have been triangulated, they are shifted
in order to ensure their centre of mass coincides with that of the 𝑢𝑑 = 𝑢(1 + 𝑘1 𝑟2 + 𝑘2 𝑟4 + 𝑘3 𝑟6 + ⋯), (4)
original grid of points, removing any systematic effects. The difference 𝑣𝑑 = 𝑣(1 + 𝑘1 𝑟2 + 𝑘2 𝑟4 + 𝑘3 𝑟6 + ⋯), (5)
between the triangulated points and the original location is then cal-
culated. This process is then repeated thousands of times with varying where (𝑢𝑑 , 𝑣𝑑 ) is the distorted
√ image coordinate of the points (𝑢, 𝑣), 𝑟 is
system properties in order to assess the expected response of the system. the radius given by 𝑟 = (𝑢 − 𝑢0 )2 + (𝑣 − 𝑣0 )2 and 𝑘𝑛 is the 𝑛th distortion
The details of how each property is perturbed will be discussed in the parameter. Typically, in order to determine the undistorted image
following sections. coordinates from the known distorted points and distortion parameters,

3
D. Sims-Waterhouse et al. Precision Engineering 63 (2020) 1–9

a non-linear solution is required. However, it is possible to define an

inverse distortion given by

𝑢𝑑 = 𝑢(1 + 𝑏1 𝑟2𝑑 + 𝑏2 𝑟4𝑑 + 𝑏3 𝑟6𝑑 + ⋯), (6)

𝑣𝑑 = 𝑣(1 + 𝑏1 𝑟2𝑑 + 𝑏2 𝑟4𝑑 + 𝑏3 𝑟6𝑑 + ⋯), (7)

where 𝑟𝑑 is the radius of the distorted points and 𝑏𝑛 is the 𝑛th inverse
distortion parameter [20]. By defining the inverse distortion, the offset
to the undistorted image points due to a difference in inverse distortion
parameters is given by the equation

𝛥𝑢 = 𝑢1 − 𝑢2 = 𝑢𝑑 (1 + 𝑏1,1 𝑟2𝑑 + 𝑏1,2 𝑟4𝑑 ) − 𝑢𝑑 (1 + 𝑏2,1 𝑟2𝑑 + 𝑏2,2 𝑟4𝑑 ), (8)

which can be further simplified

Fig. 2. Scale factor measurement set-up. (a) Stereo-photogrammetry system, (b) gantry
𝛥𝑢 = 𝑢𝑑 (𝛥𝑏1 𝑟2𝑑 + 𝛥𝑏2 𝑟4𝑑 ), (9) plate, (c) chequerboard artefact, (d) retro-reflector, (e) beam splitter and (f) laser
interferometer.
where 𝛥𝑏𝑛 is the offset on the 𝑛th inverse distortion parameter and 𝛥𝑢
is the corresponding offset to the pixel position. The conversion from
distortion parameters to the inverse distortion parameters is given by 𝑇𝑧
𝑇𝜃 = 𝑎𝑟𝑐𝑐𝑜𝑠 , (18)
𝑇𝑟
𝑏1 = −𝑘1 , (10) 𝑇𝑦
𝑇𝜙 = 𝑎𝑟𝑐𝑡𝑎𝑛 , (19)
𝑏2 = 3𝑘21 − 𝑘2 , (11) 𝑇𝑥
𝑏3 = −12𝑘31 + 8𝑘1 𝑘2 − 𝑘3 , (12) where 𝑇𝑟 , 𝑇𝜃 and 𝑇𝜙 is the radius, elevation and azimuth of the second
camera with respect to the first camera. As the magnitude of 𝑇𝑟 will
𝑏4 = 55𝑘41 − 55𝑘21 𝑘2 + 5𝑘22 + 10𝑘1 𝑘3 − 𝑘4 . (13) only affect the effective scale of the reconstruction, its uncertainty is
Higher order terms for b can be defined, but four were used as taken to be zero for the Monte Carlo simulation and will instead be
evaluated in Section 2.4. Instead only the angular uncertainty of the
the contribution from higher orders is negligible. Using Eqs. (10)–(13),
second camera is considered using the equations
it is possible to calculate the uncertainty in each inverse distortion
parameter using 𝑇𝑧 𝑇𝑥 𝑇𝑧 𝑇𝑦
𝑢2 (𝑇𝜃 ) = ( √ )2 𝑢2 (𝑇𝑥 ) + ( √ )2 𝑢2 (𝑇𝑦 )
√
√ 4 𝑇𝑧2 𝑇𝑧2
√∑ 𝛿𝑏 𝑇𝑟3 1 − 𝑇𝑟3 1 −
𝑈 (𝑏𝑛 ) = √ ( 𝑛 )𝑈 2 (𝑘𝑖 ).
𝑇𝑟2 𝑇𝑟2
(14)
𝑖=1
𝛿𝑘𝑖 𝑇𝑥2 + 𝑇𝑦2
+( √ )2 𝑢2 (𝑇𝑧 ), (20)
By using the associated uncertainties for the distortion parameters, 𝑇𝑧2
3
𝑇𝑟 1 − 2
the uncertainty on each inverse distortion parameter can be calculated 𝑇𝑟
and used to generate a normally distributed random offset to each
parameter. With the random offsets to the inverse distortion parameters 𝑇𝑥2 𝑇𝑦
𝛥𝑏𝑛 , the offset in the pixel position can then be computed using Eq. (9). 𝑢2 (𝑇𝜙 ) = ( )2 𝑢2 (𝑇𝑥 ) + ( )2 𝑢2 (𝑇𝑦 ). (21)
𝑇𝑥2 + 𝑇𝑦2 𝑇𝑥2 + 𝑇𝑦2

2.2.3. Extrinsic properties According to the angular uncertainties calculated using Eqs. (20)
The extrinsic properties of the stereo-photogrammetry system corre- and (21), normally distributed random angular offsets are added to
the spherical polar coordinates of the second camera before being
spond to the relative orientation between the two cameras. Physically,
converted back to Cartesian coordinates.
this represents the rotation and translation between the two cameras.
For this work, one camera was fixed in space at the origin, with
2.3. Evaluation of the global scale factor
the second camera being subject to random perturbations. The rota-
tional component to this transformation is implemented by applying a The scale factor of the stereo-photogrammetry system was evaluated
rotation matrix to the second camera in the form using a high-accuracy laser interferometer (Renishaw XL-80). As can
𝑅 = 𝑅𝑥 (𝜃)𝑅𝑦 (𝜙)𝑅𝑧 (𝜌), (15) be seen in Fig. 2, the stereo-photogrammetry system was used to track
a chequerboard artefact placed on the gantry plate of a linear actua-
where 𝑅𝑥 (𝜃) is a rotation of 𝜃 around the 𝑥-axis, 𝑅𝑦 (𝜙) is a rotation of tor. The linear actuator was then used to translate the chequerboard
𝜙 around the 𝑦-axis and 𝑅𝑧 (𝜌) is a rotation of 𝜌 around the 𝑧-axis. The artefact over a 90 mm travel range whilst the linear position was
angles 𝜃, 𝜙 and 𝜌 are randomly generated from a normal distribution monitored using the laser interferometer. The linear motion observed
determined by the uncertainty on those angles and converted into a by the stereo-photogrammetry system could then be compared to the
rotation matrix. This rotational offset matrix is then applied to the linear distance provided by the laser interferometer in order to correct
original matrix in order to produce the perturbation in rotation. for scale factor errors introduced by the chequerboard calibration
The translation matrix between the cameras is of the form procedure.
The laser interferometer system utilises a refractive index corrector
⎡𝑇𝑥 ⎤ unit and has a stated relative accuracy of ±5 × 10−5 %, corresponding to
𝑇 = ⎢ 𝑇𝑦 ⎥ , (16)
⎢ ⎥ a maximum error of around 45 nm over the complete range of motion
⎣ 𝑇𝑧 ⎦ used in this work (100 mm). The uncertainty on the calculated scale
where 𝑇𝑥 , 𝑇𝑦 and 𝑇𝑧 are the translations of the second camera in the 𝑥, factor is given by
𝑦 and 𝑧 axis, respectively. However, as the magnitude of the translation √
𝑢(𝑙𝑒𝑛𝑔𝑡ℎ) = 𝑢2 (𝑐𝑎𝑙) + 𝑢2 (𝑝), (22)
matrix will only lead to a scale factor error, it is useful to isolate the
magnitude and direction of the translation uncertainty. The camera where 𝑢(𝑐𝑎𝑙) is the uncertainty of the laser interferometer measure-
translation can be represented in a spherical-polar coordinate system ments and 𝑢(𝑝) is the uncertainty of the stereo-photogrammetry mea-
according to the equations surements. The uncertainty of the stereo-photogrammetry measure-
√ ments was evaluated by carrying out 20 repeat measurements through
𝑇𝑟 = 𝑇𝑥2 + 𝑇𝑦2 + 𝑇𝑧2 , (17) the entire travel range of 100 mm.

4
D. Sims-Waterhouse et al. Precision Engineering 63 (2020) 1–9

Table 2
Measurement standard deviations for each factor.
Influence factor Value DoF Uncertainty /μm
x y z
𝑢(𝑢𝑝 ), 𝑢(𝑣𝑝 ) 0.1 (pixels) ∞ 3.1 2.3 6.7
𝑢(𝑓𝑢 ), 𝑢(𝑓𝑣 ) 0.82, 0.84 (pixels) 2244 1.7 1.3 3.6
𝑢(𝜃), 𝑢(𝜙), 𝑢(𝜌) 1.84, 1.52, 14 9.1 4.7 9.1
0.71 (∗ 10−4 radians)
𝑢(𝑇𝑥 ), 𝑢(𝑇𝑦 ), 𝑢(𝑇𝑧 ) 13, 8, 27 (μm) 14 1.5 0.4 3.2
𝑢(𝑘1 ), 𝑢(𝑘2 ) 0.001, 0.0362 2244 2.1 2.5 8.2

Fig. 3. Standard deviation of errors in 𝑥 axis against number of iterations.

2.4. Verification through gauge block measurement

In order to verify the performance of the system, a calibrated 5 mm

gauge block that has been wrung to a reference platen was measured.
Although the gauge block exhibits a highly specular surface, a laser line
projected on to the surface provides sufficient diffusely reflected light
to triangulate points on both the gauge block and platen surface. As
the laser line does not provide a pixel-to-pixel correspondence, a line
was fit to the two surfaces and the epipolar geometry [16] of the stereo- Fig. 4. Plot of average uncertainty in the 𝑥 axis against level of feature location
uncertainty.
system was used to extract the corresponding points. The measurement
was performed five times with the stereo-cameras rigidly mounted with
respect to the gauge block, with the laser line scanned over the surface
in order to provide enough points to fit a plane to the data. resolution of 33 μm due to the pixel pitch of 1.85 μm. However, an f-
stop value of 6 was used in order to provide a sufficient depth-of-field,
3. Results leading to diffraction limited system with an airy disc with a radius
of 8 μm. The diffraction limitation leads to an optical resolution of
3.1. Software model results 0.145 mm at 301 mm away from the camera. The first and second order
radial distortion parameters for the cameras were found to be −0.135
3.1.1. Verification of distribution and 0.24 for the first camera and −0.13 and 0.17 for the second.
In order to ensure that the number of iterations was sufficient to Table 2 shows the values of the influence factors that were evalu-
accurately represent the measurement uncertainty, the first step was ated, as well as their contribution to the combined standard uncertainty
to evaluate the standard deviation of the three-dimensional coordinate in the 𝑥, 𝑦 and 𝑧 axes. The combined standard uncertainty is calculated
error as a function of number of iterations. Fig. 3 shows the mean for each axis as
√
standard deviation of the position errors in the 𝑥 axis as a function √𝑁
√∑
of number of iterations. As Fig. 3 shows, the variation in the uncer- 𝑢𝑐𝑜𝑚𝑏 = √ 𝑢2 , 𝑛 (23)
tainty very quickly drops to less than 2 nm after only 1000 iterations. 𝑛=1
Between 4000 and 10 000 iterations, the standard deviation of the in which 𝑢𝑛 is the uncertainty contribution of influence factor 𝑛 and
results falls below 1 nm. As the results are calculated as the mean 𝑁 is the total number of influence factors. The combined standard
of over 1000 points within the measurement volume, the effective uncertainty for the contributions in Table 2 was 10.1 μm, 6.0 μm and
number of iterations is a thousand times more than the actual number 14.8 μm in the 𝑥, 𝑦 and 𝑧 axis, respectively.
of iterations. 10 000 iterations were performed in order to ensure the The Degrees of Freedom (DoF) for the influence factors are also
extracted values are as representative as possible. shown in Table 2. Using the Welch–Satterthwaite equation, the DoF on
the combined standard uncertainties was found to be 3, 37 and 96 giv-
3.1.2. Contributions of each factor ing coverage factors of 3.2, 2.0 and 2.0 for a 95% confidence interval.
Each influence factor from Table 1 was evaluated through the Based on the coverage factors, the expanded standard uncertainties are
calibration of the stereo-photogrammetry system using a chequerboard 32.1 μm, 12.1 μm and 29.2 μm for a 95% confidence interval.
calibration artefact. The cameras used for this work were 12 mega-
pixel sensors (4024 pixels by 3036 pixels) with 16 mm lenses that 3.1.3. Influence of feature uncertainty
were found to be 16.74 mm and 16.66 mm when calibrated. The In order to assess the effect of feature location uncertainty, the
stereo-photogrammetry was found to have a baseline distance between Monte Carlo simulation was performed for ten different feature location
the cameras of 199.35 mm and an angle of 38.64◦ between cameras. errors between 0 and 1 pixels. The choice in range of pixels to simulate
This baseline and camera angle leads to a convergence point of the was chosen arbitrarily to reflect of values that would be typically
two camera views at 301 mm away from each camera. At 301 mm, expected. As can be seen in Fig. 4, the relationship between pixel error
the magnification of each camera is around 0.055, with an optical and uncertainty in the 𝑥 axis is approximately linear.

5
D. Sims-Waterhouse et al. Precision Engineering 63 (2020) 1–9

Fig. 5. Spatial variation of errors in 𝑥, 𝑦 and 𝑧 axis in the presence of an offset in the Fig. 7. Spatial variation of errors in 𝑥, 𝑦 and 𝑧 axis in the presence of an offset in the
focal length. rotation of the second camera.

Fig. 8. Spatial variation of errors in 𝑥, 𝑦 and 𝑧 axis in the presence of an offset in the
Fig. 6. Spatial variation of errors in 𝑥, 𝑦 and 𝑧 axis in the presence of an offset in the radial distortion parameters.
translation of the second camera.

the measurement volume are all observed for individual factors, with
3.1.4. Systematic error contribution
real measurements being the result of a combination of all the observed
Although the uncertainties in the intrinsic and extrinsic stereo-
effects.
camera parameters have been assumed to be randomly distributed,
they represent the uncertainty in the characterisation process. For any
3.1.5. Spatial variation of measurement uncertainty
given measurement, the errors on the system parameters are likely to
In addition to the spatial variation caused by systematic errors,
be mainly systematic, with a substantially smaller random component there is also a variation of the combined standard uncertainty within
representing the mechanical stability of the system. As a result, it the measurement volume. An additional Monte Carlo simulation was
is useful to visualise how systematic offsets will propagate through performed for a line of points in the 𝑥 axis and the standard deviation
the measurement and to see how they manifest in the measurement in each axis was calculated for 100 000 iterations. As can be seen in
errors. However, as the uncertainties cannot be characterised easily, Fig. 9, the triangulation uncertainty shows significant variation through
the combined standard uncertainties quoted in Section 3.1.2 represent a the measurement volume. Variations of the order of 5 μm can be seen in
realistic expectation of what can be achieved. However, it is important both the 𝑦 and 𝑧 axes, leading to substantial variations in the combined
to understand how systematic errors in the extrinsic and intrinsic standard uncertainty throughout the measurement volume.
properties manifest. In order to demonstrate the effect of systematic
offsets, the computational model was used to simulate the triangulation 3.2. Scaling factor results
errors in the presence of offsets applied to the system properties. As
can be seen in Figs. 5–8, systematic offsets in the system properties The uncertainty on the scale factor of the stereo-photogrammetry
lead to complex distortions in the measurement volume. Spatial offsets, system was evaluated using a linear actuator tracked with a laser inter-
differences in scale factor for each axis and non-linear distortions in ferometer. A chequerboard artefact was placed on the linear actuator

6
D. Sims-Waterhouse et al. Precision Engineering 63 (2020) 1–9

Fig. 9. Spatial variation of average uncertainty in 𝑥, 𝑦 and 𝑧 axis for random errors. Fig. 11. Residual error of photogrammetry measurements.

Fig. 12. Point cloud of gauge block points (Red points correspond to the gauge block
surface and blue to the platen surface).

3.3. Gauge block measurement

Fig. 10. Plot of translation of stage according to PG and the laser interferometer.
The calibrated gauge block was measured using a laser line to
provide correspondance for the stereo-photogrammetry system. As can
and tracked with the stereo-photogrammetry system. By then taking the be seen in Fig. 12, the gauge block measurement was split in order
average translation between the various chequerboard positions and to provide points for a plane fit to be applied to the upper and lower
comparing with the interferometer data, the scale factor of the system surfaces. Once orientated with the platen surface on the 𝑧 axis plane,
could be determined. the standard deviations and heights of each surface were calculated.
The gauge block and platen points were found to have standard devia-
Fig. 10 shows the stereo-photogrammetry based displacement, com-
tions from a fitted plane of 32 μm and 27 μm respectively. According
pared to the interferometer translation for two different orientations of
to the stereo-photogrammetry measurement, the gauge block height
the actuator in the measurement volume. A linear regression was then
was found to be 5.0086 mm, compared to the calibrated value of
used to determine the scale factor for each repeat measurement with
5.0006 mm, resulting in a length measurement error of 8 μm.
a standard deviation of the scale factor over all 20 measurements of
0.0068%. The uncertainty on the scale factor corresponds to a mea-
4. Discussion
surement uncertainty contribution of 6.8 μm on a length measurement
of 100 mm.
4.1. Software model results
A result of the scale factor data is the shape of the plot of the linear
displacement error for the photogrammetry system. Fig. 11 shows the The results from the Monte Carlo simulations provide information
displacement errors for twenty repeat measurements of a 70 mm travel about the importance of specific factors when considering improve-
range with associated uncertainty on those points. As can be seen, ments of a measuring system. As Table 2 shows, the rotation and
there is a highly repeatable pattern in the displacement data, likely translation between the two cameras are by far the largest influence
corresponding to the systematic offsets discussed in Section 3.1.4. In factor to the triangulation uncertainty. With a combined standard
fact, the standard deviation on any one point was typically on the order uncertainty of 32.1 μm, 12.1 μm and 29.2 μm for the 𝑥, 𝑦 and 𝑧 axes
of 5 μm, despite a lack of accurate alignment between measurements. respectively, the contribution due to most of the properties is on the

7
D. Sims-Waterhouse et al. Precision Engineering 63 (2020) 1–9

order of 5 μm, demonstrating the impact of the uncertainty on the intrinsic properties are likely systematic offsets, rather than randomly
camera rotation and distortion parameters. However, the rotation and varying. Additionally, the error in the length measurement is somewhat
distortion properties were only the main factors for the case of feature higher than would be expected, based on the scale factor uncertainty.
localisation uncertainties of 0.1 pixels. Fig. 4 shows the effect that However, as can be seen in Fig. 11, non-linear distortions of the
feature location uncertainty has on the standard uncertainty. Although measurement volume could explain the 8 μm error in the gauge block
the standard uncertainty is below 10 μm at 0.1 pixels, there is a linear measurement. In order to verify this, repeat measurements of the gauge
increase up to around 35 μm for a feature location uncertainty of around block will be taken throughout the measurement volume.
1 pixel. Although in ideal circumstances 0.1 pixels of uncertainty is
obtainable, the vast number of contributions to the feature location 5. Conclusion
uncertainty mean that, in reality, there will be a distribution of feature
uncertainties within a measurement. The contrast of visible surface This paper presents the first attempt at an uncertainty analysis of a
texture, focus and the feature detection algorithm used all have an stereo-photogrammetry system, as well as a method for the traceable
impact on the feature uncertainty and depend on a range of internal and calibration of the photogrammetry scale factor. The outcome of this
external factors, making the true value of feature location uncertainty work is an estimate of the expanded measurement uncertainty of a
difficult to quantity. stereo-photogrammetry system, as well as the identification of key
Although the rotational extrinsic properties were shown to be one influence factors that must be addressed in future work. Addition-
of the main contributing factors in the combined standard uncertainty, ally, this work has also provided insight into the manifestation of
it was also considered that in reality, the majority of the error con- characterisation uncertainty within a triangulation-based measurement
tributions from the intrinsic and extrinsic factors would be systematic
system. Systematic errors in system properties lead to highly complex,
rather than random. As such, the effect of systematic offsets to the
non-linear effects in the resultant point cloud. In future work, the ad-
focal length, translation, rotation and distortion factors were modelled
justment of systematic effects will be addressed as well as determining
in order to understand their effect on the triangulated point cloud. As
a methodology for calculating anticipated feature uncertainty.
can be seen in Figs. 4–8, the effects of systematic errors manifest as
complex distortions to the triangulated point cloud, often being non-
Declaration of competing interest
linear. Effectively, these distortions will result in length measurement
being affected by both position and orientation within the measurement
volume, being stretched and skewed in different axes. Although the The authors declare that they have no known competing finan-
values in Table 2 demonstrate the effect of a random variation, they cial interests or personal relationships that could have appeared to
give some indication of the possible contribution of systematic offsets influence the work reported in this paper.
to the measurement uncertainty.
In addition to the spatial variation caused by systematic offsets, Acknowledgements
there is also a spatial variation to the combined measurement uncer-
tainty for purely random influence factors. The nature of photogram- The authors would like to thank EPSRC, United Kingdom (Grants
metry means that the uncertainty on a triangulated point will vary EP/M008983/1 and EP/L0165 67/1) for funding this work as well as
throughout the measurement volume, as shown in Fig. 9. Although the Chien Jakub Nguygen Van for the design of the stereo-photogrammetry
effect is relatively small, Fig. 9 shows that the combined uncertainty system and the National Physical Laboratory for calibrating and supply-
can vary by around 5 μm throughout the entire measurement volume. ing the gauge blocks.

4.2. Scaling factor results References

Despite using an uncalibrated chequerboard artefact in order to [1] Luhmann Thomas, Robson Stuart. Close range photogrammetry: Principles,
provide the initial extrinsic parameters, the accuracy of the scale fac- techniques and applications. Whittles Publishing; 2011.
[2] Fragione A, Sanchez Salmeron AJ, Modica F, Percoco G. Multi-step approach for
tor appears to be high, with an error of 0.0068% over the baseline
automated scaling of photogrammetric micro-measurements. Int J Adv Manuf
length. Based on the linear regression of the length measurement data, Technol 2019;747–57.
the uncertainty on the scale factor was on the order of 7 μm over [3] Lavecchia F, Guerra MG, Galantucci LM. Performance verification of a pho-
the entire 100 mm measurement volume. The interesting result from togrammetric scanning system for micro-parts using a three-dimensional artifact:
the stereo-photogrammetry comparison with the laser interferometer adjustment and calibration. Int J Adv Manuf Technol 2018;96(9–12):4267–79.
[4] Luhmann Thomas. Close range photogrammetry for industrial applications. ISPRS
measurements is the shape of the residuals from the linear fit. As
J Photogramm Remote Sens 2010;65(6):558–69.
can be seen in Fig. 11, there is a repeatable trend to the error on [5] Sun Peng, Lu Nai Guang, Dong Ming Li, Yan Bi Xi, Wang Jun. Simultaneous
the chequerboard position. There are a range of external factors that all-parameters calibration and assessment of a stereo camera pair using a scale
could have resulted in the trend of the residuals in Fig. 11, however, bar. Sensors (Basel, Switzerland) 2018;18(11):1–19.
systematic errors in the extrinsic and intrinsic properties are likely a [6] Duan Xiaojie, Wang Chongqi, Wang Jianming, Zhao He. A new calibration
method and optimization of structure parameters under the non-ideal condition
substantial contributing factor. In fact, the maximum deviation of the
for 3D measurement system based on fiber-optic interference fringe projection.
length measurements was typically on the order of 5 μm or less for all Optik 2018;172(May):424–30.
repeated measurements. The high repeatability of the length measure- [7] International Organization for Standardization. ISO 3650:1998 Geometrical
ments would suggest that the scale factor uncertainty can be reduced by Product Specifications (GPS) — Length standards — Gauge blocks, 1998.
compensating for the non-linear distortion of the measurement volume. [8] Li Jinlong, Ren Hongbing, Luo Peng, Gao Xiaorong, Wang Zeyong. Specular
reflection compensation in homography fringe projection profilometry. Optik
2017;140:413–22.
4.3. Gauge block measurement [9] Zhang Zhengyou. Flexible camera calibration by viewing a plane from unknown
orientations. In: Proceedings of the Seventh IEEE International Conference on
Although the feature location uncertainty is unknown for the gauge Computer Vision, vol. 1. 1999, p. 666–73.
block measurements, the standard deviation from a plane for both the [10] Sims-Waterhouse Danny, Piano Samanta, Leach Richard. Verification of micro-
scale photogrammetry for smooth three-dimensional object measurement. Meas
gauge block and platen surfaces agrees with the values predicted by the
Sci Technol 2017;28(5).
Monte Carlo simulations. Furthermore, the feature location uncertainty [11] Balsamo A, Di Ciommo M, Mugno R, Rebaglia BI, Ricci E, Grella R. Evaluation of
is expected to be much higher than for other methods due to the CMM uncertainty through Monte Carlo simulations. CIRP Ann - Manuf Technol
challenging surface properties, further suggesting that the extrinsic and 1999;48(1):425–8.

8
D. Sims-Waterhouse et al. Precision Engineering 63 (2020) 1–9

[12] Miura Yuka, Nakanishi Shoichi, Higuchi Eiichi, Takamasu Kiyoshi, Abe Makoto, [17] Hartley Richard, Zisserman Andrew. Multiple view geometry in computer vision.
Sato Osamu. Comparative evaluation of estimation of hole plate measurement second ed.. Cambridge University Press; 2004.
uncertainty via Monte Carlo simulation. Precis Eng 2019;56(May 2018):496–505. [18] Joint Committee for Guides in Metrology. Evaluation of measurement data —
[13] Di Leo G, Liguori C, Paolillo A. Propagation of uncertainty through stereo Supplement 1 to the ‘‘Guide to the expression of uncertainty in measurement’’ —
triangulation. In: 2010 IEEE international instrumentation and measurement Propagation of distributions using a Monte Carlo method. Evaluation 2008;JCGM
technology conference, I2MTC 2010 - Proceedings. IEEE; 2010, p. 12–7. 101:2:90.
[14] Lavecchia F, Guerra MG, Galantucci LM. The influence of software algorithms [19] Joint Committee For Guides In Metrology Jcgm. Evaluation of measurement data
on photogrammetric micro-feature measurement’s uncertainty. Int J Adv Manuf — Guide to the expression of uncertainty in measurement. Int Organ Stand
Technol 2017;93(9):3991–4005. Geneva ISBN 2008;50(September):134.
[15] Xu Yafan, Zhao Yan, Wu Falin, Yang Kui. Error analysis of calibration param- [20] Drap Pierre, Lefèvre Julien. An exact formula for calculating inverse radial lens
eters estimation for binocular stereo vision system. In: IST 2013 - 2013 IEEE distortions. Sensors (Switzerland) 2016;16(6):1–18.
international conference on imaging systems and techniques, Proceedings. IEEE;
2013, p. 317–20.
[16] Sankowski W, Włodarczyk M, Kacperski D, Grabowski K. Estimation of
measurement uncertainty in stereo vision system. Image Vis Comput
2017;61:70–81.