Process Improvement Using Data

Release 547b16

Kevin Dunn

05 May 2026
Copyright © 2010 to 2026 Kevin G. Dunn
4
 CONTENTS

Preface iii

1 Visualizing Process Data 1

1.1 Data visualization in context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 References and readings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Time-series plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 Bar plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.5 Box plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.6 Relational graphs: scatter plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.7 Tables as a form of data visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.8 Topics of aesthetics and style . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.9 General summary: revealing complex data graphically . . . . . . . . . . . . . . . . . . . . 19
1.10 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2 Univariate Data Analysis 37

2.1 Univariate data analysis in context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.2 References and readings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.3 What is variability? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.4 Histograms and probability distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.5 Some terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.6 Binary (Bernoulli) distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
2.7 Uniform distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
2.8 Normal distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
2.9 The t-distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
2.10 Poisson distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
2.11 Confidence intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
2.12 Testing for differences and similarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
2.13 Paired tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
2.14 Other types of confidence intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
2.15 Statistical tables for the normal- and t-distribution . . . . . . . . . . . . . . . . . . . . . . . 95
2.16 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

3 Process Monitoring 149

3.1 Process monitoring in context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
 3.2 References and readings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
3.3 What is process monitoring about? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
3.4 Shewhart charts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
3.5 CUSUM charts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
3.6 EWMA charts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
3.7 Other types of monitoring charts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
3.8 Process capability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
3.9 The industrial practice of process monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . 171
3.10 Industrial case study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
3.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
3.12 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

4 Least Squares Modelling Review 209

4.1 Least squares modelling in context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
4.2 References and readings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
4.3 Covariance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
4.4 Correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
4.5 Some definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
4.6 Least squares models with a single x-variable . . . . . . . . . . . . . . . . . . . . . . . . . . 218
4.7 Least squares model analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
4.8 Investigating an existing linear model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
4.9 Summary of steps to build and investigate a linear model . . . . . . . . . . . . . . . . . . . 250
4.10 More than one variable: multiple linear regression (MLR) . . . . . . . . . . . . . . . . . . . 253
4.11 Outliers: discrepancy, leverage, and influence of the observations . . . . . . . . . . . . . . 260
4.12 Enrichment topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
4.13 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269

5 Design and Analysis of Experiments 319

5.1 Design and analysis of experiments in context . . . . . . . . . . . . . . . . . . . . . . . . . 319
5.2 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320
5.3 Usage examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
5.4 References and readings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
5.5 Why learning about systems is important . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322
5.6 Experiments with a single variable at two levels . . . . . . . . . . . . . . . . . . . . . . . . 325
5.7 Changing one single variable at a time (COST) . . . . . . . . . . . . . . . . . . . . . . . . . 328
5.8 Full factorial designs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330
5.9 Fractional factorial designs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346
5.10 Blocking and confounding for disturbances . . . . . . . . . . . . . . . . . . . . . . . . . . . 362
5.11 Response surface methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365
5.12 Evolutionary operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
5.13 General approach for experimentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
5.14 Extended topics related to designed experiments . . . . . . . . . . . . . . . . . . . . . . . . 374
5.15 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379

6 Latent Variable Modelling 411

6.1 In context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411
6.2 References and readings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411
6.3 Extracting value from data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413
6.4 What is a latent variable? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419
6.5 Principal Component Analysis (PCA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421
6.6 Principal Component Regression (PCR) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 468

6
 6.7 Introduction to Projection to Latent Structures (PLS) . . . . . . . . . . . . . . . . . . . . . . 472
6.8 Applications of Latent Variable Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 490

7 Applications of Process Improvement using Data 501

7.1 Product development and product improvement . . . . . . . . . . . . . . . . . . . . . . . . 501

Index 511

i
ii
 PREFACE

This book is a guide on how to improve processes using the large quantities of data that are routinely
collected from process systems. It is in a state of a semi-permanent draft.

We cover data visualization (page 1) first, in Chapter 1, since most data analysis studies start by plotting
the data. This is an extremely brief introduction to this topic, only illustrating the most basic plots
required for this book. Please consult the references in this chapter for more exciting plots that provide
insight to your data.

This is followed by Chapter 2 on univariate data analysis (page 37), which is a comprehensive treatment
of univariate techniques to quantify variability and then to compare variability. We look at various
univariate distributions and consider tests of significance from a confidence-interval viewpoint. This is
arguably a more useful and intuitive way, instead of using hypothesis tests.

The next chapter, Chapter 3, is on monitoring charts (page 149) to track variability: a straightforward
application of univariate data analysis and data visualization from the previous two chapters.

Chapter 4 introduces the area of multivariate data. The first natural application is least squares
modelling (page 209), where we learn how variation in one variable is related to another variable. This
chapter briefly covers multiple linear regression and outliers. We don’t cover nonlinear regression
models but hope to add that in future updates to the book.

Chapter 5 covers designed experiments (page 319), where we intentionally introduce variation into our
system to learn more about it. We learn how to use the models from the experiments to optimize our
process (e.g. for improved profitability).

The final chapter, Chapter 6, is on latent variable modelling (page 411) where we learn how to deal with
multiple variables and extract information from them. This section is divided in several chapters
(PCA, PLS, and applications). It is still a work in progress and will be improved in the future.

Because this is a predominantly electronic book, we resort to many hyperlinks in the text. We
recommend a good PDF reader that allows forward and back navigation of links. However, we have
ensured that a printed copy can be navigated just as easily, especially if you use the table of contents
and index for cross referencing.

Updates: This book is continually updated; there isn’t a fixed edition. You should view it as a wiki.
You might currently have an incomplete or older draft of the document. The latest version is always
available at [Link]

Acknowledgements: I would like to thank my students, teaching assistants, and instructors from
McMaster University, as well as other universities who have, over the years, made valuable comments,
suggestions and corrections. They have graciously given permission to use their solutions to various
questions. Particular thanks to Emily Nichols (2010), Ian Washington (2011), Ryan McBride (2011),

iii
Stuart Young (2011), Mudassir Rashid (2011), Yasser Ghobara (2012), Pedro Castillo (2012), Miles
Montgomery (2012), Cameron DiPietro (2012), Andrew Haines (2012), Krishna Patel (2012), Xin Yuan
(2013), Sean Johnstone (2013), Jervis Pereira (2013), and Ghassan Marjaba (2014), Kyla Sask (2015, and
2016). Their contributions are greatly appreciated.

The textbook was used in an online course from July to August 2014, Experimentation for
Improvement1 . Comments and feedback from that course have greatly improved this book. Thanks to
all the Courserians. That Coursera course was relaunched, and is still active. All videos created for that,
as well as videos created for the Ontario Online Initiative have been embedded in the textbook. Look
for the YouTube videos on the web version of this book, or if reading it from a PDF, watch for the icon
shown.

In particular, I’d like to thank Devon Mordell, from McMaster University, for her informal help on
editing parts of the book. As well as countless others who have via email or web forms provided
feedback. Any errors, poor spelling and grammar are entirely my own fault – any feedback to improve
them will be appreciated2 .

Thanks also to instructors at other universities who have used these notes and slides in their courses
and provided helpful feedback.

 Tip

Copyright and Your Rights

This book is unusual in that it is not available from a publisher. You may download it
electronically, use it for yourself, or share it with anyone.

The copyright to the book is held by Kevin Dunn, but it is licensed to you under the permissive
Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)3 license.

In particular, you are free to

• share - copy, distribute and transmit the work (which includes printing it).

• adapt - but you must distribute the new result under the same or similar license to this one.

• commercialize - you are allowed to create commercial applications based on this work.

• attribute - but you must attribute the work as follows:

– Using selected portions: “Portions of this work are the copyright of Kevin Dunn.”

– Or if used in its entirety: “This work is the copyright of Kevin Dunn.”

You don’t have to, but it would be nice if you tell us you are using this book. That way we can let
you know of any errors.

• Please tell us if you find errors in these chapters, or have suggestions for improvements.

• Please email to ask permission if you would like changes to the above terms and conditions.

Thanks, Kevin4

1 [Link]
2 [Link]

iv
3 [Link]
4 kgdunn@[Link]

v
CHAPTER 1
VISUALIZING PROCESS DATA

1.1 Data visualization in context

This is the first chapter in the book. Why? Many of you have heard the phrase “plot your data,” but
seldom are we shown what appropriate plots look like.

In this section we consider quantitative plots – plots that show numbers. We cover various plots that
will help you gain more insight from your data. We end with a list of tips for effective data
visualization.

Usage examples

You can use the material in this chapter when you must learn more about your system from the data.
For example, you may get these questions:

• Co-worker: Here are the yields (final output value) from a given system for the last 3 years (1256 data
points). Can you help me:

– effectively communicate what the time trends are in the data?

– summarize the yield values?

• Manager: How can we effectively summarize the (a) number and (b) types of defects on our 17
products for the last 12 months?

• Yourself : We produce products in a batchwise manner. For each batch we have 25 different sensors
that we record a value for at a rate of 5 readings per minute, over a total interval of 300 minutes.
How can we visualize these 25 × 5 × 300 = 37500 data points?

1
 Process Improvement Using Data

What we will cover

1.2 References and readings

1. Edward Tufte, Envisioning Information, Graphics Press, 1990. (10th printing in 2005)

2. Edward Tufte, The Visual Display of Quantitative Information, Graphics Press, 2001.

3. Edward Tufte, Visual Explanations: Images and Quantities, Evidence and Narrative, 2nd edition,
Graphics Press, 1997.

4. Stephen Few, Show Me the Numbers and Now You See It: Simple Visualization Techniques for
Quantitative Analysis; both from Analytics Press.

5. William Cleveland, Visualizing Data, 1st edition, Hobart Press, 1993.

6. William Cleveland, The Elements of Graphing Data, 2nd edition, Hobart Press, 1994.

7. Su, It’s Easy to Produce Chartjunk Using Microsoft Excel 2007 but Hard to Make Good Graphs5 ,
Computational Statistics and Data Analysis, 52 (10), 4594-4601, 2008.

1.3 Time-series plots

Video for
this section We start off by considering a plot most often seen in engineering applications: the time-series plot. The
time-series plot is a univariate plot: it shows only one variable. It is a 2-dimensional plot in which one
axis, the time-axis, shows graduations at an appropriate scale (seconds, minutes, weeks, quarters,
years), while the other axis shows the numeric values. Usually, the time-axis is displayed horizontally,
but this is not a requirement: some interesting analysis can be done with time running vertically.

Many statistical packages call this a line plot, as it can be used generally to display any sort of
sequence, whether it is along time or some other ordering. The time-series plot is an excellent way to
visualize long sequences of data. It tells a visual story along the sequence axis, and the human brain is
incredible at absorbing this high density of data, locating patterns in the data such as sinusoids, spikes,
and outliers, and separating any noise from signal.

Here are some tips for effective time-series plots:

• The software should have horizontal and vertical zooming ability. Once zoomed in, there must be
tools to scroll up, down, left and right.

• Always label the x-axis appropriately with (time) units that make sense.
5 [Link]

2 Chapter 1. Visualizing Process Data

 Release 547b16

This plot, found on the Internet, shows a computer’s CPU temperature with time. There are several
problems with the plot, but the key issue here is the x-axis. This plot is probably the result of poor
default settings in the software, but as you will start to realize, bad defaults are very common in
most software packages. They waste your time when you have to repeatedly modify the charts,
especially if you are just starting out with exploring the data. Good software will sensibly label the
time-based axis for you.

The Python code below shows how to do this correctly: a real [Link], a Plotly time
axis formatted with month/year tick labels, and a rolling-mean overlay that helps separate signal
from noise.

import pandas as pd
import plotly.graph_objects as go

[Link] = "plotly"

# Real ammonia concentrations from a waste-water treatment plant.

waste = pd.read_csv("[Link]

# Give the x-axis real time units rather than raw integer indices.
# The CSV ships without a timestamp; attach an hourly index that
# starts at the beginning of 2010 so the time axis is meaningful.
[Link] = pd.date_range(
start="2010-01-01", periods=len(waste), freq="h"
)

# A clean line plot of the raw signal, with month/year tick labels.
fig = waste["Ammonia"].[Link]()
fig.update_layout(
xaxis_title_text="Date",
yaxis_title_text="Ammonia [mmol/L]",
width=800,
height=400,
showlegend=False,
)
fig.update_xaxes(tickformat="%b %Y")
[Link]()

(continues on next page)

1.3. Time-series plots 3

Process Improvement Using Data

(continued from previous page)

# Overlay a 5-day rolling mean to separate signal from noise.
rolling = waste["Ammonia"].rolling("5D", center=True).mean()
fig = [Link]()
fig.add_trace([Link](x=[Link], y=waste["Ammonia"], name="Ammonia"))
fig.add_trace(
[Link](
x=[Link],
y=rolling,
name="5-day rolling mean",
line_color="black",
)
)
fig.update_layout(
xaxis_title_text="Date",
yaxis_title_text="Ammonia [mmol/L]",
width=800,
height=400,
)
fig.update_xaxes(tickformat="%b %Y")
[Link]()

• When plotting more than one trajectory (a vector of values) against time, it is helpful if the lines do
not cross or jumble too much. This allows you to clearly see the relationship with other variables.
The use of a second y-axis on the right-hand side is helpful when plotting two trajectories, but when
plotting three or more trajectories that are in the same numeric range, it is better to use several
parallel axes.

• Using the same data as in the previous tip, a much improved visualization technique is to use
sparklines to represent the sequence of data.

4 Chapter 1. Visualizing Process Data

 Release 547b16

Sparklines are small graphics that carry a high density of information. The human eye is easily
capable of absorbing about 100 dots or points per linear centimeter and around 10000 points per
square centimeter. These sparklines convey the same amount of information as the previous plots
and are easy to consume on hand-held devices such as cellphones and tablet computing devices
that are common in chemical plants and other engineering facilities. Read more about them from
this hyperlink6 .

• When plotting money values over time (e.g. sales of your product over the past 10 years), adjust for
inflation effects by dividing by the consumer price index or an appropriate factor. Distortions due to
the time value of money can be very misleading, as this example of retail sales shows7 . For
Canadians, here is a Canadian inflation calculator8 from the Bank of Canada that can help you. For
most countries you can almost certainly find something similar from the country’s national bank or
a government office.

• If you ever ask yourself, “Why are we being shown so little?” then you must request more data
before and after the time period or current sequence shown. A typical example is stock-price data
(see example figure of Apple’s stock (page 5)). There are numerous graphical “lies” in magazines and
reports where the plot shows a drastic change in trend, but in the context of prior data, that trend is
a small aberration. Again, this brings into play the brain’s remarkable power to discern signal from
noise, but to do this, our brains require context. Ask for the extra context, or look for it, if not
provided.
6 [Link]
7 [Link]
8 [Link]

1.3. Time-series plots 5

 Process Improvement Using Data

1.4 Bar plots

Video for
this section The bar plot is another univariate plot on a two-dimensional axis. The two axes are not called x- or
y-axes. Instead, one axis is called the category axis showing the category name, while the other, the
value axis, shows the numeric value of that category, given by the length of the bar.

Here is some advice for bar plots:

• Use a bar plot when there are many categories and interpretation of the plot does not differ if the
category axis is reshuffled. (It might be easier to interpret the plot with a particular ordering;

6 Chapter 1. Visualizing Process Data

 Release 547b16

however, the interpretation won’t be different with a different ordering of the categories.)

• A time-series plot is more appropriate than a bar plot when there is a time-based ordering to the
categories, because usually you want to imply some sort of trend with time-ordered data. Therefore
do not use a bar plot for time trends, rather use a time-series plot.

Use this Python code to draw the figures:

import pandas as pd

[Link] = "plotly"

labels = ["2008 Q1", "Q2", "Q3", "Q4", "2009 Q1", "Q2", "Q3", "Q4"]
profit = [Link](
data=[45, 32, 67, 23, 42, 56, 64, 92],
index=labels,
columns=["Quarterly profit ($ '000)"],
) + 40

# Draw a bar plot.

fig = [Link]()
fig.update_traces(
text=profit["Quarterly profit ($ '000)"],
textposition="outside",
)
fig.update_layout(
yaxis_title_text="Quarterly profit ($ '000)",
showlegend=False,
)
[Link]()

# Now rather use a line plot.

fig = [Link](markers=True)
fig.update_layout(
yaxis_title_text="Quarterly profit ($ '000)",
showlegend=False,
)
[Link]()

or this R code:

1.4. Bar plots 7

Process Improvement Using Data

labels = c("2008 Q1", "Q2", "Q3", "Q4",

"2009 Q1", "Q2", "Q3", "Q4")
profit = c(45, 32, 67, 23, 42, 56, 64, 92)+40

# Draw a bar-plot
bp = barplot(profit,
[Link]=labels,
axisnames=TRUE,
ylab="Quarterly profit ($ '000)",
border = TRUE)
text(bp, profit+3,
labels=format(profit),
xpd = TRUE,
col = "black")

# Now rather use a line plot.

# Graph profit, but turn off axes
# and annotations
plot(profit, type="b", axes=TRUE,
ann=FALSE, xaxt="n")

# Show the x-axis using our labels

axis(1, at=1:8, lab=labels)

# Plot title
title(ylab="Quarterly profit ($ '000)")

• Bar plots can be wasteful as each data point is repeated several times:

1. Left edge (line) of each bar

2. Right edge (line) of each bar

3. The height of the colour in the bar

4. The number’s position (up and down along the y-axis)

5. The top edge of each bar, just below the number

6. The number itself

To this end, Tufte defines the data-ink ratio as:

total ink for data
Data-ink ratio =
total ink for graphics
= 1 − proportion of ink that can be erased without loss of data information

The heuristic is to maximize this ratio as far as possible by using the ink (pixels) for only the data.

• Rather use a table than a bar plot for a handful of data points.

• Don’t use cross-hatching, textures or unusual shading in the plots. This creates distracting visual
vibrations.

8 Chapter 1. Visualizing Process Data

 Release 547b16

• Use horizontal bars if

– there is some ordering to the categories (it is often easier to read the category labels from
top-to-bottom), or

– if the labels do not fit side-by-side: don’t make the reader have to rotate the page to interpret the
plot; rotate the plot for the reader.

• You can place the labels inside the bars.

• You should start the noncategory axis at zero: the bar’s area shows the magnitude. Starting bars at a
nonzero value distorts the meaning.

1.5 Box plots

Video for
this section Box plots are an efficient summary of one variable (univariate chart), but can also be used effectively to
compare variables that are in the same units of measurement.

The box plot shows the so-called five-number summary of a univariate data series:

1. Minimum sample value

2. 25th percentile9 (1st quartile10 )

3. 50th percentile (median)

4. 75th percentile (3rd quartile)

5. Maximum sample value

The 25th percentile is the value below which 25% of the observations in the sample are found. The
distance from the 3rd to the 1st quartile is also known as the interquartile range (IQR) and represents
the data’s spread, similar to the standard deviation.

The following data are thickness measurements of 2-by-6 boards (2-by-6 refers for the thickness and
depth of a wooden board), taken at six locations around the edge. Here is a sample of the
9 [Link]
10 [Link]

1.5. Box plots 9

Process Improvement Using Data

measurements and a summary of the first 100 boards (code in Python and R respectively):

import pandas as pd

[Link] = "plotly"

all_boards = pd.read_csv("[Link]
boards = all_boards.iloc[0:100, 1:7]

# Look at the start and end of the data.

# Examine and summarize your data before
# doing anything else.
[Link]()
[Link]()
[Link]()

fig = [Link]()
fig.update_layout(yaxis_title_text="Thickness [mils]")
[Link]()

[Link] = [Link]("[Link]
boards = [Link][1:100, 2:7]

# Look at the start and end of the data

# Examine and summarize your data before
# doing anything else
head(boards)
tail(boards)

summary(boards)

boxplot(boards)

The following box plot is a graphical summary of these numbers.

A box plot is great for comparisons. In this figure we see how the thickness at position 1 is greater than
at the other positions. It is also the position with high variability, indicating that something about the
saw blade at that position is not what it should be. The median is also not balanced between the two
quantiles for this box plot, when compared to the others.

10 Chapter 1. Visualizing Process Data

 Release 547b16

Some variations for the box plot are possible:

• Show outliers as dots, where an outlier is most commonly defined as any point 1.5 IQR distance
units away from the box. The box’s upper bound is at the 25th percentile, and the boxes lower
bound is at the 75th percentile.

• The whiskers on the plots are drawn at most 1.5 IQR distance units away from the box, however, if
the whisker is to be drawn beyond the bound of the data vector, then it is redrawn at the edge of the
data instead (i.e. it is clamped, to avoid it exceeding).

• Use the mean instead of the median [not too common].

• Use the 2% and 98% percentiles rather than the upper and lower hinge values.

Example

In a final exam for a particular course at McMaster University there was an open-ended question.
These data values are the grades11 achieved for the answer to that question, broken down by whether
the student used a systematic method, or not. No grades were given for using a systematic method;
grades were awarded only for answering the question.

A systematic method is any method that assists the student with problem solving. For example, a
strategry could be to: define the problem, identify knowns/unknowns and assumptions, explore
alternatives, plan a strategy, implement the strategy and then check the solution.

Draw two box plots next to each other that compare the grades of students who did, or did not use a
problem solving strategy. Comment on any features you notice in the comparison.

Answer

Several points are apparent in the box plot:

• students in either category achieved the highest grade possible

• the spread (interquartile distance) when using the problem solving method is smaller

• both box plots show a skew to the lower left tail (compare the median to the first and third quartiles)
11 [Link]

1.5. Box plots 11

 Process Improvement Using Data

• we will use a confidence interval (page 85) in a later chapter to judge whether this difference is
statistically significant or not.

More readings

You can read more about box plots in the paper by Hadley Wickham and Lisa Stryjewsk12 . It
summarizes variations of this plot, such as the violin plot, and two-dimensional versions of it. It is a
power summary plot that has been around since 1970.

1.6 Relational graphs: scatter plots

Video for
this section This is a plot many people are comfortable using. It helps you understand the relationship between
two variables - a bivariate plot - as opposed to the previous charts that are univariate. A scatter plot is
a collection of points shown inside a box formed by two axes at 90 degrees to each other. The marker’s
position is located at the intersection of the values shown on the horizontal (x) axis and vertical (y) axis.

The unspoken intention of a scatter plot is usually to ask the reader to draw a causal relationship
between the two variables. However, not all scatter plots actually show causal phenomena, as the
figure below tries to convince you:

This source code generates similar, but not identical, figures to those shows here in the text.

import numpy as np
import pandas as pd

[Link] = "plotly"

# Plot of temperature vs vapour pressure.

data_file = "[Link]
distillation = pd.read_csv(data_file)
fig = [Link](x="Temp9", y="VapourPressure")
fig.update_layout(
xaxis_title_text="Temperature (F)",
yaxis_title_text="Vapour pressure (kPa)",
)
[Link]()
(continues on next page)

12 [Link]

12 Chapter 1. Visualizing Process Data

 Release 547b16

(continued from previous page)

# Plot of white hairs vs BMD.

# Osteoporosis (fake) data: number of white
# hairs per square inch vs bone mineral
# density (measurement of osteoporosis)
# in kg/m^3 (1500 kg/m3 is typical).
N = 50
white_hairs = [Link]([Link](loc=500, scale=150, size=N))
bone_mineral_density = (
-0.25 * white_hairs + 1550 + [Link](loc=0, scale=25, size=N)
)
osteo = [Link](
{"white_hairs": white_hairs, "bone_mineral_density": bone_mineral_density}
)
fig = [Link](x="white_hairs", y="bone_mineral_density")
fig.update_layout(
xaxis_title_text="Number of white hairs per square inch of scalp",
yaxis_title_text="Bone mineral density (kg/m^3) [measure of osteoporosis]",
)
[Link]()

The equivalent code in R:

# Plot of temperature vs vapour pressure

data_file = "[Link]
distillation = [Link](data_file)

plot(distillation$Temp9,
distillation$VapourPressure,
xlab="Temperature (F)",
ylab="Vapour pressure (kPa)")

# Plot of white hairs vs BMD

# Osteoporosis (fake) data: number of white
# hairs per square inch vs bone mineral
# density (measurement of osteoporosis)
# vs kg/m^3 (1500 kg/m3 is typical)
N = 50
[Link] = round(rnorm(N,
mean=500,
sd=150))
[Link] = -0.25 * [Link] + 1550 + rnorm(N, mean=0, sd=25)

plot([Link], [Link],
xlab = "Number of white hairs per square inch of scalp",
ylab = "Bone mineral density (kg/m^3) [measure of osteoporosis]")

Strive for graphical excellence by doing the following:

• Make each axis as tight as possible.

• Avoid heavy grid lines.

• Use the least amount of ink.

• Do not distort the axes.

There is an unfounded fear that others won’t understand your 2D scatter plot. Tufte (Visual Display of
Quantitative Information, p 83) shows that there are no scatter plots in a sample (1974 to 1980) of U.S.,
German and British dailies, despite studies showing that 12-year-olds can interpret such plots:
Japanese newspapers frequently use them.

1.6. Relational graphs: scatter plots 13

Process Improvement Using Data

You will see this in industrial settings as well. The next time you go into an industrial control room (or
look carefull at some screens in online videos), try finding any scatter plots. The audience is not to
blame: it is the producers of these charts who assume the audience is incapable of interpreting them.

ò Note

Assume that if you can understand the plot, so will your audience.

Further improvements can be made to your scatter plots. For example, extend the frames only as far as
your data, and add a regression line where appropriate:

You can add box plots and histograms to the side of the axes to aide interpretation:

Add a third variable to the plot by adjusting the marker size, and add a fourth variable with the use of

14 Chapter 1. Visualizing Process Data

 Release 547b16

colour:

This example, from GapMinder ([Link] ) , shows data until 2007 for:

1. income per person (x-axis);

2. against fertility (y-axis);

3. the size of each data point is proportional to the country’s population;

4. the marker colour shows life expectancy at birth (years).

5. The GapMinder website allows you to “play” the graph over time, effectively adding a
fifth dimension to the 2D plot.

So 5 dimensions in a 2D surface. A 6th dimension cab be added if using technology such as

VR glasses, to create a 3rd dimension, to display another variable from the data set.

Use the hyperlink above to see how richer countries move towards lower fertility and
higher income over time.

1.7 Tables as a form of data visualization

A data table, or a spreadsheet, is an efficient format for comparative data analysis on categorical
objects. Usually, the items being compared are placed in a column, while the categorical objects are in
the rows. The quantitative value is then placed at the intersection of the row and column, called the
cell. The following examples demonstrate data tables.

This table compares monthly payments for buying or leasing various cars (categories). The first two
columns are being compared; the other columns contain additional, secondary information.
13 [Link]

1.7. Tables as a form of data visualization 15

Process Improvement Using Data

The next table compares defect types (number of defects) for different product grades (categories).

This particular table raises more questions:

• Which defects cost us the most money?

• Which defects occur most frequently? The table does not contain any information about production
rate. For example, if there were 1850 lots of grade A4636 (first row) produced, then defect A occurs
at a rate of 37/1850 = 1/50. And if 250 lots of grade A2610 (last row) were produced, then, again,
defect A occurs at a rate of 1/50. Redrawing the table on a production-rate basis would be useful if
we are making changes to the process and want to target the most problematic defect.

• If we are comparing a type of defect over different grades, then we are now comparing down the
table, instead of across the table. In this case, the fraction of defects for each grade would be a more
useful quantity to display.

• If we are comparing defects within a grade, then we are comparing across the table. Here again, the
fraction of each defect type, weighted according to the cost of that defect, would be more
appropriate.

Three common pitfalls to avoid:

1. Avoid using pie charts when tables will do.

Pie charts are tempting when we want to graphically break down a quantity into components. I
have used them erroneously myself (here is an example on a website that I helped with:
[Link] We won’t go into details here, but I strongly
suggest you read the convincing evidence of Stephen Few in: “Save the pies for dessert”14 . The key
problem is that the human eye cannot adequately decode angles; however, we have no problem
with linear data.
14 [Link]

16 Chapter 1. Visualizing Process Data

 Release 547b16

2. Avoid arbitrary ordering along the first column; usually, alphabetically or in time order is better.

Listing the car types alphabetically is trivial: instead, list them by some other third criterion of
interest, perhaps minimum down payment required, typical lease duration, or total amount of
interest paid on the loan. That way you get some extra context to the table for free.

3. Avoid using excessive grid lines.

Tabular data should avoid vertical grid lines, except when the columns are so close that mistakes
will be made. The human eye will use the visual white space between the numbers to create its own
columns.

To wrap up this section is a demonstration of tabular data in a different format, based on an idea of
Tufte in The Visual Display of Quantitative Information, p. 158. Here we compare the corrosion resistance
and roughness of a steel surface for two different types of coatings, A and B.

A layout that you expect to see in a standard engineering report:

Product Corrosion resistance Surface roughness

Coating A Coating B Coating A Coating B
K135 0.30 0.22 30 42
K136 0.45 0.39 86 31
P271 0.22 0.24 24 73
P275 0.40 0.44 74 52
S561 0.56 0.36 70 75
S567 0.76 0.51 63 70

And the layout advocated by Tufte:

1.7. Tables as a form of data visualization 17

Process Improvement Using Data

Note how the slopes carry the information about the effect of changing the coating type. The
rearranged row ordering shows these changes as well. This idea is effective for two treatments but
could be extended to three or four treatments by adding extra “columns”. Only the extremes are
numbered, but every point could be numbered if the values are also required by the readers.

1.8 Topics of aesthetics and style

We won’t cover these topics, but Tufte’s books (page 2) (see Tufte) contain remarkable examples that
discuss effective use of colour for good contrast, varying line widths, and graph layout (e.g. use more
horizontal than vertical - an aspect ratio of about 1.4 to 2.0; and flow the graphics into the location in
the text where discussed).

1.8.1 Data frames (axes)

Frames are the basic containers that surround the data and give context to our numbers. Here are
some tips:

1. Use round numbers.

2. Generally, tighten the axes as much as possible, except . . .

3. When showing comparison plots, all axes must have the same minima and maxima.

1.8.2 Colour
Colour is very effective in all graphical charts. However, you must bear in mind that your readers
might be colour-blind, or the document might be read from a grayscale printout, or viewed on an
electronic device where colours are shown differently than you might intend.

Note also that a standard colour progression does not exist. We often see dark blues and purples
representing low numbers and reds the higher numbers, with greens, yellows and orange in-between.
There are several such colour schemes15 - there isn’t a universal standard. The only safest colour
15 [Link]

18 Chapter 1. Visualizing Process Data

 Release 547b16

progression is the grayscale axis, ranging from black to white at each extreme: this satisfies both
colour-blind readers and users of your grayscale printed output.

See the section on scatter plots (page 15) for an example of the effective use of colour.

1.9 General summary: revealing complex data graphically

There is no generic advice that applies in every instance. These tips are useful, though, in most cases:

• If the question you want answered is causality, then show causality (the most effective way is with
bivariate scatter plots). If trying to answer a question with alternatives, show comparisons (with
tiles of plots or a simple table).

• Words and graphics belong together. Add labels to plots for outliers, and explain interesting points.
Add equations and even small summary tables on top of your plots. Remember that a graph should
be like a paragraph of text, not necessarily just a graphical display of numbers that you discuss later
on.

• Avoid obscure coding on the graph. Don’t label points as “A”, “B”, “C”, . . . . and then put a legend:
“A: grade TK133”, “B: grade RT231”, “C: grade TK134”. Just put the labels directly on the plot.

• Do not assume your audience is ignorant and won’t understand a complex plot. Conversely, don’t
try to enliven a plot with decorations and unnecessary graphics (flip through a copy of almost any
weekly news magazine for examples of this sort of embellishment). As Tufte mentions more than
once in his books, “If the statistics are boring, then you’ve got the wrong numbers.”. The graph should
stand on its own.

• When the graphics involve money and time, make sure you adjust the money for inflation.

• Maximize the data-ink ratio = (ink for data) / (total ink for graphics). Maximizing this ratio, within
reason, means you should (a) eliminate nondata ink and (b) erase redundant data-ink.

• Maximize data density. Humans can interpret data displays16 of around 100 data points per
centimeter (250 data points per linear inch) and around 10000 per square centimeter (60000 data
points per square inch).

1.10 Exercises

ò Question

The data shown here are the number of visits to a university website for a particular statistics
course. There are 90 students in the class.

1. What are the names (type) of the 2 plots shown?

2. List any 2 interesting features in these data.

16 [Link]

1.9. General summary: revealing complex data graphically 19

Process Improvement Using Data

ò Solution

1. The plots are a time-series plot and a sparkline. The sparkline shows exactly the same data, just
a more compact form (without the labelling on the axes).

2. Features shown in the data are:

• A noticeable weekly cycle; probably assignments are due the next day!

• A sustained, high level of traffic in the first week February - maybe a midterm test.

• Some days have more than 90 visits, indicating that students visit the site more than once per
day, or due to external visitors to the site.

ò Question

What are the names of the axes on a bar plot?

ò Solution

The category axis and value axis.

ò Question

Which types of features can can the human eye easily pick out of a time series plot?

ò Solution

Features such as sinusoids, spikes, gaps (missing values), upward and downward trends are
quickly picked out by the human eye, even in a poorly drawn plot.

ò Question

Why is the principle of minimizing “data ink” so important in an effective visualization? Give an
scientific or engineering example of why this important.

ò Solution

It reduces the time or work to interpret that plot, by eliminating elements that are non-essential to
the plot’s interpretation. Situations which are time or safety critical are examples, for example in an
operator control room, or medical facility (operating room).

20 Chapter 1. Visualizing Process Data

 Release 547b16

ò Question

Describe what the main difference(s) between a bar chart and a histogram are.

ò Solution

The solution is taken directly from:

[Link]

• Histograms are used to show distributions of variables while bar charts are used to compare
variables.

• Histograms plot quantitative data with ranges of the data grouped into bins or intervals while
bar charts plot categorical data.

• Bars can be reordered in bar charts but not in histograms.

• There are no spaces between the bars of a histogram since there are no gaps between the bins.
An exception would occur if there were no values in a given bin but in that case the value is zero
rather than a space. On the other hand, there are spaces between the variables of a bar chart.

• The bars of bar charts typically have the same width. The widths of the bars in a histogram need
not be the same as long as the total area is one hundred percent if percents are used or the total
count if counts are used. Therefore, values in bar charts are given by the length of the bar while
values in histograms are given by areas.

ò Question

Write out a list of any features that can turn a plot into a poor visualization. Think carefully about
plots you encountered in textbooks and scientific publications, or the lab reports you might have
recently created for a university or college course.

ò Question

The following graphics were shown in the print issue of The Economist in the 28 November 2009
issue, page 85. The article attempts to argue that there are enough similarities between Japan’s
stagnant economic experience in the 1990’s (known as “Japan’s Lost Decade”), and the current
experience in the “rich world” western countries to give their policymakers pause for concern. You
can read the full article here17 . What problems do you notice with the graphics?

1.10. Exercises 21
Process Improvement Using Data

ò Solution

There are several problems with this graphical comparison, but the main concerns are with
showing time trends as bar plots, and the alignment of the time trends.

• The purpose of the plot is to show the similarities between Japan in the 1990’s to the current
trends (2000’s) in Britain and USA. The data from 2000 onwards for Japan is therefore irrelevant
in this case.

• The data are time-based: a bar-plot is a poor choice to show time-based trends.

• Notice the symmetry above and below the zero line: in colours: “light blue + grey = dark blue”,
i.e. “General government balance + Net capital inflow = Private financial balance”. Given this
constraint, only 2 of the 3 variables are required. As I’m not an economics expert, I have no idea
which 2 of the 3 would be most relevant.

• The data for Japan from the 1990’s should be shown on the same plot for USA and Britain for the
2000’s, since that was the purpose - to show a comparison between Japan’s experience and the
USA/Britain experience. One way to do this: plot three lines on a time-series chart: one for each
of Japan, Britain and USA for “General government balance”. The have a second plot, similar to
the first, showing the “Private financial balance”.

Minor problems are:

• The colour scheme is poor: four different shades of blue are used (two background shades, and
two of the time-based parameters)

17 [Link]

22 Chapter 1. Visualizing Process Data

 Release 547b16

• The use of stacked bar plots is almost always problematic: the user is never sure if the bars are
cumulative or additive, unless they know the subject matter or read the accompanying text.

ò Question

This figure is a screenshot from a Toronto Star article18 about mortgage payments as a function of
the interest rate. Redraw the same information in a more suitable form.

ò Solution

The data from this article are needlessly embellished with a picture of a house, a $20,000 bill and a
stake in the ground.

A simple annotated table will show the data well enough. A bar chart, horizontally or vertically
aligned is not suitable.

Some people have suggested using a scatter plot - I never thought of that, but it works. It shows a
straight line relationship between interest rate and the monthly payment. I suppose the advantage
of that plot is that you can see (a) the relationship is linear, which it should be, and (b) you can
visually interpolate the monthly payment given any interest rate between 2 and 5%.

18 [Link]

1.10. Exercises 23
Process Improvement Using Data

A key point though: the mortgage amount and the amortization rate must be shown with the plot
or table. The cost of the house and the downpayment are actually irrelevant. You are paying
interest on the mortgage amount, where
mortgage amount = cost of the house + mortgage insurance − downpayment. The table or the plot
will change if either of those two variables change. Your monthly payment is higher for shorter
amortization periods, and for larger mortgage amounts.

ò Question

This question is an extension to visualizing more than 3 variables. Investigate on your own the
term “scatterplot matrix”, and draw one for the Food texture data set19 . See the car library in R to
create an effective scatterplot matrix with the scatterplotMatrix function. List some
bullet-points that interpret the plot.

ò Solution

import pandas as pd
import [Link] as px

data_file = '[Link]
food = pd.read_csv(data_file)

fig = px.scatter_matrix(food, dimensions=[Link][1:6])

[Link]()

19 [Link]

24 Chapter 1. Visualizing Process Data

 Release 547b16

library(car)
data_file = '[Link]
food <- [Link](data_file)

# Hide the smoother and bounds

scatterplotMatrix(food[,2:6])

From this plot we see histograms of the 5 univariate distributions on the diagonal plots; the
off-diagonal plots are the bivariate correlations between each combination of variable. The trend
line (solid light green) shows the linear regression between the two variables. The lower diagonal
part of the plot is a 90 degree rotation of the upper diagonal part. Some software packages will just
draw either the upper or lower part.

From these plots we quickly gain an insight into the data:

• Most of the 5 variables have a normal-like distribution, except for Crispy, but notice the small
notches on the middle histogram: they are equally spaced, indicating the variable is not
continuous; it is quantized20 . The Fracture variable also displays this quantization.

• There is a strong negative correlation with oiliness and density: oilier pastries are less dense (to
be expected).

• There is a positive correlation with oiliness and crispiness: oilier pastries are more crisp (to be
expected).

• There is no relationship between the oiliness and hardness of the pastry.

• There is a negative correlation between density and crispiness (based on the prior relationship
with Oil): less dense pastries (e.g. more air in them) and crispier.

• There is a positive correlation between Density and Fracture. As described in the dataset
file, Fracture is the angle by which the pastry can be bent, before it breaks; more dense
pastries have a higher fracture angle.

• Similarly, a very strong negative correlation between Crispy and Fracture, indicating the
expected effect that very crispy pastries have a low fracture angle.

• The pastry’s hardness seems to be uncorrelated to all the other 4 variables.

ò Question

Using the Website traffic data set21

1. Create a chart that shows the variability in website traffic for each day of the week.

2. Use the same data set to describe any time-based trends that are apparent.

ò Solution

1. A suitable chart for displaying variability on a per-day basis is the boxplot, one box for each day

20 [Link]
21 [Link]

1.10. Exercises 25
Process Improvement Using Data

of the week. This allows you to see between-day variation when comparing the boxes side by
side, and get an impression of the variability within each variable, by examining how the box’s
horizontal lines are spread out (25th, 50th and 75th percentiles).

2. A box plot is an effective way to summarize and compare the data for each day of the week.

import pandas as pd
[Link] = "plotly"

web = pd.read_csv('[Link]

# Re-order the days

day_names = ["Saturday", "Sunday", "Monday", "Tuesday",
"Wednesday", "Thursday", "Friday"]
web["DayOfWeek"] = [Link](web["DayOfWeek"],
categories=day_names,
ordered=True)
web = web.sort_values("DayOfWeek")

fig = [Link](x="DayOfWeek", y="Visits")

[Link]()

web = [Link]('[Link]

# Re-order the factors in this order

[Link] = c("Saturday", "Sunday", "Monday", "Tuesday", "Wednesday","Thursday", "Friday
˓→" )

days = factor(web$DayOfWeek, level=[Link])

boxplot(web$Visits ~ days)

The box plot shows:

• Much less website traffic on Saturdays and Sundays, especially Sunday which has less spread
than Saturday.

26 Chapter 1. Visualizing Process Data

 Release 547b16

• Visits increase during the weekday, peaking on Wednesday and then dropping down by
Friday.

• All week days seem to have about the same level of spread, except Friday, which is more
variable.

• This is a website of academic interest, so these trends are expected.

3. A time-series plot of the data shows increased visits in September and October, and declining
visits in November and December. This coincides with the phases of the academic term. A plot
of the total number of visits within each month will show this effect clearly. The lowest number
of visits were recorded in late June and July.

The best way to draw the time-series plot is to use proper time-based labelling on the x-axis, but we
won’t cover that topic here. If you are interested, read up about the xts package (see the R
tutorial22 ) and it’s plot command. See how it is used in the code below:

import pandas as pd
[Link] = "plotly"

web = pd.read_csv('[Link]

# Sequence plot of the raw integer index

fig = web["Visits"].[Link](markers=True)
fig.update_layout(xaxis_title_text="Sequence order",
yaxis_title_text="Visits")
[Link]()

# A better plot using a real date axis

web["Date"] = pd.to_datetime(web["MonthDay"].[Link](),
format="%B %d")
web = web.sort_values("Date").set_index("Date")
fig = web["Visits"].[Link]()
fig.update_xaxes(tickformat="%b")

1.10. Exercises 27
Process Improvement Using Data

fig.update_layout(yaxis_title_text="Visits")
[Link]()

web = [Link]('[Link]

layout(matrix(c(1,2), 1, 2))
plot(web$Visits, type="o")

# A better plot using the xts library

library(xts)
[Link] = [Link](web$MonthDay, format=" %B %d")
[Link] = xts(web$Visits, [Link]=[Link])
plot([Link], [Link]="%b")

ò Question

1. What type of plot is shown here?

2. Describe the two phenomenon displayed.

3. Which plot type asks you to draw a cause and effect relationship between two variables?

4. Use rough values from the given plot to construct an approximate example of the plot you
proposed in part 3.

5. What advantage is there to the plot given here, over the type in your answer to part 3.

ò Solution

1. A time-series plot.

2. The rate of cellphone usage (expected to be proportional to number of mobile phone antennae)
has increased in Columbia, especially since 2002. Likely this is this usual case where the price
comes down, leading to greater use. Though some other political or economic change may have
taken place in 2002 leading to increased phone use.

22 [Link]

28 Chapter 1. Visualizing Process Data

 Release 547b16

The rate of kidnappings peaked in 2000, at a rate of 8 per 100,000 residents, and has steadily
decreased since that peak.

3. A scatter plot.

4. A scatter plot, from approximate values on the plot, is generated by the following code (you
may use any software to construct your plot)

# Data from 1996 to 2007

bitmap('[Link]', pointsize=14, res=300)
kidnap <- c( 4, 5, 6.5, 7.5, 8.75, 7, 7, 5, 3.25, 2, 1.5, 1.25)
mobile <- c(0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 2, 3.5, 4.25, 6.5, 7.25)
plot(mobile, kidnap, type='p', xlab="Mobile phone antennae [thousands]",
ylab="Kidnappings per 100,000 residents")
[Link]()

5. The advantage of the time-series plot is that you are able to clearly see any time-based trends -
those are lost in the scatter plot (though you can recover some time-based information when you
connect the dots in time order).

Comment:

The general negative correlation in the scatter plot, and the trends observed in the time-series plots
ask you to infer a relationship between the two trajectories. In this case the plot’s author would like
you to infer that increased cellphone penetration in the population has been (partly) responsible for
the reduction in kidnappings.

This relationship may, or may not be, causal in nature. The only way to ascertain causality would
be to do an experiment: in this case, you would remove cellphone antennae and see if kidnappings
increased again. This example outlines the problem with trends and data observed from society -
we can never be sure the phenomena are causal:

1.10. Exercises 29
Process Improvement Using Data

• firstly we couldn’t possibly perform that experiment of removing cell towers, and

• even if we could, the time scales are too long to control the experimental conditions: something
else would change while we were doing the experiment.

To compensate for that, social science studies compare similar countries - for example the original
article from The Economist’s website23 shows how the same data from Mexico and Venezuela were
compared to Columbia’s data. The article also shows how much of the trend was due to political
changes in the country that were happening at the same time: in particular a 3rd factor not shown
in the plots was largely responsible for the decrease in kidnappings. Kidnappings would probably
have remained at the same level if it were not also for the increase in the number of police officers,
who are able to respond to citizen’s cellphone calls.

Fortunately in engineering situations we deal with much shorter time scales, and are able to better
control our experiments. However the case of an uncertain 3rd factor is prevalent and must be
guarded for - we’ll learn about this is the section on design of experiments.

ò Question

Load the room temperature24 dataset into R, Python or MATLAB, or whichever software tool you
prefer to plot with.

1. Plot the 4 trajectories, FrontLeft, FrontRight, BackLeft and BackRight on the same plot.

2. Comment on any features you observe in your plot.

3. Be specific and describe how sparklines of these same data would improve the message the data
is showing.

ò Solution

1. You could use the following code to plot the data:

import pandas as pd
[Link] = "plotly"

data_file = '[Link]
roomtemp = pd.read_csv(data_file)
[Link]()

fig = roomtemp[["FrontLeft", "FrontRight",

"BackLeft", "BackRight"]].[Link](
color_discrete_sequence=["blue", "royalblue",
"black", "dimgray"],
)
fig.update_layout(
yaxis_range=[290, 300],
xaxis_title_text="Sequence order",
yaxis_title_text="Room temperature [K]",
)
[Link]()

23 [Link]
24 [Link]

30 Chapter 1. Visualizing Process Data

 Release 547b16

data_file = '[Link]
roomtemp <- [Link](data_file)
summary(roomtemp)
ylim = c(290, 300)

plot(roomtemp$FrontLeft,
type='l',
col="blue",
ylim=c(290, 300),
xlab="Sequence order",
ylab="Room temperature [K]")
lines(roomtemp$FrontRight,
type='b',
pch='o',
col="blue")
lines(roomtemp$BackLeft,
type='l',
col="black")
lines(roomtemp$BackRight,
type='b',
pch='o',
col="black")

legend(20, 300,
legend=c("Front left",
"Front right",
"Back left",
"Back right"),
col=c("blue", "blue",
"black", "black"),
lwd=2,
pch=c(NA, "o", NA, "o"))

A sequence plot of the data is good enough, though a time-based plot is better.

2. • Oscillations, with a period of roughly 48 to 50 samples (corresponds to 24 hours) shows a

daily cycle in the temperature.

• All 4 temperatures are correlated (move together).

• There is a break in the correlation around samples 50 to 60 on the front temperatures (maybe a
door or window was left open?). Notice that the oscillatory trend still continues within the
offset region - just shifted lower.

1.10. Exercises 31
Process Improvement Using Data

• A spike up in the room’s back left temperature, around sample 135.

3. The above plot was requested to be on one axis, which leads to some clutter in the presentation.
Sparklines show each trajectory on their own axis, so it is less cluttered, but the same features
would still be observed when the 4 tiny plots are stacked one on top of each other.

If you looked around for how to generate sparklines in R you may have come across this
website25 . Notice in the top left corner that the sparklines function comes from the
YaleToolkit, which is an add-on package to R. We show how to install packages in the
tutorial26 . Once installed, you can try out that sparklines function:

• First load the library: library(YaleToolkit)

• Then see the help for the function: help(sparklines) to see how to generate your
sparklines

ò Question

Load the six point board thickness27 dataset, available from datasets website.

1. Plot a boxplot of the first 100 rows of data to match the figure in these notes (page 10)

2. Explain why the thick center line in the box plot is not symmetrical with the outer edges of the
box.

ò Solution

1. The following code will load the data, and plot a boxplot for the first 100 rows:

import pandas as pd
[Link] = "plotly"

data_file = '[Link]
boards = pd.read_csv(data_file)
[Link]()

# Ignore the first date/time column; use only Pos1...Pos6

first100 = [Link][:100, 1:7]
fig = [Link]()
fig.update_layout(yaxis_title_text="Thickness [mils]")
[Link]()

25 [Link]
26 [Link]
27 [Link]

32 Chapter 1. Visualizing Process Data

 Release 547b16

data_file = '[Link]
boards <- [Link](data_file)
summary(boards)

plot(boards[1:100,2], type='l')
plot(boards[1:100,5], type='l')
first100 <- boards[1:100, 2:7]

# Ignore the first date/time column: using only Pos1, Pos2, ... Pos6 columns
boxplot(first100, ylab="Thickness [mils]")

2. The thick center line on each boxplot is the median (50th percentile) of that variable. The top and
bottom edges of the box are the 25th and 75th percentile, respectively. If the data are from a
symmetric distribution, such as the 𝑡 or normal distribution, then the median should be
approximately centered with respect to those 2 percentiles. The fact that it is not, especially for
position 1, indicates the data are skewed either to the left (median is closer to upper edge) or the
the right (median closer to the lower edge).

ò Question

Consider this plot from the Economist article regarding “Working hours”28

1.10. Exercises 33
Process Improvement Using Data

1. What is the plot’s author trying to convey with this scatter plot?

2. Do you believe this an effective and complete message (i.e. could you improve it somehow?)

3. Is there a causal mechanism at play between the two variables?

4. How would you confirm or disprove the message the plot’s author is making?

ò Solution

1. The message is likely that longer working hours do not translate into greater earnings
(measured with GDP) as might be expected. In fact, the opposite holds: longer working hours
are correlated with lower earnings (we say: “there’s a negative correlation between working
hours and earnings”). The axes have been scaled to account for purchasing power.

2. As the original article alludes, there are differences between countries; and given the large
number of points on the plot (well over 200) it is safe to assume that there are several points per
country, showing the shifts over time. As a result, colour coding, or using different markers to
show each country’s shift and change over time will provide some additional insight. For
example, the line of points stretching from 2200 to 2600 on the x-axis: is that due to one country
and in which direction is it moving over time (left or right)?

Some students rightly pointed out that policy shifts occurred during this period; some countries
joined the EU, and that may have lead to a change in the plots. So the picture is by no means
complete. However, the picture is almost never complete for any data set.

3. This is a tough one to answer. The data are compelling in their lack of scatter. Usually systems
with dubious correlations show a high degree of scatter. As before, colour or shaped codes for
each country will give a better idea of cause-effect. I suspect this plot shows a strong correlation
simply because there are small clusters for each country that are close together, but the negative
trend simply comes from a country-to-country difference.

As emphasized before in this course, we can only truly tell causality by doing an experiment.

28 [Link]

34 Chapter 1. Visualizing Process Data

 Release 547b16

Here there are no major ethical obligations, however it is unlikely that you would be able to
convince companies to enforce short vs long working hours so you can observe productivity.
The time before the change also takes effect is likely very long.

So the answer is yes, maybe there is a causal mechanism here that is plausible (we’ve often
heard that people whose work-life balanced is better are more productive), but we cannot test it
explicitly.

4. Also see the prior answer: require experiments over a broad range of employment types and
regions, using shorter and longer working hours, and measure the corresponding earnings.

ò Question

Read the short, clearly written article by Stephen Few on the pitfalls of pie charts: Save the pies for
dessert, [Link] . The article presents an
easy-to-read argument against pie charts that will hopefully convince you.

Here’s a great example that proves his point30 from the Canada Revenue Agency.

ò Question

Enrichment:

• Watch this 20 minute video31 that shows how a 2-dimensional plot comes alive to show 5
dimensions of data. What are the 5 dimensions?

• A condensed version from this, 4 minute YouTube video32 shows Hans Rosling giving a new
perspective on the same data. This Economist article33 has some interesting background on Dr.
Rosling, as does this page, giving a selection of his work34 .
Video for
this section

29 [Link]
30 [Link]
html
31 [Link]
32 [Link]
33 [Link]
34 [Link]

1.10. Exercises 35
Process Improvement Using Data

36 Chapter 1. Visualizing Process Data

 CHAPTER 2
UNIVARIATE DATA ANALYSIS

2.1 Univariate data analysis in context

This section gives a starting idea to the general area of data analysis. We cover concepts from
univariate data analysis shown in the pictorial outline below. This section is only a review of these
concepts for one single variable. If you have more than one variable, you can repeat the analysis for
each one. Later, in the multivariate chapter (page 411), we learn how to extract information from
multiple variables at the same time.

Some introductory statistics textbooks, for more detailed background, are recommend further down.

2.1.1 Usage examples

Video for
this section The material in this section is used whenever you want to learn more about a single variable in your
data set. For example:

• Co-worker: Here are the final output values, on a scale from 0 to 100%, from a batch system for the
last 3 years (1256 data points).

– What sort of distribution do the data have?

– Yesterday our output value was less than 50%, what are the chances of that happening under
typical conditions?

• Yourself : We have historical failure rate data for certain equipment in our factories. What is the
probability that 3 of the same type of equipment will fail this year?

• Manager: We have 2 duplicate reactors. Does reactor 1 have better final product purity, on average,
than reactor 2?

• Colleague: What does the 95% confidence interval for the density of our powder ingredient really
mean?

37
Process Improvement Using Data

2.1.2 What we will cover

2.2 References and readings

Any standard statistics text book will cover the topics from this part of the book in much greater depth
than these notes. Some that you might refer to:

1. Recommended: Box, Hunter and Hunter, Statistics for Experimenters, Chapter 2.

2. Hodges and Lehmann, Basic Concepts of Probability and Statistics.

3. Hogg and Ledolter, Engineering Statistics.

4. Montgomery and Runger, Applied Statistics and Probability for Engineers.

2.3 What is variability?

Life is pretty boring without variability, and this book, and almost all the field of statistics would be
unnecessary if things did not naturally vary.

Fortunately, we have plenty of variability in the recorded data from our processes and systems:

• Raw material properties are not constant.

• Unknown sources, often called “error” (note that the word error in statistics does not have the usual
negative connotation from English). These errors are all sources of variation which our imperfect

38 Chapter 2. Univariate Data Analysis

 Release 547b16

knowledge of the process cannot account for.

• Measurement and sampling variability: sensor drift, spikes, noise, recalibration shifts, errors in our
sample analysis and laboratory equipment.

• Production disturbances:

– external conditions change, such as ambient temperature, or humidity, and

– pieces of plant equipment break down, wear out and are replaced.

• Feedback control systems introduce variability in your process, in order to reduce variability in
another part of the process. Think of what a feedback control system (page 40) does. See page 222 or
page 879 of the freely available textbook by Dr. Thomas Marlin35 for visual illustrations.

• Operating staff: introduce variability into a process in feedback manner (i.e. they react to process
upsets) or in a feed-forward manner, for example, to preemptively act on the process to counteract a
known disturbance. By doing so they introduce variability into a process.

All this variability, although a good opportunity to keep many of use employed, comes at a price as
described next.
35 [Link]

2.3. What is variability? 39

Process Improvement Using Data

2.3.1 The high cost of variability in your final product

Assertion
Customers expect both uniformity and low cost when they buy your product. Variability defeats
both objectives.

Three broad outcomes are possible when you sell a highly variable product:

1. The customer may be totally unable to use your product for the intended purpose. Imagine a food
ingredient such as fresh milk, or a polymer with viscosity that is too high, or a motor oil with
unsuitable properties that causes engine failure.

2. Your product leads to poor performance. The user must compensate for the poor properties through
additional cost: more energy will be required to work with a polymer whose melting point is higher
than expected, longer reaction times will be required if the catalyst purity is not at specification.

3. Your brand is diminished: your products, even though acceptable will be considered with suspicion
in the future.

An extreme example was the food poisoning and deaths that occurred due to the listeriosis
outbreak at Maple Leaf Foods, Canada in 2008. The bacterial count in food products is always
non-zero, however the established tolerance limits were exceeded during this outbreak.

Another example was the inadvertent acceleration that occurred in some Toyota car models in
2010. It is still uncertain whether this was manufacturer error or driver error.

In addition to the risk of decreasing your market share (see the above 3 points), variability in your
product also has these costs:

1. Inspection costs: to mitigate the above risks you must inspect your product before you ship it to
your customers. It is prohibitively expensive and inefficient to test every product (known as
“inspecting quality into your product”). A production line with low variability on the other hand,
requires less inspection of every product.

The pharmaceutical industry is well known to be inefficient in this respect, with terms such as
“100% inspection” and even “200% inspection”. Furthermore, some types of inspection are
destructive, and therefore 100% inspection is not feasible.

2. Off-specification products: must be reworked, disposed of, or sold at a loss or much lower profit.
These costs are ultimately passed onto your customers, costing you money.

Note: the above discussion assumes that you are able to quantify product quality with one or more
univariate quality metrics and that these metrics are independent of each other. Quality is almost
always a multivariate attribute of the product. We will discuss the use of multivariate methods (page 411)
to judge product quality later.

2.3.2 The high cost of variability in your raw materials

Turning the above discussion around, with you on the receiving end of a highly variable raw material:

• If you do not implement any sort of process control system, then any variability in these raw
materials that you receive and process is manifest as variability in your final product. This usually
shows up in proportion: higher variability in the inputs results in higher variability in the product
quality.

40 Chapter 2. Univariate Data Analysis

 Release 547b16

• Even if you do take feedback or feed-forward corrective control: you have to incur additional cost,
since you have to process materials that are not to specification: this will require energy and/or
time, reducing your profit due to the supplier’s raw material variability.

Note: Feedback control around a given set point can be seen as introducing additional variation into
a process to counteract other sources of variation (called disturbances in the process control lingo).
This is done with the hope of reducing the output variability.

2.3.3 Dealing with variability

So, how do we make progress despite this variability? This whole book, and all of statistical data
analysis, is about variability:

• in the data visualization section (page 1) we gave some hints how to plot graphics that show the
variability in our process clearly

• in this chapter we learn how to quantify variability and then compare variability

• later we consider how to construct monitoring charts (page 149) to track variability

• in the section on least squares modelling (page 209) we learn how variation in one variable might
affect another variable

• with designed experiments (page 319) we intentionally introduce variation into our process to learn
more about the process (e.g. so that we can optimize our process for improved profitability); and

• and in the latent variable modelling (page 411) section we learn how to deal with multiple variables,
simultaneously extracting information from the data to understand how variability affects the
process.

2.3. What is variability? 41

 Process Improvement Using Data

2.4 Histograms and probability distributions

Video for
this section The previous section (page 38) has hopefully convinced you that variation in a process is inevitable. This
section aims to show how we can visualize and quantify any variability in a recorded vector of data.

A histogram is a summary of the variation in a measured variable. It shows the number of samples that
occur in a category: this is called a frequency distribution. For example: number of children born,
categorized against their birth gender: male or female.

The raw data in the above example was a vector that consisted of 2739 text entries, with 1420 of them
as Male and 1319 of them as Female. In this case Female and Male represent the two categories.

Histograms make sense for categorical variables, but a histogram can also be derived from a
continuous variable. Here is an example showing the mass of cartons of 1 kg of flour. The continuous
variable, mass, is divided into equal-size bins that cover the range of the available data. Notice how
the packaging system has to overfill each carton so that the vast majority of packages weigh over 1 kg
(what is the average package mass?). If the variability in the packaging system could be reduced - the
spread of the data made narrower - then the histogram can be shifted to the left, thereby reducing
overfill.

42 Chapter 2. Univariate Data Analysis

 Release 547b16

# Create 500 normally distributed points

# with a mean of 1100 and standard deviation
# of 50 units.
import numpy as np
import pandas as pd

[Link] = "plotly"

N = 500
values = [Link]([Link](loc=1100, scale=50, size=N))

fig = [Link](nbins=8)
fig.update_layout(
xaxis_title_text="Mass [g] of each package",
yaxis_title_text=f"Number of packages (N={N})",
showlegend=False,
)
[Link]()

# Create 500 normally distributed points

# with a mean of 1100 and standard deviation
# of 50 units.
data = rnorm(500, mean=1100, sd=50)
hist(data,
xlab="Mass [g] of each package",
ylab="Number of packages (N=500)")

Try creating a fictitious histogram for each of the following situations:

• The grades for a class for a really easy test.

• The numbers thrown from a 6-sided die.

• The annual income for people in your country.

• Analytical measurements taken in a laboratory, by the same person or computerized process.

In preparing the above histograms, what have you implicitly inferred about time-scales? These
histograms show the long-term distribution (probabilities) of the system being considered. This is why

2.4. Histograms and probability distributions 43

Process Improvement Using Data

concepts of chance and random phenomena can be use to described systems and processes. Probabilities
can be used to describe our long-term expectations. Let us contrast some long-term and short-term
expectations next:

• The long-term sex ratio at birth 1.06:1 (boy:girl) is expected in Canada; but a newly pregnant mother
would not know the sex.

• The long-term data from a process shows an 85% output yield from our batch reactor; but tomorrow
it could be 59% and the day after that 86%.

• We know that a fair die has a 16.67% chance of showing a 4 when thrown, but we cannot predict the
value of the next throw.

Even if we have complete mechanistic knowledge of our process, the concepts from probability and
statistics are useful to summarize and communicate information about past behaviour, and the
expected future behaviour.

Steps to creating a frequency distribution, illustrated with 4 examples, labelled A, B, C, and D.

1. Decide what you are measuring:

A. acceptable or unacceptable metal appearance: yes/no

B. number of defects on a metal sheet: none, low, medium, high

C. yield from the batch reactor: somewhat continuous - quantized due to rounding to the closest
integer

D. daily ambient temperature, in Kelvin: continuous values

2. Decide on a resolution for the measurement axis:

A. acceptable/unacceptable (1/0) code for the metal’s appearance

B. use a scale from 1 to 4 that grades the metal’s appearance

C. batch yield is measured in 1% increments, reported either as 78, 79, 80, 81%, etc.

D. temperature is measured to a 0.05 K precision, but we can report the values in bins of 5K

3. Report the number of observations in the sample or population that fall within each bin (resolution
step):

A. number of metal pieces with appearance level “acceptable” and “unacceptable” are added up

B. number of pieces with defect level 1, 2, 3, 4 are counted

C. number of batches with yield inside each bin level are calculated

D. number of temperature values inside each bin level are computed

4. Plot the number of observations in category as a bar plot. If you plot the number of observations
divided by the total number of observations, 𝑁 , then you are plotting the relative frequency.

A relative frequency, also called density, is sometimes preferred:

• we do not need to report the total number of observations, 𝑁

• it can be compared to other distributions

• if 𝑁 is large enough, then the relative frequency histogram starts to resemble the population’s
distribution

44 Chapter 2. Univariate Data Analysis

 Release 547b16

• the area under the histogram is equal to 1, and related to probability

# Create 1000 normally distributed points

# with mean of 0 and standard deviation of 1.
import numpy as np
import pandas as pd

[Link] = "plotly"

N = 1000
values = [Link]([Link](loc=0, scale=1, size=N))

# Frequency histogram (counts).

fig = [Link]()
fig.update_layout(
yaxis_title_text=f"Frequency (N={N})",
showlegend=False,
)
[Link]()

# Relative-density histogram (area sums to 1).

fig = [Link](histnorm="probability density")
fig.update_layout(
yaxis_title_text="Relative density",
showlegend=False,
)
[Link]()

# 1000 normally distributed values

N = 1000
values = rnorm(N)
hist(values, freq=TRUE, xlab="Random values",
[Link]=1.5, [Link]=1.8, lwd=2,
[Link]=1.8, [Link]=1.8,
ylab=paste0("Frequency (N=",N,")"))
hist(values, freq=FALSE, xlab="Random values",
[Link]=1.5, [Link]=1.8, lwd=2,
[Link]=1.8, [Link]=1.8,
ylab="Relative density")

# Compare the two plots: only the y-axis

# changes but the general shape remains.

2.4. Histograms and probability distributions 45

 Process Improvement Using Data

2.5 Some terminology

Video for
this section We review a couple of concepts that you should have seen in a prior statistical course or elsewhere. If
unfamiliar, please type the word or concept in a search engine for more background.

Population

A large collection of observations that might occur; a set of potential measurements. Some texts
consider an infinite collection of observations, but a large number of observations is good
enough.

Sample

A collection of observations that have actually occurred; a set of existing measurements that we
have recorded in some way, usually electronically.

In engineering applications where we have plenty of data, we can characterize the population
from all available data. The figure here shows the viscosity of a motor oil, from all batches
produced in the last 5 years (about 1 batch per day). These 1825 data points, though technically a
sample are an excellent surrogate for the population viscosity because they come from such a long
duration. Once we have characterized these samples, future viscosity values will likely follow
that same distribution, provided the process continues to operate in a similar manner.

Distribution

Distributions are used to summarize, in a compact way, a much larger collection of a much larger
collection of data points. Histograms, just discussed above, are one way of visualizing a
distribution. We can also express distributions by a few numerical parameters. See below.

Probability

The area under a plot of relative frequency distribution is equal to 1. Probability is then the
fraction of the area under the frequency distribution curve (also called density curve).

Superimpose a vertical line on your fictitious histograms you drew earlier to indicate:

• the probability of a test grades less than 80%;

• the probability that the number thrown from a 6-sided die is less than or equal to 2;

• the probability of someone’s income exceeding $60000;

• the probability of the measurement exceeding a certain critical value.

Parameter

46 Chapter 2. Univariate Data Analysis

 Release 547b16

A parameter is a value that describes the population’s distribution in some way. For example,
the population mean.

Statistic

A statistic is an estimate of a population parameter.

Mean (location)

The mean, or average, is a measure of location of the distribution. For each measurement, 𝑥𝑖 , in
your sample, the population mean is also called the expected value, written as ℰ {𝑥}.

1 ∑︁
population mean: ℰ {𝑥} = 𝜇 = 𝑥
𝑁
𝑛
1 ∑︁
sample mean: 𝑥= 𝑥𝑖
𝑛 𝑖=1

where 𝑁 represents the size of the entire population, and 𝑛 is the number of samples measured
from the population.

# A vector of 50 normally distributed

# random numbers. If you have Python 3.8
# or higher, consider using the
# 'statistics' package instead.

import numpy as np

N = 50
x = [Link](size=N)
print([Link](x))
# Run the code several times, to check
# that the mean is approximately 0
# Check what the 'x' variable contains.

# A vector of 50 normally distributed

# random numbers
N = 50
x = rnorm(N)
mean(x)

# Run the code several times, to check

# that the mean is approximately 0
# Check what the 'x' variable contains.

This is only one of several statistics that describes your data: if you told your customer that the
average density of your liquid product was 1.421 g/L, and nothing further, the customer might
assume all lots of the same product have a density of 1.421 g/L. But we know from our earlier
discussion (page 38) that there will be variation. We need information, in addition to the mean, to
quantify the distribution of values: the spread.

Variance (spread)

A measure of spread, or variance, is also essential to quantify your distribution.

1 ∑︁
𝒱 {𝑥} = ℰ (𝑥 − 𝜇)2 = 𝜎 2 = (𝑥 − 𝜇)2
{︀ }︀
Population variance :
𝑁
𝑛
1 ∑︁
Sample variance : 𝑠2 = (𝑥𝑖 − 𝑥)2
𝑛 − 1 𝑖=1

2.5. Some terminology 47

 Process Improvement Using Data

Dividing by 𝑛 − 1 makes the variance statistic, 𝑠2 , an unbiased estimator of the population

variance, 𝜎 2 . However, in many data sets our value for 𝑛 is large, so using a divisor of 𝑛, which
you might come across in computer software or other texts, rather than 𝑛 − 1 as shown here,
leads to little difference.

# A vector of 50 normally distributed

# random numbers with a standard
# deviation of 5.
# If you have Python 3.8
# or higher, consider using the
# 'statistics' package instead.

import numpy as np

N = 50
spread = 5
x = [Link](loc=0, scale=spread, size=N)
print("Standard deviation = " +\
str([Link](x)))
print("The variance is = " +\
str([Link](x)))
print("Square root of variance = " +\
str([Link]([Link](x))))

# Run the code several times.

# A vector of 50 normally distributed

# random numbers with a standard
# deviation of 5
N = 50
spread = 5
x = rnorm(N, sd=spread)

paste0('Standard deviation = ',

round(sd(x), 3))
paste0('The variance is = ',
round(var(x), 3))
paste0('Square root of variance = ',
round(sqrt(var(x)), 3))

# Run the code several times.

The square root of variance, called the standard deviation is a more useful measure of spread: it
is easier to visualize on a histogram and has the advantage of being in the same units of
measurement as the variable itself.

Degrees of freedom

The denominator in the sample variance calculation, 𝑛 − 1, is called the degrees of freedom. We
have one fewer than 𝑛 degrees of freedom, because there is a constraint that the sum of the
deviations around 𝑥 must add up to zero. This constraint is from the definition of the mean.
However, if we knew what the sample mean was without having to estimate it, then we could
subtract each 𝑥𝑖 from that value, and our degrees of freedom would be 𝑛.

Video for Outliers

this section
Outliers are hard to define precisely, but an acceptable definition is that an outlier is a point that
is unusual, given the context of the surrounding data. Another definition which is less useful, but
nevertheless points out the problem of concretely defining what an outlier is, is this: “An outlier -
I know it when I see it!”

48 Chapter 2. Univariate Data Analysis

 Release 547b16

The following 2 sequences of numbers show the number 4024 that appears in the first sequence,
has become an outlier in the second sequence. It is an outlier based on the surrounding context.

• 4024, 5152, 2314, 6360, 4915, 9552, 2415, 6402, 6261

• 4, 61, 12, 64, 4024, 52, -8, 67, 104, 24

Median (robust measure of location)

The median is an alternative measure of location. It is a sample statistic, not a population

statistic, and is computed by sorting the data and taking the middle value (or average of the
middle 2 values, for even 𝑛). It is also called a robust statistic, because it is insensitive (robust) to
outliers in the data.

ò Note

The median is the most robust estimator of the sample location: it has a breakdown of 50%,
which means that just under 50% of the data need to be replaced with unusual values before
the median breaks down as a suitable estimate. The mean on the other hand has a breakdown
value of 1/𝑛, as only one of the data points needs to be unusual to cause the mean to be a
poor estimate. To compute the median in R, use the median(x) function on a vector x.

Governments will report the median income, rather than the mean, to avoid influencing the
value with the few very high earners and the many low earners. The median income per person
is a more fair measure of location in this case.

Median absolute deviation, MAD (robust measure of spread)

A robust measure of spread is the MAD, the median absolute deviation. The name is descriptive
of how the MAD is computed:

mad {𝑥𝑖 } = 𝑐 · median {‖𝑥𝑖 − median {𝑥𝑖 } ‖} where 𝑐 = 1.4826

The constant 𝑐 makes the MAD consistent with the standard deviation when the observations 𝑥𝑖
are normally distributed. The MAD has a breakdown point of 50%, because like the median, we
can replace just under half the data with outliers before the MAD estimate becomes unbounded.
To compute the MAD in R, use the mad(x) function on a vector x.

# A vector of 500 normally distributed

# random numbers.
import numpy as np
from [Link] import median_abs_deviation

x = [Link](size=500)

print("Without any outliers:")

print(f"Standard deviation = {[Link](x, ddof=1)}")
# scale='normal' applies the 1.4826 factor so the
# MAD agrees with sd() for normal data.
print(f"The MAD is = "
f"{median_abs_deviation(x, scale='normal')}")
print("These two should agree mostly.")

# Run it several times to verify that the

# two are similar, when there are no
# outliers.

(continues on next page)

2.5. Some terminology 49

Process Improvement Using Data

(continued from previous page)

# Now add a huge outlier:
x[1] = 9876
print("But now add an outlier...")
print(f"*Standard deviation = {[Link](x, ddof=1)}")
print(f"*The MAD is = "
f"{median_abs_deviation(x, scale='normal')}")
print("See how MAD is not affected.")

# A vector of 500 normally distributed

# random numbers

x <- rnorm(500)

paste0('Without any outliers:')

paste0('Standard deviation = ', sd(x))
paste0('The MAD is = ', mad(x))
print('These two should agree mostly')

# Run it several times to verify that the

# two are similar, when they are not
# outliers

# Now add a huge outlier:

x[2] <- 9876
paste0('But now add an outlier...')
paste0('*Standard deviation = ', sd(x))
paste0('*The MAD is = ', mad(x))
paste0('See how MAD is not affected.')

Enrichment reading: read pages 1 to 8 of “Tutorial to Robust Statistics36 ”, PJ Rousseeuw, Journal of

Chemometrics, 5, 1-20, 1991.

2.6 Binary (Bernoulli) distribution

Systems that have binary outcomes (pass/fail; yes/no) must obey the probability principle that:
𝑝(pass) + 𝑝(fail) = 1. That is, the sum of the probabilities of the two possible outcomes must add up to
exactly one. A Bernoulli distribution only has a single parameter, 𝑝1 , the probability of observing event
1. The probability of the second event is the difference with 1: that is 𝑝2 = 1 − 𝑝1 .

An example: a histogram for a system that produces 70% acceptable product, 𝑝(pass) = 0.7, could look
like:

36 [Link]

50 Chapter 2. Univariate Data Analysis

 Release 547b16

If each observation is independent of the other, then:

• For the above system where 𝑝(pass) = 0.7, what is probability of seeing the following sequential
outcomes: pass, pass, pass (3 times in a row)?

(0.7)(0.7)(0.7) = 0.343, about one third

• What is the probability of seeing the sequence: pass, fail, pass, fail, pass, fail?

(0.7)(0.3)(0.7)(0.3)(0.7)(0.3) = 0.0093, less than 1%

Another example: you work in a company that produces tablets. The machine creates acceptable,
unbroken tablets 97% of the time, so 𝑝acceptable = 0.97, so 𝑝defective = 0.03.

• In a future batch of 850,000 tablets, how many tablets are expected to be defective? (Most companies
will call this quantity “the cost of waste”.)

850000 × (1 − 0.97) = 25500 tablets per batch will be defective

• You take a random sample of 𝑛 tablets from a large population of 𝑁 tablets. What is the chance that
all 𝑛 tablets are acceptable if 𝑝 is the Bernoulli population parameter of finding acceptable tablets:

Sample size 𝑝 = 95% 𝑝 = 97%

𝑛 = 10
𝑛 = 50
𝑛 = 100

• Are you surprised by the large reduction in the number of defective tablets for only a small increase
in 𝑝? It is for this reason that a well-performing process producing accetable product does not need
to have inspection of every product produced.

2.7 Uniform distribution

A uniform distribution arises when an observation’s value is equally as likely to occur as all the other
options of the recorded values. The classic example are dice: each face of a die is equally as likely to
show up as any of the other faces. This forms a discrete, uniform distribution.

The histogram for an event with 4 possible outcomes that are uniformly distributed is shown below.
Notice that the sample histogram will not necessarily have equal bar heights for all categories (bins),
especially for small sample sizes.

2.7. Uniform distribution 51

Process Improvement Using Data

You can simulate uniformly distributed random numbers in most software packages. As an example,
to generate 50 uniformly distributed random integers between 2 and 10, inclusive, in various
languages:

import numpy as np
([Link](50) * (10 - 2) + 2).round()

# run the code several times to verify

# the numbers are between 2 and 10

[Link](runif(50, 2, 11))

# run the code several times to verify

# the numbers are between 2 and 10

A continuous, uniform distribution arises when there is equal probability of every measurement
occurring within a given lower- and upper-bound. This sort of phenomena is not often found in
practice. Usually, continuous measurements follow some other distribution, of which we will discuss
the normal and 𝑡-distribution next.

2.8 Normal distribution

Before introducing the normal distribution, we first look at two important concepts: the Central limit
theorem, and the concept of independence. Both concepts are used in important derivations, based on
the normal distribution.

52 Chapter 2. Univariate Data Analysis

 Release 547b16

2.8.1 Central limit theorem

Video for
this section The Central limit theorem plays an important role in the theory of probability and in the derivation of
the normal distribution. We don’t prove this theorem here, but we only use the result that:

The average of a sequence of values from any distribution will approach the normal distribution,
provided the original distribution has finite variance.

The condition of finite variance is true for almost all systems of practical interest.

The critical requirement for the central limit theorem to be true is that the samples used to compute
that average are independent of each together. The average produced from such samples will be more
nearly normal though. Note: we do not require the original data to be normally distributed. This is a
common misconception though.

Imagine a case where we are throwing dice. The distributions, shown below, are obtained when we
throw a die 𝑀 times and we plot the distribution of the average of these 𝑀 throws.

As one sees from the above figures, the distribution from these averages quickly takes the shape of the
so-called normal distribution. As 𝑀 increases, the y-axis starts to form a peak. Try it yourself:

2.8. Normal distribution 53

Process Improvement Using Data

import numpy as np
import pandas as pd
from [Link] import make_subplots
import plotly.graph_objects as go

N = 500

# Throw N six-sided dice, ten times.

throws = [Link](1, 7, size=(10, N))

groupings = [(1, "One throw", 6),

(2, "Average of two throws", 8),
(4, "Average of 4 throws", 8),
(6, "Average of 6 throws", 8),
(8, "Average of 8 throws", 12),
(10, "Average of 10 throws", 12)]

fig = make_subplots(rows=2, cols=3,

subplot_titles=[g[1] for g in groupings])
for k, (m, _label, bins) in enumerate(groupings):
averages = throws[:m].mean(axis=0)
fig.add_trace(
[Link](x=averages, nbinsx=bins, showlegend=False),
row=k // 3 + 1, col=k % 3 + 1,
)
[Link]()

N = 500

# Layout the plots in 2 rows and 3 columns

m <- t(matrix(seq(1,6), 3, 2))
layout(m)

# Throw the dice several times

s1 <- [Link](runif(N, 1, 7))
s2 <- [Link](runif(N, 1, 7))
s3 <- [Link](runif(N, 1, 7))
s4 <- [Link](runif(N, 1, 7))
s5 <- [Link](runif(N, 1, 7))
s6 <- [Link](runif(N, 1, 7))
s7 <- [Link](runif(N, 1, 7))
s8 <- [Link](runif(N, 1, 7))
s9 <- [Link](runif(N, 1, 7))
s10 <- [Link](runif(N, 1, 7))

hist(s1, main="", xlab="One throw", breaks=seq(0,6)+0.5)

bins = 8
hist((s1+s2)/2, breaks=bins,
main="", xlab="Average of two throws")
hist((s1+s2+s3+s4)/4, breaks=bins,
main="", xlab="Average of 4 throws")
hist((s1+s2+s3+s4+s5+s6)/6, breaks=bins,
main="", xlab="Average of 6 throws")
bins=12
hist((s1+s2+s3+s4+s5+s6+s7+s8)/8, breaks=bins,
main="", xlab="Average of 8 throws")
hist((s1+s2+s3+s4+s5+s6+s7+s8+s9+s10)/10, breaks=bins,
main="", xlab="Average of 10 throws")

What is the engineering significance of this averaging process (which is really just a weighted sum)?
Many of the quantities we measure are bulk properties, such as viscosity, density, or particle size. We
can conceptually imagine that the bulk property measured is the combination of the same property,

54 Chapter 2. Univariate Data Analysis

 Release 547b16

measured on smaller and smaller components. Even if the value measured on the smaller component
is not normally distributed, the bulk property will be as if it came from a normal distribution.

2.8.2 Independence
The assumption of independence is widely used in statistical work and is a condition for using the
central limit theorem.

ò Note

The assumption of independence means that the samples we have in front of us are randomly taken
from a population. If two samples are independent, there is no possible relationship between them.

We frequently violate this assumption of independence in engineering applications. Think about these
examples for a while:

• A questionnaire is given to a group of people. What happens if they discuss the questionnaire in
sub-groups prior to handing it in?

We are not going to receive 𝑛 independent answers, rather we will receive as many
independent opinions as there are sub-groups.

• The rainfall amount, recorded every day, over the last 30 days.

These data are not independent: if it rains today, it can likely rain tomorrow as the weather
usually stays around for some days. These data are not useful as a representative sample of
typical rainfall, however they are useful for complaining about the weather. Think about the
case if we had considered rainfall in hourly intervals, rather than daily intervals.

• The snowfall, recorded on 3 January for every year since 1976: independent or not?

These sampled data will be independent.

• The impurity values in the last 100 batches of product produced is shown below. Which of the 3
time sequences has independent values?

In chemical processes there is often a transfer from batch-to-batch: we usually use the same lot of
raw materials for successive batches, the batch reactor may not have been cleaned properly between
each run, and so on. It is very likely that two successive batches (𝑘 and 𝑘 + 1) are somewhat related,
and less likely that batch 𝑘 and 𝑘 + 2 are related. In the figure below, can you tell which sequence of
values are independent?

2.8. Normal distribution 55

 Process Improvement Using Data

Sequence 2 (sequence 1 is positively correlated, while sequence 3 is negatively correlated).

• We need a highly reliable pressure release system. Manufacturer A sells a system that fails 1 in
every 100 occasions, and manufacturer B sells a system that fails 3 times in every 1000 occasions.
Given this information, answer the following:

– The probability that system A fails: 𝑝(Afails ) = 1/100

– The probability that system B fails:𝑝(Bfails ) = 3/1000

– The probability that both system A and fail at the same time:
1
𝑝(both A and B fail) = 100 3
· 1000 = 3 × 10−5 , but only if system A and B are totally independent.

– For the previous question, what does it mean for system A to be totally independent of system B?

It means the 2 systems must be installed in parallel, so that there is no interaction between
them at all.

– How would the probability of both A and B failing simultaneously change if A and B were not
independent?

The probability of both failing simultaneously will increase.

2.8.3 Formal definition for the normal distribution

Video for
this section
2
(𝑥 − 𝜇)
1 −
𝑝(𝑥) = √ 𝑒 2𝜎 2
2𝜋𝜎 2

56 Chapter 2. Univariate Data Analysis

 Release 547b16

• 𝑥 is the variable of interest

• 𝑝(𝑥) is the probability of obtaining that value of 𝑥

• 𝜇 is the population average for the distribution (first parameter)

• 𝜎 is the population standard deviation for the distribution, and is always a positive quantity
(second parameter)

Some questions:

1. What is the maximum value of 𝑝(𝑥) and where does it occur, using the formula above?

2. What happens to the shape of 𝑝(𝑥) as 𝜎 gets larger ?

3. What happens to the shape of 𝑝(𝑥) as 𝜎 → 0 ?

4. Fill out this table:

𝑥 𝜎 𝜇 𝑝(𝑥)
0 1 0
1 1 0
-1 1 0

To calculate the point on the curve 𝑝(𝑥) we use the dnorm(...) function in R. It requires you specify
the two parameters:

from [Link] import norm

# x=0, mu=0, and sigma=1

# This is the maximum of the curve
[Link](x=0, loc=0, scale=1) # 0.3989423

# x=1, mu=0, and sigma=1

[Link](x=1, loc=0, scale=1) # 0.2419707

# x=-1, mu=0, and sigma=1

# It is symmetrical
(continues on next page)

2.8. Normal distribution 57

Process Improvement Using Data

(continued from previous page)

[Link](x=-1, loc=0, scale=1) # 0.2419707

# x=+3, mu=0, and sigma=1

# This is at a point very far from center
[Link](x=+3, loc=0, scale=1) # 0.00443185

# x=0, mu=0, and sigma=1

# This is the maximum of the curve
dnorm(x = 0, mean = 0, sd = 1) # 0.3989423

# x=1, mu=0, and sigma=1

dnorm(x = 1, mean = 0, sd = 1) # 0.2419707

# x=-1, mu=0, and sigma=1

# It is symmetrical
dnorm(x = -1, mean = 0, sd = 1) # 0.2419707

# x=+3, mu=0, and sigma=1

# This is at a point very far from center
dnorm(x = +3, mean = 0, sd = 1) # 0.00443185

Some useful points:

• The total area from 𝑥 = −∞ to 𝑥 = +∞ is 1.0; we cannot calculate the integral of 𝑝(𝑥) analytically.

• 𝜎 is the distance from the mean, 𝜇, to the point of inflection

• The normal distribution only requires two parameters to describe it: 𝜇 and 𝜎

• The area from 𝑥 = −𝜎 to 𝑥 = 𝜎 is about 70% (68.3% exactly) of the distribution. So we have a
probability of about 15% of seeing an 𝑥 value greater than 𝑥 = 𝜎, and also 15% of 𝑥 < −𝜎

• The tail area outside ±2𝜎 is about 5% (2.275 outside each tail)

It is more useful to calculate the area under 𝑝(𝑥) from 𝑥 = −∞ to a particular point 𝑥. This is called the
cumulative distribution, and is discussed more fully in the next section (page 61).

from [Link] import norm

# gives area from -Inf to -1,

# for mu=0, sigma=1
[Link](-1, loc=0, scale=1) # 0.1586553

# Gives area from -Inf to +1,

# for mu=0, sigma=1
[Link](1, loc=0, scale=1) # 0.8413447

# Spread is wider, but the

# fractional area is the same
[Link](3, loc=0, scale=3) # 0.8413447

# gives area from -Inf to -1,

# for mu=0, sigma=1
pnorm(-1, mean = 0, sd = 1) # 0.1586553

# Gives area from -Inf to +1,

# for mu=0, sigma=1
pnorm(1, mean = 0, sd = 1) # 0.8413447
(continues on next page)

58 Chapter 2. Univariate Data Analysis

 Release 547b16

(continued from previous page)

# Spread is wider, but the

# fractional area is the same
pnorm(3, mean = 0, sd = 3) # 0.8413447

You might still find yourself having to refer to tables of cumulative area under the normal distribution,
instead of using the pnorm() function (for example in a test or exam). If you look at the appendix of
most statistical texts you will find these tables, and there is one at the end of this chapter (page 95). Since
these tables cannot be produced for all combinations of mean and standard deviation parameters, they
Video for use what is called standard form.
this section

𝑥𝑖 − mean
𝑧𝑖 =
standard deviation
The values of the mean and standard deviation are either the population parameters, if known, or
using the best estimate of the mean and standard deviation from the sampled data. The transformed
quantity is called the z-value (or 𝑧-deviate).

For example, if our values of 𝑥𝑖 come from a normal distribution with mean of 34.2 and variance of 55.
Then we could write 𝑥 ∼ 𝒩 (34.2, 55), which is short-hand notation of saying the same thing. The
𝑥𝑖 − 34.2
equivalent 𝑧-values for these 𝑥𝑖 values would be: 𝑧𝑖 = √ .
55
This transformation to standard form does not change the distribution of the original 𝑥, it only
changes the parameters of the distribution. You can easily prove to yourself that 𝑧 is normally
distributed as 𝑧 ∼ 𝒩 (0.0, 1.0); the standard normal distribution has mean of zero and unit variance. So
statistical tables only report the area under the distribution of a 𝑧 value with mean of zero, and unit
variance.

This is a common statistical technique, to standardize a variable, which we will see several times.
Standardization takes our variable from 𝑥 ∼ 𝒩 (some mean, some variance) and converts it to
𝑧 ∼ 𝒩 (0.0, 1.0). It is just as easy to go backwards, from a given 𝑧-value and return back to our original
𝑥-value.

The units of 𝑧 are dimensionless, no matter what the original units of 𝑥 were. Standardization also
allows us to straightforwardly compare 2 variables that may have different means and spreads. For
example if our company has two reactors at different locations, producing the same product. We can
standardize a variable of interest, e.g. viscosity, from both reactors and then proceed to use the
standardized variables to compare performance.

Consult a statistical table found in most statistical textbooks for the normal distribution, such as the
one found at the end of this chapter (page 95). Make sure you can firstly understand how to read the
table. Secondly, duplicate a few entries in the table using R. Complete these small exercises by
estimating what the rough answer should be. Use the tables first, then use R to get a more accurate
estimate.

1. Assume 𝑥, the measurement of biological activity for a drug, is normally distributed with mean of
26.2 and standard deviation of 9.2. What is the probability of obtaining an activity reading less than
or equal to 30.0?

# We know that the probability should be 50%

# if the activity is equal to the mean.
(continues on next page)

2.8. Normal distribution 59

Process Improvement Using Data

(continued from previous page)

from [Link] import norm

x = 26.2
mu = 26.2
sigma = ____
[Link](x, loc=mu, scale=sigma)

# Now modify this above to answer the question.

# We know that the probability should be 50%

# if the activity is equal to the mean.

x <- 26.2
mu <- 26.2
sigma <- ____
pnorm(x, mean=mu, sd=sigma)

# Now modify this above to answer the question.

2. Assume 𝑥 is the yield for a batch process, with mean of 85 g/L and variance of 16 g2 .L−2 . What
proportion of batch yield values lie between 75 and 95 g/L?

import numpy as np
from [Link] import norm

mu = 85 # g/L
sigma = [Link](16) # g/L
x_left = ___
area_left_tail = [Link](x_left,
loc=mu,
scale=sigma)

x_right = ___
area_right_tail = [Link](x_right,
loc=mu,
scale=sigma)

# Now subtract the two areas to get

# the answer. Why?

mu <- 85 # g/L
sigma <- sqrt(16) # g/L
[Link] <- ___
[Link] <- pnorm([Link],
mean=mu,
sd=sigma)

[Link] <- ___

[Link] <- pnorm([Link],
mean=mu,
sd=sigma)

# Now subtract the two areas to get

# the answer. Why?

60 Chapter 2. Univariate Data Analysis

 Release 547b16

2.8.4 Checking for normality: using a q-q plot

Video for
this section Often we are not sure if a sample of data can be assumed to be normally distributed. This section
shows you how to test whether the data are normally distributed, or not.

Before we look at this method, we need to introduce the concept of the inverse cumulative distribution
function (inverse CDF). Recall the cumulative distribution is the area underneath the distribution
function, 𝑝(𝑧), which goes from −∞ to 𝑧. For example, the area from −∞ to 𝑧 = −1 is about 15%, as
we showed earlier, and we can use the pnorm() function in R to verify that.

Now the inverse cumulative distribution is used when we know the area, but want to get back to the
value along the 𝑧-axis. For example, below which value of 𝑧 does 95% of the area lie for a standardized
normal distribution? Answer: 𝑧 = 1.64. In R we use the qnorm(0.95, mean=0, sd=1) to calculate
this value. The q stands for quantile37 , because we give it the quantile and it returns the 𝑧-value: e.g.
qnorm(0.5) gives 0.0.

On to checking for normality. We start by first constructing some quantities that we would expect for
truly normally distributed data. Secondly, we construct the same quantities for the actual data. A plot
of these 2 quantities against each other will reveal if the data are normal, or not.

1. Imagine we have 𝑁 observations which are normally distributed. Sort the data from smallest to
largest. The first data point should be the (1/𝑁 × 100) quantile, the next data point is the
(2/𝑁 × 100) quantile, the middle, sorted data point is the 50th quantile, (1/2 × 100), and the last,
sorted data point is the (𝑁/𝑁 × 100) quantile.

The middle, sorted data point from this truly normal distribution must have a 𝑧-value on the
standardized scale of 0.0 (we can verify that by using qnorm(0.5)). By definition, 50% of the data
should lie below this mid point. The first data point will be at qnorm(1/N), the second at
qnorm(2/N), the middle data point at qnorm(0.5), and so on. In general, the 𝑖th sorted point
should be at qnorm((i-0.5)/N), for values of 𝑖 = 1, 2, . . . , 𝑁 . We subtract off 0.5 by convention to
37 [Link]

2.8. Normal distribution 61

Process Improvement Using Data

account for the fact that qnorm(1.0) = Inf. So we construct this vector of theoretically expected
quantities from the inverse cumulative distribution function.

N = 10
index = seq(1, N)
P = (index - 0.5) / N
P
[1] 0.05 0.15 0.25 0.35 0.45 0.55 0.65 0.75 0.85 0.95
[Link] = qnorm(P)
[1] -1.64 -1.04 -0.674 -0.385 -0.126 0.125 0.385 0.6744 1.036 1.64

2. We also construct the actual quantiles for the sampled data. First, standardize the sampled data by
subtracting off its mean and dividing by its standard deviation. Here is an example of 10 batch
yields (see actual values below). The mean yield is 80.0 and the standard deviation is 8.35. The
standardized yields are found by subtracting off the mean and dividing by the standard deviation.
Then the standardized values are sorted. Compare them to the theoretical quantities.

yields <- c(86.2, 85.7, 71.9, 95.3, 77.1, 71.4, 68.9, 78.9, 86.9, 78.4)
[Link] <- mean(yields) # 80.0
[Link] <- sd(yields) # 8.35

yields.z = (yields - [Link])/[Link]

[1] 0.734 0.674 -0.978 1.82 -0.35 -1.04 -1.34 -0.140 0.818 -0.200

[Link] = sort(yields.z)
[1] -1.34 -1.04 -0.978 -0.355 -0.200 -0.140 0.674 0.734 0.818 1.82

[Link] # numbers are rounded in the printed output

[1] -1.64 -1.04 -0.674 -0.385 -0.126 0.125 0.385 0.6744 1.036 1.64

3. The final step is to plot this data in a suitable way. If the sampled quantities match the theoretical
quantities, then a scatter plot of these numbers should form a 45 degree line.

plot([Link], [Link], type="p")

A built-in function exists in R that runs the above calculations and shows a scatter plot. The 45 degree
line is added using the qqline(...) function. However, a better function that adds a confidence
limit envelope is included in the car library (see the Package Installer menu in R for adding libraries

62 Chapter 2. Univariate Data Analysis

 Release 547b16

from the internet).

qqnorm(yields)
qqline(yields)

# or, using the ``car`` library

library(car)
qqPlot(yields)

All the above code together in one script for you to test out:

import numpy as np
import pandas as pd
from [Link] import norm, probplot

[Link] = "plotly"

N = 10
index = [Link](1, N + 1)
P = (index - 0.5) / N
theoretical_quantity = [Link](P)

yields = [Link]([86.2, 85.7, 71.9, 95.3, 77.1,

71.4, 68.9, 78.9, 86.9, 78.4])
mean_yield = [Link]() # 80.0
sd_yield = [Link](ddof=1) # 8.35

yields_z = (yields - mean_yield) / sd_yield

yields_z_sorted = [Link](yields_z)

# Manual q-q plot, on standardized scale.

fig = [Link](
{"theoretical": theoretical_quantity,
"sample": yields_z_sorted}
).[Link](x="theoretical", y="sample")
[Link]()

# Built-in q-q plot, on the original scale.

# [Link] returns (osm, osr) and
# the slope/intercept of the regression line.
osm, osr = probplot(yields, dist="norm",
fit=False)
(continues on next page)

2.8. Normal distribution 63

Process Improvement Using Data

(continued from previous page)

fig = [Link](
{"theoretical quantile": osm,
"yields": osr}
).[Link](x="theoretical quantile",
y="yields")
[Link]()

N = 10
index <- seq(1, N)
P <- (index - 0.5) / N
[Link] <- qnorm(P)

yields <- c(86.2, 85.7, 71.9, 95.3, 77.1,

71.4, 68.9, 78.9, 86.9, 78.4)
[Link] <- mean(yields) # 80.0
[Link] <- sd(yields) # 8.35

yields.z <- (yields - [Link])/[Link]

[Link] <- sort(yields.z)

plot([Link],
[Link],
type="p")

qqnorm(yields)
qqline(yields)

# or, using the ``car`` library

library(car)
qqPlot(yields)

The R plot rescales the 𝑦-axis (sample quantiles) back to the original units to make interpretation
easier. We expect some departure from the 45 degree line due to the fact that these are only a sample of
data. However, large deviations indicates the data are not normally distributed. An error region, or
confidence envelope, may be superimposed around the 45 degree line.

The q-q plot, quantile-quantile plot, shows the quantiles of 2 distributions against each other. In fact,
we can use the horizontal axis for any distribution, it need not be the theoretical normal distribution.
We might be interested if our data follow an 𝐹 -distribution then we could use the quantiles for that
theoretical distribution on the horizontal axis.

We can use the q-q plot to compare any 2 samples of data, even if they have different values of 𝑁 , by
calculating the quantiles for each sample at different step quantiles (e.g. 1, 2, 3, 4, 5, 10, 15, . . . . 95, 96,
97, 98, 99), then plot the q-q plot for the two samples. You can calculate quantiles for any sample of
data using the quantile function in R. The simple example below shows how to compare the q-q
plot for 1000 normal distribution samples against 2000 𝐹 -distribution samples.

import numpy as np
import pandas as pd
from [Link] import f, probplot

[Link] = "plotly"

# 1000 normal values

rand_norm = [Link](size=1000)

# 2000 values from F-distribution

rand_f = [Link](dfn=200, dfd=150, size=2000)
(continues on next page)

64 Chapter 2. Univariate Data Analysis

 Release 547b16

(continued from previous page)

# looks sort of normally distributed

fig = [Link](rand_f).[Link](
histnorm="probability density",
title="Are these data from a normal distribution?",
)
fig.update_layout(yaxis_title_text="Frequency")
[Link]()

# But your eye is being fooled ...

# See the heavy tail in the q-q plot.
osm, osr = probplot(rand_f, dist="norm",
fit=False)
fig = [Link](
{"theoretical quantile": osm,
"sample": osr}
).[Link](x="theoretical quantile",
y="sample")
[Link]()

# 1000 normal values

[Link] <- rnorm(1000)

# 2000 values from F-distribution

rand.f <- rf(2000, df1=200, df=150)

# looks sort of normally distributed

hist(rand.f, freq=FALSE, ylim=c(0, 2.6),
main="Are these data from a normal distribution?",
ylab="Frequency")

# Add the density line on top

lines(density(rand.f))

# But your eye is being fooled ...

# See the heavy tail
library(car)
qqPlot(rand.f, distribution="norm")

Even though the histogram of the 𝐹 -distribution samples looks normal to the eye (left), the q-q plot

2.8. Normal distribution 65

Process Improvement Using Data

(right) quickly confirms it is definitely not normal, particularly, that the right-tail is too heavy.

2.8.5 Introduction to confidence intervals from the normal distribution

We introduce the concept of confidence intervals here as a straightforward application of the normal
distribution, Central limit theorem, and standardization.

Suppose we have a quantity of interest from a process, such as the daily profit. We have many
measurements of this profit, and we can easily calculate the average profit. But we know that if we
take a different data set of profit values and calculate the average, we will get a similar, but different
average. Since we will never know the true population average, the question we want to answer is:

What is the range within which the true (population) average value lies? E.g. give a range for the
true, but unknown, daily profit.

This range is called a confidence interval, and we study them in more depth later on (page 77). We will
use an example to show how to calculate this range.

Let’s take 𝑛 values of this daily profit value, let’s say 𝑛 = 5.

𝑖=𝑛
1 ∑︁
1. An estimate of the population mean is given by 𝑥 = 𝑥𝑖 (we saw this before (page 47))
𝑛 𝑖
𝑖=𝑛
1 ∑︁
2. The estimated population variance is 𝑠2 = (𝑥𝑖 − 𝑥)2 (we also saw this before (page 47))
𝑛−1 𝑖

3. This is new: the estimated mean, 𝑥, is a value that is also normally distributed with mean of 𝜇 and
variance of 𝜎 2 /𝑛, with only one requirement: this result holds only if each of the 𝑥𝑖 values are
independent of each other.

Mathematically we write: 𝑥 ∼ 𝒩 𝜇, 𝜎 2 /𝑛 .
(︀ )︀

This important result helps answer our question above. It says that repeated estimates of the mean
will be an accurate, unbiased estimate of the population mean, and interestingly, the variance of
that estimate is decreased by using a greater number of samples, 𝑛, to estimate that mean. This
makes intuitive sense: the more independent samples of data we have, the better our estimate
(“better” in this case implies lower error, i.e. lower variance).

We can illustrate this result as shown below:

66 Chapter 2. Univariate Data Analysis

 Release 547b16

The true population (but unknown to us) profit value is $700.

• The 5 samples come from the distribution given by the thinner line: 𝑥 ∼ 𝒩 𝜇, 𝜎 2
(︀ )︀

• The 𝑥 average comes from the distribution given by the thicker line: 𝑥 ∼ 𝒩 𝜇, 𝜎 2 /𝑛 .
(︀ )︀

4. Creating 𝑧 values for each 𝑥𝑖 raw sample point:

𝑥𝑖 − 𝜇
𝑧𝑖 =
𝜎

5. The 𝑧-value for 𝑥 would be:

𝑥−𝜇
𝑧= √
𝜎/ 𝑛

which subtracts off the unknown population mean from our estimate of the mean, and divides
through by the standard deviation for 𝑥. We can illustrate this as:

2.8. Normal distribution 67

Process Improvement Using Data

6. Using the known normal distribution for 𝑥 ∼ 𝒩 𝜇, 𝜎 2 /𝑛 , we can find the vertical, dashed red lines
(︀ )︀

shown in the previous figure, that contain 95% of the area under the distribution for 𝑥.

7. These vertical lines are symmetrical about 0, and we will call them −𝑐𝑛 and +𝑐𝑛 , where the
subscript 𝑛 refers to the fact that they are from the normal distribution (it doesn’t refer to the 𝑛
samples). From the preceding section on q-q plots we know how to calculate the 𝑐𝑛 value from R:
using qnorm(1 - 0.05/2), so that there is 2.5% area in each tail.

8. Finally, we construct an interval for the true population mean, 𝜇, using the standard form:

−𝑐𝑛 ≤ 𝑧 ≤ +𝑐𝑛
𝑥−𝜇
−𝑐𝑛 ≤ √ ≤ +𝑐𝑛
𝜎/ 𝑛 (2.1)
𝜎 𝜎
𝑥 − 𝑐𝑛 √ ≤ 𝜇 ≤ 𝑥 + 𝑐𝑛 √
𝑛 𝑛
LB ≤ 𝜇 ≤ UB

Notice that the lower and upper bound are a function of the known sample mean, 𝑥, the values for
𝑐𝑛 which we chose, the known sample size, 𝑛, and the unknown population standard deviation, 𝜎.

So to estimate our bounds we must know the value of this population standard deviation. This is
not very likely, (I can’t think of any practical cases where we know the population standard
deviation, but not the population mean, which is the quantity we are constructing this range for),
however there is a hypothetical example in the next section (page 72) to illustrate the calculations.

The 𝑡-distribution is required to remove this impractical requirement of knowing the population
standard deviation.

68 Chapter 2. Univariate Data Analysis

 Release 547b16

2.9 The t-distribution

Suppose we have a quantity of interest from a process, such as the daily profit. In the preceding
section we started to answer the useful and important question:

What is the range within which the true average value lies? E.g. the range for the true, but
unknown, daily profit.

But we got stuck, because the lower and upper bounds we calculated for the true average, 𝜇 were a
function of the unknown population standard deviation, 𝜎. Repeating the prior equation for confidence
interval (page 79) where we know the variance:

𝑥−𝜇
−𝑐𝑛 ≤ √ ≤ +𝑐𝑛
𝜎/ 𝑛
𝜎 𝜎
𝑥 − 𝑐𝑛 √ ≤ 𝜇 ≤ 𝑥 + 𝑐𝑛 √
𝑛 𝑛
LB ≤ 𝜇 ≤ UB

𝑥−𝜇
which we derived by using the fact that √ is normally distributed.
𝜎/ 𝑛
An obvious way out of our dilemma is to replace 𝜎 by the sample standard deviation, 𝑠, which is
𝑥−𝜇
exactly what we will do, however, the quantity 𝑠/ √ is not normally distributed, but is 𝑡-distributed.
𝑛
Before we look at the details, it is helpful to see how similar in appearance the 𝑡 and normal
distribution are: the 𝑡-distribution peaks slightly lower than the normal distribution, but it has broader
tails. The total area under both curves illustrated here is 1.0.

There is one other requirement we have to ensure in order to use the 𝑡-distribution: the values that we
sample, 𝑥𝑖 must come from a normal distribution (carefully note that in the previous section we didn’t
have this restriction!). Fortunately it is easy to check this requirement: just use the q-q plot method
described earlier (page 61). Another requirement, which we had before, was that we must be sure these
measurements, 𝑥𝑖 , are independent.

2.9. The t-distribution 69

Process Improvement Using Data

So given our 𝑛 samples, which are independent, and from a normal distribution, we can now say:
𝑥−𝜇
√ ∼ 𝑡𝑛−1 (2.2)
𝑠/ 𝑛

Compare this to the previous case where our 𝑛 samples are independent, and we happen to know, by
some unusual way, what the population standard deviation is, 𝜎:
𝑥−𝜇
√ ∼ 𝒩 (0, 1)
𝜎/ 𝑛
𝑥−𝜇
√ ∼ 𝑡𝑛−1 can now be used to construct an interval
So the more practical and useful case where 𝑧 = 𝑠/ 𝑛
for 𝜇. We say that 𝑧 follows the 𝑡-distribution with 𝑛 − 1 degrees of freedom, where the degrees of
freedom refer to those from the calculating the estimated standard deviation, 𝑠.

Note that the new variable 𝑧 only requires we know the population mean (𝜇), not the population
√
standard deviation; rather we use our estimate of the standard deviation 𝑠/ 𝑛 in place of the
population standard deviation.

We will come back to (2.2) in a minute; let’s first look at how we can calculate values from the
𝑡-distribution in computer software.

2.9.1 Calculating the t-distribution

• In R we use the function dt(x=..., df=...) to give us the values of the probability density
values, 𝑝(𝑥), of the 𝑡-distribution (compare this to the dnorm(x, mean=..., sd=...) function
for the normal distribution).

from [Link] import norm, t

x = 0.0

# Recall, for the normal distribution:

[Link](x, loc=0, scale=1) # 0.3989423

# For the t-distribution we don't have

# a sigma, but we do need to say how
# many degrees of freedom we have:

dof = 8
[Link](x, df=dof) # 0.386699

# Shows that the t-distribution has a

# lower peak than the normal distribution.
# Try it again, but with fewer and
# greater degrees of freedom (`dof`).

x = 0.0

# Recall, for the normal distribution:

(continues on next page)

70 Chapter 2. Univariate Data Analysis

 Release 547b16

(continued from previous page)

dnorm(x, mean=0, sd=1) # 0.3989423

# For the t-distribution we don't have

# a sigma, but we do need to say how
# many degrees of freedom we have:

dof <- 8
dt(x, df=dof) # 0.386699

# Shows that the t-distribution has a

# lower peak than the normal distribution.
# Try it again, but with fewer and
# greater degrees of freedom (`dof`).

• The cumulative area from −∞ to 𝑥 under the probability density curve gives us the probability that
values less than or equal to 𝑥 could be observed. It is calculated in R using pt(q=..., df=...).
For example, pt(1.0, df=8) is 0.8267. Compare this to the R function for the standard normal
distribution: pnorm(1.0, mean=0, sd=1) which returns 0.8413.

from [Link] import norm, t

q = 1.0

# Recall, for the normal distribution:

[Link](q, loc=0, scale=1) # 0.8413447

# For the t-distribution we need to

# specify the degrees of freedom:

dof = 8
[Link](q, df=dof) # 0.8267032

# Shows that the t-distribution is

# similar, but the areas are slightly
# different.

q = 1.0

# Recall, for the normal distribution:

pnorm(q, mean=0, sd=1) # 0.8413447

# For the t-distribution we need to

# specify the degrees of freedom:

dof <- 8
pt(q, df=dof) # 0.8267032

# Shows that the t-distribution is

# similar, but the areas are slightly
# different.

• And similarly to the qnorm function which returns the ordinate for a given area under the normal
distribution, the function qt(0.8267, df=8) returns 0.9999857, close enough to 1.0, which is the
inverse of the previous example.

from [Link] import norm, t

p = 0.5
(continues on next page)

2.9. The t-distribution 71

 Process Improvement Using Data

(continued from previous page)

# Recall, for the normal distribution:

[Link](p, loc=0, scale=1) # 0.0

# For the t-distribution:

dof = 8
[Link](p, df=dof) # 0.0

# Both distributions have their 50%

# quantile at p=0. But try it for
# other values of probability, p.

p = 0.5

# Recall, for the normal distribution:

qnorm(p, mean=0, sd=1) # 0.0

# For the t-distribution:

dof <- 8
qt(p, df=dof) # 0.0

# Both distributions have their 50%

# quantile at p=0. But try it for
# other values of probability, p.

2.9.2 Using the t-distribution to calculate our confidence interval

Video for
this section Returning back to (2.2) we stated that
𝑥−𝜇
√ ∼ 𝑡𝑛−1
𝑠/ 𝑛
We can plot the 𝑡-distribution for a given value of 𝑛 − 1, the degrees of freedom. Then we can locate
vertical lines on the 𝑥-axis at −𝑐𝑡 and +𝑐𝑡 so that the area between the verticals covers say 95% of the
total distribution’s area. The subscript 𝑡 refers to the fact that these are critical values from the
𝑡-distribution.

Then we write:
−𝑐𝑡 ≤ 𝑧 ≤ +𝑐𝑡
𝑥−𝜇
−𝑐𝑡 ≤ √ ≤ +𝑐𝑡
𝑠/ 𝑛 (2.3)
𝑠 𝑠
𝑥 − 𝑐𝑡 √ ≤ 𝜇 ≤ 𝑥 + 𝑐𝑡 √
𝑛 𝑛
LB ≤ 𝜇 ≤ UB

Now all the terms in the lower and upper bound are known, or easily calculated.

So we finish this section off with an example. We produce large cubes of polymer product on our
process. We would like to estimate the cube’s average viscosity, but measuring the viscosity is a
destructive laboratory test. So using 9 independent samples taken from this polymer cube, we get the
9 lab values of viscosity: 23, 19, 17, 18, 24, 26, 21, 14, 18.

If we repeat this process with a different set of 9 samples we will get a different average viscosity. So
we recognize the average of a sample of data, is itself just a single estimate of the population’s average.
What is more helpful is to have a range, given by a lower and upper bound, that we can say the true
population mean lies within.

72 Chapter 2. Univariate Data Analysis

 Release 547b16

1. The average of these nine values is 𝑥 = 20 units.

2. Using the Central limit theorem, what is the distribution from which 𝑥 comes?
(︀ )︀
𝑥 ∼ 𝒩 𝜇, 𝜎 2 /𝑛

This also requires the assumption that the samples are independent estimates of the
population viscosity. We don’t have to assume the 𝑥𝑖 are normally distributed.

3. What is the distribution of the sample average? What are the parameters of that distribution?
(︀ )︀
The sample average is normally distributed as 𝒩 𝜇, 𝜎 2 /𝑛

4. Assume, for some hypothetical reason, that we know the population viscosity standard deviation is
𝜎 = 3.5 units. Calculate a lower and upper bound for 𝜇:

The interval is calculated using from an earlier equation when discussing the normal distribution
(page 68):
𝜎
LB = 𝑥 − 𝑐𝑛 √
𝑛
3.5
= 20 − 1.95996 · √
9
= 20 − 2.286 = 17.7
UB = 20 + 2.286 = 22.3

5. We can confirm these 9 samples are normally distributed by using a q-q plot (not shown, but you
can use the code below to generate the plot). This is an important requirement to use the
𝑡-distribution, next.

6. Calculate an estimate of the standard deviation.

𝑠 = 3.81

7. Now construct the 𝑧-value for the sample average and from what distribution does this 𝑧 come
from?
𝑥−𝜇
It comes the 𝑡-distribution with 𝑛 − 1 = 8 degrees of freedom, and is given by 𝑧 = √
𝑠/ 𝑛
8. Construct an interval, symbolically, that will contain the population mean of the viscosity. Also
calculate the lower and upper bounds of the interval assuming the internal to span 95% of the area
of this distribution.

The interval is calculated using (2.3):

𝑠
LB = 𝑥 − 𝑐𝑡 √
𝑛
3.81
= 20 − 2.306004 · √
9
= 20 − 2.929 = 17.1
UB = 20 + 2.929 = 22.9

using from R that qt(0.025, df=8) and qt(0.975, df=8), which gives 2.306004

import numpy as np
import pandas as pd
from [Link] import t, probplot

(continues on next page)

2.9. The t-distribution 73

Process Improvement Using Data

(continued from previous page)

[Link] = "plotly"

# Step 0: the raw data

viscosity = [Link]([23, 19, 17, 18,
24, 26, 21, 14, 18])
n = len(viscosity)

# Step 1:
x_avg = [Link]()

# Step 5: Verify the data are normal

osm, osr = probplot(viscosity, dist="norm",
fit=False)
fig = [Link](
{"theoretical quantile": osm,
"viscosity": osr}
).[Link](x="theoretical quantile",
y="viscosity")
[Link]()

# Step 6:
x_sd = [Link](ddof=1)

# Step 7: t-distribution
dof = n - 1

# Step 8:
conf_level = 0.95

# Can be calculated at either

# the lower tail
c_t = [Link](q=(1 - conf_level) / 2,
df=dof)

# or the upper tail

c_t = [Link](q=1 - (1 - conf_level) / 2,
df=dof)

LB = x_avg - c_t * x_sd / [Link](n)

UB = x_avg + c_t * x_sd / [Link](n)
print(f"The {round(conf_level * 100):.0f}"
f"% confidence interval is: ")
print(f"[{round(LB, 1)}; {round(UB, 1)}]")

# Step 0: the raw data

viscosity <- c(23, 19, 17, 18,
24, 26, 21, 14, 18)
n <- length(viscosity)

# Step 1:
[Link] <- mean(viscosity)

# Step 5: Verify the data are normal

library(car)
qqPlot(viscosity)

# Step 6:
[Link] <- sd(viscosity)

# Step 7: t-distribution
dof <- n - 1

# Step 8:
[Link] <- 0.95

(continues on next page)

74 Chapter 2. Univariate Data Analysis

 Release 547b16

(continued from previous page)

# Can be calculated at either
# the lower tail
c.t <- qt(p = ([Link])/2,
df = dof)

# or the upper tail

c.t <- qt(p = 1-([Link])/2,
df = dof)

LB <- [Link] - c.t * [Link] / sqrt(n)

UB <- [Link] + c.t * [Link] / sqrt(n)
paste0('The ', round([Link]*100, 0),
'% confidence interval is: ')
paste0('[', round(LB, 1), '; ', round(UB, 1), ']')

Comparing the answers for parts 4 and 8 we see the interval, for the same level of 95% certainty, is
wider when we have to estimate the standard deviation. This makes sense: the standard deviation is
an estimate (meaning there is error in that estimate) of the true standard deviation. That uncertainty
must propagate, leading to a wider interval within which we expect to locate the true population
viscosity, 𝜇.

We will interpret confidence intervals in more detail a little later on (page 77).

2.10 Poisson distribution

The Poisson distribution is useful to characterize rare events (number of cell divisions in a small time
unit), system failures and breakdowns, or number of flaws on a product (contaminations per cubic
millimetre). These are events that have a very small probability of occurring within a given time
interval or unit area (e.g. pump failure probability per minute = 0.000002), but there are many
opportunities for the event to possibly occur (e.g. the pump runs continuously). A key assumption is
that the events must be independent. If one pump breaks down, then the other pumps must not be
affected; if one flaw is produced per unit area of the product, then other flaws that appear on the
product must be independent of the first flaw.

Let 𝑛 = number of opportunities for the event to occur. If this is a time-based system, then it would be
the number of minutes the pump is running. If it were an area/volume based system, then it might be
the number of square inches or cubic millimetres of the product. Let 𝑝 = probability of the event
occurring: e.g. 𝑝 = 0.000002 chance per minute of failure, or 𝑝 = 0.002 of a flaw being produced per
square inch. The rate parameter at which the event occurs is then given by 𝜂 = 𝑛𝑝 and is a count of
events per unit time or per unit area. A value for 𝑝 can be found using long-term, historical data.

There are two important properties:

1. The mean of the distribution for the rate happens to be the rate at which unusual events occur =
𝜂 = 𝑛𝑝

2. The variance of the distribution is also 𝜂. This property is particularly interesting - state in your own
words what this implies.
𝑒−𝜂 𝜂 𝑥
Formally, the Poisson distribution can be written as , with a plot as shown for 𝜂 = 4. Please note
𝑥!
the lines are only guides, the probability is only defined at the integer values marked with a circle.

2.10. Poisson distribution 75

Process Improvement Using Data

𝑝(𝑥) expresses the probability that there will be 𝑥 occurrences (must be an integer) of this rare event in
the same interval of time or unit area as 𝜂 was measured.

Example: Equipment in a chemical plant can and will fail. Since it is a rare event, let’s use the Poisson
distribution to model the failure rates. Historical records on a plant show that a particular supplier’s
pumps are, on average, prone to failure in a month with probability 𝑝 = 0.01 (1 in 100 chance of failure
each month). There are 50 such pumps in use throughout the plant. What is the probability that either 0,
1, 3, 6, 10, or 15 pumps will fail this year? (Create a table)
pump failures
𝜂 = 12 months failure
year × 50 pumps × 0.01 month = 6 year

𝑥 𝑝(𝑥)
0 0.25% chance
1 1.5%
3 8.9
6 16%
10 4.1%
15 0.1%

from [Link] import poisson

x = [0, 1, 3, 6, 10, 15]

# scipy calls the Poisson parameter 'mu'.

[Link](x, mu=6)

# Output:
# array([0.00247875, 0.01487251, 0.08923508,
# 0.16062314, 0.04130303, 0.00103326])

76 Chapter 2. Univariate Data Analysis

 Release 547b16

x <- c(0, 1, 3, 6, 10, 15)

# Note: R calls the Poisson parameter 'lambda'

dpois(x, lambda=6)

# Output:
# 0.0025 0.0149 0.0892 0.161 0.0413 0.001

2.11 Confidence intervals

So far we have calculated point estimates of parameters, called statistics. In the last section in the
𝑡-distribution we already calculated a confidence interval. In this section we formalize the idea,
starting with an example.

Example: a new customer is evaluating your product, they would like a confidence interval for the
impurity level in your sulphuric acid. You can tell them: “the range from 429ppm to 673ppm contains the
true impurity level with 95% confidence”. This is a compact representation of the impurity level. You
could have told your potential customer that

• the sample mean from the last year of data is 551 ppm

• the sample standard deviation from the last year of data is 102 ppm

• the last year of data are normally distributed

But a confidence interval conveys a similar concept, in a useful manner. It gives an estimate of the
location and spread and uncertainty associated with that parameter (e.g. impurity level in this case).

Let’s return to the previous viscosity example, where we had the 9 viscosity measurements 23, 19,
17, 18, 24, 26, 21, 14, 18. The sample average was 𝑥 = 20.0 and the standard deviation was
𝑥−𝜇
𝑠 = 3.81. The 𝑧-value is: 𝑧 = √ . And we showed this was distributed according to the
𝑠/ 𝑛
𝑡-distribution with 8 degrees of freedom.

Calculating a confidence interval requires we find a range within which that 𝑧-value occurs. Most
often we are interested in symmetrical confidence intervals, so the procedure uses critical values from
the relevant distribution:

−𝑐𝑡 ≤ 𝑧 ≤ +𝑐𝑡
𝑥−𝜇
−𝑐𝑡 ≤ √ ≤ +𝑐𝑡
𝑠/ 𝑛 (2.4)
𝑠 𝑠
𝑥 − 𝑐𝑡 √ ≤ 𝜇 ≤ 𝑥 + 𝑐𝑡 √
𝑛 𝑛
LB ≤ 𝜇 ≤ UB

The critical values of 𝑐𝑡 are qt(1 - 0.05/2, df=8) = 2.306004 when we used the 95%
confidence interval (2.5% in each tail). We calculated that LB = 20.0 - 2.92 = 17.1 and that UB = 20.0 +
2.92 = 22.9.

2.11.1 Interpreting the confidence interval

Video for
this section • The expression in (2.4) should not be interpreted to mean that the viscosity is 20 units and lies inside
the LB (lower-bound) to UB (upper-bound) range of 17.1 to 22.9 with a 95% probability. In fact, the
sample mean lies exactly at the mid-point of the range with 100% certainty - that is how the range
was calculated.

2.11. Confidence intervals 77

Process Improvement Using Data

• What the expression in (2.4) does imply is that 𝜇 lies in this interval. The confidence interval is a
range of possible values for 𝜇, not for 𝑥. Confidence intervals are for parameters, not for statistics.

• Notice that the upper and lower bounds are a function of the data sample used to calculate 𝑥 and
the number of points, 𝑛. If we take a different sample of data, we will get different upper and lower
bounds.

• What does the level of confidence mean?

It is the probability that the true population viscosity, 𝜇 is in the given range. At 95%
confidence, it means that 5% of the time the interval will not contain the true mean. So if we
collected 20 sets of 𝑛 samples, 19 times out of 20 the confidence interval range will contain the
true mean, but one of those 20 confidence intervals is expected not to contain the true mean.

• What happens if the level of confidence changes? Calculate the viscosity confidence intervals for
90%, 95%, 99%.

Confidence LB UB
90% 17.6 22.4
95% 17.1 22.9
99% 15.7 24.2

As the confidence level is increased, our interval widens, indicating that we have a more
reliable region, but it is less precise. With a wider interval we have greater confidence that the
true parameter will be inside that region.

Try it out:

import numpy as np
from [Link] import t

# Try varying this value:

conf_level = 0.90

viscosity = [Link]([23, 19, 17, 18,

24, 26, 21, 14, 18])
n = len(viscosity)
x_avg = [Link]()
x_sd = [Link](ddof=1)
dof = n - 1
c_t = [Link](q=1 - (1 - conf_level) / 2,
df=dof)
LB = x_avg - c_t * x_sd / [Link](n)
UB = x_avg + c_t * x_sd / [Link](n)
print(f"The {round(conf_level * 100):.0f}"
f"% confidence interval is: ")
print(f"[{round(LB, 1)}; {round(UB, 1)}]")

# Try varying this value:

[Link] <- 0.90

viscosity <- c(23, 19, 17, 18,

24, 26, 21, 14, 18)
n <- length(viscosity)
[Link] <- mean(viscosity)
(continues on next page)

78 Chapter 2. Univariate Data Analysis

 Release 547b16

(continued from previous page)

[Link] <- sd(viscosity)
dof <- n - 1
c.t <- qt(p = 1-([Link])/2,
df = dof)
LB <- [Link] - c.t * [Link] / sqrt(n)
UB <- [Link] + c.t * [Link] / sqrt(n)
paste0('The ', round([Link]*100, 0),
'% confidence interval is: ')
paste0('[', round(LB, 1), '; ', round(UB, 1), ']')

• What happens if the level of confidence is 100%?

The confidence interval is then infinite. We are 100% certain this infinite range contains the
population mean, however this is not a useful interval. Test it out in the code above; also try
creating an interval with 99.9% confidence, and then 99.99% confidence.

• What happens if we increase the value of 𝑛?

As intuitively expected, as the value of 𝑛 increases, the confidence interval decreases in width.

• Returning to the case above, where at the 95% level we found the confidence interval was
[17.1; 22.9] for the bale’s viscosity. What if we were to analyze the bale thoroughly, and found the
population viscosity to be 23.2. What is the probability of that occurring?

Less than 5% of the time.

2.11.2 Confidence interval for the mean from a normal distribution

Video for
this section The aim here is to formalize the calculations for the confidence interval of 𝑥, given a sample of 𝑛

a) independent points, taken from

b) the normal distribution.

Be sure to check those two assumptions before going ahead.

There are 2 cases: one where you know the population standard deviation (unlikely), and one where
you do not (the usual case). It is safer to use the confidence interval for the case when you do not know
the standard deviation, as it is a more conservative (i.e. wider) interval.

The detailed derivation for the two cases was covered in earlier sections.

Case A. Variance is known

When the variance is known, the confidence interval is given by (2.5) below, derived from this
𝑥−𝜇
𝑧-deviate: 𝑧 = √ back in the section on the normal distribution (page 68).
𝜎/ 𝑛

−𝑐𝑛 ≤ 𝑧 ≤ +𝑐𝑛
𝑥−𝜇
−𝑐𝑛 ≤ √ ≤ +𝑐𝑛
𝜎/ 𝑛 (2.5)
𝜎 𝜎
𝑥 − 𝑐𝑛 √ ≤ 𝜇 ≤ 𝑥 + 𝑐𝑛 √
𝑛 𝑛
LB ≤ 𝜇 ≤ UB

The values of 𝑐𝑛 are qnorm(1 - 0.05/2) = 1.96 when we happen to use the 95% confidence
interval (2.5% in each tail).

2.11. Confidence intervals 79

 Process Improvement Using Data

Case B. Variance is unknown

In the more realistic case when the variance is unknown we use the equation derived in the section on the
𝑥−𝜇
t-distribution (page 72), and repeated here below. This is derived from the 𝑧-deviate: 𝑧 = √ :
𝑠/ 𝑛

−𝑐𝑡 ≤ 𝑧 ≤ +𝑐𝑡
𝑥−𝜇
−𝑐𝑡 ≤ √ ≤ +𝑐𝑡
𝑠/ 𝑛 (2.6)
𝑠 𝑠
𝑥 − 𝑐𝑡 √ ≤ 𝜇 ≤ 𝑥 + 𝑐𝑡 √
𝑛 𝑛
LB ≤ 𝜇 ≤ UB

The values of 𝑐𝑡 are qt(1 - 0.05/2, df=...) when we use the 95% confidence interval (2.5% in
each tail). This 𝑧-deviate is distributed according to the 𝑡-distribution, since we have additional
uncertainty when using the standard deviation estimate, 𝑠, instead of the population standard
deviation, 𝜎.

Comparison

If we have the fortunate case where our estimated variance, 𝑠2 , is equal to the population variance, 𝜎 2 ,
then we can compare the 2 intervals in equations (2.5) and (2.6). The only difference would be the
value of the 𝑐𝑛 from the normal distribution and 𝑐𝑡 from the 𝑡-distribution. For typical values used as
confidence levels, 90% to 99.9%, values of 𝑐𝑡 > 𝑐𝑛 for any degrees of freedom.

This implies the confidence limits are wider for the case when the standard deviation is unknown,
leading to more conservative results, reflecting our uncertainty of the standard deviation parameter, 𝜎.

2.12 Testing for differences and similarity

Video for
this section These sort of questions often arise in data analysis:

• We want to change to a cheaper material, B. Does it work as well as A?

• We want to introduce a new catalyst B. Does it improve our product properties over the current
catalyst A?

Either we want to confirm things are statistically the same, or confirm they have changed. Notice that
in both the above cases we are testing the population mean (location). Has the mean shifted or is it the
same? There are also tests for changes in variance (spread), which we will cover. We will work with an
example throughout this section.

Example: A process operator needs to verify that a new form of feedback control on the batch reactor
leads to improved yields. Yields under the current control system, A, are compared with yields under
the new system, B. The last ten runs with system A are compared to the next 10 sequential runs with
system B. The data are shown in the table, and shown in graphical form as well. (Note that the box
plot uses the median, while the plots on the right show the mean.)

80 Chapter 2. Univariate Data Analysis

 Release 547b16

import pandas as pd

[Link] = "plotly"

# Generate the boxplot

A = [92.7, 73.3, 80.5, 81.2, 87.1,
69.2, 81.9, 73.9, 78.6, 80.5]
B = [83.5, 78.9, 82.7, 93.2, 86.3,
74.7, 81.6, 92.4, 83.6, 72.4]

data = [Link]([
[Link]({"observe": A, "method": "A"}),
[Link]({"observe": B, "method": "B"}),
])

fig = [Link](
(continues on next page)

2.12. Testing for differences and similarity 81

Process Improvement Using Data

(continued from previous page)

x="method", y="observe",
title="Batch yield (%) for two trials",
)
[Link]()

# Generate the boxplot

A <- c(92.7, 73.3, 80.5, 81.2, 87.1,
69.2, 81.9, 73.9, 78.6, 80.5)
B <- c(83.5, 78.9, 82.7, 93.2, 86.3,
74.7, 81.6, 92.4, 83.6, 72.4)

data.A <- [Link](observe=A, method='A')

data.B <- [Link](observe=B, method='B')
data <- rbind(data.A, data.B)

limits <- range(data$observe)

boxplot(data$obs ~ data$method, lwd=2,
main="Batch yield (%) for two trials")

We address the question of whether or not there was a significant difference between system A and B. A
significant difference means that when system B is compared to a suitable reference, that we can be
sure that the long run implementation of B will lead, in general, to a different yield (%). We want to be
sure that any change in the 10 runs under system B were not only due to chance, because system B will
cost us $100,000 to install, and $20,000 in annual software license fees.

Note: those with a traditional statistical background will recognize this section as one-sided
hypothesis tests. We will only consider tests for a significant increase or decrease, i.e. one-sided
tests, in this section. We use confidence intervals, rather than hypothesis tests; the results are
exactly the same. Arguably the confidence interval approach is more interpretable, since we get a
bound, rather that just a clear-cut yes/no answer.

There are two main ways to test for a significant increase or significant decrease.

2.12.1 Comparison to a long-term reference set

Continuing the above example we can compare the past 10 runs from system B with the 10 runs from
system A. The average difference between these runs is 𝑥𝐵 − 𝑥𝐴 = 82.93 − 79.89 = 3.04 units of
improved yield. Now, if we have a long-term reference data set available, we can compare if any 10
historical, sequential runs from system A, followed by another 10 historical, sequential runs under
system A had a difference that was this great. If not, then we know that system B leads to a definite
improvement, not likely to be caused by chance alone.

Here’s the procedure:

1. Imagine that we have 300 historical data points from this system, tabulated in time order: yield
from batch 1, 2, 3 . . . (the data are available on the website38 ).

2. Calculate the average yields from batches 1 to 10. Then calculate the average yield from batches 11
to 20. Notice that this is exactly like the experiment we performed when we acquired data for
system B: two groups of 10 batches, with the groups formed from sequential batches.

3. Now subtract these two averages: (group average 11 to 20) minus (group average 1 to 10).
38 [Link]

82 Chapter 2. Univariate Data Analysis

 Release 547b16

4. Repeat steps 2 and 3, but use batches 2 to 11 and 12 to 21. Repeat until all historical batch data are
used up, i.e. batches 281 to 290 and 291 to 300. The plot below can be drawn, one point for each of
these difference values.

The vertical line at 3.04 is the difference value recorded between system B and system A. From this we
can see that historically, there were 31 out of 281 batches, about 11% of historical data, that had a
difference value of 3.04 or greater. So there is a 11% probability that system B was better than system A
purely by chance, and not due to any technical superiority. Given this information, we can now judge,
if the improved control system will be economically viable and judge, based on internal company
criteria, if this is a suitable investment, also considering the 11% risk that our investment will fail.

Notice that no assumption of independence or any form of distributions was required for this work!
The only assumption made is that the historical data are relevant. We might know this if, for example,
no substantial modification was made to the batch system for the duration over which the 300 samples
were acquired. If however, a different batch recipe were used for sample 200 onwards, then we may
have to discard those first 200 samples: it is not fair to judge control system B to the first 200 samples
under system A, when a different operating procedure was in use.

So to summarize: we can use a historical data set if it is relevant. And there are no assumptions of
independence or shape of the distribution, e.g. a normal distribution.

In fact, for this example, the data were not independent, they were autocorrelated. There was a
relationship from one batch to the next: 𝑥[𝑘] = 𝜑𝑥[𝑘 − 1] + 𝑎[𝑘], with 𝜑 = −0.3, and
(︀ )︀
𝑎[𝑘] ∼ 𝒩 𝜇 = 0, 𝜎 2 = 6.72 . As an aside you can simulate your own set of autocorrelated data using
this R code:

import numpy as np
import pandas as pd

[Link] = "plotly"

N = 300
phi = -0.3
spread = 6.7
location = 79.9

A_hist = [Link](N)
for k in range(1, N):
A_hist[k] = (phi * A_hist[k - 1]
+ [Link](loc=0,
scale=spread))
A_hist = A_hist + location

# Note: your plot will look different to

# the text, because it will be from a
# different set of random numbers.
(continues on next page)

2.12. Testing for differences and similarity 83

Process Improvement Using Data

(continued from previous page)

title = ("Autocorrelation between "
"successive values of batch yield")
fig = [Link](
{"x[k]": A_hist[:-1],
"x[k+1]": A_hist[1:]}
).[Link](x="x[k]", y="x[k+1]",
title=title)
fig.update_xaxes(range=[60, 100])
fig.update_yaxes(range=[60, 100])
[Link]()

# Hint: run the code several times, with

# different values of variable `phi`.

N <- 300
phi <- -0.3
spread <- 6.7
location <- 79.9

# create a vector of zeros

[Link] <- numeric(N)
for (k in 2:N)
{
[Link][k] <- phi*([Link][k-1]) +
rnorm(1, mean = 0, sd = spread)
}
[Link] <- [Link] + location

# Note: your plot will look different to

# the text, because it will be from a
# different set of random numbers
title = paste0("Autocorrelation between ",
"successive values of batch yield")
plot([Link][1:N-1], [Link][2:N],
xlab = "x[k]",
ylab = "x[k+1]",
main = title,
lwd = 3,
xlim = c(60,100),
ylim = c(60,100))

lines(lowess([Link][1:N-1], [Link][2:N]))

# Hint: run the code several times, with

# different values of variable `phi`.

We can visualize this autocorrelation by plotting the values of 𝑥[𝑘] against 𝑥[𝑘 + 1]:

84 Chapter 2. Univariate Data Analysis

 Release 547b16

We can immediately see the data are not independent, because the slope is non-zero.

2.12.2 Comparison when a reference set is not available

A reference data set may not always be available; we may only have the data from the 20 experimental
runs (10 from system A and 10 from B) and nothing else. We can proceed to compare the data, but we
will require a strong assumption of random sampling (independence), which is often not valid in
engineering data sets. Fortunately, engineering data sets are usually large - we are good at collecting
data - so the methodology in the preceding section on using a reference set, is greatly preferred, when
possible.

How could the assumption of independence (random sampling) be made more realistically? How is
the lack of independence detrimental? We show below that the assumption of independence is made
twice: the samples within group A and B must be independent; furthermore, the samples between the
groups should be independent. But first we have to understand why the assumption of independence
is required, by understanding the usual approach for estimating if differences are significant or not.

The usual approach for assessing if the difference between 𝑥𝐵 − 𝑥𝐴 is significant follows this approach:

1. Assume the data for sample A and sample B have been independently sampled from their
respective populations.

2. Assume the data for sample A and sample B have the same population variance, 𝜎𝐴 = 𝜎𝐵 = 𝜎
(there is a test for this, see the next section).

3. Let the sample A have population mean 𝜇𝐴 and sample B have population mean 𝜇𝐵 .

4. From the central limit theorem (this is where the assumption of independence of the samples within

2.12. Testing for differences and similarity 85

Process Improvement Using Data

each group comes), we know that:

2 2
𝜎𝐴 𝜎𝐵
𝒱 {𝑥𝐴 } = 𝒱 {𝑥𝐵 } =
𝑛𝐴 𝑛𝐵

5. Assuming independence again, but this time between groups, this implies the average of each
sample group is independent, i.e. 𝑥𝐴 and 𝑥𝐵 are independent of each other. This allows us to write:

𝜎2 𝜎2
(︂ )︂
1 1
𝒱 {𝑥𝐵 − 𝑥𝐴 } = + = 𝜎2 + (2.7)
𝑛𝐴 𝑛𝐵 𝑛𝐴 𝑛𝐵

6. Using the central limit theorem, even if the samples in A and the samples in B are non-normal, the
sample averages 𝑥𝐴 and 𝑥𝐵 will be more normal as the sample size becomes progressively larger.
So the difference between these means will also be more normal: 𝑥𝐵 − 𝑥𝐴 . Now express this
difference in the form of a 𝑧-deviate (standard form):

(𝑥𝐵 − 𝑥𝐴 ) − (𝜇𝐵 − 𝜇𝐴 )
𝑧= √︃ (︂ )︂
2
1 1 (2.8)
𝜎 +
𝑛𝐴 𝑛𝐵

We could ask, what is the probability of seeing a 𝑧 value from equation (2.8) of that magnitude?
Recall that this 𝑧-value is the equivalent of 𝑥𝐵 − 𝑥𝐴 , expressed in deviation form, and we are
interested if this difference is due to chance. So we should ask, what is the probability of getting a
value of 𝑧 greater than this, or smaller that this, depending on the case?

The only question remains is what is a suitable value for 𝜎? As we have seen before, when we have
a large enough reference set, then we can use the value of 𝜎 from the historical data, called an
external estimate. Or we can use an internal estimate of spread; both approaches are discussed below.

Now we know the approach required, using the above 6 steps, to determine if there was a significant
difference. And we know the assumptions that are required: normally distributed and independent
samples. But how can we be sure our data are independent? This is the most critical aspect, so let’s
look at a few cases and discuss, then we will return to our example and calculate the 𝑧-values with
both an external and internal estimate of spread.

Discuss whether these experiments would lead to independent data or not, and how we might
improve the situation.

a) We are testing a new coating to repel moisture. The coating is applied to packaging sheets that are
already hydrophobic, however this coating enhances the moisture barrier property of the sheet. In
the lab, we take a large packaging sheet and divide it into 16 blocks. We coat the sheet as shown in
the figure and then use the 𝑛𝐴 = 8 and 𝑛𝐵 = 8 values of hydrophobicity to judge if coating B is
better than coating A.

86 Chapter 2. Univariate Data Analysis

 Release 547b16

Some problems with this approach:

• The packaging sheet to which the new coating is applied may not be uniform. The sheet is
already hydrophobic, but the hydrophobicity is probably not evenly spread over the sheet, nor
are any of the other physical properties of the sheet. When we measure the moisture repelling
property with the different coatings applied, we will not have an accurate measure of whether
coating A or B worked better. We must randomly assign blocks A and B on the packaging sheet.

• Even so, this may still be inadequate, because what if the packaging sheet selected has overly
high or low hydrophobicity (i.e. it is not representative of regular packaging sheets). What
should be done is that random packaging sheets should be selected, and they should be selected
across different lots from the sheet supplier (sheets within one lot are likely to be more similar
than between lots). Then on each sheet we apply coatings A and B, in a random order on each
sheet.

• It is tempting to apply coating A and B to one half of the various sheets and measure the difference
between the moisture repelling values from each half. It is tempting because this approach
would cancel out any base variation between difference sheets, as long as that variation is
present across the entire sheet. Then we can go on to assess if this difference is significant.

There is nothing wrong with this methodology, however, there is a different, specific test for
paired data, covered in a later section (page 92). If you use the above test, you violate the
assumption in step 5, which requires that 𝑥𝐴 and 𝑥𝐵 be independent. Values within group A and
B are independent, but not their sample averages, because you cannot calculate 𝑥𝐴 and 𝑥𝐵
independently.

b) We are testing an alternative, cheaper raw material in our process, but want to be sure our product’s
final properties are unaffected. Our raw material dispensing system will need to be modified to
dispense material B. This requires the production line to be shut down for 15 hours while the new
dispenser, lent from the supplier, is installed. The new supplier has given us 8 representative
batches of their new material to test, and each test will take 3 hours. We are inclined to run these 8
batches over the weekend: set up the dispenser on Friday night (15 hours), run the tests from
Saturday noon to Sunday noon, then return the line back to normal for Monday’s shift. How might
we violate the assumptions required by the data analysis steps above when we compare 8 batches
of material A (collected on Thursday and Friday) to the 8 batches from material B (from the
weekend)? What might we do to avoid these problems?

• The 8 tests are run sequentially, so any changes in conditions between these 8 runs and the 8 runs
from material A will be confounded (confused) in the results. List some actual scenarios how

2.12. Testing for differences and similarity 87

Process Improvement Using Data

confounding between the weekday and weekend experiments occur:

– For example, the staff running the equipment on the weekend are likely not the same staff that
run the equipment on weekdays.

– The change in the dispenser may have inadvertently modified other parts of the process, and
in fact the dispenser itself might be related to product quality.

– The samples from the tests will be collected and only analyzed in the lab on Monday, whereas
the samples from material A are usually analyzed on the same day: that waiting period may
degrade the sample.

This confounding with all these other, potential factors means that we will not be able to
determine whether material B caused a true difference, or whether it was due to the other
conditions.

• It is certainly expensive and impractical to randomize the runs in this case. Randomization
would mean we randomly run the 16 tests, with the A and B chosen in random order, e.g. A B A
B A A B B A A B B B A B A. This particular randomization sequence would require
changing the dispenser 9 times.

• One suboptimal sequence of running the system is A A A A B B B B A A A A B B B B.

This requires changing the dispenser 4 times (one extra change to get the system back to material
A). We run each (A A A A B B B B) sequence on two different weekends, changing the
operating staff between the two groups of 8 runs, making sure the sample analysis follows the
usual protocols: so we reduce the chance of confounding the results.

Randomization might be expensive and time-consuming in some studies, but it is the insurance we
require to avoid being misled. These two examples demonstrate this principle: block what you can
and randomize what you cannot (see blocking). We will review these concepts again in the design and
analysis of experiments section (page 319). If the change being tested is expected to improve the process,
then we must follow these precautions to avoid a process upgrade/modification that does not lead to
the expected improvement; or the the converse - a missed opportunity of implementing a change for
the better.

External and internal estimates of spread

So to recap the progress so far, we are aiming to test if there is a significant, long-term difference between
two systems: A and B. We showed the most reliable way to test this difference is to compare it with a
body of historical data, with the comparison made in the same way as when the data from system A
and B were acquired; this requires no additional assumptions, and even allows one to run experiments
for system B in a non-independent way.

But, because we do not always have a large and relevant body of data available, we can calculate the
difference between A and B and test if this difference could have occurred by chance alone. For that
we use equation (2.8), but we need an estimate of the spread, 𝜎.

External estimate of spread

The question we turn to now is what value to use for 𝜎 in equation (2.8). We got to that equation by
assuming we have no historical, external data. But what if we did have some external data? We could
at least estimate 𝜎 from that. For example, the 300 historical batch yields has 𝜎 = 6.61:

Check the probability of obtaining the 𝑧-value in (2.8) by using the hypothesis that the value
𝜇𝐵 − 𝜇𝐴 = 0. In other words we are making a statement, or a test of significance. Then we calculate

88 Chapter 2. Univariate Data Analysis

 Release 547b16

this 𝑧-value and its associated cumulative probability:

(𝑥𝐵 − 𝑥𝐴 ) − (𝜇𝐵 − 𝜇𝐴 )
𝑧= √︃ (︂ )︂
1 1
𝜎2 +
𝑛𝐴 𝑛𝐵
(82.93 − 79.89) − (𝜇𝐵 − 𝜇𝐴 )
𝑧= √︃ (︂ )︂
1 1
6.612 +
10 10
3.04 − 0
𝑧= = 1.03
2.956
The probability of seeing a 𝑧-value from −∞ up to 1.03 is 84.8% (use the pnorm(1.03) function in R).
But we are interested in the probability of obtaining a 𝑧-value larger than this. Why? Because 𝑧 = 0
represents no improvement, and a value of 𝑧 < 0 would mean that system B is worse than system A.
So what are the chances of obtaining 𝑧 = 1.03? It is (100-84.8)% = 15.2%, which means that system B’s
performance could have been obtained by pure luck in 15.2% of cases.

import numpy as np
from [Link] import norm

A = [Link]([92.7, 73.3, 80.5, 81.2, 87.1,

69.2, 81.9, 73.9, 78.6, 80.5])
B = [Link]([83.5, 78.9, 82.7, 93.2, 86.3,
74.7, 81.6, 92.4, 83.6, 72.4])

xA_avg = [Link]()
xB_avg = [Link]()
n_A = len(A)
n_B = len(B)
sigma_external = 6.61 # given

den = sigma_external ** 2 * (1 / n_A + 1 / n_B)

z = (xB_avg - xA_avg) / [Link](den)

# Probability of this z?
# We have normalized to zero mean
# and to unit standard deviation:
p = [Link](z, loc=0, scale=1) # 0.8481164
print(f"Probability by chance: "
f"{round((1 - p) * 100, 1)}%")

A <- c(92.7, 73.3, 80.5, 81.2, 87.1,

69.2, 81.9, 73.9, 78.6, 80.5)
B <- c(83.5, 78.9, 82.7, 93.2, 86.3,
74.7, 81.6, 92.4, 83.6, 72.4)

[Link] <- mean(A)

[Link] <- mean(B)
n.A <- length(A)
n.B <- length(B)
[Link] <- 6.61 # given

den <- [Link]**2 * (1/n.A + 1/n.B)

z <- ([Link] - [Link]) / sqrt(den)

# Probability of this z?
# We have normalized to zero mean
# and to unit standard deviation:
p <- pnorm(z, mean=0, sd=1) # 0.8481164
paste0('Probability by chance: ',
(continues on next page)

2.12. Testing for differences and similarity 89

Process Improvement Using Data

(continued from previous page)

round((1-p)*100, 1), '%')

We interpret this number of “15.2%” in the summary section, but let’s finally look at what happens if
we have no historical data - then we generate an internal estimate of 𝜎 from the 20 experimental runs
alone.

Internal estimate of spread

The sample variance from each system was 𝑠2𝐴 = 6.812 and 𝑠2𝐵 = 6.702 , and in this case it happened
that 𝑛𝐴 = 𝑛𝐵 = 10, although the sample sizes do not necessarily have to be the same.

If the variances are comparable (there is a test for that below (page 94)), then we can calculate a pooled
variance, 𝑠2𝑃 , which is a weighted sum of the sampled variances:

(𝑛𝐴 − 1)𝑠2𝐴 + (𝑛𝐵 − 1)𝑠2𝐵

𝑠2𝑃 =
𝑛𝐴 − 1 + 𝑛𝐵 − 1
2
9 × 6.81 + 9 × 6.702
𝑠2𝑃 =
18
𝑠2𝑃 = 45.63

Now using this value of 𝑠𝑃 instead of 𝜎 in (2.8):

(𝑥𝐵 − 𝑥𝐴 ) − (𝜇𝐵 − 𝜇𝐴 )
𝑧= √︃ (︂ )︂
1 1
𝜎2 +
𝑛𝐴 𝑛𝐵
(82.93 − 79.89) − (𝜇𝐵 − 𝜇𝐴 )
= √︃ (︂ )︂
1 1
𝑠2𝑃 +
10 10
3.04 − 0
= √︀
45.63 × 2/10
𝑧 = 1.01

The probability of obtaining a 𝑧-value greater than this can be calculated as 16.4% using the
𝑡-distribution with 18 degrees of freedom (use 1-pt(1.01, df=18) in R). We use a 𝑡-distribution
because an estimate of the variance is used, 𝑠2𝑝 , not a population variance, 𝜎 2 .

import numpy as np
from [Link] import t

A = [Link]([92.7, 73.3, 80.5, 81.2, 87.1,

69.2, 81.9, 73.9, 78.6, 80.5])
B = [Link]([83.5, 78.9, 82.7, 93.2, 86.3,
74.7, 81.6, 92.4, 83.6, 72.4])

xA_avg = [Link]()
xB_avg = [Link]()
n_A = len(A)
n_B = len(B)
# degrees of freedom
dof = n_A - 1 + n_B - 1
var_pooled = ((n_A - 1) * [Link](ddof=1)
+ (n_B - 1) * [Link](ddof=1)) / dof

den = var_pooled * (1 / n_A + 1 / n_B)

(continues on next page)

90 Chapter 2. Univariate Data Analysis

 Release 547b16

(continued from previous page)

z = (xB_avg - xA_avg) / [Link](den)

# Probability of this z?
# Compare it against the t-distribution:
p = [Link](z, df=dof) # 0.8361346

print(f"Probability by chance: "

f"{round((1 - p) * 100, 1)}%")

A <- c(92.7, 73.3, 80.5, 81.2, 87.1,

69.2, 81.9, 73.9, 78.6, 80.5)
B <- c(83.5, 78.9, 82.7, 93.2, 86.3,
74.7, 81.6, 92.4, 83.6, 72.4)

[Link] <- mean(A)

[Link] <- mean(B)
n.A <- length(A)
n.B <- length(B)
# degrees of freedom
dof <- n.A - 1 + n.B - 1
[Link] <- ((n.A - 1) * var(A) +
(n.B - 1) * var(B)) / dof

den <- [Link] * (1/n.A + 1/n.B)

z <- ([Link] - [Link]) / sqrt(den)

# Probability of this z?
# Compare it against the t-distribution:
p <- pt(z, df = dof) # 0.8361346

paste0('Probability by chance: ',

round((1-p)*100, 1), '%')

As an aside: we used a normal distribution for the external 𝜎 and a 𝑡-distribution for the internal 𝑠.
Both cases had a similar value for 𝑧 (compare 𝑧 = 1.01 to 𝑧 = 1.03). Note however that the probabilities
are higher in the 𝑡-distribution’s tails, which means that even though we have similar 𝑧-values, the
probability is greater: 16.4% against 15.2%. While this difference is not much from a practical point of
view, it illustrates the difference between the 𝑡-distribution and the normal distribution.

The results from this section were achieved by only using the 20 experimental runs, no external data.
However, it made some strong assumptions:

• The variances of the two samples are comparable, and can therefore be pooled (page 94) to provide an
estimate of 𝜎.

• The usual assumption of independence within each sample is made (which we know not to be true
for many practical engineering cases).

• The assumption of independence between the samples is also made (this is more likely to be true in
this example, because the first runs to acquire data for A are not likely to affect the runs for system
B).

• Each sample, A and B, is assumed to be normally distributed.

2.12. Testing for differences and similarity 91

 Process Improvement Using Data

Summary and comparison of methods

Let’s compare the 3 estimates. Recall our aim is to convince ourself/someone that system B will have
better long-term performance than the current system A.

If we play devil’s advocate, our null hypothesis is that system B has no effect. Then it is up to us to
prove, convincingly, that the change from A to B has a systematic, permanent effect. That is what the
calculated probabilities represent :, the probability of us being wrong.

1. Using only reference data: 11% (about 1 in 10)

2. Using the 20 experimental runs, but an external estimate of 𝜎: 15.2% (about 1 in 7)

3. Using the 20 experimental runs only, no external data: 16.4% (about 1 in 6)

The reference data method shows that the trial with 10 experiments using system B could have
actually been taken from the historical data with a chance of 11%. A risk adverse company may want
this number to be around 5%, or as low as 1% (1 in 100), which essentially guarantees the new system
will have better performance.

When constructing the reference set, we have to be sure the reference data are appropriate. Were the
reference data acquired under conditions that were similar to the time in which data from system B
were acquired? In this example, they were, but in practice, careful inspection of plant records must be
made to verify this.

The other two methods mainly use the experimental data, and provide essentially the same answer in
this case study, though that is not always the case. The main point here is that our experimental data are
usually not independent. However, by careful planning, and expense, we can meet the requirement of
independence by randomizing the order in which we acquire the data. Randomization is the insurance
(cost) we pay so that we do not have to rely on a large body of prior reference data. But in some cases
it is not possible to randomize, so blocking is required. More on blocking in the design of experiments
section (page 362).

2.13 Paired tests

Video for
this section A paired test is a test that is run twice on the same object or batch of materials. You might see the
nomenclature of “two treatments” being used in the literature. For example:

• A drug trial could be run in two parts: each person randomly receives a placebo or the drug, then 3
weeks later they receive the opposite, for another 3 weeks. Tests are run at 3 weeks and 6 weeks and
the difference in the test result is recorded.

• We are testing two different additives, A and B, where the additive is applied to a base mixture of
raw materials. Several raw material lots are received from various suppliers, supposedly uniform.
Split each lot into 2 parts, and run additive A and B on each half. Measure the outcome variable, e.g.
conversion, viscosity, or whatever the case might be, and record the difference.

• We are testing a new coating to repel moisture. The coating is applied to randomly selected sheets
in a pattern [A|B] or [B|A] (the pattern choice is made randomly). We measure the repellent
property value and record the difference.

In each case we have a table of 𝑛 samples recording the difference values. The question now is
whether the difference is significant, or is it essentially zero?

The advantage of the paired test is that any systematic error in our measurement system, what ever it
might be, is removed as long as that error is consistent. Say for example we are measuring blood

92 Chapter 2. Univariate Data Analysis

 Release 547b16

pressure, and the automated blood pressure device has a bias of -5 mmHg. This systematic error will
cancel out when we subtract the 2 test readings. In the example of the raw materials and additives:
any variation in the raw materials and its (unintended) effect on the outcome variable of interest will
be cancelled.

The disadvantage of the paired test is that we lose degrees of freedom. Let’s see how:

1. Calculate the 𝑛 differences: 𝑤1 = 𝑥𝐵,1 − 𝑥𝐴,1 ; 𝑤2 = 𝑥𝐵,2 − 𝑥𝐴,2 , . . . to create the sample of values
w = [𝑤1 , 𝑤2 , . . . , 𝑤𝑛 ]

2. Assume these values, 𝑤𝑖 , are independent, because they are taken on independent objects (people,
base packages, sheets of paper, etc)

3. Calculate the mean, 𝑤 and the standard deviation, 𝑠𝑤 , of these 𝑛 difference values.

4. What do we need to assume about the population from which 𝑤 comes? Nothing. We are not
interested in the 𝑤 values, we are interested in 𝑤. OK, so what distribution would values of 𝑤 come
from? By the central limit theorem, the 𝑤 values should be normally distributed as
(︀ 2
)︀
𝑤 ∼ 𝒩 𝜇𝑤 , 𝜎𝑤 /𝑛 , where 𝜇𝑤 = 𝜇𝐴−𝐵 .

5. Now calculate the 𝑧-value, but use the sample standard deviation, instead of the population
standard deviation.
𝑤 − 𝜇𝑤
𝑧= √
𝑠𝑤 / 𝑛

6. Because we have used the sample standard deviation, 𝑠𝑤 , we have to use to the 𝑡-distribution with
𝑛 − 1 degrees of freedom, to calculate the critical values.

7. We can calculate a confidence interval, below, and if this interval includes zero, then the change
from treatment A to treatment B had no effect.
𝑠𝑤 𝑠𝑤
𝑤 − 𝑐𝑡 √ < 𝜇𝑤 < 𝑤 + 𝑐𝑡 √
𝑛 𝑛

The value of 𝑐𝑡 is taken from the 𝑡-distribution with 𝑛 − 1 degrees of freedom at the level of
confidence required: use the qt(...) function in R to obtain the values of 𝑐𝑡 .

The loss of degrees of freedom can be seen when we use exactly the same data and treat the problem
as one where we have 𝑛𝐴 and 𝑛𝐵 samples in groups A and B and want to test for a difference between
𝜇𝐴 and 𝜇𝐵 . You are encouraged to try this out. There are more degrees of freedom, 𝑛𝐴 + 𝑛𝐵 − 2 in fact
when we use the 𝑡-distribution with the pooled variance shown here (page 90). Compare this to the case
just described above where there are only 𝑛 degrees of freedom.

2.14 Other types of confidence intervals

There are several other confidence intervals that you might come across in your career. We merely
mention them here and don’t cover their derivation. What is important is that you understand how to
interpret a confidence interval. Hopefully the previous discussion achieved that.

2.14.1 Confidence interval for the variance

This confidence interval finds a region in which the normal distribution’s variance parameter, 𝜎, lies.
The range is obviously positive, since variance is a positive quantity. For reference, this range is:
[︃ ]︃
(𝑛 − 1)𝑆 2 (𝑛 − 1)𝑆 2
to
𝜒2𝑛−1,𝛼/2 𝜒2𝑛−1,1−𝛼/2

2.14. Other types of confidence intervals 93

Process Improvement Using Data

• 𝑛 is the number of samples

• 𝑆 2 is the sample variance

• 𝜒2𝑛−1,𝛼/2 are values from the 𝜒2 distribution with 𝑛 − 1 and 𝛼/2 degrees of freedom

• 1 − 𝛼: is the level of confidence, usually 95%, so 𝛼 = 0.05 in that case.

2.14.2 Confidence interval for the ratio of two variances

One way to test whether we can pool (combine) two variances, taken from two different normal
𝑠2
distributions, is to construct the ratio: 21 . We can construct a confidence interval, and if this interval
𝑠2
contains the value of 1.0, then we have no evidence to presume they are different (i.e. we can assume
the two population variances are similar).

𝑠22 𝜎22 𝑠22

𝐹𝛼/2,𝜈1 ,𝜈2 < < 𝐹1−𝛼/2,𝜈1 ,𝜈2
𝑠21 𝜎12 𝑠21

where we use 𝐹𝛼/2,𝜈1 ,𝜈2 to mean the point along the cumulative 𝐹 -distribution which has area of 𝛼/2
using 𝜈1 degrees of freedom for estimating 𝑠1 and 𝜈2 degrees of freedom for estimating 𝑠2 . For
example, in R, the value of 𝐹0.05/2,10,20 can be found from qf(0.025, 10, 20) as 0.2925. The point
along the cumulative 𝐹 -distribution which has area of 1 − 𝛼/2 is denoted as 𝐹1−𝛼/2,𝜈1 ,𝜈2 , and 𝛼 is the
level of confidence, usually 𝛼 = 0.05 to denote a 95% confidence level.

2.14.3 Confidence interval for proportions: the binomial proportion confidence interval
Sometimes we measure the proportion of successes (passes). For example, if we take a sample of 𝑛
independent items from our production line, and with an inspection system we can judge pass or
failure. The proportion of passes is what is important, and we wish to construct a confidence region for
the population proportion. This allows one to say the population proportion of passes lies between the
given range. As in the proportion of packaged pizzas with 20 or more pepperoni slices is between 86 and 92%.

Incidentally, it is this confidence interval that is used in polls to judge the proportion of people that
prefer a political party. One can run this confidence interval backwards and ask: how many
independent people do I need to poll to achieve a population proportion that lies within a range of
±2%, 19 times out of 20? The answer actually is function of the poll result! But the worst case scenario
is a split-poll, and that requires 2400 respondents.

94 Chapter 2. Univariate Data Analysis

 Release 547b16

2.15 Statistical tables for the normal- and t-distribution

If interested, here is the code used to generate these figures

2.15. Statistical tables for the normal- and t-distribution 95

Process Improvement Using Data

# The source code used to generate the

# *normal distribution* section:
q <- c(seq(-3.0, -2.0, 0.25),
c(-1.8, -1.5, -1.0, -0.5, 0, 0.5,
1.0, 1.5, 1.8),
seq(2.0, 3.0, 0.25))
[Link] = pnorm(q)

p <- c(0.001, 0.0025, 0.005, 0.010, 0.025,

0.05, 0.075, 0.10, 0.3, 0.5, 0.7,
0.9, 0.925, 0.950, 0.975, 0.99,
0.995, 0.9975, 0.999)
[Link] = qnorm(p)

layout(matrix(c(1,2), 1, 2))
par(mar = c(4.2, 4.2, 0.2, 1))
plot(q, [Link],
type = "b",
main = "",
xlab = "z",
ylab = "q = cumulative area under the normal distribution",
[Link] = 1.4,
[Link] = 1.8,
lwd = 4,
[Link] = 1.8,
[Link] = 1.8,
ylim = c(0, 1))
grid(col="gray30")
a1 = -0.6
arrows(a1, y = -0.2, x1 = a1,
y1 = pnorm(a1),
code = 0, lwd = 2)
arrows(a1, y = pnorm(a1), x1 = -3,
y1 = pnorm(a1), code = 2, lwd = 2)
text(-2, pnorm(a1) + 0.05, "pnorm(z)",
cex = 1.5)

plot([Link], p,
type = "b",
main = "",
xlab = "z",
ylab = "q = cumulative area under the normal distribution",
[Link] = 1.4,
[Link] = 1.8,
lwd = 4,
[Link] = 1.8,
[Link] = 1.8,
ylim = c(0, 1))
grid(col = "gray30")
a1 = qnorm(0.65)
arrows(a1, y = 0, x1 = a1,
y1 = pnorm(a1), code = 1, lwd = 2)
arrows(a1, y=pnorm(a1), x1 = -5,
y1 = pnorm(a1), code = 0, lwd = 2)
text(-2, pnorm(a1)+0.05, "qnorm(q)",
cex = 1.5)

# The source code used to generate the t-distribution section:

dof <- c(1, 2, 3, 4, 5, 10, 15, 20,
30, 60, Inf)
[Link] <- c(0.4, 0.25, 0.1,
0.05, 0.025, 0.01, 0.005)

[Link] <- length(dof)

[Link] <- length([Link])

(continues on next page)

96 Chapter 2. Univariate Data Analysis

 Release 547b16

(continued from previous page)

values <- matrix(0, nrow=[Link], ncol=[Link])
k = 0
for (entry in [Link]){
k = k + 1
values[ , k] <- abs(qt(entry, dof))
}
round(values,3)

par(mar=c(4.2, 4.2, 0.2, 1))

z <- seq(-5, 5, 0.01)
probabilty <- dt(z, df=5)
plot(z, probabilty,
type = "l",
main = "",
xlab = "z",
ylab = "Probabilities from the t-distribution",
[Link] = 1.4,
[Link] = 1.8,
lwd = 4,
[Link] = 1.8,
[Link] = 1.8)
abline(h = 0)
z = 1.5
abline(v = z)
abline(v = 0)

2.16 Exercises

ò Question

1 ∑︀ {︀ }︀ 1 ∑︀
Recall that 𝜇 = ℰ(𝑥) = 𝑥 and 𝒱 {𝑥} = ℰ (𝑥 − 𝜇)2 = 𝜎 2 = (𝑥 − 𝜇)2 .
𝑁 𝑁
1. What is the expected value thrown of a fair 6-sided die? (Note: plural of die is dice)

2. What is the expected variance of a fair 6-sided die?

ò Solution

Often the mean and standard deviation of a uniform distribution are not actual values from the
distribution, however the definitions for them hold:
1 ∑︀ 1
1. 𝜇 = ℰ(𝑥) = 𝑥 = (1 + 2 + 3 + 4 + 5 + 6) = 3.5
𝑁 6
{︀ }︀
2. 𝒱 {𝑥} = ℰ (𝑥 − 𝜇)2 =
1 ∑︀
(1 − 3.5)2 + (2 − 3.5)2 + (3 − 3.5)2 + (4 − 3.5)2 + (5 − 3.5)2 + (6 − 3.5)2 = 17.5/6 = 2.92
𝑁
If you’re feeling adventurous, you can simulate random dice rolls and verify your answers:

> N = 10000
> hist([Link](runif(N, 1, 7))) # make sure you get a uniform distribution
> mean([Link](runif(N, 1, 7))) # 3.4929
> var([Link](runif(N, 1, 7))) # 2.885426

2.16. Exercises 97
Process Improvement Using Data

ò Question

Characterizing a distribution: Compute the mean, median, standard deviation and MAD for salt
content for the various soy sauces given in this report39 (page 41) as described in the the article
from the Globe and Mail40 on 24 September 2009. Plot a box plot of the data and report the
interquartile range (IQR). Comment on the 3 measures of spread you have calculated: standard
deviation, MAD, and interquartile range.

The raw data are given below in units of milligrams of salt per 15 mL serving:

[460, 520, 580, 700, 760, 770, 890, 910, 920, 940, 960, 1060, 1100]

ò Solution

salt <- c(460, 520, 580, 700, 760, 770, 890, 910, 920, 940, 960, 1060, 1100)
mean(salt) # 813.077
median(salt) # 890
sd(salt) # 202.0885
mad(salt) # 192.738
1.4826*median(abs(salt - median(salt))) # 192.738
IQR(salt) # 240 = 940 - 700 (see below)
summary(salt)
# Min. 1st Qu. Median Mean 3rd Qu. Max.
# 460.0 700.0 890.0 813.1 940.0 1100.0

boxplot(salt, ylab="Salt content (mg / 15mL serving)")

# Testing robustness. Create an artificial outlier

# ------------------
salt[2] = 91941
mean(salt) # 7845.462 <---- not robust at all
median(salt) # 910 <---- robust
sd(salt) # 25268.22 <---- way off!
mad(salt) # 222.39 <---- robust
1.4826*median(abs(salt - median(salt))) # 222.39 <---- manual calculation of MAD
IQR(salt) # 200 <---- robust

39 [Link]
40 [Link]

98 Chapter 2. Univariate Data Analysis

 Release 547b16

Note that the units of spread are the same as the variable being quantified. The IQR is 240 mg
salt/15 mL serving. The standard deviation (202 mg salt/15 mL serving), and MAD (193 mg
salt/15 mL serving), are 2 other ways to quantify the spread of the data. Note that the IQR, for
normally distributed data, will only be consistent if you divide the result by 1.349. Read the help
for the IQR function in R for more details. Note from the code how the IQR is a distance between
two points.

In this example the numbers are mostly in agreement, because there are no major outliers. The
MAD and IQR are two robust methods of quantifying spread, while the standard deviation is
extremely sensitive to outliers - due to the squaring of residuals about the mean. You can verify this
by replacing one of the values and recalculating the numbers.

ò Question

Give a reason why Statistics Canada reports the median income when reporting income by
geographic area. Where would you expect the mean to lie, relative to the median? Use this table41
to look up the income for Hamilton. How does it compare to Toronto? And all of Canada?

ò Solution

We described how easily the mean is influenced by unusual data points (page 49). Take any group of
people anywhere in the world, and there will always be a few who earn lots of money (not
everyone can be the CEO, especially of a bank!). Also, since no one earns negative income, the
distribution piles up at the left, with fewer people on the right. This implies that the mean will lie
above the median, since 50% of the histogram area must lie below the median, by definition. A
previous student pointed out that low income earners are less likely to file tax returns, so they are
underrepresented in the data.

41 [Link]

2.16. Exercises 99
Process Improvement Using Data

Even though the median is a more fair way of reporting income, and robust to unusual earners
(many low income earners, very few super-rich), I would prefer if Statistics Canada released a
histogram - that would tell a lot more - even just the MAD, or IQR would be informative. It was
surprising that Hamilton showed higher median earnings per family than Toronto. I infer from this
that there are more low income earners in Toronto and Canada than in Hamilton, but without the
histograms it is hard to be sure. Also, I wasn’t able to find exactly what StatsCan means by a family
- did they include single people as a “family”? Maybe there are more, wealthy singles in Toronto,
but they are aren’t included in the numbers. The median income per person would be a useful
statistic to help judge that.

ò Question

Use the data set on raw materials42 .

• How many variables in the data set?

• How many observations?

• The data are properties of a powder. Plot each variable, one at a time, and locate any outliers.
R-users will benefit from the R tutorial43 (see the use of the identify function).

ò Solution

See the code below that generates the plots. Outliers were identified by visual inspection of these
plots. Recall an outlier is an unusual/interesting point, and a function of the surrounding data.
You can use a box plot to locate preliminary outliers, but recognize that you are leaving the
computer to determine what is unusual. Automated outlier detection systems work moderately
well, but there is no substitute (yet!) for visual inspection of the data.

The same few samples appear to be outliers in most of the variables.

rm <- [Link]('[Link]

ncol(rm) # 7 columns
nrow(rm) # 36 rows

# Plot the data as you normally would

plot(rm$size1, ylab="Particle size: level 1")

# Now use the identify(...) command, with the same data as you plotted. Use the
# "labels" option to let R use the "Sample" column to label points where you click
identify(rm$size1, labels=rm$Sample)

# After issuing the "identify(...)" command, click on any interesting points in the
# plot. Right-click anywhere to stop selecting points.

# Repeat with the other columns

plot(rm$size2, ylab="Particle size: level 2")
identify(rm$size2, labels=rm$Sample)
plot(rm$size3, ylab="Particle size: level 3")
identify(rm$size3, labels=rm$Sample)
plot(rm$density1, ylab="Particle density: level 1")
identify(rm$density1, labels=rm$Sample)
plot(rm$density2, ylab="Particle density:
42 [Link] level 2")
identify(rm$density2, labels=rm$Sample)
43 [Link]
plot(rm$density3, ylab="Particle density: level 3")
identify(rm$density3, labels=rm$Sample)

100 Chapter 2. Univariate Data Analysis

 Release 547b16

2.16. Exercises 101

Process Improvement Using Data

102 Chapter 2. Univariate Data Analysis

 Release 547b16

ò Question

2.16. Exercises 103

Process Improvement Using Data

Write a few notes on the purpose of feedback control, and its effect on variability of process quality.

ò Solution

• Purpose is to keep the process close to a desired set point (or mean).

• Sometimes used to maintain the process variability within a desired tolerance limit (or standard
deviation).

• Lowers the variability of the process outputs (i.e., narrow the distribution) by actually
introducing greater variability into the process, to counteract external variation in the the process
inputs. For example, variation from the raw materials, or ambient conditions, such as seasonal
temperature are process inputs.

• Feedback control allows us to move the process operation closer to targets, without less
likelihood of deviation outside these limits. (In the next section on process monitoring we will
learn how to track and quantify this).

ò Question

Use the section on Historical data44 from Environment Canada’s website and use the Customized
Search option to obtain data for the HAMILTON A station from 2000 to 2009. Use the settings as
Year=2000, and Data interval=Monthly and request the data for 2000, then click Next year
to go to 2001 and so on.

• For each year from 2000 to 2009, get the total snowfall and the average of the Mean temp over
the whole year (the sums and averages are reported at the bottom of the table).

• Plot these 2 variables against time

• Now retrieve the long-term averages for these data from a different section of their website45
(use the same location, HAMILTON A, and check that the data range is 1971 to 2000).
Superimpose the long-term average as a horizontal line on your previous plot.

• Note: the purpose of this exercise is more for you to become comfortable with web-based data
retrieval, which is common in most companies.

• Note: please use any other city for this question if you prefer.

ò Solution

These are the data, and the code to plot the results. The temperature for the last decade trended
higher than the average for the prior 3 decades, 1971 to 2000.

snow <- c(170.9, 94.1, 138.0, 166.2, 175.8, 218.4, 56.6, 182.4, 243.2)
temp <- c(7.6, 8.8, 8.8, 7.3, 7.7, 8.2, 9.1 , 8.2, 7.7)
year <- seq(2000, 2008)

[Link] = 161.8

44 [Link]
45 [Link]

104 Chapter 2. Univariate Data Analysis

 Release 547b16

[Link] = 7.6

plot(year, snow, xlab="Year", ylab="Snowfall (cm)", type="b")

abline(h=[Link], col = "gray40")
text(2006, [Link]-5, "Total annual snowfall, averaged over 1971 to 2000 data")

plot(year, temp, xlab="Year", ylab="Average annual temperature (°C)", type="b")

abline(h=[Link], col = "gray40")
text(2006, [Link]-0.1, "Average annual temperature, averaged over 1971 to 2000
˓→data")

2.16. Exercises 105

Process Improvement Using Data

ò Question

Does the number of visits in the website traffic46 data set follow a normal distribution? If so, what
are the parameters for the distribution? What is the likelihood that you will have between 10 and
30 visits to the website?

ò Solution

web <- [Link]('[Link]

summary(web)

# Check for normality: use a qq-plot

qqnorm(web$Visits)
qqline(web$Visits)

# Use the [Link] function from the "car" package for qq-plots with error bars
library(car)
[Link](web$Visits)

# Mean and standard deviation

[Link] <- mean(web$Visits) # 22.23364
[Link] <- sd(web$Visits) # 8.331826

[Link] <- (10 - [Link]) / [Link]

[Link] <- (30 - [Link]) / [Link]
c([Link], [Link]) # -1.4683029 0.9321312
pnorm([Link]) - pnorm([Link]) # 0.7533546

The above source code was used to generate these plots to answer the question. The data do
appear to follow a normal distribution. This means we can calculate the mean and standard
deviation from the data.

• Mean number of visits = 22 visits

• Standard deviation of the number of visits = 8.3 visits

• Probability that there are between 10 and 30 visits to the site each day: 75.3%

We should use the 𝑡-distribution to answer the last part, but at this stage we had not yet looked at
the 𝑡-distribution. However, the large number of observations (214) means the 𝑡-distribution is no
different than the normal distribution.

ò Question

The ammonia concentration in your wastewater treatment plant is measured every 6 hours. The
data for one year are available from the dataset website47 .

1. Use a visualization plot to hypothesize from which distribution the data might come. Which
distribution do you think is most likely? Once you’ve decided on a distribution, use a qq-plot to
test your decision.

2. Estimate location and spread statistics assuming the data are from a normal distribution. You
can investigate using the fitdistr function in R, in the MASS package.

46 [Link]

106 Chapter 2. Univariate Data Analysis

 Release 547b16

3. What if you were told the measured values are not independent. How does it affect your
answer?

4. What is the probability of having an ammonia concentration greater than 40 mg/L when:

• you may use only the data (do not use any estimated statistics)

• you use the estimated statistics for the distribution?

Note: Answer this entire question using computer software to calculate values from the normal
distribution. But also make sure you can answer the last part of the question by hand, (when
given the mean and variance), and using a table of normal distributions.

ò Solution

nh4 <- [Link]('[Link]

summary(nh4$Ammonia) # just to check if the data seem OK
# Min. 1st Qu. Median Mean 3rd Qu. Max.
# 9.99 30.22 36.18 36.09 42.37 58.74

# Investigate the histogram or density plots first

hist(nh4$Ammonia)
plot(density(nh4$Ammonia))

# The qq-plot confirms it is normal, apart from the right-hand-side tail

library(car)
png(file='[Link]')
qqPlot(nh4$Ammonia)
[Link]()

# Estimate the parameters of the distribution

[Link] = mean(nh4$Ammonia) # 36.09499
[Link] = sd(nh4$Ammonia) # 8.518928

# Advanced: use the MASS package in R to estimate

# (very similar results)
fitdistr(nh4$Ammonia, "normal")

level <- 40

# Using only the data to calculate p(Ammonia > level):

# calculate fraction of samples greater than ``level``
sum(nh4$Ammonia > level) / length(nh4$Ammonia)

# Using the normal distribution to estimate p(Ammonia > level):

# Calculate a z-value first, then the cumulative probability

z <- (level - [Link])/[Link]
1 - pnorm(z)

# Or, you can get the answer more directly:

1 - pnorm(level, mean=[Link], sd=[Link])

# More correctly, we should have used the t-distribution,

# because we actually estimated the standard deviation
# We basically get the same answer
1 - pt(z, df=(length(nh4$Ammonia)-1))

1. When plotting a histogram, it seems that an appropriate distribution might be the normal

47 [Link]

2.16. Exercises 107

Process Improvement Using Data

distribution. A qq-plot shows it it mostly normal, apart from the right hand side tail (upper tail)
which is slightly heavier, outside the given limits, than would be found on the normal
distribution.

2. Assuming the data are normal, we can calculate the distribution’s parameters as 𝑥 = 𝜇
^ = 36.1
and 𝑠 = 𝜎
^ = 8.52.

3. The fact that the data are not independent is not an issue. To calculate estimates of the
parameter’s distribution (the mean and standard deviation) we do not need to assume
independence. One way to see this: if I randomly reorder the data, I will still get the same value
for the mean and standard deviation. The assumption of independence is required for the
central limit theorem, but we have not used that theorem here.

4. The probability of having an ammonia concentration greater than 40 mg/L:

• When counting the fraction of the samples greater than 40 mg/L (i.e. we only use the data
themselves): 3.44% (see code)

• When using the estimated values of the mean and standard deviation from the normal
distribution, we can calculate a 𝑧-value, then find the area under the normal distribution
corresponding to this 𝑧: 3.23% (see code)

Note: We should use actually be using the 𝑡-distribution, since we used an estimate of the
population variance and not the true population variance to calculate 𝑧. However, since the
degrees of freedom, 𝑛 − 1 = 1439, are so large, there is no practical difference in our answer.

ò Question

We take a large bale of polymer composite from our production line and using good sampling
techniques, we take 9 samples from the bale and measure the viscosity in the lab for each sample.
These samples are independent estimates of the population (bale) viscosity. We will believe these
samples follow a normal distribution (we could confirm this in practice by running tests and
verifying that samples from any bale are normally distributed). Here are 9 sampled values: 23,
19, 17, 18, 24, 26, 21, 14, 18.

• The sample average

• An estimate of the standard deviation

• What is the distribution of the sample average, 𝑥? What are the parameters of that distribution?

Additional information: I use a group of samples and calculate the mean, 𝑥, then I take
another group of samples and calculate another 𝑥, and so on. Those values of 𝑥 are not
going to be the same, but they should be similar. In other words, the 𝑥 also has a
distribution. So this question asks what that distribution is, and what its parameters are.

• Construct an interval, symbolically, that will contain, with 95% certainty (probability), the
population mean of the viscosity.

Additional information: To answer this part, you should move everything to 𝑧-coordinates
first. Then you need to find the points −𝑐 and +𝑐 in the following diagram that mark the
boundary for a 95% of the total area under the distribution. This region is an interval that
will contain, with 95% certainty, the population mean of the viscosity, 𝜇. Write your answer
in form: LB < 𝜇 < UB.

108 Chapter 2. Univariate Data Analysis

 Release 547b16

• Now assume that for some hypothetical reason we know the standard deviation of the bale’s
viscosity is 𝜎 = 3.5 units, calculate the population mean’s interval numerically.

Additional information: In this part you are just finding the values of LB and UB

ò Solution

data <- c(23, 19, 17, 18, 24, 26, 21, 14, 18)
N = length(data)

# Mean and standard deviation

[Link] <- mean(data)
[Link] <- sd(data)
cn <- qnorm(1 - 0.05/2)

# Assume we know sigma:

sigma <- 3.5
LB <- [Link] - cn * sigma / sqrt(N)
UB <- [Link] + cn * sigma / sqrt(N)
c(LB, UB)

• Sample average = 20

• Sample standard deviation = 3.81

• By the central limit theorem, and if the samples are taken independently, the mean,
√
𝑥 ∼ 𝒩 (𝜇, 𝜎/ 𝑛)
𝑥−𝜇
• The z-value for 𝑥 can be constructed as 𝑧 = √ . An interval within which we can find 𝜇 with
𝜎/ 𝑛
95% certainty is given below where 𝑐𝑛 is found from the normal distribution, and in R:
qnorm(0.975) = 1.959964, approximately 1.96.
𝑥−𝜇
−𝑐𝑛 ≤ √ ≤ +𝑐𝑛
𝜎/ 𝑛
𝜎 𝜎
𝑥 − 𝑐𝑛 √ ≤ 𝜇 ≤ 𝑥 + 𝑐𝑛 √
𝑛 𝑛
LB ≤ 𝜇 ≤ UB
• The 95% confidence interval for 𝜇 is from 17.7 to 22.3.

2.16. Exercises 109

Process Improvement Using Data

ò Question

You are responsible for the quality of maple syrup produced at your plant. Historical data show
that the standard deviation of the syrup viscosity is 40 cP. How many lab samples of syrup must
you measure so that an estimate of the syrup’s long-term average viscosity is inside a range of 60
cP, 95% of the time? This question is like the previous one: except this time you are given the range
of the interval UB − LB, and you need to find 𝑛.

ò Solution

We can write the range symbolically as:

𝜎
LB = 𝑥 − 𝑐𝑛 √
𝑛
𝜎
UB = 𝑥 + 𝑐𝑛 √
𝑛

Subtracting and setting equal to 60 cP:

𝜎
UB − LB = 60 = 2𝑐𝑛 · √
𝑛
(︂ )︂2
(2)(1.96)(40)
𝑛=
60
𝑛 ≈ 7 samples

ò Question

Your manager is asking for the average viscosity of a product that you produce in a batch process.
Recorded below are the 12 most recent values, taken from consecutive batches. State any
assumptions, and clearly show the calculations which are required to estimate a 95% confidence
interval for the mean. Interpret that confidence interval for your manager, who is not sure what a
confidence interval is.

Raw data: [13.7, 14.9, 15.7, 16.1, 14.7, 15.2, 13.9, 13.9, 15.0, 13.0, 16.7, 13.2]
Mean: 14.67
Standard deviation: 1.16

Ensure you can also complete the question by hand, using statistical tables.

ò Solution

The confidence interval for a mean requires the assumption that the individual numbers are taken
from a normal distribution, and they are sampled independently (no sample has an effect on the
others). Under these assumptions we can calculate a 𝑧-value for the sampled mean, 𝑥, and
construct upper and lower bounds reflecting the probability of sampling that 𝑧-value.
𝑥−𝜇
−𝑐𝑛 ≤ √ ≤ 𝑐𝑛
𝜎/ 𝑛
Since we don’t know the value of 𝜎, we use the sampled value, 𝑠 = 1.16. But this means our

110 Chapter 2. Univariate Data Analysis

 Release 547b16

𝑧-value is no longer normally distributed, rather it is 𝑡-distributed. The limits, ±𝑐𝑡 that contain 95%
of the area under the 𝑡-distribution, with 11 degrees of freedom, are 2.20 (or any close
approximation from the tables provided). From this we get the confidence interval:

𝑥−𝜇
−𝑐𝑡 ≤ √ ≤ 𝑐𝑡
𝑠/ 𝑛
2.20 × 1.16 2.20 × 1.16
14.67 − √ ≤ 𝜇 ≤ 14.67 + √
12 12
13.93 ≤ 𝜇 ≤ 15.41

This confidence interval means that we have 95% confidence that the true average viscosity lies
within these bounds. If we took 100 groups of 12 samples, then the limits calculated from 95 of
those groups are expected to contain the true mean. It is incorrect to say that there is 95%
probability the true mean lies within these bounds; the true mean is fixed, there is no probability
associated with it.

ò Question

A new wastewater treatment plant is being commissioned and part of the commissioning report
requires a statement of the confidence interval of the biochemical oxygen demand (BOD)48 . How
many samples must you send to the lab to be sure the true BOD is within a range of 2 mg/L,
centered about the sample average? If there isn’t enough information given here, specify your own
numbers and assumptions and work with them to answer the question.

ò Solution

The objective is to calculate 𝑛, the number of samples. Let 𝑥 be the average of these 𝑛 samples, and
this average will be distributed according to the normal distribution with mean and standard
deviation as shown below, if the samples are taken independently (which may not be possible in
practice!):

𝑥BOD − 𝜇BOD
𝑧=
𝜎BOD
The value of 𝑧 will lie within this confidence interval:
𝑥BOD − 𝜇BOD
−𝑐𝑛 ≤ √ ≤ +𝑐𝑛
𝜎BOD / 𝑛
𝜎BOD 𝜎BOD
𝑥BOD − 𝑐𝑛 √ ≤ 𝜇BOD ≤ 𝑥BOD + 𝑐𝑛 √
𝑛 𝑛
LB ≤ 𝜇BOD ≤ UB

At this point all we know is that UB - LB = 2 mg/L. These are the rest of the assumptions we have
to make:

• assume a standard deviation of 𝜎

^BOD = 4 mg/L

• use 95% confidence intervals

• assume we know the population standard deviation, so we use the normal distribution to
calculate 𝑐𝑛 as qnorm(1-0.05/2) in R.

48 [Link]

2.16. Exercises 111

Process Improvement Using Data

(︂ )︂2
2(1.96)(^
𝜎BOD )
Solving for 𝑛 at these values gives: 𝑛 = = (1.96 × 4)2 ∼ 62. This large number of
2
samples makes sense: compare the range (2 mg/L) to the standard deviation of 4 mg/L: you have
to take a large number of samples to get your precision up when you have so much noise in your
signal.

ò Question

One of the questions we posed at the start of this chapter was: Here are the yields from a batch
bioreactor system49 for the last 3 years (300 data points; we run a new batch about every 3 to 4
days).

1. What sort of distribution do the yield data have?

2. A recorded yield value was less than 60%, what are the chances of that occurring? Express your
answer as: there’s a 1 in n chance of it occurring.

3. Which assumptions do you have to make for the second part of this question?

ò Solution

1. Assume the 300 data points represent an entire population. Plot a qqPlot(...) using the car
package:

The data appear to follow a normal distribution, based on the visual test of this qq-plot.

2. We need to find the probability that the yield, 𝑌 , is less than or equal to 60, stated as 𝑃 (𝑌 ≤ 60).
If we assume 𝑌 ∼ 𝒩 (𝜇, 𝜎 2 ) then we first need to find the 𝑧-value bound corresponding to 60,
and then find the probability of finding values below, or equal to that bound.
𝑦−𝜇 60 − 80.353
𝑧bound = = = −3.085
𝜎 6.597

49 [Link]

112 Chapter 2. Univariate Data Analysis

 Release 547b16

In this data set of 300 numbers there are zero entries below this limit. But using the
distribution’s fit, we can calculate the probability as pnorm(-3.085), which is ≈ 0.001. This is
equivalent to saying that there is a 1 in 1000 chance of achieving a yield less than 60%.

3. We only had to assume the data are normally distributed - we did not need the data to be
independent - in order to use the estimated parameters from the distribution to calculate the
probability.

# import data
data <- [Link]('[Link]

# Determine statistics
summary(data)
yield <- data$Yield
[Link] <- mean(yield)
[Link] <- sd(yield)

# Rather use a qqplot with limits

library(car)
png(file='[Link]')
qqPlot(yield)
[Link]()

# Assuming normal distribution determine probability of x < 60

z <- ([Link])/[Link]
p <- pnorm(z,0,1)

ò Question

One aspect of your job responsibility is to reduce energy consumption on the plant floor. You ask
the electrical supplier for the energy requirements (W.h) for running a particular light fixture for 24
hours. They won’t give you the raw data, only their histogram when they tested randomly selected
bulbs (see the data and code below).

> [Link] <- c(4025, 4075, 4125, 4175, 4225, 4275, 4325, 4375)
> [Link] <- c(4, 19, 14, 5, 4, 1, 2, 1)
> barplot([Link], [Link]=[Link], ylab="Number of bulbs (N=50)",
xlab="Energy required over 24 hours (W.h)", col="White", ylim=c(0,20))

2.16. Exercises 113

Process Improvement Using Data

• Calculate an estimate of the mean and standard deviation, even though you don’t have the
original data.

• What is a confidence interval for the mean at 95% probability, stating and testing any
assumptions you need to make.

ò Solution

• The mean and standard deviation can be estimated as shown in the code below. The estimates
are: the mean energy usage is 4127 [Link], and the standard deviation is 79 [Link]. This
corresponds very closely to the raw data I used to generate this question (mean of actual data =
4125, sd of actual data = 77.2).

[Link] <- c(4025, 4075, 4125, 4175, 4225, 4275, 4325, 4375)
[Link] <- c(4, 19, 14, 5, 4, 1, 2, 1)
N <- 50
[Link] <- sum([Link] * [Link])/N # 4127
[Link] <- sqrt(sum([Link]*([Link] - [Link])^2)/(N-1)) # 78.9

• Strictly speaking we cannot calculate a confidence interval for the mean, as the data are not
normally distributed. We can see that there is a heavy tail to the right hand side. Why do we
require the data to be normally distributed? To create the confidence interval we have to use an
estimate of the standard deviation, and then use the 𝑡-distribution to estimate the confidence
interval bounds. However, the 𝑡-distribution requires that we assume the raw data come from a

114 Chapter 2. Univariate Data Analysis

 Release 547b16

normal distribution.

But if we do calculate the confidence interval, we have to use the 𝑡-distribution at the 95%
cumulative area, with 50 - 1 = 49 degrees of freedom. In R: qt(0.025, df=49) gives
−𝑐𝑡 = −2.009575. Using our estimates of 𝑠 = 79 and 𝑥 = 4127

𝑥−𝜇
−𝑐𝑡 ≤ √ ≤ +𝑐𝑡
𝑠/ 𝑛
𝑠 𝑠
𝑥 − 𝑐𝑡 √ ≤ 𝜇 ≤ 𝑥 + 𝑐𝑡 √
𝑛 𝑛
79 79
4127 − 2.01 × ≤ 𝜇 ≤ 4127 + 2.01 ×
7 7
4104 ≤ 𝜇 ≤ 4150

Look at this answer and compare it to the original histogram; does it make sense to you?

ò Question

The confidence interval for the population mean takes one of two forms below, depending on
whether we know the variance or not. At the 90% confidence level, for a sample size of 13, compare
and comment on the upper and lower bounds for the two cases. Assume that 𝑠 = 𝜎 = 3.72.

𝑥−𝜇
−𝑐𝑛 ≤ √ ≤ 𝑐𝑛
𝜎/ 𝑛

𝑥−𝜇
−𝑐𝑡 ≤ √ ≤ 𝑐𝑡
𝑠/ 𝑛

ò Solution

This question aims for you to prove to yourself that the 𝑡-distribution is wider (more broad) than
the normal distribution. The 90% region spanned by the 𝑡-distribution with 12 degrees of freedom
has upper and lower limits at qt((1-0.9)/2, df=12), i.e. from -1.782 to 1.782. The equivalent
90% region spanned by the normal distribution is qnorm((1-0.9)/2), spanning from z=-1.64 to
z=1.64. Everything else in the center of the 2 inequalities is the same, so we only need to compare 𝑐𝑡
and 𝑐𝑛 .

ò Question

A major aim of many engineers is/will be to reduce the carbon footprint of their company’s
high-profile products. Next week your boss wants you to evaluate a new raw material that requires
kg CO2
2.6 less than the current material, but the final product’s brittleness must be the same
kg product
as achieved with the current raw material. This is a large reduction in CO2 , given your current
production capacity of 51,700 kg of product per year. Manpower and physical constraints prevent
you from running a randomized test; you don’t have a suitable database of historical data either.

2.16. Exercises 115

Process Improvement Using Data

One idea you come up with is to use to your advantage the fact that your production line has three
parallel reactors, TK104, TK105, and TK107. They were installed at the same time, they have the
same geometry, the same instrumentation, etc; you have pretty much thought about every factor
that might vary between them, and are confident the 3 reactors are identical. Typical production
schedules split the raw material between the 3 reactors. Data on the website50 contain the
brittleness values from the three reactors for the past few runs on the current raw material.

1. Which two reactors would you pick to run your comparative trial on next week?

2. Repeat your calculations assuming pairing.

ò Solution

The purpose of this question is to compare two systems. There are two ways: either compare
one group to another group, or to have paired tests. We could consider this a paired test,
because the material is run in both reactors at the same conditions. In this answer we compare
reactor I to reactor J as groups. Our answer will be to run experiments in the reactors that
show the smallest difference.

ò Note

This question also has missing data, denote as NA in R. Most real data sets that you deal with will
have missing data and the questions will expect to deal with them. For example, the degrees of
freedom will be reduced because of the missing data. Use this solution to see how to write code
in R that deals with missing values.

We can start by looking at the data. A box plot is a reasonable way to compare both the location
and spread of the brittleness values from each reactor.

50 [Link]

116 Chapter 2. Univariate Data Analysis

 Release 547b16

The standard way to test for differences between two groups of samples is given by equation (2.8) -
it is derived as coming from the normal distribution with mean of 𝜇𝐴 − 𝜇𝐵 and the standard
deviation as shown in the denominator.
(𝑥𝐵 − 𝑥𝐴 ) − (𝜇𝐵 − 𝜇𝐴 )
𝑧= √︃ (︂ )︂
2
1 1
𝜎 +
𝑛𝐴 𝑛𝐵

Assuming the two population means are identical, the 𝑧-value is a direct estimate of the probability
with which that assumption is wrong. A 𝑧-value around zero indicates that the assumption was
true, a large or small 𝑧-value indicates that the assumption was wrong.

So we can calculate the 𝑧-value, and the corresponding probability for each pair of reactor
differences using the code below.

But the next problem we face is that we don’t know the value of 𝜎. We can estimate it however, by
pooling the variances of the two groups. Strictly speaking we should do a check for comparable
variances before pooling them - described in a previous section (page 94).

When we use the pooled variance now, then the assumption that the 𝑧-value follows the normal
distribution is not correct anymore; it follows the 𝑡-distribution, with the pooled number of degrees
of freedom. Once we have the 𝑧-value we can calculate the probability of finding a 𝑧-value of at
least that big. Anything beyond that is the risk that we are wrong.

We can also expand the 𝑧 value into a confidence interval at a given confidence level. We do this in
the code at the 95% level (see LB and UB terms).

• 𝜇104 − 𝜇105 : 𝑧 = 1.25; risk we are wrong: 89.1%; CI: −31.4 ≤ 𝜇104 − 𝜇105 ≤ 134

• 𝜇104 − 𝜇107 : 𝑧 = 1.41; risk we are wrong: 91.6%; CI −21.4 ≤ 𝜇104 − 𝜇107 ≤ 120

• 𝜇105 − 𝜇107 : 𝑧 = -0.0532; risk we are wrong: 52.1% and −81.8 ≤ 𝜇105 − 𝜇107 ≤ 77.6 (note that the
minimum risk is 50%; the risk is not 47.8%)

While all three reactors have confidence intervals that span zero at the 95% level, notice how the
interval gives us a feel for the degree of difference. Clearly reactors TK105 and TK107 are the most
similar, however all 3 are statistically equivalent from a confidence interval point of view. Contrast
this to using a hypothesis test, which you may have encountered in other statistical courses. A
hypothesis test just tells you “yes” or “no”; a confidence interval gives a much better engineering
feel for the degree of difference.

A full solution to this question require you report the 𝑧-values and its corresponding risk.

# We are going to be doing this 3 times. Rather write a function to

# do the work for general "groupA" and "groupB" vectors.
group_difference <- function(groupA, groupB)
{
# This function assumes either group has missing data. Calculate
# the mean and variance omitting the missing values

[Link] <- mean(groupA[])

[Link] <- var(groupA[])
A.N <- length(groupA[])

[Link] <- mean(groupB[])

[Link] <- var(groupB[])
B.N <- length(groupB[])

difference <- [Link] - [Link]

[Link] <- (A.N - 1 + B.N - 1)

2.16. Exercises 117

Process Improvement Using Data

[Link] <- ((A.N - 1) * [Link] + (B.N - 1) * [Link]) / [Link]

[Link] <- sqrt([Link] *(1/A.N + 1/B.N))

z <- (difference - 0) / [Link]
[Link] <- pt(z, [Link])

LB <- difference - qt(0.975, df=[Link])*[Link]

UB <- difference + qt(0.975, df=[Link])*[Link]
return(list(z, [Link], LB, UB))
}

brittle <- [Link]('[Link]

attach(brittle) # Now we can access the variables directly, without $ symbols

# Let's start though by plotting boxplots of the data

bitmap('../images/[Link]', type="png256", width=7, height=7, res=250,
˓→pointsize=14)

par(mar=c(4.2, 4.2, 0.2, 0.2)) # (bottom, left, top, right);

boxplot(brittle, ylab="Brittleness index", [Link]=1.5, [Link]=1.8, [Link]=1.8, cex.
˓→axis=1.8)

[Link]()

# 104 vs 105
group_difference(TK104, TK105)
# z = 1.253729
# [Link] = 0.891298 (1-0.1087021)
# -31.4 < [Link] < 134

# 104 vs 107
group_difference(TK104, TK107)
# z = 1.405639
# [Link] = 0.9163178 (1-0.0836822)
# -21.4 < [Link] < 120

# 105 vs 107
group_difference(TK105, TK107)
# z = -0.05326222
# [Link] = 0.4788878 (1-0.5211122)
# -81.8 < [Link] < 77.6

Using a paired test

Pairing assumes that each reactor was run with the same material, except that the material was
split into thirds: one third for each reactor. As described in the section on paired tests (page 92) we
rely on calculating the difference in brittleness, then calculating the 𝑧-value of the average
difference. Contrast this to the unpaired tests, where we calculated the difference of the averages.

The code below shows how the paired differences are evaluated for each of the 3 combinations.
The paired test highlights the similarity between TK105 and TK107, the same as the unpaired test.
However the paired test shows much more clearly how different tanks TK104 and TK105 are, and
especially TK104 and TK107.

In the case of TK104 and TK105 the difference might seem surprising - take a look back at the box
plots and how much they overlap. However a paired test cannot be judged by a box plot, because it
looks at the case-by-case difference, not the overall between group difference. A better plot with
which to confirm the really large 𝑧-value for the TK105 and TK107 difference is the plot of the
differences.

118 Chapter 2. Univariate Data Analysis

 Release 547b16

brittle <- [Link]('[Link]

attach(brittle)

# Calculates the paired difference

paired_difference <- function(groupA, groupB, alpha=0.95)
{
# This function assumes either group has missing data.
# Find the subset of observations in common.

[Link] <- groupA[ & ]

[Link] <- groupB[ & ]

diffs <- [Link] - [Link]

[Link] <- mean(diffs)
[Link] <- sd(diffs)
diffs.N <- length(diffs)

plot([Link], type="b")

z <- ([Link] - 0) / ([Link]/sqrt(diffs.N))

[Link] <- pt(z, df=(diffs.N-1))
c.t <- qt(1-(1-alpha)/2, df=(diffs.N-1))
LB <- [Link] - c.t * [Link] / sqrt(diffs.N)
UB <- [Link] + c.t * [Link] / sqrt(diffs.N)

return(list(z, [Link], diffs.N-1, LB, UB))

}

paired_difference(TK104, TK105, alpha=0.95)

# (z=2.64, [Link]=0.991, DOF=17, LB=9.81, UB=88.4)

paired_difference(TK104, TK107, alpha=0.95)

# (z=12, [Link]=1, DOF=19, LB=48.3, UB=68.7)

paired_difference(TK105, TK107, alpha=0.95)

# (z=-0.33, [Link]=0.37, DOF=20, LB=-46.1, UB=33.5)

Not required for the full grade, but one can show the confidence intervals are:

9.81 ≤ 𝜇105−104 ≤ 88.4

48.3 ≤ 𝜇107−104 ≤ 68.7
−46.1 ≤ 𝜇107−105 ≤ 33.5

Advanced students should look at how the reduction in degrees of freedom affects this test; and
contrast the results to those when using an unpaired test.

ò Question

Use the website traffic data51 from the dataset website:

• Write down, symbolically, the z-value for the difference in average visits on a Friday and
Saturday.

• Estimate a suitable value for the variance and justify your choice.

• What is the probability of obtaining a z-value of this magnitude or smaller? Would you say the
difference is significant?

• Pick any other 2 days that you would find interesting to compare and repeat your analysis.

2.16. Exercises 119

Process Improvement Using Data

ò Solution

• Let our variable of interest be the difference between the average of the 2 groups: 𝑥Fri − 𝑥Sat . This
variable will be distributed normally (why? - see the notes) according to
(︀ 2
)︀
𝑥Fri − 𝑥Sat ∼ 𝒩 𝜇Fri − 𝜇Sat , 𝜎diff . So the z-value for this variable is:
(𝑥Fri − 𝑥Sat ) − (𝜇Fri − 𝜇Sat )
𝑧=
𝜎diff
(︂ )︂
2 1 1
• The variance of the difference, 𝜎diff = 𝜎2 + , where 𝜎 2 is the variance of the number
𝑛Fri 𝑛Sat
of visits to the website on Friday and Saturday. Since we don’t know that value, we can estimate
it from pooling the 2 variances of each group. We should calculate first that these variances are
comparable (they are; but you should confirm this yourself (page 94)).
(𝑛Fri − 1)𝑠2Fri + (𝑛Sat − 1)𝑠2Sat
𝜎 2 ≈ 𝑠2𝑃 =
𝑛Fri − 1 + 𝑛Sat − 1
29 × 45.56 + 29 × 48.62
=
58
= 47.09
• The z-value calculated from this pooled variance is:

20.77 − 15.27
𝑧= (︂ )︂ = 3.1
1 1
47.09 +
30 30

But since we used an estimated variance, we cannot say that 𝑧 comes from the normal
distribution anymore. It now follows the 𝑡-distribution with 58 degrees of freedom (which is still
comparable to the normal distribution - see question 7 below). The corresponding probability
that 𝑧 < 3.1 is 99.85%, using the 𝑡-distribution with 58 degrees of freedom. This difference is
significant; there is a very small probability that this difference is due to chance alone.

• The code was modified to generate the matrix of z-value results in the comments below. The
largest difference is between Sunday and Wednesday, and the smallest difference is between
Monday and Tuesday.

51 [Link]

120 Chapter 2. Univariate Data Analysis

 Release 547b16

website <- [Link]('[Link]

attach(website)

[Link] <- Visits[DayOfWeek=="Monday"]

[Link] <- Visits[DayOfWeek=="Tuesday"]
[Link] <- Visits[DayOfWeek=="Wednesday"]
[Link] <- Visits[DayOfWeek=="Thursday"]
[Link] <- Visits[DayOfWeek=="Friday"]
[Link] <- Visits[DayOfWeek=="Saturday"]
[Link] <- Visits[DayOfWeek=="Sunday"]

# Look at a boxplot of the data from Friday and Saturday

bitmap('[Link]', type="png256", width=7, height=7,
res=250, pointsize=14)
par(mar=c(4.2, 4.2, 0.2, 0.2)) # (bottom, left, top, right)
boxplot([Link], [Link], names=c("Friday", "Saturday"), ylab="Number of visits",
[Link]=1.5, [Link]=1.8, [Link]=1.8, [Link]=1.8)
[Link]()

# Use the "group_difference" function from question 4

group_difference([Link], [Link])
# z = 3.104152
# [Link] = 0.9985255 (1-0.001474538)

# All differences: z-values

# ----------------------------
# Mon Tue Wed Thu Fri Sat Sun
# Mon 0.0000000 NA NA NA NA NA NA
# Tue -0.2333225 0.000000 NA NA NA NA NA
# Wed -0.7431203 -0.496627 0.000000 NA NA NA NA
# Thu 0.8535025 1.070370 1.593312 0.000000 NA NA NA
# Fri 2.4971347 2.683246 3.249602 1.619699 0.000000 NA NA
# Sat 5.4320361 5.552498 6.151868 4.578921 3.104152 0.000000 NA
# Sun 3.9917201 4.141035 4.695493 3.166001 1.691208 -1.258885 0

ò Question

You plan to run a series of 22 experiments to measure the economic advantage, if any, of switching
to a corn-based raw material, rather than using your current sugar-based material. You can only
run one experiment per day, and there is a high cost to change between raw material dispensing
systems. Describe two important precautions you would implement when running these
experiments, so you can be certain your results will be accurate.

ò Solution

Some important precautions one has to take are:

1. Keep all disturbance factors as constant as possible: e.g. use the same staff for all experiments
(Corn and Sugar), keep other variables on the process as constant as possible.

2. Randomize the order of the experiments, despite the cost, to obtain independent experimental
measurements. For example, if you cannot use the same staff for all experiments, then the
experiment order must be randomization. Do not, for example, use group A staff to run the Corn
experiments and group B staff to run the Sugar experiments.

Randomization is expensive and inconvenient, but is the insurance we pay to ensure the results

2.16. Exercises 121

Process Improvement Using Data

are not confounded by unmeasured disturbances.

3. Use representative lots of corn- and sugar-based materials. You don’t want to run all your
experiments on one batch of corn or sugar. What if the batch of corn-based material was an
unusual in some way and showed no difference, when really there is a long-term difference? Or
the opposite could have occurred as well.

ò Question

There are two analytical techniques for measuring biochemical oxygen demand (BOD)52 . You wish
to evaluate the two testing procedures, so that you can select the test which has lower cost, and
fastest turn-around time, but without a compromise in accuracy. The table contains the results of
the each test, performed on a sample that was split in half.

1. Is there a statistical difference in accuracy between the two methods?

2. Review the raw data and answer whether there is a practical difference in accuracy.

Dilution method Manometric method

11 25
26 3
18 27
16 30
20 33
12 16
8 28
26 27
12 12
17 32
14 16

ò Solution

The temptation is to jump into the code and calculate the 𝑡-values and averages differences
(𝑥𝐷 = 16.4, and 𝑥𝑀 = 22.6). But start with a plot of the data, specifically a plot of the differences
between the two methods. The immediate problem you see is that average difference of 6.2
between the methods is strongly influenced by a single observation (the second one). In general,
the dilution method always produced a smaller result than the manometric method. We expect to
see that in our analytical results.

52 [Link]

122 Chapter 2. Univariate Data Analysis

 Release 547b16

6.27
Now let’s look at the analytical answer. As before, we can calculate 𝑧 = 1.86 = (where
3.375
2
𝑠𝑝 = 62.7), with a probability of 96.1% that we will have a value smaller than this (risk = 3.9% that
we are wrong). A confidence interval would be −0.77 < 𝜇M − 𝜇D < 13.3. And it is at this point that
you should realize the problem, even if you didn’t plot your data. The fact that the confidence
interval only just includes zero is what should raise concern; if the two methods were roughly
equivalent, then the interval should span zero with rough symmetry. But this is too close.
9.20
So omitting the second point and repeating the analysis gives: calculate 𝑧 = 3.24 = (where
2.84
2
𝑠𝑝 = 40.4), with a probability of 99.8% that we will have a value smaller than this (risk = 0.2% that
we are wrong). A confidence interval would be 3.2 < 𝜇M − 𝜇D < 15.2; this is a result that is much
more aligned with the plotted data.

ò Note

You may have discovered/used the [Link](...) function in R. If you know what you are
doing with this function, you are welcome to use it; however I’m reluctant to advocate its use at
this point, because these exercises are all about understanding what is going on with confidence
intervals and calculating them yourself.

ò Question

Plot the cumulative probability function for the normal distribution and the 𝑡-distribution on the
same plot.

• Use 6 degrees of freedom for 𝑡-distribution.

2.16. Exercises 123

Process Improvement Using Data

• Repeat the plot for a larger number of degrees of freedom.

• At which point is the 𝑡-distribution indistinguishable from the normal distribution?

• What is the practical implication of this result?

ò Solution

z <- seq(-5, 5, 0.1)

norm <- pnorm(z)

bitmap('[Link]', type="png256", width=12, height=7,

res=300, pointsize=14)
par(mar=c(4.2, 4.2, 2.2, 0.2))

layout(matrix(c(1,2), 1, 2))
plot(z, norm, type="p", pch=".", cex=5, main="Normal and t-distribution (df=6)",
ylab="Cumulative probability")
lines(z, pt(z, df=6), type="l", lwd=2)
legend(0.5, y=0.35, legend=c("Normal distribution", "t-distribution (df=8)"),
pch=c(".", "-"), [Link]=c(5, 2))

plot(z, norm, type="p", pch=".", cex=5, main="Normal and t-distribution (df=35)",

ylab="Cumulative probability")
lines(z, pt(z, df=35), type="l", lwd=2)
legend(0.5, y=0.35, legend=c("Normal distribution", "t-distribution (df=35)"),
pch=c(".", "-"), [Link]=c(5, 2))
[Link]()

The above source code and figure output shows that the 𝑡-distribution starts being
indistinguishable from the normal distribution after about 35 to 40 degrees of freedom. This means
that when we deal with large sample sizes (over 40 or 50 samples), then we can use critical values
from the normal distribution rather than the 𝑡-distribution. Furthermore, it indicates that our
estimate of the variance is a pretty good estimate of the population variance for largish sample

124 Chapter 2. Univariate Data Analysis

 Release 547b16

sizes.

ò Question

Explain why tests of differences are insensitive to unit changes. If this were not the case, then one
could show a significant difference for a weight-loss supplement when measuring waist size in
millimetres, yet show no significant difference when measuring in inches!

ò Question

A food production facility fills bags with potato chips. The advertised bag weight is 35.0 grams.
But, the current bagging system is set to fill bags with a mean weight of 37.4 grams, and this done
so that only 1% of bags have a weight of 35.0 grams or less.

• Back-calculate the standard deviation of the bag weights, assuming a normal distribution.

• Out of 1000 customers, how many are lucky enough to get 40.0 grams or more of potato chips in
their bags?

ò Solution

• Calculate the z-value and find which fraction of 𝑧 falls at or below 1% of the probability area.
From the tables this is -2.326.

Then solve for 𝜎:

35 − 37.4
𝑧= = −2.326
𝜎
35 − 37.4
𝜎= = 1.03 grams
−2.326

• Probability of 40.0 grams of more is the area above the corresponding 𝑧-value:

40 − 37.4
𝑧>
1.03
𝑧 > 2.52

The exact answer is (1 - pnorm(2.52))*1000 = 5.86, though using tables you could use
the value corresponding to 𝑧 = 2.5, which is 99.38%, which is the area below that z-value. The
area above it is 0.62%, corresponding to 6.2 people. Either 5, 6 or 7 people is an acceptable
answer, depending on your rounding error.

ò Question

A food production facility fills bags with potato chips with an advertised bag weight of 50.0 grams.

1. The government’s Weights and Measures Act requires that at most 1.5% of customers may receive
a bag containing less than the advertised weight. At what setting should you put the target fill
weight to meet this requirement exactly? The check-weigher on the bagging system shows the

2.16. Exercises 125

Process Improvement Using Data

long-term standard deviation for weight is about 2.8 grams.

2. Out of 100 customers, how many are lucky enough to get 55.0 grams or more of potato chips in
their bags?

ò Solution

1. Given that it is a long-term standard deviation, we have 𝜎 = 2.8 grams. Calculate the 𝑧-value
and find which fraction of 𝑧 falls at or below 1.5% of the probability area. From the tables this is
𝑧 = −2.17 = qnorm(0.015).

Then solve for 𝜇:

50 − 𝜇
𝑧= = −2.17
2.8
𝜇 = 56.08 grams

The check weigher should be set at 56.08 grams.

2. From the prior answer, we can see many customers will receive a bag with 55.0 grams or more.
Probability of 55.0 grams of more is the area area the corresponding 𝑧-value:

55 − 56.08
𝑧>
2.8
𝑧 > −0.385

The exact answer is (1 - pnorm(-0.385))*100 = 64.98, so around 65 customers out of

every 100 (you might have a slightly different number if you used tables to answer your
question - make sure you can use the statistical tables to answer this problem too.)

ò Question

The following confidence interval is reported by our company for the amount of sulphur dioxide
measured in parts per billion (ppb) that we send into the atmosphere.

123.6 ppb ≤ 𝜇 ≤ 240.2 ppb

Only 𝑛 = 21 raw data points (one data point measured per day) were used to calculate that 90%
confidence interval. A 𝑧-value would have been calculated as an intermediate step to get the final
𝑥−𝜇
confidence interval, where 𝑧 = √ .
𝑠/ 𝑛
1. What assumptions were made about those 21 raw data points to compute the above confidence
interval?

2. Which lower and upper critical values would have been used for 𝑧? That is, which critical values
are used before unpacking the final confidence interval as shown above.

3. What is the standard deviation, 𝑠, of the raw data?

4. Today’s sulphur dioxide reading is 460 ppb and your manager wants to know what’s going on;
you can quickly calculate the probability of seeing a value of 460 ppb, or greater, to help judge

126 Chapter 2. Univariate Data Analysis

 Release 547b16

the severity of the pollution. How many days in a 365 calendar-day year are expected to show a
sulphur dioxide value of 460 ppb or higher?

5. Explain clearly why a wide confidence interval is not desirable, from an environmental
perspective.

ò Solution

1. The 21 data points are independent and come from any distribution of finite variance.

2. From the 𝑡-distribution at 20 degrees of freedom, with 5% in each tail: 𝑐𝑡 = 1.72 = qt(0.95,
df=20). The 𝑡-distribution is used because the standard deviation is estimated, rather than
being a population deviation.

3. The standard deviation may be calculated from:

𝑠 𝑠
𝑈 𝐵 − 𝐿𝐵 = 240.2 − 123.6 = 2 × 𝑐𝑡 √ = (2)(1.72) √
𝑛 𝑛
√
(116)( 𝑛)
𝑠=
(2)(1.72)
𝑠 = 154.5 ppb

Note the very large standard deviation relative to the confidence interval range. This is the
reason why so many data points were taken (21), to calculate the average, because the raw data
comes from a distribution with such a large variation.

An important note here is the large estimated value for the standard deviation and realized it
was so wide, that it would imply the distribution produced values with negative sulphur
dioxide concentration (which is physically impossible). However, note that when dealing with
large samples (21 in this case), the distinction between the normal and the 𝑡-distribution is
minimal. Further, the raw data are not necessarily assumed to be from the normal distribution,
they could be from any distribution, including one that is heavy-tailed, such as the
F-distribution53 (see the yellow and green lines in particular).

4. The probability calculation requires a mean value. Our best guess for the mean is the midpoint
of the confidence interval, which is always symmetric about the estimated process mean,
240.2 − 123.6
𝑥= + 123.6 = 181.9. Note that this is not the value for 𝜇, since 𝜇 is unknown.
2
460 − 181.9
𝑧= = 1.80
154.5
Probability is 1 - pt(1.8, df=20) = 1 − 0.9565176 = 0.0434824, or about
0.0434824 × 365 = 15.9, or about 16 days in the year (some variation is expected, if you have
used a statistical table)

5. A wide confidence interval implies that our sulphur dioxide emissions are extremely variable
(the confidence interval bounds are a strong function of the process standard deviation). Some
days we are putting more pollution up into the air and balancing it out with lower pollution on
other days. Those days with high pollution are more environmentally detrimental.

53 [Link]

2.16. Exercises 127

Process Improvement Using Data

ò Question

A common unit operation in the pharmaceutical area is to uniformly blend powders for tablets.
One such unit is illustrated below (figure taken from Wikipedia54 ). In this question we consider
blending an excipient (an inactive magnesium stearate base), a binder, and the active ingredient.
The mixing process is tracked using a wireless near infrared (NIR) probe embedded in a V-blender.
The mixer is stopped when the NIR spectra become stable. A new supplier of magnesium stearate
is being considered that will save $ 294,000 per year.

The 15 most recent runs with the current magnesium stearate supplier had an average mixing time
of 2715 seconds, and a standard deviation of 390 seconds. So far you have run 6 batches from the
new supplier, and the average mixing time of these runs is 3115 seconds with a standard deviation
of 452 seconds. Your manager is not happy with these results so far - this extra mixing time will
actually cost you more money via lost production.

The manager wants to revert back to the original supplier, but is leaving the decision up to you;
what would be your advice? Show all calculations and describe any additional assumptions, if
required.

ò Solution

This question, similar to most real statistical problems, is open-ended. This problem considers
whether a significant difference has occurred. And in many cases, even though there is significant
difference, it has to be weighed up whether there is a practical difference as well, together with the
potential of saving money (increased profit).

You should always state any assumptions you make, compute a confidence interval for the
difference and interpret it.

54 [Link]

128 Chapter 2. Univariate Data Analysis

 Release 547b16

The decision is one of whether the new material leads to a significant difference in the mixing time.
It is desirable, from a production point of view, that the new mixing time is shorter, or at least the
same. Some notation:

𝜇
^Before = 𝑥𝐵 = 2715 𝜇
^After = 𝑥𝐴 = 3115
𝜎
^Before = 𝑠𝐵 = 390 𝜎
^After = 𝑠𝐴 = 452
𝑛𝐵 = 15 𝑛𝐴 = 6

Assumptions required to compare the two groups:

• The individual samples within each group were taken independently, so that we can invoke the
central limit theorem and assume these means and standard deviation are normally distributed.

• Assume the individual samples within each group are from a normal distribution as well.

• Assume that we can pool the variances, i.e. 𝜎Before and 𝜎After are from comparable distributions.

• Using the pooled variance implies that the 𝑧-value follows the 𝑡-distribution.

• The mean of each group (before and after) is independent of the other (very likely true).

• No other factors were changed, other than the raw material (we can only hope, though in
practice this is often not true, and a paired test would eliminate any differences like this).

Calculating the pooled variance:

(𝑛𝐴 − 1)𝑠2𝐴 + (𝑛𝐵 − 1)𝑠2𝐵

𝑠2𝑃 =
𝑛𝐴 − 1 + 𝑛𝐵 − 1
(6 − 1)4522 + (15 − 1)3902
=
6 − 1 + 15 − 1
= 165837

Computing the z-value for this difference:

(𝑥𝐵 − 𝑥𝐴 ) − (𝜇𝐵 − 𝜇𝐴 )
𝑧= √︂ (︁ )︁
𝑠2𝑃 𝑛1𝐴 + 𝑛1𝐵

(2715 − 3115) − (𝜇𝐵 − 𝜇𝐴 )

𝑧= √︁
165837 16 + 15
1
(︀ )︀

−400 − (𝜇𝐵 − 𝜇𝐴 )
𝑧= = −2.03 on the hypothesis that 𝜇𝐵 = 𝜇𝐴
196.7

The probability of obtaining this value of 𝑧 can be found using the 𝑡-distribution at 6 + 15 - 2 = 19
degrees of freedom (because the standard deviation is an estimate, not a population value). Using
tables, a value of 0.025, or 2.5% is found (in R, it would be pt(-2.03, df=19) = 0.0283, or
2.83%). At this point one can argue either way that the new excipient leads to longer times, though
I would be inclined to say that this probability is too small to be due to chance alone. Therefore
there is a significant difference, and we should revert back to the previous excipient. Factors such
as operators, and other process conditions could have affected the 6 new runs.

Alternatively, and this is the way I prefer to look at these sort of questions, is to create a confidence
interval. At the 95% level, the value of 𝑐𝑡 in the equation below, using 19 degrees of freedom is
qt(0.975, df=19) = 2.09 (any value close to this from the tables is acceptable):

2.16. Exercises 129

Process Improvement Using Data

−𝑐𝑡 ≤ 𝑧 ≤ +𝑐𝑡
√︂ (︁ )︁ √︂ (︁ )︁
(𝑥𝐵 − 𝑥𝐴 ) − 𝑐𝑡 𝑠2𝑃 𝑛1𝐴 + 𝑛1𝐵 ≤ 𝜇𝐵 − 𝜇𝐴 ≤ (𝑥𝐵 − 𝑥𝐴 ) + 𝑐𝑡 𝑠2𝑃 𝑛1𝐴 + 𝑛1𝐵
√︁ √︁
−400 − 2.09 165837 16 + 15 1
≤ −400 + 2.09 165837 61 + 151
(︀ )︀ (︀ )︀
≤ 𝜇𝐵 − 𝜇𝐴
−400 − 412 ≤ 𝜇𝐵 − 𝜇𝐴 ≤ −400 + 412
−812 ≤ 𝜇𝐵 − 𝜇𝐴 ≤ 12

The interpretation of this confidence interval is that there is no difference between the current and
new magnesium stearate excipient. The immediate response to your manager could be “keep using
the new excipient”.

However, the confidence interval’s asymmetry should give you pause, certainly from a practical
point of view (this is why I prefer the confidence interval - you get a better interpretation of the
result). The 12 seconds by which it overlaps zero is so short when compared to average mixing
times of around 3000 seconds, with standard deviations of 400 seconds. The practical
recommendation is that the new excipient has longer mixing times, so “revert to using the previous
excipient”.

One other aspect of this problem that might bother you is the low number of runs (batches) used.
Let’s take a look at how sensitive the confidence interval is to that. Assume that we perform one
extra run with the new excipient (𝑛𝐴 = 7 now), and assume the pooled variance, 𝑠2𝑝 = 165837
remains the same with this new run. The new confidence interval is:
√︂ (︁ )︁ √︂ (︁ )︁
(𝑥𝐵 − 𝑥𝐴 ) − 𝑐𝑡 𝑠2𝑃 𝑛1𝐴 + 𝑛1𝐵 ≤ 𝜇𝐵 − 𝜇𝐴 ≤ (𝑥𝐵 − 𝑥𝐴 ) + 𝑐𝑡 𝑠2𝑃 𝑛1𝐴 + 𝑛1𝐵
√︁ √︁
(𝑥𝐵 − 𝑥𝐴 ) − 2.09 165837 17 + 15 1
≤ 𝜇𝐵 − 𝜇𝐴 ≤ (𝑥𝐵 − 𝑥𝐴 ) + 2.09 165837 17 + 15 1
(︀ )︀ (︀ )︀

(𝑥𝐵 − 𝑥𝐴 ) − 390 ≤ 𝜇𝐵 − 𝜇𝐴 ≤ (𝑥𝐵 − 𝑥𝐴 ) + 390

So comparing this ±390 with 7 runs, to the ±412 with 6 runs, shows that the confidence interval
shrinks in quite a bit, much more than the 12 second overlap of zero. Of course we don’t know
what the new 𝑥𝐵 − 𝑥𝐴 will be with 7 runs, so my recommendation would be to perform at least
one more run with the new excipient, but I suspect that the new run would show there to be a
significant difference, and statistically confirm that we should “revert to using the previous excipient”.

ò Question

List an advantage of using a paired test over an unpaired test. Give an example, not from the notes,
that illustrates your answer.

ò Solution

One primary advantage of pairing is that any systematic difference between the two groups (A and
B) is eliminated. For example, a bias in the measurement will cancel out when calculating the pairs
of differences. Any example is suitable as an answer: e.g. laboratory miscalibration; an offset in an
on-line sensor, etc.

Other advantages are that the raw data do not need to be normally distributed, only the paired
differences.

130 Chapter 2. Univariate Data Analysis

 Release 547b16

Another advantage is that randomization of the trials is required in the unpaired case (often a
costly extra expense), whereas in the paired case, we only need to be sure the pairs are independent
of each other (that’s much easier to assume, and often true). For example testing drug A and B on a
person, some time apart. The pairs are run on the same person, but each person in the drug trial is
independent of the other.

ò Question

An unpaired test to distinguish between group A and group B was performed with 18 runs: 9
samples for group A and 9 samples for group B. The pooled variance was 86 units.

Also, a paired test on group A and group B was performed with 9 runs. After calculating the paired
differences, the variance of these differences was found to be 79 units.

Discuss, in the context of this example, an advantage of paired tests over unpaired tests. Assume
95% confidence intervals, and that the true result was one of “no significant difference between
method A and method B”. Give numeric values from this example to substantiate your answer.

ò Solution

One advantage of the paired test is that often a fewer number of samples are required to obtain a
more sensitive result than when analyzing the data as from two distinct, unpaired groups.

Construct the confidence interval for both cases, substitute in these values and then compare the
confidence intervals. The equations for both confidence intervals are derived directly from the
𝑧-value.

Unpaired case:
(𝑥𝐵 − 𝑥𝐴 ) − (𝜇𝐵 − 𝜇𝐴 )
−𝑐𝑡 ≤ √︃ (︂ )︂ ≤ +𝑐𝑡
1 1
𝑠2𝑃 +
𝑛𝐴 𝑛𝐵
√︃ (︂ )︂ √︃ (︂ )︂
2 1 1 1 1
(𝑥𝐵 − 𝑥𝐴 ) − 𝑐𝑡 𝑠𝑃 + ≤ 𝜇𝐵 − 𝜇𝐴 ≤ (𝑥𝐵 − 𝑥𝐴 ) + 𝑐𝑡 𝑠2𝑃 +
𝑛 𝑛𝐵 𝑛 𝑛
√︃ 𝐴(︂ )︂ √︃ 𝐴(︂ 𝐵 )︂
1 1 1 1
(𝑥𝐵 − 𝑥𝐴 ) − 2.12 × 86 + ≤ 𝜇𝐵 − 𝜇𝐴 ≤ (𝑥𝐵 − 𝑥𝐴 ) + 2.12 × 86 +
9 9 9 9
(𝑥𝐵 − 𝑥𝐴 ) − 9.27 ≤ 𝜇𝐵 − 𝜇𝐴 ≤ (𝑥𝐵 − 𝑥𝐴 ) + 9.27
The 𝑐𝑡 value for the unpaired case is from the 𝑡-distribution with 16 degrees of freedom, a value of
around 2.12.

Paired case:

In this case the vector of differences is 𝑤, and by the central limit theorem it is distributed as
(︀ 2
)︀
𝑤 ∼ 𝒩 𝜇𝐵−𝐴 , 𝜎𝑤 /𝑛 , but we use the estimated variance, 𝑠2𝑤 instead.

𝑤 − 𝜇𝐵−𝐴
−𝑐𝑡 ≤ √ ≤ +𝑐𝑡
𝑠𝑤 / 𝑛

𝑠𝑤 𝑠𝑤
𝑤 − 𝑐𝑡 √ ≤ 𝜇𝑤 ≤ 𝑤 + 𝑐𝑡 √
√𝑛 √𝑛
79 79
𝑤 − 2.3 √ ≤ 𝜇𝑤 ≤ 𝑤 + 2.3 √
9 9
𝑤 − 6.81 ≤ 𝜇𝑤 ≤ 𝑤 + 6.81
2.16. Exercises 131
Process Improvement Using Data

The 𝑐𝑡 value for the paired case is from the 𝑡-distribution with 8 degrees of freedom, a value of
around 2.3.

The key result of this question is that the confidence interval for the paired case is tighter
(narrower) than the confidence interval from the unpaired case. Given that the true result was one
of no significant difference, it implies that 𝜇𝐴 = 𝜇𝐵 and that 𝜇𝑤 = 0. The tighter confidence√︂interval
79
comes purely from the fact that the standard deviation used for the paired case is smaller, vs
√︃ (︂ 9
√ √
)︂
1 1
the 86 + from the unpaired case. This is not due to the variances, since 86 ≈ 79, i.e.
9 9
(9.27 vs 8.88), but rather due to the fact that that unpaired standard deviation is multiplied by
√︀ √︀
2/9, while the paired standard deviation is multiplied by 1/9.

So while the 𝑐𝑡 value for the paired case is actually larger (widening the confidence interval due to
the fewer degrees of freedom), the overall effect is that the paired confidence interval is narrower
than the unpaired confidence interval. This result holds for most cases of paired and unpaired
studies, though not always.

ò Question

You are convinced that a different impeller (mixing blade) shape for your tank will lead to faster,
i.e. shorter, mixing times. The choices are either an axial blade or a radial blade, as shown in this
figure from Wikipedia55 .

Before obtaining approval to run some experiments, your team wants you to explain how you will
interpret the experimental data. Your reply is that you will calculate the average mixing time from
each blade type and then calculate a confidence interval for the difference. A team member asks
you what the following 95% confidence intervals would mean:

1. −453 seconds ≤ 𝜇Axial − 𝜇Radial ≤ 390 seconds

2. −21 seconds ≤ 𝜇Axial − 𝜇Radial ≤ 187 seconds

For both cases (a) explain what the confidence interval means in the context of this experiment, and
(b) whether the recommendation would be to use radial or axial impellers to get the shortest
mixing time.

132 Chapter 2. Univariate Data Analysis

 Release 547b16

3. Now assume the result from your experimental test was

−21 seconds ≤ 𝜇Axial − 𝜇Radial ≤ 187 seconds; how can you make the confidence interval narrower?

ò Solution

1. This confidence interval spans zero, and nearly symmetrically. This implies the population
difference is likely zero, while the symmetry implies their is no preference either way: the
difference in mixing times is as low as -453 seconds or as high as 390 seconds. The
recommendation is that either the axial or radial impeller could be used, with no expected
long-term difference. Use the cheaper impeller; or use the axial impeller if the costs are the same
(only because of the very slight imbalance in the CI). Note that there is a 5% chance that the
confidence interval does not contain the true difference.

2. This confidence interval also spans zero, so there is no statistical difference between the two
impellers. However the CI does not span zero symmetrically. The asymmetry of the interval
makes me much less comfortable recommending that there is no practical difference between
the impellers. It often happens in these cases that by removing a single data point that the
confidence interval does not span zero anymore. In this case I would recommend either
impeller, but if there is no cost difference, I would prefer the radial impeller, as it might have
shorter mixing times, especially if the confidence interval quoted here is only due to one
observation. A careful review of the raw data would be useful in this case.

3. The confidence interval can be made narrower in 2 ways (as long as the sample mean and
sample standard deviation remain stable):

• Use more data points, 𝑛 in both groups.

• Choose a lower degree of confidence, e.g. 90% instead of 95%, which is really just an artificial
reduction of the interval.

One can also reduce the interval by shrinking the standard deviation, but that’s usually not a
practical possibility. You cannot perform a paired test, as you only have one mixing tank.

ò Interpreting confidence intervals

Recall the definition of the confidence interval is subtle: it says 95% of the time, the upper and
lower bounds of the confidence interval contain the true value of the parameter; it does not say
there is a 95% probability the true value of the parameter lies inside the bounds. That last part
is incorrect because it implies the true value of the parameter can vary, which it can’t: the true
parameter value is fixed, only the bounds change.

ò Question

The paper by PJ Rousseeuw, “Tutorial to Robust Statistics56 ”, Journal of Chemometrics, 5, 1-20, 1991
discusses the breakdown point of a statistic.

1. Describe what the breakdown point is, and give two examples: one with a low breakdown point,
and one with a high breakdown point. Use a vector of numbers to help illustrate your answer.

55 [Link]

2.16. Exercises 133

Process Improvement Using Data

2. What is an advantage of using robust methods over their “classical” counterparts?

ò Solution

1. PJ Rousseeuw defines the breakdown point on page 3 of his paper as “. . . the smallest fraction of
the observations that have to be replaced to make the estimator unbounded. In this definition
one can choose which observations are replaced, as well as the magnitude of the outliers, in the
least favourable way”.

A statistic with a low breakdown point is the mean, of the 𝑛 values used to calculate the mean,
only 1 needs to be replaced to make the estimator unbounded; i.e. its breakdown point is 1/𝑛.
The median though has a breakdown point of 50%, as one would have to replace 50% of the 𝑛
data points in the vector before the estimator becomes unbounded.

Use this vector of data as an example: [2, 6, 1, 9151616, −4, 2]. The mean is 1525270, while the
median is 2.

2. • Robust methods are insensitive to outliers, which is useful when we need a measure of
location or spread that is calculated in an automated way. It is increasingly prevalent to skip
out the “human” step that might have detected the outlier, but our data sets are getting so
large that we can’t possibly visualize or look for outliers manually anymore.

• As described in the above paper by Rousseeuw, robust methods also emphasize outliers.
Their “lack of sensitivity to outliers” can also be considered an advantage.

ò Question

1. Why are robust statistics, such as the median or MAD, important in the analysis of modern data
sets? Explain, using an example, if necessary.

2. What is meant by the break-down point of a robust statistic? Give an example to explain your
answer.

ò Solution

1. Data sets you will have to deal with in the workplace are getting larger and larger (lengthwise),
and processing them by trimming outliers (see Question 5 later) manually is almost impossible.
Robust statistics are a way to summarize such data sets without point-by-point investigation.

This is especially true for automatic systems that you will build that need to (a) acquire and (b)
process the data to then (c) produce meaningful output. These systems have to be capable of
dealing with outliers and missing values.

2. The breakdown point is the number of contaminating data points required before a statistic
(estimator) becomes unbounded, i.e. useless. For example, the mean requires only 1
contaminating value, while the median requires 50% + 1 data points before it becomes useless.

Consider the sequence [2, 6, 1, 91511, −4, 2]. The mean is 15253, while the median is 2, which is a

56 [Link]

134 Chapter 2. Univariate Data Analysis

 Release 547b16

far more useful estimate of the central tendency in the data.

ò Question
1 1
{︀ }︀
𝑥 and 𝒱 {𝑥} = ℰ (𝑥 − 𝜇)2 = 𝜎 2 = (𝑥 − 𝜇)2 .
∑︀ ∑︀
Recall that 𝜇 = ℰ(𝑥) = 𝑁 𝑁

1. What is the expected value thrown of a fair, 12-sided dice?

2. What is the expected variance of a fair, 12-sided dice?

3. Simulate 10,000 throws in a software package (R, MATLAB, or Python) from this dice and see if
your answers match those above. Record the average value from the 10,000 throws, call that
average 𝑥.

4. Repeat the simulation 10 times, calculating the average value of all the dice throws. Calculate
the mean and standard deviation of the 10 𝑥 values and comment whether the results match the
theoretically expected values.

ò Solution

The objective of this question is to recall basic probability rules.

1. Each value on the dice is equally probable, so the expected value thrown will be:

12 12
∑︁ ∑︁ 1
ℰ(𝑋) = 𝑥𝑖 𝑃 (𝑥𝑖 ) = 𝑃 (𝑥) 𝑥𝑖 = (1 + 2 + · · · + 12) = 6.5
𝑖=1 𝑖=1
12
This value is the population mean, 𝜇.

2. Continuing the notation from the above question we can derive the expected variance as,

12
1 ∑︁ 1 [︀
(𝑥𝑖 − 𝜇)2 = · (1 − 6.5)2 + (2 − 6.5)2 + . . . + (12 − 6.5)2 ≈ 11.9167
]︀
𝒱(𝑋) =
𝑁 𝑖 12

3. Simulating 10,000 throws corresponds to 10,000 independent and mutually exclusive random
events, each with an outcome between 1 and 12. The sample mean and variance from my
sample was calculated using this code in R:
𝑥 = 6.5219
𝑠2 = 12.03732

# Set the random seed to a known point, to allow

# us to duplicate pseudorandom results
[Link](13)

[Link] <- [Link](runif(10000, 1, 13))

# Verify that it is roughly uniformly distributed

# across 12 bins
hist([Link], breaks=seq(0,12))

[Link] <- mean([Link])

[Link] <- var([Link])
c([Link], [Link])

2.16. Exercises 135

Process Improvement Using Data

4. Repeating the above simulation 10 times (i.e. 10 independent experiments) produces 10 different
estimates of 𝜇 and 𝜎 2 . Note, your answer should be slightly different, and different each time
you run the simulation.

N <- 10
n <- 10000
[Link] <- numeric(N)
[Link] <- numeric(N)
for (i in 1:N) {
[Link] <- [Link](runif(n, 1, 13))
[Link][i] <- mean([Link])
[Link][i] <- var([Link])
}

[Link]
# [1] 6.5527 6.4148 6.4759 6.4967 6.4465
# [6] 6.5062 6.5171 6.4671 6.5715 6.5485

[Link]
# [1] 11.86561 11.84353 12.00102 11.89658 11.82552
# [6] 11.83147 11.95224 11.88555 11.81589 11.73869

# You should run the code several times and verify whether
# the following values are around their expected, theoretical
# levels. Some runs should be above, and other runs below
# the theoretical values.
# This is the same as increasing "N" in the first line.

# Is it around 6.5?
mean([Link])

# Is it around 11.9167?
mean([Link])

# Is it around \sigma^2 / n = 11.9167/10000 = 0.00119167 ?

var([Link])

(︀ )︀
Note that each 𝑥 ∼ 𝒩 𝜇, 𝜎 2 /𝑛 , where 𝑛 = 10000. We know what 𝜎 2 is in this case: it is our
theoretical value of 11.92, calculated earlier, and for 𝑛 = 10000 samples, our theoretical
expectation is that 𝑥 ∼ 𝒩 (6.5, 0.00119167).

Calculating the average of those 10 means, let’s call that 𝑥, shows a value close to 6.5, the
theoretical mean.

Calculating the variance of those 10 means shows a number around 0.00119167, as expected.

ò Question

Removed. Was a duplicate of a prior question (number 13).

ò Question

1. At the 95% confidence level, for a sample size of 7, compare and comment on the upper and
lower bounds of the confidence interval that you would calculate if:

a) you know the population standard deviation

136 Chapter 2. Univariate Data Analysis

 Release 547b16

b) you have to estimate it for the sample.

Assume that the calculated standard deviation from the sample, 𝑠 matches the population
𝜎 = 4.19.

2. As a follow up, overlay the probability distribution curves for the normal and 𝑡-distribution that
you would use for a sample of data of size 𝑛 = 7.

3. Repeat part of this question, using larger sample sizes. At which point does the difference
between the 𝑡- and normal distributions become practically indistinguishable?

4. What is the implication of this?

ò Solution

1. This question aims for you to prove to yourself that the 𝑡-distribution is wider (more broad)
than the normal distribution, and as a result, the confidence interval is wider as well. This is
because we are less certain of the data’s spread when using the estimated variance.

The confidence intervals are:

𝑥−𝜇
−𝑐𝑛 ≤ √ ≤ 𝑐𝑛
𝜎/ 𝑛

𝑥−𝜇
−𝑐𝑡 ≤ √ ≤ 𝑐𝑡
𝑠/ 𝑛
The 95% region spanned by the 𝑡-distribution with 6 degrees of freedom has upper and lower
limits at 𝑐𝑡 = ± qt((1-0.95)/2, df=6), i.e. from -2.45 to 2.45. The equivalent 95% region
spanned by the normal distribution is 𝑐𝑛 = ± qnorm((1-0.95)/2), spanning from z=-1.96 to
z=1.96. Everything else in the center of the 2 inequalities is the same, so we only need to
compare 𝑐𝑡 and 𝑐𝑛 .

2. The question asked to overlay the probability distributions (not cumulative probability
distributions):

where the above figure was generated with the R-code:

n <- 7
z <- seq(-5, 5, 0.01)
[Link] <- dnorm(z)
prob.t <- dt(z, df=n-1)

bitmap('[Link]', res=300, width=7, height=4)

plot(z, [Link], type="l", ylab="Normal and t-distributions", lwd=2)
lines(z, prob.t, lty=8, lwd=2) # dashed line
legend(x=1.35, y=0.25, legend=c("Normal distribution", "t-distribtion"),
lty=c(1,8), lwd=c(2,2))
[Link]()
Exercises 137
Process Improvement Using Data

3. Repeated use of the above code, but changing 𝑛, shows that little practical difference between the
distributions with as few as 𝑛 = 20 samples. After 𝑛 = 40 and especially 𝑛 = 60, there is almost
no theoretical difference between them.

4. This implies that when we do any analysis of large samples of data, say 𝑛 > 50, and if those data
are independently sampled, then we can just use the normal distribution’s critical value (e.g. the
±1.96 value for 95% confidence, which you now know from memory), instead of looking up the
𝑡-distribution’s values.

Since the wider values from the 𝑡-distribution reflect our uncertainty in using an estimate of the
variance, rather than the population variance, this result indicates that our estimated variances
are a good estimate of the population variance for largish sample sizes.

ò Question

Engineering data often violate the assumption of independence. In this question you will create
(simulate) sequences of autocorrelated data, i.e. data that lack independence, and investigate how
lack of independence affects our results.

The simplest form of autocorrelation is what is called lag-1 autocorrelation, when the series of
values, 𝑥𝑘 is correlated with itself only 1 step back in time, 𝑥𝑘−1 :

𝑥𝑘 = 𝜑𝑥𝑘−1 + 𝑎𝑘
(︀ )︀
The 𝑎𝑘 value is a random error and for this question let 𝑎𝑘 ∼ 𝒩 𝜇 = 0, 𝜎 2 = 25.0 .

Create 3 sequences of autocorrelated data with:

A: 𝜑 = +0.7 (positively correlated)

B: 𝜑 = 0.0 (uncorrelated data)

C: 𝜑 = −0.6 (negatively correlated)

For case A, B and C perform the following analysis. Repeat the following 1000 times (let
𝑖 = 1, 2, . . . , 1000):

• Create a vector of 100 autocorrelated 𝑥 values using the above formula, using the current level of
𝜑

• Calculate the mean of these 100 values, call it 𝑥𝑖 and store the result

At this point you have 1000 𝑥𝑖 values for case A, another 1000 𝑥𝑖 values for case B, and similarly for
case C. Now answer these questions:

1. Assuming independence, which is obviously not correct for 2 of the 3 cases, nevertheless, from
which population should 𝑥 be from, and what are the 2 parameters of that population?

2. Now, using your 1000 simulated means, estimate those two population parameters.

3. Compare your estimates to the theoretical values.

Comment on the results, and the implication of this regarding tests of significance (i.e. statistical
tests to see if a significant change occurred or not).

138 Chapter 2. Univariate Data Analysis

 Release 547b16

ò Solution

We expect that case B should match the theoretical case the closest, since data from case B are truly
independent, since the autocorrelation parameter is zero. We expect case A and C datasets, which
violate that assumption of independence, to be biased one way or another. This question aims to
see how they are biased.

nsim <- 1000 # Number of simulations

[Link] <- numeric(nsim) # An empty vector to store the results

[Link](37) # so that you can reproduce these results

for (i in 1:nsim)
{
N <- 100 # number of points in autocorrelated sequence
phi <- +0.7 # ** change this line for case A, B and C **
spread <- 5.0 # standard deviation of random variables
x <- numeric(N)
x[1] = rnorm(1, mean=0, sd=spread)
for (k in 2:N){
x[k] <- phi*x[k-1] + rnorm(1, mean=0, sd=spread)
}
[Link][i] <- mean(x)
}
theoretical <- sqrt(spread^2/N)

# Show some output to the user

c(theoretical, mean([Link]), sd([Link]))

You should be able to reproduce the results I have below, because the above code uses the
[Link](...) function, which forces R to generate random numbers in the same order on my
computer as yours (as long as we all use the same version of R).

• Case A: 0.50000000, 0.00428291, 1.65963302

• Case B: 0.50000000, 0.001565456, 0.509676562

• Case C: 0.50000000, 0.0004381761, 0.3217627596

The first output is the same for all 3 cases: this is the theoretical standard deviation of the
(︀ )︀
distribution from which the 𝑥𝑖 values come: 𝑥𝑖 ∼ 𝒩 𝜇, 𝜎 2 /𝑁 , where 𝑁 = 100, the number of
points in the autocorrelated sequence. This result comes from the central limit theorem, which tells
us that 𝑥𝑖 should be normally distributed, with the same mean as our individual 𝑥-values, but have
smaller variance. That variance is 𝜎 2 /𝑁 , where 𝜎 is the variance of the distribution from which we
took the raw 𝑥 values. That theoretical variance value is 25/100, or theoretical standard deviation
√︀
of 25/100 = 0.5.

But, the central limit theorem only has one crucial assumption: that those raw 𝑥 values are
independent. We intentionally violated this assumption for case A and C.

We use the 1000 simulated values of 𝑥𝑖 and calculate the average of the 1000 𝑥𝑖 values and the
standard deviation of the 1000 𝑥𝑖 values. Those are the second and third values reported above.

We see in all cases that the mean of the 1000 values nearly matches 0.0. If you run the simulations
again, with a different seed, you will see it above zero, and sometimes below zero for all 3 cases. So
we can conclude that lack of independence does not affect the estimated mean.

The major disagreement is in the variance though. Case B matches the theoretical variance; data
that are positively correlated have an inflated standard deviation, 1.66; data that are negatively
correlated have a deflated standard deviation, 0.32 when 𝜑 = −0.6.

2.16. Exercises 139

Process Improvement Using Data

This is problematic for the following reason. When doing a test of significance, we construct a
confidence interval:
𝑥−𝜇
−𝑐𝑡 ≤ √ ≤ +𝑐𝑡
𝑠/ 𝑛
𝑠 𝑠
𝑥 − 𝑐𝑡 √ ≤ 𝜇 ≤ 𝑥 + 𝑐𝑡 √
𝑛 𝑛
LB ≤ 𝜇 ≤ UB

We use an estimated standard deviation, 𝑠, whether that is found from pooling the variances or
found separately (it doesn’t really matter), but the main problem is that 𝑠 is not accurate when the
data are not independent:

• For positive correlations (quite common in industrial data): our confidence interval will be too
wide, likely spanning zero, indicating no statistical difference, when in fact there might be one.

• For negative correlations (less common, but still seen in practice): our confidence interval will be
too narrow, more likely to indicate there is a difference.

The main purpose of this question is for you to see how use to understand what happens when a
key assumption is violated. There are cases when an assumption is violated, but it doesn’t affect
the result too much.

In this particular example there is a known theoretical relationship between 𝜑 and the
inflated/deflated variance that can be derived (with some difficulty). But in most situations the
affect of violating assumptions is too difficult to derive mathematically, so we use computer power
to do the work for us: but then we still have to spend time thinking and interpreting the results.

ò Question

Sulphur dioxide is a byproduct from ore smelting, coal-fired power stations, and other sources.

These 11 samples of sulphur dioxide, SO2 , measured in parts per billion [ppb], were taken from our
plant. Environmental regulations require us to report the 90% confidence interval for the mean SO2
value.

180, 340, 220, 410, 101, 89, 210, 99, 128, 113, 111

1. What is the confidence interval that must be reported, given that the sample average of these 11
points is 181.9 ppb and the sample standard deviation is 106.8 ppb?

2. Why might Environment Canada require you to report the confidence interval instead of the
mean?

ò Solution

1. From the central limit theorem, assuming the 11 values are independent, the mean SO2 value,
{︀ }︀
𝑥 ∼ 𝒩 𝜇, 𝜎 2 /𝑛 , where 𝜇 and 𝜎 are the distribution from which the raw values come.

Using an estimate for 𝜎 = 𝑠^ = 106.8 we can construct the 𝑧-value and confidence interval. 𝑧 will
be 𝑡-distributed with 𝑛 − 1 = 10 degrees of freedom, so 𝑐𝑡 = 1.81 At the 90% confidence level we

140 Chapter 2. Univariate Data Analysis

 Release 547b16

can then write:

𝑥−𝜇
−𝑐𝑡 ≤ √ ≤ +𝑐𝑡
𝑠/ 𝑛
𝑠 𝑠
𝑥 − 𝑐𝑡 √ ≤ 𝜇 ≤ 𝑥 + 𝑐𝑡 √
𝑛 𝑛
181.9 − 1.81 × 106.8
√
11
≤ 𝜇 ≤ 181.9 + 1.81 × 106.8
√
11
123.6 ppb ≤ 𝜇 ≤ 240.2 ppb

2. Environment Canada may require the confidence interval since in addition to providing an
estimate of the mean (just the midpoint of the CI), it also provides an estimate of the spread –
variability in your process – if 𝑛 is known, without requiring access to the raw data.

A wide CI gives an indication that you might in fact be polluting too much on some days, and
compensating on others, which is not desirable. The confidence interval’s width can also be
compared between plants to find the most variable polluters.

ò Question

A concrete slump test is used to test for the fluidity, or workability, of concrete. It’s a crude, but
quick test often used to measure the effect of polymer additives that are mixed with the concrete to
improve workability.

The concrete mixture is prepared with a polymer additive. The mixture is placed in a mold and
filled to the top. The mold is inverted and removed. The height of the mold minus the height of the
remaining concrete pile is called the “slump”, as shown in this figure from Wikipedia57 .

Your company provides the polymer additive, and you are developing an improved polymer
formulation, call it B, that hopefully provides the same slump values as your existing polymer, call
it A. Formulation B costs less money than A, but you don’t want to upset, or lose, customers by
varying the slump value too much.

1. You have a single day to run your tests (experiments). Preparation, mixing times, measurement
and clean up take 1 hour, only allowing you to run 10 experiments. Describe all precautions, and
why you take these precautions, when planning and executing your experiment. Be very
specific in your answer (use bullet points).

2. The following slump values were recorded over the course of the day:

2.16. Exercises 141

Process Improvement Using Data

Additive Slump value [cm]

A 5.2
A 3.3
B 5.8
A 4.6
B 6.3
A 5.8
A 4.1
B 6.0
B 5.5
B 4.5

What is your conclusion on the performance of the new polymer formulation (system B)? Your
conclusion must either be “send the polymer engineers back to the lab” or “let’s start making
formulation B for our customers”. Explain your choice clearly.

To help you, 𝑥𝐴 = 4.6 and 𝑠𝐴 = 0.97. For system B: 𝑥𝐵 = 5.62 and 𝑠𝐵 = 0.69.

Note: In your answer you must be clear on which assumptions you are using and, where
necessary, why you need to make those assumptions.

3. Describe the circumstances under which you would rather use a paired test for differences
between polymer A and B.

4. What are the advantage(s) of the paired test over the unpaired test?

5. Clearly explain which assumptions are used for paired tests, and why they are likely to be true
in this case?

6. The slump tests were actually performed in a paired manner, where pairing was performed
based on the cement supplier. Five different cement suppliers were used:

Supplier Slump value [cm] from A Slump value [cm] from B

1 5.2 5.8
2 3.3 4.5
3 4.6 6.0
4 5.8 5.5
5 4.1 6.2

Use these data, and provide, if necessary, an updated recommendation to your manager.

ò Solution

1. The basic rule is to control what you can and randomize against what you cannot. You should
have mentioned some of these items:

• Control: clean equipment thoroughly between runs.

57 [Link]

142 Chapter 2. Univariate Data Analysis

 Release 547b16

• Control: other factors that might affect the slump: temperature, humidity.

• Control: ensure the same person prepares all mixtures, or randomize the allocation of people
if you have to use more than 1 person. Don’t let person 1 prepare all the A mixtures and
person 2 the B mixtures.

• Control: mixing times and how the mixture is created could have an effect. This should
ideally be done by the same person.

• Randomize the order of all the A and B experiments: don’t run all the A’s, then all the B’s, as
that will confound with other factors. For example, even though temperature might vary
during the day, if we randomize the run order, then we prevent temperature from affecting
the results.

• Use raw materials (cement, binder, other ingredients) from all possible suppliers. And the
supplier raw materials should be representative.

2. We will initially assume that 𝜇𝐴 = 𝜇𝐵 , in other words, the outcome is “let’s start making
formulation B for our customers”. We will construct a confidence interval for the difference,
𝜇𝐵 − 𝜇𝐴 and interpret that CI.

• Assume the slump values within each group are independent, which will be true if we take
the precautions above. We do this because then we can use the central limit theorem (CLT) to
(︀ 2
)︀ (︀ 2
)︀
state 𝑥𝐴 ∼ 𝒩 𝜇𝐴 , 𝜎𝐴 /𝑛𝐴 and that 𝑥𝐵 ∼ 𝒩 𝜇𝐵 , 𝜎𝐵 /𝑛𝐵 .

• Note: we don’t require the samples within each group to be normally distributed.
2 2
• Assume the variances are the same: 𝜎𝐴 = 𝜎𝐵 = 𝜎 2 : this is required to simplify the next step.

• Assume the 𝑥𝐴 and 𝑥𝐵 means are independent. This allows us to calculate a variance value,
𝒱 {𝑥𝐵 − 𝑥𝐴 } from which we can create a 𝑧-value for 𝜇𝐵 − 𝜇𝐴 :

(𝑥𝐵 − 𝑥𝐴 ) − (𝜇𝐵 − 𝜇𝐴 )
𝑧= √︀
𝒱 {𝑥𝐵 − 𝑥𝐴 }

That denominator variance can be written as:

𝒱 {𝑥𝐵 − 𝑥𝐴 } = 𝒱 {𝑥𝐵 } + 𝒱 {𝑥𝐴 }

(︂ )︂
1 1
= 𝜎2 +
𝑛𝐵 𝑛𝐴

using our previous assumption that the variances are equal. We can verify this with an 𝐹 -test,
but won’t do it here.

Because we do not have an external estimate of the variance, 𝜎 2 , available, we must assume a
good estimate for it can be found by pooling the estimated variances of the group A and B
samples (which requires our equal variance assumption from earlier).

4𝑠2𝐴 + 4𝑠2𝐵
𝑠2𝑃 =
4+4
2
4(0.97) + 4(0.69)2
𝑠2𝑃 = = 0.709
4+4

This pooling also gives us 8 degrees of freedom for the 𝑡-distribution, which is how the
𝑧-value is distributed.

2.16. Exercises 143

Process Improvement Using Data

Using that 𝑧-value and filling our assumed difference of zero for the true means, we can
construct a 95% confidence interval:

√︂ (︁ −𝑐𝑡 ≤ 𝑧 ≤ +𝑐𝑡 √︂
)︁ (︁ )︁
(𝑥𝐵 − 𝑥𝐴 ) − 𝑐𝑡 𝑠2𝑃 𝑛1𝐵 + 𝑛1𝐴 ≤ 𝜇𝐵 − 𝜇𝐴 ≤ (𝑥𝐵 − 𝑥𝐴 ) + 𝑐𝑡 𝑠2𝑃 𝑛1𝐵 + 1
𝑛𝐴
√︁ √︁
1.02 − 2.3 0.709 15 + 15 1.02 + 2.3 0.709 51 + 15
(︀ )︀ (︀ )︀
≤ 𝜇𝐵 − 𝜇𝐴 ≤
−0.21 ≤ 𝜇𝐵 − 𝜇𝐴 ≤ 2.2

The statistical conclusion is that there is no difference between formulation A and B, since
the CI spans zero. However, the practical interpretation is that the CI only just contains zero,
and this should cause us to stop, and really consider the risk of the statistical conclusion.

If one of the data points were in error just slightly, or if we ran a single additional experiment,
it is quite possible the CI will not span zero anymore. In my mind, this risk is too great, and we
risk upsetting the customers.

So my conclusion would be to “send the polymer engineers back to the lab” and have them
improve their formulation until that CI spans zero more symmetrically.

3. A paired test should be used when there is something is common within pairs of samples in
group A and B, but that commonality does not extend between the pairs. Some examples
though you could have mentioned:

Pairing is appropriate: person 1 mixes polymer for test A and B; person 2 mixes polymer for test
A and B (but with different time and agitation level that person 2); person 3 mixes . . . etc Pairing
not appropriate: person 1 mixes all the polymer A samples; person 2 mixes all the polymer B
samples (pairing won’t fix this, and even the unpaired results will be inaccurate - see
precautions mentioned above). Pairing appropriate: you only have enough cement and raw
materials to create the concrete mixture for 2 samples: one for A and one for B. You repeat this 5
times, each time using a different supplier’s raw materials.

In other words, pairing is appropriate when there is something the prevents the 𝑥𝐴 and 𝑥𝐵
quantities from being independent.

4. The one advantage of the paired test is that it will cancel out any effect that is common between
the pairs (whether that effect actually affects the slump value or not). Pairing is a way to guard
against potential effect.

This makes the test more sensitive to the difference actually being tested for (formulation A vs B)
and prevents confounding from the effect we are not testing for (suppliers’ raw material).

Unpaired tests, but with randomization will only prevent us from being misled, however that
supplier effect is still present in the 10 experimental values. The 5 difference values used in the
paired tests will be free from that effect.

5. Pairing requires/assumes that the paired objects have something in common (e.g. a common
bias due to the cement raw material). This common bias will be cancelled out once we calculate
the difference in measurements.

• The difference values calculated, 𝑤𝑖 , are assumed to be independent. This is likely true in this
case because each raw material supplier is different (unrelated) to the other.

• If the differences are independent, then the central limit theorem can be safely assumed so
(︀ 2
)︀
that the average of these differences, 𝑤 ∼ 𝒩 𝜇𝑤 , 𝜎𝑤 /𝑛 .

144 Chapter 2. Univariate Data Analysis

 Release 547b16

6. The 5 difference values are 𝑤𝑖 = [0.6, 1.2, 1.4, −0.3, 2.1] and the average difference value is
𝑤 = 1 and its estimated variance is 𝑠2𝑤 = 0.815.

Create the 𝑧-value against the 𝑡-distribution with 4 degrees of freedom (𝑐𝑡 = 2.78), at the 95%
confidence level, and unpack it into a confidence interval.

−𝑐 ≤ 𝑧 ≤ +𝑐𝑡
√︁ 𝑡 √︁
𝑠2 𝑠2
𝑤 − 𝑐𝑡 𝑛 ≤ 𝜇𝑤 ≤ 𝑤 + 𝑐𝑡
√︁ √︁𝑛
1 − 2.78 0.815
4 ≤ 𝜇𝑤 ≤ 1 + 2.78 0.815
4
−0.12 ≤ 𝜇𝑤 ≤ 2.12

The interpretation is that the true difference in slump, 𝜇𝑤 , when accounting for variation from
the cement raw material, is again not statistically significant, at the 95% confidence level.

Practically though, there is a bit of a risk, due to the imbalance (asymmetry) in the confidence
interval. It would be reluctant to hinge my company’s profitability on this result, especially with
the fact that there are only 4 experiments. So my personal conclusion would be to still “send the
polymer engineers back to the lab”.

ò Question

You are planning a series of experiments to test alternative conditions in a store and see which
conditions lead to higher sales.

Which practical steps would you take to ensure independence in the experimental data, when
investigating:

1. adjustable halogen lighting: A = soft and dim lighting and B = brighter lighting

2. alternative shelving: A = solid white metal shelves and B = commercial stainless steel racking

ò Solution

By Cameron DiPietro and Andrew Haines (2012 class)

Randomization is expensive and inconvenient; however, the high cost is to ensure that the results
attained in each study are not affected by unmeasured disturbances. We also have to take care to
control measured disturbances as far as possible.

1. To ensure independence when investigating adjustable halogen lighting: A = soft and dim
lighting and B = brighter lighting, the following experiments and conditions may be run:

• All light fixtures are changed correctly during the swap from A to B and the same scenario
from B to A

• Keep prices of all products the same during days with A lighting and days with B lighting

• Do not inform customers of A to B swap or B to A swap in lighting

• Ensure product quality

• Use the same amount of voltage throughout the store for each lighting arrangement

2.16. Exercises 145

Process Improvement Using Data

• Keep the store stocked the same for everyday during experiment

• Use random days for each light fixture

• Maintain the same advertisements for the store during the study

• Do not inform employees of lighting swaps to ensure identical employee to customer

relationships

• Compensate for any holiday or unexpected short days of store hours

• Have employees work randomized shifts to ensure no patterns in employees moods during
light fixture swaps

• Employees have the same mindset to customers (if a retail business) during both A and B
lighting arrangements

• Assume all data from A and B light fixtures have identical population variance

If lighting A and B are installed simultaneously, then it might be possible to even run different
tests during the day, randomly allocated.

2. To ensure independence when investigating alternative shelving: A = solid white metal shelves
and B = commercial stainless steel racking, the following experiments and conditions may be
run:

• Shelving size remains the same and in the same location

• Identical product placement on both shelves A and B, if possible

• Being able to control everything other than the variable being studied of shelves

• Distances between shelves identical

• Ensure employees have the same mindset during each customer visit

• Identical number of items per shelf

• Same shelf distances from checkout

• Clean each shelf in the same manner for both A and B

• Keep prices and sales the same throughout the study period

Clearly the shelf study cannot be easily implemented, since the logistics of unstocking, removing
shelf A, replacing with shelf B and restocking them is extremely costly.

One thing to consider in such cases is to run the experiments in two separate stores that are as
similar as possible in all other respects (e.g. built in the area with similar profiles of customers,
similar store layout, etc.).

ò Question

This question gives you exposure to analyzing a larger data set than seen in the preceding
questions.

Your manager has asked you to describe the flow rate characteristics of the overhead stream
leaving the top of the distillation column58 at your plant. You are able to download one month of

146 Chapter 2. Univariate Data Analysis

 Release 547b16

data, available from this website59 , from 1 March to 31 March, taken at one minute intervals to
answer this question.

58 [Link]
59 [Link]

2.16. Exercises 147

Process Improvement Using Data

148 Chapter 2. Univariate Data Analysis

 CHAPTER 3
PROCESS MONITORING

3.1 Process monitoring in context

In the first section we learned about visualizing data (page 1), then we moved on to reviewing univariate
statistics (page 37). This section now combines both topics, showing how to create a system that
monitors a single, univariate, value from any process. These monitoring systems are easily
implemented online, and generate great value for companies that use them in day-to-day production.
This is one of their greatest advantages: almost no training is required to interpret the visualization
and secondly the human eye can quickly pick up any patters or trends in the plots; both expected and
unexpected patterns.

Monitoring charts are a graphical tool, enabling anyone to rapidly detect a problem by visual analysis.
The next logical step after detection of a problem is to diagnose it, but we will cover diagnosis in the
section on latent variable models (page 411).

This section is the last section where we deal with univariate data; after this section we start to use and
deal with 2 or more variables.

3.1.1 Usage examples

Video for
this section The material in this section is used whenever you need to rapidly detect problems. It has tangible
application in many areas - in fact, you have likely encountered these monitoring charts in areas such
as a hospital (monitoring a patient’s heart beat), stock market charts (for intraday trading), or in a
processing/manufacturing facility (control room computer screens).

• Co-worker: We need a system to ensure an important dimension on our product is stable and
consistent over the entire shift.

• Yourself : We know that as the position of a manufacturing robot moves out of alignment that our
product starts becoming inconsistent; more variable. How can we quickly detect this slow drift in
alignment and predict when to stop the process and perform preventative maintenance?

• Manager: the hourly average profit, and process throughput is important to the head-office; can we
create a system for them to track that?

• Potential customer: what is your process capability - we are looking for a new supplier that can
provide a low-variability raw material for us with Cpk of at least 1.6, preferably higher.

149
Process Improvement Using Data

Note: process monitoring is mostly reactive and not proactive. So it is suited to incremental process
improvement, which is typical of most improvements. However, using the monitoring charts to make
proactive changes to avoid a bigger problem later in time is certainly possible by adding additional
rules and calculations to the plots. For example, rules to forecast a few steps ahead, with prediction
intervals, can be easily added.

We point out in the next section (page 151) that process monitoring is not a feedback control system. So
that section should be read in the context of thinking reactively and proactively (in a feed forward
anticipatory manner).

3.1.2 What we will cover

We will consider 3 main charts after introducing some basic concepts: Shewhart charts, CUSUM charts
and (exponentially weighted moving average) charts. The EWMA chart has an adjustable parameter
that captures the behaviour of a Shewhart chart at one extreme and a CUSUM chart at the other
extreme, or a combination of both is possible by settings this parameter on a sliding scale.

3.1.3 Concepts
Concepts and acronyms that you must be familiar with by the end of this section:

• Shewhart chart, CUSUM chart and EWMA chart

• Phase 1 and phase 2 when building a monitoring system

• False alarms

• Type 1 and type 2 errors

• LCL and UCL

• Target

• Cp and Cpk

• Outliers

• Real-time implementation of monitoring systems

3.2 References and readings

1. Recommended: Box, Hunter and Hunter, Statistics for Experimenters, Chapter 14 (2nd edition)

2. Recommended: Montgomery and Runger, Applied Statistics and Probability for Engineers.

3. Hunter, J.S. “The Exponentially Weighted Moving Average60 ”, Journal of Quality Technology, 18 (4) p
203 - 210, 1986.

4. MacGregor, J.F. “Using On-Line Process Data to Improve Quality: Challenges for Statisticians61 ”,
International Statistical Review, 65, p 309-323, 1997.
60 [Link]
61 [Link]

150 Chapter 3. Process Monitoring

 Release 547b16

3.3 What is process monitoring about?

Most industries have now realized that product quality is not an option. There was historical thinking
that quality is the equivalent of “gold-plating” your product, but that has mostly fallen away. Product
quality is not always a cost-benefit trade-off: it is beneficial to you in the long-term to improve your
product quality, and for your customers as well.

As we spoke about in the univariate review section (page 37), good quality products (low variability)
actually boost your profits by lowering costs in the long term. You have lower costs when you do not
have to scrap off-specification product, or have to rework bad product. You have increased long-term
sales with more loyal customers and improved brand reputation as a reliable and consistent supplier.

An example that most people in North America can relate to is the rise in Asian car manufacturers’
market share, at the expense American manufacturers’ market share. The market has the perception
that Asian cars are more reliable than American cars and resale rates certainly reflect that. The
perception has started to change since 2010, as North American manufacturers have become more
quality conscious. That is an illustration of how lack of variability in your product can benefit you.

In order to achieve this high level of final product quality, our systems should be producing low
variability product at every step of the manufacturing process. Rather than wait till the end of the
process to discover poor quality product, we should be monitoring, in real-time, the purchased raw
materials and also the intermediate steps in our process. When we discover unusual variability the
lofty aim is to make (permanent) process adjustments to avoid that variability from ever occurring
again.

Notice here that process monitoring is not intended to be automatic feedback control. It has the same
principles of quantifying unusual operation (errors) and reacting to them in some way, but the
intention with process monitoring is:

• that any process adjustments are infrequent [not frequently on a set cycle, as feedback control does],

• these adjustments are made manually [not automatically with actuators],

• and take place due to special causes [not due to regularly occurring process disturbances].

As seen by the items in square brackets above, automatic feedback control is applied continuously by
computer systems and makes short-term, temporary changes to the system to keep it at the desired
target (called the setpoint) in the face of process disturbances. Process monitoring is very different
therefore to feedback control.

Note that process monitoring is often called statistical process control (SPC). This can lead to
unnecessary confusion with process control, i.e. the design and implementation of feedback control,
feed-forward control and other automated control systems. We will not use the term SPC, rather we
will use the term process monitoring.

3.3.1 Monitoring charts

We use monitoring charts, also called control charts, to display and detect this unusual variability. A
monitoring chart is a display of one value (variable), against time, or in sequence order. These
time-based plots also show some additional information: usually a target value, and one or more
limits lines are superimposed on the plot. The plots are most useful when displayed in real-time, or
close to real-time. There are various technical ways to express what a monitoring chart does exactly,
but a general definition is that a monitoring chart helps you detect outliers and other unusual
time-based behaviour.

The key points are that a monitoring chart:

3.3. What is process monitoring about? 151

Process Improvement Using Data

• is most often a time-series plot, or some sort of sequence plot,

• a target value (center line) may be shown,

• one or more limit lines are shown, such as the UCL (upper control limit) or LCL (lower control
limit),

• they are displayed and updated in real-time, or as close to real-time as possible, so that the chart
appears to move from right to left.

Here is an example that shows these properties.

3.3.2 General approach

Monitoring charts are developed in 2 phases. You will see the terminology of:

• Phase 1: building and testing the chart from historical data that you have collected. This phase is
performed off-line, it is very iterative, and you will spend most of your time here. The primary
purpose of this phase is to

– find portions of the data that are from stable operation

– use these stable portions to calculate suitable control chart limits

– ensure that your chart works as expected based on historical data

• Phase 2: We use the monitoring chart on new, fresh data from the process. This phase is
implemented with computer hardware and software for real-time display of the charts. This phase
is skipped if the phase 1 testing is not successful (e.g. too many false alarms). We discuss reasons for
failure in the section on judging the chart’s performance (page 158).

3.3.3 What should we monitor?

Any variable can be monitored. However, the purpose of process monitoring is so that you can react
early to bad, or unusual operation. This implies we should monitor variables as soon as they become
available, preferably in real-time. They are more suitable than variables that take a long time to acquire
(e.g. laboratory measurements). We should not have to wait to the end of the production line to find
our process was out of statistical control.

Data/measurements available at the start of your process, suc as raw material data from your supplier
should also be monitored as soon as it is available, e.g. when received by your company, or even
earlier - before the supplier ships it to you.

Intermediate variables measured from sensors at all points along the production process are (a)
available much more frequently and without delay, (b) are more precise, (c) are usually more

152 Chapter 3. Process Monitoring

 Release 547b16

meaningful to the operating staff than final quality variables from the lab, and (d) contain the
“fingerprint” of the fault, helping the engineers with diagnosis of what the problem is and point to
which part(s) of the process need adjustment (see MacGregor, 1997).

Note that we do not have to monitor variables that are measured only from on-line sensors. The
variable could be a calculation made from the on-line measurements.

For example, an energy balance could be calculated from various thermocouples on the process and
the degree of mismatch in the energy balance could be critical to quality. For example, the mismatch
could indicate an unexpected source of heat into or out of the process - so monitor that mismatch,
rather than the raw temperature data. Similarly, a mass balance can be monitored in real-time, such as
a total mass balance, or a carbon (or other elemental) balance. This is common in the mining industry
and bio-processing industries.

Discuss one of these unit operations with your colleague. Which variables would you monitor?

• Waste water treatment process

• Tablet/pharmaceutical manufacturing

• Oil and gas (e.g. a distillation column)

• Food-processing or bio-engineering (e.g. fermentation) unit

• Mineral processing plant (e.g. a flotation cell)

• Plastics processing (e.g. a twin-screw extruder)

3.3.4 In-control vs out-of-control

Every book on quality control gives a slightly different viewpoint, or uses different terminology for
these terms.

In this book we will take “in-control” to mean that the behaviour of the process is stable over time.
Note though, that in-control does not mean the variable of interest meets the specifications required by
the customer, or set by the plant personnel. All that “in control” means is that there are no special
causes in the data, i.e. the process is stable. A special cause, or an assignable cause is an event that
occurs to move the process, or destabilize it. Process monitoring charts aim to detect such events. The
opposite of “special cause” operation is common cause operation, or stable process operation.

ò Note

Our objective: quickly detect abnormal variation, and fix it by finding the root cause. In this section
we look at the “detection” problem. Diagnosis and process adjustment are two separate steps that
follow detection.

3.4 Shewhart charts

Video for
this section A Shewhart chart, named after Walter Shewhart from Bell Telephone and Western Electric, monitors
that a process variable remains on target and within given upper and lower limits. It is a monitoring
chart for location. It answers the question whether the variable’s location is stable over time. It does not
track anything else about the measurement, such as its standard deviation. Looking ahead: we show
later (page 160) that a pure Shewhart chart needs extra rules to help monitor the location of a variable
effectively.

3.4. Shewhart charts 153

Process Improvement Using Data

The defining characteristics of a Shewhart chart are: a target, upper and lower control limits (UCL and
LCL). These action limits are defined so that no action is required as long as the variable plotted
remains within the limits. In other words a special cause is not likely present if the points remain
within the UCL and LCL.

3.4.1 Derivation using theoretical parameters

Define the variable of interest as 𝑥, and assume that we have samples of 𝑥 available in sequence order.
No assumption is made regarding the distribution of 𝑥. The average of 𝑛 of these 𝑥-values is defined
as 𝑥, which from the Central limit theorem (page 53) we know will be more normally distributed with
unknown population mean 𝜇 and unknown population variance 𝜎 2 /𝑛, where 𝜇 and 𝜎 refer to the
distribution that samples of 𝑥 came from. The figure here shows the case for 𝑛 = 5.

So by taking subgroups of size 𝑛 values, we now have for each subgroup a newly calculated variable,
√
𝑥 and we will define a shorthand symbol for its standard deviation: 𝜎𝑋 = 𝜎/ 𝑛. Writing a 𝑧-value for
𝑥, and its associated confidence interval for 𝜇 is now easy after studying the section on confidence
intervals (page 77):

𝑥−𝜇
𝑧=
𝜎𝑋

Assuming we know 𝜎𝑋 , which we usually do not in practice, we can invoke the normal distribution
and calculate the probability of finding a value of 𝑧 between 𝑐𝑛 = −3 to 𝑐𝑛 = +3:

𝑥−𝜇
−𝑐𝑛 ≤ ≤ +𝑐𝑛
𝜎𝑋

𝑥 − 𝑐𝑛 𝜎𝑋 ≤ 𝜇 ≤ 𝑥 + 𝑐𝑛 𝜎 𝑋 (3.1)

LCL ≤ 𝜇 ≤ UCL

The reason for 𝑐𝑛 = ±3 is that the total area between that lower and upper bound spans 99.73% of the
area (in R: pnorm(+3) - pnorm(-3) gives 0.9973). So it is highly unlikely, a chance of 1 in 370, that
a data point, 𝑥, calculated from a subgroup of 𝑛 raw 𝑥-values, will lie outside these bounds.

The following illustration should help connect the concepts: the raw data’s distribution happens to
have a mean of 6 and standard deviation of 2, while it is clear the distribution of the subgroups of 5
samples (thicker line) is much narrower.

154 Chapter 3. Process Monitoring

 Release 547b16

3.4.2 Using estimated parameters instead

The derivation in equation (3.1) requires knowing the population variance, 𝜎, and assuming that our
target for 𝑥 is 𝜇. The latter assumption is reasonable, but we will estimate a value for 𝜎 instead, using
the data.

Let’s take a look at phase 1, the step where we are building the monitoring chart’s limits from
𝐾
1 ∑︁
historical data. Create a new variable 𝑥 = 𝑥𝑘 , where 𝐾 is the number of 𝑥 samples we have
𝐾
𝑘=1
available to build the monitoring chart, called the phase 1 data. Note that 𝑥 is sometimes called the
grand mean. Alternatively, just set 𝑥 to the desired target value for 𝑥 or use a long portion of stable
data to estimate a suitable target

The next hurdle is 𝜎. Define 𝑠𝑘 to be the standard deviation of the 𝑛 values in the 𝑘 th subgroup. We do
𝑆
not show it here, but for a subgroup of 𝑛 samples, an unbiased estimator of 𝜎 is given by , where
𝑎𝑛
𝐾
1 ∑︁
𝑆= 𝑠𝑘 is simply the average standard deviation calculated from 𝐾 subgroups. Values for 𝑎𝑛 are
𝐾
𝑘=1
looked up from a table, or using the formula below, and depend on the number of samples we use
within each subgroup.

𝑛 2 3 4 5 6 7 8 10 15
𝑎𝑛 0.7979 0.8862 0.9213 0.9400 0.9515 0.9594 0.9650 0.9727 0.9823

More generally, using the Γ(...) function, for example gamma(...) in R or MATLAB, or
[Link](...) in Python, you can reproduce the above 𝑎𝑛 values.
√
2 Γ(𝑛/2)
𝑎𝑛 = √
𝑛 − 1 Γ(𝑛/2 − 0.5)

3.4. Shewhart charts 155

Process Improvement Using Data

Notice how the 𝑎𝑛 values tend to 1.0 the larger the subgroup size, indicating we need less of a
correction to make the standard deviation less biased. Once we have this unbiased estimator for the
standard deviation from these 𝐾 subgroups, we can write down suitable lower and upper control
limits for the Shewhart chart:

LCL = 𝑥 − 3 · 𝑆√ UCL = 𝑥 + 3 · 𝑆√ (3.2)

𝑎𝑛 𝑛 𝑎𝑛 𝑛
It is highly unlikely that all the data chosen to calculate the phase 1 limits actually lie within these
calculated LCL and UCLs. Those portions of data not from stable operation, which are outside the
limits, should not have been used to calculate these limits. Those unstable data bias the limits to be
wider than required.

Exclude these outlier data points and recompute the LCL and UCLs. Usually this process is repeated 2
to 3 times. It is wise to investigate the data being excluded to ensure they truly are from unstable
operation. If they are from stable operation, then they should not be excluded. These data may be
violating the assumption of independence (page 161). One may consider using wider limits, or use an
EWMA control chart (page 164).

Example

Bales of rubber are being produced, with every 10th bale automatically removed from the line for
testing. Measurements of colour intensity are made on 5 sides of that bale, using calibrated digital
cameras under controlled lighting conditions. The rubber compound is used for medical devices, so it
needs to have the correct colour, as measured on a scale from 0 to 255. The average of the 5 colour
measurements is to be plotted on a Shewhart chart. So we have a new data point appearing on the
monitoring chart after every 10th bale.

In the above example the raw data are the bale’s colour. There are 𝑛 = 5 values in each subgroup.
Collect say 𝐾 = 20 samples of good production bales considered to be from stable operation. No
special process events occurred while these bales were manufactured.

The data below represent the average of the 𝑛 = 5 samples from each bale, there are 𝐾 = 20 of these
subgroups.

𝑥 = [245, 239, 239, 241, 241, 241, 238, 238, 236, 248, 233, 236, 246, 253, 227, 231, 237, 228, 239, 240]

The overall average is 𝑥 = 238.8 and 𝑆 = 9.28. The raw data are available on this website62 and you
can verify the values of 𝑥 and 𝑆 were correctly calculated.

• Calculate the lower and upper control limits for this Shewhart chart.

• Were there any points in the phase 1 data (training phase) that exceeded these limits?
𝑆√ = 238.8 − 3 · 9.28
– LCL = 𝑥 − 3 · √ = 225.6
𝑎𝑛 𝑛 (0.94)( 5)
𝑆√ = 238.8 + 3 · 9.28
– UCL = 𝑥 + 3 · √ = 252.0
𝑎𝑛 𝑛 (0.94)( 5)
– The group with 𝑥 = 253 exceeds the calculated upper control limit.

– That 𝑥 point should be excluded and the limits recomputed. You can show the new 𝑥 = 238.0
and 𝑆 = 9.68 and the new LCL = 224 and UCL = 252.

In source code:
62 [Link]

156 Chapter 3. Process Monitoring

 Release 547b16

import numpy as np
from [Link] import gamma

# Given information (but calculate yourself

# from [Link]
xbar = [Link]([245, 239, 239, 241, 241, 241, 238,
238, 236, 248, 233, 236, 246, 253,
227, 231, 237, 228, 239, 240],
dtype=float)

# Number of measurements per subgroup

N_sub = 5

# Average of the 20 standard deviations

# of the 20 subgroups
S = 9.28

# xdb = x double bar = overall mean =

# mean of the means
xdb = [Link]()

num_an = [Link](2) * gamma(N_sub / 2)

den_an = [Link](N_sub - 1) * gamma((N_sub - 1) / 2)
an = num_an / den_an

LCL = xdb - (3 * S / (an * [Link](N_sub)))

UCL = xdb + (3 * S / (an * [Link](N_sub)))
print(f"Control limits: [{round(LCL, 2)}; "
f"{round(UCL, 2)}]")

print(f"Number > UCL: {(xbar > UCL).sum()}")

print(f"Number < LCL: {(xbar < LCL).sum()}")

# Exclude the one subgroup above the UCL.

# Do this by setting it to NaN (missing).
xbar[xbar > UCL] = [Link]

# Calculate the mean, ignoring NaN.

xdb = [Link](xbar)

# 'S' will change also. If you download the

# raw data (link above), you can prove
# that the new 'S' will be:
S = 9.68

# 'an' and N_sub will not change.

LCL = xdb - (3 * S / (an * [Link](N_sub)))

UCL = xdb + (3 * S / (an * [Link](N_sub)))
print(f"Control limits: [{round(LCL)}; "
f"{round(UCL)}]")

# Given information (but calculate yourself

# from [Link]
xbar = c(245, 239, 239, 241, 241, 241, 238,
238, 236, 248, 233, 236, 246, 253,
227, 231, 237, 228, 239, 240)

# Number of measurements per subgroup

[Link] = 5

# Average of the 20 standard deviations

# of the 20 subgroups
S = 9.28

(continues on next page)

3.4. Shewhart charts 157

 Process Improvement Using Data

(continued from previous page)

# xdb = x double bar = overall mean =
# mean of the means
xdb = mean(xbar)

[Link] = sqrt(2) * gamma([Link]/2)

[Link] = sqrt([Link]-1) * gamma(([Link]-1)/2)
an = [Link] / [Link]

LCL = xdb - (3 * S/(an * sqrt([Link])))

UCL = xdb + (3 * S/(an * sqrt([Link])))
paste0('Control limits: [', round(LCL, 2),
'; ', round(UCL,2), ']')

paste0('Number > UCL: ', sum(xbar > UCL))

paste0('Number < LCL: ', sum(xbar < LCL))

# Exclude the one subgroup above the UCL.

# Do this by setting it to 'NA' (missing)
xbar[xbar > UCL] = NA

# Calculate the mean, removing missing

# values (ignore it).
xdb = mean(xbar, [Link]=TRUE)

# 'S' will change also. If you download the

# raw data (link above), you can prove
# that the new 'S' will be:
S = 9.68

# The 'an' and '[Link]' will not change.

LCL = xdb - (3 * S/(an * sqrt([Link])))

UCL = xdb + (3 * S/(an * sqrt([Link])))
paste0('Control limits: [', round(LCL, 0),
'; ', round(UCL,0), ']')

3.4.3 Judging the chart’s performance

Video for
this section There are 2 ways to judge performance of a monitoring chart. In particular here we discuss the
Shewhart chart:

1. Error probability.

We define two types of errors, Type I and Type II, which are a function of the lower and upper control
limits (LCL and UCL).

You make a type I error when your sample is typical of normal operation, yet, it falls outside the UCL
or LCL limits. We showed in the theoretical derivation that the area covered by the upper and lower
control limits is 99.73%. The probability of making a type I error, usually denoted as 𝛼 is then
100 − 99.73 = 0.27%.

Synonyms for a type I error: false alarm, false positive (used mainly for testing of diseases), producer’s
risk (used for acceptance sampling, because here as the producer you will be rejecting an acceptable
sample), false rejection rate, or alpha.

You make a type II error when your sample really is abnormal, but falls within the the UCL and LCL
limits and is therefore not detected. This error rate is denoted by 𝛽, and it is a function of the degree of
abnormality, which we derive next.

Synonyms for a type II error: false negative (used mainly for testing of diseases), consumer’s risk (used

158 Chapter 3. Process Monitoring

 Release 547b16

for acceptance sampling, because your consumer will be receiving available product which is
defective), false acceptance rate, or beta.

To quantify the probability 𝛽, recall that a Shewhart chart is for monitoring location, so we make an
assumption that the new, abnormal sample comes from a distribution which has shifted its location
from 𝜇 to 𝜇 + Δ𝜎 (e.g. Δ can be positive or negative). Now, what is the probability this new sample,
which come from the shifted distribution, will fall within the existing LCL and UCL? This figure
shows the probability is 𝛽 = (1 − the shaded area).

𝛼 = 𝑃 𝑟 (𝑥 is in control, but lies outside the limits) = type I error rate

𝛽 = 𝑃 𝑟 (𝑥 is not in control, but lies inside the limits) = type II error rate

The table highlights that 𝛽 is a function of the amount by which the process shifts = Δ, where Δ = 1
implies the process has shifted up by 1𝜎. The table was calculated for 𝑛 = 4 and used critical limits of
±3𝜎𝑋 . You can calculate your own values of 𝛽 using this line of R code: beta <- pnorm(3 -
delta*sqrt(n)) - pnorm(-3 - delta*sqrt(n))

Δ 0.25 0.50 0.75 1.00 1.50 2.00

𝛽 when 𝑛 = 4 0.9936 0.9772 0.9332 0.8413 0.5000 0.1587

import numpy as np
from [Link] import norm

delta = 1
n = 4
beta = ([Link](+3 - delta * [Link](n))
- [Link](-3 - delta * [Link](n)))

print(f"When delta={delta} and n={n} "

f"then beta = {round(beta, 4)}")

delta <- 1
n <- 4
(continues on next page)

3.4. Shewhart charts 159

Process Improvement Using Data

(continued from previous page)

beta <- pnorm(+3 - delta*sqrt(n)) -
pnorm(-3 - delta*sqrt(n))

paste0('When delta=', delta, ' and n=', n,

' then beta = ', round(beta, 4))

The key point you should note from the table is that a Shewhart chart is not good (it is slow) at
detecting a change in the location (level) of a variable. This is surprising given the intention of the plot
is to monitor the variable’s location. Even a moderate shift of 0.75𝜎 units (Δ = 0.75) will only be
detected around 6.7% of the time (100 − 93.3%) when 𝑛 = 4. We will discuss CUSUM charts (page 162)
and the Western Electric rules, next, as a way to overcome this issue.

It is straightforward to see how the type I, 𝛼, error rate can be adjusted - simply move the LCL and
UCL up and down, as required, to achieve your desired error rates. There is nothing wrong in
arbitrarily shifting these limits - more on this later (page 161) in the section on adjusting limits.

However what happens to the type II error rate as the LCL and UCL bounds are shifted away from the
target? Imagine the case where you want to have 𝛼 → 0. As you make the UCL higher and higher, the
value for 𝛼 drops, but the value for 𝛽 will also increase, since the control limits have become wider!
You cannot simultaneously have low type I and type II error, or as said more colloquially, “there is
no free lunch”.

2. Using the average run length (ARL)

The average run length (ARL) is defined as the average number of sequential samples we expect
before seeing an out-of-bounds, or out-of-control signal. This is given by the inverse of 𝛼, as ARL = 𝛼1 .
Recall for the theoretical distribution we had 𝛼 = 0.0027, so the ARL = 370. Thus we expect a run of
370 samples before we get an out-of-control signal.

3.4.4 Extensions to the basic Shewhart chart to help monitor stability of the location
The Western Electric rules: we saw above how sluggish the Shewhart chart is in detecting a small shift
in the process mean, from 𝜇 to 𝜇 + Δ𝜎. The Western Electric rules are an attempt to more rapidly
detect a process shift, by raising an alarm when these improbable events occur:

1. Two out of 3 points lie beyond 2𝜎 on the same side of the centre line

2. Four out of 5 points lie beyond 1𝜎 on the same side of the centre line

3. Eight successive points lie on the same side of the center line

However, an alternative chart, the CUSUM chart is more effective at detecting a shift in the mean.
Notice also that the theoretical ARL, 1/𝛼, is reduced by using these rules in addition to the LCL and
UCL bounds.

Adding robustness: the phase I derivation of a monitoring chart is iterative. If you find a point that
violates the LCL and UCL limits, then the approach is to remove that point, and recompute the LCL
and UCL values. That is because the LCL and UCL limits would have been biased up or down by
these unusual points 𝑥𝑘 points.

This iterative approach can be tiresome with data that has spikes, missing values, outliers, and
other problems typical of data pulled from a process database (historian). Robust monitoring
charts are procedures to calculate the limits so the LCL and UCL are resistant to the effect of
outliers. For example, a robust procedure might use the medians and MAD instead of the mean

160 Chapter 3. Process Monitoring

 Release 547b16

and standard deviation. An examination of various robust procedures, especially that of the
interquartile range, is given in the paper by D. M. Rocke, Robust Control Charts63 , Technometrics,
31 (2), p 173 - 184, 1989.

Note: do not use robust methods to calculate the values plotted on the charts during phase 2, only
use robust methods to calculate the chart limits in phase 1!

Warning limits: it is common to see warning limits on a monitoring chart at ±2𝜎, while the ±3𝜎 limits
are called the action limits. Real-time computer systems usually use a colour scheme to distinguish
between the warning state and the action state. For example, the chart background changes from
green, to orange to red as the deviations from target become more severe.

Adjusting the limits: The ±3𝜎 limits are not set in stone. Depending on the degree to which the source
data obey the assumptions, and the frequency with which spikes and outliers contaminate your data,
you may need to adjust your limits, usually wider, to avoid frequent false alarms. Nothing makes a
monitoring chart more useless to operators than frequent false alarms (”crying wolf64 ”). However,
recall that there is no free lunch (page 158): you cannot simultaneously have low type I and type II error.

Changing the subgroup size: It is perhaps a counterintuitive result that increasing the subgroup size,
𝑛, leads to a more sensitive detection system for shifts in the mean, because the control limits are
pulled in tighter. However, the larger 𝑛 also means that it will take longer to see the detection signal as
the subgroup mean is averaged over more raw data points. So there is a trade-off between subgroup
size and the run length (time to detection of a signal).

3.4.5 Mistakes to avoid

1. Imagine you are monitoring an aspect of the final product’s quality, e.g. viscosity, and you have a
product specification that requires that viscosity to be within, say 40 to 60 cP. It is a mistake to place
those specification limits on the monitoring chart as a guide when to take action. It is also a mistake
to use the required specification limits instead of the LCL and UCL. The monitoring chart is to
detect abnormal variation in the process and gives a signal on when to take action, not to inspect for
quality specifications. You can certainly have another chart for that, but the process monitoring
chart’s limits are intended to monitor process stability, and these Shewhart stability limits are
calculated differently. Ideally the specification limits lie beyond the LCL and UCL action limits.

2. Shewhart chart limits were calculated with the assumption of independent subgroups (e.g.
subgroup 𝑖 has no effect on subgroup 𝑖 + 1). For a process with mild autocorrelation, the act of
creating subgroups, with 𝑛 samples in each group, removes most, if not all, of the relationship
between subgroups. However processes with heavy autocorrelation (slow moving processes
sampled at a high rate, for example), will have LCL and UCL calculated from equation (3.2) that
will raise false alarms too frequently. In these cases you can widen the limits, or remove the
autocorrelation from the signal. More on this in the later section on exponentially weighted moving
average (EWMA) charts (page 164).

3. Using Shewhart charts on two or more highly correlated quality variables, usually on your final
product measurement, can increase your type II (consumer’s risk) dramatically. We will come back
to this very important topic in the section on latent variable models (page 495), where we will
counterintuitively prove that even having individual charts each within their respective limits can
result where it is outside the joint limits.
63 [Link]
64 [Link]

3.4. Shewhart charts 161

Process Improvement Using Data

3.5 CUSUM charts

We showed earlier (page 159) that the Shewhart chart is not too sensitive to detecting shifts in the mean.
Depending on the subgroup size, 𝑛, we showed that it can take several consecutive samples before a
warning or action limit is triggered. The cumulative sum chart, or CUSUM chart, allows more rapid
detection of these shifts away from a target value, 𝑇 .

The following equation shows how this chart works.

𝑆0 = (𝑥0 − 𝑇 )
𝑆1 = (𝑥0 − 𝑇 ) + (𝑥1 − 𝑇 ) = 𝑆0 + (𝑥1 − 𝑇 )
𝑆2 = (𝑥0 − 𝑇 ) + (𝑥1 − 𝑇 ) + (𝑥2 − 𝑇 ) = 𝑆1 + (𝑥2 − 𝑇 ) (3.3)

In general 𝑆𝑡 = 𝑆𝑡−1 + (𝑥𝑡 − 𝑇 )

162 Chapter 3. Process Monitoring

 Release 547b16

Values of 𝑆𝑡 are the values plotted on the y-axis of a CUSUM chart. Imagine during a period of good,
stable, in-control process operation around the target 𝑇 , then these 𝑆𝑡 numbers are just random errors,
with mean of zero. The long-term sum of 𝑆𝑡 is also zero, as the positive and negative errors keep
cancelling out.

So imagine a CUSUM chart where at some time point the process mean shifts up by Δ units, causing
future values of 𝑥𝑡 to be 𝑥𝑡 + Δ instead. Now the summation in the last equation of (3.3) has an extra Δ
term added at each step to 𝑆𝑡 . Every point will build up an accumulation of Δ, which shows up as a

3.5. CUSUM charts 163

Process Improvement Using Data

positive or negative slope in the CUSUM chart.

The CUSUM chart is extremely sensitive to small changes. The example chart is shown here for a
process where the mean is 𝜇 = 20, and 𝜎 = 3. A small shift of 0.4 × 3 = 1.2 units (i.e from 20 to 21.2)
occurs at 𝑡 = 150. This shift is almost imperceptible in the raw data (see the 3rd row in the figure).
However, the CUSUM chart rapidly picks up the shift by showing a consistent rising slope.

This figure also shows how the CUSUM chart is used with the 2 masks. Notice that there are no lower
and upper bounds for 𝑆𝑡 . A process that is on target will show a “wandering” value of 𝑆, moving up
and down. In fact, as the second row in the figure shows, a surprising amount of movement up and
down occurs even when the process is in control.

What is of interest however is a persistent change in slope in the CUSUM chart. The angle of the
superimposed V-mask is the control limit: the narrower the mouth of the mask, the more sensitive the
CUSUM chart is to deviations from the target. Both the type I and II error are set by the angle of the V
and the leading distance (the distance from the short vertical line to the apex of the V).

The process is considered in control as long as all points are within the arms of the V shape. The mask
in the second row of the plot shows “in control” behaviour, while the mask in the fourth row detects
the process mean has shifted, and an alarm should be raised.

Once the process has been investigated the CUSUM value, 𝑆𝑡 is often reset to zero; though other
resetting strategies exist. A tabular version of the CUSUM chart also exists which tends to be the
version used in software systems.

The purpose of this section is not to provide formulas for the V-mask or tabular CUSUM charts, only
to explain the CUSUM concept to put the next section on EWMA control charts in perspective.

3.6 EWMA charts

The two previous charts highlight 2 extremes of monitoring charts. On the one hand, a Shewhart chart
assumes each subgroup sample is independent (unrelated) to the next - implying there is no
“memory” in the chart. On the other hand, a CUSUM chart has an infinite memory, all the way back to
the time the chart was started or reset at 𝑡 = 0 (see the equation in the prior section (page 162)).

As an introduction to the exponentially weighted moving average (EWMA) chart, consider first the
simple moving average (MA) chart. This chart is used just like a Shewhart chart, except the samples
that make up each subgroup are calculated using a moving window of width 𝑛. The case of 𝑛 = 5 is
shown below.

The MA chart plots values of 𝑥𝑡 , calculated from groups of size 𝑛, using equal weight for each of the 𝑛
most recent raw data.
1 1 1
𝑥𝑡 = 𝑥𝑡−1 + 𝑥𝑡−2 + . . . + 𝑥𝑡−𝑛
𝑛 𝑛 𝑛
The EWMA chart is similar to the MA chart, but uses different weights; heavier weights for more
recent observations, tailing off exponentially to very small weights further back in history. Let’s take a
look at a derivation.

164 Chapter 3. Process Monitoring

 Release 547b16

Define the process target as 𝑇 and define 𝑥𝑡 as a new data measurement arriving now. We then try to
create an estimate of that incoming value, giving some weight, 𝜆, to the actual measured value, and the rest of the
weight, 1 − 𝜆, to the prior estimate.

^𝑡 , with the ∧ mark above the 𝑥𝑡 to indicate that it is a prediction of

Let us write the estimate of 𝑥𝑡 as 𝑥
the actual measured 𝑥𝑡 value. The prior estimate is therefore written as 𝑥 ^𝑡−1 .

So putting into equation form that “an estimate of that incoming value, is given by some weight, 𝜆 and
the rest of the weight, 1 − 𝜆, to the prior estimate”:

𝑥
^𝑡 = 𝜆𝑥𝑡 + (1 − 𝜆) 𝑥
^𝑡−1
𝑥
^𝑡 = ^𝑡−1 + 𝜆 (𝑥𝑡 − 𝑥
𝑥 ^𝑡−1 )
(3.4)
𝑥
^𝑡+1 = ^𝑡 + 𝜆 (𝑥𝑡+1 − 𝑥
𝑥 ^𝑡 )
𝑥
^𝑡+1 = 𝜆𝑥𝑡+1 + (1 − 𝜆) 𝑥^𝑡

To start the EWMA sequence we define the value for 𝑥 ^1 = 𝜆𝑥1 + 𝑇 (1 − 𝜆). A worked
^0 = 𝑇 and 𝑥
example is given further on in this section.

The last line in the equation group above shows that a 1-step-ahead prediction for 𝑥 at time 𝑡 + 1 is a
weighted sum of two components: the current measured value, 𝑥𝑡 , and secondly the predicted value,
^𝑡 , with the weights summing up to 1. This gives a way to experimentally find a suitable 𝜆 value from
𝑥
historical data: adjust it up and down until the differences between 𝑥
^𝑡+1 and the actual measured
values of 𝑥𝑡+1 are small.

The next plot shows visually what happens as the weight of 𝜆 is changed. In this example a shift of
Δ = 1𝜎 = 3 units occurs abruptly at 𝑡 = 150. This is of course not known in practice, but the purpose
here is to illustrate the effects of choosing 𝜆. Prior to that change the process mean is 𝜇 = 20 and the
raw data has 𝜎 = 3.

The first chart is the raw data and also a Shewhart chart with subgroup size of 1; the control limits are
at ±3 time the standard deviation, so at 11.0 and 19.0 units. This control chart barely picks up the shift,
as was explained in a prior section (page 160).

The second, third and fourth charts are EWMA charts with different values of 𝜆; the line is the value
on the left-hand side of equation (3.4), in other words it is 𝑥
^𝑡+1 , the EWMA value at time 𝑡. We see that
as 𝜆 decreases, the charts are smoother, since the averaging effect is greater: more and more weight is
given to the history, 𝑥
^𝑡 , and less weight to the current data point, 𝑥𝑡 . See equation (3.4) to understand
that interpretation. Also note carefully how the control limits become narrower as the 𝜆 decreases, as is
explained shortly below.

To see why 𝑥
^𝑡 represents historical data, you can recursively substitute and show that:
𝑖=𝑡
∑︁
𝑥
^𝑡+1 = 𝑤𝑖 𝑥𝑖 = 𝑤0 𝑥0 + 𝑤1 𝑥1 + 𝑤2 𝑥2 + . . .
𝑖=0

where the weights are: 𝑤𝑖 = 𝜆(1 − 𝜆)𝑡−𝑖

which emphasizes that the prediction is a just a weighted sum of the raw measurements, with weights
declining in time.

The final chart of the sequence of 5 charts is a CUSUM chart, which is the ideal chart (page 162) for
picking up such an abrupt shift in the level.

3.6. EWMA charts 165

Process Improvement Using Data

166 Chapter 3. Process Monitoring

 Release 547b16

In the next figure, we show a comparison of the weights used in different monitoring charts studied so
far.

From the above discussion and the weights shown for the 4 different charts, it should be clear now
how an EWMA chart is a tradeoff between a Shewhart chart and a CUSUM chart. As 𝜆 → 1, the
EWMA chart behaves more as a Shewhart chart, giving only weight to the most recent observation.
While as 𝜆 → 0 the EWMA chart starts to have an infinite memory (like a CUSUM chart). There are 12
data points used in the example, so the CUSUM ‘weight’ is one twelfth or ≈ 0.0833.

The upper and lower control limits for the EWMA plot are plotted in the same way as the Shewhart
limits, but calculated differently:
√︁ √︁
LCL = 𝑥 − 𝐾 · 𝜎Shewhart 𝜆 UCL = 𝑥 + 𝐾 · 𝜎Shewhart 𝜆 (3.5)
2−𝜆 2−𝜆
where 𝜎Shewhart represents the standard deviation as calculated for the Shewhart chart. 𝐾 is usually a
value of 3, similar to the 3 standard deviations used in a Shewhart chart, but can of course be set to any
level that balances the type I (false alarms) and type II errors (not detecting a deviation which is
present already).

An interesting implementation can be to show both the Shewhart and EWMA plot on the same chart,
with both sets of limits. The EWMA value plotted is actually the one-step ahead prediction of the next
𝑥-value, which can be informative for slow-moving processes.

The code here shows one way of calculating the EWMA values for a vector of data. Once you have
defined the function, use it as ewma(x, lam=..., target=...).

3.6. EWMA charts 167

Process Improvement Using Data

import numpy as np

def ewma(x, lam, target=None):

if target is None:
target = x[0]
y = [Link](len(x))
y[0] = target
for k in range(1, len(x)):
error = x[k - 1] - y[k - 1]
y[k] = y[k - 1] + lam * error
return y

# Try using this function now:

x = [Link]([200, 210, 190, 190, 190, 190])
ewma(x, lam=0.3, target=200)

ewma <- function(x, lambda, target=x[1]){

N <- length(x)
y <- numeric(N)
y[1] = target
for (k in 2:N){
error = x[k - 1] - y[k - 1]
y[k] = y[k - 1] + lambda*error
}
return(y)
}

# Try using this function now:

x <- c(200, 210, 190, 190, 190, 190)
ewma(x, lambda = 0.3, target = 200)

Here is a worked example, starting with the assumption the process is at the target value of 𝑇 = 200
units, and 𝜆 = 0.3. We intentionally show what happens if the new value stays fixed at 190: you see
the value plotted gets only a weight of 0.3, while the 0.7 weight is for the prior historical value. Slowly
the value plotted catches up, but there is always a lag. The value plotted on the chart is from the last
equation in the set of (3.4).

Sample number Raw data 𝑥𝑡 Value plotted on chart: 𝑥

^𝑡
0 NA 200
1 200 0.3 × 200 + 0.7 × 200 = 200
2 210 0.3 × 210 + 0.7 × 200 = 203
3 190 0.3 × 190 + 0.7 × 203 = 199.1
4 190 0.3 × 190 + 0.7 × 199.1 = 196.4
5 190 0.3 × 190 + 0.7 × 196.4 = 194.5
6 190 0.3 × 190 + 0.7 × 194.5 = 193.1

3.7 Other types of monitoring charts

You may encounter other charts in practice:

• The S chart is for monitoring the subgroup’s standard deviation. Take the group of 𝑛 samples and
show their standard deviation on a Shewhart-type chart. The limits for the chart are calculated
using similar correction factors as were used in the derivation for the 𝑥 Shewhart chart. This chart
has a LCL ≥ 0.

168 Chapter 3. Process Monitoring

 Release 547b16

• The R chart was a precursor for the S chart, where the R stands for range, the subgroup’s maximum
minus minimum. It was used when charting was done manually, as standard deviations were
tedious to calculate by hand.

• The np chart and p chart are used when monitoring the proportion of defective items using a
pass/fail criterion. In the former case the sample size taken is constant, while in the latter the
proportion of defective items is monitored. These charts are derived using the binomial distribution.

• The exponentially weight moving variance (EWMV) chart is an excellent chart for monitoring for an
increase in product variability. Like the 𝜆 from an EWMA chart, the EWMV also has a sliding
parameter that can balance current information and historical information to trade-off sensitivity.
More information is available in the paper by J.F. MacGregor, and T.J. Harris, “The Exponentially
Weighted Moving Variance65 ”, Journal of Quality Technology, 25, p 106-118, 1993.

3.8 Process capability

ò Note

This section is not about a particular monitoring chart, but is relevant to the topic of process
monitoring.
Video for
this section
3.8.1 Centered processes
Purchasers of your product may request a process capability ratio (PCR) for each of the quality
attributes of your product. For example, your plastic product is characterized by its Mooney viscosity
and melting point. A PCR value can be calculated for either property, using the definition below:

Upper specification limit − Lower specification limit USL − LSL

PCR = = (3.6)
6𝜎 6𝜎
Since the population standard deviation, 𝜎, is not known, an estimate of it is used. Note that the lower
specification limit (LSL) and upper specification limit (USL) are not the same as the lower control limit
(LCL) and upper control limit (UCL) as were calculated for the Shewhart chart. The LSL and USL are
the tolerance limits required by your customers, or set from your internal specifications.

Interpretation of the PCR:

• assumes the property of interest follows a normal distribution

• assumes the process is centered (i.e. your long term mean is halfway between the upper and lower
specification limits)

• assumes the PCR value was calculated when the process was stable

The PCR is often called the process width. Let’s see why by taking a look at a process with PCR=0.5
and then PCR=2.0. In the first case USL − LSL = 3𝜎. Since the interpretation of PCR assumes a
centered process, we can draw a diagram as shown below:
65 [Link]

3.8. Process capability 169

Process Improvement Using Data

The diagram is from a process with mean of 80 and where LSL=65 and USL=95. These specification are
fixed, set by our production guidelines. If the process variation is 𝜎 = 10, then this implies that
PCR=0.5. Assuming further that the our production is centered at the mean of 80, we can calculate how
much defective product is produced in the shaded region of the plot. Assuming a normal distribution:

• 𝑧 for LSL = (65 − 80)/10 = −1.5

• 𝑧 for USL = (95 − 80)/10 = 1.5

• Shaded area probability = pnorm(-1.5) + (1-pnorm(1.5)) = 13.4% of production is out of the

specification limits.

Contrast this to the case where PCR = 2.0 for the same system. To achieve that level of process
capability, using the same upper and lower specifications we have to reduce the standard deviation by a
factor of 4, down to 𝜎 = 2.5. The figure below illustrates that almost no off-specification product is
produced for a centered process at PCR = 2.0. There is a width of 12𝜎 units from the LSL to the USL,
giving the process location (mean) ample room to drift left or right without creating additional
off-specification product.

ò Note

You will probably come across the terminology Cp , especially when dealing with 6 sigma
programs. This is the same as PCR for a centered process.

170 Chapter 3. Process Monitoring

 Release 547b16

3.8.2 Uncentered processes

Video for
this section Processes are not very often centered between their upper and lower specification limits. So a measure
of process capability for an uncentered processes is defined:
(︂ )︂
Upper specification limit − 𝑥 𝑥 − Lower specification limit
PCRk = Cpk = min ; (3.7)
3𝜎 3𝜎

The 𝑥 term would be the process target from a Shewhart chart, or simply the actual average operating
point. Notice that Cpk is a one-sided ratio, only the side closest to the specification is reported. So even
an excellent process with Cp = 2.0 that is running off-center will have a lower Cpk .

It is the Cpk value that is requested by your customer. Values of 1.3 are usually a minimum
requirement, while 1.67 and higher are requested for health and safety-critical applications. A value of
Cpk ≥ 2.0 is termed a six-sigma process, because the distance from the current operating point, 𝑥, to
the closest specification is at least 6𝜎 units.

You can calculate that a shift of 1.5𝜎 from process center will introduce only 3.4 defects per million.
This shift would reduce your Cpk from 2.0 to 1.5.

ò Note

It must be emphasized that Cpk and Cp numbers are only useful for a process which is stable.
Furthermore the assumption of normally distributed samples is also required to interpret the Cpk
value.

3.9 The industrial practice of process monitoring

This preceding section of the book is only intended to give an overview of the concepts of process
monitoring. As you move into an industrial environment you will find there are many such systems
already in place. Higher levels of management track statistics from a different point of view, often
summarizing data from an entire plant, geographic region, or country. The techniques learned in this
book, while focusing mainly on unit operations, are equally applicable though to data from a plant,
region, or country.

You may come across systems called dashboards, which are often part of enterprise resource planning
(ERP) systems. These dashboards are supposed to monitor the pulse of a company and are tracked like
any other monitoring chart discussed above. Another area is called business intelligence (BI) systems.
These typically track sales and other financial information.

Yet another acronym is the KPI, key performance indicator, which is a summary variable, such as
profit per hour, or energy cost per unit of production. These are often monitored and acted on by site
managers on a daily or weekly basis. Sites in a global company with the lowest KPIs receive the
greatest scrutiny.

But at the unit operation and plant level, you will likely find the hardest part of getting a monitoring
chart implemented is the part where you need access to the data. Getting data out of most database
systems is not easy, though it has improved quite a bit in the last few years.

It is critical that your monitoring chart display the quantity as close to real-time as possible. It is almost
as if the monetary value of the information in a monitoring chart decays exponentially from the time
an event occurs. It is hard to diagnose and correct a problem detected yesterday, and harder still if the
problem occurred last week or month.

3.9. The industrial practice of process monitoring 171

 Process Improvement Using Data

You will also realize that good operator training to interpret and act on the monitoring chart is
time-consuming; operators are often cycled between different units or plants, so frequent re-training is
required. Concepts from the data visualization (page 1) section are helpful to minimize training effort -
make sure the online plots contain the right level of information, without clutter, so they can be acted
on accurately.

Another side effect of large quantities of data are that you will have to work with IT groups to
manipulate large chunks of data on dedicated networks, separate from the rest of the plant. The last
thing you want to be responsible for is clogging the company network with your traffic. Most
industries now have a “production” network running in parallel to the “corporate” network. The
production network carries real-time data, images from cameras and so forth, while the company
network carries office email and web traffic.

3.9.1 Approach to implement a monitoring chart in an industrial setting

Here is some general guidance; feel free to adjust the steps as required for your unique situation.

1. Identify the variable(s) to monitor. Make sure each variable shows different, uncorrelated
phenomena to avoid redundancy. If unsure which variables to select, use a multivariate monitoring
system (page 495).

2. Retrieve historical data from your computer systems, or lab data, or paper records.

3. Import the data and just plot it. Do you see any time trends, outliers, spikes, missing data gaps?
Investigate these (to learn more about your process), but then remove them to create the phase 1
data set.

4. Locate any regions of data which are from generally stable operation. Remove spikes and outliers
that will bias your control limits calculations. In other words, find regions of common-cause
operation.

5. Split your phase 1 data into say a 60% and 40% split. Set aside 40% of the cleaned portion to use as
phase 1 testing data later on. Keep the data from outliers, spikes and unstable process operation
aside as another testing data set (to ensure that these problems are actually detectable).

6. Using the cleaned 60% portion, estimate limits that you would expect to contain this stable region of
operation just by looking at the plots.

7. On the 60% portion, calculate preliminary control limits (UCL, LCL), using the formulas shown in
this section. They should agree with limits in the previous step.

8. How does your chart work? Test your chart on the 40% cleaned portion. These testing data should
not raise many alarms. Any alarms raised will be type I errors, so you can quantify your type I error
rate from the fraction of false alarms raised.

9. Test your chart on the unusual data you found earlier. You can quantify the type II error by
counting the fraction of this bad data that went undetected by your chart.

10. Adjust the limits and monitoring chart parameters (e.g. 𝜆) if necessary, to achieve the required type
I and type II balance that is acceptable to your operation staff. You may even have to resort to using
a different chart, or monitoring based on a different variable.

11. Test the chart on your desktop computer for a couple of days. When you detect an unusual event,
go and check with the process operators and verify the event. Would they have reacted to it, had
they known about it? Or, would this have been a false alarm? You may need to refine your limits, or
the value you are plotting again.

172 Chapter 3. Process Monitoring

 Release 547b16

12. Remember that this form of charting is not an expert system - it will not diagnose problems: you
have to use your engineering knowledge by looking at patterns in the chart, and use knowledge of
other process events.

13. Demonstrate the system to your colleagues and manager. But show them economic estimates of the
value of early detection. They are usually not interested in the plots alone, so convert the statistics
into monetary values. For example, dollars saved if we had detected that problem in real-time,
rather than waiting till later.

14. Installation and operator training will take time. This assumes that you have real-time data
acquisition systems and real-time processing systems in place - most companies do. You will have
to work with your company’s IT staff to get this implemented.

15. Listen to your operators for what they want to see. Use principles of good data visualization (page 1)
to reduce unnecessary information. Make your plots interactive - if you click on an unusual point it
should “drill-down” and give you more details and historical context.

16. Future monitoring charts are easier to get going, once the first system is in place.

3.10 Industrial case study

ArcelorMittal’s steel mill in Hamilton, Ontario, (formerly called Dofasco) has used multivariate
process monitoring tools in many areas of their plant for decades now. One of their most successful
applications is that applied to their casting operation. In this section we only focus on the application;
the sort of multivariate calculations used by this system are discussed later on (page 411).

The computer screenshot shows the monitoring system, called Caster SOS (Stable Operation
Supervisor), which is followed by the operators. There are several charts on the screen: two charts,
called “Stability Index 1” and “Stability Index 2”, are one-sided monitoring charts. Notice the warning
limits and the action limits. In the middle is a two-sided chart. A wealth of information is presented on
the screen - their design was heavily influenced and iterated on several times, working with the
operators. The screen shot is used with permission of Dr. John MacGregor.

3.10. Industrial case study 173

Process Improvement Using Data

The economics of monitoring charts cannot be overstated. The ArcelorMittal example above was
introduced around 1997. The calculations required by this system are complex - however the computer
systems performs them in near real-time, allowing the operators to take corrective action within a few
seconds. The data show a significant reduction in breakouts since 1997 (used with permission of Dr. John
MacGregor).

The economic savings and increased productivity is in the millions of dollars per year, as each
breakout costs around $200,000 to $500,000 due to process shutdowns and/or equipment damage.

3.11 Summary

Montgomery and Runger list 5 reasons why monitoring charts are widely used. After this section of
the book you should understand the following about the charts and process monitoring:

1. These tools are proven to improve productivity (i.e. to reduce scrap and rework, as described
above), and to increase process throughput.

174 Chapter 3. Process Monitoring

 Release 547b16

2. They detect defective production, consistent with the concept of “doing it right the first time”, a
mantra that you will increasingly hear in the manufacturing workplace.

3. A monitoring chart with good limits will prevent over-control of the process. Operators are trained
not to make process adjustments unless there is a clear warning or alarm from the chart.

4. The patterns generated by the plots often help determine what went wrong, providing some
diagnostic value to the operators. We will see a more formal tool for process diagnosis though in the
latent variable section (page 411).

5. Monitoring charts are required to judge if a process is stable over time. A stable process allows us to
calculate our process capability, which is an important metric for your customers.

3.12 Exercises

ò Question

Is it fair to say that a monitoring chart is like an online version of a confidence interval (page 77)?
Explain your answer.

ò Solution

This question is likely to generate a wide range of answers. No surprise, since there are strong
feelings on this point in the quality control literature66 as well. The confusion stems from the fact
that if you are in phase 1, then no, a monitoring chart is not a confidence interval, but in phase 2,
then you can argue that confidence intervals have many similarities to monitoring charts.

But, in general, I feel the above statement is incorrect. Even in phase 2 a monitoring chart is not
really like an on-line confidence interval. Mainly because a monitoring chart is intended to check
for system stability, and to alarm quickly if the system moves away from the assumed distribution
(usually a normal distribution). The monitoring limits are calculated to provide the required alarm
level (the ARL). A confidence interval, on the other hand, defines the limits within which we expect
to find the true population mean with a certain degree of confidence when we use a given sample
of data.

The similarity comes from the way the monitoring chart’s limits are calculated: by using the
concept of a confidence interval. But a monitoring chart’s limits can and should be adjusted up or
down to improve your type I and II error levels, while for a confidence interval, the only way to
alter the limits is to take a different sample size, take a new sample of data, and choose a different
level of confidence. But doing this, will still only find you bounds within which you expect the
population mean to lie. A monitoring chart’s bounds are only there to signal when things are not
the same any more.

ò Question

Use the batch yields data67 and construct a monitoring chart using the 300 yield values. Use a
subgroup of size 5. Report your target value, lower control limit and upper control limit, showing

66 [Link]

3.12. Exercises 175

Process Improvement Using Data

the calculations you made. I recommend that you write your code so that you can reuse it for other
questions.

ò Solution

Please see the code below. The Shewhart chart’s parameters are as below, with plots generated
from the R code.

• Target = 80.4

• Lower control limit at 3 standard deviations = 71.1

• Upper control limit at 3 standard deviations = 89.6

Try it yourself:

import numpy as np
import pandas as pd
from [Link] import gamma

[Link] = "plotly"

data_file = "[Link]
batch = pd.read_csv(data_file)

# Make sure we have the expected data.

[Link]()
Yield = batch["Yield"].to_numpy()

# To get a feel for the data; looks pretty

# good, no unusual outliers.
[Link](Yield).[Link]().show()

N = len(Yield)
N_sub = 5 # subgroup size
# Reshape so each column is one subgroup.
subgroup = [Link](N // N_sub, N_sub).T

67 [Link]

176 Chapter 3. Process Monitoring

 Release 547b16

N_groups = [Link][1]
[Link] # (5, 60)

subgroup_sd = [Link](axis=0, ddof=1)

subgroup_xbar = [Link](axis=0)

# Take a look at what these numbers mean.

[Link](subgroup_xbar).[Link]().update_layout(
yaxis_title_text="Subgroup average").show()
[Link](subgroup_sd).[Link]().update_layout(
yaxis_title_text="Subgroup spread").show()

# Report your target value, lower control

# limit and upper control limit, showing
# the calculations you made.
target = subgroup_xbar.mean()
Sbar = subgroup_sd.mean()

# a_n value is from the table when

# subgroup size = 5
an_num = [Link](2) * gamma(N_sub / 2)
an_den = [Link](N_sub - 1) * gamma(N_sub / 2 - 0.5)
an = an_num / an_den
sigma_estimate = Sbar / an
LCL = target - 3 * sigma_estimate / [Link](N_sub)
UCL = target + 3 * sigma_estimate / [Link](N_sub)
(LCL, target, UCL)

fig = [Link](subgroup_xbar).[Link](
title="Shewhart chart")
fig.update_layout(
yaxis=dict(range=[LCL - 5, UCL + 5],
title_text="Subgroup means"))
fig.add_hline(y=target, line_color="green")
fig.add_hline(y=UCL, line_color="red")
fig.add_hline(y=LCL, line_color="red")
[Link]()

data_file <- '[Link]

batch <- [Link](data_file)

# make sure we have the expected data

summary(batch)
attach(batch)

# To get a feel for the data;

# looks pretty good; no unusual outliers
plot(Yield)

N = length(Yield)
[Link] = 5 # subgroup size
subgroup <- matrix(Yield, [Link], N/[Link])
[Link] <- ncol(subgroup)
dim(subgroup) # 5 by 60 matrix

[Link] <- apply(subgroup, 2, sd)

[Link] <- apply(subgroup, 2, mean)

# Take a look at what these numbers mean

plot([Link],
type="b",
ylab="Subgroup average")
plot([Link],
type="b",
ylab="Subgroup spread")

# Report your target value, lower control

# limit and upper control limit, showing
# the calculations you made.
target <- mean([Link])
Sbar <- mean([Link])
3.12. Exercises 177
# a_n value is from the table when
# subgroup size = 5
an <- 0.94
Process Improvement Using Data

ò Question

The boards data68 on the website are from a line which cuts spruce, pine and fir (SPF) to produce
general quality lumber that you could purchase at Rona, Home Depot, etc. The price that a saw
mill receives for its lumber is strongly dependent on how accurate the cut is made. Use the data for
the 2 by 6 boards (each row is one board) and develop a monitoring system using these steps.

a) Plot all the data.

b) Now assume that boards 1 to 500 are the phase 1 data; identify any boards in this subset that
appear to be unusual (where the board thickness is not consistent with most of the other
operation)

c) Remove those unusual boards from the phase 1 data. Calculate the Shewhart monitoring limits
and show the phase 1 data with these limits. Note: choose a subgroup size of 7 boards.

d) Test the Shewhart chart on boards 501 to 2000, the phase 2 data. Show the plot and calculate the
type I error rate (𝛼) from the phase 2 data; assuming, of course, that all the phase 2 data are from
in-control operation.

e) Calculate the ARL and look at the chart to see if the number looks about right. Use the time
information in the raw data and your ARL value to calculate how many minutes between a false
alarm. Will the operators be happy with this?

f) Describe how you might calculate the consumer’s risk (𝛽).

g) How would you monitor if the saws are slowly going out of alignment?

ò Solution

This questions answers are derived in the source code (at the end).

1. A plot of the raw data:

68 [Link]

178 Chapter 3. Process Monitoring

 Release 547b16

2. A plot of just the phase 1 data shows no particular outliers. Most people found a few outliers,
that’s OK - remember it is a subjective test, and if this were a process you were responsible for,
then you would know more clearly what an outlier was. For me though, I didn’t think any of
these points were particularly unusual.

3. The initial Shewhart parameters found were:

• UCL = 1701

• Target = 1676

• LCL = 1652

When plotting these limits on the phase 1 data, there was only one subgroup that was found
outside the limits (the first subgroup). This subgroup is removed and the limits recalculated.
(For this case there was only one, very moderate, subgroup outside the limits - the new limits are
basically the same). The new limits

• UCL = 1700

3.12. Exercises 179

Process Improvement Using Data

• Target = 1676

• LCL = 1651

A Shewhart chart of all the phase 1 data (including outliers, to highlight them) is shown here.
The limits were the final limits, after iteratively removing the first unusual subgroup . The code
contains all the calculation steps.

4. Using these parameters on the phase 2 data generates the following plot:

Assuming the subgroups in phase 2 are all in control, the 𝛼 value is sum of the points outside
the limits, divided by the total number of subgroups in phase 2 = 9/214 = 4.2%. This is much
greater than the theoretically expected 𝛼 of 0.27%.

Notice though there is a group of points all on one side of the target line. According to the
Western Electric rules, a group of more than 8 points on one side of the target line is highly
improbable and an alarm should be raised. This indicates that these phase 2 testing data are
likely not from in-control operation.

5. The ARL = 1/𝛼 = 1/0.042 = 23.8; i.e. 1 subgroup in every 24 will lie outside the control limits,

180 Chapter 3. Process Monitoring

 Release 547b16

even if that subgroup is from in-control operation. That number looks about right from the
above phase 2 chart, although, most of the outliers seem to occur in the last half of the chart (see
answer to part 4). The data set comes from about 5 hours and 15 minutes (315 minutes) of
operation; during this time there were 286 subgroups that would have been shown on a real
Shewhart chart. With an ARL of 24 subgroups, there would be about 12 (286/24) false alarms
over these 315 minutes. In other words a false alarm about once every 26 minutes. This is much
too high for practical use. Either the limits must be made wider, or this data really is not from
in-control operation.

6. To calculate the consumer’s risk (𝛽) we require a period of data where we know the blades have
shifted, so that the board thickness has been increased or decreased to a new level (mean
operating point). Using that out of control, or unstable data, we calculate Shewhart subgroups
as usual, and count the number of data points falling within the current LCL and UCL. A count
of those in control subgroups divided by the total number of these out of control subgroups
would be an estimate of 𝛽.

7. As the blades go out of alignment, the variability in the thickness values increases. Two ways to
monitor this are

• To plot the subgroup standard deviation over time. I have added the nonparametric
regression lines against time on the plot to highlight how the variability increases over time.
This indicates to me that this data probably was not from in control operation. This is the
reality in most processes: we are never sure that the data are from in-control operation; it is
always trial and error.

• Use a CUSUM chart.

• A more sensitive monitoring chart for this would be the exponentially weighted moving
variance: MacGregor, J.F. and Harris, T.J., “The Exponentially Weighted Moving Variance”,
Journal of Quality Technology, 25, p 106-118, 1993.

boards <- [Link]('[Link]

attach(boards)

[Link] <- length(Thickness) # Total number of boards

[Link] <- 7 # subgroup size
N.phase1 <- 500 # number of phase 1 raw data points

3.12. Exercises 181

Process Improvement Using Data

# a) Plot all the data.

plot(Thickness, type="p", main="Thickness: all data")

# b) Now assume that boards 1 to 500 are the phase I data; identify unusual boards
plot(Thickness[1:N.phase1], type="p", main="Phase 1 raw data", ylab="Thickness")
identify(Thickness) # There are no major outliers

# c) Remove unusual boards from the phase I data.

# Calculate the Shewhart monitoring limits and show the phase I data with these limits.
˓→
# Note: choose an appropriate subgroup size of your choice.

subgroup <- matrix(Thickness, [Link], [Link]/[Link])

[Link] <- ncol(subgroup)

[Link] <- apply(subgroup, 2, sd)

[Link] <- apply(subgroup, 2, mean)

[Link] <- 1
[Link] <- floor(N.phase1/[Link])

target <- mean([Link][[Link]:[Link]])

Sbar <- mean([Link][[Link]:[Link]])

an <- c(NA, 0.793, 0.886, 0.921, 0.940, 0.952, 0.959, 0.965)

[Link] <- Sbar / an[[Link]]
LCL <- target - 3 * [Link]/sqrt([Link])
UCL <- target + 3 * [Link]/sqrt([Link])
round(c(LCL, target, UCL), 0)

# But there is 1 point outside the limits; happens to be the first one: take a short-cut
[Link] = 2
[Link] = floor(N.phase1/[Link])

target <- mean([Link][[Link]:[Link]])

Sbar <- mean([Link][[Link]:[Link]])
[Link] <- Sbar / an[[Link]]
LCL <- target - 3 * [Link]/sqrt([Link])
UCL <- target + 3 * [Link]/sqrt([Link])
round(c(LCL, target, UCL), 0)
# Limits are pretty much the same: not surprising.

[Link] <- [Link][1:[Link]]

plot([Link], type="b", ylim=c(LCL-10, UCL+10), pch=".", cex=5, ylab="Subgroup mean",
˓→main="Shewhart chart for phase I: training data")
abline(h=UCL, col="red")
abline(h=LCL, col="red")
abline(h=target, col="green")

# d) Test the Shewhart chart on boards 501 to 2000, the phase II data.
# Show the plot and calculate the type I error rate (alpha) from the phase II data.
[Link] <- [Link][([Link]+1):length([Link])]
plot([Link], type="b", pch=".", cex=5, ylab="Subgroup mean", main="Shewhart chart
˓→for phase II: testing data")
abline(h=UCL, col="red")
abline(h=LCL, col="red")
abline(h=target, col="green")

phase2.N <- length([Link])

[Link] <- (sum([Link]<LCL) + sum([Link]>UCL))
[Link] <- [Link] / phase2.N
c([Link], phase2.N, [Link])
[Link] = 0.27/100

# e) Calculate the ARL. Given the time information in the raw data,
# use the ARL and calculate how many minutes between a false alarm.
# Will the operators be happy this?

182 Chapter 3. Process Monitoring

 Release 547b16

ARL = ceiling(1/[Link]) # On average, you will have about 1 in

˓→24

# subgroups giving a false alarm

# units = [subgroups per false alarm]
[Link] = 15 + 60 + 4*60 # Or use some other approximation
[Link] = ([Link]/[Link]) / [Link] # [subgroups per minute]: i.e. how often
# a new subgroup is shown on the screen
[Link] = ARL / [Link] # [subgroups per false alarm]/[subgroups
˓→per minute]
# = [minutes/false alarm]
c([Link], [Link], [Link]/[Link], [Link])

# g) How would you monitor if the saws are slowly going out of alignment?
# Show the increase in variance over time with a nonparametric smooth of subgroup
˓→standard deviation
plot([Link], type="p", ylab="Subgroup standard deviation", main="Slow increase in
˓→subgroup variability over the day")

lines(lowess([Link]))

ò Question

Your process with Cpk of 2.0 experiences a drift of 1.5𝜎 away from the current process operating
point towards the closest specification limit. What is the new Cpk value; how many defects per
million items did you have before the drift? And after the drift?

ò Solution

The new Cpk value is 1.5. The number of defects per million items at Cpk = 2.0 is 0.00098
(essentially no defects), while at Cpk = 1.5 it is 3.4 defects per million items. You only have to
consider one-side of the distribution, since Cpk is by definition for an uncentered process, and
deals with the side closest to the specification limits.

from [Link] import norm

Cpk = 1.5
n_sigma_distance = 3 * Cpk
dpm = [Link](-n_sigma_distance,
loc=0,
scale=1) * 1e6
print(f"Defects per million = {round(dpm, 3)}")

Cpk <- 1.5

[Link] <- 3 * Cpk
dpm <- pnorm(-[Link],
mean=0,
sd=1) * 1E6
paste0('Defects per million = ', round(dpm,3))

ò Question

Which type of monitoring chart would be appropriate to detect unusual spikes (outliers) in your

3.12. Exercises 183

Process Improvement Using Data

production process?

ò Solution

A Shewhart chart has no memory, and is suited to detecting unusual spikes in your production.
CUSUM and EWMA charts have memory, and while they would pick up this spike, they would
also create a long duration of false alarms after that. So those charts are much less appropriate.

ò Question

A tank uses small air bubbles to keep solid particles in suspension. If too much air is blown into the
tank, then excessive foaming and loss of valuable solid product occurs; if too little air is blown into
the tank the particles sink and drop out of suspension.

1. Which monitoring chart would you use to ensure the airflow is always near target?

2. Use the aeration rate dataset69 from the website and plot the raw data (total litres of air added in
a 1 minute period). Are you able to detect any problems?

3. Construct the chart you described in part 1, and show it’s performance on all the data. Make any
necessary assumptions to construct the chart.

4. At what point in time are you able to detect the problem, using this chart?

5. Construct a Shewhart chart, choosing appropriate data for phase 1, and calculate the Shewhart
limits. Then use the entire dataset as if it were phase 2 data.

• Show this phase 2 Shewhart chart.

• Compare the Shewhart chart’s performance to the chart in part 3 of this question.

69 [Link]

184 Chapter 3. Process Monitoring

 Release 547b16

ò Solution

Solution based on work by Ryan and Stuart (2011 class)

1. A CUSUM chart would be a suitable chart to monitor that the airflow is near target. While a
Shewhart chart is also intended to monitor the location of a variable, it has a much larger run
length for detecting small shifts. An EWMA chart with small 𝜆 (long memory) would
approximate a CUSUM chart, and so would also be suitable

2. The aeration rate dataset is depicted below:

It is very difficult to assess problems from the raw data plot. There might be a slight upward
shift around 300 and 500 minutes.

3. Assumptions for the CUSUM chart:

• We will plot the CUSUM chart on raw data, though you could use subgroups if you wanted
to.

• The target value can be the mean (24.17) of all the data, or more robustly, use the median
(24.1), especially if we expect problems with the raw data (true of almost every real data set).

4. The CUSUM chart, using the median as target value showed a problem starting to occur around
𝑡 = 300. So we recalculated the median, using only data from 0 to 𝑡 = 200, to avoid biasing the
target value. Using this median instead, 23.95, we get the following CUSUM chart:

5. The revised CUSUM chart suggests that the error occurs around 275 min, as evidenced by the

3.12. Exercises 185

Process Improvement Using Data

steep positive slope thereafter. It should be noted that the CUSUM chart begins to bear a
positive slope around 200 min, but this initial increase in the cumulative error would likely not
be diagnosable (i.e. using a V-mask).

# Code by Ryan and Stuart (2011 class)

CUSUM <- function(x, target){

N <- length(x)
S <- numeric(N)
S[1] = x[1] - target
for (t in 2:N){
S[t] = S[t-1] + (x[t] - target)
}
return(S)
}

# Import data and remove missing values (NA)

[Link] <- [Link]('[Link]
aeration <- [Link]([Link]$Aeration)

# Plot raw data

bitmap('[Link]', type="png256",
width=10, height=4, res=300, pointsize=14)
plot(aeration, type="l", xlab="Time (min)", ylab="Aeration rate (L/min)")
grid()
[Link]()

# Plot CUSUM Chart

target <- median(aeration[1:200])
bitmap('[Link]', type="png256",
width=10, height=4, res=300, pointsize=14)
plot(CUSUM(aeration, target), type="l", xlab="Time (min)",
ylab="CUSUM cumulative deviations")
grid()
[Link]()

# Plot the Shewhart chart: see code from the other question to
# calculate the control limits
LCL <- 22.1
UCL <- 25.8
N <- 5
subgroups <- matrix(aeration, N, length(aeration)/N)
[Link] <- numeric(length(aeration)/N)
[Link] <- numeric(length(aeration)/N)

# Calculate mean and sd of subgroups (see R-tutorial)

[Link] <- apply(subgroups, 2, mean)
[Link] <- apply(subgroups, 2, sd)
ylim <- range([Link]) + c(-5, +5)
xdb <- target # use the same CUSUM target !

bitmap('[Link]',
type="png256", width=10, height=4, res=300, pointsize=14)
par(mar=c(4.2, 4.2, 0.5, 0.5))
par([Link]=1.3, [Link]=1.5, [Link]=1.5, [Link]=1.5)
plot(seq(1,length([Link])*N, N), [Link], type="b", pch=".", cex=5, main="",
ylab="Phase II subgroups", xlab="Time order", ylim=ylim)
abline(h=UCL, col="red")
abline(h=LCL, col="red")
abline(h=xdb, col="green")
lines(c(275, 275), ylim, col="blue")
text(280, 29, "CUSUM detected problem at t=275",adj = c(0,0))
[Link]()

6. Using the iterative Shewhart code from the previous question, we used

186 Chapter 3. Process Monitoring

 Release 547b16

• Phase I was taken far enough away from the suspected error: 0 - 200 min

• Subgroup size of 𝑛 = 5
¯ = 23.9
• 𝑥

• 𝑆¯ = 1.28

• 𝑎𝑛 = 0.940
1.28√
• LCL = 23.9 − 3 · 0.940 5
= 22.1
1.28√
• UCL = 23.9 + 3 · 0.940 5
= 25.8

The Shewhart chart applied to the entire dataset is shown below. In contrast to the CUSUM chart,
the Shewhart chart is unable to detect the problem in the aeration rate. Unlike the CUSUM chart,
which has infinite memory, the Shewhart chart has no memory and cannot adequately assess the
location of the monitored variable in relation to its specified target. Instead, the Shewhart chart
merely monitors aeration rate with respect to the control limits for the process. Since the aeration
rate does not exceed the control limits for the process (i.e. process remains in control), the Shewhart
chart does not detect any abnormalities.

If you used the Western Electric rules, in addition to the Shewhart chart limits, you would have
picked up a consecutive sequence of 8 points on one side of the target around 𝑡 = 350.

ò Question

Do you think a Shewhart chart would be suitable for monitoring the closing price of a stock on the
stock market? Please explain your answer if you agree, or describe an alternative if you disagree.

ò Solution

No, a Shewhart chart is not suitable for monitoring stock prices. Stock prices are volatile variables
(not stable), so there is no sense in monitoring their location. Hopefully the stock is moving up,
which it should on average, but the point is that stock prices are not stable. Nor are stock prices
independent day-to-day.

So what aspect of a stock price is stable? The difference between the opening and closing price of a

3.12. Exercises 187

Process Improvement Using Data

stock is remarkably stationary. Monitoring the day-to-day change in a stock price would work.
Since you aren’t expected to know this fact, any reasonable answer that attempts to monitor a stable
substitute for the price will be accepted. E.g. another alternative is to remove the linear up or down
trend from a stock price and monitor the residuals.

There are many alternatives; if this sort of thing interests you, you might find the area called
technical analysis70 worth investigating. An EWMA chart is widely used in this sort of analysis.

ò Question

Describe how a monitoring chart could be used to prevent over-control of a batch-to-batch process.
(A batch-to-batch process is one where a batch of materials is processed, followed by another batch,
and so on).

ò Solution

Over-control of any process takes place when too much corrective action is applied. Using the
language of feedback control, your gain is the right sign, but the magnitude is too large. Batch
processes are often subject to this phenomenon: e.g. the operator reduces the set-point temperature
for the next batch, because the current batch produced product with a viscosity that was too high.
But then the next batch has a viscosity that is too low, so the operator increases the temperature
set-point for the following batch. This constant switching is known as over-control (the operator is
the feedback controller and his/her gain is too high, i.e. they are over-reacting).

A monitoring chart such as a Shewhart chart would help the operator: if the previous batch was
within the limits, then s/he should not take any corrective action. Only take action when the
viscosity value is outside the limits. An EWMA chart would additionally provide a one-step ahead
prediction, which is an advantage.

ò Question

You need to construct a Shewhart chart. You go to your company’s database and extract data from
10 periods of time lasting 6 hours each. Each time period is taken approximately 1 month apart so
that you get a representative data set that covers roughly 1 year of process operation. You choose
these time periods so that you are confident each one was from in control operation. Putting these
10 periods of data together, you get one long vector that now represents your phase 1 data.

• There are 8900 samples of data in this phase 1 data vector.

• You form subgroups: there are 4 samples per subgroup and 2225 subgroups.

• You calculate the mean within each subgroup (i.e. 2225 means). The mean of those 2225 means is
714.

• The standard deviation within each subgroup is calculated; the mean of those 2225 standard
deviations is 98.

1. Give an unbiased estimate of the process standard deviation?

70 [Link]

188 Chapter 3. Process Monitoring

 Release 547b16

2. Calculate lower and upper control limits for operation at ±3 of these standard deviations from
target. These are called the action limits.

3. Operators like warning limits on their charts, so they don’t have to wait until an action limit
alarm occurs. Discussions with the operators indicate that lines at 590 and 820 might be good
warning limits. What percentage of in control operation will lie inside the proposed warning
limit region?

ò Solution

𝑆 98
1. An unbiased estimate of the process standard deviation is 𝜎
^= 𝑎𝑛 = 0.921 = 106.4, since the
subgroup size is 𝑛 = 4.

2. Using the data provided in the question:

𝑆 98
UCL = 𝑥 + 3 √ = 714 + 3 × = 874
𝑎𝑛 𝑛 0.921 × 2
𝑆 98
LCL = 𝑥 − 3 √ = 714 − 3 × = 554
𝑎𝑛 𝑛 0.921 × 2

3. Since Shewhart charts assume a normal distribution in their derivation, we can use the same
principle to calculate a 𝑧-value, and the fraction of the area under the distribution. But you have
to be careful here: which standard deviation do you use to calculate the 𝑧-value? You should use
the subgroup’s standard deviation, not the process standard deviation. The Shewhart chart
shows the subgroup averages, so the values of 590 and 820 refer to the subgroup values.

If that explanation doesn’t make sense, think of the central limit theorem: the mean of a group of
(︀ )︀
samples, 𝑥 ∼ 𝒩 𝜇, 𝜎 2 /𝑛 , where 𝜎 2 is the process variance, and 𝜎 2 /𝑛 is the subgroup variance of 𝑥.

𝑥low − 𝑥 590 − 714

𝑧low = √ = √ = −2.33
𝜎
^/ 𝑛 106.4/ 4
𝑥high − 𝑥 820 − 714
𝑧high = √ = √ = +2.00
𝜎^/ 𝑛 106.4/ 4

The area below -2.33 is pnorm(-2.33) = 0.009903076, though I will accept any value around
1%, eyeballed from the printed tables. The area below +2.00 is 97.73%, which was on the tables
already. So the total amount of normal operation within the warning limits is 97.73-1.00 = 96.7%.

The asymmetry in their chosen warning limits might be because a violation of the lower bound is
more serious than the upper bound.

ò Question

If an exponentially weighted moving average (EWMA) chart can be made to approximate either a
CUSUM or a Shewhart chart by adjusting the value of 𝜆, what is an advantage of the EWMA chart
over the other two? Describe a specific situation where you can benefit from this.

3.12. Exercises 189

Process Improvement Using Data

ò Solution

The EWMA chart not only provides control limits for monitoring a process, it also provides a
one-step-ahead prediction of the variable being monitored. This is particularly beneficial as the
EWMA chart’s prediction can be used to adjust process conditions, should the prediction show the
process heading towards, or outside, the control limits. This means that changes to the process are
only made if they are required. This is extremely important on slow-moving processes, which are
prone to overly aggressive control.

ò Question

The most recent estimate of the process capability ratio for a key quality variable was 1.30, and the
average quality value was 64.0. Your process operates closer to the lower specification limit of 56.0.
The upper specification limit is 93.0.

What are the two parameters of the system you could adjust, and by how much, to achieve a
capability ratio of 1.67, required by recent safety regulations. Assume you can adjust these
parameters independently.

ò Solution

The process capability ratio for an uncentered process, PCRk , is given by:
(︂ )︂
Upper specification limit − 𝑥 𝑥 − Lower specification limit
PCRk = min ;
3𝜎 3𝜎

The two adjustable parameters are 𝑥, the process target (operating point) and 𝜎, the process
variance. The current process standard deviation is:

64.0 − 56.0
1.30 =
3𝜎
64.0 − 56.0
𝜎= = 2.05
3 × 1.30
• Adjusting the operating point (we would expect to move the operating point away from the LSL):

𝑥 − 56.0
1.67 =
3 × 2.05
𝑥 = 56.0 + 1.67 × 3 × 2.05 = 66.3

So the operating point increases from 64.0 to 66.3 to obtain a higher capability ratio.

• Adjusting the process standard deviation (we would have to assume we can decrease the standard
deviation, keeping the operating point fixed):

64.0 − 56.0
1.67 =
3×𝜎
64.0 − 56.0
𝜎= = 1.60
3 × 1.67
Decrease the process standard deviation from 2.05 to 1.60.

190 Chapter 3. Process Monitoring

 Release 547b16

ò Question

A bagging system fills bags with a target weight of 37.4 grams and the lower specification limit is
35.0 grams. Assume the bagging system fills the bags with a standard deviation of 0.8 grams:

1. What is the current Cpk of the process?

2. To what target weight would you have to set the bagging system to obtain Cpk=1.3?

3. How can you adjust the Cpk to 1.3 without adjusting the target weight (i.e. keep the target
weight at 37.4 grams)?

ò Solution

1. Recall the Cpk is defined relative to the closest specification limit. So in this case it must be due
to the lower limit. Cpk = 𝑥−𝐿𝑆𝐿
3𝜎 = 37.4−35.0
3×0.8 = 1.0

2. To obtain Cpk = 1.3 we solve the above equation for 𝑥 = 1.3 × 3 × 0.8 + 35.0 = 38.12 grams.

3. Changing the lower specification limit is not an option to raise Cpk, because the bags are sold as
containing 35.0 grams of snackfood. Changing the specification limit is in general an artificial
way of changing Cpk. The only practical way to improve Cpk is to decrease the process variance
(e.g. using better equipment with tighter control). The new 𝜎 = 37.4−35.0
3×1.3 = 0.615 grams.

ò Question

Plastic sheets are manufactured on your blown film line. The Cp value is 1.7. You sell the plastic
sheets to your customers with specification of 2 mm ± 0.4 mm.

1. List three important assumptions you must make to interpret the Cp value.

2. What is the theoretical process standard deviation, 𝜎?

3. What would be the Shewhart chart limits for this system using subgroups of size 𝑛 = 4?

4. Illustrate your answer from part 2 and 3 of this question on a diagram of the normal distribution.

ò Solution

1. The notes show that Cp values require us to assume that (a) the process values follow a normal
distribution, the process was centered when the data were collected, and (c) that the process was
stable (use a monitoring chart to verify this last assumption).

2. The range from the lower to the upper specification limit is 0.8 mm, which spans 6 standard
deviations. Given the Cp value of 1.7, the process standard deviation must have been
0.8
𝜎 = 1.7×6 = 0.0784 mm.

3. This time we have the process standard deviation, so there is no need to estimate it from
historical phase 1 data (remember the assumption that Cp and Cpk value are calculated from
stable process operation?). The Shewhart control limits would be: 𝑥 ± 3 × √𝜎𝑛 = 2 ± 3 × 0.0784/2.
The LCL = 1.88 mm and the UCL = 2.12 mm.

3.12. Exercises 191

Process Improvement Using Data

4. An illustration is shown here with the USL, LSL, LCL and UCL, and target values. This question
merely required you to show the LCL and UCL within the LSL and USL, on any normal
distribution curve. However, for illustration, I have added to the diagram the distribution for
the Shewhart chart (thicker line) and distribution for the raw process data (thinner line).

The R code used to generate this figure:

n = 4 # subgroup size
Cp = 1.7
[Link] = 0.8
[Link] = [Link]/Cp/6 # process [Link].
[Link] = 2 # process target
LSL = [Link] - 0.4
USL = [Link] + 0.4
LCL = [Link] - 3 * [Link] / sqrt(n)
UCL = [Link] + 3 * [Link] / sqrt(n)

# A vector of 500 equally spaced points, with 10% offset on either side
x <- seq(LSL-0.1*[Link], USL+0.1*[Link], [Link]/500)
px <- dnorm(x, mean=[Link], sd=[Link])
[Link] <- dnorm(x, mean=[Link], sd=[Link]/sqrt(n))

plot(x, [Link], type="l", xlab=(expression(""*x)), ylab="", [Link]=FALSE,

main=(expression(""*mu*"=2.0")), xlim=c(LSL, USL), [Link]=1.8,
[Link]=1.8, lwd=3, [Link]=1.8, [Link]=1.8, yaxt="n")
lines(x, px, type="l", col="gray50")
abline(v=[Link])

[Link] = max([Link])*0.8
segments(x0=LSL,y0=0, x1=LSL, y1=[Link], col="gray30")
segments(x0=USL,y0=0, x1=USL, y1=[Link], col="gray30")
segments(x0=LCL,y0=0, x1=LCL, y1=[Link], col="red")
segments(x0=UCL,y0=0, x1=UCL, y1=[Link], col="red")
text(LSL, [Link], "LSL", cex=1.3, pos=3)
text(USL, [Link], "USL", cex=1.3, pos=3)
text(LCL, [Link], "LCL", cex=1.3, pos=3)
text(UCL, [Link], "UCL", cex=1.3, pos=3)

192 Chapter 3. Process Monitoring

 Release 547b16

# Sigma for the process

y <- dnorm([Link]+[Link], mean=[Link], sd=[Link])
arrows(x0=[Link], y0=y, x1=[Link]+[Link], y1=y, code=3, angle=15, length=0.1)
text([Link] + [Link], y+0.2, (expression(""*sigma)), cex=1.8, pos=4)

# Sigma for the Shewhart chart

y <- dnorm([Link]+[Link]/sqrt(n), mean=[Link], sd=[Link]/sqrt(n))
arrows(x0=[Link], y0=y, x1=[Link]+[Link]/sqrt(n), y1=y, code=3, angle=15, length=0.1)
text([Link]+[Link]/sqrt(n), y+0.2, (expression(""*sigma/sqrt(n))), cex=1.8, pos=4)

ò Question

The following charts show the weight of feed entering your reactor. The variation in product
quality leaving the reactor was unacceptably high during this period of time.

1. What can your group of process engineers learn about the problem, using the time-series plot
(100 consecutive measurements, taken 1 minute apart).

2. Why is this variability not seen in the Shewhart chart?

3. Using concepts described elsewhere in this book, why might this sort of input to the reactor
have an effect on the quality of the product leaving the reactor?

ò Solution

1. The time-series plot shows a cyclical, almost saw-tooth, pattern in the weight of feed entering. I
would investigate the feeding equipment to see what is leading to these fluctuations in the feed
weight. Perhaps some rotary device is responsible for the periodic variation.

2. The variability is not seen in the Shewhart monitoring chart. The Shewhart chart used subgroups
of size 5 (20 Shewhart samples for 100 time-series samples). These fluctuations obviously cancel
out when calculating the Shewhart subgroups (a limitation of the Shewhart chart).

3.12. Exercises 193

Process Improvement Using Data

3. As engineers we are aiming for stability in our processes; stability in the raw material
characteristics, stability in how we operate the process over time and minimizing as many
disturbances as possible. If we can do this, it will lead to greatly improved consistency in our
products (low output variability). Having this sort of input to the reactor means we have to
provide apply (feedback) control to counteract it. In this case the feedback control may not have
been effective to eliminate the feed variation, or the feedback control itself caused other
disruptions to the process quality.

ò Question

You will come across these terms in the workplace. Investigate one of these topics, using the
Wikipedia link below to kick-start your research. Write a paragraph that (a) describes what your
topic is and (b) how it can be used when you start working in a company after you graduate, or
how you can use it now if you are currently working.

• Lean manufacturing71

• Six sigma72 and the DMAIC cycle. See the list of companies73 that use six sigma tools.

• Kaizen74 (a component of The Toyota Way75 )

• Genchi Genbutsu76 (also a component of The Toyota Way77 )

In early 2010 Toyota experienced some of its worst press coverage on this very topic. Here is
an article78 in case you missed it.

ò Question

The Kappa number is a widely used measurement in the pulp and paper industry. It can be
measured on-line, and indicates the severity of chemical treatment that must be applied to a wood
pulp to obtain a given level of whiteness (i.e. the pulp’s bleachability). Data on the website79
contain the Kappa values from a pulp mill. Use the first 2000 data points to construct a Shewhart
monitoring chart for the Kappa number. You may use any subgroup size you like. Then use the
remaining data as your phase 2 (testing) data. Does the chart perform as expected?

ò Solution

The intention of this question is for you to experience the process of iteratively calculating limits
from phase 1 data and applying them to phase 2 data.

The raw data for the entire data set looks as follows. There are already regions in the phase 2 data

71 [Link]
72 [Link]
73 [Link]
74 [Link]
75 [Link]
76 [Link]
77 [Link]
78 [Link]
79 [Link]

194 Chapter 3. Process Monitoring

 Release 547b16

that we expect to not be from normal operation (around 2500 and 2900)

I used subgroups of size 6 for the figures in this answer, however, the code below is very general,
and you can regenerate the plots if you chose a different subgroup size. Just change one of the lines
near the top.

The upper and lower control limits are calculated, and with a subgroup size of 𝑛 = 6, there are 333
subgroups and the limits are: UCL = 18.26, target = 21.73, and UCL = 25.21. This is illustrated on
the phase 1 data here:

Next we remove the subgroups which lie outside the limits. Please try using the R code to see how
to do it automatically. The new limits, after removing the subgroups beyond the limits from the
first round are: LCL = 18.24, target = 21.71 and UCL = 25.18. They barely changed. But the updated
plot with subgroups removed is now shown below. There is no need to perform another round of
pruning. Only if you used a subgroup size of 4 would you need to do a third round. You could also
have just shifted the limits to a different level, for example, to ±4 standard deviations. We can do
this if we have enough process knowledge to understand the implication of it, in terms of profit.

3.12. Exercises 195

Process Improvement Using Data

Now apply these control limits to the phase 2 data. The plot is shown below:

The limits identify 2 prolonged periods of unusual operation at sequence point 80 and 140. If we
apply the Western Electric rules, we see a third unusual region around sequence step 220. A few
other alarms are scattered in the phase 2 data. About 7% of the subgroups lie outside these control
limits, so these phase 2 data are definitely not from in-control operation; which we expected from
the raw data plot at the start of this question.

The code for all the calculation steps is provided here:

kappa <- [Link]('[Link]

summary(kappa)
attach(kappa) # gives access to the variable "Kappa"

[Link] <- length(Kappa)

N.phase1 <- 2000
N.phase2 <- [Link] - N.phase1
[Link] <- 5

[Link] <- 1
[Link] <- floor(N.phase1/[Link])
[Link] <- [Link] + 1
[Link] <- floor([Link]/[Link])

# Plot all the data

plot(Kappa, type="p", pch=".", cex=2, main="", ylab="Kappa number: raw data",
xlab="Sequence order")
abline(v=N.phase1, col="gray50")
text(N.phase1/2, 10, "Phase I data", col="blue")

196 Chapter 3. Process Monitoring

 Release 547b16

text(N.phase1 + N.phase2/2, 10, "Phase II data", col="blue")

# We won't check the phase I raw data for outliers; we will use the phase I
# subgroups to check for outliers.

# Create the subgroups on ALL the raw data. Form a matrix with `[Link]` rows
# placing the vector of data down each row, then going across to form the columns.

# Calculate the mean and standard deviation within each subgroup (columns of the matrix)
[Link] <- matrix(Kappa, [Link], [Link]/[Link])
[Link] <- apply([Link], 2, mean)
subgroup.S <- apply([Link], 2, sd)

[Link] <- [Link][[Link]:[Link]]

phase1.S <- subgroup.S[[Link]:[Link]]
[Link] <- [Link][[Link]:[Link]]
phase2.S <- subgroup.S[[Link]:[Link]]

# We are going to repeatedly have to calculate the phase 1 limits. Create a function.
shewhart_limits <- function(xbar, S, [Link], [Link]=3){
# Give the xbar and S vector containing the subgroup means and standard deviations.
# Also give the subgroup size used. Returns the lower and upper control limits
# for the Shewhart chart (UCL and LCL) which are [Link] away from the target.

[Link] <- mean(xbar)

[Link] <- mean(S)
an = c(NA, 0.793, 0.886, 0.921, 0.940, 0.952, 0.959, 0.965)
LCL <- [Link] - 3*[Link]/an[[Link]]/sqrt([Link])
UCL <- [Link] + 3*[Link]/an[[Link]]/sqrt([Link])
c(LCL, UCL)

return(list(LCL, [Link], UCL))

}
limits <- shewhart_limits([Link], phase1.S, [Link])
LCL <- limits[1]
xdb <- limits[2]
UCL <- limits[3]
c(LCL, xdb, UCL)

# Any points outside these limits? Yup, quite a few.

plot([Link], type="b", pch=".", cex=5, main="", ylab="Phase I subgroups: round 1",
xlab="Sequence order")
abline(h=UCL, col="red")
abline(h=LCL, col="red")
abline(h=xdb, col="green")
lines([Link], type="b", pch=".", cex=5)

# Find the indices of the point outside the limits. You could use the identify function,
# or you can find them programatically, using boolean (logical) vectors. Take a look
# at what the variables "outside" and "inside" look like to understand what they do.
outside <- ([Link] > UCL) + ([Link] < LCL)
outside <- [Link](outside)
inside <- !outside

# Now use only the data inside the existing limits to recalculate the phase I limits.
[Link] <- [Link][inside]
phase1.S <- phase1.S[inside]
limits <- shewhart_limits([Link], phase1.S, [Link])
LCL <- limits[1]
xdb <- limits[2]
UCL <- limits[3]
c(LCL, xdb, UCL)

# Replot the data: everything is inside the limits this time

plot([Link], type="b", pch=".", cex=5, main="", ylab="Phase I subgroups: round 2",
xlab="Sequence order")
abline(h=UCL, col="red")
abline(h=LCL, col="red")

3.12. Exercises 197

Process Improvement Using Data

abline(h=xdb, col="green")
lines([Link], type="b", pch=".", cex=5)

outside <- ([Link] > UCL) + ([Link] < LCL)

sum(outside) # yay, it is zero!

# Using subgroups of size 4 or smaller will require an additional

# round of pruning subgroups
outside <- [Link](outside)
inside <- !outside
[Link] <- [Link][inside]
phase1.S <- phase1.S[inside]
limits <- shewhart_limits([Link], phase1.S, [Link])
LCL <- limits[1]
xdb <- limits[2]
UCL <- limits[3]
c(LCL, xdb, UCL)

# Replot the data: everything is inside the limits after the second
# or third round of pruning.
plot([Link], type="b", pch=".", cex=5, main="", ylab="Phase I subgroups: round 3",
xlab="Sequence order")
abline(h=UCL, col="red")
abline(h=LCL, col="red")
abline(h=xdb, col="green")
lines([Link], type="b", pch=".", cex=5)

outside <- ([Link] > UCL) + ([Link] < LCL)

sum(outside) # yeah, it is zero!

# Now test the Shewhart limits on the phase II data

plot([Link], type="b", pch=".", cex=5, main="", ylab="Phase II subgroups: testing",
xlab="Sequence order")
abline(h=UCL, col="red")
abline(h=LCL, col="red")
abline(h=xdb, col="green")
lines([Link], type="b", pch=".", cex=5)
outside.phase2 <- ([Link] > UCL) + ([Link] < LCL)
alpha <- sum(outside.phase2) / length([Link])
alpha

# Alpha = 7.9% for this phase 2 data, much higher than the 0.27% expected for
# 3 sigma limits to be expected, because there are process problems on at
# least 3 occasions in this phase 2 data.

ò Question

In this section we showed how one can monitor any variable in a process. Modern instrumentation
though capture a wider variety of data. It is common to measure point values, e.g. temperature,
pressure, concentration and other hard-to-measure values. But it is increasingly common to
measure spectral data. These spectral data are a vector of numbers instead of a single number.

Below is an example from a pharmaceutical process: a complete spectrum can be acquired many
times per minute, and it gives a complete chemical fingerprint or signature of the system. There are
460 spectra in figure below; they could have come, for example, from a process where they are
measured 5 seconds apart. It is common to find fibre optic probes embedded into pipelines and
reactors to monitor the progress of a reaction or mixing.

Write a few bullet points how you might monitor a process where a spectrum (a vector) is your
data source, and not a “traditional” single point measurement, like a temperature value.

198 Chapter 3. Process Monitoring

 Release 547b16

ò Solution

A complete spectrum (vector) of values is obtained with every observation. To monitor a process
using one of the charts learned about so far (Shewhart, CUSUM, or EWMA chart) we have to
reduce this vector down to a single number. Any of these methods will do:

• Use a single point at a particular wavelength in the spectrum (e.g. the peak at 1200 nm or 1675
nm).

• Use a weighted sum of a region of the spectrum, or the integrated area under a region in the
spectrum (these 2 approaches are similar/equivalent)

• Use the spectrum to predict a certain property of interest, and then monitor that property
instead. For example: use the spectrum to predict the colour of cookies (i.e. how well baked they
are) and monitor the “well-bakedness” characteristic.

Later on we will learn about multivariate monitoring methods (page 495).

ò Question

The carbon dioxide measurement is available from a gas-fired furnace80 . These data are from phase
1 operation.

1. Calculate the Shewhart chart upper and lower control limits that you would use during phase 2
with a subgroup size of 𝑛 = 6.

2. Is this a useful monitoring chart? What is going in this data?

3. How can you fix the problem?

80 [Link]

3.12. Exercises 199

Process Improvement Using Data

ò Solution

Solution based on work by Ryan and Stuart (2011 class)

First a plot of the raw data will be useful:

1. Assuming that the CO2 data set is from phase 1 operation, the control limits were calculated as
follows:

• Assume subgroups are independent

𝐾
¯= 1
∑︀
• 𝑥 𝐾 𝑥
¯𝑘 = 53.5
𝑘=1

𝐾
• 𝑆¯ = 1
∑︀
𝐾 𝑠𝑘 = 1.10
𝑘=1

• 𝑎𝑛 = 0.952
1.10√
• LCL = 53.5 − 3 · 0.952 6
= 52.08
1.10√
• UCL = 53.5 + 3 · 0.952 6
= 54.92

2. The Shewhart chart using a subgroup of size 6 is not a useful monitoring chart. There are too
many false alarms, which will cause the operators to just ignore the chart. The problem is that
the first assumption of independence is not correct and has a detrimental effect, as shown in a
previous question (page 138).

200 Chapter 3. Process Monitoring

 Release 547b16

3. One approach to fixing the problem is to subsample the data, i.e. only use every 𝑘 th data point as
the raw data, e.g. 𝑘 = 10, and then form subgroups from that sampled data.

Another is to use a larger subgroup size. Use the autocorrelation function81 , and the
corresponding acf(...) function in R to verify the degree of relationship. Using this function
we can see the raw data are unrelated after the 17th lag, so we could use subgroups of that size.
However, even then we see the Shewhart chart showing frequent violation, though fewer than
before.

Yet another alternative is to use an EWMA chart, which takes the autocorrelation into account.
However, the EWMA chart limits are found from the assumption that the subgroup means (or
raw data, if subgroup size is 1), are independent.

So we are finally left with the conclusion that perhaps there data really are not from in control
operation, or, if they are, we must manually adjust the limits to be wider.

import numpy as np
import pandas as pd
from [Link] import gamma
from [Link] import plot_acf

[Link] = "plotly"

file = "[Link]
data = pd.read_csv(file)
CO2 = data["CO2"].to_numpy()
N_raw = len(CO2)
N_sub = 6

# Change N_sub to 10, 15, 20, etc.

# At N_sub = 17 we see the
# autocorrelation disappear.

# Plot all the data.

[Link](CO2).[Link]().update_layout(
xaxis_title_text="Sequence order",
yaxis_title_text="CO2: raw data").show()

# Create the subgroups on ALL the raw data.

# Reshape into N_sub rows by N_raw/N_sub
# columns; each column is one subgroup.
# Calculate the mean and standard deviation
# within each subgroup.
subgroups = [Link](N_raw // N_sub, N_sub).T
subgroups_S = [Link](axis=0, ddof=1)
subgroups_xbar = [Link](axis=0)
ylim = (subgroups_xbar.min() - 3,
subgroups_xbar.max() + 3)

# Keep adjusting N_sub until you don't see

# any autocorrelation between subgroups.
plot_acf(subgroups_xbar)

# Create a function to calculate Shewhart

# chart limits.
def shewhart_limits(xbar, S, sub_n,
N_stdev=3):
"""Return (LCL, xdb, UCL) which are
N_stdev away from the target, given the
subgroup means xbar, subgroup standard
deviations S, and subgroup size sub_n."""
xdb = [Link]() # x-double-bar
s_bar = [Link]()
num_an = [Link](2) * gamma(sub_n / 2)
den_an = [Link](sub_n - 1) * gamma(

3.12. Exercises 201

Process Improvement Using Data

(sub_n - 1) / 2)
an = num_an / den_an
LCL = xdb - N_stdev * s_bar / (
an * [Link](sub_n))
UCL = xdb + N_stdev * s_bar / (
an * [Link](sub_n))
return LCL, xdb, UCL

LCL, xdb, UCL = shewhart_limits(

subgroups_xbar, subgroups_S, N_sub)
(LCL, xdb, UCL)

# Any points outside these limits?

fig = [Link](subgroups_xbar).[Link](
title="Phase I subgroups: round 1")
fig.update_layout(
xaxis_title_text="Sequence order",
yaxis=dict(range=ylim))
fig.add_hline(y=UCL, line_color="red")
fig.add_hline(y=LCL, line_color="red")
fig.add_hline(y=xdb, line_color="green")
[Link]()

file <- '[Link]

data <- [Link](file)
CO2 <- data$CO2
[Link] <- length(CO2)
[Link] <- 6

# Change ``[Link]`` to 10, 15, 20, etc

# At [Link] <- 17 we see the
# autocorrelation disappear

# Plot all the data

par(mar=c(4.2, 4.2, 0.5, 0.5))
par([Link]=1.3, [Link]=1.5,
[Link]=1.5, [Link]=1.5)
plot(CO2, type="p", pch=".", cex=2,
main="", ylab="CO2: raw data",
xlab="Sequence order")

# Create the subgroups on ALL the raw data.

# Form a matrix with `[Link]` rows by
# placing the vector of data down each row,
# then going across to form the columns.
# Calculate the mean and standard deviation
# within each subgroup (columns of the matrix)

subgroups <- matrix(CO2, [Link], [Link]/[Link])

subgroups.S <- apply(subgroups, 2, sd)
[Link] <- apply(subgroups, 2, mean)
ylim <- range([Link]) + c(-3, +3)

# Keep adjusting [Link] until you don't see

# any autocorrelation between subgroups
acf([Link])

# Create a function to calculate

# Shewhart chart limits
shewhart_limits <- function(xbar, S,
sub.n, [Link]=3){
# Give the xbar and S vector containing
# the subgroup means and standard
# deviations. Also give the subgroup
# size used. Returns the lower and upper
# control limits for the Shewhart chart
# (UCL and LCL) which are [Link] away

202 Chapter 3. Process Monitoring

 Release 547b16

# from the target.

# xdb = [Link] = mean of means

xdb <- mean(xbar)
[Link] <- mean(S)
[Link] <- sqrt(2)*gamma(sub.n/2)
[Link] <- sqrt(sub.n-1)*gamma((sub.n-1)/2)
an <- [Link] / [Link]
LCL <- xdb - 3*[Link]/(an*sqrt(sub.n))
UCL <- xdb + 3*[Link]/(an*sqrt(sub.n))
return(list(LCL, xdb, UCL))
}

limits <- shewhart_limits([Link],

subgroups.S, [Link])
LCL <- limits[1]
xdb <- limits[2]
UCL <- limits[3]
c(LCL, xdb, UCL)

# Any points outside these limits?

par(mar=c(4.2, 4.2, 0.5, 0.5))
par([Link]=1.3, [Link]=1.5,
[Link]=1.5, [Link]=1.5)
plot([Link], type="b", pch=".",
cex=5, main="", ylim=ylim,
ylab="Phase I subgroups: round 1",
xlab="Sequence order")
abline(h=UCL, col="red")
abline(h=LCL, col="red")
abline(h=xdb, col="green")
lines([Link], type="b", pch=".",
cex=5)

ò Question

The percentage yield from a batch reactor, and the purity of the feedstock are available as the Batch
yield and purity82 data set. Assume these data are from phase 1 operation and calculate the
Shewhart chart upper and lower control limits that you would use during phase 2. Use a subgroup
size of 𝑛 = 3.

1. What is phase 1?

2. What is phase 2?

3. Show your calculations for the upper and lower control limits for the Shewhart chart on the yield
value.

4. Show a plot of the Shewhart chart on these phase 1 data.

ò Solution

Solution based on work by Ryan McBride, Stuart Young, and Mudassir Rashid (2011 class)

1. Phase 1 is the period from which historical data is taken that is known to be “in control”. From

81 [Link]
82 [Link]

3.12. Exercises 203

Process Improvement Using Data

this data, upper and lower control limits can be established for the monitored variable that
contain a specified percent of all in control data.

2. Phase 2 is the period during which new, unseen data is collected by process monitoring in
real-time. This data can be compared with the limits calculated from the “in control” data.

3. Assuming the dataset was derived from phase 1 operation, the batch yield data was grouped
into subgroups of size 3. However, since the total number of data points (N=241) is not a
multiple of three, the data set was truncated to the closest multiple of 3, i.e. 𝑁𝑛𝑒𝑤 = 240, by
removing the last data point. Subsequently, the mean and standard deviation were calculated
for each of the 80 subgroups. From this data, the lower and upper control limits were calculated
as follows:

80
1 ∑︁
𝑥= 𝑥𝑘 = 75.3
80
𝑘=1
80
1 ∑︁
𝑆= 𝑠𝑘 = 5.32
80
𝑘=1
𝑆
LCL = 𝑥 − 3 · √ = 64.9
𝑎𝑛 𝑛
𝑆
UCL = 𝑥 + 3 · √ = 85.7
𝑎𝑛 𝑛
using 𝑎𝑛 = 0.886 for a subgroup size of 3
and 𝑥 = 75.3

Noticing that the mean for subgroup 42, 𝑥42 = 63.3, falls below this LCL, the control limits were
recalculated excluding this subgroup from phase 1 data (see R-code). Following this adjustment,
the new control limits were calculated to be:

• LCL = 65.0

• UCL = 85.8

4. Shewhart charts for both rounds of the yield data (before and after removing the outlier):

204 Chapter 3. Process Monitoring

 Release 547b16

import numpy as np
import pandas as pd
from [Link] import gamma

[Link] = "plotly"

# Recursively calculate the Shewhart limits,

# trimming subgroups outside the limits each
# round, until no more points are excluded.
file = ("[Link]
"[Link]")
data = pd.read_csv(file)
y = data["yield"].to_numpy()
variable = "Yield"
N = 3

# No further changes required. The code

# below will work for any new data set.
# Truncate to the closest multiple of N
# so the reshape works.
y = y[: (len(y) // N) * N]
subgroups = [Link](len(y) // N, N).T
x_mean = [Link](axis=0)
x_sd = [Link](axis=0, ddof=1)
ylim = (x_mean.min() - 5, x_mean.max() + 5)

num_an = [Link](2) * gamma(N / 2)

den_an = [Link](N - 1) * gamma((N - 1) / 2)
an = num_an / den_an

k = 1
doloop = True
# Prevent infinite loops.
while doloop and k < 5:
S = x_sd.mean()
xdb = x_mean.mean() # x-double-bar
LCL = xdb - (3 * S / (an * [Link](N)))
UCL = xdb + (3 * S / (an * [Link](N)))
print((LCL, UCL))

fig = [Link](x_mean).[Link](
title=f"Phase I subgroups: round {k}")
fig.update_layout(
xaxis_title_text="Sequence order",
yaxis=dict(range=ylim))
fig.add_hline(y=UCL, line_color="red")
fig.add_hline(y=LCL, line_color="red")
fig.add_hline(y=xdb, line_color="green")
[Link]()

3.12. Exercises 205

Process Improvement Using Data

if not ((x_mean < LCL).any()

or (x_mean > UCL).any()):
# Finally! No more points to exclude.
doloop = False
k += 1

# Keep only subgroups whose mean falls

# inside the current control limits.
keep = (x_mean >= LCL) & (x_mean <= UCL)
x_sd = x_sd[keep]
x_mean = x_mean[keep]

# Thanks to Mudassir for his source code to

# recursively calculate the limits. Some
# updates were made.

file <- '[Link]

data <- [Link](file)
y <- data$yield
variable <- "Yield"
N <- 3

# No further changes required. The code

# below will work for any new data set
subgroups <- matrix(y, N, length(y)/N)
[Link] <- numeric(length(y)/N)
[Link] <- numeric(length(y)/N)

# Calculate mean and sd of subgroups

# (see R-tutorial)
[Link] <- apply(subgroups, 2, mean)
[Link] <- apply(subgroups, 2, sd)
ylim <- range([Link]) + c(-5, +5)
k <- 1
doloop <- TRUE

# Prevent infinite loops

while (doloop & k < 5){

[Link] <- sqrt(2)*gamma(N/2)

[Link] <- sqrt(N-1)*gamma((N-1)/2)
an <- [Link] / [Link]

S <- mean([Link])
xdb <- mean([Link]) # x-double bar
LCL <- xdb - (3*S/(an*sqrt(N)))
UCL <- xdb + (3*S/(an*sqrt(N)))
print(c(LCL, UCL))

# Create a figure on every loop

par(mar=c(4.2, 4.2, 0.5, 0.5))
par([Link]=1.3, [Link]=1.5,
[Link]=1.5, [Link]=1.5)
plot([Link], type="b", pch=".",
cex=5, main="",
ylab=paste("Phase I subgroups: round", k),
xlab="Sequence order", ylim=ylim)
abline(h=UCL, col="red")
abline(h=LCL, col="red")
abline(h=xdb, col="green")
lines([Link], type="b", pch=".", cex=5)

if (!(any([Link] < LCL) | any([Link] > UCL))){

# Finally! No more points to exclude
doloop <- FALSE
}
k <- k + 1

206 Chapter 3. Process Monitoring

 Release 547b16

# Retain in [Link] and [Link] only those

# entries that are within the control
# limits
[Link] <- [Link][[Link]>=LCL]
[Link] <- [Link][[Link]>=LCL]
[Link] <- [Link][[Link]<=UCL]
[Link] <- [Link][[Link]<=UCL]
} # end: while doloop

ò Question

You will hear about 6-sigma processes frequently in your career. What does it mean exactly that a
process is “6-sigma capable”? Draw a diagram to help illustrate your answer.

3.12. Exercises 207

Process Improvement Using Data

208 Chapter 3. Process Monitoring

 CHAPTER 4
LEAST SQUARES MODELLING REVIEW

4.1 Least squares modelling in context

This section begins a new part: we start considering more than one variable at a time. However, you
will see the tools of confidence intervals and visualization from the previous sections coming into play
so that we can interpret our least squares models both analytically and visually.

The following sections, on design and analysis of experiments and latent variable models, will build
on the least squares model we learn about here.

4.1.1 Usage examples

Video for
this section The material in this section is used whenever you need to interpret and quantify the relationship
between two or more variables.

• Colleague: How is the yield from our lactic acid batch fermentation related to the purity of the
sucrose substrate?

You: The yield can be predicted from sucrose purity with an error of plus/minus 8%

Colleague: And how about the relationship between yield and glucose purity?

You: Over the range of our historical data, there is no discernible relationship.

• Engineer 1: The theoretical equation for the melt index is non-linearly related to the viscosity

Engineer 2: The linear model does not show any evidence of that, but the model’s prediction ability
does improve slightly when we use a non-linear transformation in the least squares model.

• HR manager: We use a least squares regression model to graduate personnel through our pay
grades. The model is a function of education level and number of years of experience. What do the
model coefficients mean?

209
 Process Improvement Using Data

4.1.2 What you will be able to do after this section

4.2 References and readings

This section is only a simple review of the least squares model. More details may be found in these
references.

• Recommended: John Fox, Applied Regression Analysis and Generalized Linear Models, Sage.

• Recommended: N.R. Draper and H. Smith, Applied Regression Analysis, Wiley.

• Box, Hunter and Hunter, Statistics for Experimenters, selected portions of Chapter 10 (2nd edition),
Wiley.

• Hogg and Ledolter, Applied Statistics for Engineers and Physical Scientists, Prentice Hall.

• Montgomery and Runger, Applied Statistics and Probability for Engineers, Wiley.

4.3 Covariance
Video for
this section You probably have an intuitive sense for what it means when two things are correlated. We will get to
correlation next, but we start by first looking at covariance. Let’s take a look at an example to formalize
this, and to see how we can learn from data.

Consider the measurements from a gas cylinder; temperature (K) and pressure (kPa). We know the
ideal gas law applies under moderate conditions: 𝑝𝑉 = 𝑛𝑅𝑇 .

• Fixed volume, 𝑉 = 20 × 10−3 m3 = 20 L

• Moles of gas, 𝑛 = 14.1 mols of chlorine gas, molar mass = 70.9 g/mol, so this is 1 kg of gas

• Gas constant, 𝑅 = 8.314 J/(mol.K)

𝑛𝑅
Given these numbers, we can simplify the ideal gas law to: 𝑝 = 𝛽1 𝑇 , where 𝛽1 = > 0. These data
𝑉
are collected from sampling the system:

210 Chapter 4. Least Squares Modelling Review

 Release 547b16

The formal definition for covariance between any two variables is: [terminology used here was
defined in a previous section (page 47)]

Cov {𝑥, 𝑦} = ℰ {(𝑥 − 𝑥)(𝑦 − 𝑦)} where ℰ {𝑧} = 𝑧 (4.1)

Use this to calculate the covariance between temperature and pressure by breaking the problem into
steps:

• First calculate deviation variables. They are called this because they are now the deviations from the
mean: 𝑇 − 𝑇 and 𝑝 − 𝑝. Subtracting off the mean from each vector just centers their frame of
reference to zero.

• Next multiply the two vectors, element-by-element, to calculate a new vector (𝑇 − 𝑇 )(𝑝 − 𝑝).

import numpy as np

temp = [Link]([273, 285, 297, 309, 321, 333,

345, 357, 369, 381])
pres = [Link]([1600, 1670, 1730, 1830, 1880,
1920, 2000, 2100, 2170, 2200])
humidity = [Link]([42, 48, 45, 49, 41, 46,
48, 48, 45, 49])

temp_centered = temp - [Link]()

pres_centered = pres - [Link]()
product = temp_centered * pres_centered

# numpy does element-by-element multiplication.

print(product)
# [16740 10080 5400 1440 180
# 60 1620 5700 10920 15660]

# Average of `product`:
[Link]() # 6780
(continues on next page)

4.3. Covariance 211

Process Improvement Using Data

(continued from previous page)

# [Link] returns the covariance matrix; the

# off-diagonal entry [0, 1] is Cov{temp, pres}
# (with N-1 normalisation, matching R).
# Calculated covariance is 7533.33
print("Covariance of temperature and "
"pressure is = "
f"{round([Link](temp, pres)[0, 1], 2)}")

# The covariance of a variable with

# itself is just the variance:
print("Covariance with itself is = "
f"{round([Link](temp, temp)[0, 1], 2)}")
print("while the variance = "
f"{round([Link](ddof=1), 2)}")

temp <- c(273, 285, 297, 309, 321, 333,

345, 357, 369, 381)
pres <- c(1600, 1670, 1730, 1830, 1880,
1920, 2000, 2100, 2170, 2200)
humidity <- c(42, 48, 45, 49, 41, 46,
48, 48, 45, 49)

[Link] <- temp - mean(temp)

[Link] <- pres - mean(pres)
product <- [Link] * [Link]

# R does element-by-element multiplication in the above line

print(product)
# [1] 16740 10080 5400 1440 180
# 60 1620 5700 10920 15660

# Average of 'product':
mean(product) # 6780

# Calculated covariance is 7533.33

paste0('Covariance of temperature and ',
'pressure is = ',
round(cov(temp, pres), 2))

# The covariance of a variable with

# itself is just the variance:
paste0('Covariance with itself is = ',
round(cov(temp, temp), 2))
paste0('while the variance = ',
round(var(temp), 2))

• The expected value of this product can be estimated by using the average, or any other suitable
measure of location. In this case mean(product) in R gives 6780. This is the covariance value.

• More specifically, we should provide the units as well: the covariance between temperature and
pressure is 6780 [[Link]] in this example. Similarly the covariance between temperature and
humidity is 35.4 [K.%].

In your own time calculate a rough numeric value and give the units of covariance for these cases:

212 Chapter 4. Least Squares Modelling Review

 Release 547b16

𝑥 𝑦
𝑥 = age of married partner 1 𝑦 = age of married partner 2
𝑥 = gas pressure 𝑦 = gas volume at a fixed temperature
𝑥 = mid term mark for this course 𝑦 = final exam mark
𝑥 = hours worked per week 𝑦 = weekly take home pay
𝑥 = cigarettes smoked per month 𝑦 = age at death
𝑥 = temperature on top tray of distillation column 𝑦 = top product purity

Also describe what an outlier observation would mean in these cases.

One last point is that the covariance of a variable with itself is the variance:
Cov {𝑥, 𝑥} = 𝒱(𝑥) = ℰ {(𝑥 − 𝑥)(𝑥 − 𝑥)}, a definition we saw earlier (page 47).

Using the cov(temp, pres) function in R gives 7533.333, while we calculated 6780. The
𝑁
difference comes from 6780 × = 7533.33, indicating that R divides by 𝑁 − 1 rather than 𝑁 . This
𝑁 −1
is because the variance function in R for a vector x is internally called as cov(x, x). Since R returns
the unbiased variance, it divides through by 𝑁 − 1. This inconsistency does not really matter for large
values of 𝑁 , but emphasizes that one should always read the documentation for the software being
used.

Note that deviation variables are not affected by a shift in the raw data of 𝑥 or 𝑦. For example,
measuring temperature in Celsius or Kelvin has no effect on the covariance number; but measuring it
in Celsius vs Fahrenheit does change the covariance value.

4.4 Correlation
Video for
this section The variance and covariance values are units dependent. For example, you get a very different
covariance when calculating it using grams vs kilograms. The correlation on the other hand removes
the effect of scaling and arbitrary unit changes. It is defined as:

ℰ {(𝑥 − 𝑥)(𝑦 − 𝑦)} Cov {𝑥, 𝑦}

Correlation = 𝑟(𝑥, 𝑦) = √︀ = √︀ (4.2)
𝒱 {𝑥} 𝒱 {𝑦} 𝒱 {𝑥} 𝒱 {𝑦}

It takes the covariance value and divides through by the units of 𝑥 and of 𝑦 to obtain a dimensionless
result. The values of 𝑟(𝑥, 𝑦) range from −1 to +1. Also note that 𝑟(𝑥, 𝑦) = 𝑟(𝑦, 𝑥).

So returning back to our example of the gas cylinder, the correlation between temperature and
pressure, and temperature and humidity can be calculated now as:

import numpy as np

temp = [Link]([273, 285, 297, 309, 321, 333, 345,

357, 369, 381])
pres = [Link]([1600, 1670, 1730, 1830, 1880, 1920,
2000, 2100, 2170, 2200])
humidity = [Link]([42, 48, 45, 49, 41, 46, 48,
48, 45, 49])

# [Link] returns the correlation matrix; the

# off-diagonal entry [0, 1] is r(x, y).

# Correlation between temperature

# and pressure is high: 0.9968355
[Link](temp, pres)[0, 1]
(continues on next page)

4.4. Correlation 213

Process Improvement Using Data

(continued from previous page)

# Correlation between temperature

# and humidity is low: 0.3803919
[Link](temp, humidity)[0, 1]

# What is correlation of humidity

# and pressure?
[Link](___, ___)[0, 1]

temp <- c(273, 285, 297, 309, 321, 333, 345,

357, 369, 381)
pres <- c(1600, 1670, 1730, 1830, 1880, 1920,
2000, 2100, 2170, 2200)
humidity <- c(42, 48, 45, 49, 41, 46, 48,
48, 45, 49)

# Correlation between temperature

# and pressure is high: 0.9968355
cor(temp, pres)

# Correlation between temperature

# and humidity is low: 0.3803919
cor(temp, humidity)

# What is correlation of humidity

# and pressure?
cor(___, ___)

Note that correlation is the same whether we measure temperature in Celsius or Kelvin. Study the
plots here to get a feeling for the correlation value and its interpretation:

214 Chapter 4. Least Squares Modelling Review

 Release 547b16

4.4.1 Visualizing the correlation matrix

When working with a real data set that has many variables, looking at correlations one pair at a time is
tedious. Pandas can compute every pairwise correlation in one call with .corr(), and we can
visualize the resulting matrix as a heat map to spot patterns at a glance.

The example below uses a flotation cell83 data set:

import pandas as pd

flot = pd.read_csv("[Link]

# A square matrix of correlation values, one

# per pair of numeric columns:
[Link]()

For a wider data set, the numeric matrix becomes hard to read; a heat map encodes the same
83 [Link]

4.4. Correlation 215

Process Improvement Using Data

information visually. Here we use a distillation column84 data set, where the goal is to predict
VapourPressure from process measurements:

import pandas as pd
import seaborn as sns

distill = pd.read_csv(
"[Link]
)

# Print the correlation matrix as numbers:

display([Link]())

# A diverging palette is helpful: red for

# positive, blue for negative correlations,
# white for near-zero.
cmap = sns.diverging_palette(220, 10, as_cmap=True)
[Link](rc={"[Link]": (15, 15)})
[Link](
[Link](),
cmap=cmap,
square=True,
linewidths=0.2,
cbar_kws={"shrink": 0.5},
)

A complementary view is the scatter plot matrix, which shows the actual data behind each correlation.
The diagonal panels are replaced by a kernel density estimate (kde) of each variable’s distribution:

from [Link] import scatter_matrix

scatter_matrix(
distill,
alpha=0.2,
figsize=(15, 15),
diagonal="kde",
)

# For large data sets, sub-sample to speed

# up the plot, e.g. every second row:
scatter_matrix(
[Link][0::2, :],
alpha=0.2,
figsize=(15, 15),
diagonal="kde",
)

When the goal is to model a particular outcome variable (here VapourPressure), the most useful
slice of the correlation matrix is its last row: it ranks every potential predictor by how strongly it
correlates with the outcome.

# The last row of the correlation matrix shows

# how each x-variable correlates with the
# outcome variable VapourPressure.
[Link]().iloc[-1, :]

84 [Link]

216 Chapter 4. Least Squares Modelling Review

 Release 547b16

4.4.2 What does a near-zero correlation look like?

The cylinder-pressure, distillation-tower and flotation-cell examples above all show strong
correlations. It is just as important to develop intuition for what 𝑟 ≈ 0 looks like in real data. The
unlimited-time-test85 data set is a useful counter-example: students were given as much time as they
wanted to write an open-book exam, and we have two columns: Time taken to finish, and the Grade
achieved.

import pandas as pd

grades = pd.read_csv(
"[Link]
)
[Link](x="Time", y="Grade", figsize=(8, 6))

# Correlation of -0.044 -- essentially zero.

[Link]()

The scatter plot shows no visible pattern, and the correlation matrix confirms it: 𝑟 = −0.044. Two
things are worth noting from this example:

• The correlation is symmetric: 𝑟(Time, Grade) = 𝑟(Grade, Time) = −0.044. There is no notion of an 𝑥-
or 𝑦-variable yet.

• The 𝑅2 value commonly reported with a least-squares model is just 𝑟2 . So without fitting any model
we already know that a straight line through these data would have 𝑅2 = (−0.044)2 ≈ 0.002. Said
differently: Time would explain about 0.2% of the variation in Grade. This is one reason 𝑅2 on its
own is a poor way to judge a regression model.

Compare this against the cheddar-cheese exercise (𝑟 around 0.5 to 0.8 between flavour and the
chemical predictors), and against the cylinder-pressure example above where 𝑟 = 0.997. Looking at
the scatter plots alongside the correlation values for these three regimes — near-zero, moderate, near-one
— is the fastest way to develop a calibrated visual sense of what a correlation coefficient really means.

4.5 Some definitions

Be sure that you can derive (and interpret!) these relationships, which are derived from the definition
of the covariance and correlation:

• ℰ{𝑥} = 𝑥

• ℰ{𝑥 + 𝑦} = ℰ{𝑥} + ℰ{𝑦} = 𝑥 + 𝑦

• 𝒱{𝑥} = ℰ{(𝑥 − 𝑥)2 }

• 𝒱{𝑐𝑥} = 𝑐2 𝒱{𝑥}

• Cov{𝑥, 𝑦} = ℰ{(𝑥 − 𝑥)(𝑦 − 𝑦)} which we take as the definition for covariance

• 𝒱{𝑥 + 𝑥} = 2𝒱{𝑥} + 2Cov{𝑥, 𝑥} = 4𝒱{𝑥}

• Cov{𝑥, 𝑦} = ℰ{𝑥𝑦} − ℰ{𝑥}ℰ{𝑦}

• Cov{𝑥, 𝑐} = 0

• Cov{𝑥 + 𝑎, 𝑦 + 𝑏} = Cov{𝑥, 𝑦}
85 [Link]

4.5. Some definitions 217

Process Improvement Using Data

• Cov{𝑎𝑥, 𝑏𝑦} = 𝑎𝑏 · Cov{𝑥, 𝑦}

• 𝒱{𝑥 + 𝑦} =
̸ 𝒱{𝑥} + 𝒱{𝑦}, which is counter to what might be expected.

• Rather:
2
𝒱{𝑥 + 𝑦} = ℰ{(𝑥 + 𝑦 − 𝑥 − 𝑦) }
2
= ℰ{((𝑥 − 𝑥) + (𝑦 − 𝑦)) }
= ℰ{(𝑥 − 𝑥)2 + 2(𝑥 − 𝑥)(𝑦 − 𝑦) + (𝑦 − 𝑦)2 }
(4.3)
= ℰ{(𝑥 − 𝑥)2 } + 2ℰ{(𝑥 − 𝑥)(𝑦 − 𝑦)} + ℰ{(𝑦 − 𝑦)2 }
= 𝒱{𝑥} + 2Cov{𝑥, 𝑦} + 𝒱{𝑦}
𝒱{𝑥 + 𝑦} = 𝒱{𝑥} + 𝒱{𝑦}, only if 𝑥 and 𝑦 are independent

4.6 Least squares models with a single x-variable

The general linear least squares model is a very useful tool (in the right circumstances), and it is the
workhorse for a number of algorithms in data analysis.

This part covers the relationship between two variables only: 𝑥 and 𝑦. In a later part on general least
squares (page 253) we will consider more than two variables and use matrix notation. But we start off
slowly here, looking first at the details for relating two variables.

We will follow these steps:

1. Model definition (this subsection)

2. Building the model

3. Interpretation of the model parameters and model outputs (coefficients, 𝑅2 , and standard error 𝑆𝐸 )

4. Consider the effect of unusual and influential data

5. Assessment of model residuals

The least squares model postulates that there is a linear relationship between measurements in vector
𝑥 and vector 𝑦 of the form:

ℰ {y} = 𝛽0 + 𝛽1 x
(4.4)
y = 𝛽0 + 𝛽1 x + 𝜖

The 𝛽0 , 𝛽1 and 𝜖 terms are population parameters, which are unknown (see the section on univariate
statistics (page 46)). The 𝜖 term represents any unmodelled components of the linear model,
measurement error, and is simply called the error term. Notice that the error is not due to 𝑥, but is the
error in fitting 𝑦; we will return to this point in the section on least squares assumptions (page 228). Also,
if there is no relationship between 𝑥 and 𝑦 then 𝛽1 = 0.

We develop the least squares method to estimate these parameters; these estimates are defined as
𝑏0 = 𝛽^0 , 𝑏1 = 𝛽^1 and 𝑒 = 𝜖^. Using this new nomenclature we can write, for a given observation 𝑖:

𝑦𝑖 = 𝑏0 + 𝑏1 𝑥𝑖 + 𝑒𝑖
(4.5)
𝑦^𝑖 = 𝑏0 + 𝑏1 𝑥𝑖

218 Chapter 4. Least Squares Modelling Review

 Release 547b16

Presuming we have calculated estimates 𝑏0 and 𝑏1 we can use the model with a new x-observation, 𝑥𝑖 ,
and predict its corresponding 𝑦^𝑖 . The error value, 𝑒𝑖 , is generally non-zero indicating out prediction
estimate of 𝑦^𝑖 is not exact. All this new nomenclature is illustrated in the figure.

4.6.1 Minimizing errors as an objective

Video for
this section Our immediate aim however is to calculate the 𝑏0 and 𝑏1 estimates from the 𝑛 pairs of data collected:
(𝑥𝑖 , 𝑦𝑖 ).

Here are some valid approaches, usually called objective functions for making the 𝑒𝑖 values small.
One could use:
∑︀𝑛 2
1. 𝑖=1 (𝑒𝑖 ) , which leads to the least squares model
∑︀𝑛 4
2. 𝑖=1 (𝑒𝑖 )

3. sum of perpendicular distances to the line 𝑦 = 𝑏0 + 𝑏1 𝑥

∑︀𝑛
4. 𝑖=1 ‖𝑒𝑖 ‖ is known as the least absolute deviations model, or the 𝑙-1 norm problem

5. least median of squared error model, which a robust form of least squares that is far less sensitive to
outliers.

The traditional least squares model, the first objective function listed, has the lowest possible variance
for 𝑏0 and 𝑏1 when certain additional assumptions are met (page 228). The low variance of these
parameter estimates is very desirable, for both model interpretation and using the model. The other
objective functions are good alternatives and may useful in many situations, particular the last
alternative.

Other reasons for so much focus on the least squares alternative is because it is computationally
tractable by hand and very fast on computers, and it is easy to prove various mathematical properties.
The other forms take much longer to calculate, almost always have to be done on a computer, may
have multiple solutions, the solutions can change dramatically given small deviations in the data
(unstable, high variance solutions), and the mathematical proofs are difficult. Also the interpretation
of the least squares objective function is suitable in many situations: it penalizes deviations
quadratically; i.e. large deviations much more than the smaller deviations.

4.6. Least squares models with a single x-variable 219

Process Improvement Using Data

You can read more about least squares alternatives in the book by Birkes and Dodge: “Alternative
Methods of Regression”.

4.6.2 Solving the least squares problem and interpreting the model
Having settled on the least squares objective function, let’s construct the problem as an optimization
problem and understand its characteristics.

The least squares problem can be posed as an unconstrained optimization problem:

𝑛
∑︁
min 𝑓 (𝑏0 , 𝑏1 ) = (𝑒𝑖 )2
𝑏0 , 𝑏1 𝑖=1
𝑛
(4.6)
∑︁ 2
= (𝑦𝑖 − 𝑏0 − 𝑏1 𝑥𝑖 )
𝑖=1

Returning to our example of the gas cylinder. In this case we know that 𝛽0 = 0 from theoretical
principles. So we can solve the above problem by trial and error for 𝑏1 . We expect
𝑛𝑅 (14.1 mol)(8.314 J/(mol.K))
𝑏1 ≈ 𝛽 1 = = = 5.861 kPa/K. So construct equally spaced points of
𝑉 20 × 10−3 m3
5.0 ≤ 𝑏1 ≤ 6.5, set 𝑏0 = 0. Then calculate the objective function using the (𝑥𝑖 , 𝑦𝑖 ) data points recorded
earlier using (4.6).

We find our best estimate for 𝑏1 roughly at 5.88, the minimum of our grid search, which is very close to
the theoretically expected value of 5.86 kPa/K.

For the case where we have both 𝑏0 and 𝑏1 varying we can construct a grid and tabulate the objective
function values at all points on the grid. The least squares objective function will always be shaped
like a bowl for these cases, and a unique minimum always be found, because the objective function is
convex.

220 Chapter 4. Least Squares Modelling Review

 Release 547b16

The above figure shows the general nature of the least-squares objective function where the two
horizontal axes are for 𝑏0 and 𝑏1 , while the vertical axis represents the least squares objective function
𝑓 (𝑏0 , 𝑏1 ).

The illustration highlights the quadratic nature of the objective function. To find the minimum
analytically we start with equation (4.6) and take partial derivatives with respect to 𝑏0 and 𝑏1 , and set
those equations to zero. This is a required condition at any optimal point (see a reference on
optimization theory), and leads to 2 equations in 2 unknowns.
𝑛
𝜕𝑓 (𝑏0 , 𝑏1 ) ∑︁
= −2 (𝑦𝑖 − 𝑏0 − 𝑏1 𝑥𝑖 ) = 0
𝜕𝑏0 𝑖
𝑛
(4.7)
𝜕𝑓 (𝑏0 , 𝑏1 ) ∑︁
= −2 (𝑥𝑖 )(𝑦𝑖 − 𝑏0 − 𝑏1 𝑥𝑖 ) = 0
𝜕𝑏1 𝑖

Now divide the first line through by 𝑛 (the number of data pairs we are using to estimate the
parameters) and solve that equation for 𝑏0 . Then substitute that into the second line to solve for 𝑏1 .
From this we obtain the parameters that provide the least squares optimum for 𝑓 (𝑏0 , 𝑏1 ):

𝑏0 = y − 𝑏1 x
(4.8)
∑︀
(𝑥𝑖 − x) (𝑦𝑖 − y)
𝑏1 = 𝑖 ∑︀ 2
𝑖 (𝑥𝑖 − x)

Verify for yourself that:

∑︀
1. The first part of equation (4.7) shows 𝑖 𝑒𝑖 = 0, also implying the average error is zero.

2. The first part of equation (4.8) shows that the straight line equation passes through the mean of the
data (x, y) without error.
∑︀
3. From second part of equation (4.7) prove to yourself that 𝑖 (𝑥𝑖 𝑒𝑖 ) = 0, just another way of saying
the dot product of the 𝑥-data and the error, 𝑥𝑇 𝑒, is zero.
∑︀
4. Also prove and interpret that 𝑖 (^ 𝑦𝑖 𝑒𝑖 ) = 0, the dot product of the predictions and the errors is zero.

5. Notice that the parameter estimate for 𝑏0 depends on the value of 𝑏1 : we say the estimates are
correlated - you cannot estimate them independently.

6. You can also compute the second derivative of the objective function to confirm that the optimum is
indeed a minimum.

Remarks:

4.6. Least squares models with a single x-variable 221

Process Improvement Using Data

1. What units does parameter estimate 𝑏1 have?

• The units of 𝑦 divided by the units of 𝑥.

2. Recall the temperature and pressure example (page 210): let 𝑝^𝑖 = 𝑏0 + 𝑏1 𝑇𝑖 :

1. What is the interpretation of coefficient 𝑏1 ?

• A one Kelvin increase in temperature is associated, on average, with an increase of 𝑏1 kPa in

pressure.

2. What is the interpretation of coefficient 𝑏0 ?

• It is the expected pressure when temperature is zero. Note: often the data used to build the
model are not close to zero, so this interpretation may have no meaning.

3. What does it mean that 𝑖 (𝑥𝑖 𝑒𝑖 ) = 𝑥𝑇 𝑒 = 0 (i.e. the dot product is zero):
∑︀

• The residuals are uncorrelated with the input variables, 𝑥. There is no information in the
residuals that is in 𝑥.

𝑦𝑖 𝑒𝑖 ) = 𝑦^𝑇 𝑒 = 0
∑︀
4. What does it mean that 𝑖 (^

• The fitted values are uncorrelated with the residuals.

5. How could the denominator term for 𝑏1 equal zero? And what would that mean?

• This shows that as long as there is variation in the 𝑥-data that we will obtain a solution. We get
no solution to the least squares objective if there is no variation in the data.

4.6.3 Example
We will refer back to the following example several times. Calculate the least squares estimates for the
model 𝑦 = 𝑏0 + 𝑏1 𝑥 from the given data. Also calculate the predicted value of 𝑦^𝑖 when 𝑥𝑖 = 5.5

𝑥 10.0 8.0 13.0 9.0 11.0 14.0 6.0 4.0 12.0 7.0 5.0
𝑦 8.04 6.95 7.58 8.81 8.33 9.96 7.24 4.26 10.84 4.82 5.68

222 Chapter 4. Least Squares Modelling Review

 Release 547b16

To calculate the least squares model:

import numpy as np
import [Link] as sm

x = [Link]([10, 8, 13, 9, 11, 14, 6, 4, 12, 7, 5])

y = [Link]([8.04, 6.95, 7.58, 8.81, 8.33, 9.96,
7.24, 4.26, 10.84, 4.82, 5.68])

# Calculate the linear model, where y

# is described by x. add_constant adds
# the intercept term.
X = sm.add_constant(x)
mod_ls = [Link](y, X).fit()

# Coefficients:
# const 3.0001
# x1 0.5001
mod_ls.params

# You can get more information with

print(mod_ls.summary())

print("The model coefficients are: ")

mod_ls.params

b0, b1 = mod_ls.params
x_new = 5.5
y_predicted = b0 + b1 * x_new
print(f"Given a new x value of {x_new} "
f"the predicted y = "
f"{round(y_predicted, 3)}")

x <- c(10, 8, 13, 9, 11, 14, 6, 4, 12, 7, 5)

y <- c(8.04, 6.95, 7.58, 8.81, 8.33, 9.96,
7.24, 4.26, 10.84, 4.82, 5.68)

(continues on next page)

4.6. Least squares models with a single x-variable 223

Process Improvement Using Data

(continued from previous page)

# "Calculate for me the linear model,
# where y is described by x"
[Link] <- lm(y ~ x)

# Call:
# lm(formula = y ~ x)
#
# Coefficients:
# (Intercept) x
# 3.0001 0.5001

# You can get more information with

summary([Link])

print('The model coefficients are: ')

coefficients([Link])

b0 <- coefficients([Link])[1]
b1 <- coefficients([Link])[2]
[Link] <- 5.5
y_predicted <- b0 + b1 * [Link]
paste0('Given a new x value of ', [Link],
' the predicted y = ',
round(y_predicted, 3))

• 𝑏0 = 3.0

• 𝑏1 = 0.5

• When 𝑥𝑖 = 5, then 𝑦^𝑖 = 3.0 + 0.5 × 5.5 = 5.75

4.6.4 A larger example with scikit-learn: predicting vapour pressure

For larger data sets, the LinearRegression class from scikit-learn provides a convenient API that
fits naturally with Pandas data frames. We will use it again on the distillation tower86 data set
introduced in the prior section (page 215).

Good statistical practice is to split the data: build the model on one part, then test it on unseen data.
Otherwise we get an inflated sense of how well the model performs. The .iloc accessor in Pandas
selects rows by position, so we use it to split the 253 observations into a “build” partition (the first 150
rows) and a “test” partition (the remaining rows):

import pandas as pd

distill = pd.read_csv(
"[Link]
)

# Use the first 150 rows to build the model,

# and the remaining rows to test it later.
build = [Link][:150]
test = [Link][150:]
[Link], [Link]

Now fit a single-variable least squares model that uses InvTemp3 (the inverse of a temperature
measurement on tray 3) to predict VapourPressure. The double-bracket idiom
build[["InvTemp3"]] returns a column matrix (a 2-D 𝑛 × 1 array), which is the shape scikit-learn
expects for the predictor matrix X:
86 [Link]

224 Chapter 4. Least Squares Modelling Review

 Release 547b16

from sklearn.linear_model import LinearRegression

# X must be a 2-D array (n_rows by n_cols).

# build[["InvTemp3"]] returns a column matrix;
# build["InvTemp3"] would return a 1-D array.
X = build[["InvTemp3"]].values
y = build["VapourPressure"].values

mymodel = LinearRegression()
[Link](X, y)

# .intercept_ is a scalar; .coef_ is an array,

# so we index it with [0] to print the slope.
print(
f"Intercept = {mymodel.intercept_:.5g}, "
f"slope = {mymodel.coef_[0]:.5g}"
)

We will return to this model in the next section (page 226) to inspect its residuals, its standard error, and
its 𝑅2 value, and again in the section on multiple linear regression (page 253) to extend it with a second
predictor.

4.7 Least squares model analysis

Once we have fitted the 𝑏0 and 𝑏1 terms using the data and the equations from the prior section
(page 221), it is of interest to know how well the model performed. That is what this section is about.
In particular:

1. Analysis of variance: breaking down the data’s variability into components

2. Confidence intervals for the model coefficients, 𝑏0 and 𝑏1

3. Prediction error estimates for the 𝑦-variable

4. We will also take a look at the interpretation of the software output.

In order to perform the second part we need to make a few assumptions about the data, and if the data
follow those assumptions, then we can derive confidence intervals for the model parameters.

4.7.1 The variance breakdown

Video for
this section Recall that variability (page 38) is what makes our data interesting. Without variance (i.e. just flat lines)
we would have nothing to do. The analysis of variance is just a tool to show how much variability in
the 𝑦-variable is explained by:

1. Doing nothing (no model: this implies 𝑦^ = 𝑦)

2. The model (^
𝑦𝑖 = 𝑏0 + 𝑏1 𝑥𝑖 )

3. How much variance is left over in the errors, 𝑒𝑖

These 3 components must add up to the total variance we started with. By definition, the variance is
computed about a mean, so the variance of no model (i.e. the “doing nothing” case) is zero. So the
total variance in vector 𝑦 is just the sum of the other two variances: the model’s variance, and the error
variance. We show this next.

Using the accompanying figure, we see that geometrically, at any fixed value of 𝑥𝑖 , that any 𝑦 value
above or below the least squares line, call it 𝑦𝑖 and shown with a circle, must obey the distance

4.7. Least squares model analysis 225

Process Improvement Using Data

relationship:

Distance relationship: 𝑦𝑖 − y) + (𝑦𝑖 − 𝑦^𝑖 )

(𝑦𝑖 − y) = (^
2
Squaring both sides: (𝑦 − y) = (^ 𝑦 − y)2 + 2(^ 𝑦𝑖 − y)(𝑦𝑖 − 𝑦^𝑖 ) + (𝑦𝑖 − 𝑦^𝑖 )2
∑︀ 𝑖 2
∑︀𝑖
𝑦𝑖 − y) + (𝑦𝑖 − 𝑦^𝑖 )2
2
∑︀
Sum and simplify: (𝑦𝑖 − y) = (^
Total sum of squares (TSS) = Regression SS (RegSS) + Residual SS (RSS)

The total sum of squares (TSS) is the total variance in the vector of 𝑦-data. This broken down into two
∑︀ 2
components: the sum of squares due to regression, (^ 𝑦𝑖 − 𝑦) , called RegSS, and the sum of squares
of the residuals (RSS), 𝑒𝑖 = 𝑒𝑇 𝑒.
∑︀ 2

It is convenient to write these sums of squares (variances) in table form, called an Analysis of Variance
(ANOVA) table:

Type of variance Distance Degrees of freedom SSQ Mean square

Regression 𝑦^𝑖 − 𝑦 𝑘 (𝑘 = 2 in the examples so far) RegSS RegSS/𝑘
Error 𝑦𝑖 − 𝑦^𝑖 𝑛−𝑘 RSS RSS/(𝑛 − 𝑘)
Total 𝑦𝑖 − 𝑦 𝑛 TSS TSS/𝑛

Interpreting the standard error

2
The term 𝑆𝐸 = RSS/(𝑛 − 𝑘) is one way of quantifying the model’s performance. The value
√︀ √︀
𝑆𝐸 = RSS/(𝑛 − 𝑘) = (𝑒𝑇 𝑒)/(𝑛 − 𝑘) is called the standard error. It is really just the standard
deviation of the error term, accounting correctly for the degrees of freedom.

Example: Assume we have a model for predicting batch yield in kilograms from x = raw material
purity, what does a standard error of 3.4 kg imply?

Answer: Recall if the assumption of normally distributed errors is correct, then this value of 3.4 kg
indicates that about two thirds of the yield predictions will lie within ±3.4 kg, and that 95% of the
yield predictions will lie within ±2 × 3.4 kg. We will quantify the prediction interval more precisely,
but the standard error is a good approximation for the error of y.

226 Chapter 4. Least Squares Modelling Review

 Release 547b16

Exercise

For two extreme cases:

1. 𝑦𝑖 = 𝑒𝑖 , i.e. where 𝑏0 = 0 and 𝑏1 = 0. In other words, our 𝑦𝑖 measurements are just random noise.

2. 𝑦𝑖 = 𝑏0 + 𝑏1 𝑥𝑖 + 𝑒𝑖 , for any values of 𝑏0 and 𝑏1 , that model fits the data perfectly, with no residuals.

Do the following in the space below:

• draw a generic plot

• create an ANOVA table with fake values

RegSS
• write down the value of the ratio
TSS
mean square of regression
• interpret what this ratio means: 𝐹0 =
mean square of residuals

From this exercise we learn that:

RegSS
• The null model (𝑦𝑖 = 𝑒𝑖 ) has ratio = 0.
TSS
RegSS
• Models where the fit is perfect have a ratio = 1. This number is called 𝑅2 , and we will see
TSS
why it is called that next.

Derivation of 𝑅2
∑︀ 2
RegSS 𝑦𝑖 − y)
(^
As introduced by example in the previous part, 𝑅 = 2
= ∑︀𝑖 2 : simply the ratio between
TSS 𝑖 (𝑦𝑖 − y)
the variance we can explain with the model (RegSS) and the total variance we started off with (TSS).
RSS
We can also write that 𝑅2 = 1 − , based on the fact that TSS = RegSS + RSS.
TSS
From the above ratios it is straightforward to see that if 𝑅2 = 0, it requires that 𝑦^𝑖 = y: we are
predicting just a flat line, the mean of the y data. On the other extreme, an 𝑅2 = 1 implies that 𝑦^𝑖 = 𝑦𝑖 ,
we have perfect predictions for every data point.

The nomenclature 𝑅2 comes from the fact that it is the square of the correlation between x and y.
Recall from the correlation section (page 213) that

ℰ {(𝑥 − 𝑥)(𝑦 − 𝑦)} Cov {𝑥, 𝑦}

𝑟(𝑥, 𝑦) = √︀ = √︀
𝒱 {𝑥} 𝒱 {𝑦} 𝒱 {𝑥} 𝒱 {𝑦}

and can range in value from −1 to +1. The 𝑅2 ranges from 0 to +1, and is the square of 𝑟(𝑥, 𝑦). 𝑅2 is
just a way to tell how far we are between predicting a flat line (no variation) and the extreme of being
able to predict the model building data, 𝑦𝑖 , exactly.

The 𝑅2 value is likely well known to anyone that has encountered least squares before. This number
must be interpreted with caution. It is most widely abused as a way to measure “how good is my model”.

These two common examples illustrate the abuse. You likely have said or heard something like this
before:

1. “the 𝑅2 value is really high, 90%, so this is a good model”.

4.7. Least squares model analysis 227

 Process Improvement Using Data

2. “Wow, that’s a really low 𝑅2 , this model can’t be right - it’s no good”.

How good, or how suitable a model is for a particular purpose is almost never related to the 𝑅2 value.
The goodness of a model is better assessed by:

• your engineering judgment: does the interpretation of model parameters make sense?

• use testing data to verify the model’s predictive performance,

• using cross-validation tools (we will see this topic later on) to see how well the model performs on
new, unseen and unused testing data.

We will see later on that 𝑅2 can be arbitrarily increased by adding terms to the linear model, as we will
see in the section on multiple linear regression (MLR) (page 253). So sometimes you will see the adjusted
R-squared used to account for the 𝑘 terms used in the model:

2 RSS/(𝑛 − 𝑘)
𝑅adj =1−
TSS/(𝑛 − 1)

where 𝑘 = 2 for the case of estimating a model 𝑦𝑖 = 𝑏0 + 𝑏1 𝑥𝑖 , as there are 2 parameters.

4.7.2 Confidence intervals for the model coefficients 𝑏0 and 𝑏1

ò Note

A good reference for this section is the book by Fox (Chapter 6), and the book by Draper and Smith.

Up to this point we have made no assumptions about the data. In fact we can calculate the model
estimates, 𝑏0 and 𝑏1 as well as predictions from the model without any assumptions on the data. It is
only when we need additional information such as confidence intervals for the coefficients and
prediction error estimates that we must make assumptions.

Recall the 𝑏1 coefficient represents the average effect on y when changing the x-variable by 1 unit. Let’s
say you are estimating a reaction rate (kinetics) from a linear least squares model, a standard step in
reactor design, you would want a measure of confidence of your coefficient. For example, if you
calculate the reaction rate as 𝑘 = 𝑏1 = 0.81 s−1 you would benefit from knowing whether the 95%
confidence interval was 𝑘 = 0.81 ± 0.26 s−1 or 𝑘 = 0.81 ± 0.68 s−1 . In the latter case it is doubtful
whether the reaction rate is of practical significance. Point estimates of the least squares model
parameters are satisfactory, but the confidence interval information is richer to interpret.

We first take a look at some assumptions in least squares modelling, then return to deriving the
confidence interval.

Assumptions required for analysis of the least squares model

Video for
this section Recall that the population (true) model is 𝑦𝑖 = 𝛽0 + 𝛽1 𝑥𝑖 + 𝜖𝑖 and 𝑏0 and 𝑏1 are our estimates of the
model’s coefficients, and e be the estimate of the true error 𝜖. Note we are assuming imperfect
knowledge of the 𝑦𝑖 by lumping all errors into 𝑒𝑖 . For example, measurement error, structural error
(we are not sure the process follows a linear structure), inherent randomness, and so on.

Furthermore, our derivation for the confidence intervals of 𝑏0 and 𝑏1 requires that we assume:

1. Linearity of the model, and that the values of x are fixed (have no error). This implies that the error
captured by 𝜖 is the error of y, since the 𝛽0 + 𝛽1 x terms are fixed.

228 Chapter 4. Least Squares Modelling Review

 Release 547b16

• In an engineering situation this would mean that your x variable has much less uncertainty than
the y variable; and is often true in many situations.

2. The variance of y is the same (constant) at all values of x, known as the constant error variance
assumption.

• The variability of y can be non-constant in several practical cases (e.g. our measurement accuracy
deteriorates at extreme high and low levels of x).

Illustration of the constant error variance assumption and the normally distributed error
assumption.

3. The errors are normally distributed: 𝑒𝑖 ∼ 𝒩 (0, 𝜎𝜖2 ). This also implies that 𝑦𝑖 ∼ 𝒩 (𝛽0 + 𝛽1 𝑥𝑖 , 𝜎𝜖2 ) from
the first linearity assumption.

4. Each error is independent of the other. This assumption is often violated in cases where the
observations are taken in time order on slow moving processes (e.g. if you have a positive error
now, your next sample is also likely to have a positive error). We will have more to say about this
later when we check for independence with an autocorrelation test (page 245).

5. In addition to the fact that the x values are fixed, we also assume they are independent of the error.
If the x value is fixed (i.e. measured without error), then it is already independent of the error.

• When the x values are not fixed, there are cases where the error gets larger as x gets
smaller/larger.

6. All 𝑦𝑖 values are independent of each other. This again is violated in cases where the data are
collected in time order and the 𝑦𝑖 values are autocorrelated.

ò Note

Derivation of the model’s coefficients do not require these assumptions, only the derivation of the
coefficient’s confidence intervals require this.

4.7. Least squares model analysis 229

Process Improvement Using Data

Also, if we want to interpret the model’s 𝑆𝐸 as the estimated standard deviation of the residuals, then
it helps if the residuals are normally distributed.

Confidence intervals for 𝛽0 and 𝛽1

Recall from our discussions on confidence intervals (page 77) that we need to know the mean and
variance of the population from which 𝑏0 and 𝑏1 come. Specifically for the least squares case:

𝑏0 ∼ 𝒩 (𝛽0 , 𝒱{𝛽0 }) and 𝑏1 ∼ 𝒩 (𝛽1 , 𝒱{𝛽1 })

Once we know those parameters, we can create a 𝑧-value for 𝑏0 and 𝑏1 , and then calculate the
confidence interval for 𝛽0 and 𝛽1 . So our quest now is to calculate 𝒱{𝛽0 } and 𝒱{𝛽1 }, and we will use
the 6 assumptions we made in the previous part.

Start from the equations that define 𝑏0 and 𝑏1 in the prior section (page 221) where we showed that:

𝑏0 = y − 𝑏1 x
∑︀
𝑖 (𝑥𝑖 − x) (𝑦𝑖 − y)
𝑏1 = ∑︀ 2
𝑖 (𝑥𝑖 − x)

∑︀ 𝑥𝑖 − x
𝑏1 = 𝑚𝑖 𝑦𝑖 where 𝑚𝑖 = ∑︀ 2
𝑗 (𝑥𝑗 − x)

That last form of expressing 𝑏1 shows that every data point contributes a small amount to the
coefficient 𝑏1 . But notice how it is broken into 2 pieces: each term in the sum has a component due to
𝑚𝑖 and one due to 𝑦𝑖 . The 𝑚𝑖 term is a function of the x-data only, and since we assume the x’s are
measured without error, that term has no error. The 𝑦𝑖 component is the only part that has error.

So we can write:
𝑏1 = 𝑚1 𝑦1 + 𝑚2 𝑦2 + . . . + 𝑚𝑁 𝑦𝑁
ℰ{𝑏1 } = ℰ{𝑚1 𝑦1 } + ℰ{𝑚2 𝑦2 } + . . . + ℰ{𝑚𝑁 𝑦𝑁 }
𝒱{𝑏1 } = 𝑚21 𝒱{𝑦1 } + 𝑚22 𝒱{𝑦2 } + . . . + 𝑚2𝑁 𝒱{𝑦𝑁 }
(︃ )︃2
∑︁ 𝑥𝑖 − x
𝒱{𝑏1 } = ∑︀ 2 𝒱{𝑦𝑖 }
𝑖 𝑗 (𝑥𝑗 − x)
𝒱{𝑦𝑖 }
𝒱{𝑏1 } = ∑︀ 2
𝑗 (𝑥𝑗 − x)

where 𝑗 is an index for all data points used to build the least squares model.

Questions:

1. So now apart from the numerator term, how could you decrease the error in your model’s 𝑏1
coefficient?

• Use samples that are far from the mean of the x-data.

• Use more samples.

2. What do we use for the numerator term 𝒱{𝑦𝑖 }?

• This term represents the variance of the 𝑦𝑖 values at a given point 𝑥𝑖 . If (a) there is no evidence of
lack-of-fit,∑︀and (b) if y has the same error at all levels of x, then we can write that 𝒱{𝑦𝑖 } =
𝑒2𝑖
𝒱{𝑒𝑖 } = , where 𝑛 is the number of data points used, and 𝑘 is the number of coefficients
𝑛−𝑘
estimated (2 in this case). The 𝑛 − 𝑘 quantity is the degrees of freedom.

230 Chapter 4. Least Squares Modelling Review

 Release 547b16

Now for the variance of 𝑏0 = y − 𝑏1 x. The only terms with error are 𝑏1 , and y. So we can derive that:
(︃ )︃
1 x2
𝒱{𝑏0 } = + ∑︀ 2 𝒱{𝑦𝑖 }
𝑁 𝑗 (𝑥𝑗 − x)

Summary of important equations

(︃ )︃
1 x2
𝒱{𝛽0 } ≈ 𝒱{𝑏0 } = + ∑︀ 2 𝒱{𝑦𝑖 }
𝑁 𝑗 (𝑥𝑗 − x)

𝒱{𝑦𝑖 }
𝒱{𝛽1 } ≈ 𝒱{𝑏1 } = ∑︀ 2
𝑗 (𝑥𝑗 − x)

𝑒2𝑖
∑︀
where 𝒱{𝑦𝑖 } = 𝒱{𝑒𝑖 } = , if there is no lack-of-fit and the y’s are independent of each other.
𝑛−𝑘
For convenience we will define some short-hand notation, which is common in least squares:
∑︀ 2 √︃ ∑︀
2 𝑒𝑖 𝑒2𝑖
𝑆𝐸 = 𝒱{𝑒𝑖 } = 𝒱{𝑦𝑖 } = or 𝑆𝐸 =
𝑛−𝑘 𝑛−𝑘
(︃ )︃
2 1 x2 2
𝑆𝐸 (𝑏0 ) = 𝒱{𝑏0 } = + ∑︀ 2 𝑆𝐸
𝑁 𝑗 (𝑥 𝑗 − x)
2
2 𝑆𝐸
𝑆𝐸 (𝑏1 ) = 𝒱{𝑏1 } = ∑︀ 2
𝑗 (𝑥𝑗 − x)

You will see that 𝑆𝐸 is an estimate of the standard deviation of the error (residuals), while 𝑆𝐸 (𝑏0 ) and
𝑆𝐸 (𝑏1 ) are the standard deviations of estimates for 𝑏0 and 𝑏1 respectively.

Now it is straight forward to construct confidence intervals for the least squares model parameters.
You will also realize that we have to use the 𝑡-distribution, because we are using an estimate of the
variance.

𝑏0 − 𝛽0 𝑏1 − 𝛽 1
−𝑐𝑡 ≤ ≤ +𝑐𝑡 −𝑐𝑡 ≤ ≤ +𝑐𝑡
𝑆𝐸 (𝑏0 ) 𝑆𝐸 (𝑏1 )
𝑏0 − 𝑐𝑡 𝑆𝐸 (𝑏0 ) ≤ 𝛽0 ≤ 𝑏0 + 𝑐𝑡 𝑆𝐸 (𝑏0 ) 𝑏1 − 𝑐𝑡 𝑆𝐸 (𝑏1 ) ≤ 𝛽1 ≤ 𝑏1 + 𝑐𝑡 𝑆𝐸 (𝑏1 )
(4.9)

Example
Video for
this section Returning back to our ongoing example (page 222), we can calculate the confidence interval for 𝛽0 and 𝛽1 .
We calculated earlier already that 𝑏0 = 3.0 and 𝑏1 = 0.5. Using these values we can calculate the
standard error:

import numpy as np
import [Link] as sm

x = [Link]([10, 8, 13, 9, 11, 14, 6, 4, 12, 7, 5])

y = [Link]([8.04, 6.95, 7.58, 8.81, 8.33, 9.96,
7.24, 4.26, 10.84, 4.82, 5.68])

# Calculate the linear model, where y

# is described by x.
X = sm.add_constant(x)
(continues on next page)

4.7. Least squares model analysis 231

Process Improvement Using Data

(continued from previous page)

mod_ls = [Link](y, X).fit()

# We can find what `b0` and `b1` are in

# several different ways:
print(mod_ls.summary())

# or using
print("The model coefficients are: ")
mod_ls.params

# Model predictions:
print("The predicted values are: ")
mod_ls.predict()
# [ 8.001 7.000 9.501 7.501 8.501
# 10.001 6.000 5.000 9.001 6.500 5.501]

# Prediction error = observed - predicted

error = y - mod_ls.predict()
N = len(x)

# The SE = standard error = 1.236603

std_error = [Link]((error ** 2).sum() / (N - 2))
print(f"Standard error SE = "
f"{round(std_error, 3)}")

x <- c(10, 8, 13, 9, 11, 14, 6, 4, 12, 7, 5)

y <- c(8.04, 6.95, 7.58, 8.81, 8.33, 9.96,
7.24, 4.26, 10.84, 4.82, 5.68)

# "Calculate for me the linear model,

# where y is described by x"
[Link] <- lm(y ~ x)

# We can find what the "b0" and "b1"

# values are in several different ways:
summary([Link])

# or using
print('The model coefficients are: ')
coefficients([Link])

# Model predictions:
print('The predicted values are: ')
predict([Link])
# 8.001 7.000 9.501 7.501 8.501
# 10.001 6.00 5.000 9.001 6.500 5.501

# Prediction error = observed - predicted

error <- y - predict([Link])
N <- length(x)

# The SE = standard error = 1.236603

[Link] <- sqrt(sum(error^2) / (N-2))
paste0('Standard error SE = ',
round([Link], 3))

Use that 𝑆𝐸 value to calculate the confidence intervals for 𝛽0 and 𝛽1 , and use that 𝑐𝑡 = 2.26 at the 95%
confidence level. You can calculate this value in R using qt(0.975, df=(N-2)). There are 𝑛 − 2
degrees of freedom, the number of degrees of freedom used to calculate 𝑆𝐸 .

First calculate the 𝑆𝐸 value and the standard errors for the 𝑏0 and 𝑏1 . Substitute these into the equation

232 Chapter 4. Least Squares Modelling Review

 Release 547b16

for the confidence interval and calculate:

𝑆𝐸 = 1.237
2
2 𝑆𝐸 1.2372
𝑆𝐸 (𝑏1 ) = ∑︀ 2 = = 0.0139
𝑗 (𝑥𝑗 − x)
110
(︃ )︃
2
92
(︂ )︂
2 1 x 2 1
𝑆𝐸 (𝑏0 ) = + ∑︀ 2 𝑆𝐸 = + 1.2372 = 1.266
𝑁 𝑗 (𝑥𝑗 − x)
11 110

The 95% confidence interval for 𝛽0 :

𝑏0 − 𝛽0
−𝑐𝑡 ≤ ≤ +𝑐𝑡
√ 𝑆𝐸 (𝑏0 ) √
3.0 − 2.26 × 1.266 ≤ 𝛽0 ≤ 3.0 + 2.26 × 1.266
0.457 ≤ 𝛽0 ≤ 5.54

The confidence interval for 𝛽1 :

𝑏1 − 𝛽 1
−𝑐𝑡 ≤ ≤ +𝑐𝑡
√ 𝑆𝐸 (𝑏1 ) √
0.5 − 2.26 × 0.0139 ≤ 𝛽1 ≤ 0.5 + 2.26 × 0.0139
0.233 ≤ 𝛽1 ≤ 0.767

The plot shows the effect of varying the slope parameter, 𝑏1 , from its lower bound to its upper bound.
Notice that the slope always passes through the mean of the data (𝑥, 𝑦).

In many cases the confidence interval for the intercept is not of any value because the data for x is so
far away from zero, or the true value of the intercept is not of concern for us.

import numpy as np
import [Link] as sm

x = [Link]([10, 8, 13, 9, 11, 14, 6, 4, 12, 7, 5])

y = [Link]([8.04, 6.95, 7.58, 8.81, 8.33, 9.96,
7.24, 4.26, 10.84, 4.82, 5.68])

# Calculate the linear model, where y

# is described by x.
X = sm.add_constant(x)
(continues on next page)

4.7. Least squares model analysis 233

 Process Improvement Using Data

(continued from previous page)

mod_ls = [Link](y, X).fit()

# You can (and should at the beginning)

# calculate the confidence intervals as shown
# above. But there is a short-cut, to save
# time, and is less error prone:
mod_ls.conf_int()

# 0 1
# const 0.455737 5.544445
# x1 0.233370 0.766812

# If you want the confidence interval at any

# other level, for example, at the 90% level:
mod_ls.conf_int(alpha=0.10)

# 0 1
# const 0.938303 5.061879
# x1 0.283957 0.716225

# Compare this to the calculated value by hand

# above. It is exactly the same!

x <- c(10, 8, 13, 9, 11, 14, 6, 4, 12, 7, 5)

y <- c(8.04, 6.95, 7.58, 8.81, 8.33, 9.96,
7.24, 4.26, 10.84, 4.82, 5.68)

# "Calculate for me the linear model,

# where y is described by x"
[Link] <- lm(y ~ x)

# You can (and should at the beginning)

# calculate the confidence intervals as shown
# above. But there is a short-cut, to save
# time, and is less error prone:
confint([Link])

# 2.5 % 97.5 %
# (Intercept) 0.4557369 5.5444449
# x 0.2333701 0.7668117

# If you want the confidence interval at any

# other level, for example, at the 90% level:
confint([Link], level=0.90)

# 5 % 95 %
# (Intercept) 0.9383030 5.061879
# x 0.2839568 0.716225

# Compare this to the calculated value by hand

# above. It is exactly the same!

4.7.3 Prediction error estimates for the y-variable

Video for
this section Apart from understanding the error in the model’s coefficient, we also would like an estimate of the
error when predicting 𝑦^𝑖 from the model, 𝑦𝑖 = 𝑏0 + 𝑏1 𝑥𝑖 + 𝑒𝑖 for a new value of 𝑥𝑖 . This is known as the
prediction interval, or prediction error interval.

234 Chapter 4. Least Squares Modelling Review

 Release 547b16

A naive first attempt

We might expect the error is related to the average size of the residuals. After all, our assumptions we
made earlier (page 228) showed
∑︀ 2 the standard error of the residuals was the standard error of the y:
2 𝑒𝑖
𝑆𝐸 = 𝒱 {𝑒𝑖 } = 𝒱 {𝑦𝑖 } = .
𝑛−𝑘

A typical histogram of the residuals looks as shown here: it is always centered around zero, and
appears to be normally distributed. So we could expect to write our prediction error as
𝑦^new = (𝑏0 + 𝑏1 𝑥new ) ± 𝑐 · 𝑆𝐸 , where 𝑐 is the number of standard deviations around the average
residual, for example we could have set 𝑐 = 2, approximating the 95% confidence limit.

But there is something wrong with that error estimate. It says that our prediction error is constant at
any value of 𝑥𝑖 , even at values far outside the range where we built the model. This is a naive estimate
of the prediction error. We have forgotten that coefficients 𝑏0 and 𝑏1 have error, and that error must be
propagated into 𝑦^new .

This estimate is however a reasonable guess for the prediction interval when you only know the
model’s 𝑆𝐸 and don’t have access to a calculator or computer to calculate the proper prediction
interval, shown next.

A better attempt to construct prediction intervals for the least squares model

ò Note

A good reference for this section is Draper and Smith, Applied Regression Analysis, page 79.

The derivation for the prediction interval is similar to that for 𝑏1 . We require an estimate for the
variance of the predicted y at at given value of x. Let’s fix our x value at 𝑥* and since 𝑏0 = y − 𝑏1 x, we
can write the prediction at this fixed x value as 𝑦^* = y − 𝑏1 (𝑥* − x).

𝒱{𝑦* } = 𝒱{y} + 𝒱{𝑏1 (𝑥* − x)} + 2Cov{y, 𝑏1 (𝑥* − x)}

2
𝑆𝐸
𝒱{𝑦* } = + (𝑥* − x)2 𝑆𝐸
2
(𝑏1 ) + 0
𝑛

4.7. Least squares model analysis 235

Process Improvement Using Data

You may read the reference texts for the interesting derivation of this variance. However, this is only
the variance of the average predicted value of y. In other words, it is the variance we expect if we
repeatedly brought in observations at 𝑥* . The prediction error of an individual observation, 𝑥𝑖 , and its
corresponding prediction, 𝑦^𝑖 , is inflated slightly further:
(︃ )︃
2
2 1 (𝑥 𝑖 − x)
𝒱{^𝑦𝑖 } = 𝑆𝐸 1 + + ∑︀ 2 , where 𝑗 is the index for all points used to build the least squares
𝑛 𝑗 (𝑥𝑗 − x)
model.
(︀ )︀
We may construct a prediction interval in the standard manner, assuming that 𝑦^𝑖 ∼ 𝒩 𝑦^𝑖 , 𝒱{^ 𝑦𝑖 } . We
will use an estimate of this variance since we do not know the population variance. This requires we
use the 𝑡-distribution with 𝑛 − 𝑘 degrees of freedom, at a given degree of confidence, e.g. 95%.

𝑦^ − 𝑦^𝑖
−𝑐𝑡 < √︀𝑖 < +𝑐𝑡
√︀ 𝑉 {^ 𝑦𝑖 } √︀
𝑦^𝑖 − 𝑐𝑡 𝑉 {^
𝑦𝑖 } < 𝑦^𝑖 < 𝑦^𝑖 + 𝑐𝑡 𝑉 {^
𝑦𝑖 }

This is a prediction interval for a new prediction, 𝑦^𝑖 given a new x value, 𝑥𝑖 . For example, if 𝑦^𝑖 = 20 at a
√︀
given value of 𝑥𝑖 , and if 𝑐𝑡 𝑉 {^𝑦𝑖 } = 5, then you will usually see written in reports and documents
that, the prediction was 20 ± 5. A more correct way of expressing this concept is to say the true
prediction at the value of 𝑥𝑖 lies within a bound from 15 to 25, with 95% confidence.

Implications of the prediction error of a new y

(︃ )︃
2
2 1 (𝑥 𝑖 − x)
Let’s understand the interpretation of 𝒱{^ 𝑦𝑖 } = 𝑆𝐸 1 + + ∑︀ 2 as the variance of the
𝑛 𝑗 (𝑥𝑗 − x)
predicted 𝑦^𝑖 at the given value of 𝑥𝑖 . Using the previous example where we calculated the least
squares line, now:

1. Now let’s say our 𝑥new happens to be x, the center point of our data. Write down the upper and
lower value of the prediction bounds for the corresponding 𝑦^, given that 𝑐𝑡 = 2.26 at the 95%
confidence level.
⎯ (︃ )︃
2
⎸
√︀ ⎸ 1 (𝑥 𝑖 − x)
• The LB = 𝑦^𝑖 − 𝑐𝑡 𝑉 {^𝑦𝑖 } = 7.5 − 2.26 × ⎷(1.237)2 1 + + ∑︀ 2 =
11 𝑗 (𝑥𝑗 − x)
7.5 − 2.26 × 1.29 = 7.50 − 2.917 = 4.58
⎯ (︃ )︃
2
⎸
√︀ ⎸ 1 (𝑥 𝑖 − x)
• The UB = 𝑦^𝑖 + 𝑐𝑡 𝑉 {^𝑦𝑖 } = 7.5 + 2.26 × ⎷(1.237)2 1 + + ∑︀ 2 =
11 𝑗 (𝑥𝑗 − x)
7.5 + 2.26 × 1.29 = 7.50 + 2.917 = 10.4

2. Now move left and right, away from x, and mark the confidence intervals. What general shape do
they have?

• The confidence intervals have a quadratic shape due to the square term under the square root.
The smallest prediction error will always occur at the center of the model, and expands
progressively wider as one moves away from the model center. This is illustrated in the figure
and makes intuitive sense as well.

236 Chapter 4. Least Squares Modelling Review

 Release 547b16

4.7.4 Interpretation of software output

To complete this section we show how to interpret the output from computer software packages. Most
packages have very standardized output, and you should make sure that whatever package you use,
that you can interpret the estimates of the parameters, their confidence intervals and get a feeling for
the model’s performance.

The following output is obtained in R for the example (page 222) we have been using in this section.
The Python version follows below.

x <- c(10, 8, 13, 9, 11, 14, 6, 4, 12, 7, 5)

y <- c(8.04, 6.95, 7.58, 8.81, 8.33, 9.96,
7.24, 4.26, 10.84, 4.82, 5.68)

# "Calculate for me the linear model,

# where y is described by x"
[Link] <- lm(y ~ x)

summary([Link])

and produces this output:

Call:
lm(formula = y ~ x)

Residuals:
Min 1Q Median 3Q Max
-1.92127 -0.45577 -0.04136 0.70941 1.83882

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 3.0001 1.1247 2.667 0.02573 *
x 0.5001 0.1179 4.241 0.00217 **
---
(continues on next page)

4.7. Least squares model analysis 237

Process Improvement Using Data

(continued from previous page)

Signif. codes: 0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1

Residual standard error: 1.237 on 9 degrees of freedom

Multiple R-squared: 0.6665, Adjusted R-squared: 0.6295
F-statistic: 17.99 on 1 and 9 DF, p-value: 0.002170

Make sure you can calculate the following values using the equations developed so far, based on the
above software output:

• The intercept term 𝑏0 = 3.0001.

• The slope term 𝑏1 = 0.5001.

• The standard error of the model, 𝑆𝐸 = 1.237, using 𝑛 − 𝑘 = 11 − 2 = 9 degrees of freedom.

• Using the standard error, calculate the standard error for the intercept = 𝑆𝐸 (𝑏0 ) = 1.1247.

• Using the standard error, calculate the standard error for the slope = 𝑆𝐸 (𝑏1 ) = 0.1179.

• The 𝑧-value for the 𝑏0 term is 2.667 (R calls this the t value in the printout, but in our notes we
𝑏0 − 𝛽0
have called this 𝑧 = ; the value that we compare to the 𝑡-statistic and used to create the
𝑆𝐸 (𝑏0 )
confidence interval).

• The 𝑧-value for the 𝑏1 term is 4.241 (see the above comment again).

• The two probability values, Pr(>|t|), for 𝑏0 and 𝑏1 should be familiar to you; they are the
probability with which we expect to find a value of 𝑧 greater than the calculated 𝑧-value (called t
value in the output above). The smaller the number, the more confident we can be the confidence
interval contains the parameter estimate.

• You can construct the confidence interval for 𝑏0 or 𝑏1 by using their reported standard errors and
multiplying by the corresponding 𝑡-value. For example, if you want 99% confidence limits, then
look up the 99% values for the 𝑡-distribution using 𝑛 − 𝑘 degrees of freedom, in this case it would be
qt((1-0.99)/2, df=9), which is ±3.25. So the 99% confidence limits for the slope coefficient
would be [0.5 − 3.25 × 0.1179; 0.5 + 3.25 × 0.1179] = [0.12; 0.88].

• The 𝑅2 = 0.6665 value.

• Be able to calculate the residuals: 𝑒𝑖 = 𝑦𝑖 − 𝑦^𝑖 = 𝑦𝑖 − 𝑏0 − 𝑏1 𝑥𝑖 . We expect the median of the residuals
to be around 0, and the rest of the summary of the residuals gives a feeling for how far the residuals
range about zero.

Using Python, you can run the following code:

import numpy as np
import [Link] as sm

X = [Link]([10, 8, 13, 9, 11, 14,

6, 4, 12, 7, 5])
y = [Link]([8.04, 6.95, 7.58, 8.81,
8.33, 9.96, 7.24, 4.26,
10.84, 4.82, 5.68])

# We do want to estimate a 'b0' term

X = sm.add_constant(X)
model = [Link](y, X)
results = [Link]()
print([Link]())
(continues on next page)

238 Chapter 4. Least Squares Modelling Review

 Release 547b16

(continued from previous page)

print('Standard error = {}'.format(\
[Link]([Link])))

which produces the following output:

OLS Regression Results

==============================================================================
Dep. Variable: y R-squared: 0.667
Model: OLS Adj. R-squared: 0.629
Method: Least Squares F-statistic: 17.99
Date: Tue, 01 Jan 2019 Prob (F-statistic): 0.00217
Time: 00:00:00 Log-Likelihood: -16.841
No. Observations: 11 AIC: 37.68
Df Residuals: 9 BIC: 38.48
Df Model: 1
Covariance Type: nonrobust
==============================================================================
coef std err t P>|t| [0.025 0.975]
------------------------------------------------------------------------------
const 3.0001 1.125 2.667 0.026 0.456 5.544
x1 0.5001 0.118 4.241 0.002 0.233 0.767
==============================================================================
Omnibus: 0.082 Durbin-Watson: 3.212
Prob(Omnibus): 0.960 Jarque-Bera (JB): 0.289
Skew: -0.122 Prob(JB): 0.865
Kurtosis: 2.244 Cond. No. 29.1
==============================================================================

Standard error = 1.2366033227263207

As for the R code, we can see at a glance:

• The intercept term 𝑏0 = 3.0001.

• The slope term 𝑏1 = 0.5001.

• The standard error of the model, 𝑆𝐸 = 1.237, using 𝑛 − 𝑘 = 11 − 2 = 9 degrees of freedom. The
summary output table does not show the standard error, but you can get it from
[Link]([Link]), where results is the Python object from fitting the linear model.

• Using the standard error, calculate the standard error for the intercept = 𝑆𝐸 (𝑏0 ) = 1.1247, which is
reported directly in the table.

• Using the standard error, calculate the standard error for the slope = 𝑆𝐸 (𝑏1 ) = 0.1179, which is
reported directly in the table.

• The 𝑧-value for the 𝑏0 term is 2.667 (Python calls this the t-value in the printout, but in our notes we
𝑏0 − 𝛽0
have called this 𝑧 = ; the value that we compare to the 𝑡-statistic and used to create the
𝑆𝐸 (𝑏0 )
confidence interval).

• The 𝑧-value for the 𝑏1 term is 4.241 (see the above comment again).

• The two probability values, P>|t|, for 𝑏0 and 𝑏1 should be familiar to you; they are the probability
with which we expect to find a value of 𝑧 greater than the calculated 𝑧-value (called t value in the
output above). The smaller the number, the more confident we can be the confidence interval
contains the parameter estimate.

4.7. Least squares model analysis 239

Process Improvement Using Data

• You can construct the confidence interval for 𝑏0 or 𝑏1 by using their reported standard errors and
multiplying by the corresponding 𝑡-value. For example, if you want 99% confidence limits, then
look up the 99% values for the 𝑡-distribution using 𝑛 − 𝑘 degrees of freedom, in this case it would be
from [Link] import t; [Link](1-(1-0.99)/2, df=9), which is ±3.25. So the 99%
confidence limits for the slope coefficient would be
[0.5 − 3.25 × 0.1179; 0.5 + 3.25 × 0.1179] = [0.117; 0.883]. However, the table output gives you the
95% confidence interval. Under the column 0.025 and 0.975 (leaving 2.5% in the lower and upper
tail respectively). For the slope coefficient, for example, this interval is [0.233; 0.767]. If you desire,
for example, the 99% confidence interval, you can adjust the code:
print([Link](alpha=1-0.99))

• The 𝑅2 = 0.6665 value.

• Be able to calculate the residuals: 𝑒𝑖 = 𝑦𝑖 − 𝑦^𝑖 = 𝑦𝑖 − 𝑏0 − 𝑏1 𝑥𝑖 .

Visualizing the fit

Returning to the larger distillation example from the prior section (page 224), the seaborn library has a
useful function, regplot, that draws the scatter plot of the raw data, overlays the least squares line,
and shades the confidence interval for the regression line in a single call.

import pandas as pd
import seaborn as sns

distill = pd.read_csv(
"[Link]
)

ax = [Link](
x="InvTemp3",
y="VapourPressure",
data=distill,
)
[Link](True)

A common upgrade is the jointplot, which adds a histogram (or kernel density estimate) of each
variable to the margins of the plot:

# Marginal histograms with the regression line:

[Link](
x="InvTemp3",
y="VapourPressure",
data=distill,
kind="reg",
)

# Or the kernel density estimate:

[Link](
x="InvTemp3",
y="VapourPressure",
data=distill,
kind="kde",
)

240 Chapter 4. Least Squares Modelling Review

 Release 547b16

Residuals, standard error and 𝑅2 for the model

Once a scikit-learn LinearRegression object has been fitted, the residuals on the building data are
obtained by subtracting the predictions from the observed values. Two simple summaries of the
residuals are useful: their average absolute size, and their standard deviation. Both are “smaller is
better”, and the standard deviation is the model’s standard error 𝑆𝐸 .

We continue with the model fitted in the prior section (page 224):

# Predictions and residuals on the building data:

X_build = build[["InvTemp3"]]
y_build = build["VapourPressure"].values

prediction_build = [Link](X_build)
errors_build = y_build - prediction_build

# Average absolute residual:

avg_absolute_error = (
[Link](errors_build).abs().mean()
)

# Standard deviation of the residuals

# (equivalent to S_E, up to the n-k correction):
std_error = errors_build.std()

print(
f"Average absolute error = "
f"{avg_absolute_error:.3f}, "
f"std. dev. of residuals = "
f"{std_error:.3f}"
)

# Plot residuals in time order to look for

# trends or autocorrelation:
[Link](errors_build).plot(
grid=True,
title="Building-data residuals (Actual - Predicted)",
)

The 𝑅2 value can be read off directly from the model object, using its .score(...) method:

# R-squared on the building data:

[Link](X_build, y_build)

As emphasized earlier in this section, a high 𝑅2 value is not a measure of prediction accuracy: it only
tells you how strongly 𝑥 and 𝑦 are correlated. The prediction quality on new data is a more honest test,
and that is what we turn to in the next section (page 242).

4.8 Investigating an existing linear model

Video for
this section
4.8.1 Summary so far
We have introduced the linear model, 𝑦 = 𝛽0 + 𝛽1 𝑥 + 𝜀 and shown how to estimate the 2 model
parameters, 𝑏0 = 𝛽^0 and 𝑏1 = 𝛽^1 . This can be done on any data set without any additional
assumptions. But, in order to calculate confidence intervals so we can better understand our model’s
performance, we must make several assumptions of the data. In the next sections we will learn how to
interpret various plots that indicate when these assumptions are violated.

4.8. Investigating an existing linear model 241

Process Improvement Using Data

Along the way, while investigating these assumptions, we will introduce some new topics:

• Transformations of the raw data to better meet our assumptions

• Leverage, outliers, influence and discrepancy of the observations

• Inclusion of additional terms in the linear model (multiple linear regression, MLR)

• The use of training and testing data

It is a common theme in any modelling work that the most informative plots are those of the residuals
- the unmodelled component of our data. We expect to see no structure in the residuals, and since the
human eye is excellent at spotting patterns in plots, it is no surprise that various types of residual plots
are used to diagnose problems with our model.

Testing the model on unseen data

The residual statistics on the building data give one view of the model. A more honest assessment is
the residuals on data that the model has never seen. Continuing the distillation example from the prior
chapter (page 241), where we held out the rows from index 150 onward as the testing partition, we can
call .predict(...) again, but on the test set:

# The held-out partition of the data set:

X_test = test[["InvTemp3"]].values
y_test = test["VapourPressure"].values

prediction_test = [Link](X_test)
errors_test = y_test - prediction_test

avg_absolute_error_test = (
[Link](errors_test).abs().mean()
)
std_error_test = errors_test.std()

print(
f"Testing-data: average absolute error = "
f"{avg_absolute_error_test:.3f}, "
f"std. dev. = {std_error_test:.3f}"
)

# Inspect the residuals in time order; any

# drift or pattern is a sign that the model
# does not generalize.
[Link](errors_test).plot(
grid=True,
title="Testing-data residuals (Actual - Predicted)",
)

If the testing-data error is much larger than the building-data error, or if the residual time series shows
an obvious drift, then the model has not generalized: there is structure in the data that is not captured
by InvTemp3 alone. The remedy is to add more explanatory variables, which is the topic of the section
on multiple linear regression (page 253).

4.8.2 The assumption of normally distributed errors

We look for normally distributed errors because if they are non-normal, then the standard error, 𝑆𝐸
and the other variances that depend on 𝑆𝐸 , such as 𝒱(𝑏1 ), could be inflated, and their interpretation
could be in doubt. This might, for example, lead us to infer that a slope coefficient is not important
when it actually is.

242 Chapter 4. Least Squares Modelling Review

 Release 547b16

This is one of the easiest assumptions to verify: use a q-q plot (see the univariate review (page 61)) to
assess the distribution of the residuals. Do not plot the residuals in sequence or some other order to
verify normality - it is extremely difficult to see that. A q-q plot highlights very clearly when tails from
the residuals are too heavy. A histogram may also be used, but for real data sets, the choice of bin
width can dramatically distort the interpretation - rather use a q-q plot. Some code for R:

model = lm(...)
library(car)
qqPlot(model) # uses studentized residuals
qqPlot(resid(model)) # uses raw residuals

If the residuals appear non-normal, then attempt the following:

• Remove the outlying observation(s) in the tails, but only after careful investigation whether that
outlier really was unusual

• Use a suitable transformation of the y-variable

• Add additional terms to the least squares model (page 253)

The simple example shown here builds a model that predicts the price of a used vehicle using only the
mileage as an explanatory variable.

The group of outliers were due to 10 observations of a certain class of vehicle (Cadillac convertibles)
that distorted the model. We removed these observations, which now limits our model to be useful
only for other vehicle types, but we gain a smaller standard error and a tighter confidence interval.
These residuals are still very non-normal though.

Before : 𝑏1 = −0.173 −0.255 ≤ 𝛽1 ≤ −0.0898 𝑆𝐸 = $9789

After : 𝑏1 = −0.155 −0.230 ≤ 𝛽1 ≤ −0.0807 𝑆𝐸 = $8655

The slope coefficient (interpretation: each extra mile on the odometer reduces the sale price on average

4.8. Investigating an existing linear model 243

Process Improvement Using Data

by 15 to 17 cents) has a tighter confidence interval after removing those unusual observations.

Removing the Cadillac cars from our model indicates that there is more than just mileage that affect
their resale value. In fact, the lack of normality, and structure in the residuals leads us to ask which
other explanatory variables can be included in the model.

In the next fictitious example the y-variable is non-linearly related to the x-variable. This non-linearity
in the y shows up as non-normality in the residuals if only a linear model is used. The residuals
become more linearly distributed when using a square root transformation of the y before building the
linear model.

More discussion about transformations of the data is given in the section on model linearity (page 248).

4.8.3 Non-constant error variance

It is common in many situations that the variability in y increases or decreases as y is increased (e.g.
certain properties are more consistently measured at low levels than at high levels). Similarly,
variability in y increases or decreases as x is increased (e.g. as temperature, x, increases the variability
of a particular y increases).

Violating the assumption of non-constant error variance increases the standard error, 𝑆𝐸 , undermining
the estimates of the confidence intervals, and other analyses that depend on the standard error.
Fortunately, it is only problematic if the non-constant variance is extreme, so we can tolerate minor
violations of this assumption.

To detect this problem you should plot:

• the predicted values of y (on the x-axis) against the residuals (y-axis)

• the x values against the residuals (y-axis)

This problem reveals itself by showing a fan shape across the plot; an example is shown in the figure.

244 Chapter 4. Least Squares Modelling Review

 Release 547b16

To counteract this problem one can use weighted least squares, with smaller weights on the
high-variance observations, i.e. apply a weight inversely proportional to the variance. Weighted least
∑︀𝑛
squares minimizes: 𝑓 (b) = 𝑖 (𝑤𝑖 𝑒𝑖 )2 , with different weights, 𝑤𝑖 for each error term. More on this
topic can be found in the book by Draper and Smith (p 224 to 229, 3rd edition).

4.8.4 Lack of independence in the data

Video for
this section The assumption of independence in the data requires that values in the y variable are independent.
Given that we have assumed the x variable to be fixed, this implies that the errors, 𝑒𝑖 are independent.
The reason for independence is required for the central limit theorem, which was used to derive the
various standard errors.

Data are not independent when they are correlated with each other. This is common on slow moving
processes: for example, measurements taken from a large reactor are unlikely to change much from
one minute to the next.

Treating this problem properly comes under the topic of time-series analysis, for which a number of
excellent textbooks exist, in particular the one by Box and Jenkins. But we will show how to detect
autocorrelation, and provide a make-shift solution to avoid it.

If you suspect that there may be lack of independence, use plots of the residuals in time order. Look
for patterns such as slow drifts, or rapid criss-crossing of the zero axis.

One way around the autocorrelation is to subsample - use only every 𝑘 th sample, where 𝑘 is a certain

4.8. Investigating an existing linear model 245

Process Improvement Using Data

number of gaps between the points. How do we know how many gaps to leave? Use the
autocorrelation function87 to determine how many samples. You can use the acf(...) function in R,
which will show how many significant lags there are between observations. Calculating the
autocorrelation accurately requires a large data set, which is a requirement anyway if you need to
subsample your data to obtain independence.

Here are some examples of the autocorrelation plot: in the first case you would have to leave at least 16
samples between each sub-sample, while the second and third cases require a gap of 1 sample, i.e. use
only every second data point.

Another test for autocorrelation is the Durbin-Watson test. For more on this test see the book by
Draper and Smith (Chapter 7, 3rd edition); in R you can use the durbinWatsonTest(model)
function in library(car). Try generating autocorrelation of varying strength (positive, e.g.
phi_long = 0.80 and negative, e.g. phi_long = -0.75) in the code below. Inspect the plots
which are generated as a result, especially the time order plot: get a feeling for what a strong and weak
positive/negative correlation looks like in the time order.

import numpy as np
import pandas as pd
import [Link] as sm

[Link] = "plotly"

# Adjust this autocorrelation parameter:

phi_long = 0.80

N = 1005
data = [Link](N)
for k in range(1, N):
data[k] = ([Link](scale=4)
(continues on next page)

87 [Link]

246 Chapter 4. Least Squares Modelling Review

 Release 547b16

(continued from previous page)

+ phi_long * data[k - 1])
x = data + 50
[Link](x).describe()

# Plot autocorrelation in the first 100 points

[Link](data[:100]).[Link](
title="Raw data").update_layout(
xaxis_title_text="Time order").show()

# Lag-1 scatter plot with a regression line.

xk = x[:1000]
xk1 = x[1:1001]
X = sm.add_constant(xk)
model = [Link](xk1, X).fit()
r = [Link](xk1, xk)[0, 1]

fig = [Link]({"x[k]": xk,

"x[k+1]": xk1}
).[Link](x="x[k]",
y="x[k+1]")
fig.update_xaxes(range=[30, 70])
fig.update_yaxes(range=[30, 70],
scaleanchor="x",
scaleratio=1)
x_line = [Link]([30, 70])
fig.add_scatter(
x=x_line,
y=[Link][0] + [Link][1] * x_line,
mode="lines",
line=dict(color="darkgreen", width=2),
showlegend=False,
)
fig.add_annotation(
x=30, y=30, xanchor="left",
text=f"Correlation = r = {round(r, 2)}",
showarrow=False,
font=dict(color="darkgreen", size=18),
)
[Link]()

# Adjust this autocorrelation parameter:

phi_long = 0.80

N = 1005
data <- numeric(N)
for (k in 2:N){
data[k] = rnorm(1, sd=4) +
phi_long * data[k-1]
}
x <- data + 50
summary(x)

# Plot autocorrelation in the first 100 points

plot(data[1:100], type='b',
main='Raw data', xlab = 'Time order')

[Link]()
lims = c(30,70)
plot(x[1:1000], x[2:1001], asp=1,
xlim=lims, ylim=lims)
model <- lm(x[2:1001] ~ x[1:1000])
abline(model, col="darkgreen", lwd=2)
text(30, 30, paste("Correlation = r = ",
round(cor(x[2:1001],
x[1:1000]), 2)),
(continues on next page)

4.8. Investigating an existing linear model 247

Process Improvement Using Data

(continued from previous page)

col="darkgreen", cex=1.5, adj = c(0, NA))

4.8.5 Linearity of the model (incorrect model specification)

Recall that the linear model is just a tool to either learn more about our data, or to make predictions.
Many cases of practical interest are from systems where the general theory is either unknown, or too
complex, or known to be non-linear.

Certain cases of non-linearity can be dealt with by simple transformations of the raw data: use a
non-linear transformation of the raw data and then build a linear model as usual. An alternative
method which fits the non-linear function, using concepts of optimization, by minimizing the sum of
squares is covered in a section on non-linear regression. Again the book by Draper and Smith (Chapter
24, 3rd edition), may be consulted if this topic is of further interest to you. Let’s take a look at a few
examples.

We saw earlier a case where a square-root transformation of the y variable made the residuals more
normally distributed. There is in fact a sequence of transformations that can be tried to modify the
distribution of a single variable: 𝑥transformed ← 𝑥𝑝original .

• When 𝑝 goes from 1 and higher, say 1.5, 1.75, 2.0, etc, it compresses small values of 𝑥 and inflates
larger values.
√
• When 𝑝 goes down from 1, 0.5 ( 𝑥), 0.25, -0.5, -1.0 (1/𝑥), -1.5, -2.0, etc, it compresses large values of
𝑥 and inflates smaller values.

• The case of log(𝑥) approximates 𝑝 = 0 in terms of the severity of the transformation.

In other instances we may know from first-principles theory, or some other means, what the expected
non-linear relationship is between an x and y variable.

• In a distillation column the temperature, 𝑇 is inversely proportional to the logarithm of the vapour
pressure, 𝑃 . So fit a linear model, 𝑦 = 𝑏0 + 𝑏1 𝑥 where 𝑥 ← 1/𝑇 and where 𝑦 ← 𝑃 . The slope
coefficient will have a different interpretation and a different set of units as compared to the case
when predicting vapour pressure directly from temperature.

• If 𝑦 = 𝑝 × 𝑞 𝑥 , then we can take logs and estimate this equivalent linear model:
log(𝑦) = log(𝑝) + 𝑥 log(𝑞), which is of the form 𝑦 = 𝑏0 + 𝑏1 𝑥. So the slope coefficient will be an
estimate of log(𝑞).
1
• If 𝑦 = , then invert both sides and estimate the model 𝑦 = 𝑏0 + 𝑏1 𝑥 where 𝑏0 ← 𝑝, 𝑏1 ← 𝑞 and
𝑝 + 𝑞𝑥
𝑦 ← 1/𝑦.

• There are plenty of other examples, some classic cases being the non-linear models that arise during
reactor design and biological growth rate models. With some ingenuity (taking logs, inverting the
equation), these can often be simplified into linear models.

• Some cases cannot be linearized and are best estimated by non-linear least squares methods.
However, a make-shift approach which works quite well for simple cases is to perform a grid
search. For example imagine the equation to fit is 𝑦 = 𝛽1 1 − 𝑒−𝛽2 𝑥 , and you are given some data
(︀ )︀

pairs (𝑥𝑖 , 𝑦𝑖 ). Then for example, create a set of trial values 𝛽1 = [10, 20, 30, 40, 50] and
𝛽2 = [0.0, 0.2, 0.4, 0.8]. Build up a grid for each combination of 𝛽1 and 𝛽2 and calculate the sum of
squares objective function for each point in the grid. By trial-and-error you can converge to an

248 Chapter 4. Least Squares Modelling Review

 Release 547b16

approximate value of 𝛽1 and 𝛽2 that best fit the data. You can then calculate 𝑆𝐸 , but not the
confidence intervals for 𝛽1 and 𝛽2 .

Before launching into various transformations or non-linear least squares models, bear in mind that
the linear model may be useful over the region of interest. In the figure we might only be concerned
with using the model over the region shown, even though the system under observation is known to
behave non-linearly over a wider region of operation.

How can we detect when the linear model is not sufficient anymore? While a q-q plot might hint at
problems, better plots are the same two plots for detecting non-constant error variance (page 244):

• the predicted values of y (on the x-axis) against the residuals (y-axis)

• the x values against the residuals (y-axis)

Here we show both plots for the example just prior (where we used a linear model for a smaller
sub-region). The last two plots look the same, because the predicted y
^ values, y
^ = 𝑏0 + 𝑏1 𝑥1 ; in other
words, just a linear transformation of the x values.

4.8. Investigating an existing linear model 249

Process Improvement Using Data

Transformations are considered successful once the residuals appear to have no more structure in
them. Also bear in mind that structure in the residuals might indicate the model is missing an
additional explanatory variable (see the section on multiple linear regression (page 253)).

Another type of plot to diagnose non-linearity present in the linear model is called a
component-plus-residual plot or a partial-residual plot. This is an advanced topic not covered here, but
well covered in the Fox reference (page 210).

4.9 Summary of steps to build and investigate a linear model

1. Plot the data to assess model structure and degree of correlation between the x and y variable.

plot(x, y) # plot the raw data

lines(lowess(x,y)) # superimpose non-parametric smoother to see correlation

2. Fit the model and examine the printed output.

model <- lm(y ~ x) # fit the model: "y as described by variable x"
summary(model)
confint(model)

• Investigate the model’s standard error, how does it compare to the range of the y variable?

• Calculate confidence intervals for the model parameters and interpret them.

3. Visualize the model’s predictions in the context of the model building data.

plot(x, y)
lines(lowess(x,y)) # show the smoother
abline(model, col="red") # and show the least squares model

4. Plot a normal probability plot, or a q-q plot, of the residuals. Are they normally distributed? If not,
investigate if a transformation of the y variable might improve them. But also see the additional
plots on checking for non-linearity and consider adding extra explanatory variables.

library(car)
qqPlot(resid(model))

5. Plot the residuals against the x-values. We expect to see no particular structure. If you see trends in

250 Chapter 4. Least Squares Modelling Review

 Release 547b16

the data, it indicates that a transformation of the x variable might be appropriate, or that there are
unmodelled phenomena in the y variable - we might need an additional x variable.

plot(x, resid(model))
abline(h=0, col="red")

6. Plot the residuals in time (sequence) order. We expect to see no particular trends in the data. If there
are patterns in the plot, assess whether autocorrelation is present in the y variable (use the acf(y)
function in R). If so, you might have to sub-sample the data, or resort to proper time-series analysis
tools to fit your model.

plot(resid(model))
abline(h=0, col="red")
lines(lowess(resid(model), f=0.2)) # use a shorter smoothing span

7. Plot the residuals against the fitted-values. By definition of the least-squares model, the covariance
∑︀𝑛
between the residuals and the fitted values is zero. You can verify that 𝑒𝑇 𝑦^ = 𝑖 𝑒𝑖 𝑦^𝑖 = 0. A
fan-shape to the residuals indicates the residual variance is not constant over the range of data: you
will have to use weighted least squares to counteract that. It is better to use studentized residuals
(page 262), rather than the actual residuals, since the actual residuals can show non-constant
variance even though the errors have constant variance.

plot(predict(model), rstudent(model))
lines(lowess(predict(model), rstudent(model)))
abline(h=0, col="red")

8. Plot the predictions of y against the actual values of y. We expect the data to fall around a 45 degree
line.

plot(y, predict(model))
lines(lowess(y, predict(model), f=0.5)) # a smoother
abline(a=0, b=1, col="red") # a 45 degree line

4.9.1 A worked example of the workflow in Python

The steps above describe a general workflow that applies to any data analysis project, not just least
squares. We illustrate the early steps of that workflow on the blender efficiency88 data set, which
records the result of designed experiments where four factors were varied to study blending efficiency:
ParticleSize, MixerDiameter, MixerRotation, and BlendingTime.

The six-step workflow on these data:

1. Define the objective: understand which factors drive BlendingEfficiency.

2. Get the data:

import pandas as pd

blender = pd.read_csv(
(continues on next page)

88 [Link]

4.9. Summary of steps to build and investigate a linear model 251

Process Improvement Using Data

(continued from previous page)

"[Link]
)

3. Explore: look at a few rows, the column types, and a numeric summary.

[Link]()
[Link]()
[Link]()
[Link]()

4. Clean: in this case the data are pre-cleaned, so we move on.

5. Calculate: a correlation matrix and scatter-plot matrix highlight which factors are most related to
the outcome variable.

from [Link] import scatter_matrix

# Numeric correlation matrix:

[Link]()

# Visual version: scatter plot of every

# pair of variables, with a kde on the
# diagonal.
scatter_matrix(
blender,
alpha=0.2,
figsize=(10, 8),
diagonal="kde",
)

Filtering and grouping are part of the daily work of anyone working with data, and Pandas makes
both very compact:

# Boolean indexing returns only the rows

# where the condition is true:
blender[blender["ParticleSize"] == 2]
blender[blender["ParticleSize"] <= 5]
blender[blender["ParticleSize"] > 5]

# groupby applies the same calculation

# to each value of the grouping variable:
[Link]("ParticleSize").mean()
[Link]("ParticleSize").std()
[Link]("ParticleSize").max()

6. Communicate: create a separate plot per particle size, so each subgroup can be inspected on its own
axes:

for psize, subset in [Link]("ParticleSize"):

ax = [Link](
x="BlendingTime",
y="BlendingEfficiency",
)
ax.set_title(f"When particle size = {psize}")

252 Chapter 4. Least Squares Modelling Review

 Release 547b16

Once the patterns are clear, you can fit a least squares model using scikit-learn (page 224) or
statsmodels (page 222) and continue with the residual diagnostics outlined in the steps above.

4.10 More than one variable: multiple linear regression (MLR)

Video for
this section We now move to including more than one explanatory x variable in the linear model. We will:

1. introduce some matrix notation for this section

2. show how the optimization problem is solved to estimate the model parameters

3. how to interpret the model coefficients

4. extend our tools from the previous section to analyze the MLR model

5. use integer (yes/no or on/off) variables in our model.

First some motivating examples:

• A relationship exists between 𝑥1 = reactant concentration and 𝑥2 = temperature with respect to 𝑦 =

reaction rate. We already have a linear model between 𝑦 = 𝑏0 + 𝑏1 𝑥1 , but we want to improve our
understanding of the system by learning about the temperature effect, 𝑥2 .

• We want to predict melt index in our reactor from the reactor temperature, but we know that the
feed flow and pressure are also good explanatory variables for melt index. How do these additional
variables improve the predictions?

• We know that the quality of our plastic product is a function of the mixing time, and also the mixing
tank in which the raw materials are blended. How do we incorporate the concept of a mixing tank
indicator in our model?

4.10.1 Multiple linear regression: notation

Video for
this section To help the discussion below it is useful to omit the least squares model’s intercept term. We do this by
first centering the data.

𝑦𝑖 = 𝑏0 + 𝑏1 𝑥𝑖
𝑦 = 𝑏0 + 𝑏1 𝑥
𝑦𝑖 − 𝑦 = 0 + 𝑏1 (𝑥𝑖 − 𝑥) by subtracting the previous lines from each other

This indicates that if we fit a model where the x and y vectors are first mean-centered, i.e. let
(︀ )︀ (︀ )︀
𝑥 = 𝑥original − mean 𝑥original and 𝑦 = 𝑦original − mean 𝑦original , then we still estimate the same slope
for 𝑏1 , but the intercept term is zero. All we gain from this is simplification of the subsequent analysis.
Of course, if you need to know what 𝑏0 was, you can use the fact that 𝑏0 = 𝑦 − 𝑏1 𝑥. Nothing else
changes: the 𝑅2 , 𝑆𝐸 , 𝑆𝐸 (𝑏1 ) and all other model interpretations remain the same. You can easily prove
this for yourself.

So in the rest of the this section we will omit the model’s intercept term, since it can always be
recovered afterwards.

4.10. More than one variable: multiple linear regression (MLR) 253
Process Improvement Using Data

The general linear model is given by:

𝑦𝑖 = 𝛽1 𝑥1 + 𝛽2 𝑥2 + . . . + 𝛽𝑘 𝑥𝑘 + 𝜖𝑖
⎡ ⎤
𝛽1
⎢ 𝛽2 ⎥
⎢ ⎥
⎢ .. ⎥ + 𝜖𝑖
𝑦𝑖 = [𝑥1 , 𝑥2 , . . . , 𝑥𝑘 ] ⎢ ⎥
⎣ . ⎦
𝛽𝑘
𝑦𝑖 = ⏟x 𝑇⏞ 𝛽 + 𝜖𝑖
⏟ ⏞
(1×𝑘) (𝑘×1)

And writing the last equation 𝑛 times over for each observation in the data:
⎡ ⎤ ⎡ ⎤⎡ ⎤ ⎡ ⎤
𝑦1 𝑥1,1 𝑥1,2 . . . 𝑥1,𝑘 𝑏1 𝑒1
𝑦 𝑥 𝑥 . . . 𝑥 𝑏 ⎢ 𝑒2 ⎥
⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥
⎢ 2 ⎥ ⎢ 2,1 2,2 2,𝑘 ⎥ ⎢ 2 ⎥
⎢ . ⎥=⎢ . . . ⎥⎢ . ⎥ + ⎢ . ⎥
⎢.⎥ ⎢ . .. .. .. ⎥
⎣.⎦ ⎣ . . ⎦ ⎣ .. ⎦ ⎣ .. ⎦
⎢ ⎥ ⎢ ⎥

𝑦𝑛 𝑥𝑛,1 𝑥𝑛,2 . . . 𝑥𝑛,𝑘 𝑏𝑘 𝑒𝑛

y = Xb + e

where:

• y: 𝑛 × 1

• X: 𝑛 × 𝑘

• b: 𝑛 × 1

• e: 𝑛 × 1

4.10.2 Estimating the model parameters via optimization

As with the simple least squares model, 𝑦 = 𝑏0 + 𝑏1 𝑥, we aim to minimize the sum of squares of the
errors in vector e. This least squares objective function can be written compactly as:

𝑓 (b) = e𝑇 e
𝑇
= (y − Xb) (y − Xb)
= y𝑇 y − 2y𝑇 Xb + bX𝑇 Xb

Taking partial derivatives with respect to the entries in b and setting the result equal to a vector of
)︀−1 𝑇
X y. You might find the Matrix Cookbook89
(︀
zeros, you can prove to yourself that b = X𝑇 X
useful in solving these equations and optimization problems.

Three important relationships are now noted:

1. ℰ{b} = 𝛽
(︀ )︀−1 2
2. 𝒱{b} = X𝑇 X 𝑆𝐸
√︂
e𝑇 e
3. An estimate of the standard error is given by: 𝜎𝑒 ≈ 𝑆𝐸 = , where 𝑘 is the number of
𝑛−𝑘
parameters estimated in the model and 𝑛 is the number of observations.
89 [Link]

254 Chapter 4. Least Squares Modelling Review

 Release 547b16

These relationships imply that our estimates of the model parameters are unbiased (the first line), and
that the variability of our parameters is related to the X𝑇 X matrix and the model’s standard error, 𝑆𝐸 .

Going back to the single variable case we showed in the section where we derived confidence intervals
(page 230) for 𝑏0 and 𝑏1 that:

2
𝑆𝐸
𝒱{𝑏1 } = ∑︀ 2
𝑗 (𝑥𝑗 − x)

(︀ )︀−1 2
Notice that our matrix definition, 𝒱{b} = X𝑇 X 𝑆𝐸 , gives exactly the same result, remembering
the x variables have already been centered in the matrix form. Also recall that the variability of these
estimated parameters can be reduced by (a) taking more samples, thereby increasing the denominator
size, and (b) by including observations further away from the center of the model.

Example

Let 𝑥1 = [1, 3, 4, 7, 9, 9], and 𝑥2 = [9, 9, 6, 3, 1, 2], and 𝑦 = [3, 5, 6, 8, 7, 10]. By inspection, the 𝑥1 and 𝑥2
variables are negatively correlated, and the 𝑥1 and 𝑦 variables are positively correlated (also positive
covariance). Refer to the definition of covariance in an equation from the prior section (page 211).

After mean centering the data we have that 𝑥1 = [−4.5, −2.5, −1.5, 1.5, 3.5, 3.5], and
𝑥2 = [4, 4, 1, −2, −4, −3] and 𝑦 = [−3.5, −1.5, −0.5, 1.5, 0.5, 3.5]. So in matrix form:
⎡ ⎤ ⎡ ⎤
−4.5 4 −3.5
⎢−2.5 4 ⎥ ⎢−1.5⎥
⎢ ⎥ ⎢ ⎥
⎢ ⎥ ⎢ ⎥
⎢−1.5 1 ⎥ ⎢−0.5⎥
X=⎢ ⎢ 1.5 −2⎥
⎥ y = ⎢
⎢ 1.5 ⎥
⎥
⎢ ⎥ ⎢ ⎥
⎣ 3.5 −4⎦ ⎣ 0.5 ⎦
⎢ ⎥ ⎢ ⎥
3.5 −3 3.5

The X𝑇 X and X𝑇 y matrices can then be calculated as:

[︃ ]︃ [︃ ]︃
55.5 −57.0 36.5
X𝑇 X = X𝑇 y =
−57.0 62 −36.0

Notice what these matrices imply (remembering that the vectors in the matrices have been centered).
The X𝑇 X matrix is a scaled version of the covariance matrix of X. The diagonal terms show how
strongly the variable is correlated with itself, which is the variance, and always a positive number. The
off-diagonal terms are symmetrical, and represent the strength of the relationship between, in this
case, 𝑥1 and 𝑥2 . The off-diagonal terms for two uncorrelated variables would be a number close to, or
equal to zero.

The inverse of the X𝑇 X matrix is particularly important - it is related to the standard error for the
(︀ )︀−1 2
model parameters - as in: 𝒱{b} = X𝑇 X 𝑆𝐸 .
[︃ ]︃
(︀ 𝑇 )︀−1 0.323 0.297
X X =
0.297 0.289

The non-zero off-diagonal elements indicate that the variance of the 𝑏1 coefficient is related to the
variance of the 𝑏2 coefficient as well. This result is true for most regression models, indicating we can’t
accurately interpret each regression coefficient’s confidence interval on its own.

4.10. More than one variable: multiple linear regression (MLR) 255
 Process Improvement Using Data

For the two variable case, 𝑦 = 𝑏1 𝑥1 + 𝑏2 𝑥2 , the general relationship is that:

1 𝑆2
𝒱 (𝑏1 ) = 2 × ∑︀ 𝐸 2
1 − 𝑟12 𝑥1
2
1 𝑆
𝒱 (𝑏2 ) = 2 ×
∑︀ 𝐸 2
1 − 𝑟12 𝑥2
2
where 𝑟12 represents the correlation between variable 𝑥1 and 𝑥2 . What happens as the correlation
between the two variables increases?

4.10.3 Interpretation of the model coefficients

Video for
this section Let’s take a look at the case where 𝑦 = 𝑏1 𝑥1 + 𝑏2 𝑥2 . We can plot this on a 3D plot, with axes of 𝑥1 , 𝑥2
and 𝑦:

The points are used to fit the plane by minimizing the sum of square distances shown by vertical lines
from each point to the plane. The interpretation of the slope coefficients for 𝑏1 and 𝑏2 is not the same as
for the case with just a single x variable.

When we have multiple x variables, then the value of coefficient 𝑏1 is the average change we would
expect in y for a one unit change in 𝑥1 provided we hold 𝑥2 fixed. It is the last part that is new: we
must assume that other x variables are fixed.

For example, let 𝑦 = 𝑏𝑇 𝑇 + 𝑏𝑆 𝑆 = −0.52𝑇 + 3.2𝑆, where 𝑇 is reactor temperature in Kelvin, and 𝑆 is
substrate concentration in g/L, and 𝑦 is yield in 𝜇g, for a bioreactor reactor system. The
𝑏𝑇 = −0.52𝜇g/K coefficient is the decrease in yield for every 1 Kelvin increase in temperature, holding
the substrate concentration fixed.

This is a good point to introduce some terminology you might come across. Imagine you have a model
where 𝑦 is the used vehicle price and 𝑥1 is the mileage on the odometer (we expect that 𝑏1 will be
negative) and 𝑥2 is the number of doors on the car. You might hear the phrase: “the effect of the
number of doors, controlling for mileage, is not significant”. The part “controlling for . . . ” indicates
that the controlled variable has been added to regression model, and its effect is accounted for. In other

256 Chapter 4. Least Squares Modelling Review

 Release 547b16

words, for two vehicles with the same mileage, the coefficient 𝑏2 indicates whether the second hand
price increases or decreases as the number of doors on the car changes (e.g. a 2-door vs a 4-door car).

In the prior example, we could say: the effect of substrate concentration on yield, controlling for
temperature, is to increase the yield by 3.2 𝜇g for every increase in 1 g/L of substrate concentration.

Fitting an MLR model with scikit-learn

In the single-x section (page 224) we built a model on the distillation tower data using only InvTemp3
as a predictor of VapourPressure. We now extend that model by adding a second predictor,
InvPressure1 (the inverse of a pressure measurement). The scikit-learn API requires almost no
change: we simply pass a list of column names instead of a single one.

import pandas as pd
from sklearn.linear_model import LinearRegression

distill = pd.read_csv(
"[Link]
)
build = [Link][:150]
test = [Link][150:]

# Specifying the predictors as a list lets us

# add or remove variables without touching
# the rest of the code:
predictors = ["InvTemp3", "InvPressure1"]

X_build_MLR = build[predictors].values
y_build = build["VapourPressure"].values

full_model = LinearRegression()
full_model.fit(X=X_build_MLR, y=y_build)

# Residuals on the building data:

predict_MLR_build = full_model.predict(X_build_MLR)
errors_MLR_build = y_build - predict_MLR_build
avg_absolute_error_MLR_build = (
[Link](errors_MLR_build).abs().mean()
)
print(
f"MLR building-data average absolute error "
f"= {avg_absolute_error_MLR_build:.3f}"
)

[Link](errors_MLR_build).plot(
grid=True,
title="MLR residuals (Actual - Predicted)",
)

The same model is then assessed on the held-out test partition. Compare the average absolute error
and standard deviation against the values reported in the single-predictor case (page 242): a useful
additional predictor should reduce both numbers, and leave the residual time series free of obvious
drift or structure.

X_test_MLR = test[predictors].values
y_test = test["VapourPressure"].values

predict_MLR_test = full_model.predict(X_test_MLR)
errors_MLR_test = y_test - predict_MLR_test

avg_absolute_error_MLR_test = (
(continues on next page)

4.10. More than one variable: multiple linear regression (MLR) 257
 Process Improvement Using Data

(continued from previous page)

[Link](errors_MLR_test).abs().mean()
)
std_error_MLR_test = errors_MLR_test.std()

print(
f"MLR testing-data average absolute error "
f"= {avg_absolute_error_MLR_test:.3f}, "
f"std. dev. = {std_error_MLR_test:.3f}"
)

4.10.4 Integer (dummy, indicator) variables in the model

Video for
this section Now that we have introduced multiple linear regression to expand our models, we also consider these
sort of cases:

• We want to predict yield, but want to indicate whether a radial or axial impeller was used in
the reactor and learn whether it has any effect on yield.

• Is there an important difference when we add the catalyst first and then the reactants, or the
reactants followed by the catalyst?

• Use an indicator variable to show if the raw material came from the supplier in Spain, India,
or Vietnam and interpret the effect of supplier on yield.

Axial and radial blades; figure from Wikipedia90

We will start with the simplest case, using the example of the radial or axial impeller. We wish to
understand the effect on yield, 𝑦[𝜇g], as a function of the impeller type, and impeller speed, 𝑥.

𝑦 = 𝛽0 + 𝛽1 𝑥 + 𝛾𝑑 + 𝜀
𝑦 = 𝑏0 + 𝑏1 𝑥 + 𝑔𝑑𝑖 + 𝑒𝑖

where 𝑑𝑖 = 0 if an axial impeller was used, or 𝑑𝑖 = 1 if a radial impeller was used. All other least
squares assumptions hold, particularly that the variance of 𝑦𝑖 is unrelated to the value of 𝑑𝑖 . For the
initial discussion let’s assume that 𝛽1 = 0, then geometrically, what is happening here is:
90 [Link]

258 Chapter 4. Least Squares Modelling Review

 Release 547b16

The 𝛾 parameter, estimated by 𝑔, is the difference in intercept when using a different impeller type.
Note that the lines are parallel.

Axial impellers: 𝑦 = 𝑏0 + 0
Radial impellers: 𝑦 = 𝑏0 + 𝑔

Now if 𝛽1 ̸= 0, then the horizontal lines in the above figure are tilted, but still parallel to each other.
Nothing else is new here, other than the representation of the variable used for 𝑑𝑖 . The interpretation of
its coefficient, 𝑔, is the same as with any other least squares coefficient. In this particular example, had
𝑔 = −56𝜇g, it would indicate that the average decrease in yield is 56 𝜇g when using a radial impeller.

The rest of the analysis tools for least squares models can be used quite powerfully. For example, a
95% confidence interval for the impeller variable might have been:

−32𝜇g ≤ 𝛾 ≤ 21𝜇g

which would indicate the impeller type has no significant effect on the yield amount, the 𝑦-variable.

Integer variables are also called dummy variables or indicator variables. Really what is happening
here is the same concept as for multiple linear regression, the equation of a plane is being estimated.
We only use the equation of the plane at integer values of 𝑑, but mathematically the underlying plane
is actually continuous.

4.10. More than one variable: multiple linear regression (MLR) 259
Process Improvement Using Data

We have to introduce additional terms into the model if we have integer variables with more than 2
levels. In general, if there are 𝑝-levels, then we must include 𝑝 − 1 terms. For example, if we wish to
test the effect of 𝑦 = yield achieved from the raw material supplier in Spain, India, or Vietnam, we
could code:

• Spain: 𝑑𝑖1 = 0 and 𝑑𝑖2 = 0

• India: 𝑑𝑖1 = 1 and 𝑑𝑖2 = 0

• Vietnam: 𝑑𝑖1 = 0 and 𝑑𝑖2 = 1.

and solve for the least squares model: 𝑦 = 𝛽0 + 𝛽1 𝑥1 + . . . + 𝛽𝑘 𝑥𝑘 + 𝛾1 𝑑1 + 𝛾2 𝑑2 + 𝜀, where 𝛾1 is the

effect of the Indian supplier, holding all other terms constant (i.e. it is the incremental effect of India
relative to Spain); 𝛾2 is the incremental effect of the Vietnamese supplier relative to the base case of the
Spanish supplier. Because of this somewhat confusing interpretation of the coefficients, sometimes
people will assume they can sacrifice an extra degree of freedom, but introduce 𝑝 new terms for the 𝑝
levels of the integer variable, instead of 𝑝 − 1 terms.

• Spain: 𝑑𝑖1 = 1 and 𝑑𝑖2 = 0 and 𝑑𝑖3 = 0

• India: 𝑑𝑖1 = 0 and 𝑑𝑖2 = 1 and 𝑑𝑖3 = 0

• Vietnam: 𝑑𝑖1 = 0 and 𝑑𝑖2 = 0 and 𝑑𝑖3 = 1

and 𝑦 = 𝛽0 + 𝛽1 𝑥1 + . . . + 𝛽𝑘 𝑥𝑘 + 𝛾1 𝑑1 + 𝛾2 𝑑2 + 𝛾3 𝑑3 + 𝜀, where the coefficients 𝛾1 , 𝛾2 and 𝛾3 are

assumed to be more easily interpreted. However, calculating this model will fail, because there is a
built-in perfect linear combination. The X𝑇 X matrix is not invertible.

4.11 Outliers: discrepancy, leverage, and influence of the observations

Unusual observations will influence the model parameters and also influence the analysis from the
model (standard errors and confidence intervals). In this section we will examine how these outliers
influence the model.

260 Chapter 4. Least Squares Modelling Review

 Release 547b16

Outliers are in many cases the most interesting data in a data table. They indicate whether there was a
problem with the data recording system, they indicate sometimes when the system is operating really
well, though more likely, they occur when the system is operating under poor conditions.
Nevertheless, outliers should be carefully studied for (a) why they occurred and (b) whether they
should be retained in the model.

4.11.1 Background

A discrepancy is a data point that is unusual in the context of the least squares model, as shown in the first
figure here. On its own, from the perspective of either x or y alone, the square point is not unusual. But
it is unusual in the context of the least squares model. When that square point is removed, the updated
least squares line (dashed line) is obtained. This square point clearly has little influence on the model,
even though it is discrepant.

The discrepant square point in model B has much more influence on the model. Given that the
objective function aims to minimize the sum of squares of the deviations, it is not surprising that the
slope is pulled towards this discrepant point. Removing that point gives a different dashed-line
estimate of the slope and intercept.

In model C the square point is not discrepant in the context of the model. But it does have high
leverage on the model: a small change in this point has the potential to be influential on the model.

Can we quantify how much influence these discrepancies have on the model; and what is leverage? The
following general formula is helpful in the rest of this discussion:

Leverage × Discrepancy = Influence on the model

4.11.2 Leverage
Leverage measures how much each observation contributes to the model’s prediction of 𝑦^𝑖 . It is also
called the hat value, ℎ𝑖 , and simply measures how far away the data point is from the center of the
model, but it takes the model’s correlation into account:

2
1 (𝑥𝑖 − 𝑥) 𝑘 1
ℎ𝑖 = + ∑︀𝑛 2 and ℎ= and ≤ ℎ𝑖 ≤ 1.0
𝑛 𝑗=1 (𝑥𝑗 − 𝑥)
𝑛 𝑛

The average hat value can be calculated theoretically. While it is common to plot lines at 2 and 3 times
the average hat value, always plot your data and judge for yourself what a large leverage means. Also

4.11. Outliers: discrepancy, leverage, and influence of the observations 261

Process Improvement Using Data

notice that smallest hat value is always positive and greater or equal to 1/𝑛, while the largest hat value
possible is 1.0. Continuing the example of models A, B and C: the hat values for models B and C are
the same, and are shown here. The last point has very high leverage.

4.11.3 Discrepancy
Discrepancy can be measured by the residual distance. However the residual is not a complete
measure of discrepancy. We can imagine cases where the point has such high leverage that it drags the
entire model towards it, leaving it only with a small residual. One way then to isolate these points is to
divide the residual by 1 − leverage = 1 − ℎ𝑖 . So we introduce a new way to quantify the residuals here,
called studentized residuals:

𝑒𝑖
𝑒*𝑖 = √
𝑆𝐸(−𝑖) 1 − ℎ𝑖

Where 𝑒𝑖 is the residual for the 𝑖th point, as usual, but 𝑆𝐸(−𝑖) is the standard error of the model when
deleting the 𝑖th point and refitting the model. This studentized residual accounts for the fact that high
leverage observations pull the model towards themselves. In practice the model is not recalculated by
omitting each point one at a time, rather there are shortcut formula that implement this efficiently. Use
the rstudent(lm(y~x)) function in R to compute the studentized residuals from a given model.

This figure illustrates how the square point in model A and B is highly discrepant, while in model C it
does not have a high discrepancy.

262 Chapter 4. Least Squares Modelling Review

 Release 547b16

4.11.4 Influence
The influence of each data point can be quantified by seeing how much the model changes when we
omit that data point. The influence of a point is a combination its leverage and its discrepancy. In
model A, the square point had large discrepancy but low leverage, so its influence on the model
parameters (slope and intercept) was small. For model C, the square point had high leverage, but low
discrepancy, so again the change in the slope and intercept of the model was small. However model B
had both large discrepancy and high leverage, so its influence is large.

One measure is called Cook’s statistic, usually called 𝐷𝑖 , and often referred to just as Cook’s D.
Conceptually, it can be viewed as the change in the model coefficients when omitting an observation,
however it is much more convenient to calculate it as follows:

𝑒2𝑖 ℎ𝑖
𝐷𝑖 = 1 ×
𝑒2𝑖 1 − ℎ𝑖
∑︀
𝑘× 𝑛

where 𝑛1
∑︀ 2
𝑒𝑖 is called the mean square error of the model (the average square error). It is easy to see
here now why influence is the product of discrepancy and leverage.

The values of 𝐷𝑖 are conveniently calculated in R using the [Link](model) function. The
results for the 3 models are shown. Interestingly for model C there is a point with even higher
influence than the square point. Can you locate that point in the least squares plot?

4.12 Enrichment topics

These topics are not covered in depth in this book, but might be of interest to you. I provide a small
introduction to each topic, showing what their purpose is, together with some examples.

4.12.1 Nonparametric models

Nonparametric modelling is a general model where the relationship between 𝑥 and 𝑦 is of the form:
𝑦 = 𝑓 (𝑥) + 𝜀, but the function 𝑓 (𝑥), i.e. the model, is left unspecified. The model is usually a smooth
function.

Consider the example of plotting Prestige (the Pineo-Porter prestige91 score) against Income, from the
1971 Canadian census. A snippet of the data is given by:

91 [Link]

4.12. Enrichment topics 263

Process Improvement Using Data

education income women prestige census type

ECONOMISTS 14.44 8049 57.31 62.2 2311 prof
[Link] 15.22 9593 34.89 58.3 2391 prof
PHYSICIANS 15.96 25308 10.56 87.2 3111 prof
[Link] 9.45 3485 76.14 34.9 3135 bc
[Link] 10.07 3739 52.27 37.2 4173 wc
[Link] 11.43 6259 39.17 35.7 4193 wc
BABYSITTERS 9.46 611 96.53 25.9 6147 <NA>
BAKERS 7.54 4199 33.30 38.9 8213 bc
MASONS 6.60 5959 0.52 36.2 8782 bc
[Link] 7.81 4549 2.46 29.9 8785 bc

The plot on the left is the raw data, while on the right is the raw data with the nonparametric model
(line) superimposed. The smoothed line is the nonparametric function, 𝑓 (𝑥), referred to above, and 𝑥 =
Income ($), and 𝑦 = Prestige.

For bivariate cases, the nonparametric model is often called a scatterplot smoother. There are several
methods to calculate the model; one way is by locally weighted scatterplot smoother (LOESS),
described as follows. Inside a fixed subregion along the 𝑥-axis (called the window):

• collect the 𝑥- and 𝑦-values inside this window

• calculate a fitted 𝑦-value, but use a weighted least squares procedure, with weights that peak at the
center of the window and declines towards the edges,

• record that average 𝑦-value against the window’s center (𝑥-value)

• slide the window along the 𝑥 axis and repeat

The model is the collection of these 𝑥- and 𝑦-values. This is why it is called nonparameteric: there are
no parameters to quantify the model. For example: if the relationship between the two variables is
linear, then a linear smooth is achieved. It is hard to express the relationship between 𝑥 and 𝑦 in
written form, so usually these models are shown visually. The nonparametric model is not immune to
outliers, but it is resistant to them.

More details can be found in W.S. Cleveland, Robust Locally Weighted Regression and Smoothing
Scatterplots92 , Journal of the American Statistical Association, 74 (368), p. 829-836, 1979.
92 [Link]

264 Chapter 4. Least Squares Modelling Review

 Release 547b16

4.12.2 Robust least squares models

Outliers are often the most interesting observations and are usually the points from which we learn the
most about the system. A manual step where we review the outliers and their influence should always
done for any important model. For example, inspection of the residual plots as described in the
preceding sections.

However, the ability to build a linear model that is not heavily influenced by outliers might be of
interest in certain cases.

• The model is built automatically and is not reviewed by a human (e.g. as an intermediate step in a
data-mining procedure). This is increasingly common in systems that build on top of the least
squares model to improve their performance in some way.

• The human reviewer is not skilled to know which plots to inspect for influential and discrepant
observations, or may not know how to interpret these plots.

Some criticism of robust methods are that there are too many different robust methods and that these
routines are much more computationally expensive than ordinary least squares. The first point is true,
as this as a rapidly evolving field, however the latter objection is not of too much concern these days.
Robust methods are now available in most decent software packages, and are stabilizing towards a
few reliable robust estimators.

If you would like to read up some more, a nice introduction targeted at engineering readers is given in
PJ Rousseeuw’s “Tutorial to Robust Statistics93 ”, Journal of Chemometrics, 5, 1-20, 1991.

In R the various efforts of international researchers is being consolidated. The robustbase package
provides basic functionality that is now well established in the field; use that package if you want to
assemble various robust tools yourself. On the other hand, a more comprehensive package called
robust is also available which provides robust tools that you should use if you are not too concerned
with the details of implementation.

For example:

> data <- [Link]('[Link]

# Using ordinary least squares

# -----------------------------
> summary(lm(data$VapourPressure ~ data$TempC2))

Call:
lm(formula = data$VapourPressure ~ data$TempC2)

Residuals:
Min 1Q Median 3Q Max
-5.59621 -2.37597 0.06674 2.00212 14.18660

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 195.96141 4.87669 40.18 <2e-16 ***
data$TempC2 -0.33133 0.01013 -32.69 <2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 2.989 on 251 degrees of freedom

Multiple R-squared: 0.8098, Adjusted R-squared: 0.8091
F-statistic: 1069 on 1 and 251 DF, p-value: < 2.2e-16

(continues on next page)

93 [Link]

4.12. Enrichment topics 265

Process Improvement Using Data

(continued from previous page)

# Use robust least squares (with automatic selection of robust method)
# --------------------------------------------------------------------
> library(robust)
> summary(lmRob(data$VapourPressure ~ data$TempC2))

Call: lmRob(formula = data$VapourPressure ~ data$TempC2)

Residuals:
Min 1Q Median 3Q Max
-5.2631296 -1.9805384 0.1677174 2.1565730 15.8846460

Coefficients:
Value Std. Error t value Pr(>|t|)
(Intercept) 179.48579886 4.92870640 36.41641120 0.00000000
data$TempC2 -0.29776778 0.01021412 -29.15256677 0.00000000

Residual standard error: 2.791 on 251 degrees of freedom

Multiple R-Squared: 0.636099

Test for Bias:

statistic p-value
M-estimate 7.962583 0.018661525
LS-estimate 12.336592 0.002094802

In this example the two models perform similarly in terms on their 𝑆𝐸 , 𝑏0 and 𝑏1 values, as well as
confidence intervals for them.

4.12.3 Logistic modelling (regression)

There are many practical cases in engineering modelling where our y-variable is a discrete entity. The
most common case is pass or failure, naturally coded as y = 0 for failure, and y = 1 is coded as success.
Some examples:

• Predict whether our product specifications are achieved (y = 0 or 1) given the batch reaction’s
temperature as 𝑥1 , the reaction duration 𝑥2 and the reactor vessel, where 𝑥3 = 0 for reactor A and
𝑥3 = 1 for reactor B.

• Predict the likelihood of making a sale in your store (y = 0 or 1), given the customer’s age 𝑥1 ,
whether they are a new or existing customers, 𝑥2 is either 0 or 1, and the day of the week as 𝑥3 .

• Predict if the final product will be y = acceptable, medium, or unsellable based on the raw material’s
properties 𝑥1 , 𝑥2 , 𝑥3 and the ambient temperature 𝑥4 .

We could naively assume that we just code our y variable as 0 or 1 (pass/fail) and build our least
squares model as usual, using the x variables. While a seemingly plausible approach, the problems are
that:

• The predictions when using the model are not dichotomous (0 or 1), which is not too much of a
problem if we interpret our prediction more as a probability. That is, our prediction is the
probability of success or failure, according to how we coded it originally. However the predictions
often lie outside the range [0, 1]. We can attempt to compensate for this by clamping the output to
zero or one, but this non-linearity causes instability in the estimation algorithms.

• The errors are not normally distributed.

• The variance of the errors are not constant and the assumption of linearity breaks down.

266 Chapter 4. Least Squares Modelling Review

 Release 547b16

A logistic model however accounts for the nature of the y-variable by creating a function, called a
logistic function, which is bounded between 0 and 1. In fact you are already familiar with such a
function: the cumulative probability of the normal distribution does exactly this.

Once the 𝑦 data are appropriately transformed, then the model can be calculated. In R one uses the
glm(y ~ x1 + x2, family=binomial) function to build a model where y must be a factor
variable: type help(factor) to learn more. The model output is interpreted as any other.

4.12.4 Testing of least-squares models

Before launching into this concept, first step back and understand why we are building least squares
models. One objective is to learn more about our systems: (a) what is the effect of one variable on
another, or (b) is the effect significant (examine the confidence interval). Another objective is purely
predictive: build a model so that we can use it to make predictions. For this last case we must test our
model’s capability for accurate predictions.

The gold standard is always to have a testing data set available to quantify how good (adequate) your
least squares model is. It is important that (a) the test set has no influence on the calculation of the
model parameters, and (b) is representative of how the model will be used in the future. We will
illustrate this with 2 examples: you need to build a predictive model for product viscosity from 3
variables on your process. You have data available, once per day, for 2006 and 2007 (730 observations).

• Use observation 1, 3, 5, 7, . . . 729 to build the least squares model; then use observation 2, 4, 6, 8, . . .
730 to test the model.

• Use observations 1 to 365 (data from 2006) to build the model, and then use observations 366 to 730
(data from 2007) to test the model.

In both cases, the testing data has no influence on the model parameters. However the first case is not
representative of how the model will be used in the future. The results from the first case are likely to
give over-optimistic results, while the second case represents the intended use of the model more
closely, and will have more honest results. Find out sooner, rather than later, that the model’s
long-term performance is not what you expect. It may be that you have to keep rebuilding the model

4.12. Enrichment topics 267

Process Improvement Using Data

every 3 months, updating the model with the most recent data, in order to maintain it’s predictive
performance.

How do we quantify this predictive performance? A common way is to calculate the root mean square
of the prediction error (RMSEP), this is very similar to the standard error (page 226) that we saw earlier
for regression models. Assuming the errors are centered at zero and follow a normal distribution, the
RMSEP can be interpreted as the standard deviation of the prediction residuals. It is important the
RMSEP be calculated only from new, unseen testing data. By contrast, you might see the term RMSEE
(root mean square error of estimation), which is the RMSEP, but calculated from the training data
(model-building data). The RMSEE ≈ 𝑆𝐸 = standard error; the small difference being due to the
denominator used (𝑛 versus 𝑛 − 𝑘).
⎯
⎸ 𝑛
⎸ 1 ∑︁ 2
RMSEP = ⎷ (𝑦new,𝑖 − 𝑦^new,𝑖 )
𝑛 𝑖

The units of RMSEP and RMSEE are the same as the units of the y-variable.

In the latent variable modelling (page 411) section of the book we will introduce the concept of
cross-validation to test a model. Cross-validation uses the model training data to simulate the testing
process. So it is not as desirable as having a fresh testing data set, but it works well in many cases.
Cross-validation can be equally well applied to least squares models. We will revisit this topic later.

4.12.5 Bootstrapping
Bootstrapping is an extremely useful tool when theoretical techniques to estimate confidence intervals
and uncertainty are not available to us.

Let’s give an example where bootstrapping is strictly not required, but is definitely useful. When
fitting a least squares model of the form 𝑦 = 𝛽0 + 𝛽1 𝑥 we are interested in the confidence interval of the
slope coefficient, 𝛽1 . Recall this coefficient indicates by how much the y-variable changes on average
when changing the x variable by one unit. The slope coefficient might represent a rate constant, or be
related to the magnitude of the feedback control loop gain. Whatever the case, it is important we
understand the degree of uncertainty associated with it, so we can make an appropriate judgement.

In the preceding section on least squares model analysis we derived this confidence interval (page 231) for
𝛽1 , repeated here:

𝑏1 − 𝛽1
−𝑐𝑡 ≤ ≤ +𝑐𝑡
𝑆𝐸 (𝑏1 )
𝑏1 − 𝑐𝑡 𝑆𝐸 (𝑏1 ) ≤ 𝛽1 ≤ 𝑏1 + 𝑐𝑡 𝑆𝐸 (𝑏1 )

Visualize this confidence in the context of the following example where x is the dose of radiation
administered (rads), and y is the survival percentage. The plot shows the data and the least square
slope coefficient (notice the y variable is a transformed variable, log(survival)).

The thick line represents the slope coefficient (−0.0059) using all the data. Clearly the unusual point
number 13 has some influence on that coefficient. Eliminating it and refitting the model makes the
slope coefficient more steep (−0.0078), which could change our interpretation of the model. This raises
the question though: what happens to the slope coefficient when we eliminate other points in the
training data? How sensitive are our model parameters to the data themselves?

268 Chapter 4. Least Squares Modelling Review

 Release 547b16

Bootstrapping gives us an indication of that sensitivity, as shown in the other plot. The original data
set had 14 observations. What bootstrapping does is to randomly select 14 rows from the original data,
allowing for duplicate selection. These selected rows are used to build a least squares model, and the
slope coefficient is recorded. Then another 14 random rows are selected and this process is repeated R
times (in this case R=1000). On some of these occasions the outlier points will be included, and other
times they will be excluded.

A histogram of the 1000 computed slope coefficients is shown here. This histogram gives us an
additional indication of the uncertainty of the slope coefficient. It shows many possible slope
coefficients that could have been obtained. One in particular has been marked, the slope when point
13 was omitted.

For completeness the confidence interval at the 95% level for 𝛽1 is calculated here, and also
superimposed on the histogram.

𝑏1 − 𝛽1
−𝑐𝑡 ≤ ≤ +𝑐𝑡
𝑆𝐸 (𝑏1 )
−0.005915 − 2.1788 × 0.001047 ≤ 𝛽1 ≤ −0.005915 + 2.1788 × 0.001047
−0.0082 ≤ 𝛽1 ≤ −0.0036

This confidence interval, together with the bootstrapped values of 𝑏1 give us additional insight when
when making our interpretation of 𝑏1 .

By now you should also be wondering whether you can bootstrap the confidence interval bounds!
That’s left as exercise for interested readers. The above example was inspired from an example in ASA
Statistics Computing and Graphics94 , 13 (1), 2002.

4.13 Exercises

ò Question

Use the distillation column data set95 and choose any two variables, one for x and one as y. Then fit
the following models by least squares in any software package you prefer:

• 𝑦𝑖 = 𝑏0 + 𝑏1 𝑥𝑖

94 [Link]

4.13. Exercises 269

Process Improvement Using Data

• 𝑦𝑖 = 𝑏0 + 𝑏1 (𝑥𝑖 − 𝑥) (what does the 𝑏0 coefficient represent in this case?)

• (𝑦𝑖 − 𝑦) = 𝑏0 + 𝑏1 (𝑥𝑖 − 𝑥)

Prove to yourself that centering the x and y variables gives the same model for the 3 cases in
terms of the 𝑏1 slope coefficient, standard errors and other model outputs.

ò Solution

Once you have created an x and y variable in R, compare the output from these 3 models:

# Model 1
summary(lm(y ~ x))

# Model 2
[Link] <- x - mean(x)
summary(lm(y ~ [Link]))

# Model 3
[Link] <- y - mean(y)
summary(lm([Link] ~ [Link]))

The same comparison in Python, using statsmodels, would be:

import pandas as pd
import [Link] as sm

distill = pd.read_csv(
"[Link]
)
x = distill["TempC2"].values
y = distill["VapourPressure"].values

# Model 1: y ~ x
print([Link](y, sm.add_constant(x)).fit().summary())

# Model 2: y ~ (x - mean(x))
x_mc = x - [Link]()
print([Link](y, sm.add_constant(x_mc)).fit().summary())

# Model 3: (y - mean(y)) ~ (x - mean(x))

y_mc = y - [Link]()
print([Link](y_mc, sm.add_constant(x_mc)).fit().summary())

ò Question

For a 𝑥new value and the linear model 𝑦 = 𝑏0 + 𝑏1 𝑥 the prediction interval for 𝑦^new is:

√︀
𝑦^𝑖 ± 𝑐𝑡 𝑉 {^
𝑦𝑖 }

where 𝑐𝑡 is the critical t-value, for example at the 95% confidence level.

Use the distillation column data set96 and with y as VapourPressure (units are kPa) and x as

95 [Link]

270 Chapter 4. Least Squares Modelling Review

 Release 547b16

TempC2 (units of degrees Farenheit) fit a linear model. Calculate the prediction interval for vapour
pressure at these 3 temperatures: 430, 480, 520 °F.

ò Solution

The prediction interval is dependent on the value of 𝑥new, i used to make the prediction. For this
model, 𝑆𝐸 = 2.989 kPa, 𝑛 = 253, 𝑗 (𝑥𝑗 − 𝑥)2 = 86999.6, and 𝑥 = 480.82.
∑︀

(︃ )︃
2
2 1 (𝑥new − 𝑥)
𝒱 (^
𝑦new,i ) = 𝑆𝐸 1 + + ∑︀ 2
𝑛 𝑗 (𝑥𝑗 − 𝑥)

Calculating this term manually, or using the predict(model, newdata=..., int="p")

function in R gives the 95% prediction interval:

• 𝑥new = 430 °F: 𝑦^new = 53.49 ± 11.97, or [47.50, 59.47]

• 𝑥new = 480 °F: 𝑦^new = 36.92 ± 11.80, or [31.02, 42.82]

• 𝑥new = 520 °F: 𝑦^new = 23.67 ± 11.90, or [17.72, 29.62]

96 [Link]

4.13. Exercises 271

Process Improvement Using Data

dist <- [Link]('[Link]

attach(dist)
model <- lm(VapourPressure ~ TempC2)
summary(model)

# From the above output

SE = sqrt(sum(resid(model)^2)/model$[Link])
n = length(TempC2)
k = model$rank
[Link] = [Link](TempC2 = c(430, 480, 520))
[Link] = mean(TempC2)
[Link] = sum(([Link])^2)
[Link] = SE^2 * (1 + 1/n + ([Link])^2/[Link])
c.t = -qt(0.025, df=n-k)
[Link] = predict(model, newdata=[Link], int="p")
[Link] = [Link][,1] - c.t*sqrt([Link])
[Link] = [Link][,1] + c.t*sqrt([Link])

# Results from [Link] agree with [Link] and [Link]

[Link]
# fit lwr upr
# 1 53.48817 47.50256 59.47379
# 2 36.92152 31.02247 42.82057
# 3 23.66819 17.71756 29.61883
[Link][,3] - [Link][,2]
plot(TempC2, VapourPressure, ylim = c(17, 65), main="Visualizing the prediction intervals")
abline(model, col="red")
library(gplots)
plotCI(x=c(430, 480, 520), y=[Link][,1], li=[Link][,2], ui=[Link][,3], add=TRUE, col="red")

ò Question

Refit the distillation model from the previous question with a transformed temperature variable.
Use 1/𝑇 instead of the actual temperature.

• Does the model fit improve?

• Are the residuals more normally distributed with the untransformed or transformed
temperature variable?

• How do you interpret the slope coefficient for the transformed temperature variable?

• Use the model to compute the predicted vapour pressure at a temperature of 480 °F, and also
calculate the corresponding prediction interval at that new temperature.

ò Solution

• Using the [Link] <- lm(VapourPressure ~ I(1/TempC2)) instruction, one obtains

the model summary below. The model fit has improved slightly: the standard error is 2.88 kPa,
reduced from 2.99 kPa.
Call:
lm(formula = VapourPressure ~ I(1/TempC2))

Residuals:
Min 1Q Median 3Q Max
-5.35815 -2.27855 -0.08518 1.95057 13.38436

272 Chapter 4. Least Squares Modelling Review

 Release 547b16

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -120.760 4.604 -26.23 <2e-16 ***
I(1/TempC2) 75571.306 2208.631 34.22 <2e-16 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 2.88 on 251 degrees of freedom

Multiple R-squared: 0.8235, Adjusted R-squared: 0.8228
F-statistic: 1171 on 1 and 251 DF, p-value: < 2.2e-16

• The residuals have roughly the same distribution as before, maybe a little more normal on the
left tail, but hardly noticeable.

• The slope coefficient of 75571 has units of kPa.°F, indicating that each one unit decrease in
temperature results in an increase in vapour pressure. Since division is not additive, the change
in vapour pressure when decreasing 10 degrees from 430 °F is a different decrease to that when
temperature is 530 °F. The interpretation of transformed variables in linear models is often a lot
harder. The easiest interpretation is to show a plot of 1/T against vapour pressure.

4.13. Exercises 273

Process Improvement Using Data

• The predicted vapour pressure at 480 °F is 36.68 kPa ±11.37, or within the range [31.0 to 42.4]
with 95% confidence, very similar to the prediction interval from question 2.

# Model with inverted temperature

[Link] <- lm(VapourPressure ~ I(1/TempC2))
summary([Link])

plot(1/TempC2, VapourPressure, xlab="1/TempC2 [1/degF]", ylab="Vapour pressure [kPa]")

abline([Link], col="red")
lines(lowess(1/TempC2, VapourPressure), lty=2, col="red")

[Link] = [Link](TempC2 = c(430, 480, 520))

[Link] = predict([Link], newdata=[Link], int="p")
[Link]
# fit lwr upr
# 1 54.98678 49.20604 60.76751
# 2 36.67978 30.99621 42.36334
# 3 24.56899 18.84305 30.29493

layout(matrix(c(1,2), 1, 2))
library(car)
qqPlot(model, main="Model with temperature", col=c(1, 1))
qqPlot([Link], main="Model with inverted temperature",col=c(1, 1))

ò Question

Again, for the distillation model, use the data from 2000 and 2001 to build the model (the first
column in the data set contains the dates). Then use the remaining data to test the model. Use x =
TempC2 and y = VapourPressure in your model.

• Calculate the RMSEP for the testing data. How does it compare to the standard error from the
model?

• Now use the influencePlot(...) function from the car library, to highlight the influential
observations in the model building data (2000 and 2001). Show your plot with observation labels
(observation numbers are OK). See part 5 of the R tutorial97 for some help.

274 Chapter 4. Least Squares Modelling Review

 Release 547b16

• Explain how the points you selected are influential on the model?

• Remove these influential points, and refit the model on the training data. How has the model’s
slope and standard error changed?

• Recalculate the RMSEP for the testing data; how has it changed?

ò Solution

• The testing data starts at index 160. The code at the end of this question shows how RMSEP was
calculated as 4.18 kPa, as compared to the standard error from the model building data
(observations 1 to 159) of 2.679 kPa. This indicates the predictions on totally new data have
greater error that those observations used to build the model - an expected result.

• The influence plot from the model building data is given below.

• The points considered as influential would be 38 and 84, which have both high leverage and
high discrepancy. Points 53 and 101 would also be considered influential: they have high
leverage, though moderately sized residuals. The other points marked in red have a large
Cook’s D value, however, their leverage is low, so it is unlikely that their removal will change
the plot and its interpretation by very much.

• The points selected for removal are [38, 53, 84, 101]. The model was rebuilt and the slope
coefficient changed from -0.368 to -0.358, while the standard error decreased from 2.679 to 2.455.
So their removal has decreased the size of the confidence intervals (before:
−0.395 ≤ 𝛽𝑇 ≤ −0.342, and after: −0.385 ≤ 𝛽𝑇 ≤ −0.332), however the slope coefficient is
roughly comparable to that from before.

• The RMSEP has reduced from 4.18kPa to 3.92 kPa, a smallish reduction, given the range of the y
variable.

97 [Link]

4.13. Exercises 275

Process Improvement Using Data

dist <- [Link]('[Link]

attach(dist)
model <- lm(VapourPressure ~ TempC2)
n = length(TempC2)
# Training and testing
# ---------------------

# Poor choice
build = seq(1,n,2); test = seq(2,n,2)

# Better choice
build = seq(1,159) # sample 159 is the last sample for 2001
test = seq(160,n) # first sample for 2002

[Link] <- lm(model, subset=build)

summary([Link])
confint([Link])
SE = sqrt(sum(resid([Link])^2)/[Link]$[Link])

[Link] = [Link](TempC2 = TempC2[test])

[Link] = predict([Link], newdata=[Link])
[Link] = VapourPressure[test]
errors <- ([Link] - [Link])
RMSEP <- sqrt(mean(errors^2))
c(RMSEP, SE)

# Find influential observations and remove them

#-----------------------------------------------
library(car)
influencePlot([Link], identify="auto")

remove = -c(38, 53, 84, 101)

[Link] <- lm(model, subset=build[remove])
summary([Link])
confint([Link])
SE = sqrt(sum(resid([Link])^2)/[Link]$[Link])
influencePlot([Link], identify="auto")

[Link] = predict([Link], newdata=[Link])

[Link] = VapourPressure[test]
errors <- ([Link] - [Link])
RMSEP <- sqrt(mean(errors^2))
c(RMSEP, SE)

ò Question

The Kappa number data set98 was used in an earlier question (page 194) to construct a Shewhart
chart. The “Mistakes to avoid” (page 161) section (Process Monitoring), warns that the subgroups for
a Shewhart chart must be independent to satisfy the assumptions used to derived the Shewhart
limits. If the subgroups are not independent, then it will increase the type I (false alarm) rate.

This is no different to the independence required for least squares models. Use the autocorrelation
tool to determine a subgroup size for the Kappa variable that will satisfy the Shewhart chart
assumptions. Show your autocorrelation plot and interpret it as well.

98 [Link]

276 Chapter 4. Least Squares Modelling Review

 Release 547b16

ò Solution

The autocorrelation plot shows significant lags up to lag 3, or even 4. So subsampling the vector
with every 4th or 5th element should yield independent samples. The autocorrelation with every
5th observation confirms this. You could also use every 6th, 7th, etc observation. Using every 30th
observation though is not too useful, since it would lead to a long delay before the control chart
showed any problems.

The ACF plot indicates that there is significant reappearance of correlation around lags 9 to 15. It
wasn’t required for you to identify why for this assignment, but usually this would be related to a
recycle stream that reenters a reactor, or due to an oscillation in a control loop.

You can also verify the autocorrelation by plotting scatterplots of the vector against itself. The first
plot below shows what an ACF coefficient of 1.0 means, while the second plot shows what it means
√
to use a lag offset of 1 position. The correlation value = 𝑅2 is shown on each plot. Compare that
value shown to the y-axis of the ACF plots.

4.13. Exercises 277

Process Improvement Using Data

kappa <- [Link]('[Link]

summary(kappa)
attach(kappa)
N = length(Kappa)

[Link] = 5
sub = seq(1, N, [Link])
[Link] <- Kappa[sub]

layout(matrix(seq(1, 2),2, 1))

acf(kappa, 50, main="Autocorrelation for the Kappa number variable", xlab="")
acf([Link], 50, main="Autocorrelation of subsampled Kappa number vector")

layout(matrix(seq(1, [Link]+1),1, [Link]+1))

for (jump in 0:[Link]){

plot(Kappa[seq(1,N-jump)], Kappa[seq(jump+1, N)],

xlab=paste("Kappa[seq(1, N-", substitute(jump, list(jump=jump)),")]", sep=""),
ylab=paste("Kappa[seq(", substitute(jump, list(jump=jump)), "+1,N)]", sep=""),
main=paste("Subsample every ", substitute(jump, list(jump=jump)), " element")
)
lines(lowess(Kappa[seq(1,N-jump)], Kappa[seq(jump+1, N)]), col="red")

correl = cor(Kappa[seq(jump+1, N)], Kappa[seq(1, N-jump)])

correl = round(correl, 2)
text(10, 35,
paste("Correlation = ", substitute(correl, list(correl=correl)), "", sep=""),
pos=4, cex=1.3)

ò Question

You presume the yield from your lab-scale bioreactor, 𝑦, is a function of reactor temperature, batch
duration, impeller speed and reactor type (one with with baffles and one without). You have
collected these data from various experiments.

Temp = 𝑇 [°C] Duration = Speed = 𝑠 [RPM] Baffles = 𝑏 [Yes/No] Yield = 𝑦 [g]

𝑑 [minutes]
82 260 4300 No 51
90 260 3700 Yes 30
88 260 4200 Yes 40
86 260 3300 Yes 28
80 260 4300 No 49
78 260 4300 Yes 49
82 260 3900 Yes 44
83 260 4300 No 59
64 260 4300 No 60
73 260 4400 No 59
60 260 4400 No 57
60 260 4400 No 62
101 260 4400 No 42
92 260 4900 Yes 38

278 Chapter 4. Least Squares Modelling Review

 Release 547b16

• Use software to fit a linear model that predicts the yield from these variables (the data set is
available from the website99 ). See the R tutorial100 for building linear models with integer
variables in R.

• Interpret the meaning of each effect in the model. If you are using R, then the confint(...)
function will be helpful as well. Show plots of each x variable in the model against yield. Use a
box plot for the baffles indicator variable.

• Now calculate the X𝑇 X and X𝑇 y matrices; include a column in the X matrix for the intercept.
Since you haven’t mean centered the data to create these matrices, it would be misleading to try
interpret them.

• Calculate the least squares model estimates from these two matrices. See the R tutorial101 for
doing matrix operations in R, but you might prefer to use MATLAB for this step. Either way,
you should get the same answer here as in the first part of this question.

ò Solution

• After importing the data, just make sure the baffles variable is imported as a factor. Then
build the model as usual. The computer output below shows the linear model’s coefficients.

bio <- [Link]('[Link]

attach(bio)
summary(bio)
[Link](baffles)
# [1] TRUE

model <- lm(yield ~ speed + baffles + temperature )

summary(model)

# Call:
# lm(formula = yield ~ speed + baffles + temperature)
#
# Residuals:
# Min 1Q Median 3Q Max
# -5.5521 -3.2543 -0.4356 2.2953 8.1519
#
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 52.483652 18.421511 2.849 0.01728 *
# speed 0.008711 0.003757 2.319 0.04288 *
# bafflesYes -9.090700 3.048811 -2.982 0.01377 *
# temperature -0.470997 0.119242 -3.950 0.00273 **
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#
# Residual standard error: 4.651 on 10 degrees of freedom
# Multiple R-squared: 0.8659, Adjusted R-squared: 0.8256
# F-statistic: 21.52 on 3 and 10 DF, p-value: 0.0001108

• The confidence intervals for each variable is significant at the 95% level. The duration variable
must be omitted from the model, because it has no variation. While it might affect the yield,
there is no variability in this data set to assess that.

– 0.00034 ≤ 𝑏speed ≤ 0.017: a 100rpm increase in impeller speed serves to increase yield by 0.87g

99 [Link]
100 [Link]
101 [Link]

4.13. Exercises 279

Process Improvement Using Data

on average, keeping all other variables constant

– −15.9 ≤ 𝑏baffles ≤ −2.30: the use of baffles decreases yield, on average, by 9.1g, keeping all
other variables constant

– −0.74 ≤ 𝑏temp ≤ −0.21: each one degree increase in temperature lowers yield by 0.47g on
average, keeping all other variables constant

– We cannot say anything about the effect of batch duration

The plots are not shown here, they can be drawn with plot(bio) to obtain a scatterplot matrix
of plots.

• For the model 𝑦 = 𝑏0 + 𝑏speed 𝑥speed + 𝑏baffles 𝑥baffles + 𝑏temp 𝑥temp let the coefficient vector be
b = [𝑏0 , 𝑏speed , 𝑏baffles , 𝑏temp ], then we can write down the following X matrix to estimate it:

⎡ ⎤
1 4300 0 82
⎢1 3700 1 90 ⎥
⎢ ⎥
⎢1 4200 1 88 ⎥
⎢ ⎥
⎢ ⎥
⎢1 3300 1 86 ⎥
⎢ ⎥
⎢1 4300 0 80 ⎥
⎢ ⎥
⎢1 4300 1 78 ⎥
⎢ ⎥
⎢ ⎥
⎢1 3900 1 82 ⎥
X=⎢
⎢1
⎥
⎢ 4300 0 83 ⎥
⎥
⎢1 4300 0 64 ⎥
⎢ ⎥
⎢ ⎥
⎢1 4400 0 73 ⎥
⎢ ⎥
⎢1 4400 0 60 ⎥
⎢ ⎥
⎢1 4400 0 60 ⎥
⎢ ⎥
⎢ ⎥
⎣1 4400 0 101⎦
1 4900 1 92

You can obtain the above X matrix in R using the [Link](model) function. The X𝑇 X
and X𝑇 y matrices are:
⎡ ⎤ ⎡ ⎤
14 59100 6 1119 668
⎢59100 251330000 24300 4714700⎥ ⎢2849600⎥
X𝑇 X = ⎢ and X𝑇 y = ⎢
⎢ ⎥ ⎢ ⎥
⎥ ⎥
⎣ 6 24300 6 516 ⎦ ⎣ 229 ⎦
1119 4714700 516 91351 52082

• Using these matrices to solve for b

⎡
⎤
52.48
)︀−1 𝑇 ⎢0.00871⎥
b = X𝑇 X
(︀
X y=⎢
⎢ ⎥
⎣ −9.09 ⎦
⎥

−0.471

This result matches the results from R. Note however that R, like most decent software packages,
(︀ )︀−1
will not solve for the inverse of X𝑇 X directly to compute b; instead it uses the QR
decomposition102 .

280 Chapter 4. Least Squares Modelling Review

 Release 547b16

# Calculate confidence intervals

confint(model)
# 2.5 % 97.5 %
# (Intercept) 1.143797e+01 93.52933596
# speed 3.396812e-04 0.01708227
# bafflesYes -1.588388e+01 -2.29752465
# temperature -7.366849e-01 -0.20530879

# Show a scatterplot matrix

plot(bio)

# Computing the model's coefficients using inv(X' * X) * X' * y

X <- [Link](model)
XtX <- t(X) %*% X
Xty <- t(X) %*% yield
b = solve(XtX) %*% Xty
# [,1]
# (Intercept) 52.483652163
# speed 0.008710973
# bafflesYes -9.090699955
# temperature -0.470996834

ò Question

In the section on comparing differences between two groups we used, without proof, the fact that:

𝒱 {𝑥𝐵 − 𝑥𝐴 } = 𝒱 {𝑥𝐵 } + 𝒱 {𝑥𝐴 }

Prove this statement, and clearly explain all steps in your proof.

ò Solution

I don’t normally concentrate on proofs in the book, unless they show something interesting, or are
used over and over. This short mathematical statement fits both criteria.

The important point with this proof is that 𝑥𝐴 and 𝑥𝐵 are the variables, not 𝑥. These variables come
from a normal distribution (Central limit theorem), as long as we assume independent sampling:
(︀ )︀
𝑥𝐴 ∼ 𝒩 𝜇; 𝜎 2 /𝑛𝐴 , and similarly for 𝑥𝐵 .

𝒱 {𝑥𝐵 − 𝑥𝐴 } = 𝒱 {𝑥𝐵 + (−𝑥𝐴 )}

= 𝒱 {𝑥𝐵 } + 2Cov {𝑥𝐵 , (−𝑥𝐴 )} + 𝒱 {−𝑥𝐴 }
2
= 𝒱 {𝑥𝐵 } + 0 + (−1) 𝒱 {𝑥𝐴 }
= 𝒱 {𝑥𝐵 } + 𝒱 {𝑥𝐴 }

The second line is a result shown earlier. The third line requires that we assume the between-group
means 𝑥𝐵 and 𝑥𝐴 are independent, and so they are uncorrelated (their covariance is zero). This was
one of the key assumptions when we studied between-group differences; and is one assumption
that is often true in many real cases.

102 [Link]

4.13. Exercises 281

Process Improvement Using Data

ò Question

The production of low density polyethylene is carried out in long, thin pipes at high temperature
and pressure (1.5 kilometres long, 50mm in diameter, 500 K, 2500 atmospheres). One quality
measurement of the LDPE is its melt index. Laboratory measurements of the melt index can take
between 2 to 4 hours. Being able to predict this melt index, in real time, allows for faster
adjustment to process upsets, reducing the product’s variability. There are many variables that are
predictive of the melt index, but in this example we only use a temperature measurement that is
measured along the reactor’s length.

These are the data of temperature (K) and melt index (units of melt index are “grams per 10
minutes”).

Temperature = 𝑇 441 453 461 470 478 481 483 485 499 500 506 516
[Kelvin]
Melt index = 𝑚 [g per 9.3 6.6 6.6 7.0 6.1 3.5 2.2 3.6 2.9 3.6 4.2 3.5
10 mins]

The following calculations have already been performed:

• Number of samples, 𝑛 = 12

• Average temperature = 𝑇 = 481 K

• Average melt index, 𝑚 = 4.925 g per 10 minutes.

∑︀ (︀ )︀
• The summed product, 𝑖 𝑇𝑖 − 𝑇 (𝑚𝑖 − 𝑚) = −422.1
∑︀ (︀ )︀2
• The sum of squares, 𝑖 𝑇𝑖 − 𝑇 = 5469.0

1. Use this information to build a predictive linear model for melt index from the reactor
temperature.

2. What is the model’s standard error and how do you interpret it in the context of this model? You
might find the following software software output helpful, but it is not required to answer the
question.

Call:
lm(formula = [Link] ~ Temperature)

Residuals:
Min 1Q Median 3Q Max
-2.5771 -0.7372 0.1300 1.2035 1.2811

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -------- 8.60936 4.885 0.000637
Temperature -------- 0.01788 -4.317 0.001519

Residual standard error: 1.322 on 10 degrees of freedom

Multiple R-squared: 0.6508, Adjusted R-squared: 0.6159
F-statistic: 18.64 on 1 and 10 DF, p-value: 0.001519

3. Quote a confidence interval for the slope coefficient in the model and describe what it means.
Again, you may use the above software output to help answer your question.

282 Chapter 4. Least Squares Modelling Review

 Release 547b16

ò Solution

1. The simplest linear predictive model possible is 𝑚 = 𝛽0 + 𝛽1 𝑇 + 𝜀, predicting the melt index
from temperature. Once we find estimates for these coefficients we write: 𝑚 = 𝑏0 + 𝑏1 𝑇 + 𝑒. And
one way to calculate these coefficients is by least squares. In the class notes we showed that for a
variable 𝑥 used to predict a variable 𝑦 that:

𝑏0 = y − 𝑏1 x
∑︀
(𝑥𝑖 − x) (𝑦𝑖 − y)
𝑏1 = 𝑖 ∑︀ 2
𝑖 (𝑥𝑖 − x)

Using the pre-calculated values, and that in our case 𝑇 = 𝑥, and that 𝑚 = 𝑦

−422.1 g per 10 minutes

𝑏1 = = −0.0772
5469.0 𝐾
𝑏0 = 4.925 + 0.0772 × 481 = 42.0g per 10 minutes

^ = 42.0 − 0.0772 × 𝑇
A predictive model of melt flow is: 𝑚

1. The standard error, 𝑆𝐸 can be read directly from the software output as 1.322 g per 10 minutes.
If you like, you could also have calculated it by hand, using the above √︂predictive model,
∑︀𝑛 2
𝑖 𝑒𝑖
calculating residuals (𝑒𝑖 = 𝑚𝑖 − 𝑚^ 𝑖 ), from which the standard error is , where 𝑛 = 12
𝑛−𝑘
and 𝑘 = 2 (there are 2 parameters in the model). However I recommend you always use the
software output and avoid these tedious hand calculations.

The interpretation of the standard error for this model is that the approximate prediction error of
melt index has a standard deviation of 1.322 grams per 10 minutes (if the residuals are normally
distributed).

1. The slope coefficient estimate, 𝑏1 has standard error of 0.01788 (from the software output), or it
𝑆2 1.3222
could be calculated as 𝑆𝐸2
(𝑏1 ) = ∑︀ (︀ 𝐸 )︀2 = = 0.017882 = 3.19 × 10−4 .
𝑇𝑗 − 𝑇 5469.0
𝑗

From this we can construct the confidence interval for the actual slope coefficient, 𝛽1 . I have used
the 95% confidence level, but you could use any level you prefer. The degrees of freedom to use for
the 𝑡-distribution are 𝑛 − 𝑘 = 12 − 2 = 10.
𝑏1 − 𝛽 1
−𝑐𝑡 ≤ ≤ +𝑐𝑡
𝑆𝐸 (𝑏1 )
𝑏1 − 𝑐𝑡 𝑆𝐸 (𝑏1 ) ≤ 𝛽1 ≤ 𝑏1 + 𝑐𝑡 𝑆𝐸 (𝑏1 )
−0.0772 − 2.23 × 0.01788 ≤ 𝛽1 ≤ −0.0772 + 2.23 × 0.01788
−0.117 ≤ 𝛽1 ≤ −0.037

You may also have chosen to answer at the 99% confidence level:

𝑏1 − 𝑐𝑡 𝑆𝐸 (𝑏1 ) ≤ 𝛽1 ≤ 𝑏1 + 𝑐𝑡 𝑆𝐸 (𝑏1 )
−0.0772 − 3.17 × 0.01788 ≤ 𝛽1 ≤ −0.0772 + 3.17 × 0.01788
−0.134 ≤ 𝛽1 ≤ −0.0205

This shows, at which ever confidence level (95% or 99%), the range within which we can expect to
find the true slope coefficient. This slope represents the magnitude by which the melt index
changes, on average, for a one degree change in temperature. If we plan to manipulate the melt
index using temperature, then this range will help us estimate an upper and lower bound for the
effort required to adjust the melt index.

4.13. Exercises 283

Process Improvement Using Data

ò Question

For a distillation column, it is well known that the column temperature directly influences the
purity of the product, and this is used in fact for feedback control, to achieve the desired product
purity. Use the distillation data set103 , and build a least squares model that predicts
VapourPressure from the temperature measurement, TempC2. Report the following values:

1. the slope coefficient, and describe what it means in terms of your objective to control the process
with a feedback loop

2. the interquartile range and median of the model’s residuals

3. the model’s standard error

4. a confidence interval for the slope coefficient, and its interpretation.

You may use any computer package to build the model and read these values off the computer
output.

ò Solution

The solution to this question can be almost entirely solved using R, though any other language
could be used. These commands, with the output that follows, were used:

> distillation <- [Link]('[Link]

> model <- lm(distillation$VapourPressure ~ distillation$TempC2)
> summary(model)

Call:
lm(formula = distillation$VapourPressure ~ distillation$TempC2)

Residuals:
Min 1Q Median 3Q Max
-5.59621 -2.37597 0.06674 2.00212 14.18660

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 195.96141 4.87669 40.18 <2e-16 ***
distillation$TempC2 -0.33133 0.01013 -32.69 <2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 2.989 on 251 degrees of freedom

Multiple R-squared: 0.8098, Adjusted R-squared: 0.8091
F-statistic: 1069 on 1 and 251 DF, p-value: < 2.2e-16

1. This predictive model allows us to achieve better control of the vapour pressure, because we can
predict it from temperature (measured in real-time), rather than wait several hours for the
laboratory vapour pressure value. The slope coefficient is -0.331, and since no units were given, I
can’t expect any in your solution; however one should report the units, which is this case would
be units of pressure divided by units temperature (e.g. psi/K). What this means, in terms of
feedback control of the vapour pressure is that we must decrease the temperature to raise the
vapour pressure. This is important when tuning the feedback control loop in 2 ways: (a) firstly,
the the sign of the gain in the feedback controller (i.e. negative gain) must be the same as the
process gain to achieve a stable feedback loop, (b) the magnitude of the slope provides an

103 [Link]

284 Chapter 4. Least Squares Modelling Review

 Release 547b16

estimate of how sensitive the vapour pressure is to temperature. For example: do we have to
add a large amount of energy into the distillation column to achieve a smallish reduction in
vapour pressure? The answer depends heavily on the units, which I omitted to provide.

2. These are reported in the above software output: (a) the residual IQR is 2.00 - (-2.38) = 4.38 units
of vapour pressure, while (b) the median residual is close to zero, as expected.

3. The model’s standard error is 2.989 in the output, or around 3.00 units of vapour pressure.
𝑏1 − 𝛽 1
4. The slope coefficient’s confidence interval can be calculated from its 𝑧-value = ; but we
𝑆𝐸 (𝑏1 )
require the standard error of the slope coefficient, which is 𝑆𝐸 (𝑏1 ) = 0.01013 from the software
output. The value for 𝑐𝑡 = 1.969 from the 𝑡-distribution at the 95% confidence level, with
𝑛 − 𝑘 = 253 − 2 = 251 degrees of freedom (a normal distribution would work equally well in
this case).

𝑏1 − 𝛽 1
−𝑐𝑡 ≤ ≤ +𝑐𝑡
𝑆𝐸 (𝑏1 )
−0.33133 − 1.969 × 0.01013 ≤ 𝛽1 ≤ −0.33133 + 1.969 × 0.01013
−0.35 ≤ 𝛽1 ≤ −0.31

This shows, at the 95% confidence level, the range within which we can expect to find the true
slope coefficient. This range is remarkably narrow; i.e. our feedback controller gain is unlikely to
change on either extreme. So we can likely design our control loop at the center point, and be
sure it will work over the entire range of expected operation. Please also cross reference the
solutions to question 2.4 in the written midterm to correctly understand what a confidence
interval is.

If you used 99% confidence levels, the answer should be: −0.358 ≤ 𝛽1 ≤ −0.305.

We have illustrated the actual slope (thick, solid line) at the upper and lower bounds of the slope
coefficient (thin, dashed lines) in the accompanying figure. Not required for this question, but
added nevertheless, are the prediction intervals for 𝑦^𝑖 .

4.13. Exercises 285

Process Improvement Using Data

I recommended that you reproduce R’s output yourself. The code below calculates these same
values.
# Calcuations with R

distillation <- [Link]('[Link]

model <- lm(distillation$VapourPressure ~ distillation$TempC2)
summary(model)

# Calculations by hand
# -----------------------

# Confidence level
alpha = 0.99

# Raw data
x <- distillation$TempC2
y <- distillation$VapourPressure
n = length(x)

# Some intermediate values

[Link] = mean(x)
[Link] = mean(y)
num <- sum((x - [Link]) * (y - [Link]))
den <- sum((x - [Link]) * (x - [Link]))

# Model coefficients
b1 <- num/den
b0 <- [Link] - b1 * [Link]
c(b0, b1)

# Model predictions and residuals, with their summary (IQR and median)
predictions <- b0 + x*b1
residuals <- y - predictions
summary(residuals)

# Calculate the 3 standard errors

SE <- sqrt(sum(residuals^2) / (n-2))
SE.b1 <- sqrt(SE^2 / den)
SE.b0 <- sqrt(SE^2 *(1/n + ([Link]^2)/den))
c(SE, SE.b0, SE.b1)

# Confidence intervals for the least squares parameters

z.b0 = b0/SE.b0
z.b1 = b1/SE.b1
c(z.b0, z.b1)
[Link] = qt(1-(1-alpha)/2, df=(n-2))
[Link]
[Link] <- b0 - [Link]*SE.b0
[Link] <- b0 + [Link]*SE.b0
[Link] <- b1 - [Link]*SE.b1
[Link] <- b1 + [Link]*SE.b1
c([Link], [Link])
c([Link], [Link])

# R2, TSS, RegSS, RSS, Adjusted R2

TSS <- sum((y - [Link])^2)
RegSS <- sum(([Link])^2)
RSS <- sum(residuals^2)
R2 <- RegSS/TSS
[Link] <- 1- (RSS/(n-2)) / (TSS/(n-1))
c(TSS, RegSS, RSS, R2, [Link])

# Error bounds for y-hat

[Link] = seq(min(x), max(x), diff(range(x))/100)
[Link] = b0 + [Link]*b1
[Link] <- [Link]*SE*sqrt(1+ 1/n + (([Link])^2)/den)

286 Chapter 4. Least Squares Modelling Review

 Release 547b16

# Plot of the raw data, least squares line, prediction interval for yhat,
# slope coefficient confidence interval range
plot(x,y, [Link]=1.5, [Link]=1.8, [Link]=1.8, [Link]=1.8, main="",
xlab="Temperature (TempC2)", ylab="VapourPressure")
grid(lwd=2)
points(x, y)
lines([Link], [Link] + [Link], col="gray40", lty=3)
lines([Link], [Link] - [Link], col="gray40", lty=3)

abline(a=b0, b=b1, col="red", lty=1, lwd=3)

# One extreme of the beta_1 slope CI

abline(a=([Link]*[Link]), b=[Link], col="red", lty=2, lwd=1)
# Other extreme of the beta_1 slope CI
abline(a=([Link]*[Link]), b=[Link], col="red", lty=2, lwd=1)

ò Question

Use the bioreactor data104 , which shows the percentage yield from the reactor when running
various experiments where temperature was varied, impeller speed and the presence/absence of
baffles were adjusted.

1. Build a linear model that uses the reactor temperature to predict the yield. Interpret the slope
and intercept term.

2. Build a linear model that uses the impeller speed to predict yield. Interpret the slope and
intercept term.

3. Build a linear model that uses the presence (represent it as 1) or absence (represent it as 0) of
baffles to predict yield. Interpret the slope and intercept term.

Note: if you use R it will automatically convert the baffles variable to 1’s and 0’s for you. If
you wanted to make the conversion yourself, to verify what R does behind the scenes, try this:

# Read in the data frame

bio <- [Link]('[Link]

# Force the baffles variables to 0's and 1's

bio$baffles <- [Link](bio$baffles) - 1

4. Which variable(s) would you change to boost the batch yield, at the lowest cost of
implementation?

5. Use the plot(bio) function in R, where bio is the data frame you loaded using the
[Link](...) function. R notices that bio is not a single variable, but a group of variables,
i.e. a data frame, so it plots what is called a scatterplot matrix instead. Describe how the
scatterplot matrix agrees with your interpretation of the slopes in parts 1, 2 and 3 of this
question.

ò Solution

The R code (below) was used to answer all questions.

104 [Link]

4.13. Exercises 287

Process Improvement Using Data

1. • The model is: 𝑦^ = 102.5 − 0.69𝑇 , where 𝑇 is tank temperature.

• Intercept = 102.5 % points is the yield when operating at 0 ∘ C. Obviously not a useful
interpretation, because data have not been collected in a range that spans, or is even close to 0
∘
C. It is likely that this bioreactor system won’t yield any product under such cold conditions.
Further, a yield greater than 100% is not realizable.

• Slope = -0.69 [[%]

∘ C] , indicating the yield decreases, on average, by about 0.7 units for every

degree increase in tank temperature.

2. • The model is: 𝑦^ = −20.3 + 0.016𝑆, where 𝑆 is impeller speed.

• Intercept = −20.3 % points is the yield when operating no agitation. Again, obviously not a
useful interpretation, because the data have not been collected under these conditions, and
yield can’t be a negative quantity.
[%]
• Slope = 0.016 [RPM] , indicating the yield increases, on average, by about 1.6 percentage points
per 100 RPM increase.

3. • The model is: 𝑦^ = 54.9 − 16.7𝐵, where 𝐵 is 1 if baffles are present and 𝐵 = 0 with no baffles.

• Intercept = 54.9 % points yield is the yield when operating with no baffles (it is in fact the
average yield of all the rows that have “No” as their baffle value).

• Slope = -16.7 %, indicating the presence of baffles decreases the yield, on average, by about
16.7 percentage points.

4. This is an open-ended, and case specific. Some factors you would include are:

• Remove the baffles, but take into account the cost of doing so. Perhaps it takes a long time
(expense) to remove them, especially if the reactor is used to produce other products that do
require the baffles.

• Operate at lower temperatures. The energy costs of cooling the reactor would factor into this.

• Operate at higher speeds and take that cost into account. Notice however there is one
observation at 4900 RPM that seems unusual: was that due to the presence of baffles, or due
to temperature in that run? We’ll look into this issue with multiple linear regression later on.

ò Note

Please note that our calculations above are not the true effect of each of the variables (tempera-
ture, speed and baffles) on yield. Our calculations assume that there is no interaction between
temperature, speed and baffles, and that each effect operates independent of the others. That’s
not necessarily true. See the section on interpreting MLR coefficients (page 256) to learn how to
“control for the effects” of other variables.

5. The scatterplot matrix, shown below, agrees with our interpretation. This is an information rich
visualization that gives us a feel for the multivariate relationships and really summarizes all the
variables well (especially the last row of plots).

• The yield-temperature relationship is negative, as expected.

• The yield-speed relationship is positive, as expected.

• The yield-baffles relationship is negative, as expected.

288 Chapter 4. Least Squares Modelling Review

 Release 547b16

• We can’t tell anything about the yield-duration relationship, as it doesn’t vary in the data we
have (there could/should be a relationship, but we can’t tell).

bio <- [Link]('[Link]

summary(bio)

# Temperature-Yield model
[Link] <- lm(bio$yield ~ bio$temperature)
summary([Link])

# Impeller speed-Yield model

[Link] <- lm(bio$yield ~ bio$speed)
summary([Link])

# Baffles-Yield model
[Link] <- lm(bio$yield ~ bio$baffles)
summary([Link])

# Scatterplot matrix
bitmap('[Link]', type="png256",
width=10, height=10, res=300)
plot(bio)
[Link]()

4.13. Exercises 289

Process Improvement Using Data

ò Question

Use the gas furnace data105 from the website to answer these questions. The data represent the gas
flow rate (centered) from a process and the corresponding CO2 measurement.

1. Make a scatter plot of the data to visualize the relationship between the variables. How would
you characterize the relationship?

2. Calculate the variance for both variables, the covariance between the two variables, and the
correlation between them, 𝑟(𝑥, 𝑦). Interpret the correlation value; i.e. do you consider this a
strong correlation?

3. Now calculate a least squares model relating the gas flow rate as the 𝑥 variable to the CO2
measurement as the 𝑦-variable. Report the intercept and slope from this model.

4. Report the 𝑅2 from the regression model. Compare the squared value of 𝑟(𝑥, 𝑦) to 𝑅2 . What do
you notice? Now reinterpret what the correlation value means (i.e. compare this interpretation
to your answer in part 2).

5. Advanced: Switch 𝑥 and 𝑦 around and rebuild your least squares model. Compare the new 𝑅2
to the previous model’s 𝑅2 . Is this result surprising? How do interpret this?

ò Solution

1. Relationship: the data are negatively correlated.

105 [Link]

290 Chapter 4. Least Squares Modelling Review

 Release 547b16

I’ve chosen to use the sp or scatterplot function from the car library. It shows the
scatterplot smoother (a.k.a. loess line) as solid red, the spread around the smoother (dashed
red), the least squares regression line (black) and boxplots for each axis.

This is a great example of an information-rich visualization: packing the maximum amount of

information into a small space. This plot answers so many questions we might have about the
data.

2. The cov(...) command supplies the variance and covariance, and the cor(...) command
gives the correlation.

• Variance of input gas flow rate = 1.15 [gas flow units] 2

• Variance of CO2 = 10.3 [CO2 units] 2

• Covariance between input gas flow and CO2 = -1.66 [gas flow units][CO2 units]

• Correlation = -0.48, i.e. around -0.5.

From my experience with data, I personally would interpret this as a reasonably strong
correlation. There is reasonably strong linear behaviour in the data cloud shown above, enough
of a relationship to confidently say that “the CO2 output does decrease at higher gas flow rates”.

3. From the R model output:

4.13. Exercises 291

Process Improvement Using Data

• intercept is -1.44 units of CO2

[units of CO2 ]
• slope is 53.4 [units of gas flow]

4. • From the R model output: 𝑅2 = 0.2347

• From earlier, the squared correlation is (−0.484)2 = 0.2347, the same value.

• Correlation can be interpreted as the square root of the 𝑅2 value when regressing 𝑦 on 𝑥 (i.e.
fitting a linear model to 𝑦 using 𝑥 as the input).

• Most novices would be misled and consider an 𝑅2 value of 0.23 quite low. But notice that
there is a repeatable and consistent negative linear relationship between 𝑥 and 𝑦 in this data.

5. This shows the interesting result that when regressing 𝑥 on 𝑦 (instead of the usual regression of 𝑦
on 𝑥), that we get the same 𝑅2 value. Note however that the intercept and slope are different
between the two regressions.

This also calls into question the interpretation of the 𝑅2 value in regression. 𝑅2 is just the square
of the correlation coefficient. Recall from class the slide on the Wikipedia examples of
√
correlation106 : there were examples where 𝑟(𝑥, 𝑦) = 𝑅2 was zero, but still a strong relationship
existing in the data. So we should interpret 𝑅2 as a measure only of the linear relationship
between two variables. And bear its quadratic nature in mind - interpreting the correlation is
actually easier, and more “linear”, in that a 0.2 improvement in correlation means the same thing
when going from 𝑟 = 0.2 to 0.4, as it does when going from 𝑟 = 0.7 to 0.9 (not so for 𝑅2 ).

gas <- [Link]('[Link]

summary(gas)

library(car)
bitmap('[Link]', type="png256",
width=6, height=6, res=300, pointsize=14)

# Use the "sp" (scatterplot) function from the "car" library

sp(gas$InputGasRate, gas$CO2, xlab="Gas flow rate", ylab="CO2",
main="Scatterplot with smoother, spread, and L/S line")
[Link]()

# (Co)variance and correlation

cov(gas)
cor(gas)

# Linear model:
model <- lm(gas$CO2 ~ gas$InputGasRate)
summary(model)

# ANOVA values
[Link] <- mean(gas$CO2)
RegSS <- sum((predict(model) - [Link])^2)
RSS <- sum(residuals(model)^2)
TSS <- sum((gas$CO2 - [Link])^2)
[Link] <- RSS / model$[Link]

# Test normality of residuals

bitmap('[Link]', type="png256",
width=6, height=6, res=300, pointsize=14)
par(mar=c(4.2, 4.2, 0.5, 0.5))
qqPlot(model) # the qqPlot "knows" what to do with a model object
[Link]()

106 [Link]

292 Chapter 4. Least Squares Modelling Review

 Release 547b16

ò Question

A new type of thermocouple107 is being investigated by your company’s process control group.
These devices produce an almost linear voltage (millivolt) response at different temperatures. In
practice though it is used the other way around: use the millivolt reading to predict the
temperature. The process of fitting this linear model is called calibration.

1. Use the following data to calibrate a linear model:

Temperature [K] 273 293 313 333 353 373 393 413 433 453
Reading [mV] 0.01 0.12 0.24 0.38 0.51 0.67 0.84 1.01 1.15 1.31

Show the linear model and provide the predicted temperature when reading 1.00 mV.

2. Are you satisfied with this model, based on the coefficient of determination (𝑅2 ) value?

3. What is the model’s standard error? Now, are you satisfied with the model’s prediction ability,
given that temperatures can usually be recorded to an accuracy of ±0.5 K with most inexpensive
thermocouples.

4. What is your (revised) conclusion now about the usefulness of the 𝑅2 value?

Note: This example explains why we don’t use the terminology of independent and dependent
variables in this book. Here the temperature truly is the independent variable, because it causes the
voltage difference that we measure. But the voltage reading is the independent variable in the least
squares model. The word independent is being used in two different senses (its English meaning vs
its mathematical meaning), and this can be misleading.

ò Solution

1. The linear model is used to predict temperature given the reading in millivolts. The reason is
that in modelling, in general, we specify as 𝑥 the variable(s) we always have available, while 𝑦 is
the variable we would like to predict from the 𝑥.

The model has the form: 𝑇 = 𝑏0 + 𝑏1 𝑉 , where 𝑇 is temperature and 𝑉 is the voltage reading.
Coefficients in the linear model are:

𝑇 = 278.6 + 135.3𝑉

implies that recording an increase in 0.1 mV means, on average, the temperature has increased
by 13.5 K in the system.

The temperature prediction at 1.00 mV would be 413.9 K.

107 [Link]

4.13. Exercises 293

Process Improvement Using Data

The following Python code was used to fit the model and draw the plot.

import numpy as np
from [Link] import *

x = [Link]([0.01, 0.12, 0.24, 0.38, 0.51, 0.67, 0.84, 1.01, 1.15, 1.31])
y = [Link]([273, 293, 313, 333, 353, 373, 393, 413, 433, 453])
n = [Link]([Link]) # the number of observations
X = [Link]([[Link](n), x]).T

# Simpler, and more accurate way:

a = [Link](X, y)[0]

# Additional calculations
resids = y - [Link](X,a) # e = y - Xa;
RSS = sum(resids**2) # residual sum of squares
TSS = sum((y - [Link](y))**2) # total sum of squares
R2 = 1 - RSS/TSS
std_error = [Link](RSS/(n-len(a)))

# Plot the data along with the fitted line:

fig = figure()
plot(x, y, 'o', label='Original data', markersize=10)
plot(x, [Link](X,a), 'r', label='Fitted line')
grid('on')
xlabel('Voltage [mV]')
ylabel('Temperature [K]')
legend(loc=0)

text(0.8, 325, 'Standard error = %0.1f K' % std_error)

plot(x, [Link](X,a)+2*std_error, 'r--')
plot(x, [Link](X,a)-2*std_error, 'r--')
[Link]('[Link]')

If you used R to fit the model, you would written something like this:

294 Chapter 4. Least Squares Modelling Review

 Release 547b16

> V <- c(0.01, 0.12, 0.24, 0.38, 0.51, 0.67, 0.84, 1.01, 1.15, 1.31)
> T <- c(273, 293, 313, 333, 353, 373, 393, 413, 433, 453)
> model <- lm(T ~ V)
> summary(model)

Call:
lm(formula = T ~ V)

Residuals:
Min 1Q Median 3Q Max
-6.9272 -2.1212 -0.1954 2.7480 5.4239

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 278.574 2.204 126.39 1.72e-14 ***
V 135.298 2.922 46.30 5.23e-11 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 3.916 on 8 degrees of freedom

Multiple R-squared: 0.9963, Adjusted R-squared: 0.9958
F-statistic: 2144 on 1 and 8 DF, p-value: 5.229e-11

2. The 𝑅2 value from this linear fit is 𝑅2 = 0.996, which being so close to 1.0, implies the linear
relationship in the data is strong (the linear model fits the data very well) - that’s all.

One cannot be satisfied with only an 𝑅2 value: it has nothing to do with whether the model’s
prediction accuracy is any good. So we can’t tell anything from this number.

3. The model’s standard error is 3.9 K. If we assume the prediction error is normally distributed
around the linear fit, this corresponds to one standard deviation. So 95% of our prediction error
lies roughly within a range of ±2 × 3.92 or ±7.8 K. These are the dashed red lines drawn on the
figure. (Please note: the true error intervals are not parallel to the regression line, they are
curved; however the ±2𝑆𝐸 limits are a good-enough approximation for most engineering
applications.

This prediction ability of ±8 K is probably not satisfying for most engineering applications, since
we can predict temperatures far more accurately, over the range from 273K to 453K, using
off-the-shelf commercial thermocouples.

4. The purpose of this question is to mainly point out the misleading nature of 𝑅2 - this value looks
really good: 99.6%, yet the actual purpose of the model, the ability to predict temperature from
the millivolt reading, has no relationship at all to this 𝑅2 value.

ò Question

1. Use the linear model you derived in the gas furnace question (page 290), where you used the gas
flow rate to predict the CO2 measurement, and construct the analysis of variance table (ANOVA)
for the dataset. Use your ANOVA table to reproduce the residual standard error, 𝑆𝐸 value, that
you get from the R software output.

Go through the R tutorial108 to learn how to efficiently obtain the residuals and predicted values
from a linear model object.

2. Also for the above linear model, verify whether the residuals are normally distributed.

4.13. Exercises 295

Process Improvement Using Data

3. Use the linear model you derived in the thermocouple question (page 293), where you used the
voltage measurement to predict the temperature, and construct the analysis of variance table
(ANOVA) for that dataset. Use your ANOVA table to reproduce the residual standard error, 𝑆𝐸
value, that you get from the R software output.

ò Solution

1. The ANOVA table values were calculated in the code solutions for question 2:

Type of variance Distance Degrees of freedom SSQ Mean square

Regression 𝑦^𝑖 − y 𝑘−2 709.9 354.9
Error 𝑦𝑖 − 𝑦^𝑖 𝑛−𝑘 2314.9 7.87
Total 𝑦𝑖 − y 𝑛 3024.8 10.2

√︁
2314.9
The residual standard error, or just standard error, 𝑆𝐸 = 296−2 = 2.8 %CO2 , which agrees with
the value from R.

2. These residuals were normally distributed, as verified in the q-q plot:

As mentioned in the help(qqPlot) output, the dashed red line is the confidence envelope at
the 95% level. The single point just outside the confidence envelope is not going to have any
practical effect on our assumption of normality. We expect 1 point in 20 to lie outside the limits.

Read ahead, if required, on the meaning of studentized residuals (page 262), which are used on the
𝑦-axis.

3. For the thermocouple data set:

Type of variance Distance Degrees of freedom SSQ Mean square

Regression 𝑦^𝑖 − y 𝑘−2 32877 16438
108 [Link]
Error 𝑦𝑖 − 𝑦^𝑖 𝑛−𝑘 122.7 15.3
Total 𝑦𝑖 − y 𝑛 33000 3300

296 Chapter 4. Least Squares Modelling Review

 Release 547b16

√︁
122.7
The residual standard error, or just standard error, 𝑆𝐸 = 10−2 = 3.9 K, which agrees with the
value from R.

ò Question

Use the mature cheddar cheese data set109 for this question.

1. Choose any 𝑥-variable, either Acetic acid concentration (already log-transformed), H2S
concentration (already log-transformed), or Lactic acid concentration (in original units) and
use this to predict the Taste variable in the data set. The Taste is a subjective measurement,
presumably measured by a panel of tasters.

Prove that you get the same linear model coefficients, 𝑅2 , 𝑆𝐸 and confidence intervals whether
or not you first mean center the 𝑥 and 𝑦 variables.

2. What is the level of correlation between each of the 𝑥-variables. Also show a scatterplot matrix
to learn what this level of correlation looks like visually.

• Report your correlations as a 3 × 3 matrix, where there should be 1.0’s on the diagonal, and
values between −1 and +1 on the off-diagonals.

3. Build a linear regression that uses all three 𝑥-variables to predict 𝑦.

• Report the slope coefficient and confidence interval for each 𝑥-variable

• Report the model’s standard error. Has it decreased from the model in part 1?

• Report the model’s 𝑅2 value. Has it decreased?

ò Solution

1. We used the acetic acid variable as 𝑥 and derived the following two models to predict taste, 𝑦:

• No mean centering of 𝑥 and 𝑦: 𝑦 = −61.5 + 15.65𝑥

• With mean centering of 𝑥 and 𝑦: 𝑦 = 0 + 15.65𝑥

These results were found from both models:

• Residual standard error, 𝑆𝐸 = 13.8 on 28 degrees of freedom

• Multiple R-squared, 𝑅2 = 0.30

• Confidence interval for the slope, 𝑏𝑎 was: 6.4 ≤ 𝑏𝐴 ≤ 24.9.

Please see the R code at the end of this question.

If you had used 𝑥 = H2S, then 𝑆𝐸 = 10.8 and if used 𝑥 = Lactic, then 𝑆𝐸 = 11.8.

2. The visual level of correlation is shown in the first 3 × 3 plots below, while the relationship of
each 𝑥 to 𝑦 is shown in the last row and column:

109 [Link]

4.13. Exercises 297

Process Improvement Using Data

The numeric values for the correlation between the 𝑥-variables are:
⎡ ⎤
1.0 0.618 0.604
⎣0.618 1.0 0.644⎦
⎢ ⎥
0.604 0.644 1.0

There is about a 60% correlation between each of the 𝑥-variables in this model, and in each case
the correlation is positive.

3. A combined linear regression model is 𝑦 = −28.9 + 0.31𝑥𝐴 + 3.92𝑥𝑆 + 19.7𝑥𝐿 where 𝑥𝐴 is the
log of the acetic acid concentration, 𝑥𝑆 is the log of the hydrogen sulphide concentration and 𝑥𝐿
is the lactic acid concentration in the cheese. The confidence intervals for each coefficient are:

• −8.9 ≤ 𝑏𝐴 ≤ 9.4

• 1.4 ≤ 𝑏𝑆 ≤ 6.5

• 1.9 ≤ 𝑏𝐴 ≤ 37

The 𝑅2 value is 0.65 in the MLR, compared to the value of 0.30 in the single variable regression.
The 𝑅2 value will always decrease when adding a new variable to the model, even if that
variable has little value to the regression model (yet another caution related to 𝑅2 ).

The MLR standard error is 10.13 on 26 degrees of freedom, a decrease of about 3 units from the
individual regression in part 1; a small decrease given the 𝑦-variable’s range of about 50 units.

Since each 𝑥-variable is about 60% correlated with the others, we can loosely interpret this by
inferring that either lactic, or acetic or H2S could have been used in a single-variable

298 Chapter 4. Least Squares Modelling Review

 Release 547b16

regression. In fact, if you compare 𝑆𝐸 values for the single-variable regressions, (13.8, 10.8 and
11.8), to the combined regression 𝑆𝐸 of 10.13, there isn’t much of a reduction in the MLR’s
standard error.

This interpretation can be quite profitable: it means that we get by with one only one 𝑥-variable
to make a reasonable prediction of taste in the future, however, the other two measurements
must be consistent. In other words we can pick lactic acid as our predictor of taste (it might be
the cheapest of the 3 to measure). But a new cheese with high lactic acid, must also have high
levels of H2S and acetic acid for this prediction to work. If those two, now unmeasured
variables, had low levels, then the predicted taste may not be an accurate reflection of the true
cheese’s taste! We say “the correlation structure has been broken” for that new observation.

Other, advanced explanations:

Highly correlated 𝑥-variables are problematic in least squares, because the confidence intervals
and slope coefficients are not independent anymore. This leads to the problem we see above: the
acetic acid’s effect is shown to be insignificant in the MLR, yet it was significant in the
single-variable regression! Which model do we believe?

This resolution to this problem is simple: look at the raw data and see how correlated each of the
𝑥-variables are with each other. One of the shortcomings of least squares is that we must invert
X′ X. For highly correlated variables this matrix is unstable in that small changes in the data
lead to large changes in the inversion. What we need is a method that handles correlation.

One quick, simple, but suboptimal way to deal with high correlation is to create a new variable,
𝑥avg = 0.33𝑥𝐴 + 0.33𝑥𝑆 + 0.33𝑥𝐿 that blends the 3 separate pieces of information into an average.
Averages are always less noisy than the separate variables the make up the average. Then use
this average in a single-variable regression. See the code below for an example.

cheese <- [Link]('[Link]

summary(cheese)

# Proving that mean-centering has no effect on model parameters

x <- cheese$Acetic
y <- cheese$Taste
summary(lm(y ~ x))
confint(lm(y ~ x))

[Link] <- x - mean(x)

[Link] <- y - mean(y)
summary(lm([Link] ~ [Link]))
confint(lm([Link] ~ [Link] ))

# Correlation amount in the X's. Also plot it

cor(cheese[,2:5])
bitmap('[Link]', type="png256",
width=6, height=6, res=300, pointsize=14)
plot(cheese[,2:5])
[Link]()

# Linear regression that uses all three X's

model <- lm(cheese$Taste ~ cheese$Acetic + cheese$H2S + cheese$Lactic)
summary(model)
confint(model)

# Use an "average" x
[Link] <- 1/3*cheese$Acetic + 1/3*cheese$H2S + 1/3*cheese$Lactic
[Link] <- lm(cheese$Taste ~ [Link])
summary([Link])
confint([Link])

4.13. Exercises 299

Process Improvement Using Data

A Pandas / scikit-learn version of the same workflow is given below for reference:

import pandas as pd
from [Link] import scatter_matrix
from sklearn.linear_model import LinearRegression

cheese = pd.read_csv(
"[Link]
)

# Drop the case identifier; it is not a

# variable to model with.
cheese = [Link](columns="Case")

# Correlation matrix and scatter plot matrix:

[Link]()
scatter_matrix(
cheese,
alpha=0.8,
marker="s",
figsize=(8, 8),
diagonal="kde",
)

# Single-variable model: predict Taste from

# acetic acid concentration.
X = cheese[["Acetic"]].values
y = cheese["Taste"].values
single = LinearRegression().fit(X, y)
print(
f"Intercept = {single.intercept_:.3f}, "
f"slope = {single.coef_[0]:.3f}"
)

# Multiple linear regression with all three

# x-variables:
X_mlr = cheese[["Acetic", "H2S", "Lactic"]].values
mlr = LinearRegression().fit(X_mlr, y)
print(
f"Intercept = {mlr.intercept_:.3f}, "
f"coefficients = {mlr.coef_}"
)
print(f"R^2 = {[Link](X_mlr, y):.3f}")

ò Question

The Kamyr digester data set110 comes from a pulp and paper plant. Use it to practise the early steps
of the data-analysis workflow before fitting a least squares model.

1. Read the data, drop any non-numeric identifier columns, and produce a histogram of every
variable. Find two variables with a clearly bimodal distribution, and two that are roughly
normally distributed.

2. For each bimodal variable, plot it in time order. Does the bimodal histogram now make sense?

3. Find the three columns most strongly positively correlated, and the three most strongly
negatively correlated, with the outcome variable Y-Kappa. Build a 7-column data frame that
combines those six predictors with Y-Kappa, and produce a scatter plot matrix for that subset
only.

4. If you needed to increase the Kappa number for this process, which variables would you change,

300 Chapter 4. Least Squares Modelling Review

 Release 547b16

and in which direction?

ò Solution

Starter code for the exploration:

import pandas as pd

digester = pd.read_csv(
"[Link]
)

# A histogram per numeric column.

# Adjust figsize and bins to taste:
[Link](figsize=(15, 12), bins=30,
color="lightblue")

# Numeric correlation matrix.

# Sort by the column we care about,
# from most negative to most positive:
correlations = [Link]()["Y-Kappa"]
correlations.sort_values()

# Pick the 3 strongest positive and 3

# strongest negative correlations, then
# build a 7-column subset:
positives = (
[Link]("Y-Kappa")
.sort_values(ascending=False)
.head(3)
.[Link]()
)
negatives = (
[Link]("Y-Kappa")
.sort_values()
.head(3)
.[Link]()
)
subset = digester[positives + negatives + ["Y-Kappa"]]

from [Link] import scatter_matrix

scatter_matrix(subset, alpha=0.4,
figsize=(12, 12), diagonal="kde")

ò Question

In this question we will revisit the bioreactor yield111 data set and fit a linear model with all
𝑥-variables to predict the yield. (This data was also used in a previous question (page 287).)

1. Provide the interpretation for each coefficient in the model, and also comment on each one’s
confidence interval when interpreting it.

2. Compare the 3 slope coefficient values you just calculated, to those from the previous question:

• 𝑦^ = 102.5 − 0.69𝑇 , where 𝑇 is tank temperature

• 𝑦^ = −20.3 + 0.016𝑆, where 𝑆 is impeller speed

110 [Link]

4.13. Exercises 301

Process Improvement Using Data

• 𝑦^ = 54.9 − 16.7𝐵, where 𝐵 is 1 if baffles are present and 𝐵 = 0 with no baffles

Explain why your coefficients do not match.

3. Are the residuals from the multiple linear regression model normally distributed?

4. In this part we are investigating the variance-covariance matrices used to calculate the linear
model.

1. First center the 𝑥-variables and the 𝑦-variable that you used in the model.

Note: feel free to use MATLAB, or any other tool to answer this question. If you are using R,
then you will benefit from this page in the R tutorial112 . Also, read the help for the
[Link](...) function to get the X-matrix. Then read the help for the sweep(...)
function, or more simply use the scale(...) function to do the mean-centering.

2. Show your calculated X𝑇 X and X𝑇 y variance-covariance matrices from the centered data.

3. Explain why the interpretation of covariances in X𝑇 y match the results from the full MLR
model you calculated in part 1 of this question.
(︀ )︀−1 𝑇
4. Calculate b = X𝑇 X X y and show that it agrees with the estimates that R calculated
(even though R fits an intercept term, while your b does not).

5. What would be the predicted yield for an experiment run without baffles, at 4000 rpm impeller
speed, run at a reactor temperature of 90 °C?

ò Solution

1. The full linear model that relates bioreactor yield to 3 factors is:

𝑦 = 52.5 − 0.47𝑥𝑇 + 0.0087𝑥𝑆 − 9.1𝑥𝐵

where 𝑥𝑇 is the temperature value in °C, 𝑥𝑆 is the speed in RPM and 𝑥𝐵 is a coded variable,
0=no baffles and 1=with baffles.

• Temperature effect: −0.74 < 𝛽𝑇 < −0.21, with 𝑏𝑇 = −0.47 indicates that increasing the
temperature by 1 °C will decrease the yield on average by 0.47 units, holding the speed and
baffle effects constant. The confidence interval does not span zero, indicating this coefficient is
significant. An ad-hoc way I sometimes use to gather the effect of a variables is to ask what is
the effect over the entire range of temperature, ∼ 40°C:

– Δ𝑦 = −0.74 × 40 = −29.6 % decrease in yield

– Δ𝑦 = −0.21 × 40 = −8.4 % decrease in yield

A tighter confidence interval will have these two values even closer, but given the range of
the y’s in the data cover about 35% units, this temperature effect is important, and will have a
noticeable effect at either end of the confidence interval.

• Speed effect: 0.34 < 𝛽𝑆 < 17.0822 with 𝑏𝑆 = 8.7 per 1000 RPM: indicates that increase the RPM
by 1000 units will increase the yield by about 8.7 units, holding the other factors constant.
While the confidence interval does not span zero, it is quite wide.

111 [Link]
112 [Link]

302 Chapter 4. Least Squares Modelling Review

 Release 547b16

• Baffles effect: −15.9 < 𝛽𝐵 < −2.29 with 𝑏𝐵 = −9.1 indicates the presence of baffles decreases
yield on average by 9.1 units, holding the temperature and speed effects constant. The
confidence interval does not span zero, indicating this coefficient is significant. It is an
important effect to consider when wanting to change yield.

2. In the previous question (page 287) we considered the separate effects:

• 𝑦^ = 102.5 − 0.69𝑇 , where 𝑇 is tank temperature

• 𝑦^ = −20.3 + 0.016𝑆, where 𝑆 is impeller speed

• 𝑦^ = 54.9 − 16.7𝐵, where 𝐵 is 1 if baffles are present and 𝐵 = 0 with no baffles

The signs of the coefficients between MLR and OLS (ordinary least squares) are in agreement,
but not the magnitudes. The problem is that when building the single-variable regression model
we place all the other effects into the residuals. For example, a model considering only
temperature, but ignoring speed and baffles is essentially saying:
𝑦 = 𝑏0 + 𝑏𝑇 𝑥𝑇 + 𝑒
𝑦 = 𝑏0 + 𝑏𝑇 𝑥𝑇 + (𝑒′ + 𝑏′𝑆 𝑥𝑆 + 𝑏′𝐵 𝑥𝐵 )
i.e. we are lumping the effect of speed and baffles which we have omitted from the model, into
the residuals, and we should see structure in our residuals due to these omitted effects.

Since the objective function for least squares is to minimize the sum of squares of the residuals,
the effect of speed and baffles can be “smeared” into the coefficient we are estimating, the 𝑏𝑇
coefficient, and this is even more so when any of the 𝑥-variables are correlated with each other.

3. The residuals from the multiple linear regression model are normally distributed. This can be
verified in the q-q plot below:

4. The X𝑇 X and X𝑇 y variance-covariance matrices from the centered data, where the order of the
variables is: temperature, speed and then baffles:
⎡ ⎤
1911 −9079 36.43
X𝑇 X = ⎣−9079 1844000 −1029⎦
⎢ ⎥
36.43 −1029 3.43
⎡ ⎤
−1310
𝑇
X y = ⎣ 29690 ⎦
⎢ ⎥
−57.3

4.13. Exercises 303

Process Improvement Using Data

The covariances show a negative relationship between temperature and yield (−1310), a positive
relationship between speed and yield (29690) and a negative relationship between baffles and
yield (−57.3). Unfortunately, covariances are unit-dependent, so we cannot interpret the relative
magnitude of these values: i.e. it would be wrong to say that speed has a greater effect than
temperature because its covariance magnitude is larger. If we had two 𝑥-variables with the same
units, then we could compare them fairly, but not in this case where all 3 units are different.

We can calculate
⎡ ⎤
−0.471
)︀−1 𝑇
b = X𝑇 X
(︀
X y = ⎣ 0.0087 ⎦
⎢ ⎥
−9.1

which agrees with the estimates that R calculated (even though R fits an intercept term, while we
do not estimate an intercept).

5. The predicted yield yield for an experiment run without baffles, at 4000 rpm impeller speed, run
at a reactor temperature of 90 °C would be 45%:

𝑦^ = 52.5 − 0.47𝑥𝑇 + 0.0087𝑥𝑆 − 9.1𝑥𝐵

𝑦^ = 52.5 − 0.47(90) + 0.0087(4000) − 9.1(0) = 45.0

All the code for this question is given below:

bio <- [Link]('[Link]

summary(bio)

# Temperature-Yield model
[Link] <- lm(bio$yield ~ bio$temperature)
summary([Link])

# Impeller speed-Yield model

[Link] <- lm(bio$yield ~ bio$speed)
summary([Link])

# Baffles-Yield model
[Link] <- lm(bio$yield ~ bio$baffles)
summary([Link])

# Model of everything
[Link] <- lm(bio$yield ~ bio$temperature + bio$speed + bio$baffles)
summary([Link])
confint([Link])

# Residuals normally distributed? Yes

library(car)
bitmap('[Link]', type="png256",
width=6, height=6, res=300, pointsize=14)
par(mar=c(4.2, 4.2, 1.5, 0.5))
qqPlot(resid([Link]))
[Link]()

# Calculate X matrix and y vector

data <- [Link]([Link])
X <- data[,2:4]
y <- matrix(bio$yield)

# Center the data first

X <- scale(X, scale=FALSE)
y <- scale(y, scale=FALSE)

# Now calculate variance-covariance matrices

XTy <- t(X) %*% y

304 Chapter 4. Least Squares Modelling Review

 Release 547b16

XTX <- t(X) %*% X

b <- solve(XTX) %*% XTy
# b agrees with R's calculation from ``[Link]``

ò Question

In this question we will use the LDPE data113 which is data from a high-fidelity simulation of a
low-density polyethylene reactor. LDPE reactors are very long, thin tubes. In this particular case
the tube is divided in 2 zones, since the feed enters at the start of the tube, and some point further
down the tube (start of the second zone). There is a temperature profile along the tube, with a
certain maximum temperature somewhere along the length. The maximum temperature in zone 1,
Tmax1 is reached some fraction z1 along the length; similarly in zone 2 with the Tmax2 and z2
variables.

We will build a linear model to predict the SCB variable, the short chain branching (per 1000 carbon
atoms) which is an important quality variable for this product. Note that the last 4 rows of data are
known to be from abnormal process operation, when the process started to experience a problem.
However, we will pretend we didn’t know that when building the model, so keep them in for now.

1. Use only the following subset of 𝑥-variables: Tmax1, Tmax2, z1 and z2 and the 𝑦 variable =
SCB. Show the relationship between these 5 variables in a scatter plot matrix.

Use this code to get you started (make sure you understand what it is doing):

LDPE <- [Link]('[Link]

subdata <- [Link](cbind(LDPE$Tmax1, LDPE$Tmax2, LDPE$z1, LDPE$z2, LDPE$SCB))
colnames(subdata) <- c("Tmax1", "Tmax2", "z1", "z2", "SCB")

Using bullet points, describe the nature of relationships between the 5 variables, and particularly
the relationship to the 𝑦-variable.

2. Let’s start with a linear model between z2 and SCB. We will call this the z2 model. Let’s
examine its residuals:

1. Are the residuals normally distributed?

2. What is the standard error of this model?

3. Are there any time-based trends in the residuals (the rows in the data are already in
time-order)?

4. Use any other relevant plots of the predicted values, the residuals, the 𝑥-variable, as described
in class, and diagnose the problem with this linear model.

5. What can be done to fix the problem? (You don’t need to implement the fix yet).

3. Show a plot of the hat-values (leverage) from the z2 model.

1. Add suitable horizontal cut-off lines to your hat-value plot.

2. Identify on your plot the observations that have large leverage on the model

3. Remove the high-leverage outliers and refit the model. Call this the [Link] model

4. Show the updated hat-values and verify whether the problem has mostly gone away

4.13. Exercises 305

Process Improvement Using Data

Note: see the R tutorial on how to rebuild a model by removing points

4. Use the influenceIndexPlot(...) function in the car library on both the z2 model and the
[Link] model. Interpret what each plot is showing for the two models. You may ignore
the Bonferroni p-values subplot.

ò Solution

1. A scatter plot matrix of the 5 variables is

• Tmax1 and z1 show a strongish negative correlation

• Tmax1 and SCB show a strong positive correlation

• Tmax2 and z2 have a really strong negative correlation, and the 4 outliers are very clearly
revealed in almost any plot with z2

• z1 and SCB have a negative correlation

• Tmax2 and SCB have a negative correlation

• Very little relationship appears between Tmax1 and Tmax2, which is expected, given
how/where these 2 data variables are recorded.

• Similarly for Tmax2 and z2.

113 [Link]

306 Chapter 4. Least Squares Modelling Review

 Release 547b16

̂︂ = 32.23 − 10.6𝑧2
2. A linear model between z2 and SCB: SCB

First start with a plot of the raw data with this regression line superimposed:

which helps when we look at the q-q plot of the Studentized residuals to see the positive and the
negative residuals:

1. We notice there is no strong evidence of non-normality, however, we can see a trend in the
tails on both sides (there are large positive residuals and large negative residuals). The
identified points in the two plots help understand which points affect the residual tails.

2. This model’s standard error is 𝑆𝐸 = 0.114, which should be compared to the range of the
𝑦-axis, 0.70 units, to get an idea whether this is large or small, so about 15% of the range.
Given that a conservative estimate of the prediction interval is ±2𝑆𝐸 , or a total range of 4𝑆𝐸 ,
this is quite large.

4.13. Exercises 307

Process Improvement Using Data

3. The residuals in time-order

Show no consistent structure, however we do see the short upward trend in the last 4 points.
The autocorrelation function (not shown here), shows there is no autocorrelation, i.e. the
residuals appear independent.

4. Three plots that do show a problem with the linear model:

• Predictions vs residuals: definite structure in the residuals. We expect to see no structure, but
a definite trend, formed by the 4 points is noticeable, as well as a negative correlation at
high predicted SCB.

• 𝑥-variable vs residuals: definite structure in the residuals, which is similar to the above
plot.

• Predicted vs measured 𝑦: we expect to see a strong trend about a 45° line (shown in blue).
The strong departure from this line indicates there is a problem with the model

308 Chapter 4. Least Squares Modelling Review

 Release 547b16

5. We can consider removing the 4 points that strongly bias the observed vs predicted plot
above.

3. A plot of the hat-values (leverage) from the regression of SCB on z2 is:

with 2 and 3 times the average hat value shown for reference. Points 52, 53 and 54 have leverage
that is excessive, confirming what we saw in the previous part of this question.

Once these points are removed, the model was rebuilt, and this time showed point 51 as an
high-leverage outlier. This point was removed and the model rebuilt.

The hat values from this updated model are:

4.13. Exercises 309

Process Improvement Using Data

which is reasonable to stop at, since the problem has mostly gone away. If you keep omitting
points, you will likely deplete all the data. At some point, especially when there is no obvious
structure in the residuals, it is time to stop interrogating (i.e. investigating) and removing
outliers.

The updated model has a slightly improved standard error 𝑆𝐸 = 0.11 and the least squares
model fit (see the R code) appears much more reasonable in the data.

4. The influence index plots for the model with all 54 points is shown first, followed by the
influence index plot of the model with only the first 50 points.

310 Chapter 4. Least Squares Modelling Review

 Release 547b16

The increasing leverage, as the abnormal process operation develops is clearly apparent. This
leverage is not “bad” (i.e. influential) initially, because it is “in-line” with the regression slope.
But by observation 54, there is significant deviation that observation 54 has high residuals
distance, and therefore a combined high influence on the model (high Cook’s D).

4.13. Exercises 311

Process Improvement Using Data

The updated model shows shows only point 8 as an influential observation, due to its moderate
leverage and large residual. However, this point does not warrant removal, since it is just above
the cut-off value of 4/(𝑛 − 𝑘) = 4/(50 − 2) = 0.083 for Cook’s distance.

The other large hat values don’t have large Studentized residuals, so they are not influential on
the model.

Notice how the residuals in the updated model are all a little smaller than in the initial model.

All the code for this question is given here:

LDPE <- [Link]('[Link]

summary(LDPE)
N <- nrow(LDPE)

sub <- [Link](cbind(LDPE$Tmax1, LDPE$Tmax2, LDPE$z1, LDPE$z2, LDPE$SCB))

colnames(sub) <- c("Tmax1", "Tmax2", "z1", "z2", "SCB")

bitmap('[Link]', type="png256",
width=6, height=6, res=300, pointsize=14)
plot(sub)
[Link]()

model.z2 <- lm(sub$SCB ~ sub$z2)

summary(model.z2)

# Plot raw data

bitmap('[Link]', type="png256",
width=6, height=6, res=300, pointsize=14)
plot(sub$z2, sub$SCB)
abline(model.z2)

312 Chapter 4. Least Squares Modelling Review

 Release 547b16

identify(sub$z2, sub$SCB)
[Link]()

# Residuals normal? Yes, but have heavy tails

bitmap('[Link]', type="png256",
width=6, height=6, res=300, pointsize=14)
library(car)
qqPlot(model.z2, [Link]="identify")
[Link]()

# Residual plots in time order: no problems detected

# Also plotted the acf(...): no problems there either
bitmap('[Link]', type="png256",
width=6, height=6, res=300, pointsize=14)
plot(resid(model.z2), type='b')
abline(h=0)
[Link]()

acf(resid(model.z2))

# Predictions vs residuals: definite structure in the residuals!

bitmap('[Link]', type="png256",
width=6, height=6, res=300, pointsize=14)
plot(predict(model.z2), resid(model.z2))
abline(h=0, col="blue")
[Link]()

# x-data vs residuals: definite structure in the residuals!

bitmap('[Link]', type="png256",
width=6, height=6, res=300, pointsize=14)
plot(sub$Tmax2, resid(model.z2))
abline(h=0, col="blue")
identify(sub$z2, resid(model.z2))
[Link]()

# Predictions-vs-y
bitmap('[Link]', type="png256",
width=6, height=6, res=300, pointsize=14)
plot(sub$SCB, predict(model.z2))
abline(a=0, b=1, col="blue")
identify(sub$SCB, predict(model.z2))
[Link]()

# Plot hatvalues
bitmap('[Link]', type="png256",
width=6, height=6, res=300, pointsize=14)
plot(hatvalues(model.z2))
[Link] <- 2/N
abline(h=2*[Link], col="darkgreen")
abline(h=3*[Link], col="red")
text(3, y=2*[Link], expression(2 %*% bar(h)), pos=3)
text(3, y=3*[Link], expression(3 %*% bar(h)), pos=3)
identify(hatvalues(model.z2))
[Link]()

# Remove observations (observation 51 was actually detected after

# the first iteration of removing 52, 53, and 54: high-leverage points)
build <- seq(1,N)
remove <- -c(51, 52, 53, 54)
[Link] <- lm(model.z2, subset=build[remove])

# Plot updated hatvalues

plot(hatvalues([Link]))
N <- length([Link]$residuals)
[Link] <- 2/N
abline(h=2*[Link], col="darkgreen")
abline(h=3*[Link], col="red")
identify(hatvalues([Link]))

4.13. Exercises 313

Process Improvement Using Data

# Observation 27 still has high leverage: but only 1 point

# Problem in the residuals gone? Yes

plot(predict([Link]), resid([Link]))
abline(h=0, col="blue")

# Does the least squares line fit the data better?

plot(sub$z2, sub$SCB)
abline([Link])

# Finally, show an influence plot

influencePlot(model.z2, [Link]="identify")
influencePlot([Link], [Link]="identify")

# Or the influence index plots

influenceIndexPlot(model.z2, [Link]="identify")
influenceIndexPlot([Link], [Link]="identify")

#-------- Use all variables in an MLR (not required for question)

[Link] <- lm(sub$SCB ~ sub$z1 + sub$z2 + sub$Tmax1 + sub$Tmax2)

summary([Link])
confint([Link])

ò Question

A concrete slump test is used to test for the fluidity, or workability, of concrete. It’s a crude, but
quick test often used to measure the effect of polymer additives that are mixed with the concrete to
improve workability.

The concrete mixture is prepared with a polymer additive. The mixture is placed in a mold and
filled to the top. The mold is inverted and removed. The height of the mold minus the height of the
remaining concrete pile is called the “slump”.

Figure from Wikipedia114

Your company provides the polymer additive, and you are developing an improved polymer
formulation, call it B, that hopefully provides the same slump values as your existing polymer, call
it A. Formulation B costs less money than A, but you don’t want to upset, or lose, customers by
varying the slump value too much.

The following slump values were recorded over the course of the day:

314 Chapter 4. Least Squares Modelling Review

 Release 547b16

Additive Slump value [cm]

A 5.2
A 3.3
B 5.8
A 4.6
B 6.3
A 5.8
A 4.1
B 6.0
B 5.5
B 4.5

You can derive the 95% confidence interval for the true, but unknown, difference between the effect
of the two additives:

√︂ −𝑐𝑡 ≤ 𝑧 ≤ +𝑐𝑡 √︂
(︁ )︁ (︁ )︁
1 1
(𝑥𝐵 − 𝑥𝐴 ) − 𝑐𝑡 𝑠2𝑃 𝑛𝐵 + 𝑛𝐴 ≤ 𝜇𝐵 − 𝜇𝐴 ≤ (𝑥𝐵 − 𝑥𝐴 ) + 𝑐𝑡 𝑠2𝑃 𝑛1𝐵 + 1
𝑛𝐴
√︁ √︁
1.02 − 2.3 0.709 15 + 15 ≤ 1.02 + 2.3 0.709 15 + 51
(︀ )︀ (︀ )︀
≤ 𝜇𝐵 − 𝜇𝐴
−0.21 ≤ 𝜇𝐵 − 𝜇𝐴 ≤ 2.2

Fit a least squares model to the data using an integer variable, 𝑥𝐴 = 0 for additive A, and 𝑥𝐴 = 1 for
additive B. The model should include an intercept term also: 𝑦 = 𝑏0 + 𝑏𝐴 𝑥𝐴 . Hint: use R to build the
model, and search the R tutorial with the term categorical variable or integer variable for assistance.

Show that the 95% confidence interval for 𝑏𝐴 gives exactly the same lower and upper bounds, as
derived above with the traditional approach for tests of differences.

ò Solution

This short piece of R code shows the expected result when regressing the slump value onto the
binary factor variable:

additive <- [Link](c("A", "A", "B", "A", "B", "A", "A", "B", "B", "B"))
slump <- c(5.2, 3.3, 5.8, 4.6, 6.3, 5.8, 4.1, 6.0, 5.5, 4.5)
confint(lm(slump ~ additive))

2.5 % 97.5 %
(Intercept) 3.7334823 5.466518
additive -0.2054411 2.245441

Note that this approach works only if your coding has a one unit difference between the two levels.
For example, you can code 𝐴 = 17 and 𝐵 = 18 and still get the same result. Usually though 𝐴 = 0
and 𝐵 = 1 or the 𝐴 = 1 and 𝐵 = 2 coding is the most natural, but all 3 of these codings would give
the same confidence interval (the intercept changes though).

114 [Link]

4.13. Exercises 315

Process Improvement Using Data

ò Question

Some data were collected from tests where the compressive strength, 𝑥, used to form concrete was
measured, as well as the intrinsic permeability of the product, 𝑦. There were 16 data points
collected. The mean 𝑥-value was 𝑥 = 3.1 and the variance of the 𝑥-values was 1.52. The average
𝑦-value was 40.9. The estimated covariance between 𝑥 and 𝑦 was −5.5.

The least squares estimate of the slope and intercept was: 𝑦 = 52.1 − 3.6𝑥.

1. What is the expected permeability when the compressive strength is at 5.8 units?

2. Calculate the 95% confidence interval for the slope if the standard error from the model was 4.5
units. Is the slope coefficient statistically significant?

3. Provide a rough estimate of the 95% prediction interval when the compressive strength is at 5.8
units (same level as for part 1). What assumptions did you make to provide this estimate?

4. Now provide a more accurate, calculated 95% prediction confidence interval for the previous
part.

ò Solution

1. It is 𝑦^ = 52.1 − 3.6(5.8) = 31.22

2. From the definition:

2
2 𝑆𝐸
𝑆𝐸 (𝑏𝑖 ) = ∑︀ 2
𝑗 (𝑥𝑗 − x)
4.52
= ∑︀ 2
𝑗 (𝑥𝑗 − x)
We need the denominator term, which can be found by back-calculation:
2
∑︀
𝑗 (𝑥𝑗 − x)
𝒱(𝑥) = 1.52 =
𝑛−1
∑︁
(𝑥𝑗 − x)2 = 1.52 × (16 − 1) = 22.8
𝑗

So the 95% confidence interval for the slope, 𝑏𝑖 :

𝑏𝑖 ± 𝑐𝑡 𝑆𝐸 (𝑏𝑖 )
√︂
4.52
−3.6 ± 2.14
22.8
−3.6 ± 2.02
where 𝑐𝑡 = 2.14 from the 𝑡-distribution with 𝑛 − 𝑘 = 16 − 2 degrees of freedom.

Since this confidence interval does not span zero, we conclude the slope coefficient is statistically
significant.

3. A rough estimate would be at 𝑦^ ± 2𝑆𝐸 , in other words, 31.2 ± 9.0, which is [22.2, 40.2]
√︀
4. A more accurate prediction interval is given by 𝑦^𝑖 ± 𝑐𝑡 𝑉 {^ 𝑦𝑖 }, where:
(︃ )︃
2 1 (𝑥𝑖 − x)2
𝑉 {^
𝑦𝑖 } = 𝑆𝐸 1 + + ∑︀ 2
𝑛 𝑗 (𝑥𝑗 − x)
(5.8 − 3.1)2
(︂ )︂
1
= 4.52 1 + +
16 22.8
= 27.99
316 Chapter 4. Least Squares Modelling Review
 Release 547b16

and represents the variance of the predicted 𝑦^𝑖 at the given value of 𝑥𝑖 = 5.8.
√︀ √
The confidence interval, or prediction interval for this 𝑦^𝑖 is ±𝑐𝑡 𝑉 {^
𝑦𝑖 } = ±2.14 27.99 = ±11.3,
a bit larger than the rough estimate above.

ò Question

A simple linear model relating reactor temperature to polymer viscosity is desirable, because
measuring viscosity online, in real time is far too costly, and inaccurate. Temperature, on the other
hand, is quick and inexpensive. This is the concept of soft sensors, also known as inferential sensors.

Data were collected from a rented online viscosity unit and a least squares model build:

𝑣^ = 1977 − 3.75𝑇

where the viscosity, 𝑣, is measured in Pa.s (Pascal seconds) and the temperature is in Kelvin. A
reasonably linear trend was observed over the 86 data points collected. Temperature values were
taken over the range of normal operation: 430 to 480 K and the raw temperature data had a sample
standard deviation of 8.2 K.

The output from a certain commercial software package was:

Analysis of Variance
---------------------------------------------------------
Sum of Mean
Source DF Squares Square
Model 2 9532.7 4766.35
Error 84 9963.7 118.6
Total 86 19496.4
Root MSE XXXXX
R-Square XXXXX

1. Which is the causal direction: does a change in viscosity cause a change in temperature, or does
a change in temperature cause a change in viscosity?

2. Calculate the Root MSE, what we have called standard error, 𝑆𝐸 in this course.

3. What is the 𝑅2 value that would have been reported in the above output?

4. What is the interpretation of the slope coefficient, -3.75, and what are its units?

5. What is the viscosity prediction at 430K? And at 480K?

6. In the future you plan to use this model to adjust temperature, in order to meet a certain
viscosity target. To do that you must be sure the change in temperature will lead to the desired
change in viscosity.

What is the 95% confidence interval for the slope coefficient, and interpret this confidence interval
in the context of how you plan to use this model.

7. The standard error features prominently in all derivations related to least squares. Provide an
interpretation of it and be specific in any assumption(s) you require to make this interpretation.

4.13. Exercises 317

Process Improvement Using Data

ò Solution

1. The causal direction is that a change in temperature causes a change in viscosity.

√︃ ∑︀ √︂
𝑒2𝑖 9963.7
2. The Root MSE = 𝑆𝐸 = = = 10.9 Pa.s.
𝑛−𝑘 84
RegSS 9532.7
3. 𝑅2 = = = 0.49
TSS 19496.4
4. The slope coefficient is −3.75 Pa.s
𝐾 and implies that the viscosity is expected to decrease by 3.75
Pa.s for every one degree increase in temperature.

5. The viscosity prediction at 430K is 1977 − 3.75 × 430 = 364.5 Pa.s and is 177 Pa.s at 480 K.

6. The confidence interval is

𝑏1 ± 𝑐𝑡 𝑆𝐸 (𝑏1 )
2
𝑆𝐸
−3.75 ± 1.98 ∑︀ 2
(𝑥𝑗 − 𝑥)
𝑗
10.9
−3.75 ± 1.98
697
−3.75 ± 0.031
2
(𝑥𝑗 − 𝑥) 2
where = 8.2, so one can solve for (𝑥𝑗 − 𝑥) (though any reasonable value/attempt to
𝑛−1
get this value should be acceptable) and 𝑐𝑡 = 1.98, using 𝑛 − 𝑘 degrees of freedom at 95%
confidence.

Interpretation: this interval is extremely narrow, i.e. our slope estimate is precise. We can be sure
that any change made to the temperature in our system will have the desired effect on viscosity
in the feedback control system.

7. The standard error, 𝑆𝐸 = 10.9 Pa.s is interpreted as the amount of spread in the residuals. In
addition, if we assume the residuals to be normally distributed (easily confirmed with a q-q
plot) and independent. If that is true, then 𝑆𝐸 is the one-sigma standard deviation for the
residuals and we can say 95% of the residuals are expected within a range of ±2𝑆𝐸 .

318 Chapter 4. Least Squares Modelling Review

 CHAPTER 5
DESIGN AND ANALYSIS OF EXPERIMENTS

ò Note

Coursera students

If you are using this chapter with the Coursera MOOC115 (massive open online course), then we
wish to welcome you and want to let you know that this book is generally part of a larger set of
notes. The cross-references in this chapter will point you to other parts, where background
knowledge is provided.
Video for
this section This chapter was written for engineers originally, but you will see the examples are very general
and can be applied to any other systems.

You can safely skip over the section on Experiments with a single variable at two levels (page 325); that
section is not covered in the MOOC. You can also initially skip the section on Why learning about
systems is important (page 322), but make sure you come back and read it.

5.1 Design and analysis of experiments in context

Video for
this section This chapter will take a totally different approach to learning about and understanding systems in
general, not only (chemical) engineering systems. The systems we could apply this to could be as
straightforward as growing plants or perfecting your favourite recipe at home. Or they may be as
complex as the entire production line in a large factory producing multiple products and shipping
them to customers.

In order to learn about a system, we have to disturb it and change it. This is to ensure cause and effect.
If we do not intentionally change the system, we are only guessing, or using our intuition. To disturb
the system, we change several factors. When we make these changes, we say that we have “run an
experiment”.

In this chapter we learn the best way to intentionally disturb the system to learn more about it. We will
use some of the tools of least squares modelling (page 209), visualization (page 1) and univariate statistics
(page 37) that were described in earlier chapters. Where necessary, we will refer back to those earlier
sections.
115 [Link]

319
 Process Improvement Using Data

5.2 Terminology
Video for
this section The area of designed experiments uses specific terminology.

Every experiment has these two components:

1. An outcome: the result or the response from an experiment.

2. One or more factors: a factor is the thing you can change to influence the outcome. Factors are also
called variables.

An important aspect about the outcome is that it is always measurable–in some way. In other words,
after you finish the experiment, you must have some measurement.

Let’s use an example of growing plants. The outcome of growing a plant might be the height of the
plant, or the average width of the leaves, or the number of flowers on the plant. These are numeric
measurements, also called quantitative measurements. Qualitative measurements are also possible.
For example, perhaps the outcome is the colour of the flower: light red, red, or dark red. A qualitative
outcome might also be a description of what happened, for example, pass or fail.

An experiment can have an objective, which combines an outcome and the need to adjust that outcome.
For example, you may want to maximize the height of the plant. Most often you want to maximize or
minimize the outcome as your objective. Sometimes, though, you want the outcome to be the same
even though you are changing factors. For example, you might want to change a recipe for your
favourite pastry to be gluten-free but keep the taste the same as the original recipe. Your outcome is
taste, and your objective is “the same”.

Every experiment always has an outcome. Every experiment does not have to have an objective, but
usually we have an objective in our mind.

Another term we will use is factors. In the plant example, you could have changed three factors:

1. The amount of water that you give the plant each day

2. The amount of fertilizer that you give the plant each week

3. The type of soil you use, A or B

All experiments must have at least one factor that is changed. We distinguish between two types of
factors: numeric factors and categorical factors.

Numeric factors are quantified by measuring, such as giving 15 mL of water or 30 mL of water to the
plant each day. An important point about numerical variables is that there is some order to them. 15
mL of water is less than 30 mL or water. Another name for this type of factor is a quantitative factor.

Categorical factors usually take on a limited number of values. For example, soil type A or soil type B
could be used to grow the plants. Categorical variables have no implicit ordering. You could have
switched the names of soil A and soil B around. Categorical variables and qualitative variables can be
used as synonyms.

Most categorical variables can be converted to continuous variables, with some careful thought. For
example: no water vs some water (categorical) can be converted to 0 mL and 40 mL (now it is
numeric). In the case of soil A vs soil B it might be that soil A contains a higher level of nutrients in
total than soil B, so a numeric version of this factor could be measured as nutrient load.

If you were working in the area of marketing, you might try three different colours of background in
your advertising poster. Those 3 colours are categorical variables in the context of the experiment.

Most experiments will have both numeric and categorical factors.

320 Chapter 5. Design and Analysis of Experiments

 Release 547b16

When we perform an experiment, we call it a run. If we perform eight experiments, we can say “there
are eight runs” in the set of experiments.

5.3 Usage examples

Video for
this section After you complete this chapter, you will be able to answer questions such as those presented in these
scenarios:

• Colleague: We have this list of eight plausible factors that affect the polymer melt index (the
outcome). How do we narrow down the list to a more manageable size and rank their effect on melt
index?

• You: Our initial screening experiments reduced the list down to three factors of interest. Now, how
do we perform the rest of the experiments?

• Manager: Two years ago someone collected these experimental data for the effectiveness of a new
chemical to treat water. What interesting results do you see in this data, and where should we
operate the system to achieve water quality that meets the required standards?

• Colleague: The current production settings for our food product gives us good shelf life, but the
energy used is high. How can we determine other settings (factors) that give long shelf life but
reduce the energy consumed?

• Colleague: We would like to run experiments by varying temperature and pressure, but operating at
both high temperature and pressure is unsafe. How do we plan such an experiment?

Here’s a visual representation of the topics we will cover in this chapter.

5.4 References and readings

• Strongly recommended: Box, Hunter and Hunter, Statistics for Experimenters, 2nd edition. Chapters
5 and 6 with topics from Chapters 11, 12, 13 and 15 are the most heavily used in this chapter.

• Søren Bisgaard: Must a Process Be in Statistical Control Before Conducting Designed

5.3. Usage examples 321

Process Improvement Using Data

Experiments?116 , with discussion (part 1117 , part 2118 , part 3119 , part 4120 , part 5121 and a rejoinder122 ).

• George Box and J. Stuart Hunter, “The 2𝑘−𝑝 Fractional Factorial Designs - Part I123 ”, Technometrics, 3,
311-351, 1961.

• George Box and J. Stuart Hunter, “The 2𝑘−𝑝 Fractional Factorial Designs - Part II124 ”, Technometrics,
3, 449-458, 1961.

• George Box, “Evolutionary Operation: A Method for Increasing Industrial Productivity125 ”, Journal
of the Royal Statistical Society (Applied Statistics), 6, 81-101, 1957.

• William G. Hunter and J. R. Kittrell, “Evolutionary Operation: A Review126 ”, Technometrics, 8,

389-397, 1966.

• Heather Tye, “Application of Statistical Design of Experiments Methods in Drug Discovery127 ”,

Drug Discovery Today, 9, 485-491, 2004.

• R.A. Fisher, Statistical Methods, Experimental Design and Scientific Inference128 , Oxford Science
Publications, 2003.

• Raymond H. Myers, Douglas C. Montgomery and Christine M. Anderson-Cook, Response Surface

Methodology: Process and Product Optimization Using Designed Experiments129 , Wiley, 2009.

• William Hill and William Hunter, “A Review of Response Surface Methodology: A Literature
Survey130 ”, Technometrics, 8, 571-590, 1966.

• Owen L. Davies, The Design and Analysis of Industrial Experiments131 , Chapter 11, revised 2nd
edition, Hafner, 1967.

5.5 Why learning about systems is important

One of the important reasons why we must experiment is that it brings us increased knowledge and a
better understanding of our system. That could lead to profit, or it could help us manufacture
products more efficiently. Once we learn what really happens in our system, we can fix problems and
optimize the system, because we have an improved understanding of cause and effect.

As described in the first reference, the book by Box, Hunter and Hunter (page 321), learning from and
improving a system is an iterative process. It usually follows this cycle:

• Make a conjecture (hypothesis), which we believe is true.

• If it is true, we expect certain consequences.

• Experiment and collect data. Are the consequences that we expected visible in the data?
116 [Link]
117 [Link]
118 [Link]
119 [Link]
120 [Link]
121 [Link]
122 [Link]
123 [Link]
124 [Link]
125 [Link]
126 [Link]
127 [Link]
128 [Link]
129 [Link]
130 [Link]
131 [Link]

322 Chapter 5. Design and Analysis of Experiments

 Release 547b16

• If so, it may lead to the next hypothesis. If not, we formulate an alternative hypothesis. Or perhaps
it is not so clear cut: we see the consequence, but not to the extent expected. Perhaps modifications
are required in the experimental conditions.

And so we go about learning. One of the most frequent reasons we experiment is to fix a problem with
our process. This is called troubleshooting. We can list several causes for the problem, change the
factors, isolate the problem, and thereby learn more about our system while fixing the problem.

5.5.1 An engineering example

Let’s look at an example. We expect that compounds A and B should combine in the presence of a
third chemical, C, to form a product D. An initial experiment shows very little of product D is
produced. Our goal is to maximize the amount of D. Several factors are considered: temperature,
reaction duration and pressure. Using a set of structured experiments, we can get an initial idea of
which factors actually impact the amount of D produced. Perhaps these experiments show that only
temperature and reaction duration are important and that pressure has little effect. Then we go ahead
and adjust only those two factors, and we keep pressure low (to save money because we can now use a
less costly, low-pressure reactor). We repeat several more systematic response surface (page 365)
experiments to maximize our production goal.

The iterations continue until we find the most economically profitable operating point. At each
iteration we learn more about our system and how to improve it. The key point is this: you must
disturb your system, and then observe it. This is the principle of causality, or cause and effect.

It is only by intentional manipulation of our systems that we learn from them. Collecting happenstance
data, (everyday) operating data, does not always help, because it is confounded by other events that
occur at the same time. Everyday, happenstance data is limited by feedback control systems.

5.5.2 Feedback control

Feedback control systems keep the region of operation to a small zone. Better yields or improved
operation might exist beyond the bounds created by our automatic control systems. Due to safety
concerns, and efficient manufacturing practices, we introduce automated feedback control systems to
prevent deviating too far from a desired region of operation. As a result, data collected from such
systems has low information quality.

An example would be making eggs for breakfast. If you make eggs the same way each morning (a bit
of butter, medium heat for 5 minutes, flip and cook it for 1 minute, then eat), you will never experience
anything different. The egg you make this morning is going to taste very similar to one last year,
because of your good control system. That’s happenstance data.

You must intentionally change the system to perturb it, and then observe it.

5.5.3 Another engineering example

Here’s a great example from the book by Box, Hunter and Hunter. Consider the negative-slope
relationship between pressure and yield (page 324): as pressure increases, the yield drops. A line could be
drawn through the points from the happenstance measurements, taken from the process at different
times in the past. That line could be from a least squares model (page 209). It is true that the observed
pressure and yield are correlated, as that is exactly what a least squares model is intended for: to
quantify correlation.

The true mechanism in this system is that pressure is increased to remove the frothing that occurs in
the reactor. Higher frothing occurs when there is an impurity in the raw material, so operators increase

5.5. Why learning about systems is important 323

Process Improvement Using Data

reactor pressure when they see frothing (i.e. high impurity). However, it is the high impurity that
actually causes the lower yield, not the pressure itself. These relationships between yield, pressure and
impurity levels are illustrated below, based an adaption from the book by Box, Hunter and Hunter,
Chapter 14 (1st edition) or Chapter 10 (2nd edition).

Pressure is correlated with the yield, but there is no cause-and-effect relationship between them. The
happenstance relationship only appears in the data because of the operating policy, causing them to be
correlated, but it is not cause and effect. That is why happenstance data cannot be relied on to imply
cause and effect. An experiment in which the pressure is changed from low to high, performed on the
same batch of raw materials (i.e. at constant impurity level), will quickly reveal that there is no causal
relationship between pressure and yield.

Another problem with using happenstance data is that they are not taken in random order.
Time-based effects, such as changes in the seasonal or daily temperatures, will affect a process. We are
all well aware of slow changes: fridges and stoves degrade over time, cars need periodic maintenance.
Even our human bodies follow this rule. If we do not randomize the order of experiments, we risk
inferring a causal relationship when none actually exists.

Designed experiments are the only way we can be sure that these correlated events are causal. You
often hear people repeat the (incomplete) phrase that “correlation does not imply causality”. That is
only half-true: the other half of the phrase is “correlation is a necessary, but not sufficient, condition for
causality”.

In summary, do not rely on anecdotal “evidence” from colleagues. Always question the system, and
always try to perturb the system intentionally. In practice you won’t always be allowed to move the
system too drastically, so at the end of this chapter we will discuss response surface methods (page 365)
and evolutionary operation (page 373), which can be implemented on-line in production processes.

Experiments are the most efficient way to extract information about a system, that is, the most
information in the fewest number of changes. So it is always worthwhile to experiment.

324 Chapter 5. Design and Analysis of Experiments

 Release 547b16

5.6 Experiments with a single variable at two levels

This is the simplest type of experiment. It involves an outcome variable, 𝑦, and one input variable, 𝑥.
The 𝑥-variable could be a continuous numeric one, such as temperature, or discrete on, such as yes/no,
on/off, A/B. This type of experiment could be used to answer questions such as the following:

• Has the reaction yield increased when using catalyst A or B?

• Does the concrete’s strength improve when adding a particular binder or not?

• Does the plastic’s stretchability improve when extruded at various temperatures (a low or high
temperature)?

We can perform several runs (experiments) at level A, and some runs at level B. These runs are
randomized (i.e. do not perform all the A runs, and then the B runs). We strive to hold all other
disturbance variables constant so we pick up only the A-to-B effect. Disturbances are any variables
that might affect 𝑦 but, for whatever reason, we don’t wish to quantify. If we cannot control the
disturbance, then at least we can use pairing (page 92) and blocking (page 362). Pairing is when there is
one factor in our experiment; blocking is when we have more than one factor.

5.6.1 Recap of group-to-group differences

We have already seen in the univariate statistics section (page 85) how to analyze this sort of data. We
first calculate a pooled variance, then a 𝑧-value, and finally a confidence interval based on this 𝑧.
Please refer back to that section to review the important assumptions we have to make to arrive at this
equation:

(𝑛𝐴 − 1)𝑠2𝐴 + (𝑛𝐵 − 1)𝑠2𝐵

𝑠2𝑃 =
𝑛𝐴 − 1 + 𝑛𝐵 − 1
(𝑥𝐵 − 𝑥𝐴 ) − (𝜇𝐵 − 𝜇𝐴 )
𝑧= √︂ (︁ )︁
𝑠2𝑃 𝑛1𝐴 + 𝑛1𝐵

√︂ −𝑐𝑡 ≤ 𝑧 ≤ 𝑐𝑡 √︂
(︁ )︁ (︁ )︁
1 1 1 1
(𝑥𝐵 − 𝑥𝐴 ) − 𝑐𝑡 × 𝑠2𝑃 𝑛𝐴 + 𝑛𝐵 ≤ 𝜇𝐵 − 𝜇𝐴 ≤ (𝑥𝐵 − 𝑥𝐴 ) + 𝑐𝑡 × 𝑠2𝑃 𝑛𝐴 + 𝑛𝐵

We consider the effect of changing from condition A to condition B to be a statistically significant effect
when this confidence interval does not span zero. However, the width of this interval and how
symmetrically it spans zero can cause us to come to a different, practical conclusion. In other words, we
override the narrow statistical conclusion based on the richer information we can infer from the width
of the confidence interval and the variance of the process.

5.6.2 Using linear least squares models

There’s another interesting way that you can analyze data from an A versus B set of tests and get the
identical result to the methods we showed in the section where we made group-to-group comparisons
(page 85). In this method, instead of using a 𝑡-test, we use a least squares model of the form:

𝑦𝑖 = 𝑏0 + 𝑔𝑑𝑖

where 𝑦𝑖 is the response variable 𝑑𝑖 is an indicator variable. For example, 𝑑𝑖 = 0 when using condition
A and 𝑑𝑖 = 1 for condition B. Build this linear model, and then examine the confidence interval for the
coefficient 𝑔. The following R function uses the 𝑦-values from experiments under condition A and the
values under condition B to calculate the least squares model:

5.6. Experiments with a single variable at two levels 325

Process Improvement Using Data

lm_difference <- function(groupA, groupB)

{
# Build a linear model with groupA = 0, and groupB = 1

y.A <- groupA[]

y.B <- groupB[]
x.A <- numeric(length(y.A))
x.B <- numeric(length(y.B)) + 1
y <- c(y.A, y.B)
x <- c(x.A, x.B)
x <- factor(x, levels=c("0", "1"), labels=c("A", "B"))

model <- lm(y ~ x)

return(list(summary(model), confint(model)))
}

brittle <- [Link]('[Link]

# We developed the "group_difference" function in the Univariate section

group_difference(brittle$TK104, brittle$TK107)
lm_difference(brittle$TK104, brittle$TK107)

Use this function in the same way you did in the carbon dioxide exercise in the univariate section
(page 115). For example, you will find when comparing TK104 and TK107 that 𝑧 = 1.4056 and the
confidence interval is −21.4 ≤ 𝜇107 − 𝜇104 ≤ 119. Similarly, when coding 𝑑𝑖 = 0 for reactor TK104 and
𝑑𝑖 = 1 for reactor TK107, we get the least squares confidence interval for parameter 𝑔: −21.4 ≤ 𝑔 ≤ 119.
This is a little surprising, because the first method creates a pooled variance and calculates a 𝑧-value
and then a confidence interval. The least squares method builds a linear model, and then calculates the
confidence interval using the model’s standard error.

Both methods give identical results, but by very different routes.

5.6.3 The importance of randomization

We emphasized in a previous section (page 85) that experiments must be performed in random order to
avoid any unmeasured, and uncontrolled, disturbances from impacting the system.

The concept of randomization was elegantly described in an example by Fisher in Chapter 2 of his
book, The Design of Experiments (page 321). A lady claims that she can taste the difference in a cup of
tea when the milk is added after the tea or when the tea is added after the milk. By setting up 𝑁 cups
of tea that contain either the milk first (M) or the tea first (T), the lady is asked to taste these 𝑁 cups
and make her assessment. Fisher shows that if the experiments are performed in random order, the
actual set of decisions made by the lady are just one of many possible outcomes. He calculates all
possibilities (we show how below), and then he calculates the probability of the lady’s actual set of
decisions being due to chance alone. If the lady has test score values better than by random chance,
then there is a reasonable claim the lady is reliable.

Let’s take a look at a more engineering-oriented example. We previously considered (page 115) the
brittleness of a material made in either TK104 or TK107. The same raw materials were charged to each
reactor. So, in effect, we are testing the difference due to using reactor TK104 or reactor TK107. Let’s
call them case A (TK104) and case B (TK107) so the notation is more general. We collected 20
brittleness values from TK104 and 23 values from TK107. We will only use the first 8 values from
TK104 and the first 9 values from TK107 (you will see why soon):

326 Chapter 5. Design and Analysis of Experiments

 Release 547b16

Case A 254 440 501 368 697 476 188 525

Case B 338 470 558 426 733 539 240 628 517

Fisher’s insight was to create one long vector of these outcomes (length of vector = 𝑛𝐴 + 𝑛𝐵 ) and
randomly assign “A” to 𝑛𝐴 of the values and “B” to 𝑛𝐵 of the values. One can show that there are
(𝑛𝐴 + 𝑛𝐵 )!
possible combinations. For example, if 𝑛𝐴 = 8 and 𝑛𝐵 = 9, then the number of unique
𝑛𝐴 !𝑛𝐵 !
ways to split these 17 experiments into two groups of 8 (A) and 9 (B) is 24,310 ways. For example, one
way is BABB ABBA ABAB BAAB, and you would therefore assign the experimental values
accordingly (B = 254, A = 440, B = 501, B = 368, A = 697, etc.).

Only one of the 24,310 sequences will correspond to the actual data printed in the above table.
Although all the other realizations are possible, they are fictitious. We do this because the null
hypothesis is that there is no difference between A and B. Values in the table could have come from
either system.

So for each of the 24,310 realizations, we calculate the difference of the averages between A and B,
𝑦 𝐴 − 𝑦 𝐵 , and plot a histogram of these differences. This is shown below, together with a vertical line
indicating the actual realization in the table. There are 4956 permutations that had a greater difference
than the one actually realized; that is, 79.6% of the other combinations had a smaller value.

Had we used a formal test of differences where we pooled the variances, we would have found a
𝑧-value of 0.8435, and the probability of obtaining that value, using the 𝑡-distribution with 𝑛𝐴 + 𝑛𝐵 − 2
degrees of freedom, would be 79.3%. See how close they agree?

The figure shows the differences in the averages of A and B for the 24,310 realizations. The vertical line
represents the difference in the average for the one particular set of numbers we measured in the
experiment.

Recall that independence is required to calculate the 𝑧-value for the average difference and compare it
against the 𝑡-distribution. By randomizing our experiments, we are able to guarantee that the results
we obtain from using 𝑡-distributions are appropriate. Without randomization, these 𝑧-values and
confidence intervals may be misleading.

The reason we prefer using the 𝑡-distribution approach over randomization is that formulating all
random combinations and then calculating all the average differences as shown here is intractable.

5.6. Experiments with a single variable at two levels 327

 Process Improvement Using Data

Even on my relatively snappy computer it would take 3.4 years to calculate all possible combinations
for the complete dataset: 20 values from group A and 23 values from group B. (It took 122 seconds to
calculate a million of them, so the full set of 960,566,918,220 combinations would take more than 3
years.)

5.7 Changing one single variable at a time (COST)

Video for
this section How do we go about running our experiments when there is more than one variable present that
affects our outcome, 𝑦? In this section we describe how not to do it.

You will certainly have seen the recommendation that we must change one single variable at a time
(COST):

• Something goes wrong with a recipe: for example, the pancakes are not as fluffy as normal, or the
muffins don’t rise as much as they should. You are convinced it is the new brand of all-purpose
flour you recently bought. You change only the flour the next time you make pancakes to check
your hypothesis.

• University labs are notorious for asking you to change one variable at a time. The reason is that
these labs intend for you to learn what the effect of a single variable is on some other variable (e.g.
change temperature in a distillation column to improve product purity). The labs teach you that this
is good scientific procedure, which is fine if your goal is to only initially learn about a system,
especially a new system that has never been explored.

However, when you want to optimize and improve a process, then a different style of experiments is
required, where multiple factors are changed simultaneously.

We have known since the mid-1930s (almost 85 years), due to the work by R. A. Fisher (page 321), that
changing one factor at a time (OFAT) is not an efficient way for experimentation. Note that OFAT is an
alternative name for COST, and an example of OFAT is illustrated in the figure.

Consider a bioreactor where we are producing a particular enzyme. The yield, our outcome variable,
is known to be affected by these six variables: dissolved oxygen level, agitation rate, reaction duration,
feed substrate concentration, substrate type and reactor temperature. For illustration purposes, let’s
assume that temperature and feed substrate concentration are chosen, as they have the greatest effect
on yield. The goal would be to maximize the yield.

The base operating point is 346 K with a feed substrate concentration of 1.5 g/L, marked with a circle
in the figure below. At these conditions, we report a yield from the reactor of approximately 63%.

328 Chapter 5. Design and Analysis of Experiments

 Release 547b16

At this point, we start to investigate the effect of temperature. We decide to move up by 10 degrees to
356 K, marked as point 1. After running the experiment, we record a lower yield value than our
starting point. So we go in the other direction and try temperatures at 338 K, 330 K and 322 K. We are
happy that the yields are increasing, but experiment 4 shows a slightly lower yield. So we figure that
we’ve reached a plateau in terms of the temperature variable. Our manager is pretty satisfied because
we’ve boosted yield from 63% to around 67%. These four runs have cost us around $10,000 in
equipment time and manpower costs so far.

We now get approval to run four more experiments, and we decide to change the substrate feed
concentration. But we’re going to do this at the best temperature found so far, 330 K, at run 3. Our
intuition tells us that higher feed concentrations should boost yield, so we try 1.75 g/L. Surprisingly,
that experiment lowers the yield. There’s likely something we don’t understand about the reaction
mechanism. Anyhow, we try the other direction, down to 1.25 g/L, and we see a yield increase. We
decide to keep going, down to 1.0 g/L, and finally to 0.75 g/L. We see very little change between these
last two runs, and we believe we have reached another plateau. Also, our budget of eight
experimental runs is exhausted.

Our final operating point chosen is marked on the plot with a hexagon, at 330 K and 0.75 g/L. We’re
proud of ourselves because we have boosted our yield from 63% to 67%, and then from 67% to 69.5%.
We have also learned something interesting about our process: the temperature appears to be
negatively correlated with yield, and the substrate concentration is negatively correlated with yield.
An unexpected observation!

The problem with this approach is that it leaves undiscovered values behind. Changing one single
variable at a time leads you into thinking you’ve reached the optimum, when all you’ve done in fact is
trap yourself at a suboptimal solution.

5.7. Changing one single variable at a time (COST) 329

 Process Improvement Using Data

Furthermore, notice that we would have got a completely different outcome had we decided to first
change substrate concentration, 𝑆, and then temperature, 𝑇 . We would have likely landed closer to the
optimum. This is very unsatisfactory: we cannot use methods to optimize our processes that depend
on the order of experiments!

We have not yet even considered the effect of the other four variables of dissolved oxygen level,
agitation rate, reaction duration and substrate type. We have suboptimally optimized the system in
two dimensions, but there are in fact six dimensions. Although the OFAT (or COST) approach can get
you close to the optimum in two variables, you have little to no hope of using this approach
successfully with multiple factors.

Designed experiments, on the other hand, provide an efficient mechanism to learn about a system,
often in fewer runs than the COST approach, and avoid misleading conclusions that might be drawn
from the COST approach. Designed experiments are always run in random order – as we will
presently see – and we will get the same result, no matter the order.

5.8 Full factorial designs

Video for
this section In this section we learn how, and why, we should change more than one variable at a time. We will use
factorial designs because

• We can visually interpret these designs, and see where to run future experiments;

• These designs require relatively few experiments; and

• They are often building blocks for more complex designs.

Most often we have two or more factors that affect our response variable, 𝑦. In this section we consider
the case when these factors are at two levels. Some examples would be to operate at low or high pH,
select long operating times or short operating times, use catalyst A or B and use mixing system A or B.
The general guidance is to choose the low and high values at the edges of normal operation. It is not
wise to use the minimum and maximum values that each factor could possibly have; they will likely
be too extreme. We will see an example of this in the section on saturated designs (page 357).

5.8.1 Using two levels for two or more factors

Let’s take a look at the mechanics of factorial designs by using our previous example where the
conversion, 𝑦, is affected by two factors: temperature, 𝑇 , and substrate concentration, 𝑆.

The range over which they will be varied is given in the table. This range was identified by the process
operators as being sufficient to actually show a difference in the conversion, but not so large as to move
the system to a totally different operating regime (that’s because we will fit a linear model to the data).

Factor Low level, − High level, +

Temperature, 𝑇 338 K 354 K
Substrate level, 𝑆 1.25 g/L 1.75 g/L

1. Write down the factors that will be varied: 𝑇 and 𝑆.

2. Write down the coded runs in standard order, also called Yates order, which alternates the sign of
the first variable the fastest and the last variable the slowest. By convention we start all runs at their
low levels and finish off with all factors at their high levels. There will be 2𝑘 runs, where 𝑘 is the
number of variables in the design and the 2 refers to the number of levels for each factor. In this

330 Chapter 5. Design and Analysis of Experiments

 Release 547b16

case, 22 = 4 experiments (runs). We perform the actual experiments in random order, but always
write the table in this standard order.

Experiment 𝑇 [K] 𝑆 [g/L]

1 − −
2 + −
3 − +
4 + +

3. Add an additional column to the table for the response variable. The response variable is a
quantitative value, 𝑦, which in this case is the conversion measured as a percentage.

Experiment Order 𝑇 [K] 𝑆 [g/L] 𝑦 [%]

1 3 − − 69
2 2 + − 60
3 4 − + 64
4 1 + + 53

Experiments were performed in random order; in this case, we happened to run experiment 4 first
and experiment 3 last.

4. For simple systems you can visualize the design and results as shown in the following figure
(page 331). This is known as a cube plot.

5.8.2 Analysis of a factorial design: main effects

The first step is to calculate the main effect of each variable. The effects are considered, by convention,
to be the difference from the high level to the low level. So the interpretation of a main effect is by how
much the outcome, 𝑦, is adjusted when changing the variable.
Video for
this section

5.8. Full factorial designs 331

Process Improvement Using Data

Consider the two runs where 𝑆 is at the − level for both experiments 1 and 2. The only change
between these two runs is the temperature, so the temperature effect is
Δ𝑇𝑆− = 60 − 69 = −9% per (354 − 338) K, that is, a −9% change in the conversion outcome per +16 K
change in the temperature.

Runs 3 and 4 both have 𝑆 at the + level. Again, the only change is in the temperature:
Δ𝑇𝑆+ = 53 − 64 = −11% per +16 K. So we now have two temperature effects, and the average of
them is a −10% change in conversion per +16 K change in temperature.

We can perform a similar calculation for the main effect of substrate concentration, 𝑆, by comparing
experiments 1 and 3: Δ𝑆𝑇 − = 64 − 69 = −5% per 0.5 g/L, while experiments 2 and 4 give
Δ𝑆𝑇 + = 53 − 60 = −7% per 0.5 g/L. So the average main effect for 𝑆 is a −6% change in conversion
for every 0.5 g/L change in substrate concentration. You should use the following graphical method
(page 332) when calculating main effects from a cube plot by hand.

This visual summary is a very effective method of seeing how the system responds to the two
variables. We can see the gradients in the system and the likely region where we can perform the next
experiments to improve the bioreactor’s conversion.

The following surface plot illustrates the true, but unknown, surface from which our measurements
are taken. Notice the slight curvature on the edges of each face. The main effects estimated above are a
linear approximation of the conversion over the region spanned by the factorial.

332 Chapter 5. Design and Analysis of Experiments

 Release 547b16

An interaction plot is an alternative way to visualize these main effects (page 333). Use this method when
you don’t have computer software to draw the surfaces. [We saw this earlier in the visualization section
(page 1)]. We will discuss interaction plots more in the next section. Here is an illustration of one such
plot for a system with little interaction.

5.8.3 Analysis of a factorial design: interaction effects

Video for
this section We expect in many real systems that the main effect of temperature, 𝑇 , for example, is different at
other levels of substrate concentration, 𝑆. It is quite plausible for a bioreactor system that the main
temperature effect on conversion is much greater if the substrate concentration, 𝑆, is also high, while at
low values of 𝑆, the temperature effect is smaller.

We call this result an interaction, when the effect of one factor is different at different levels of the other
factors. Let’s give a practical, everyday example: assume your hands are covered with dirt or oil. We
know that if you wash your hands with cold water, it will take longer to clean them than washing with
hot water. So let factor A be the temperature of the water; factor A has a significant effect on the time
taken to clean your hands.

Consider the case when washing your hands with cold water. If you use soap with cold water, it will
take less time to clean your hands than if you did not use soap. It is clear that factor B, the categorical
factor of using no soap vs some soap, will reduce the time to clean your hands.

Now consider the case when washing your hands with hot water. The time taken to clean your hands
with hot water when you use soap is greatly reduced, far faster than any other combination. We say

5.8. Full factorial designs 333

Process Improvement Using Data

there is an interaction between using soap and the temperature of the water. This is an example of an
interaction that works to help us reach the objective faster.

The effect of warm water enhances the effect of soap. Conversely, the effect is soap is enhanced by
using warm water. So symmetry means that if soap interacts with water temperature, then we also
know that water temperature interacts with soap.

In summary, interaction means the effect of one factor depends on the level of the other factor. In this
example, that implies the effect of soap is different, depending on if we use cold water or hot water.
Interactions are also symmetrical. The soap’s effect is enhanced by warm water, and the warm water’s
effect is enhanced by soap.

Let’s use a different system here to illustrate (page 334) interaction effects, but still using 𝑇 and 𝑆 as the
variables being changed and keeping the response variable, 𝑦, as the conversion, shown by the contour
lines.

Experiment 𝑇 [K] 𝑆 [g/L] 𝑦 [%]

1 − (390 K) − (0.5 g/L) 77
2 + (400 K) − (0.5 g/L) 79
3 − (390 K) + (1.25 g/L) 81
4 + (400 K) + (1.25 g/L) 89

The main effect of temperature for this system is

• Δ𝑇𝑆− = 79 − 77 = 2% per 10 K

• Δ𝑇𝑆+ = 89 − 81 = 8% per 10 K

which means that the average temperature main effect is 5% per 10 K.

Notice how different the main effect is at the low and high levels of 𝑆. So the average of the two is an
incomplete description of the system. There is some other aspect to the system that we have not
captured.

Similarly, the main effect of substrate concentration is

334 Chapter 5. Design and Analysis of Experiments

 Release 547b16

• Δ𝑆𝑇 − = 81 − 77 = 4% per 0.75 g/L

• Δ𝑆𝑇 − = 89 − 79 = 10% per 0.75 g/L

which gives the average substrate concentration main effect as 7% per 0.75 g/L.

The data may also be visualized using an interaction plot (page 335) here, showing a higher degree of
interaction.

The lack of parallel lines is a clear indication of interaction. The temperature effect is stronger at high
levels of 𝑆, and the effect of 𝑆 on conversion is also greater at high levels of temperature. What is
missing is an interaction term, given by the product of temperature and substrate. We represent this as
𝑇 × 𝑆 and call it the temperature-substrate interaction term.

This interaction term should be zero for systems with no interaction, which implies the lines are
parallel in the interaction plot. Such systems will have roughly the same effect of 𝑇 at both low and
high values of 𝑆 (and in between). So then, a good way to quantify interaction is by how different the
main effect terms are at the high and low levels of the other factor in the interaction. The interaction
must also be symmetrical: if 𝑇 interacts with 𝑆, then 𝑆 interacts with 𝑇 by the same amount.

We can quantify the interaction of our current example in this way. For the 𝑇 interaction with 𝑆:

• Change in conversion due to 𝑇 at high 𝑆: 89 − 81 = +8

• Change in conversion due to 𝑇 at low 𝑆: 79 − 77 = +2

• The half difference: [+8 − (+2)]/2 = 3

For the 𝑆 interaction with 𝑇 ,

• Change in conversion due to 𝑆 at high 𝑇 : 89 − 79 = +10

• Change in conversion due to 𝑆 at low 𝑇 : 81 − 77 = +4

• The half difference: [+10 − (+4)]/2 = 3

A large, positive interaction term indicates that temperature and substrate concentration will increase
conversion by a greater amount when both 𝑇 and 𝑆 are high. Similarly, these two terms will rapidly
reduce conversion when they both are low.

We will get an improved appreciation for interpreting main effects and the interaction effect when we
consider the analysis in the form of a linear, least squares model.

5.8. Full factorial designs 335

 Process Improvement Using Data

5.8.4 Analysis by least squares modelling

Video for
this section Let’s review the original system (the one with little interaction) (page 330) and analyze the experimental
data using a least squares model. We represent the original data here, with the baseline conditions:

Experiment 𝑇 [K] 𝑆 [g/L] 𝑦 [%]

Baseline 346 K 1.50
1 − (338 K) − (1.25 g/L) 69
2 + (354 K) − (1.25 g/L) 60
3 − (338 K) + (1.75 g/L) 64
4 + (354 K) + (1.75 g/L) 53

It is standard practice to represent the data from designed experiments in a centered and scaled form:
variable − center point
. This gives the following values:
range/2
338 − 346 −8
• 𝑇− = = = −1
(354 − 338)/2 8
1.25 − 1.50 −0.25
• 𝑆− = = = −1
(1.75 − 1.25)/2 0.25
Similarly, 𝑇+ = +1 and 𝑆+ = +1, while the center points (baseline experiment) would be 𝑇0 = 0 and
𝑆0 = 0.

We will propose a least squares model that describes this system:

Population model : 𝑦 = 𝛽 0 + 𝛽 𝑇 𝑥 𝑇 + 𝛽𝑆 𝑥 𝑆 + 𝛽𝑇 𝑆 𝑥 𝑇 𝑥 𝑆 + 𝜀
Sample model : 𝑦 = 𝑏0 + 𝑏𝑇 𝑥 𝑇 + 𝑏𝑆 𝑥 𝑆 + 𝑏𝑇 𝑆 𝑥 𝑇 𝑥 𝑆 + 𝑒

We have four parameters to estimate and four data points. This means when we fit the model to the
data, we will have no residual error, because there are no degrees of freedom left. If we had replicate
experiments, we would have degrees of freedom to estimate the error, but more on that later. Writing
the above equation for each observation,
⎡ ⎤ ⎡ ⎤⎡ ⎤ ⎡ ⎤
𝑦1 1 𝑇− 𝑆− 𝑇− 𝑆− 𝑏0 𝑒1
⎢𝑦 ⎥ ⎢1 𝑇 𝑆− 𝑇+ 𝑆− ⎥ ⎢ 𝑏𝑇 ⎥ ⎢𝑒2 ⎥
⎥ ⎢ ⎥ ⎢
⎢ 2⎥ ⎢ +
⎢ ⎥=⎢ ⎥+⎢ ⎥
⎥
⎥⎢
⎣𝑦3 ⎦ ⎣1 𝑇− 𝑆+ 𝑇− 𝑆+ ⎦ ⎣ 𝑏𝑆 ⎦ ⎣𝑒3 ⎦
𝑦4 1 𝑇+ 𝑆+ 𝑇+ 𝑆+ 𝑏𝑇 𝑆 𝑒4
⎡ ⎤ ⎡ ⎤⎡ ⎤ ⎡ ⎤
69 1 −1 −1 +1 𝑏0 𝑒1
⎢60⎥ ⎢1 +1 −1 −1⎥ ⎢ 𝑏𝑇 ⎥ ⎢𝑒2 ⎥
⎥ ⎢ ⎥ ⎢
⎢ ⎥=⎢ ⎥+⎢ ⎥
⎢ ⎥ ⎢ ⎥
⎣64⎦ ⎣1 −1 −1⎦ ⎣ 𝑏𝑆 ⎦ ⎣𝑒3 ⎦
⎥⎢
+1
53 1 +1 +1 +1 𝑏𝑇 𝑆 𝑒4
y = Xb + e

Where the last line is a more compact representation. Notice then that the matrices from linear regression

336 Chapter 5. Design and Analysis of Experiments

 Release 547b16

(page 253) are:

⎡ ⎤
4 0 0 0
⎢0 4 0 0⎥
X𝑇 X = ⎢
⎢ ⎥
⎥
⎣0 0 4 0⎦
0 0 0 4
⎡ ⎤
246
⎢−20⎥
X𝑇 y = ⎢
⎢ ⎥
⎣−12⎦
⎥

−2
⎡ ⎤⎡ ⎤ ⎡ ⎤
1/4 0 0 0 246 61.5
⎥ ⎢−20⎥ ⎢ −5 ⎥
⎢ 0 1/4 0 0 ⎥ ⎢ ⎥ ⎢ ⎥
b = (X𝑇 X)−1 X𝑇 y = ⎢ ⎥=⎢
⎢
0 ⎦ ⎣−12⎦ ⎣ −3 ⎦
⎥⎢ ⎥
⎣ 0 0 1/4
0 0 0 1/4 −2 −0.5

Some things to note are (1) the orthogonality of X𝑇 X and (2) the interpretation of these coefficients.
Video for
this section 1. Note how the X𝑇 X matrix has only zeros on the off-diagonals: it indicates that matrix X is
orthogonal and confirms, algebraically, what we knew intuitively. The change we made in
temperature, 𝑇 , was independent of the changes we made in substrate concentration, 𝑆. This means
that we can separately calculate and interpret the slope coefficients in the model.

2. What is the interpretation of, for example, 𝑏𝑇 = −5? Recall, from the section on linear regression
interpretation (page 256), that it is the effect of increasing the temperature by 1 unit. In this case, the
𝑥𝑇 variable has been normalized, but this slope coefficient represents the effect of changing 𝑥𝑇 from
0 to 1, which in the original units of the variables is a change from 346 to 354 K, that is, an 8 K
increase in temperature. It equally well represents the effect of changing 𝑥𝑇 from −1 to 0: a change
from 338 K to 346 K decreases conversion by 5%.

Similarly, the slope coefficient for 𝑏𝑆 = −3 represents the expected decrease in conversion when 𝑆 is
increased from 1.50 g/L to 1.75 g/L.

Now contrast these numbers with those in the graphical analysis done previously (page 331) and
repeated below. They are the same, as long as we are careful to interpret them as the change over
half the range.

The 61.5 term in the least squares model is the expected conversion at the baseline conditions.

5.8. Full factorial designs 337

 Process Improvement Using Data

Notice from the least squares equations how it is just the average of the four experimental values,
even though we did not actually perform an experiment at the center.

Let’s return to the system with high interaction (page 333) where the four outcome values in standard
order were 77, 79, 81 and 89. Looking back, the baseline operation was 𝑇 = 395 K and 𝑆 = 1.25−0.5
2 =
0.875 g/L; you should prove to yourself that the least squares model is

𝑦 = 81.5 + 2.5𝑥𝑇 + 3.5𝑥𝑆 + 1.5𝑥𝑇 𝑥𝑆

The interaction term can now be readily interpreted: it is the additional increase in conversion seen
when both temperature and substrate concentration are at their high level. If 𝑇 is at the high level and
𝑆 is at the low level, then the least squares model shows that conversion is expected at
81.5 + 2.5 − 3.5 − 1.5 = 79. The interaction term has decreased conversion by 1.5 units.

Finally, out of interest, the nonlinear surface that was used to generate the experimental data for the
interacting system is coloured in the illustration. In practice we never know what this surface looks
like, but we estimate it with the least squares plane, which appears below the nonlinear surface as
black and white grids. The corners of the box are outer levels at which we ran the factorial
experiments.

The corner points are exact with the nonlinear surface, because we have used the four values to
estimate four model parameters. There are no degrees of freedom left, and the model’s residuals are
therefore zero. Obviously, the linear model will be less accurate away from the corner points when the
true system is nonlinear, but it is a useful model over the region in which we will use it later in the
section on response surface methods (page 365).

5.8.5 Example: design and analysis of a three-factor experiment

Video for
this section This example should be done by yourself. It is based on Question 19 in the exercises for Chapter 5 in
Box, Hunter and Hunter (2nd edition).

The data are from a plastics molding factory that must treat its waste before discharge. The 𝑦-variable
represents the average amount of pollutant discharged (lb per day), while the three factors that were
varied were

• 𝐶 = the chemical compound added (choose either chemical P or chemical Q)

338 Chapter 5. Design and Analysis of Experiments

 Release 547b16

• 𝑇 = the treatment temperature (72 °F or 100 °F)

• 𝑆 = the stirring speed (200 rpm or 400 rpm)

• 𝑦 = the amount of pollutant discharged (lb per day)

Experiment Order 𝐶 𝑇 [°F] 𝑆 [rpm] 𝑦 [lb]

1 5 Choice P 72 200 5
2 6 Choice Q 72 200 30
3 1 Choice P 100 200 6
4 4 Choice Q 100 200 33
5 2 Choice P 72 400 4
6 7 Choice Q 72 400 3
7 3 Choice P 100 400 5
8 8 Choice Q 100 400 4

1. Draw a geometric figure that illustrates the data from this experiment.

2. Calculate the main effect for each factor by hand.

For the C effect, there are four estimates of 𝐶:

(+25) + (+27) + (−1) + (−1) 50
= = 12.5
4 4
For the T effect, there are four estimates of 𝑇 :
(+1) + (+3) + (+1) + (+1) 6
= = 1.5
4 4
For the S effect, there are four estimates of 𝑆:
(−27) + (−1) + (−29) + (−1) −58
= = −14.5
4 4

3. Calculate the 3 two-factor interactions (2fi) by hand, recalling that interactions are defined as the
half difference going from high to low.

For the CT interaction, there are two estimates of 𝐶𝑇 . Recall that interactions are calculated as
the half difference going from high to low. Consider the change in 𝐶 when

• 𝑇high (at 𝑆 high) = 4 − 5 = −1

• 𝑇low (at 𝑆 high) = 3 − 4 = −1

This gives a first estimate of [(−1) − (−1)]/2 = 0. Similarly,

• 𝑇high (at 𝑆 low) = 33 − 6 = +27

• 𝑇low (at 𝑆 low) = 30 − 5 = +25

gives a second estimate of [(+27) − (+25)]/2 = +1.

The average CT interaction is therefore (0 + 1)/2 = 0.5. You can interchange 𝐶 and 𝑇 and still
get the same result.

For the CS interaction, there are two estimates of 𝐶𝑆. Consider the change in 𝐶 when

• 𝑆high (at 𝑇 high) = 4 − 5 = −1

• 𝑆low (at 𝑇 high) = 33 − 6 = +27

5.8. Full factorial designs 339

 Process Improvement Using Data

This gives a first estimate of [(−1) − (+27)]/2 = −14. Similarly,

• 𝑆high (at 𝑇 low) = 3 − 4 = −1

• 𝑆low (at 𝑇 low) = 30 − 5 = +25

gives a second estimate of [(−1) − (+25)]/2 = −13.

The average CS interaction is therefore (−13 − 14)/2 = −13.5. You can interchange 𝐶 and 𝑆
and still get the same result.

For the ST interaction, there are two estimates of 𝑆𝑇 : (−1 + 0)/2 = −0.5. Calculate in the
same way as above.

4. Calculate the single three-factor interaction (3fi).

There is only a single estimate of 𝐶𝑇 𝑆. The 𝐶𝑇 effect at high 𝑆 is 0, and the 𝐶𝑇 effect at low 𝑆
is +1. The 𝐶𝑇 𝑆 interaction is then [(0) − (+1)]/2 = −0.5.

You can also calculate this by considering the 𝐶𝑆 effect at the two levels of 𝑇 , or by
considering the 𝑆𝑇 effect at the two levels of 𝐶. All three approaches give the same result.

5. Compute the main effects and interactions using matrix algebra and a least squares model.
Video for
this section
⎡ ⎤ ⎡ ⎤⎡ ⎤
5 +1 −1 −1 −1 +1 +1 +1 −1 𝑏0
⎢30⎥ ⎢+1 +1
⎢ ⎥ ⎢ −1 −1 −1 −1 +1 +1⎥⎥ ⎢ 𝑏𝐶 ⎥
⎢ ⎥
⎢ 6 ⎥ ⎢+1 −1 +1 −1 −1 +1 −1 +1⎥ ⎢ 𝑏𝑇 ⎥
⎢ ⎥ ⎢ ⎥⎢ ⎥
⎢ ⎥ ⎢ ⎥⎢ ⎥
⎢33⎥ ⎢+1 +1
⎢ ⎥=⎢ +1 −1 +1 −1 −1 −1⎥⎥ ⎢ 𝑏𝑆 ⎥
⎢ ⎥
⎢ 4 ⎥ ⎢+1 −1 −1 +1 +1 −1 −1 +1⎥ ⎢ 𝑏𝐶𝑇 ⎥
⎥ ⎢
⎢ ⎥ ⎢ ⎥
⎢ 3 ⎥ ⎢+1 +1 −1 +1 −1 +1 −1 −1⎥ ⎢ 𝑏𝐶𝑆 ⎥
⎢ ⎥ ⎢ ⎥⎢ ⎥
⎢ ⎥ ⎢ ⎥⎢ ⎥
⎣ 5 ⎦ ⎣+1 −1 +1 +1 −1 −1 +1 −1⎦ ⎣ 𝑏𝑇 𝑆 ⎦
4 +1 +1 +1 +1 +1 +1 +1 +1 𝑏𝐶𝑇 𝑆
y = Xb

6. Use computer software to build the following model and verify that:

𝑦 = 11.25 + 6.25𝑥𝐶 + 0.75𝑥𝑇 − 7.25𝑥𝑆 + 0.25𝑥𝐶 𝑥𝑇 − 6.75𝑥𝐶 𝑥𝑆 − 0.25𝑥𝑇 𝑥𝑆 − 0.25𝑥𝐶 𝑥𝑇 𝑥𝑆

Learning notes:

• The chemical compound could be coded either as (chemical P = −1, chemical Q = +1) or (chemical P
= +1, chemical Q = −1). The interpretation of the 𝑥𝐶 coefficient is the same, regardless of the coding.

• Just the tabulation of the raw data gives us some interpretation of the results. Why? Since the
variables are manipulated independently, we can just look at the relationship of each factor to 𝑦,
without considering the others. It is expected that the chemical compound and speed have a strong
effect on 𝑦, but we can also see the chemical × speed interaction. You can see this last interpretation
by writing out the full X design matrix and comparing the bold column, associated with the 𝑏CS
term, with the 𝑦 column.

A note about magnitude of effects

In this text we quantify the effect as the change in response over half the range of the factor. For
example, if the center point is 400 K, the lower level is 375 K and the upper level is 425 K, then an effect
of "-5" represents a reduction in 𝑦 of 5 units for every increase of 25 K in 𝑥.

340 Chapter 5. Design and Analysis of Experiments

 Release 547b16

We use this representation because it corresponds with the results calculated from least-squares
software. Putting the matrix of −1 and +1 entries into the software as X, along with the corresponding
(︀ )︀−1
vector of responses, 𝑦, you can calculate these effects as b = X𝑇 X Xy.

Other textbooks, specifically Box, Hunter and Hunter, will report effects that are double ours. This is
because they consider the effect to be the change from the lower level to the upper level (double the
distance). The advantage of their representation is that binary factors (catalyst A or B; agitator on or
off) can be readily interpreted, whereas in our notation, the effect is a little harder to describe (simply
double it!).

The advantage of our methodology, though, is that the results calculated by hand would be the same
as those from any computer software with respect to the magnitude of the coefficients and the
standard errors, particularly in the case of duplicate runs and experiments with center points.

Remember: our effects are half those reported in Box, Hunter and Hunter, and in some other
textbooks; our standard error would also be half of theirs. The conclusions drawn will always be the
same, as long as one is consistent.

5.8.6 Assessing significance of main effects and interactions

When there are no replicate points, then the number of factors to estimate from a full factorial is 2𝑘
from the 2𝑘 observations. There are no degrees of freedom left to calculate the standard error or the
confidence intervals for the main effects and interaction terms.

The standard error can be estimated if complete replicates are available. However, a complete replicate
is onerous, because a complete replicate implies the entire experiment is repeated: system setup,
running the experiment and measuring the result. Taking two samples from one actual experiment
and measuring 𝑦 twice is not a true replicate. That is only an estimate of the measurement error and
analytical error.

Furthermore, there are better ways to spend our experimental budget than running complete replicate
experiments – see the section on screening designs (page 357) later on. Only later in the overall
experimental procedure should we run replicate experiments as a verification step and to assess the
statistical significance of effects.

There are two main ways we can determine if a main effect or interaction is significant: by using a
Pareto plot or the standard error.

Pareto plot

ò Note

This is a makeshift approach that is only applicable if all the factors are centered and scaled.

A full factorial with 2𝑘 experiments has 2𝑘 parameters to estimate. Once these parameters have been
calculated, for example, by using a least squares model (page 336), then plot as shown the absolute value
of the model coefficients in sorted order, from largest magnitude to smallest, ignoring the intercept
term. Significant coefficients are established by visual judgement – establishing a visual cutoff by
contrasting the small coefficients to the larger ones.

5.8. Full factorial designs 341

Process Improvement Using Data

The example shown in the bar graph was from a full factorial experiment where the results for 𝑦 in
standard order were 𝑦 = [45, 71, 48, 65, 68, 60, 80, 65, 43, 100, 45, 104, 75, 86, 70, 96].

We would interpret that factors A, C and D, as well as the interactions of AC and AD, have a
significant and causal effect on the response variable, 𝑦. The main effect of B on the response 𝑦 is small,
at least over the range that B was used in the experiment. Factor B can be omitted from future
experimentation in this region, though it might be necessary to include it again if the system is
operated at a very different point.

The reason why we can compare the coefficients this way, which is not normally the case with least
squares models, is that we have both centered and scaled the factor variables. If the centering is at
typical baseline operation, and the range spanned by each factor is that expected over the typical
operating range, then we can fairly compare each coefficient in the bar plot. Each bar represents the
influence of that term on 𝑦 for a one-unit change in the factor, that is, a change over half its operating
range.

Obviously, if the factors are not scaled appropriately, then this method will be error prone. However,
the approximate guidance is accurate, especially when you do not have a computer or if additional
information required by the other methods (discussed below) is not available. It is also the only way to
estimate the effects for highly fractionated and saturated designs (page 357).

Standard error: from replicate runs or from an external dataset

ò Note

It is often better to spend your experimental budget screening for additional factors rather than
replicating experiments.

If there are more experiments than parameters to be estimated, then we have extra degrees of freedom.
Having degrees of freedom implies we can calculate the standard error, 𝑆𝐸 . Once 𝑆𝐸 has been found,
we can also calculate the standard error for each model coefficient, and then confidence intervals can
be constructed for each main effect and interaction. And because the model matrix is orthogonal, the
confidence interval for each effect is independent of the other. This is because the general confidence
(︀ )︀−1 2
interval is 𝒱 (b) = X𝑇 X 𝑆𝐸 , and the off-diagonal elements in X𝑇 X are zero.

For an experiment with 𝑛 runs, and where we have coded our X matrix to contain −1 and +1

342 Chapter 5. Design and Analysis of Experiments

 Release 547b16

elements, and when √︃

the X matrix is orthogonal, the standard error for coefficient 𝑏𝑖 is
√︀ 𝑆2
𝑆𝐸 (𝑏𝑖 ) = 𝒱 (𝑏𝑖 ) = ∑︀ 𝐸 2 . Some examples:
𝑥𝑖

• A 23 factorial where every combination has been repeated √︂ will have 𝑛 = 16 runs, so the standard
2
𝑆𝐸 𝑆𝐸
error for each coefficient will be the same, at 𝑆𝐸 (𝑏𝑖 ) = = .
16 4
• A 23 factorial with three additional runs at the center point would have the following least squares
representation:

y = Xb + e
⎡ ⎤ ⎡ ⎤ ⎡ ⎤
𝑦1 1 𝐴− 𝐵− 𝐶− 𝐴− 𝐵 − 𝐴− 𝐶 − 𝐵− 𝐶− 𝐴− 𝐵− 𝐶− 𝑒1
⎢ 𝑦 ⎥ ⎢1 𝐴 𝐵− 𝐶− 𝐴+ 𝐵 − 𝐴+ 𝐶− 𝐵− 𝐶− 𝐴+ 𝐵− 𝐶− ⎥ ⎥ ⎡ 𝑏 ⎤ ⎢ 𝑒2 ⎥
⎢ ⎥
⎢ 2 ⎥ ⎢ +
0
⎢ 𝑦3 ⎥ ⎢1 𝐴− 𝐵+ 𝐶− 𝐴− 𝐵+ 𝐴− 𝐶 − 𝐵+ 𝐶 − 𝐴− 𝐵+ 𝐶− ⎥ ⎢ ⎢ 𝑒3 ⎥
⎢ ⎥ ⎢ ⎥ ⎢ ⎥
⎢ ⎥ ⎢ ⎥ ⎢ 𝑏𝐴 ⎥⎥ ⎢𝑒 ⎥
⎢
⎢ 𝑦4 ⎥ ⎢1 𝐴+ 𝐵+ 𝐶− 𝐴+ 𝐵 + 𝐴+ 𝐶− 𝐵+ 𝐶 − 𝐴+ 𝐵 + 𝐶 − ⎥ 4 ⎥
⎥⎢⎢ 𝑏𝐵 ⎥ ⎢
⎢ ⎥ ⎢ ⎥ ⎥
⎢ 𝑦 ⎥ ⎢1 𝐴 𝐵− 𝐶+ 𝐴− 𝐵 − 𝐴− 𝐶 + 𝐵− 𝐶+ 𝐴− 𝐵− 𝐶+ ⎥ ⎥ ⎢ 𝑒5 ⎥
⎢ 5⎥ ⎢ − ⎢ ⎥
⎥ ⎢ 𝑏𝐶 ⎥
⎢ ⎥
⎢ 𝑦6 ⎥ = ⎢1 𝐴+ 𝐵− 𝐶+ 𝐴+ 𝐵 − 𝐴+ 𝐶 + 𝐵− 𝐶+ 𝐴+ 𝐵− 𝐶+ ⎥ ⎢ ⎥+⎢ 𝑒
⎢ ⎥ ⎢ ⎥
⎢ 6 ⎥
⎢ ⎥ ⎢ ⎥⎢ 𝑏𝐴𝐵 ⎥ ⎢ ⎥
⎢ 𝑦7 ⎥ ⎢1 𝐴− 𝐵+ 𝐶+ 𝐴− 𝐵 + 𝐴− 𝐶 + 𝐵+ 𝐶+ 𝐴− 𝐵+ 𝐶+ ⎥ ⎢
⎥ ⎢
⎥ ⎢ 𝑒7 ⎥
⎥ ⎢
⎢ ⎥ ⎢
⎢ 𝑦 ⎥ ⎢1 𝐴 ⎢ 𝑏𝐴𝐶 ⎥ ⎥
⎢ 8⎥ ⎢ + 𝐵+ 𝐶+ 𝐴+ 𝐵 + 𝐴+ 𝐶 + 𝐵+ 𝐶+ 𝐴+ 𝐵+ 𝐶+ ⎥ ⎥⎢ ⎥ ⎢ 𝑒8 ⎥
⎥ 𝑏𝐵𝐶
⎣ ⎦ ⎢ ⎥
𝑦 1 0 0 0 0 0 0 0 ⎢𝑒𝑐,1 ⎥
⎢ ⎥ ⎢ ⎢ ⎥
⎢ 𝑐,1 ⎥ ⎢ ⎥
⎢ ⎥ ⎢ ⎥ 𝑏𝐴𝐵𝐶 ⎢ ⎥
⎣𝑦𝑐,2 ⎦ ⎣1 0 0 0 0 0 0 0 ⎦ ⎣𝑒𝑐,2 ⎦
𝑦𝑐,3 1 0 0 0 0 0 0 0 𝑒𝑐,3

And substituting in the values, using vector shortcut notation for y and e:
⎡ ⎤
1 −1 −1 −1 +1 +1 +1 −1
⎢1 +1 −1 −1 −1 −1 +1 +1⎥
⎢ ⎥⎡ 𝑏 ⎤
0
⎢1 −1 +1 −1 −1 +1 −1 +1⎥ ⎢
⎢ ⎥
⎢ ⎥ ⎢ 𝑏𝐴 ⎥
⎢1 +1 +1 −1 +1 −1 −1 −1⎥ ⎥
⎥⎢ 𝑏
⎥
𝐵 ⎥
⎢
⎢1 −1 −1 +1 +1 −1 −1 +1⎥
⎢
⎥⎢ ⎥
⎥ ⎢ 𝑏𝐶 ⎥
⎢
y = ⎢1 +1 −1 +1 −1 +1 −1 −1⎥ ⎢ ⎥+e
⎢
⎢ ⎥⎢ 𝑏𝐴𝐵 ⎥
⎢1 −1 +1 +1 −1 −1 +1 −1⎥ ⎢
⎥ ⎢ ⎥
⎢ 𝑏𝐴𝐶 ⎥
⎢ ⎥
⎢1 +1 +1 +1 +1 +1 +1 +1⎥ ⎢
⎥ ⎥
⎥ ⎣ 𝑏𝐵𝐶 ⎦
⎢
⎢1 0 0 0 0 0 0 0⎥
⎢
⎢ ⎥ 𝑏𝐴𝐵𝐶
⎣1 0 0 0 0 0 0 0⎦
1 0 0 0 0 0 0 0

Note that the center point runs do not change the orthogonality of X (verify this by writing out and
computing the X𝑇 X matrix and observing that all off-diagonal entries are zeros). However, as we
expect after having studied the section on least squares modelling (page 209), additional runs decrease
the variance of the model parameters, 𝒱(b). In this case, there are 𝑛 = 23 + 3 = 11 runs, so the
2 e𝑇 e
standard error is decreased to 𝑆𝐸 = . However, the center points do not further reduce the
√︃ 11 − 8
𝑆2
variance of the parameters in ∑︀ 𝐸 2 , because the denominator is still 2𝑘 (except for the intercept
𝑥𝑖
term, whose variance is reduced by the center points).

Once we obtain the standard error for our system and calculate the variance of the parameters, we can
multiply it by the critical 𝑡-value at the desired confidence level in order to calculate the confidence
limit. However, it is customary to just report the standard error next to the coefficients, so that users
can apply their own level of confidence. For example,

5.8. Full factorial designs 343

Process Improvement Using Data

Temperature effect, 𝑏𝑇 = 11.5 ± 0.707

Catalyst effect, 𝑏𝐾 = 1.1 ± 0.707

Even though the confidence interval of the temperature effect would be

11.5 − 𝑐𝑡 × 0.707 ≤ 𝛽𝑇 ≤ 11.5 + 𝑐𝑡 × 0.707, it is clear that at the 95% significance level, the above
representation shows the temperature effect is significant, while the catalyst effect is not (𝑐𝑡 ≈ 2).

Refitting the model after removing nonsignificant effects

After having established which effects are significant, we can exclude the nonsignificant effects and
increase the degrees of freedom. (We do not have to recalculate the model parameters – why?) The
residuals will be nonzero now, so we can then estimate the standard error and apply all the tools from
least squares modelling to assess the residuals. Plots of the residuals in experimental order, against
fitted values, q-q plots and all the other assessment tools from earlier are used, as usual.

Continuing the above example, where a 24 factorial was run, the response values in standard order
were 𝑦 = [71, 61, 90, 82, 68, 61, 87, 80, 61, 50, 89, 83, 59, 51, 85, 78]. The significant effects were from A, B,
D and BD. Now, omitting the nonsignificant effects, there are only five parameters to estimate,
2 39
including the intercept, so the standard error is 𝑆𝐸 = = 3.54, with 11 degrees of freedom. The
√︂− 5
16
2
𝑆𝐸
𝑆𝐸 (𝑏𝑖 ) value for all coefficients, except the intercept, is = 0.471, and the critical 𝑡-value at the
16
95% level is qt(0.975, df=11) = 2.2. So the confidence intervals can be calculated to confirm that
these are indeed significant effects.

There is some circular reasoning here: postulate that one or more effects are zero and increase the
degrees of freedom by removing those parameters in order to confirm the remaining effects are
significant. Some general advice is to first exclude effects that are definitely small, and then retain
medium-size effects in the model until you can confirm they are not significant.

Variance of estimates from the COST approach versus the factorial approach

Finally, we end this section on factorials by illustrating their efficiency. Contrast the two cases: COST
and the full factorial approach. For this analysis we define the main effect simply as the difference
between the high and low values (normally we divide through by 2, but the results still hold). Define
the variance of the measured 𝑦 value as 𝜎𝑦2 .

344 Chapter 5. Design and Analysis of Experiments

 Release 547b16

COST approach Fractional factorial approach

The main effect of 𝑇 is 𝑏𝑇 = 𝑦2 − 𝑦1 . The main effect is 𝑏𝑇 = 0.5(𝑦2 − 𝑦1 ) + 0.5(𝑦4 − 𝑦3 ).
The variance is 𝒱(𝑏𝑇 ) = 𝜎𝑦2 + 𝜎𝑦2 . The variance is 𝒱(𝑏𝑇 ) = 0.25(𝜎𝑦2 + 𝜎𝑦2 ) + 0.25(𝜎𝑦2 + 𝜎𝑦2 ).
So 𝒱(𝑏𝑇 ) = 2𝜎𝑦2 . And 𝒱(𝑏𝑇 ) = 𝜎𝑦2 .

Not only does the factorial experiment estimate the effects with much greater precision (lower
variance), but the COST approach cannot estimate the effect of interactions, which is incredibly
important, especially as systems approach optima that are on ridges (see the contour plots earlier in
this section for an example).

Factorial designs make each experimental observation work twice.

5.8.7 Summary so far

• The factorial experimental design is intentionally constructed so that each factor is independent of
the others. There are 2𝑘 experiments for 𝑘 factors.

– This implies the X𝑇 X matrix is easily constructed (a diagonal matrix, with a value of 2𝑘 for each
diagonal entry).

– These coefficients have the lowest variability possible: (X𝑇 X)−1 𝑆𝐸

2
.

– We have uncorrelated estimates of the slope coefficients in the model. That is, we can be sure the
value of the coefficient is unrelated to the other values.

• However, we still need to take the usual care in interpreting the coefficients. The usual precaution,
using the example below, is that the temperature coefficient 𝑏𝑇 is the effect of a one-degree change,
holding all other variables constant. That’s not possible if 𝑏𝑇 𝑆 , the interaction between 𝑇 and 𝑆, is
significant: we cannot hold the 𝑇 𝑆 constant while changing 𝑏𝑇 .

𝑦 = 𝑏0 + 𝑏𝑇 𝑥𝑇 + 𝑏𝑆 𝑥𝑆 + 𝑏𝑇 𝑆 𝑥𝑇 𝑥𝑆 + 𝑒

We cannot interpret the main effects separately from the interaction effects when we have significant
interaction terms in the model. Also, if you conclude the interaction term is significant, then you
must also include all main factors that make up that interaction term in the model.

For another example, with an interpretation, please see Box, Hunter and Hunter (2nd edition), page
185.

• Factorial designs use the collected data much more efficiently than one-at-a-time experimentation.
As shown in the preceding section (page 344), the estimated variance is halved when using a factorial
design compared to a COST approach.

• A small or zero effect from an 𝑥 variable to the 𝑦 response variable implies the 𝑦 is insensitive to that
𝑥. This is desirable in some situations. It means we can adjust that 𝑥 without affecting 𝑦, sometimes
stated as “the 𝑦 is robust to changes in 𝑥”.

5.8.8 Example: analysis of systems with 4 factors

In the prior sections you have seen how to analyze experiments with 2 factors and 3 factors. The logic
to analyze systems with 4 or more factors proceeds in exactly the same way. The video here shows
how to go about this.
Video for
this section

5.8. Full factorial designs 345

 Process Improvement Using Data

5.9 Fractional factorial designs

When there are many factors that we have identified as being potentially important, then the 2𝑘 runs
required for a full factorial can quickly become large and too costly to implement.

For example, you are responsible for a cell-culture bioreactor at a pharmaceutical company and there
is a drive to minimize the production of an inhibiting by-product. Variables thought to be related with
this by-product are: two types of T = temperature profile (𝑇− : a slow ramp over time or 𝑇+ a fast
initial ramp then constant temperature), the D = dissolved oxygen at a low and high level, the A =
agitation rate at a slow and faster speed, P = pH at a low and high level, and two blends of S =
substrate type. These five factors, at two levels, require 25 = 32 runs. It would take almost a year to
collect the data at all the combinations of T, D, A, P and S if each experiment requires 10 days (typical
in this industry), and if parallel reactors are not available.

Furthermore, we are probably interested in only the 5 main effects and 10 two-factor interactions (2fi).
The remaining 10 three-factor interactions, 5 four-factor interactions, a single five-factor interaction are
likely not too important either, at least initially. A full factorial would estimate 32 effects, even if we
likely only interested in at most 16 of them (5 main effects + 10 of the 2fi’s + 1 intercept).

Running a half fraction, or quarter fraction, of the full set will allow us to estimate the main effects and
two-factor interactions (2fi) in many cases, at the expense of confounding the higher interactions. [We
will explain exactly what is meant by that term confounding later on, but for now you can interpret it is
as ‘confused with’].

For many real systems it is the main effects that are mostly of interest, and at most two-factor
interactions. Very very seldom do three-factor interactions occur, nor are they often of practical
significance. As such, we are willing to allow some confounding (confusing) with these factors.

So let’s move into this section where we show how to construct and analyze these fractional factorials,
which are tremendously useful when screening many variables - especially for a first-pass at
experimenting on a new system. They are used when you, as the experimenter, suspect that you have
more factors on your list than are actually needed. Which ones can you eliminate?

5.9.1 Half fractions

Video for
this section A half fraction has 12 2𝑘 = 2𝑘−1 runs. But which half of the runs do we omit? Let’s use an example of a
23 full factorial which has 8 experiments. The half-fraction would have 4 runs. Since 4 runs can be
represented by a 22 factorial, we start by writing down the usual 22 factorial for any 2 factors (we will
use A and B in this example, but you can use any 2 factors). Now create the 3rd factor as the product of
the first two, C = AB.

Experiment A B C = AB
1 − − +
2 + − −
3 − + −
4 + + +

So this is our half-factorial designed experiment in 3 factors, but it only requires 4 experiments as
shown by the open points in the figure. The experiments given by the solid points are not run.

346 Chapter 5. Design and Analysis of Experiments

 Release 547b16

What have we lost by running only half of the full factorial? Let’s write out the full design and matrix
of all interactions, then construct the X matrix for the least squares model.

Experiment A B C AB AC BC ABC Intercept

1 − − + + − − + +
2 + − − − − + + +
3 − + − − + − + +
4 + + + + + + + +

Before even constructing the X-matrix, you can see that A=BC, and that B=AC and C=AB (this last
association was intentional), and intercept I=ABC.

The least squares model would be:

y = Xb + e
𝑦𝑖 = 𝑏0 + 𝑏𝐴 𝑥𝐴 + 𝑏𝐵 𝑥𝐵 + 𝑏𝐶 𝑥𝐶 + 𝑏𝐴𝐵 𝑥𝐴𝐵 + 𝑏𝐴𝐶 𝑥𝐴𝐶 + 𝑏𝐵𝐶 𝑥𝐵𝐶 + 𝑏𝐴𝐵𝐶 𝑥𝐴𝐵𝐶 + 𝑒𝑖
⎡ ⎤
𝑏0
⎢ 𝑏 ⎥
𝐴 ⎥
⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎡ ⎤
𝑦1 1 −1 −1 +1 +1 −1 −1 +1 ⎢ 𝑏𝐵 ⎥ 𝑒1
⎢
⎢𝑦 ⎥ ⎢1 +1 −1 −1 −1 −1 +1 +1⎥ ⎢ 𝑏 ⎥ ⎢ ⎢𝑒 ⎥
⎢ 2⎥ ⎢ ⎥ ⎢ 𝐶 ⎥ ⎢ 2⎥
⎥
⎢ ⎥=⎢ ⎥+⎢ ⎥
⎣𝑦3 ⎦ ⎣1 −1 +1 −1 −1 +1 −1 +1⎦ ⎢
⎥
⎢ 𝑏𝐴𝐵 ⎥ ⎣𝑒3 ⎦
𝑦4 1 +1 +1 +1 +1 +1 +1 +1 ⎢ 𝑏𝐴𝐶 ⎥ 𝑒4
⎢ ⎥
⎢ ⎥
⎣ 𝑏𝐵𝐶 ⎦
𝑏𝐴𝐵𝐶

The X matrix is not orthogonal (page 337) anymore, because one or more columns are exactly identical
to another column, also known as collinearity. Notice that 4 of the columns are the same as the other 4:
we have perfect collinearity between 4 pairs of columns. Also note this system is underdetermined as
there are more unknowns than equations.

For these reasons the least squares model cannot be solved by inverting the X𝑇 X matrix. Prove it to
yourself by using this code:

5.9. Fractional factorial designs 347

Process Improvement Using Data

import numpy as np

intercept = [Link](4)
A = [Link]([-1, +1, -1, +1])
B = [Link]([-1, -1, +1, +1])
C = A * B
X = np.column_stack([intercept, A, B, C,
A * B, A * C, B * C,
A * B * C])

XtX = X.T @ X
print("The X'X matrix is = ")
print(XtX)

print("Calculate the inverse (it will fail!)")

[Link](XtX)

# We cannot, since the determinant is 0:

[Link](XtX)

int <- c(1, 1, 1, 1)

A <- c(-1, +1, -1, +1)
B <- c(-1, -1, +1, +1)
C = A * B
X = cbind(int, A, B, C, AB=A*B,
AC=A*C, BC=B*C, ABC=A*B*C)

XtX <- t(X) %*% X

print("The X'X matrix is = ")
print(XtX)

print('Calculate the inverse (it will fail!)')

solve(XtX)

# We cannot, since the determinant is 0:

det(XtX)

To resolve this problem we can reformulate the model to obtain independent columns, grouping
together the columns which are identical. There are now 4 equations and 4 unknowns:

y = Xb + e
⎡ ⎤ ⎡ ⎤⎡ ⎤ ⎡ ⎤
𝑦1 1 −1 −1 +1 𝑏0 + 𝑏𝐴𝐵𝐶 𝑒1
⎢𝑦 ⎥ ⎢1 +1 −1 −1⎥ ⎢ 𝑏𝐴 + 𝑏𝐵𝐶 ⎥ ⎢𝑒2 ⎥
⎥ ⎢ ⎥ ⎢
⎢ 2⎥ ⎢
⎢ ⎥=⎢ ⎥+⎢ ⎥
⎥
⎣𝑦3 ⎦ ⎣1 −1 +1 −1⎦ ⎣ 𝑏𝐵 + 𝑏𝐴𝐶 ⎦ ⎣𝑒3 ⎦
⎥⎢

𝑦4 1 +1 +1 +1 𝑏𝐶 + 𝑏𝐴𝐵 𝑒4

Writing it this way clearly shows how the main effects and two-factor interactions are confounded.

• 𝑏0 + 𝑏𝐴𝐵𝐶 = 𝛽̂︀0 → I + ABC

• 𝑏𝐴 + 𝑏𝐵𝐶 = 𝛽̂︀𝐴 → A + BC : this implies 𝛽𝐴 estimates the A main effect and the BC interaction

• 𝑏𝐵 + 𝑏𝐴𝐶 = 𝛽̂︀𝐵 → B + AC

• 𝑏𝐶 + 𝑏𝐴𝐵 = 𝛽̂︀𝐶 → C + AB

It means we cannot separate, for example, the effect of the BC interaction from the main effect of A: the
least-squares coefficient is a sum of both these effects. Similarly for the other pairs. This is why we say
the factor A is confounded with the two-factor interaction BC. Factor B is confounded with AC, and

348 Chapter 5. Design and Analysis of Experiments

 Release 547b16

factor C is confounded with AB. Also the intercept is not a pure estimate of the intercept; it is
confounded with the 3-factor interaction ABC.

This is what we have lost by running a half-fraction: the benefit of doing fewer experiments is paid by
the price of confounding within the factors we estimate.

We introduce the terminology that A is an alias for BC, similarly that B is an alias for AC, etc, because
we cannot separate these aliased effects.

5.9.2 Generators and defining relationships

Video for
this section Calculating which main effects and two-factor interactions will be confounded with each other, called
the confounding pattern, can be tedious for larger values of 𝑘. Here we introduce an easy way to
calculate the confounding pattern.

Recall for the half-fraction of a 2𝑘 factorial that the first 𝑘 − 1 main factors are written down, then the
final 𝑘 th factor is generated from the product of the previous 𝑘 − 1 factors. Consider the case of a 24 half
fraction with factors A, B, C and D. The half-fraction has 21 24 = 23 = 8 experiments, so we write this 23
factorial in factors A, B, and C, then set:

D = ABC

This is called the generating relation for the design. Here are some rules when working with this
notation:

• A factor multiplied by itself is the identity, or intercept column: A × A = I, B × B = I, etc. Think

about that: if you look at the previous designs we have written out, this makes sense. Any column
multiplied by itself is equal to a column of ones.

• A factor multiplied by a column of ones is equal to itself. For example: D × I = D

• The intercept I is simply a column of ones, which is what the intercept column is. And for emphasis:
I × I = I.

• You can substitute in the generating relation of D = ABC, and like with an algebraic equation, we can
multiply both sides by D to get D × D = ABC × D, which simplifies to I = ABCD. Another way to
get this same result it to substitute the generating relationship in twice: ABC × D = ABC × ABC =
AABBCC = I I I = I = ABCD.

This last part, I = ABCD, is called the defining relation for this design. Notice that we started with the
generating relation and simplified it by multiplying the terms in that relationship with each other. Since
there were two terms, ABC and D, we multiplied them, and ended up with I = ABCD.

This is our defining relationship for this design:

I = ABCD

We will discuss this topic again later with more examples. The main point though is that the effects
which are aliased (confounded) with each other can be found quickly by multiplying the effect we are
interested in by the defining relationship. For example, if we wanted to know what the main effect A
would be confounded with in this 24−1 half fraction we should multiply A by the defining relationship
as in

A = A × I = A × ABCD = BCD

indicating that A is aliased with the 3-factor interaction BCD. What is the aliasing for these effects:

• What is main effect B aliased with? (Answer: ACD)

5.9. Fractional factorial designs 349

 Process Improvement Using Data

• What is the 2fi AC aliased with? (Answer: BD)

Another example:

Returning back to the 23−1 half fraction in the previous section (page 346), use the generating
relation to verify the aliasing of main-effects and two-factor interactions derived earlier by hand.

• First calculate the defining relationship. It is I = . . . ..

• Aliasing for A? (Answer: BC)

• Aliasing for B? (Answer: AC)

• Aliasing for C? (Answer: AB: recall this is how we generated that half fraction)

• Aliasing for the intercept term, I? (Answer: ABC)

Yet another example:

Which aliasing (confounding) would occur if you decided for a 24−1 design to generate the
half-fraction by using the 2-factor interaction term AC rather than the 3-factor interaction term
ABC.

• First write out your generating relationship: D = AC

• Now calculate the defining relationship: I = . . . .

• Aliasing for A? (Answer: CD)

• Aliasing for B? (Answer: ABCD)

• Aliasing for C? (Answer: AD)

Why is this a poorer choice than using D = ABC to generate the half-fraction? Answer: the main
effects of A and C which could be important, are aliased with 2fi. Had we generated the design
with the usual 3fi term, ABC, the main effects would only be aliased with three-factor
Video for
interactions (3fi).
this section
5.9.3 Generating the complementary half-fraction
Returning to our example in the previous section (page 346) of a half-fraction from a full 23 factorial, and
imagine the half-fraction of 4 runs was completed. Imagine that all 3 factors showed significant effect
on the outcome. Further, imagine that one of the factors actually gave a direction opposite to what was
expected. This is really interesting, and unexpected new knowledge.

The original generator was C = AB and the defining relation was I = ABC; so factor C was aliased with
the 2fi of AB. If it was factor C that had an opposite sign, it could be due to C, or due to AB. So you
wish to complete the full-factorial and run the other half fraction to find out. This will help clarify that
interesting factor, because it will remove the aliasing when you then analyze all 8 data points together.

The defining relation for the complementary half-fraction is I = -ABC, or multiply both sides by C to
equivalently obtain IC = C = -AB. This shows the complementary half fraction is in fact generated by
C = -AB, while the original half-fraction was generated by C = AB. This is a general rule that applies to
half-fractions.

Let’s return to the table in the previous section (page 346) and generate the other 4 runs from that C =
-AB defining relationship:

350 Chapter 5. Design and Analysis of Experiments

 Release 547b16

Experiment A B C = − AB
5 − − −
6 + − +
7 − + +
8 + + −

After running these additional 4 experiments shown (in random order of course) we have a complete
set of 8 runs. Analyzing the data together we can calculate the main effects and two-factor interactions
without aliasing because we are back to the usual full factorial of 23 runs. Confirm it for yourself
visually in the plot alongside.

So we see that we can always complete our half-fraction by creating a complementary fraction. This
complimentary fraction is found by flipping the sign on the generating factor. For example, changing
the sign from C = AB to -C = AB. In the illustration this is equivalent to running the 4 experiments at
the closed circles.

5.9.4 Generators: to determine confounding due to blocking

Generators are also great for determining the blocking pattern. Recall the case described earlier where
we only had enough material to run two sets of 4 experiments to complete our 23 full factorial. An
unintended disturbance could have been introduced by running the first half-fraction on different
materials to the second half-fraction. We intentionally decided (page 363) to confound the two blocks of
experiments with the 3-factor interaction, ABC. So if there is an effect due to the blocks (i.e. the raw
materials) or if there truly was a 3-factor interaction, it will show up as a significant coefficient for
𝑏𝐴𝐵𝐶 .

So in general if you run a full 2𝑘 factorial in two blocks you should create a 2𝑘−1 half fraction to run as
the first block, and then run the other block on the complementary half-fraction. You should always
confound your block effect on the highest possible interaction term. Then block 1 runs will have that
highest interaction factor with all positive signs, and block 2 will have all negative signs for that
interaction factor.

Here are the block generators you can use when splitting a 2𝑘 factorial in 2 blocks:

5.9. Fractional factorial designs 351

Process Improvement Using Data

𝑘 Design Block 1 defining relation Block 2 defining relation

3 23−1 I=ABC I=-ABC
4 24−1 I=ABCD I=-ABCD
5 25−1 I=ABCDE I=-ABCDE

5.9.5 Highly fractionated designs: beyond half-fractions

Running a half-fraction of a 2𝑘 factorial is not the only way to reduce the number of runs. In general,
we can run a 2𝑘−𝑝 fractional factorial. A system with 2𝑘−1 is called a half fraction, while a 2𝑘−2 design is
a quarter fraction, and so on.

The purpose of a fractionated design is to reduce the number of experiments when your budget - or
time - does not allow you to complete a full factorial. Also, the full factorial is often not required,
especially when 𝑘 is greater than about 4, since the higher-order interaction terms are almost always
insignificant. If we have a budget - or time - for only 8 experiments, then our options are to run a:

• 23 full factorial on 3 factors

• 24−1 half fraction, investigating 4 factors

• 25−2 quarter fraction looking at the effects of 5 factors

• 26−3 fractional factorial with 6 factors, or a

• 27−4 fractional factorial with 7 factors.

At the early stages of our work we might prefer to screen many factors, 𝑘 = 6 or 7, accepting a very
complex confounding pattern, because we are uncertain which factors actually affect our response.
Later, as we are optimizing our process, particularly as we approach an optimum, then the 2 factor and
perhaps 3-factor interactions are more dominant. So investigating and calculating these effects more
accurately and more precisely becomes important and we have to use full factorials. But by then we
have hopefully identified much fewer factors 𝑘 than what we started off with.

So this section is concerned with the trade-offs as we go from a full factorial with 2𝑘 runs to a highly
fractionated factorial, 2𝑘−𝑝 .

Example 1

You have identified 7 factors that affect your response.

• What is the smallest number of runs that can be performed to screen for the main effects?

A 27−𝑝 fractional factorial would have 64 (𝑝 = 1), 32 (𝑝 = 2), 16 (𝑝 = 3), or 8 (𝑝 = 4) runs. The
last case is the smallest number that can be used to estimate the intercept and 7 main effects (8
data points, 8 unknowns).

• What would be the generators and defining relation for this 27−4 = 8 run experiment and
what is the aliasing structure?

1. Assign A, B, . . . G as the 7 factor names. Since there are 8 runs, start by writing out the first
3 factors, A, B, and C, in the usual full factorial layout for these 23 = 8 runs.

2. Next we assign the remaining factors to the highest-possible interaction terms from these 3
factors. Since we’ve already assigned A, B and C we only have to assign the other 4 factors,
D, E, F, and G. Pick the 4 interaction terms that we are least interested in. In this particular
example, we have to use all (saturate) the interaction terms.

352 Chapter 5. Design and Analysis of Experiments

 Release 547b16

– D = AB

– E = AC

– F = BC

– G = ABC

Record the experimental results for 𝑦 in the last column.

Experiment A B C D=AB E=AC F=BC G=ABC 𝑦

1 − − − + + + − 77.1
2 + − − − − + + 68.9
3 − + − − + − + 75.5
4 + + − + − − − 72.5
5 − − + + − − + 67.9
6 + − + − + − − 68.5
7 − + + − − + − 71.5
8 + + + + + + + 63.7

1. So the 4 generators we used to create, or generate, the experimental design are I = ABD, I =
ACE, I = BCF and I = ABCG. The generator terms such as ABD and ACE are called
“words”.

2. The defining relationship is a sequence of words which are all equal to I. The defining relation
is found from the product of all possible generator combinations, and then simplified to be
written as I = . . . ..

The rule is that a 2𝑘−𝑝 factorial design is produced by 𝑝 generators and has a defining
relationship of 2𝑝 words. So in this example there are 𝑝 generators and 2𝑝 = 24 = 16 words
in our defining relation. They are:

– Intercept: I [1]

– Each generator combined with I: I = ABD = ACE = BCF = ABCG [2,3,4,5]

– Two combinations of generators: I = BDCE = ACDF = CDG = ABEF = BEG = AFG [6 to

11]

– Three combinations of generators: I = DEF = ADEG = CEFG = BDFG [12 to 15]

– Four combinations of generators: I = ABCDEFG [16]

The 16 words in the defining relationship are written as: I = ABD = ACE = BCF = ABCG =
BCDE = ACDF = CDG = ABEF = BEG = AFG = DEF = ADEG = CEFG = BDFG =
ABCDEFG. The shortest length word, not counting the intercept, has 3 letters. We will refer
to this later as the design’s resolution (page 355).

3. The aliasing or confounding pattern for any desired effect can be calculated by multiplying
the defining relationship by that effect. Let’s take A as an example, below, and multiply it
by the 16 words in the defining relation:

AI = BD = CE = ABCF = BCG = ABDCE = CDF = ACDG = BEF = ABEG = FG = ADEF =

DEG = ACEFG = ABDFG = BCDEFG. So our factor A estimate is not just factor A, it is also
a combined estimate of:

5.9. Fractional factorial designs 353

Process Improvement Using Data

𝛽̂︀𝐴 → A + BD + CE + ABCF + BCG + ABCDE + CDF + ACDG + BEF + ABEG + FG +

ADEF + DEG + ACEFG + ABDFG + BCDEFG.

𝛽̂︀𝐴 ≈ A + BD + CE + FG + . . .

So by performing 8 runs instead of the full 27 , we confound the main effects with a large
number of 2-factor and higher interaction terms. In particular, the main effect of A is
confounded here with the BD, CE and FG two-factor interactions. Any 3 and higher-order
interaction confounding is usually not of interest.

Listed below are all the aliases for the main effects, reporting only the two-factor
interactions. The bold words indicate the confounding that was intentionally created when
we set up the design.

– 𝛽̂︀0 = ABCDEFG

– 𝛽̂︀A → A + BD + CE + FG

– 𝛽̂︀B → B + AD + CF + EG

– 𝛽̂︀C → C + AE + BF + DG

– 𝛽̂︀D → D + AB + CG + EF

– 𝛽̂︀E → E + AC + BG + DF

– 𝛽̂︀F → F + BC + AG + DE

– 𝛽̂︀G → G + CD + BE + AF

4. If this confounding pattern is not suitable, for example, if you expect interaction BG to be
important but also main effect E, then choose a different set of generators before running
the experiment. Or more simply, reassign your variables (temperature, pressure, pH,
agitation, etc) to different letters of A, B, etc to obtain another, hopefully more desirable,
confounding relationship.

Example 2

From a cause-and-effect analysis, flowcharts, brainstorming session, expert opinions, operator

opinions, and information from suppliers you might determine that there are 8 factors that could
impact an important response variable. Rather than running 28 = 256 experiments, you can run
28−4 = 16 experiments. (Note: you cannot have fewer experiments: 28−5 = 8 runs are not
sufficient to estimate the intercept and 8 main effects).

So the 28−4 factorial will have 24 runs. Assign the first 4 factors to A, B, C and D in the usual full
factorial manner to create these 16 runs, then assign the remaining 4 factors to the three-factor
interactions:

• E = ABC, or I = ABCE

• F = ABD, or I = ABDF

• G = BCD, or I = BCDG

• H = ACD, or I = ACDH

So by multiplying all combinations of the words we obtain the complete defining relationship.
We expect 𝑝 = 4 generators and 2𝑝 = 16 words in the defining relationship.

• Two at a time: I = ABCE × ABDF = CDEF, I = ABCE × BCDG = ADEG, etc

354 Chapter 5. Design and Analysis of Experiments

 Release 547b16

• Three at a time: I = ABCE × ABDF × BCDG = BEFG, etc

• Four at a time: I = ABCE × ABDF × BCDG × ACDH = ABCDEFGH.

The defining relationship is I = ABCE = ABDF = BCDG = ACDH = CDEF = ADEG = . . . =

ABCDEFGH, and the shortest word, not counting the intercept, has 4 characters. We will refer to
this later as the design’s resolution (page 355).

Next we can calculate all aliases of the main effects. So for A = IA = BCE = BDF = ABCDG =
CDH = ACDEF = DEG = . . . = BCDEFGH, indicating that A will be confounded with BCE +
BDF + ABCDG + . . . . In this example none of the main effects have been aliased with two-factor
interactions. The aliasing is only with 3-factor and higher interaction terms.

Summary

1. It is tedious and error prone to calculate the aliasing structure by hand, so computer software is
useful in this case. For example, for the 27−4 system can be created in R by first loading the BHH2
package, then using the command ffDesMatrix(k=7, gen=list(c(4,1,2), c(5,1,3),
c(6,2,3), c(7,1,2,3))). See the R tutorial132 for more details on how to install packages like
BHH2.

2. The choice of generators is not unique and other choices may lead to a different, more preferable
confounding pattern. But it is often easier to use the letters A, B, C, etc, then just reassign the factors
to the letters to achieve the “least-worst” confounding for your situation.

3. In general, a 2𝑘−𝑝 factorial design is produced by 𝑝 generators and has a defining relationship of 2𝑝
words.

There is a quick way to calculate if main effects will be confounded with 2fi or 3fi without having to go
through the process shown in this section. This is described next when we look at design resolution
(page 355).

5.9.6 Design resolution

The resolution of a design is given by the length of the shortest word in the defining relation. We
normally write the resolution as a subscript to the factorial design using Roman numerals. Some
examples:

1. The 27−4 example 1 in the previous section had the shortest word of 3 characters, so this would be
called a 27−4
III design. Main effects were confounded with 2-factor interactions in that example.

2. The 28−5 example 2 had as the shortest word length of 4 characters, so this would be a 28−5
IV design.
Main effects were confounded with 3-factor interactions.

3. Finally, imagine the case of 25−1 , in other words a half-fraction design. The first four factors are
written in the standard factorial way, but the fifth factor is generated from E = ABCD. So its
defining relation is I = ABCDE, where the shortest word, not counting the intercept, is 5 characters -
it is a 25−1
V design. You can verify for yourself that main effects will be confounded with 4-factor
interactions in this design.

You can consider the resolution of a design to be an indication of how clearly the effects can be
separated in a design. The higher the resolution, the lower the degree of confounding: this is desirable.
Within reason, always aim for a higher resolution design given your experimental budget, but also
132 [Link]

5.9. Fractional factorial designs 355

 Process Improvement Using Data

accept a lower resolution, at least initially, in order to test for more factors. We will discuss this further
in the section on screening designs (page 357).

You can interpret the resolution index as follows: let main effects = 1, two-factor interactions = 2,
three-factor interactions = 3, etc. Then subtract this number from the resolution index to show how
that effect is aliased. Consider a resolution IV design, since 4 − 1 = 3, it indicates that main effects are
aliased with 3fi, but not with two-factor interactions; and 4 − 2 = 2 indicates that 2fi are aliased with
each other. Here is a summary:

Resolution III designs: are good for screening

• Are excellent for initial screening: to separate out the important factors from the many potential
factors. To identify important variables that can be studied in the next experiments.

• Main effects are not confounded with each other.

• Main effects are aliased with two-factor interactions (3 − 1 = 2).

• Two-factor interactions are aliased with main effects (3 − 2 = 1).

Resolution IV designs: are good for characterizing

• Most useful for characterizing (learning about and understanding) a system, both the main
effects and the interactions, since:

• Interactions are very interesting in many systems, since they go beyond just the main effects and
can improve your process even more (or sometimes take away from it too!).

• Main effects are not confounded with each other.

• Main effects are not aliased with two-factor interactions either (4 − 1 = 3).

• Main effects are aliased with three-factor interactions though (4 − 1 = 3).

• Two-factor interactions are still aliased with each other though (4 − 2 = 2).

Resolution V designs: are good for optimizing

• For optimizing a process, learning about complex effects, and developing high-accuracy
predictive models, since:

• Main effects are not confounded with each other.

• Main effects are not aliased with two-factor interactions.

• Two-factor interactions are not aliased with each other either.

• But two-factor interactions are aliased with three-factor interactions (5 − 2 = 3).

The above guidance about using resolution IV and V designs for characterization and optimization is
fairly general - there are many cases where a satisfactory optimization can be performed with a
resolution IV experiment.

In this text currently, for resolution III, IV and V designs we look at factorial designs. However, there
are a number of other design types which can also be used. If you are interested, please research
Plackett-Burman designs, Box-Behnken designs, central composite designs, and definitive screening
designs (page 377).
Video for
this section You can use the following table to visualize the trade-off between design resolution, the number of
factors (𝑘), the number of runs required, and the aliasing pattern. The table is adapted from the text by
Box, Hunter and Hunter (2nd edition, p 272), and (1st edition, p 410).

356 Chapter 5. Design and Analysis of Experiments

 Release 547b16

5.9.7 Saturated designs for screening

A saturated design can be likened to a well trained doctor asking you a few, but very specific,
questions to identify a disease or problem. On the other hand, if you sit there just tell the doctor all
your symptoms, you may or may not get an accurate diagnosis. Designed experiments, like visiting
this doctor, shortens the time required to identify the major effects in a system, and to do so as
accurately as possible, within limited budget.

Saturated designs are most suited for screening, and should always be run when you are investigating
a new system with many factors. These designs are usually of resolution III and allow you to
determine the main effects with a low number of experiments.

For example, a 27−4

III factorial, introduced in the section on highly fractionated designs (page 352), will
screen 7 factors in 8 experiments. Once you have run the 8 experiments you can quickly tell which
subset of the 7 factors are actually important, and spend the rest of your budget on clearly
understanding these effects and their interactions. Bear in mind that there is a risk of confounding, as
previously described in that section.

Let’s see how by continuing the previous example, repeated again below with the corresponding
values of 𝑦. Recall it was a set of eight experiments in seven factors (page 352):

5.9. Fractional factorial designs 357

Process Improvement Using Data

Experiment A B C D=AB E=AC F=BC G=ABC 𝑦

1 − − − + + + − 77.1
2 + − − − − + + 68.9
3 − + − − + − + 75.5
4 + + − + − − − 72.5
5 − − + + − − + 67.9
6 + − + − + − − 68.5
7 − + + − − + − 71.5
8 + + + + + + + 63.7

Use a least squares model to estimate the coefficients in the model:

y = Xb
)︀−1 𝑇
b = X𝑇 X
(︀
X y

where b = [𝑏0 , 𝑏𝐴 , 𝑏𝐵 , 𝑏𝐶 , 𝑏𝐷 , 𝑏𝐸 , 𝑏𝐹 , 𝑏𝐺 ]. The matrix X is essentiall a copy of the above table, but with
an added column of 1’s for the intercept term. Notice that the X𝑇 X matrix will be diagonal. Make sure
you can calculate X𝑇 X by hand, at least once. It is also straightforward to calculate the solution vector
(by hand!), which you can confirm to be b = [70.7, −2.3, 0.1, −2.8, −0.4, 0.5, −0.4, −1.7].

How do you assess which main effects are important? There are eight data points and eight
parameters, so there are no degrees of freedom and the residuals are all zero. In this case you have to
use a Pareto plot (page 341), which requires that your variables have been suitably scaled in order to
judge importance of the main effects relative to each other. The Pareto plot would be given as shown
below, and as usual, it does not show the intercept term.

import itertools
import numpy as np
import pandas as pd
import [Link] as sm

[Link] = "plotly"

# Create vectors for each factor in the experiment.

# [Link] gives the full 2^3 design.
design = [Link](
list([Link]([-1, +1],
[-1, +1],
[-1, +1])),
columns=["A", "B", "C"],
)
A = design["A"].to_numpy()
B = design["B"].to_numpy()
C = design["C"].to_numpy()
D = A * B
E = A * C
F = B * C
G = A * B * C
y = [Link]([77.1, 68.9, 75.5, 72.5,
67.9, 68.5, 71.5, 63.7])

X = np.column_stack([A, B, C, D, E, F, G])
X = sm.add_constant(X)
demo = [Link](y, X).fit()
print([Link]())

# OK, now we are ready to generate the Pareto plot.

(continues on next page)

358 Chapter 5. Design and Analysis of Experiments

 Release 547b16

(continued from previous page)

# Sort the absolute coefficient values, dropping
# the intercept (matching the R paretoPlot conventions).
names = ["A", "B", "C", "D", "E", "F", "G"]
effects = [Link](
[Link]([Link][1:]),
index=names,
).sort_values()
fig = [Link](orientation="h")
fig.update_layout(
xaxis_title_text="|effect|",
yaxis_title_text="Factor",
showlegend=False,
)
[Link]()

# Try getting the results manually:

XtX = X.T @ X
print("The XtX matrix is:")
print(XtX)

Xty = X.T @ y
b = [Link](XtX, Xty)
print("The solution vector is:")
print(b)

# Create vectors for each factor in the experiment

A = B = C = c(-1, +1)
design = [Link](A=A, B=B, C=C)
A = design$A
B = design$B
C = design$C
D = A*B
E = A*C
F = B*C
G = A*B*C
y = c(77.1, 68.9, 75.5, 72.5, 67.9, 68.5, 71.5, 63.7)

demo = lm(y ~ A + B + C + D + E + F + G)
summary(demo)

# OK, now we are ready to generate the Pareto plot.

# Let's use a library to do that for us.

# library(pid) <-- best to use this!

# It is better to uncomment and use the line above.

# But this embedded R script on this website does not have the
# "pid" library available. So we will load the required function
# from an external server instead:
source('[Link]

# And now we can generate the plot:

paretoPlot(demo)

# Try getting the results manually:

X_matrix = [Link](demo)
XtX <- t(X_matrix) %*% X_matrix
print('The XtX matrix is:')
print(XtX)

Xty <- t(X_matrix) %*% y

b = solve(XtX) %*% Xty
print('The solution vector is:')
print(b)

5.9. Fractional factorial designs 359

 Process Improvement Using Data

Significant effects would be A, C and G. The next largest effect, E, though fairly small, could be due to
the main effect E or due to the AC interaction, because recall the confounding pattern, up to the 2
factor-interactions, for main effect was 𝛽̂︀E → E + AC + BG + DF.

The factor B is definitely not important to the response variable in this system and can be excluded in
future experiments, as could F and D likely. Future experiments should focus on the A, C and G
factors and their interactions. We show how to use these existing 8 experiments in the above table, but
add a few new ones in the next section on design foldover and by understanding projectivity.

A side note on screening designs is a mention of Plackett and Burman designs. These designs can
sometimes be of greater use than a highly fractionated design. A fractional factorial must have 2𝑘−𝑝
runs, for integers 𝑘 and 𝑝: i.e. either 4, 8, 16, 32, 64, 128, . . . runs. Plackett-Burman designs are screening
designs that can be run in any multiple of 4 greater or equal to 12:, i.e. 12, 16, 20, 24, . . . runs. The Box,
Hunter, and Hunter book has more information in Chapter 7, but another interesting paper on these
topic is by Box and Bisgaard: “What can you find out from 12 experimental runs?”, which shows how
to screen for 11 factors in 12 experiments.
Video for
this section An important mention to readers interested in other, arguable better screening strategies, is to consider
definitive screening designs (page 377).

5.9.8 Design foldover

Experiments are not a one-shot operation. They are almost always sequential, as we learn more and
more about our system. Once the first screening experiments are complete there will always be
additional questions. In this section we consider two common questions that arise after an initial set of
fractional factorials have been run.

Dealias a single main effect (switch sign of one factor)

In the previous example we had a 27−4 III system with generators D=AB, E=AC, F=BC, and G=ABC.
Effect C was the largest effect. But we cannot be sure it was large due to factor C alone: it might have
been one of the interactions it is aliased with. The aliasing pattern for effect C was: 𝛽̂︀C → C + AE + BF
+ DG. For example, we might have reason to believe the AE interaction is large. We would like to do
some additional experiments so that C becomes unconfounded with any two-factor interactions.

360 Chapter 5. Design and Analysis of Experiments

 Release 547b16

The way we can do this is to run another 8 experiments, but this time just change the sign of C to -C; in
other words, re-run the original 8 experiments where the only thing that is changed is to flip the signs
on column C; the columns which are generated from column C should remain as they were. This
implies that the generators have become D=AB, E=-AC, F=-BC, and G=-ABC. We must emphasize, do
not re-create these generated columns from new signs in column C. What we have now is another 27−4 III
design. You can calculate the aliasing pattern for the recent 8 experiments is 𝛽̂︀C → C - AE - BF - DG.

Now consider putting all 16 runs together and analyzing the joint results. There are now 16
parameters that can be estimated. Using computer software you can see that factor C will have no
confounding with any two-factor interactions. Also, any two-factor interactions involving C are
removed from the other main effects. For example, factor A was originally confounded with CE with
the first 8 experiments; but that will be removed when analyzing all 16 together.

So our general conclusion is: switching the sign of one factor will de-alias that factor’s main effect,
and all its associated two-factor interactions when analyzing the two fractional factorials together. In
the above example, we will have an unconfounded estimate of C and 2-factor interactions involving C
will be unconfounded with main effects: i.e. AC, BC, CD, CE, CF and CG.

Increase design resolution (switching all signs)

One can improve the aliasing structure of a design when switching all the signs of all factors from the
first fraction. Switching the signs means that we take the complete design matrix of factor settings, and
simply flip the signs to create the second fraction. A factor setting that was run at the low level in the
first fraction is then run at a high level in the second fraction. This includes generated factors: imagine
that D = AB, and we had A = −1, B = +1 for a particular run in the first fraction. In the first fraction
we would have D = (−1)(+1) = −1, while in the second fraction it would simply be D = +1, with A
= +1 and B = −1 respectively, since we simply flip signs. We do not regenerate the generated factors.

In the 27−4
III example, we ran 8 experiments. If we now run another 8 experiments with all the signs
switched, then these 8+8 experiments would be equivalent to a 27−3 IV design. This resolution IV design
means that all the main effects can now be estimated without confounding from any two-factor
interactions. However, the two-factor interactions are still confounded with themselves.

This is a good strategy in general: to run the first fraction of runs to assess the main effects. It serves as
a good checkpoint, as well providing intermediate results to colleagues, it can be used to get
approval/budget to run the next set of experiments. If we then perform another set of runs we know
that we are doing them in a way that captures the most additional information, and with the least
confounding. Remember experimentation is not done in a single go; it is sequential. We perform
experiments, analyze the results, and design further experiments to reach our goal.

5.9.9 Projectivity
A final observation for this section is how fractional factorials will collapse down to a full factorial
under certain conditions.

Consider the diagram here, where a half fraction in factors A, B and C was run (4 experiments) at the
closed points.

5.9. Fractional factorial designs 361

 Process Improvement Using Data

On analyzing the data, the experimenter discovers that factor C does not actually have an impact on
response 𝑦. This means the C dimension could have been removed from the experiment, and is
illustrated by projecting the A and B factors forward, removing factor C. Notice that this is now a full
factorial in factors A and B. The same situation would hold if either factor B or factor A were found to
be unimportant. Furthermore if two factors are found to be unimportant, then this corresponds to 2
replicate experiments in 1 factor.

This projectivity of factorials holds in general for a larger number of factors. The above example,
actually a 23−1
III experiment, was a case of projectivity = 2. In general, projectivity = 𝑃 = resolution − 1.
So if you have a resolution IV fractional factorial, then it has projectivity = 𝑃 = 4 − 1, implying that it
6−2
contains a full factorial in 3 factors. So a 2IV (16 runs) system with 6 factors, contains an embedded
full factorial using a combination of any 3 factors; if any 3 factors were found unimportant, then a
replicated full factorial exists in the remaining 3 factors.

5.10 Blocking and confounding for disturbances

5.10.1 Characterization of disturbances

Video for
this section External disturbances will always have an effect on our response variable, 𝑦. Operators, ambient
conditions, physical equipment, lab analyses, and time-dependent effects (catalyst deactivation,
fouling), will impact the response. This is why it is crucial to randomize (page 326) the order of
experiments: so that these unknown, unmeasurable, and uncontrollable disturbances cannot
systematically affect the response.

However, certain disturbances are known, or controllable, or measurable. For these cases we perform
pairing and blocking. We have already discussed pairing in the univariate section: pairing is when
two experiments are run on the same subject and we analyze the differences in the two response
values, rather than the actual response values. If the effect of the disturbance has the same magnitude
on both experiments, then that disturbance will cancel out when calculating the difference. The
magnitude of the disturbance is expected to be different between paired experiments, but is expected
to be the same within the two values of the pair.

Blocking is slightly different: blocking is a special way of running the experiment so that the
disturbance actually does affect the response, but we construct the experiment so that this effect is not
misleading.

Finally, a disturbance can be characterized as a controlled disturbance, in which case it isn’t a

362 Chapter 5. Design and Analysis of Experiments

 Release 547b16

disturbance anymore, as it is held constant for all experiments, and its effect cancels out. But it might
be important to investigate the controlled disturbance, especially if the system is operated later on
when this disturbance is at a different level.

5.10.2 Blocking and confounding

Video for
this section It is common for known, or controllable or measurable factors to have an effect on the response.
However these disturbance factors might not be of interest to us during the experiment. Cases are:

• Known and measurable, not controlled: Reactor vessel A is known to achieve a slightly better response,
on average, than reactor B. However both reactors must be used to complete the experiments in the
short time available.

• Known, but not measurable nor controlled: There is not enough material to perform all 23 runs, there is
only enough for 4 runs. The impurity in either the first batch A for 4 experiments, or the second
batch B for the other 4 runs will be different, and might either increase or decrease the response
variable (we don’t know the effect it will have).

• Known, measurable and controlled: Reactor vessel A and B have a known, measurable effect on the
output, 𝑦. To control for this effect we perform all experiments in either reactor A or B, to prevent
the reactor effect from confounding (confusing) our results.

In this section then we will deal with disturbances that are known, but their effect may or may not be
measurable. We will also assume that we cannot control that disturbance, but we would like to
minimize its effect.

For example, if we don’t have enough material for all 23 runs, but only enough for 4 runs, the question
is how to arrange the 2 sets of 4 runs so that the known, by unmeasurable disturbance from the
impurity has the least effect on our results and interpretation of the 3 factors.

Our approach is to intentionally confound the effect of the disturbance with an effect that is expected to
be the least significant. The 𝐴 × 𝐵 × 𝐶 interaction term is almost always going to be small for many
systems, so we will split the runs that the first 4 are run at the low level of 𝐴𝐵𝐶 and the other four at
the high level, as illustrated.

Each group of 4 runs is called a block and the process of creating these 2 blocks is called blocking. The
experiments within each block must be run randomly.

5.10. Blocking and confounding for disturbances 363

Process Improvement Using Data

Experiment A B C AB AC BC ABC Response, 𝑦

1 − − − + + + − (batch 1) 𝑦̃︀1
2 + − − − − + + (batch 2) ˚
𝑦2
3 − + − − + − + (batch 2) ˚
𝑦3
4 + + − + − − − (batch 1) 𝑦̃︀4
5 − − + + − − + (batch 2) ˚
𝑦5
6 + − + − + − − (batch 1) 𝑦̃︀6
7 − + + − − + − (batch 1) 𝑦̃︀7
8 + + + + + + + (batch 2) ˚
𝑦8

If the raw material has a significant effect on the response variable, then we will not be able to tell
whether it was due to the 𝐴 × 𝐵 × 𝐶 interaction, or due to the raw material, since
𝛽^𝐴𝐵𝐶 → ABC
⏟ interaction
⏞ +raw material effect.
expected to be small

But the small loss due to this confusion of effects, is the gain that we can still estimate the main effects
and two-factor interactions without bias, provided the effect of the disturbance is constant. Let’s see
how we get this result by denoting 𝑦̃︀𝑖 as a 𝑦 response from the first batch of materials and let ˚
𝑦𝑖 denote
a response from the second batch.

Using the least squares equations you can show for yourself that (some are intentionally left blank for
you to complete):

𝛽^𝐴 = −̃︀ 𝑦2 − ˚
𝑦1 + ˚ 𝑦3 + 𝑦̃︀4 − ˚
𝑦5 + 𝑦̃︀6 − 𝑦̃︀7 + ˚
𝑦8
^
𝛽𝐵 =
𝛽^𝐶 =
𝛽^𝐴𝐵 = +̃︀
𝑦1 − ˚
𝑦2 − ˚ 𝑦5 − 𝑦̃︀6 − 𝑦̃︀7 + ˚
𝑦3 + 𝑦̃︀4 + ˚ 𝑦8
^
𝛽𝐴𝐶 =
𝛽^𝐵𝐶 =
𝛽^𝐴𝐵𝐶 =

Imagine now the 𝑦 response was increased by 𝑔 units for the batch 1 experiments, and increased by ℎ
units for batch 2 experiments. You can prove to yourself that these biases will cancel out for all main
effects and all two-factor interactions. The three factor interaction of 𝛽^𝐴𝐵𝐶 will however be heavily
confounded.

Another way to view this problem is that the first batch of materials and the second batch of materials
can be represented by a new variable, called 𝐷 with value of 𝐷− = batch 1 and 𝐷+ = batch 2. We will
show next that we must consider this new factor to be generated from the other three: D = ABC.

We will also address the case when there are more than two blocks in the next section on the use of
generators (page 349). For example, what should we do if we have to run a 23 factorial but with only
enough material for 2 experiments at a time?

364 Chapter 5. Design and Analysis of Experiments

 Release 547b16

5.11 Response surface methods

Video for
this section The purpose of response surface methods (RSM) is to optimize a process or system. RSM is a way to
explore the effect of operating conditions (the factors) on the response variable, 𝑦. As we map out the
unknown response surface of 𝑦, we move our process as close as possible towards the optimum,
taking into account any constraints.

Initially, when we are far away from the optimum, we will use factorial experiments. As we approach
the optimum then these factorials are replaced with better designs that more closely approximate
conditions at the optimum.

Notice how it is a sequential approach. RSM then is a tool the describes how we should run these
sequential sets of experiments. At the start of this section on designed experiments (page 328) we showed
how sequential experimentation (COST) leads to sub-optimal solutions. Why are we advocating
sequential experimentation now? The difference is that here we use sequential experiments by
changing multiple factors simultaneously, and not changing only one factor at a time.

RSM concept for a single variable: COST approach

We will however first consider just the effect of a single factor, 𝑥1 as it relates to our response, 𝑦. This is
to illustrate the general response surface process.
Video for
this section

We start at the point marked 𝑖 = 0 as our initial baseline (cp=center point). We run a 2-level
experiment, above and below this baseline at −1 and +1, in coded units of 𝑥1 , and obtain the
corresponding response values of 𝑦0,− and 𝑦0,+ . From this we can estimate a best-fit straight line and
move in the direction that increases 𝑦. The sloping tangential line, also called the path of steepest ascent.
Make a move of step-size = 𝛾1 units along 𝑥1 and measure the response, recorded as 𝑦1 . The response
variable increased, so we keep going in this direction.

Make another step-size, this time of 𝛾2 units in the direction that increases 𝑦. We measure the response,
𝑦2 , and are still increasing. Encouraged by this, we take another step of size 𝛾3 . The step-sizes, 𝛾𝑖
should be of a size that is big enough to cause a change in the response in a reasonable number of

5.11. Response surface methods 365

 Process Improvement Using Data

experiments, but not so big as to miss an optimum.

Our next value of 𝑦3 is about the same size as 𝑦2 , indicating that we have plateaued. At this point we
can take some exploratory steps and refit the tangential line (which now has a slope in the opposite
direction). Or we can just use the accumulated points 𝑦 = [𝑦0− , 𝑦0+ , 𝑦1 , 𝑦2 , 𝑦3 ] and their
corresponding 𝑥-values to fit a non-linear curve. Either way, we can then estimate a different step-size
𝛾4 that will bring us closer to the optimum.

This univariate example is in fact what experimenters do when using the COST approach (page 328)
described earlier. We have:

• exploratory steps of different sizes towards an optimum

• refit the model once we plateau

• repeat
Video for
this section This approach works well if there really is only a single factor that affects the response. But with most
systems there are multiple factors that affect the response. We show next how the exact same idea is
used, only we change multiple variables at a time to find the optimum on the response surface.

5.11.1 Response surface optimization via a 2-variable system example

Video for
this section This example considers a new system here where two factors, temperature T, and substrate
concentration S are known to affect the yield from a bioreactor. But in this example we are not just
interested in yield, but actually the total profit from the system. This profit takes into account energy
costs, raw materials costs and other relevant factors. The illustrations in this section show the contours
of profit in light grey, but in practice these are obviously unknown.

We currently operate at this baseline condition:

• T = 325 K

• S = 0.75 g/L

• Profit = $407 per day

We start by creating a full factorial around this baseline by choosing Δ𝑇 = 10 K, and Δ𝑆 = 0.5 g/L
based on our knowledge that these are sufficiently large changes to show an actual difference in the
response value, but not too large so as to move to a totally different form of operation in the bioreactor.

The results from the full factorial are in the table here:

Experiment T (actual) S (actual) T (coded) S (coded) Profit

Baseline 325 K 0.75 g/L 0 0 407
1 320 K 0.50 g/L − − 193
2 330 K 0.50 g/L + − 310
3 320 K 1.0 g/L − + 468
4 330 K 1.0 g/L + + 571

Clearly the promising direction to maximize profit is to operate at higher temperatures and higher
substrate concentrations. But how much much higher and in what ratio should we increase 𝑇 and 𝑆?
These answers are found by building a linear model of the system from the factorial data:

𝑦^ = 𝑏0 + 𝑏𝑇 𝑥𝑇 + 𝑏𝑆 𝑥𝑆 + 𝑏𝑇 𝑆 𝑥𝑇 𝑥𝑆
𝑦^ = 389.8 + 55𝑥𝑇 + 134𝑥𝑆 − 3.50𝑥𝑇 𝑥𝑆

366 Chapter 5. Design and Analysis of Experiments

 Release 547b16

𝑥𝑇,actual − center𝑇 𝑥𝑇,actual − 325 𝑥𝑆,actual − 0.75

where 𝑥𝑇 = = and similarly, 𝑥𝑆 = .
Δ𝑇 /2 5 0.25
The model shows that we can expect an increase of $55/day of profit for a unit increase in 𝑥𝑇 (coded
units). In real-world units that would require increasing temperature by Δ𝑥𝑇,actual = (1) × Δ𝑇 /2 = 5K
to achieve that goal. That scaling factor comes from the coding we used:
𝑥𝑇,actual − center𝑇
𝑥𝑇 =
Δ𝑇 /2
Δ𝑥𝑇,actual
Δ𝑥𝑇 =
Δ𝑇 /2

Similarly, we can increase 𝑆 by Δ𝑥𝑆 = 1 unit = 1 × Δ𝑆 /2 = 0.5/2 = 0.25 g/L real-world units, to
achieve a $134 per day profit increase.

The interaction term is small, indicating the response surface is mostly linear in this region. The
illustration shows the model’s contours (straight, green lines). Notice that the model contours are a
good approximation to the actual contours (dotted, light grey), which are unknown in practice.

To improve our profit in the optimal way we move along our estimated model’s surface, in the
direction of steepest ascent. This direction is found by taking partial derivatives of the model function,
ignoring the interaction term, since it is so small.

𝜕 𝑦^ 𝜕 𝑦^
= 𝑏𝑇 = 55 = 𝑏𝑆 = 134
𝜕𝑥𝑇 𝜕𝑥𝑆
This says for every 𝑏𝑇 = 55 coded units that we move by in 𝑥𝑇 we should also move 𝑥𝑆 by 𝑏𝑆 = 134
coded units. Mathematically:

Δ𝑥𝑆 134
=
Δ𝑥𝑇 55
The simplest way to do this is just to pick a move size in one of the variables, then change the move
size of the other one.

5.11. Response surface methods 367

Process Improvement Using Data

So we will choose to increase Δ𝑥𝑇 = 1 coded unit, which means:

Δ𝑥𝑇 = 1
Δ𝑥𝑇,actual = 5 K
𝑏𝑆 134
Δ𝑥𝑆 = Δ𝑥𝑇 = Δ𝑥𝑇
𝑏𝑇 55
𝑥𝑆,actual
but we know that Δ𝑥𝑆 =
Δ𝑆 /2
134
Δ𝑥𝑆,actual = × 1 × Δ𝑆 /2 by equating previous 2 lines
55
134
Δ𝑥𝑆,actual = × 1 × 0.5/2 = 0.61 g/L
55
• 𝑇5 = 𝑇baseline + Δ𝑥𝑇,actual = 325 + 5 = 330 K

• 𝑆5 = 𝑆baseline + Δ𝑥𝑆,actual = 0.75 + 0.6 = 1.36 g/L

So when we run the next experiment at these conditions. The daily profit is 𝑦5 = $ 669, improving
quite substantially from the baseline case.

We decide to make another move, in the same direction of steepest ascent, i.e. along the vector that
points in the 134
55 direction. We move the temperature up 5K, although we could have used a larger or
smaller step size if we wanted:

• 𝑇6 = 𝑇5 + Δ𝑥𝑇,actual = 330 + 5 = 335 K

• 𝑆6 = 𝑆5 + Δ𝑥𝑆,actual = 1.36 + 0.61 = 1.97 g/L

Again, we determine profit at 𝑦6 = $ 688. It is still increasing, but not by nearly as much. Perhaps we
are starting to level off. However, we still decide to move temperature up by another 5 K and increase
the substrate concentration in the required ratio:

• 𝑇7 = 𝑇6 + Δ𝑥𝑇,actual = 335 + 5 = 340 K

• 𝑆7 = 𝑆6 + Δ𝑥𝑆,actual = 1.97 + 0.61 = 2.58 g/L

The profit at this point is 𝑦7 = $ 463. We have gone too far as profit has dropped off. So we return back
to our last best point, because the surface has obviously changed, and we should refit our model with a
new factorial in this neighbourhood:

Experiment T (actual) S (actual) T S Profit

6 335 K 1.97 g/L 0 0 688
8 331 K 1.77 g/L − − 694
9 339 K 1.77 g/L + − 725
10 331 K 2.17 g/L − + 620
11 339 K 2.17 g/L + + 642

This time we have deciding to slightly smaller ranges in the factorial range𝑇 = 8 = (339 − 331) K and
range𝑆 = 0.4 = (2.17 − 1.77) g/L so that we can move more slowly along the surface.

368 Chapter 5. Design and Analysis of Experiments

 Release 547b16

A least squares model from the 4 factorial points (experiments 8, 9, 10, 11, run in random order), seems
to show that the promising direction now is to increase temperature but decrease the substrate
concentration.

𝑦^ = 𝑏0 + 𝑏𝑇 𝑥𝑇 + 𝑏𝑆 𝑥𝑆 + 𝑏𝑇 𝑆 𝑥𝑇 𝑥𝑆
𝑦^ = 673.8 + 13.25𝑥𝑇 − 39.25𝑥𝑆 − 2.25𝑥𝑇 𝑥𝑆

As before we take a step in the direction of steepest ascent of 𝑏𝑇 units along the 𝑥𝑇 direction and 𝑏𝑆
units along the 𝑥𝑆 direction. Again we choose Δ𝑥𝑇 = 1 unit, though we must emphasize that we
could used a smaller or larger amount, if desired.

Δ𝑥𝑆 −39
=
Δ𝑥𝑇 13
−39
Δ𝑥𝑆 = ×1
13
−39
Δ𝑥𝑆,actual = × 1 × 0.4/2 = −0.6 g/L
13
Δ𝑥𝑇,actual = 4 K

• 𝑇12 = 𝑇6 + Δ𝑥𝑇,actual = 335 + 4 = 339K

• 𝑆12 = 𝑆6 + Δ𝑥𝑆,actual = 1.97 - 0.6 = 1.37 g/L

We determine that at run 12 the profit is 𝑦12 = $ 716. But our previous factorial had a profit value of
$725 on one of the corners. Now it could be that we have a noisy system; after all, the difference
between $716 and $725 is not too much, but there is a relatively large difference in profit between the
other points in the factorial.

One must realize that as one approaches an optimum we will find:

5.11. Response surface methods 369

Process Improvement Using Data

• The response variable will start to plateau, since, recall that the first derivative is zero at an
optimum, implying the surface flattens out, and all points, in all directions away from the optimum
are worse off.

• The response variable remains roughly constant for two consecutive jumps, because one has
jumped over the optimum.

• The response variable decreases, sometimes very rapidly, because we have overshot the optimum.

• The presence of curvature can also be inferred when interaction terms are similar or larger in
magnitude than the main effect terms.

An optimum therefore exhibits curvature, so a model that only has linear terms in it will not be
suitable to use to find the direction of steepest ascent along the true response surface. We must add
terms that account for this curvature.

Checking for curvature

The factorial’s center point can be predicted from (𝑥𝑇 , 𝑥𝑆 ) = (0, 0), and is just the intercept term. In the
last factorial, the predicted center point was 𝑦^cp = $670; yet the actual center point from run 6 showed a
profit of $ 688. This is a difference of $18, which is substantial when compared to the main effects’
coefficients, particularly of temperature.

So when the measured center point value is quite different from the predicted center point in the linear
model, then that is a good indication there is curvature in the response surface. The way to
accommodate for that is to add quadratic terms to the estimate model.

Adding higher-order terms using central composite designs

We will not go into too much detail about central composite designs, other than to show what they
look like for the case of 2 and 3 variables. These designs take an existing orthogonal factorial and
augment it with axial points. This is great, because we can start off with an ordinary factorial and
always come back later to add the terms to account for nonlinearity.

The axial points are placed 40.25 = 1.4 coded units away from the center for a 2 factor system, and
80.25 = 1.7 units away for a 𝑘 = 3 factor system. Rules for higher numbers of factors, and the reasoning
behind the 1.4 and 1.7 unit step size can be found, for example in the textbook by Box, Hunter and
Hunter.

So a central composite design layout was added to the factorial in the above example and the
experiments run, randomly, at the 4 axial points.

370 Chapter 5. Design and Analysis of Experiments

 Release 547b16

The four response values were 𝑦13 = 720, 𝑦14 = 699, 𝑦15 = 610, and 𝑦16 = 663. This allows us to
estimate a model with quadratic terms in it: 𝑦 = 𝑏0 + 𝑏𝑇 𝑥𝑇 + 𝑏𝑆 𝑥𝑆 + 𝑏𝑇 𝑆 𝑥𝑇 𝑥𝑆 + 𝑏𝑇 𝑇 𝑥2𝑇 + 𝑏𝑆𝑆 𝑥2𝑆 . The
parameters in this model are found in the usual way, using a least-squares model:

y = Xb + e
⎡ ⎤ ⎡ ⎤
𝑦8 1 −1 −1 +1 +1 +1
⎢ 𝑦9 ⎥ ⎢1 +1 −1 −1 +1 +1⎥ ⎡
⎢ ⎥ ⎢ ⎥ ⎤
⎢ ⎥ ⎢ ⎥ 𝑏0
⎢𝑦10 ⎥ ⎢1 −1 +1 −1 +1 +1⎥ ⎢
⎥
⎢ 𝑏𝑇 ⎥
⎢ ⎥ ⎢ ⎥
⎢𝑦 ⎥ ⎢1 +1 +1 +1 +1 +1⎥
⎢ 11 ⎥ ⎢ ⎥⎢ 𝑏 ⎥
⎢
𝑆 ⎥
⎢ 𝑦6 ⎥ = ⎢1 0 0 0 0 0 ⎥⎢ ⎥+e
⎢ ⎥ ⎢ ⎥
⎢ ⎥ ⎢ ⎥⎢ 𝑏𝑇 𝑆 ⎥
⎢𝑦13 ⎥ ⎢1 0 −1.41 0 0 2⎥ ⎢ ⎥
⎥⎢ 𝑏
⎥
𝑇 𝑇
⎢ ⎥ ⎢
⎢𝑦 ⎥ ⎢1 1.41 0 0 2 0⎥
⎣ ⎦
⎢ 14 ⎥ ⎢ ⎥ 𝑏
⎥ 𝑆𝑆
⎣𝑦15 ⎦ ⎣1 0 1.41 0 0 2⎦
⎢ ⎥ ⎢
𝑦16 1 −1.41 0 0 2 0
𝑦 = 688 + 13𝑥𝑇 − 39𝑥𝑆 − 2.4𝑥𝑇 𝑥𝑆 − 4.2𝑥2𝑇 − 12.2𝑥2𝑆

Notice how the linear terms estimated previously are the same! The quadratic effects are clearly
significant when compared to the other effects, which was what prevented us from successfully using
a linear model to project out to point 12 previously.

The final step in the response surface methodology is to plot this model’s contour plot and predict
where to run the next few experiments. As the solid contour lines in the illustration show, we should
run our next experiments roughly at 𝑇 = 343K and 𝑆 = 1.60 g/L where the expected profit is around
$736. We get those two values by eye-balling the solid contour lines, drawn from the above non-linear
model. You could find this point analytically as well.

This is not exactly where the true process optimum is, but it is pretty close to it (the temperature of 𝑇 =
343K is just a little lower that where the true optimum is.

This example has demonstrated how powerful response surface methods are. A minimal number of
experiments has quickly converged onto the true, unknown process optimum. We achieved this by
building successive least squares models that approximate the underlying surface. Those least squares
models are built using the tools of fractional and full factorials and basic optimization theory, to climb
the hill of steepest ascent.
Video for
this section
5.11.2 The general approach for response surface modelling
1. Start at your baseline conditions and identify the main factors based on physics of the process,
operator input, expert opinion input, and intuition. Also be aware of any constraints, especially for
safe process operation. Perform factorial experiments (full or fractional factorials), completely
randomized. Use the results from the experiment to estimate a linear model of the system:

𝑦^ = 𝑏0 + 𝑏𝐴 𝑥𝐴 + 𝑏𝐵 𝑥𝐵 + 𝑏𝐶 𝑥𝐶 . . . + 𝑏𝐴𝐵 𝑥𝐴 𝑥𝐵 + 𝑏𝐴𝐶 𝑥𝐴 𝑥𝐶 + . . .

2. The main effects are usually significantly larger than the two-factor interactions, so these higher
interaction terms can be safely ignored. Any main effects that are not significant may be dropped
for future iterations.

3. Use the model to estimate the path of steepest ascent (or descent if minimizing 𝑦):
𝜕 𝑦^ 𝜕 𝑦^
= 𝑏1 = 𝑏2 ...
𝜕𝑥1 𝜕𝑥2

5.11. Response surface methods 371

 Process Improvement Using Data

The path of steepest ascent is climbed. Move any one of the main effects, e.g. 𝑏𝐴 by a certain
amount, Δ𝑥𝐴 . Then move the other effects: Δ𝑥𝑖 = 𝑏𝑏𝐴𝑖 Δ𝑥𝐴 . For example, Δ𝑥𝐶 is moved by 𝑏𝑏𝐶𝐴 Δ𝑥𝐴 .

If any of the Δ𝑥𝑖 values are too large to safely implement, then take a smaller proportional step in
all factors. Recall that these are coded units, so unscale them to obtain the move amount in
real-world units.

4. One can make several sequential steps until the response starts to level off, or if you become certain
you have entered a different operating mode of the process.

5. At this point you repeat the factorial experiment from step 1, making the last best response value
your new baseline. This is also a good point to reintroduce factors that you may have omitted
earlier. Also, if you have a binary factor; investigate the effect of alternating its sign at this point.
These additional factorial experiments should also include center points.

6. Repeat steps 1 through 5 until the linear model estimate starts to show evidence of curvature, or
that the interaction terms start to dominate the main effects. This indicates that you are reaching an
optimum.

• Curvature can be assessed by comparing the predicted center point, i.e. the model’s intercept =
𝑏0 , against the actual center point response(s). A large difference in the prediction, when
compared to the model’s effects, indicates the response surface is curved.

7. If there is curvature, add axial points to expand the factorial into a central composite design. Now
estimate a quadratic model of the form:

𝑦 = 𝑏0 + 𝑏1 𝑥1 + 𝑏2 𝑥2 + . . . + 𝑏12 𝑥1 𝑥2 + . . . + 𝑏11 𝑥21 + 𝑏22 𝑥22 + . . .

8. Draw contour plots of this estimated response surface (all data analysis software packages have
contour plotting functions) and determine where to place your sequential experiments. You can
also find the model’s optimum analytically by taking derivatives of the model function.

What is the response variable when optimizing more than one outcome?
Video for
this section Response surface methods consider optimization of a single outcome, or response variable, called
𝑦. In many instances we are interested in just a single response, but more often we are interested in
a multi-objective response, i.e. there are trade-offs. For example we can achieve a higher
production rate, but it is at the expense of more energy.

One way to balance all competing objectives is to rephrase the 𝑦 variable in terms of total costs, or
better still, net profit. This makes calculating the 𝑦 value more complex, as we have to know the
various costs and their relative weightings to calculate the profit. Now you have a single 𝑦 to work
with.

Another way is to superimpose the response surfaces of two or more 𝑦-variables. This is
tremendously helpful when discussing and evaluating alternate operating points, because plant
managers and operators can then visually see the trade-offs.

Summary

1. In the previous sections we used factorials and fractional factorials for screening the important
factors. When we move to process optimization, we are assuming that we have already identified
the important variables. In fact, we might find that variables that were previously important,

372 Chapter 5. Design and Analysis of Experiments

 Release 547b16

appear unimportant as we approach the optimum. Conversely, variables that might have been
dropped out earlier, become important at the optimum.

2. Response surface methods generally work best when the variables we adjust are numerically
continuous. Categorical variables (yes/no, catalyst A or B) are handled by fixing them at one or the
other value, and then performing the optimization conditional on those selected values. It is always
worth investigating the alternative values once the optimum has been reached.

3. Many software packages provide tools that help with an RSM study. If you would like to use R in
your work, we highly recommend the rsm package by Russel Lenth, available in R. You can read
more about the package in this article133 as well as a case-study134 .

5.12 Evolutionary operation

Evolutionary operation (EVOP) is a tool to help maintain a full-scale process at its optimum. Since the
process is not constant, the optimum will gradually move away from its current operating point.
Chemical processes drift due to things such as heat-exchanger fouling, build-up inside reactors and
tubing, catalyst deactivation, and other slowly varying disturbances in the system.

EVOP is an iterative hunt for the process optimum by making small perturbations to the system.
Similar to response surface methods, once every iteration is completed, the process is moved towards
the optimum. The model used to determine the move direction and levels of next operation are from
full or fractional factorials, or designs that estimate curvature, like the central composite design.

Because every experimental run is a run that is expected to produce saleable product (we don’t want
off-specification product), the range over which each factor is varied must be small. Replicate runs are
also made to separate the signal from noise, because the optimum region is usually flat.

Some examples of the success of EVOP and a review paper are in these readings:

• George Box: Evolutionary Operation: A Method for Increasing Industrial Productivity135 ”, Journal of
the Royal Statistical Society (Applied Statistics), 6, 81 - 101, 1957.

• William G. Hunter and J. R. Kittrell, “Evolutionary Operation: A Review136 ”, Technometrics, 8,

389-397, 1966.

Current day examples of EVOP do not appear in the scientific literature much, because this
methodology is now so well established.

5.13 General approach for experimentation

We complete this section with some guidance for experimentation in general. The main point is that
experiments are never run in one go. You will always have more questions after the first round. Box,
Hunter and Hunter provide two pieces of guidance on this:

1. The best time to run an experiment is after the experiment. You will discover things from the
previous experiment that you wish you had considered the first time around.

2. For the above reason, do not spend more than 20% to 25% of your time and budget on your first
group of experiments. Keep some time aside to add more experiments and learn more about the
system.
133 [Link]
134 [Link]
135 [Link]
136 [Link]

5.12. Evolutionary operation 373

Process Improvement Using Data

The first phase is usually screening. Screening designs are used when developing new products and
tremendous uncertainty exists; or sometimes when a system is operating so poorly that one receives
the go-ahead to manipulate the operating conditions wide enough to potentially upset the process, but
learn from it.

• The ranges for each factor may also be uncertain; this is a perfect opportunity to identify suitable
ranges for each factor.

• You also learn how to run the experiment on this system, as the operating protocol isn’t always
certain ahead of time. It is advisable to choose your first experiment to be the center point, since the
first experiment will often “fail” for a variety of reasons (you discover that you need more
equipment midway, you realize the protocol isn’t sufficient, etc). Since the center point is not
required to analyze the data, it worth using that first run to learn about your system. If successful
though, that center point run can be included in the least squares model.

• Include as many factors into as few runs as possible. Use a saturated, resolution III design, or a
Plackett and Burman design.

• Main effects will be extremely confounded, but this is a preliminary investigation to isolate the
important effects.

The second phase is to add sequential experiments to the previous experiments.

• Use the concept of foldover: switching the sign of the factor of interest to learn more about a single
factor, or switch all signs to increase the design’s resolution.

• If you had a prior screening experiment, use the concept of projectivity in the important factors to
limit the number of additional experiments required.

• Move to quarter and half-fractions of higher resolution, to better understand the main effects and
2-factor interactions.

The third phase is to start optimizing by exploring the response surface using the important variables
discovered in the prior phases.

• Use full or half-fraction factorials to estimate the direction of steepest ascent or descent.

• Once you reach the optimum, then second order effects and curvature must be added to assess the
direction of the optimum.

The fourth phase is to maintain the process optimum using the concepts of evolutionary operation
(EVOP).

• An EVOP program should always be in place on a process, because raw materials change, fouling
and catalyst deactivation take place, and other slow moving disturbances have an effect. You
should be always hunting for the process optimum.

5.14 Extended topics related to designed experiments

This section is just an overview of some interesting topics, together with references to guide you to
more information.

374 Chapter 5. Design and Analysis of Experiments

 Release 547b16

5.14.1 Experiments with mistakes, missing values, or belatedly discovered constraints

Video for
this section Many real experiments do not go smoothly. Once the experimenter has established their −1 and +1
levels for each variable, they back that out to real units. For example, if temperature was scaled as
𝑇actual − 450K
𝑇 = , then 𝑇 = −1 corresponds to 425K and 𝑇 = +1 corresponds to 475K.
25K
But if the operator mistakenly sets the temperature to 𝑇actual = 465𝐾, then it doesn’t quite reach the +1
465 − 450
level required. This is not a wasted experiment. Simply code this as 𝑇 = = 0.6, and enter
25
that value in the least squares model for matrix X. Then proceed to calculate the model parameters
using the standard least squares equations. Note that the columns in the X-matrix will not be
orthogonal anymore, so X𝑇 X will not be a diagonal matrix, but it will be almost diagonal.

Similarly, it might be discovered that temperature cannot be set to 475K when the other factor, for
example concentration, is also at its high level. This might be due to physical or safety constraints. On
the other hand, 𝑇 = 475𝐾 can be used when concentration is at its low level. This case is the same as
described above: set the temperature to the closest possible value for that experiment, and then
analyze the data using a least squares model. The case when the constraint is known ahead of time is
dealt with later on (page 375), but in this case, the constraint was discovered just as the run was to be
performed.

Also see the section on optimal designs (page 376) for how one can add one or more additional
experiments to fix an existing bad set of experiments.

The other case that happens occasionally is that samples are lost, or the final response value is missing
for some reason. Not everything is lost: recall the main effects for a full 2𝑘 factorial are estimated 𝑘
times at each combination of the factors (page 344).

If one or more experiments have missing 𝑦 values, you can still estimate these main effects, and
sometimes the interaction parameters by hand. Furthermore, analyzing the data in a least squares
model will be an undetermined system: more unknowns than equations. You could choose to drop out
higher-order interaction terms to reduce the equations to a square system: as many unknowns as
equations. Then proceed to analyze the results from the least squares model as usual. There are
actually slightly more sophisticated ways of dealing with this problem, as described by Norman
Draper in “Missing Values in Response Surface Designs137 ”, Technometrics, 3, 389-398, 1961.

The above discussion illustrates clearly our preference for using the least squares model: whether the
experimental design was executed accurately or not: the least squares model always works, whereas
the short cut tools (page 330) developed for perfectly executed experiments will fail.

5.14.2 Handling of constraints

Most engineering systems have limits of performance, either by design or from a safety standpoint. It
is also common that optimum production levels are found close to these constraints. The factorials we
use in our experiments must, by necessity, span a wide range of operation so that we see systematic
change in our response variables, and not merely measure noise. These large ranges that we choose for
the factors often hit up again constraints.

A simple bioreactor example for 2 factors is shown: at high temperatures and high substrate
concentrations we risk activating a different, undesirable side-reaction. The shaded region represents
the constraint where we may not operate. We could for example replace the (𝑇+ , 𝐶+ ) experiment with
two others, and then analyze these 5 runs using least squares.
137 [Link]

5.14. Extended topics related to designed experiments 375

Process Improvement Using Data

Unfortunately, these 5 runs do not form an orthogonal (independent) X matrix anymore. We have lost
orthogonality. We have also reduced the space (or volume when we have 3 or more factors) spanned
by the factorial design.

It is easy to find experiments that obey the constraints for 2-factor cases: run them on the corner
points. But for 3 or more factors the constraints form planes that cut through a cube. We then use
optimal designs (page 376) to determine where to place our experiments. A D-optimal design works
well for constraint-handling because it finds the experimental points that would minimize the loss of
orthogonality (i.e. they try to achieve the most orthogonal design possible). A compact way of stating
this is to maximize the determinant of X𝑇 X, which is why it is called D-optimal (it maximizes the
determinant).

These designs are generated by a computer, using iterative algorithms. See the D-optimal reference in
the section on optimal designs (page 376) for more information.

5.14.3 Optimal designs

If you delve into the modern literature on experimental methods you will rapidly come across the
concept of an optimal design. This begs the question, what is sub-optimal about the factorial designs
we have focussed on so far?

A full factorial design spans the maximal space possible for the 𝑘 factors. From least squares
modelling we know that large deviations from the model center reduces the variance of the parameter
estimates. Furthermore, a factorial ensures the factors are moved independently, allowing us to
estimate their effects independently as well. These are all “optimal” aspects of a factorial.

So again, what is sub-optimal about a factorial design? A factorial design is an excellent design in most
cases. But if there are constraints that must be obeyed, or if the experimenter has an established list of
possible experimental points to run, but must choose a subset from the list, then an “optimal” design is
useful.

All an optimal design does is select the experimental points by optimizing some criterion, subject to
constraints. Some examples:

• The design region is a cube with a diagonal slice cut-off on two corner due to constraints. What is
the design that spans the maximum volume of the remaining cube?

• The experimenter wishes to estimate a non-standard model, e.g.

376 Chapter 5. Design and Analysis of Experiments

 Release 547b16

𝑑𝑥 +𝑒
− 𝑓 𝑥A +𝑔
𝑦 = 𝑏0 + 𝑏A 𝑥A + 𝑏AB 𝑥AB + 𝑏B 𝑥B + 𝑏AB exp B for fixed values of 𝑑, 𝑒, 𝑓 and 𝑔.

• For a central composite design, or even a factorial design with constraints, find a smaller number of
experiments than required for the full design, e.g. say 14 experiments (a number that is not a power
of 2).

• The user might want to investigate more than 2 levels in each factor.

• The experimenter has already run 𝑛 experiments, but wants to add one or more additional
experiments to improve the parameter estimates, i.e. decrease the variance of the parameters. In the
case of a D-optimal design, this would find which additional experiment(s) would most increase the
determinant of the X𝑇 X matrix.

The general approach with optimal designs is

1. The user specifies the model (i.e. the parameters).

2. The computer finds all possible combinations of factor levels that satisfy the constraints, including
center-points. These are now called the candidate points or candidate set, represented as a long list of
all possible experiments. The user can add extra experiments they would like to run to this list.

3. The user specifies a small number of experiments they would actually like to run.

4. The computer algorithm finds this required number of runs by picking entries from the list so that
those few runs optimize the chosen criterion.

The most common optimality criteria are:

• A-optimal designs minimize the average variance of the parameters, i.e. minimizes
trace (X𝑇 X)−1
{︀ }︀

(︀ )︀
• D-optimal designs minimize the general variance of the parameters, i.e. maximize det X𝑇 X

• G-optimal designs minimize the maximum variance of the predictions

• V-optimal designs minimize the average variance of the predictions

It must be pointed out that a full factorial design, 2𝑘 is already A-, D- G- and V-optimal. Also notice
that for optimal designs the user must specify the model required. This is actually no different to
factorial and central composite designs, where the model is implicit in the design.

The algorithms used to find the subset of experiments to run are called candidate exchange algorithms.
They are usually just a brute force evaluation of the objective function by trying all possible
combinations. They bring a new trial combination into the set, calculate the objective value for the
criterion, and then iterate until the final candidate set provides the best objective function value.

Readings

• St. John and Draper: “D-Optimality for Regression Designs: A Review138 ”, Technometrics, 17, 15-,
1975.

5.14.4 Definitive screening designs

The final type of design to be aware of is a class of designs called the definitive screening design, and
below is a link that you can read up some more information.

These designs are a type of optimal design (page 376). Optimal designs can be very flexible. For
example, if you had a limited budget you can create an optimal design for a given number of factors
138 [Link]

5.14. Extended topics related to designed experiments 377

Process Improvement Using Data

you are investigating to maximize one of these optimality criteria to fit your budget. A computer
algorithm is used to find the settings for each one of the budgeted number of runs, so that the
optimization criterion is maximized. In other words the computer is designing the experiments for
you, so they have some very distinct advantages.

The readings below give more details, and a practical implementation of these designs using the R
software package.

Readings

• John Lawson “DefScreen: Definitive Screening Designs, in package “daewr”: Design and Analysis
of Experiments with R139 ”.

• Bradley Jones: “Class of Three-Level Designs for Definitive Screening in the Presence of
Second-Order Effects140 ”, Journal of Quality Technology, 2011.

5.14.5 Mixture designs

The area of mixture designs is incredibly important for optimizing recipes, particularly in the area of
fine chemicals, pharmaceuticals, food manufacturing, and polymer processing. Like factorial designs,
there are screening and optimization designs for mixtures also.

A mixture design is required when the factors being varied add up to 100% to form a mixture. Then
these factors cannot be adjusted in an independent, factorial-like manner, since their proportion in the
∑︀
recipe must add to 100%: 𝑖 𝑥𝑖 = 1. These designs result in triangular patterns (called simplexes). The
experimental points at the 3 vertices are for pure components 𝑥𝐴 , 𝑥𝐵 , or 𝑥𝐶 . Points along the sides
represent a 2-component mixture, and points in the interior represent a 3-component blend.

In the above figure on the right, the shaded region represents a constraint that cannot be operated in.
A D-optimal algorithm must then be used to select experiments in the remaining region. The example
is for finding the lowest cost mixture for a fruit punch, while still meeting certain taste requirements
(e.g. watermelon juice is cheap, but has little taste). The constraint represents a region where the
acidity is too high.
139 [Link]
140 [Link]

378 Chapter 5. Design and Analysis of Experiments

 Release 547b16

5.15 Exercises

ò Question

These readings are to illustrate the profound effect that designed experiments have had in some
areas.

• Application of Statistical Design of Experiments Methods in Drug Discovery141 and using DOE
for high-throughput screening to locate new drug compounds142 .

• High traffic websites offer a unique opportunity to perform testing and optimization. This is
because each visitor to the site is independent of the others (randomized), and these tests can be
run in parallel. Read more in this brief writeup143 on how Google uses testing tools to optimize
YouTube, one of their web properties. Unfortunately they use the term “multivariate” incorrectly
- a better term is “multi-variable”; nevertheless, the number of factors and combinations to be
tested is large. It’s well known that fractional factorial methods are used to analyze these data.

• See three chemical engineering examples of factorial designs in Box, Hunter, and Hunter:
Chapter 11 (1st edition), or page 173 to 183 in the second edition.

ò Question

Your family runs a small business selling low dollar value products over the web. They want to
improve sales. There is a known effect from the day of the week, so to avoid that effect they run the
following designed experiment every Tuesday for the past eight weeks. The first factor of interest is
whether to provide free shipping over $30 or over $50. The second factor is whether or not the
purchaser must first create a profile (user name, password, address, etc) before completing the
transaction. The purchaser can still complete their transaction without creating a profile.

These are the data collected:

Date Free shipping over . . . Profile required before Total sales made
transaction
05 January 2010 $30 Yes $ 3275
12 January 2010 $50 No $ 3594
19 January 2010 $50 No $ 3626
26 January 2010 $30 No $ 3438
02 February 2010 $50 Yes $ 2439
09 February 2010 $30 No $ 3562
16 February 2010 $30 Yes $ 2965
23 February 2010 $50 Yes $ 2571

1. Calculate the average response from replicate experiments to calculate the 4 corner points.

2. Calculate and interpret the main effects in the design.

3. Show the interaction plot for the 2 factors.

141 [Link]
142 [Link]
143 [Link]

5.15. Exercises 379

Process Improvement Using Data

4. We will show in the next class how to calculate confidence intervals for each effect, but
would you say there is an interaction effect here? How would you interpret the interaction
(whether there is one or not)?

5. What is the recommendation to increase sales?

6. Calculate the main effects and interactions by hand using a least squares model. You may
confirm your result using software, but your answer should not just be the computer
software output.

ò Solution

1. This is a 22 factorial system with a replicate at each point. We might not have covered replicates
(page 341) in class by the time you had to do this assignment. So you should average the
replicate points and then calculate the main effects and other terms for this system. You will get
the same result if you analyze it as two separate factorials and then average the results - it’s just
more work that way though.

2. The experiment results in standard form with 4 corner points:

A B Average sales
1
− − 2 (3438 + 3562) = $3, 500
1
+ − 2 (3594 + 3626) = $3, 610
1
− + 2 (3275 + 2965) = $3, 120
1
+ + 2 (2439 + 2571) = $2, 505

where A = free shipping over $30 (low level) and $50 (high level), and let B = −1 if no profile is
required, or +1 if a profile is required before completing the transaction.
−505
• The main effect for free shipping (A) is = 12 (3610 − 3500 + 2505 − 3120) = = −252.50
2
This indicates that sales decrease by $252.50, on average, when going from free shipping over
$30 to $50. One might expect, within reason, that higher sales are made when the free
shipping value is higher (people add purchases so they reach the free shipping limit). That is
shown by the very small effect of $50 when no profile is required. However when a profile is
required, we see the opposite: a drop in sales!
−1485
• The main effect of creating a profile B = 21 (3120 − 3500 + 2505 − 3610) = = −742.50
2
Indicating that sales drop by $742.50 on average when requiring a profile before completing
the transaction vs not requiring a profile. The drop in sales is less when offering free shipping
over $30 than when free shipping is for $50 or more in purchases.

Not required for this question, but one of the best ways to visualize a small factorial, or a subset
of a larger factorial, is with a cube plot:

380 Chapter 5. Design and Analysis of Experiments

 Release 547b16

3. The interaction plot which visually shows the main effects described above is:

4. The interaction term can be calculated in two ways, both giving the same answer. Only one way
is shown here:

• A at high B: -$615.00

• A at low B: $ 110.00
1 −725
• AB interaction = 2 (−615 − 110) = = −362.50
2
This interaction term is larger than one of the main effects, so I would judge this to be important.
Also, it is roughly 10% of the 𝑦𝑖 = daily sales values, so it is definitely important.

In part 1 we showed the main effect of requiring a profile is to decrease sales. The strong
negative interaction term here indicates that sales are even further reduced when free shipping
is over $50, rather than $30. Maybe it’s because customers “give up” making their purchase
when free shipping is at a higher amount and they need to create a profile - perhaps they figure
this isn’t worth it. If they get free shipping over $30, the penalty of creating a profile is not as
great anymore. This last effect might be considered counterintuitive - but I’m not an expert on
why people buy stuff.

In general, an interaction term indicates that the usual main effects are increased or decreased
more or less than they would have been when acting on their own.

5. Sales can be definitely increased by not requiring the user to create a profile before completing

5.15. Exercises 381

Process Improvement Using Data

the transaction (creating a profile is a strong deterrent to increasing sales, whether free shipping
over $30 or $50 is offered). The effect of free shipping when not requiring a profile is small. The
raw data for the case when no profile was required (below), show slightly higher sales when free
shipping over $50 is required. Further experimentation to assess if this is significant or not
would be required.

Date Free shipping over Profile required before Total sales made that
... transaction day
12 January 2010 $50 No $ 3594
19 January 2010 $50 No $ 3626
26 January 2010 $30 No $ 3438
09 February 2010 $30 No $ 3562

6. A least squares model can be calculated from the average of each replicate. Then there are 4
observations and 4 unknowns. Using the design matrix, in standard order, we can set up the
following least squares model:

y = Xb + e
⎡ ⎤ ⎡ ⎤⎡ ⎤ ⎡ ⎤
𝑦1 1 −1 −1 +1 𝑏0 𝑒1
⎢𝑦 ⎥ ⎢1 +1
⎢ 2⎥ ⎢ −1 −1⎥ ⎢ 𝑏A ⎥ ⎢𝑒2 ⎥
⎥ ⎢ ⎥ ⎢
⎢ ⎥=⎢ ⎥+⎢ ⎥
⎥
⎣𝑦3 ⎦ ⎣1 −1 +1 −1⎦ ⎣ 𝑏A ⎦ ⎣𝑒3 ⎦
⎥⎢

𝑦4 1 +1 +1 +1 𝑏AB 𝑒4
⎡ ⎤ ⎡ ⎤⎡ ⎤ ⎡ ⎤
3500 1 −1 −1 +1 𝑏0 𝑒1
⎢3610⎥ ⎢1 +1 −1 −1⎥ ⎢ 𝑏A ⎥ ⎢𝑒2 ⎥
⎥ ⎢ ⎥ ⎢
⎥=⎢ ⎥+⎢ ⎥
⎢ ⎥ ⎢ ⎥
⎣3120⎦ ⎣1 −1 +1 −1⎦ ⎣ 𝑏A ⎦ ⎣𝑒3 ⎦
⎢ ⎥⎢

2505 1 +1 +1 +1 𝑏AB 𝑒4

And solving the regression coefficients (note the orthogonality in the X𝑇 X matrix):
)︀−1 𝑇
b = X𝑇 X
(︀
X y
⎛ ⎞−1 ⎡ ⎤
4 0 0 0 +3500 + 3610 + 3120 + 2505
⎜0 4 0 0⎟ ⎢−3500 + 3610 − 3120 + 2505⎥
b=⎜
⎜ ⎟ ⎢ ⎥
⎝0 0 4 0⎠ ⎣−3500 − 3610 + 3120 + 2505⎦
⎟ ⎢ ⎥

0 0 0 4 +3500 − 3610 − 3120 + 2505

⎡1 ⎤⎡ ⎤
4 0 0 0 12735
⎢ 0 1 0 0 ⎥ ⎢ −505 ⎥
b=⎢
⎢ 4 ⎥⎢ ⎥
⎣ 0 0 41 0 ⎦ ⎣−1485⎦
⎥⎢ ⎥

0 0 0 14 −725
⎡ ⎤ ⎡ ⎤
𝑏0 3184
⎢ 𝑏 ⎥ ⎢−126⎥
⎢ A⎥ ⎢
⎥=⎢
⎥
⎣ 𝑏A ⎦ ⎣−371⎦
⎢ ⎥

𝑏AB −181
The final model is 𝑦 = 3184 − 126𝑥A − 371𝑥B − 181𝑥AB .

Compare the values in the X𝑇 y vector to the calculations for the main effects and interactions to
see the similarity. The least squares model parameters are half the size of the main effects and
interactions reported above, because of how the parameters are interpreted in the least squares
model.

382 Chapter 5. Design and Analysis of Experiments

 Release 547b16

Particularly the effect of requiring a profile, 𝑥𝐵 , is to reduce sales by 2×371 = 742.

ò Question

More readings:

1. It is worth reading this paper by Bisgaard to see how the same tools shown in these notes were
used to solve a real industrial problem: designed experiments, autocorrelation plots, data
visualization, and quality control charts. Also he describes how the very real pressure from
managers, time-constraints and interactions with team-members impacted the work.

“The Quality Detective: A Case Study144 ” (and discussion), Philosophical Transactions of the Royal
Society A, 327, 499-511, 1989.

2. George Box, The R. A. Fisher Memorial Lecture, 1988, “Quality Improvement - An Expanding
Domain for the Application of Scientific Method145 ”, Philosophical Transactions of the Royal Society
- A, 327: pages 617-630, 1989.

ò Question

ò Note

This is a tutorial-type question: all the sub-questions build on each other. All questions deal with
a hypothetical bioreactor system, and we are investigating four factors:
• A = feed rate: slow or medium
• B = initial inoculant size (300g or 700g)
• C = feed substrate concentration (40 g/L or 60 g/L)
• D = dissolved oxygen set-point (4mg/L or 6 mg/L)

The 16 experiments from a full factorial, 24 , were randomly run, and the yields from the bioreactor,
𝑦, are reported here in standard order: y = [60, 59, 63, 61, 69, 61, 94, 93, 56, 63, 70, 65, 44, 45, 78, 77].

1. Calculate the 15 main effects and interactions and the intercept, using computer software.

2. Use a Pareto-plot to identify the significant effects. What would be your advice to your
colleagues to improve the yield?

3. Refit the model using only the significant terms identified in the second question.

• Explain why you don’t actually have to recalculate the least squares model parameters.

• Compute the standard error and confirm that the effects are indeed significant at the 95%
level.

4. Write down the exact settings for A, B, C, and D you would provide to the graduate student
running a half-fraction in 8 runs for this system.

5. Before the half-fraction experiments are even run you can calculate which variables will be
confounded (aliased) with each other. Report the confounding pattern for these main effects and

144 [Link]
145 [Link]

5.15. Exercises 383

Process Improvement Using Data

for these two-factor interactions. Your answer should be in this format:

• Generator =

• Defining relationship =

• Confounding pattern:

– 𝛽̂︀A →

– 𝛽̂︀B →

– 𝛽̂︀C →

– 𝛽̂︀D →

– 𝛽̂︀AB →

– 𝛽̂︀AC →

– 𝛽̂︀AD →

– 𝛽̂︀BC →

– 𝛽̂︀BD →

– 𝛽̂︀CD →

6. Now use the 8 yield values corresponding to your half fraction, and calculate as many
parameters (intercept, main effects, interactions) as you can.

• Report their numeric values.

• Compare your parameters from this half-fraction (8 runs) to those from the full factorial (16
runs). Was much lost by running the half fraction?

• What was the resolution of the half-fraction?

• What is the projectivity of this half-fraction? And what does this mean in light of the fact that
factor A was shown to be unimportant?

• Factor C was found to be an important variable from the half-fraction; it had a significant
coefficient in the linear model, but it was aliased with ABD. Obviously in this problem, the
foldover set of experiments to run would be the other half-fraction. But we showed a way to
de-alias a main effect. Use that method to show that the other 8 experiments to de-alias factor
C would just be the other 8 experiment not included in your first half-fraction.

ò Solution

1. Using the computer code (at the end of the question), we found the complete model for all
effects and interaction as:
𝑦^ = 66 − 0.6𝑥𝐴 + 9𝑥𝐵 + 4𝑥𝐶 − 3.9𝑥𝐷 − 0.5𝑥𝐴 𝑥𝐵 − 0.5𝑥𝐴 𝑥𝐶 + 0.9𝑥𝐴 𝑥𝐷 + 6.4𝑥𝐵 𝑥𝐶 + 1.3𝑥𝐵 𝑥𝐷 − 5.3𝑥𝐶 𝑥𝐷
+ 1.1𝑥𝐴 𝑥𝐵 𝑥𝐶 − 1.2𝑥𝐴 𝑥𝐵 𝑥𝐷 + 0.3𝑥𝐴 𝑥𝐶 𝑥𝐷 − 0.1𝑥𝐵 𝑥𝐶 𝑥𝐷 + 0.1𝑥𝐴 𝑥𝐵 𝑥𝐶 𝑥𝐷

2. The Pareto plot shows the important main effects are B, C, D and these two-factor interactions:
BC and CD.

The advice to improve yield would be to:

384 Chapter 5. Design and Analysis of Experiments

 Release 547b16

• A: use either the slow or medium feedrate, whichever has the better process economics

• B: operate with the larger inoculant size: 700g

• C: use a higher feed concentration 60 g/L

• D: use the lower dissolved oxygen set point of 4 mg/L

• BC: in this case the BC interaction works in our favour (high × high)

• CD: the CD interaction also works in our favour, since -5.3 × (+1) × (-1) leads to an increased
yield.

At these conditions the expected yield is in the region of 93 to 94% (runs 7 and 8 from the
standard order).

3. The model does not have to be refitted because the columns in matrix X are orthogonal,
meaning that the coefficient estimates do not depend on the levels of any other variables.

By dropping out the insignificant coefficients and keeping only the 5 parameters from the Pareto
plus the intercept, we have 6 parameters, 16 data points, so 10 degrees of freedom. The residual
vector is found from e = y − y^ , where y
^ = Xsub bsub .
⏟ ⏞ ⏟ ⏞
16×6 6×1

The subset matrix of Xsub is found by sub-sampling from the full 16 × 16 matrix; similarly for
the coefficient vector b. From this we can calculate:

• The standard error is 𝑆𝐸 = 3.1, which is pretty tight, considering the ranges of y-values in the
data set

5.15. Exercises 385

Process Improvement Using Data

• The critical 𝑡-value for the 95% confidence level = 2.23

(︀ )︀−1 2
• The standard error for the parameters in the model is given by X𝑇 X 𝑆𝐸 . We can use this
form because
√︂ apart from the intercept column, each column is centered around zero. So
3.12
𝑆𝐸 (𝑏𝑖 ) = = 0.78.
16
• The confidence intervals for each of the significant effects are:

7.3 ≤ 𝛽𝐵 ≤ 10.7
2.3 ≤ 𝛽𝐶 ≤ 5.7
−5.6 ≤ 𝛽𝐷 ≤ −2.1
4.6 ≤ 𝛽𝐵𝐶 ≤ 8.1
−7.0 ≤ 𝛽𝐶𝐷 ≤ −3.5

4. A half-fraction of a 24 factorial has 8 experiments. We can generate the levels for 3 of the factors,
A, B and C from a full factorial in these 8 runs. The generating term for the fourth factor D is
best set to the highest level of confounding, the ABC term.

Using that concept, we would ask the graduate student to run these 8 experiments in random
order:

Experiment Feed rate Inoculant size Feed concentration DO set point

1 Slow 300g 40 g/L 4 mg/L
2 Medium 300g 40 g/L 6 mg/L
3 Slow 700g 40 g/L 6 mg/L
4 Medium 700g 40 g/L 4 mg/L
5 Slow 300g 60 g/L 6 mg/L
6 Medium 300g 60 g/L 4 mg/L
7 Slow 700g 60 g/L 4 mg/L
8 Medium 700g 60 g/L 6 mg/L

5. • Generator = D = ABC

• Defining relationship = I = ABCD

• Confounding pattern:

– 𝛽̂︀A → A + BCD

– 𝛽̂︀B → B + ACD

– 𝛽̂︀C → C + ABD

– 𝛽̂︀D → D + ABC

– 𝛽̂︀AB → AB + CD

– 𝛽̂︀AC → AC + BD

– 𝛽̂︀AD → AD + BC

– 𝛽̂︀BC → BC + AD

– 𝛽̂︀BD → BD + AC

386 Chapter 5. Design and Analysis of Experiments

 Release 547b16

– 𝛽̂︀CD → CD + AB

6. Selecting the rows from the full factorial design which correspond to the 8 runs from the half
factorial we get 𝑦 = [60, 63, 70, 61, 44, 61, 94, 77] corresponding to the table order in question 5.

Then forming the X matrix from the table in question 5 we solve for the parameters as follows:

• ̂︀𝑏0 = 66.25 → I + ABCD

• ̂︀𝑏A = −0.75 → A + BCD (previous estimate for A was -0.625)

• ̂︀𝑏B = 9.25 → B + ACD (previous estimate for B was 9.9)

• ̂︀𝑏C = 2.75 → C + ABD (previous estimate for C was 4.0)

• ̂︀𝑏D = −2.75 → D + ABC (previous estimate for A was -3.9)

• ̂︀𝑏AB = −5.75 → AB + CD (previous estimate for AB was insignificant, while CD was -5.25)

• ̂︀𝑏AC = 0.75 → AC + BD (previous estimates for both AC and BD were insignificant)

• ̂︀𝑏AD = 7.25 → AD + BC (previous estimate for AD was insignificant, while BC was 6.4)

You can verify for yourself that each coefficient from the half fraction is just the sum of the effects
estimated from the full factorial. For example, ̂︀𝑏AD = 7.25 → AD + BC = 0.875 + 6.375 = 7.25.

So these estimates from the half-fraction are comparable to the estimates from the full fraction.

R code for this question

# Generate the design matrix

A <- B <- C <- D <- c(-1, 1)
f <- [Link](A=A, B=B, C=C, D=D)
A <- f$A
B <- f$B
C <- f$C
D <- f$D
# Set the response values in standard order, and solve thefull factorial model
y <- c(60, 59, 63, 61, 69, 61, 94, 93, 56, 63, 70, 65, 44, 45, 78, 77)

[Link] <- lm(y ~ (A+B+C+D)^4)

b <- coef([Link])

# Pareto plot
[Link] <- coef([Link])[2:length(coef([Link]))]
library(lattice)

bitmap('[Link]', type="png256", width=8,

height=8, res=300, pointsize=14)
library(lattice)
coeff <- sort(abs([Link]), [Link]=TRUE)
barchart(coeff$x,
xlim=c(0, max(abs([Link]))+0.1),
xlab=list("Magnitude of effect", cex=1.5),
ylab = list("Effect", cex=1.5),
groups=([Link]>0)[coeff$ix], col=c("lightblue", "orange"),
scales=list(cex=1.5)
)
[Link]()

# Refit the model with only: B, C, D, BC, CD and intercept

[Link] <- lm(y ~ B + C + D + B*C + C*D)
summary([Link])

5.15. Exercises 387

Process Improvement Using Data

# and check confidence intervals

confint([Link])

# Half-fraction generated from D = A*B*C defining relationship I = ABCD

# Create a logical vector, indicating which subset of the full runs to use:
subset <- D == A*B*C
A.s <- A[subset]
B.s <- B[subset]
C.s <- C[subset]
D.s <- D[subset]

y[subset]

[Link] <- lm(y[subset] ~ A.s + B.s + C.s + D.s + A.s*B.s + A.s*C.s + A.s*D.s)
summary([Link])
[Link] <- coef([Link])[2:length(coef([Link]))]

bitmap('[Link]', type="png256", width=8,

height=8, res=300, pointsize=14)

coeff <- sort(abs([Link]), [Link]=TRUE)

barchart(coeff$x,
xlim=c(0, max(abs([Link]))+0.1),
xlab=list("Magnitude of effect", cex=1.5),
ylab = list("Effect", cex=1.5),
groups=([Link]>0)[coeff$ix], col=c("lightblue", "orange"),
scales=list(cex=1.5)
)
[Link]()

ò Question

Your group is developing a new product, but have been struggling to get the product’s stability,
measured in days, to the level required. You are aiming for a stability value of 50 days or more.
Four factors have been considered:

• A = monomer concentration: 30% or 50%

• B = acid concentration: low or high

• C = catalyst level: 2% or 3%

• D = temperature: 393K or 423K

These eight experiments have been run so far:

Experiment Order A B C D Stability

1 5 − − − − 40
2 6 + − − + 27
3 1 − + − + 35
4 4 + + − − 21
5 2 − − + + 39
6 7 + − + − 27
7 3 − + + − 27
8 8 + + + + 20

388 Chapter 5. Design and Analysis of Experiments

 Release 547b16

Where would you run the next experiment to try get the stability above 50 or greater?

ò Question

The following diagram shows data from a central composite design. The factors were run at their
standard levels, and there were 4 runs at the center point.

1. Calculate the parameters for a suitable quadratic model in these factors. Show your matrices for
X and y.

2. Draw a response surface plot of A vs B over a suitably wide range beyond the experimental
region.

3. Where would you move A and B if your objective is to increase the response value?

1. Report your answer in coded units.

2. Report your answer in real-world units, if the full factorial portion of the experiments were
ran at:

• A = stirrer speed, 200rpm and 340 rpm

• B = stirring time, 30 minutes and 40 minutes

You might feel more comfortable setting up the problem in MATLAB. You can use the contour
plot146 functions in MATLAB to visualize the results.

If you are using R, you can use the rbind(...) or cbind(...) functions to build up your X
matrix row-by-row or column-by-column. The equivalent of meshgrid in R is the
[Link](...) function. See the R code on the course website147 that shows how to generate
surface plots in R.

146 [Link]
147 [Link]

5.15. Exercises 389

Process Improvement Using Data

ò Question

A full 23 factorial was run as shown:

Experiment A B C
1 30% 232 Larry
2 50% 232 Larry
3 30% 412 Larry
4 50% 412 Larry
5 30% 232 Terry
6 50% 232 Terry
7 30% 412 Terry
8 50% 412 Terry

• What would be the D-optimal objective function value for the usual full 23 factorial model?

• If instead experiment 2 was run at (A,B,C) = (45%, 200, Larry), and experiment 3 run at (A, B, C)
= (35%, 400, Larry); what would be the D-optimal objective function value?

• What is the ratio between the two objective function values?

ò Solution

• The D-optimal objective function is to maximize the determinant of the design matrix, i.e.
(︀ )︀
det X𝑇 X .

Since this is a full factorial in 3 factors, with all runs perfectly at the −1 and +1 levels, then the
determinant is the product of the diagonal entries and is 88 = 16777216. In MATLAB, this would
be det(eye(8) * 8).

• Assuming the columns in X are in the order of [intercept, A, B, C, AB, AC, BC, ABC], then row
2 in matrix X would be [1, 0.5, −1.35, −1, −0.675, −0.5, 1.35, 0.675] and row 3 would be
[1, −0.5, 0.867, −1, −0.4333, 0.5, −0.867, 0.4333]

The determinant with these two rows replaced in now 6.402 × 106 .
6.402×106
• The ratio is 16777216 = 0.38, a fairly large reduction in the objective.

ò Question

In your start-up company you are investigating treatment options for reducing the contamination
level of soil that has been soaked with hydrocarbon products. You have two different heaps of
contaminated soil from two different sites. You expect your treatment method to work on any soil
type though.

Your limited line of credit allows only 9 experiments, even though you have identified at least 6
factors which you expect to have an effect on the treatment.

1. Write out the set of experiments that you believe will allow you to learn the most relevant
information, given your limited budget. Explain your thinking, and present your answer with 7

390 Chapter 5. Design and Analysis of Experiments

 Release 547b16

columns: 6 columns showing the settings for the 6 factors and one column for the heap from
which the test sample should be taken. There should be 9 rows in your table.

2. What is the projectivity and resolution of your design?

ò Solution

1. When given a constraint on the number of experiments, we would like to examine the highest
number of factors, but with the lowest tradeoff in the associated resolution.

There are 6 factors to examine. As stated, we would like our treatment method to work on any
contaminated soil sample, however we have testing soil only from 2 sites. This is a blocking
variable, since we might expect differences due to the site where the soil came from, but we
want it to have the least possible effect on our results.

An alternative way to view this problem is to assume that soil is an extra factor in the
experiment, but when choosing the generators, we will associate it with the highest level of
confounding possible. This latter interpretation makes it easier to use the table in the notes.

Using the table in the notes (page 356), and looking down the column with 7 factors, we are
constrained to the cell with 8 experiments, since the next cell down has 16 experiments, which is
too many. So a 27−4
III design would be most appropriate.

We would write out our usual 23 full factorial, then assign D=AB, E=AC, F=BC and G=ABC. We
will let that last factor be the heap of soil factor, as it has the highest level of confounding.

We can run a 9th experiment. In this case, I would put all variables at the center point (if they are
continuous), and use a 50/50 blend of the two soil samples. Also, I would run this experiment
first, to iron out any experimental protocol issues that I will didn’t think of; rather discover them
on this first run, which can be discarded in the analysis later on.

Alternatively, if I’m confident with my experimental procedure, I can choose to do experiment 9

last, if at all, as a replicate of any interesting previous experiment that gives an unexpected
(good or bad) result.

A table for the experiments would be:

Experiment A B C D=AB E=AC F=BC G=ABC

1 − − − + + + Heap 1
2 + − − − − + Heap 2
3 − + − − + − Heap 2
4 + + − + − − Heap 1
5 − − + + − − Heap 2
6 + − + − + − Heap 1
7 − + + − − + Heap 1
8 + + + + + + Heap 2
9 0 0 0 0 0 0 50/50

2. The design has resolution = 𝑅 = 3, from the table in the notes. The projectivity is 𝑅 − 1 = 2.

5.15. Exercises 391

Process Improvement Using Data

ò Question

A factorial experiment was run to investigate the settings that minimize the production of an
unwanted side product. The two factors being investigated are called A and B for simplicity, but
are:

• A = reaction temperature: low level was 420 K, and high level was 440 K

• B = amount of surfactant: low level was 10 kg, high level was 12 kg

A full factorial experiment was run, randomly, on the same batch of raw materials, in the same
reactor. The system was run on two different days though, and the operator on day 2 was a
different person. The recorded amount, in grams, of the side product was:

Experiment Run order Day A B Side product formed

1 2 1 420 K 10 kg 89 g
2 4 2 440 K 10 kg 268 g
3 5 2 420 K 12 kg 179 g
4 3 1 440 K 12 kg 448 g
5 1 1 430 K 11 kg 196 g
6 6 2 430 K 11 kg 215 g

1. What might have been the reason(s) for including experiments 5 and 6?

2. Was the blocking for a potential day-to-day effect implemented correctly in the design? Please
show your calculations.

3. Write out a model that will predict the amount of side product formed. The model should use
coded values of A and B. Also write out the X matrix and y vector that can be used to estimate
−1
the model coefficients using the equation b = (X′ X) X′ y.
−1
4. Solve for the coefficients of your linear model, either by using b = (X′ X) X′ y directly, or by
some other method.

5. Assuming the blocking for the day-to-day effect was implemented correctly, does your model
show whether this was an important effect on the response or not? Explain your answer.

6. You have permission to run two further experiments to find an operating point that reduces the
unwanted side product. Where would you place your next two runs, and show how you select
these values. Please give your answer in the original units of A and B.

7. As you move along the response surface, performing new experiments to approach the
optimum, how do you know when you are reaching an optimum? How does your experimental
strategy change? Please give specific details, and any model equations that might help illustrate
your answer.

ò Solution

1. Experiments 5 and 6 from the standard order might have been included as baseline experiments,
since they appear at the center point for factors A and B.

These two runs give 2 degrees of freedom as well, which helps with estimating confidence

392 Chapter 5. Design and Analysis of Experiments

 Release 547b16

intervals on the least squares parameters.

Also, since one of them was performed first, it could have been used to establish the
experimental workflow. In other words, the experiment was used to see how to run the
experiment the first time. If things go horribly wrong, then this data point can just be discarded.
If we had started with a corner of the factorial, we would have had to repeat that experiment if it
failed, or if it succeeded, had a duplicate experiment at the one corner but not the others.

Finally, it could also have been used to assess the effect of the operators, since runs 5 and 6 are
identical, though in this case runs 5 and 6 are on different days, so it could be the day-to-day
being measured here.

2. Yes. If we consider the day effect to be a new factor, C, then we could runs 1 to 4 as a half
fraction in 3 factors. The least disruptive generator would be C = AB. Using this we can see that
runs 1 and 4 should be run on one day, and runs 2 and 3 on the next day: this is what was done.
The center points can be run on either day, and in this case one was run on each day.

Using this generator confounds the interaction effect, AB with the day-to-day (and
operator-to-operator) effect. We can never clear up that confounding with this set of
experiments.

3. The model would have the form:

𝑦 = 𝑏0 + 𝑏𝐴 𝑥𝐴 + 𝑏𝐵 𝑥𝐵 + 𝑏𝐴𝐵 𝑥𝐴𝐵 + 𝑒

The matrices and vectors to solves this least squares model are:
⎡ ⎤ ⎡ ⎤ ⎡ ⎤
𝑦1 1 −1 −1 +1 𝑒
⎥⎡ ⎤ ⎢ 1⎥
⎢𝑦2 ⎥ ⎢1 +1 −1 −1⎥ 𝑏0 ⎢𝑒2 ⎥
⎢ ⎥ ⎢
⎢ ⎥ ⎢ ⎥ ⎥ ⎢ ⎥
⎢𝑦3 ⎥ ⎢1 −1 +1 −1⎥ ⎢ 𝑏 𝐴 ⎢𝑒3 ⎥
⎢ ⎥=⎢ ⎥⎢⎢ ⎥
+⎢ ⎥
⎢
⎢𝑦 ⎥ ⎢1 +1 +1 +1⎥ ⎣ 𝑏 ⎥
⎢𝑒4 ⎥
⎥
⎢ 4⎥ ⎢ ⎥ 𝐵 ⎦
⎣𝑦5 ⎦ ⎣1 0 0 0 ⎦ 𝑏𝐴𝐵 ⎣𝑒5 ⎦
⎢ ⎥ ⎢ ⎥ ⎢ ⎥
𝑦6 1 0 0 0 𝑒6
⎡ ⎤ ⎡ ⎤ ⎡ ⎤
89 1 −1 −1 +1 𝑒
⎥⎡ ⎤ ⎢ 1⎥
⎢268⎥ ⎢1 +1 −1 −1⎥ 𝑏0 ⎢𝑒2 ⎥
⎢ ⎥ ⎢
⎢ ⎥ ⎢ ⎥ ⎥ ⎢ ⎥
⎢179⎥ ⎢1 −1 +1 −1⎥ ⎢ 𝑏 𝐴 ⎢𝑒3 ⎥
⎢ ⎥=⎢ ⎥⎢⎢ ⎥
+⎢ ⎥
⎢
⎢448⎥ ⎢1 +1 +1 +1⎥ ⎣ 𝑏 ⎥
⎢𝑒4 ⎥
⎥
⎢ ⎥ ⎢ ⎥ 𝐵 ⎦
⎣196⎦ ⎣1 0 0 0 ⎦ 𝑏𝐴𝐵 ⎣𝑒5 ⎦
⎢ ⎥ ⎢ ⎥ ⎢ ⎥
215 1 0 0 0 𝑒6
y = Xb + e

−1
4. Using the above matrices we can calculate b = (X′ X) X′ y, even by hand!
⎡ ⎤
6 0 0 0
⎢0 4 0 0⎥
X′ X = ⎢
⎢ ⎥
⎥
⎣0 0 4 0⎦
0 0 0 4
⎡ ⎤ ⎡ ⎤
89 + 268 + 179 + 448 + 196 + 215 1395
⎢ −89 + 268 − 179 + 448 ⎥ ⎢ 448 ⎥
X′ y = ⎢ ⎥=⎢
⎢ ⎥ ⎢ ⎥
−89 − 268 + 179 + 448
⎥
⎣ ⎦ ⎣ 270 ⎦
+89 − 268 − 179 + 448 90
⎡ ⎤
232.5
⎢ 112 ⎥
−1
b = (X′ X) X′ y = ⎢
⎢ ⎥
⎥
⎣ 67.5 ⎦
22.5
5.15. Exercises 393
Process Improvement Using Data

5. The above least squares solution shows the two main effects are large: 112 and 67.5 for a one unit
change (coded units). Relative to these two, the interaction term of 𝑏𝐴𝐵 = 22.5 is small. This
implies the day-to-day effect (which is confounded with the operator effect) is small.

6. A new run in A and B would be at lower values of A and B, since we want to reduce the side
product. We will make a move from the baseline point by reducing factor A by 1 unit, and then
ratio that with the necessary change in B to go down the direction of steepest descent:

Δ𝑥𝐴 = −1
Δ𝑥𝐴,actual = −10 K
𝑏𝐵 67.5
Δ𝑥𝐵 = Δ𝑥𝐴 = Δ𝑥𝐴
𝑏𝐴 112
𝑥𝐵,actual
but we know that Δ𝑥𝐵 =
Δ𝐵 /2
𝑏𝐵
Δ𝑥𝐵,actual = Δ𝑥𝐴 × Δ𝐵 /2 by equating previous 2 lines
𝑏𝐴
67.5
Δ𝑥𝐵,actual = × (−1) × 2kg/2
112
Δ𝑥𝐵,actual = −0.60 kg

Note that Δ𝐵 ̸= Δ𝑥𝐵 . The former is the range for factor B, the latter is the amount by which we
change factor B from the baseline. So the new settings for the next experiment would be at:

• A = 430 - 10 = 420 K

• B = 11 − 0.60 = 10.4 kg

7. An optimum is present in a factorial experiment if you notice that:

• interaction terms start to become large,

• the center point in the factorial has higher/lower values than any of the corner points
(remember that with an optimum you are the peak or the valley)

• curvature terms, i.e. quadratic terms, in the model are larger than the main effect.

The experimental strategy changes by included axial points into the factorial design, allowing
one to calculate the quadratic terms in the model, such as a 𝑏𝐴𝐴 𝑥2𝐴 term for the quadratic effect
of factor A.

ò Question

Adapted from Box, Hunter and Hunter

A liquid polymer formulation is being made that is applied as a polish to wood surfaces. The
group responsible for the product have identified 3 elements to the formulation that have an effect
of the liquid polish’s final quality attributes (FQAs: this acronym is becoming a standard in most
companies these days).

• A: amount of reactive monomer in the recipe (10% at the low level and 30% at the high level)

• B: the amount of chain length regulator (1% at the low level and 4% at the high level)

• C: the type of chain length regulator (regulator P at the − level or regulator Q at the + level)

In class we have focused on the case where our 𝑦-variable is continuous, but it could also be

394 Chapter 5. Design and Analysis of Experiments

 Release 547b16

descriptive. In this question we also see what happens when we have more than one 𝑦-variable.

• 𝑦1 = Milky appearance: either Yes or No

• 𝑦2 = Viscous: either Yes or No

• 𝑦3 = Yellow colour: either No or Slightly

The following table captures the 8 experiments in standard order, although the experiments were
run in a randomized order.

Experiment A B C 𝑦1 𝑦2 𝑦3
1 − − P Yes Yes No
2 + − P No Yes No
3 − + P Yes No No
4 + + P No No No
5 − − Q Yes Yes No
6 + − Q No Yes Slightly
7 − + Q Yes No No
8 + + Q No No Slightly

1. What is the cause of a milky appearance?

2. What causes a more viscous product?

3. What is the cause of a slight yellow appearance?

4. Which conditions would you use to create a product was not milky, was of low viscosity, and
had no yellowness?

5. Which conditions would you use to create a product was not milky, was of low viscosity, and
had some yellowness?

ò Solution

Tables are often frowned on by people, but the reality is they are sometimes one of the best forms of
visualizing data. In this example we see:

1. The milky appearance is caused by low levels of A = amount of reactive monomer (10% in this
recipe), since milky appearance is correlated with that column.

2. A more viscous product is caused by low levels of B = amount of chain length regulator (1% in
this recipe), since the change in signs in B match the viscous column.

3. The yellow appearance is due to an interaction: in this case only when using chain length
regulator Q and when using high levels of reactive monomer in the recipe (30%) do we see the
yellow appearance.

4. Such a product can be obtained by using

• A = high amount of reactive monomer in the recipe (30%)

• B = high amounts of chain length regulator (4%)

• C = use chain length regulator P

5.15. Exercises 395

Process Improvement Using Data

These correspond to conditions in experiment 4.

5. Such a product can be obtained by using

• A = high amount of reactive monomer in the recipe (30%)

• B = high amounts of chain length regulator (4%)

• C = use chain length regulator Q

These correspond to conditions in experiment 8.

In all these questions we can conclusively state there is cause and effect, since we see repeated
changes in the factors (holding the other variables and disturbances constant) and the
corresponding effects in the 3 𝑦-variables.

ò Question

Using a 23 factorial design in 3 variables (A = temperature, B = pH and C = agitation rate), the

conversion, 𝑦, from a chemical reaction was recorded.

Experiment A B C 𝑦
1 − − − 72
2 + − − 73
3 − + − 66
4 + + − 87
5 − − + 70
6 + − + 73
7 − + + 67
8 + + + 87

temperature − 150°C
• A=
10°C
pH − 7.5
• B=
0.5
agitation rate − 50rpm
• C=
5rpm
1. Show a cube plot for the recorded data.

2. Estimate the main effects and interactions by hand.

3. Interpret any results from part 2.

4. Show that a least squares model for the full factorial agrees with the effects and interactions
calculated by hand.

5. Approximately, at what conditions (given in real-world units), would you run the next
experiment to improve conversion. Give your settings in coded units, then unscale and uncenter
them to get real-world units.

396 Chapter 5. Design and Analysis of Experiments

 Release 547b16

ò Solution

1. A cube plot for the data from these 8 runs is:

2. The main effects and interactions are:

• A effect: There are 4 estimates of

(73 − 72) + (87 − 66) + (73 − 70) + (87 − 67) 45
𝐴= = = 11.25
4 4
• B effect: There are 4 estimates of
(66 − 72) + (87 − 73) + (67 − 70) + (87 − 73) 19
𝐵= = = 4.75
4 4
• C effect: Again 4 estimates of
(70 − 72) + (73 − 73) + (67 − 66) + (87 − 87) −1
𝐶= = = −0.25
4 4
• AB interaction: There are 2 estimates of 𝐴𝐵. Recall that interactions are calculated as the half
difference going from high to low. Consider the change in 𝐴 when

– 𝐵high (at 𝐶 high) = 87 - 67 = 20

– 𝐵low (at 𝐶 high) = 73-70 = 3

– First estimate = [(20) - (3)]/2 = 8.5

– 𝐵high (at 𝐶 low) = 87 - 66 = 21

– 𝐵low (at 𝐶 low) = 73 - 72 = +1

– Second estimate = [(21) - (1)]/2 = 10

– Average AB interaction = (8.5 + 10)/2 = 9.25

– You can interchange 𝐴 and 𝐵 and still get the same result.

• AC interaction: There are 2 estimates of 𝐴𝐶. Consider the change in 𝐶 when

– 𝐴high (at 𝐵 high) = 87 - 87 = 0

– 𝐴low (at 𝐵 high) = 67 - 66 = 1

– First estimate = [(0) - (+1)]/2 = -0.5

– 𝐴high (at 𝐵 low) = 73 - 73 = 0

5.15. Exercises 397

Process Improvement Using Data

– 𝐴low (at 𝐵 low) = 70 - 72 = -2

– Second estimate = [(0) - (-2)]/2 = 1

– Average AC interaction = (-0.5 + 1)/2 = 0.25

– You can interchange 𝐴 and 𝐶 and still get the same result.

• BC interaction: There are 2 estimates of 𝐵𝐶: 0 (at high A) and 1.5 (at low A), giving an
average BC interaction of 0.75.

• ABC interaction: There is only a single estimate:

– 𝐴𝐵 effect at high 𝐶 = 8.5

– 𝐴𝐵 effect at low 𝐶 = 10

– 𝐴𝐵𝐶 interaction = [(8.5) - (10)] / 2 = -0.75

– You can calculate this also by considering the 𝐴𝐶 effect at the two levels of 𝐵

– Or, you can calculate this by considering the 𝐵𝐶 effect at the two levels of 𝐴.

– All 3 approaches give the same result.

3. These results show that temperature, A, has by far the greatest effect on the conversion: an
increase in conversion of 11.25 % for every 10 °C increase in temperature. The agitation rate, C,
has a negligible effect and the effect of pH, B, is between the two: an expected 4.75% increase for
every 0.5 units of increased pH.

There are no interactions between agitation rate (the BC and AC interactions are both small), so
we can safely drop the agitation, factor C, from future consideration in this system.

There is however an interaction between temperature and pH, the AB interaction. This shows
that conversion is further increased when both these factors are operated at their high levels.
−1
4. A least squares model was found by solving for b = (X′ X) X′ y, where
⎡ ⎤ ⎡ ⎤⎡ ⎤
72 +1 −1 −1 −1 +1 +1 +1 −1 𝑏0
⎢73⎥ ⎢+1 +1 −1 −1 −1 −1 +1 +1⎥
⎥ ⎢ 𝑏𝐴 ⎥
⎢ ⎥
⎢ ⎥ ⎢
⎢66⎥ ⎢+1 −1 +1 −1 −1 +1 −1 +1⎥ ⎢ 𝑏𝐵 ⎥
⎢ ⎥ ⎢ ⎥⎢ ⎥
⎢ ⎥ ⎢ ⎥⎢ ⎥
⎢87⎥ ⎢+1 +1 +1 −1 +1 −1 −1 −1⎥
⎥ ⎢ 𝑏𝐶 ⎥
⎢ ⎥
⎢ ⎥=⎢
⎢70⎥ ⎢+1 −1 −1 +1 +1 −1 −1 +1⎥ ⎢ 𝑏𝐴𝐵 ⎥
⎥ ⎢
⎢ ⎥ ⎢ ⎥
⎢73⎥ ⎢+1 +1 −1 +1 −1 +1 −1 −1⎥ ⎢ 𝑏𝐴𝐶 ⎥
⎢ ⎥ ⎢ ⎥⎢ ⎥
⎢ ⎥ ⎢ ⎥⎢ ⎥
⎣67⎦ ⎣+1 −1 +1 +1 −1 −1 +1 −1⎦ ⎣ 𝑏𝐵𝐶 ⎦
87 +1 +1 +1 +1 +1 +1 +1 +1 𝑏𝐴𝐵𝐶
y = Xb

which solved for b gives the expected model:

𝑦 = 74.4 + 5.625𝑥𝐴 + 2.375𝑥𝐵 − 0.125𝑥𝐶 + 4.625𝑥𝐴 𝑥𝐵 + 0.125𝑥𝐴 𝑥𝐶 + 0.375𝑥𝐵 𝑥𝐶 − 0.375𝑥𝐴 𝑥𝐵 𝑥𝐶

that agrees with the hand-calculations (where the effects are double those from the least squares
model).

5. The experiments can be run at any level of agitation, so it makes sense to use the current
midpoint of 50 rpm. We are then left with selecting A and B.

398 Chapter 5. Design and Analysis of Experiments

 Release 547b16

Since we want to increase conversion, we would want to go in any direction that has higher
levels of A and B. I would tentatively select A = 1.5 and B as 1.5 in coded units. I used the
following MATLAB code and figure to help my decision (we will see how to select the new point
more formally in the section on response surface methods (page 365)).

X = [1 -1 -1 -1; ...
1 1 -1 -1; ...
1 -1 1 -1; ...
1 1 1 -1; ...
1 -1 -1 1; ...
1 1 -1 1; ...
1 -1 1 1; ...
1 1 1 1];
X(:,5) = X(:,2) .* X(:,3);
X(:,6) = X(:,2) .* X(:,4);
X(:,7) = X(:,3) .* X(:,4);
X(:,8) = X(:,2) .* X(:,3) .* X(:,4);
y = [72, 73, 66, 87, 70, 73, 67, 87]';
inv(X'*X) % verify that X is orthogonal
b=inv(X'*X) * X'*y

[A,B] = meshgrid(-2:.2:2, -2:.2:2);

y_hat = b(1) + b(2) .*A + b(3).*B + b(5) .* A .* B;

contour(A, B, y_hat, 'LineWidth', 2)

xlabel('A [Temperature]', 'FontSize', 14)
ylabel('B [pH]', 'FontSize', 14)
title('Predicted contours of y=conversion for A and B', 'FontSize', 14)
hold on
grid on
plot(-1, -1, 'ko', 'Markersize', 10, 'LineWidth',4)
plot(+1, -1, 'ko', 'Markersize', 10, 'LineWidth',4)
plot(-1, +1, 'ko', 'Markersize', 10, 'LineWidth',4)
plot(+1, +1, 'ko', 'Markersize', 10, 'LineWidth',4)
plot(+1.5, +1.5, 'k*', 'Markersize', 10, 'LineWidth',1)
% Add a colour bar; see the MATLAB help for other colour schemes
colormap(hsv)
axis equal
colorbar
print -dpng -r200 [Link]

5.15. Exercises 399

Process Improvement Using Data

In real-world units these points correspond to:

• 𝐴actual = 1.5 × 10°C + 150°C = 165 °C.

• 𝐵actual = 1.5 × 0.5°C + 7.5°C = 8.25 pH units.

ò Question

1. Why do we block groups of experiments?

2. Write a 23 factorial design in two blocks of 4 runs, so that no main effect or 2 factor interaction is
confounded with block differences.

ò Solution

1. When performing experiments in groups, for example, half the experiments are run on day one
and the others on day 2, we must block the experiments we choose to run on each day, to avoid
inadvertently introducing a new effect, a day-to-day effect in the model. In other words, we
must choose in a careful way the half group of runs we place on day 1 and day 2.

Blocking applies in many other cases: sometimes we have to use two batches of raw materials to
do an experiment, because there is not enough for the full factorial. We must block to prevent
the effect of raw materials to affect our 𝑦-variable.

Or to run the experiments efficiently in a short time, we choose to do them in parallel in two
different reactors. Here we must block against the reactor effect.

2. For a 23 system we have factors A, B and C. To avoid the main effect being confounded with any
2 factor interactions we must assign the blocks to the highest interaction, i.e. the ABC
interaction.

400 Chapter 5. Design and Analysis of Experiments

 Release 547b16

Writing out the design in standard order:

Experiment A B C ABC
1 − − − −
2 + − − +
3 − + − +
4 + + − −
5 − − + +
6 + − + −
7 − + + −
8 + + + +

This table indicates we should do all experiments in column ABC with a − in one block, and the
experiments with a + should be done in the second block. The main effects will not be
confounded with any 2-factor interactions in this case.

Another way you can interpret blocking is as follows. Consider the block to be a new factor in
your experiment, call it factor D, where D at the low level corresponds to experiments in the first
block, and D at the high level would be experiments in the second block.

But we can only run 8 experiments, so we now use the table in the course notes (derived from
page 272 in Box, Hunter and Hunter, 2nd edition), and see the layout that will cause least
disruption is to assign D = ABC. This gives the same experimental layout above.

ò Question

Factors related to the shrinkage of plastic film, produced in an injection molding device, are being
investigated. The following factors have been identified by the engineer responsible:

• A = mold temperature

• B = moisture content

• C = holding pressure

• D = cavity thickness

• E = booster pressure

• F = cycle time

• G = gate size

Experiment A B C D E F G 𝑦
1 − − − + + + − 14.0
2 + − − − − + + 16.8
3 − + − − + − + 15.0
4 + + − + − − − 15.4
5 − − + + − − + 27.6
6 + − + − + − − 24.0
7 − + + − − + − 27.4
8 + + + + + + + 22.6

5.15. Exercises 401

Process Improvement Using Data

You can obtain a copy of this data set if you install the BsMD package in R. Then use the following
commands:

library(BsMD)
data([Link])

# Use only a subset of the original experiments

X <- [Link][1:8, 2:10]

1. How many experiments would have been required for a full factorial experiment?

2. What type of fractional factorial is this (i.e. is it a half fraction, quarter fraction . . . )?

3. Identify all the generators used to create this design. A table, such as on page 272 in Box, Hunter
and Hunter, 2nd edition will help.

4. Write out the complete defining relationship.

5. What is the resolution of this design?

6. Use a least squares approach to calculate a model that fits these 8 experiments.

7. What effects would you judge to be significant in this system? The engineer will accept your
advice and disregard the other factors, and spend the rest of the experimental budget only on
the factors deemed significant.

8. What are these effects aliased with (use your defining relationship to find this).

9. Why is in necessary to know the confounding pattern for a fractional factorial design.

ò Solution

1. There are 7 factors in this experiment, so a full factorial would require 27 = 128 experiments.

2. This is a one-sixteenth fraction, 8/128 = 1/16.

3. Since the are 7 factors in 8 runs, the DOE tradeoff table (page 356) indicates the possible
generators are D = AB, E = AC, F = BC and G = ABC. However, that doesn’t mean the
experiments were generated with exactly those factors. For example, these experiments could
have interchanged the A and B columns, in which case factors E and F would be different.

However, when checking the columns in our table against these generators we see that the
experiments were derived from exactly these same generators. It is customary to record the
generators in the form I = . . . , so our generators are:

• I = ABD

• I = ACE

• I = BCF

• I = ABCG.

4. The defining relationship is the product of all possible generator combinations. Since there are 4
generators, there are 24 words in the defining relationship. A similar example in the course notes
shows that the defining relationship is:

402 Chapter 5. Design and Analysis of Experiments

 Release 547b16

I = ABD = ACE = BCF = ABCG = BCDE = ACDF = CDG = ABEF = BEG = AFG = DEF =
ADEG = CEFG = BDFG = ABCDEFG

5. It is a resolution III design, by noting the shortest word in the defining relationship is of length 3
(and verified in the table above).

6. The least squares model would be found by setting − = −1 and + = +1 in the table above as the
X matrix, and adding an additional column of 1’s to account for the intercept. This gives a total
of 8 columns in the matrix. The X𝑇 X will be diagonal, with 8’s on the diagonal. The y vector is
just the table of results above.
(︀ )︀−1 𝑇
From this we calculate b = X𝑇 X X y (MATLAB and R code is given at the end).

𝑦 = 20.35 − 0.65𝑥𝐴 − 0.25𝑥𝐵 + 5.05𝑥𝐶 − 0.45𝑥𝐷 − 1.45𝑥𝐸 − 0.15𝑥𝐹 + 0.15𝑥𝐺

7. From this we judge effect C, E and to a smaller extent, effect A, to be significant.

8. However, these main effects are aliased with:

• C (multiply C by every word in the defining relationship)

– CABD = ABCD

– CACE = AE

– CBCF = BF

– CABCG = ABG

– CBCDE = BDE

– CACDF = ADF

– CCDG = DG

– CABEF = ABCEF

– CBEG = CBEG

– CAFG = ACFG

– CDEF = CDEF

– CADEG = ACDEG

– CCEFG = EFG

– CBDFG = BCDFG

– CABCDEFG = ABDEFG

• E (reporting only the 2 factor interactions)

– AC

– BG

– DF

• A (reporting only the 2 factor interactions)

– BD

5.15. Exercises 403

Process Improvement Using Data

– CE

– FG

9. It is necessary to know the confounding pattern because it helps to interpret the coefficients. For
example, we see that factor C is aliased with the AE interaction, and we also see that factors A
and E are important. We cannot be sure though if that large coefficient for C is due purely to C,
or if it is also due to the AE interaction.

The only way we can uncouple that coefficient is by performing additional, foldover experiments.

The R code for this question are given below, and also code to draw the Pareto plot to determine
the most important coefficients.

A <- B <- C <- c(-1, 1)

d <- [Link](A=A, B=B, C=C)
y <- c(14.0, 16.8, 15.0, 15.4, 27.6, 24.0, 27.4, 22.6)
A <- d$A
B <- d$B
C <- d$C
D <- A*B
E <- A*C
F <- B*C
G <- A*B*C
model <- lm(y ~ A + B + C + D + E + F + G)
summary(model)

coeff <- coef(model)[2:length(coef(model))]

# Pareto plot of the absolute coefficients

library(lattice)
bitmap('[Link]', type="png256",
width=6, height=6, res=300, pointsize=14)
barchart(sort(abs(coeff)), xlab="Magnitude of effect",
ylab = "Effect", col=0)
[Link]()

404 Chapter 5. Design and Analysis of Experiments

 Release 547b16

ò Question

One of the experiment projects investigated by a previous student of this course was
understanding effects related to the preparation of uncooked, breaded chicken strips.

The student investigated these 3 factors in a full factorial design * :

• D = duration: low level at 15 minutes; and high level = 22 minutes.

• R = position of oven rack: low level = use middle rack; high level = use low oven rack (this
coding, though unusual, was used because the lower rack applies more heat to the food).

• P = preheated oven or not: low level = short preheat (30 seconds); high level = complete
preheating.
*
The student actually investigated 4 factors, but found the effect of oven temperature to be
negligible!

The response variable was 𝑦 = taste, the average of several tasters, with higher values being more
desirable.

Experiment D R P Taste
1 − − − 3
2 + − − 9
3 − + − 3
4 + + − 7
5 − − + 3
6 + − + 10
7 − + + 4
8 + + + 7

A full factorial model, using the usual coding, was calculated from these 8 experiments:

𝑦 = 5.75 + 2.5𝑥D − 0.5𝑥R + 0.25𝑥P − 0.75𝑥D 𝑥R − 0.0𝑥D 𝑥P − 0.0𝑥R 𝑥P − 0.25𝑥D 𝑥R 𝑥P

1. What is the physical interpretation of the +2.5𝑥D term in the model?

2. From the above table, at what real-world conditions should you run the system to get the
highest taste level?

3. Does your previous answer match the above model equation? Explain, in particular, how the
non-zero two factor interaction term affects taste, and whether the interaction term reinforces the
taste response variable, or counteracts it, when the settings you identified in part 2 are used.

4. If you decided to investigate this system, but only had time to run 4 experiments, write out the
fractional factorial table that would use factors D and R as your main effects and confound
factor P on the DR interaction.

Now add to your table the response column for taste, extracting the relevant experiments from
the above table.

Next, write out the model equation and estimate the 4 model parameters from your reduced set
of experiments. Compare and comment on your model coefficients, relative to the full model
equation from all 8 experiments.

5.15. Exercises 405

Process Improvement Using Data

ò Solution

1. (22-15)/2 increase in cooking time results in a 2.5 taste level increase.

2. 22 minutes, middle over rack, with preheating.

3. Yes: the D+ and P+ levels both have positive coefficients, while the R- level has a negative
coefficient. The DR interaction has a negative coefficient. This actually reinforced (improves) the
taste, because D = + and R = −, so the the DR term adds the taste value. This term also makes
physical sense: if D = + and R = +, then the taste deteriorates, likely to the food being
overcooked. Similarly, if D = − and R = −, then the chicken is undercooked.

4. The table, in standard order has 𝑦 = 3, 9, 3, 7 and the model is 𝑦 = 5.5 + 2.5𝑥𝐷 − 0.5𝑥𝑅 − 0.5𝑥𝑃 .
The D and R coefficients are the same, only the P coefficient = Poriginal + DRoriginal
= +0.25 − 0.75 = −0.5, due to the aliasing, which is expected.

ò Question

Your company is developing a microgel-hydrogel composite, used for controlled drug delivery
with a magnetic field. A previous employee did the experimental work but she has since left the
company. You have been asked to analyze the existing experimental data.

• Response variable: 𝑦 = sodium fluorescein (SF) released [mg], per gram of gel

• The data collected, in the original units:

Experiment Order M = microgel weight [%] H = hydrogel weight [%] 𝑦

1 4 4 10 119
2 1 8 10 93
3 6 4 16 154
4 3 8 16 89
5 2 6 13 85
6 5 6 13 88
7 9 3.2 13 125
8 7 8.8 13 111
9 10 6 17.2 136
10 8 6 8.8 98

1. What was likely the reason the experimenter added experiments 5 and 6?

2. Why might the experimenter have added experiments 7, 8, 9 and 10 after the first six? Provide a
rough sketch of the design, and all necessary calculations to justify your answer.

3. What is the name of the type of experimental design chosen by the employee for all 10
experiments in the table?

4. Using these data, you wish to estimate a nonlinear approximation of the response surface using
a model with quadratic terms. Write out the equation of such a model that can be calculated
from these 10 experiments (also read the next question).

5. Write out

406 Chapter 5. Design and Analysis of Experiments

 Release 547b16

• the X matrix,

• the corresponding symbolic entries in b

• and the y vector

(︀ )︀−1 𝑇
that you would use to solve the equation b = X𝑇 X X y to obtain the parameter estimates
of the model you proposed in the previous part. You must use data from all 10 experiments.

6. How many degrees of freedom will be available to estimate the standard error and confidence
intervals?

ò Solution

1. These are centerpoint (baseline) runs. They may have been run for some of the following
reasons:

• To give degrees of freedom for calculating the standard error and then confidence intervals
for the slopes.

• Trial runs, though they were not done first, so that’s unlikely.

• To obtain baseline values for later response surface optimization.

• To test the factorial model.

• To assess repeatability at the center point.

• There might be one or more days that elapsed between the runs, so this assesses the
robustness of the model over time

• To test for curvature: if the average of the centerpoints, 87, is very different from the model’s
intercept, 𝑏0 = 0.25(119 + 93 + 154 + 89) = 114, as it is in this case, then there is evidence of
curvature.

2. It’s clear that there is evidence of curvature, also, it it is feasible the employee was wanting to
optimize the response variable. In that case, she would likely use response surface techniques of
climbing the path of steepest ascent.

In this model, the presence of curvature at the center point has already been shown. Also a quick
calculation from the 4 corner points shows a significant 2 factor interaction.

Using response surface methods in with only the linear terms will be misleading in this case.
That’s why the employee decided to add the extra experiments; they are the axial experiments to
support quadratic terms.

3. Central composite design, with the full factorial experiment, in two factors.

4. 𝑦 = 𝑏0 + 𝑏𝑀 𝑥𝑀 + 𝑏𝐻 𝑥𝐻 + 𝑏𝑀 𝐻 𝑥𝑀 𝑥𝐻 + 𝑏𝑀 𝑀 𝑥2𝑀 + 𝑏𝐻𝐻 𝑥2𝐻

5. X has 10 rows and 6 columns to support the 6 terms in the above model. The last for points have
√︀
± (2) and 2 terms in those rows.

6. There 4 degrees of freedom (10 observations, 6 parameters)

5.15. Exercises 407

Process Improvement Using Data

ò Question

Biological drugs are rapidly growing in importance in the treatment of certain diseases, such as
cancers and arthritis, since they are designed to target very specific sites in the human body. This
can result in treating diseases with minimal side effects. Such drugs differ from traditional drugs in
the way they are manufactured – they are produced during the complex reactions that take place in
live cell culture. The cells are grown in lab-scale bioreactors, harvested, purified and packaged.

These processes are plagued by low yields which makes these treatments very costly. Your group
has run an initial set of experiments to learn more about the system and find better operating
conditions to boost the yield. The following factors were chosen in the usual factorial manner:

• G = glucose substrate choice: a binary factor, either Gm at the low level code or Gp at the high
level.

• A = agitation level: low level = 10 rpm and high level = 20 rpm, but can only be set at integer
values.

• T = growth temperature: 30°C at the low level, or 36°C at the high level, and can only be set at
integer values in the future, with a maximum value of 40°C.

• C = starting culture concentration: low level = 1100 and high level = 1400, and can only be
adjusted in multiples of 50 units and within a range of 1000 to 2000 units.

A fractional factorial in 8 runs at the above settings, created by aliasing C = GAT, gave the
following model in coded units:

𝑦 = 24 + 3𝑥G − 1.0𝑥A + 4.0𝑥T − 0.2𝑥G 𝑥A − 0.79𝑥G 𝑥T − 0.25𝑥A 𝑥T + 3.5𝑥G 𝑥A 𝑥T

The aim is to find the next experiment that will improve the yield, measured in milligrams, the
most.

1. What settings might have been used for the baseline conditions for this factorial experiment?

2. What is the resolution of this design?

3. Using the method of steepest ascent, state all reasonable assumptions you need to find the
experimental conditions for all 4 factors for the next experiment. Give these 4 conditions in both
the real-world units, as well as in the usual coded units of the experiment. Note however that
your manager has seen that temperature has a strong effect on yield, and he has requested the
next experiment be run at 40°C.

4. Report the expected yield at these proposed experimental conditions.

ò Solution

1. Baseline conditions are at G = Gm or Gp (either would work), A at 15 rpm, T at 30°C, and C at

1250 concentration units.

2. It is a four factor experiment, with 8 runs; from the table, for the given aliasing, it is a resolution
IV design.

3. We assume that we can ignore all 2fi and 3fi - i.e. that they are small. Specifically, this implies
that the 3.5 coefficient is for C and not for the product of 𝑥G 𝑥A 𝑥T

408 Chapter 5. Design and Analysis of Experiments

 Release 547b16

• Fix temperature at 40°C, implying that 𝑇 (next) = 40°C and 𝑥(next)

T = 40−33
3 = 2.33.

• Factor G must be run at the highest level possible, i.e. G = Gp

• Factor A must be run at a lower level, specifically Δ𝐴 = −0.25 × 2.33 = −0.583, or a deviation
of -2.9 rpm from the baseline. Since we have to use integer values, that implies 𝐴(next) = 12
rpm and 𝑥(next)
A = 12−15
5 = −0.6.

• Factor C must be run at a higher level, specifically Δ𝐶 = 3.5/4 × 2.33 = 2.04, or a deviation of
+306 in actual units from the baseline. Since we have to round to the closest 50, that implies
𝐶 (next) = 1550 rpm and 𝑥(next)
C = 1550−1250
150 = +2.

4. The predicted yield can be found by substituting the coded values into the model equation,
choosing to either use or ignore the small interactions:

With the interactions:

𝑦 = 24 + 3(+1) − 1.0(−0.6) + 4.0(2.33) − 0.2(+1)(−0.6) − 0.79(+1)(2.33) − 0.25(−0.6)(2.33) + 3.5(+2)

𝑦 = 42.3

Without interactions:

𝑦 = 24 + 3(+1) − 1.0(−0.6) + 4.0(2.33) + 3.5(+2) = 43.9

5.15. Exercises 409

Process Improvement Using Data

410 Chapter 5. Design and Analysis of Experiments

CHAPTER 6
LATENT VARIABLE MODELLING

6.1 In context

This section considers the important area of latent variable modelling. These models have been
shown, about 20 to 30 years ago, to be very powerful tools in dealing with the very data that
(chemical) engineers face frequently. Our main goal of this section is to show how one can extract
value from these data. But we first introduce the concept of a latent variable, and specifically the
principal component analysis (PCA) model: the cornerstone of all latent variable models. Then we
consider different ways to use our databases for interesting applications such as troubleshooting,
soft-sensors, process monitoring, and new product development.

6.1.1 What we will cover

6.2 References and readings

These readings cover a variety of topics in the area of latent variable methods:

• General: A collection of important latent variable publications are collected at

[Link]

411
Process Improvement Using Data

• General: John MacGregor, Honglu Yu, Salvador García-Muñoz, Jesus Flores-Cerrillo, “Data-Based
Latent Variable Methods for Process Analysis, Monitoring and Control148 ”. Computers and Chemical
Engineering, 29, 1217-1223, 2005.

• General: Ericsson, Johansson, Kettaneth-Wold, Trygg, Wikström, Wold: “Multivariate and

Megavariate Data Analysis”.

• About PCA: Svante Wold, Kim Esbensen, Paul Geladi: “Principal Component Analysis149 ”,
Chemometrics and Intelligent Laboratory Systems, 2, 37-52, 1987.

• PLS: Svante Wold, Michael Sjöström, Lennart Eriksson: “PLS-regression: A Basic Tool of
Chemometrics150 ”, Chemometrics and Intelligent Laboratory Systems, 58, 109-130, 2001.

• PLS: S. Wold, S. Hellberg, T. Lundstedt, M. Sjöström and H. Wold, “PLS Modeling With Latent
Variables in Two or More Dimensions”, Frankfurt PLS meeting, 1987 (available on request, by email to
kgdunn@[Link])

• PLS: Paul Geladi and Bruce Kowalski, “Partial Least-Squares Regression: A Tutorial151 ”, Analytica
Chimica Acta, 185, 1-17, 1986.

• PLS: Paul Garthwaite, “An Interpretation of Partial Least Squares152 ”, Journal of the American
Statistical Association, 89, 122-127, 1994.

• Process monitoring: John MacGregor and Theodora Kourti “Statistical Process Control of
Multivariate Processes153 ”, Control Engineering Practice, 3, p 403-414, 1995.

• Process monitoring: J.V. Kresta, T.E. Marlin, and J.F. MacGregor “Multivariate Statistical
Monitoring of Process Operating Performance154 ”, Canadian Journal of Chemical Engineering, 69,
35-47, 1991.

• Contribution plots: P Miller, RE Swanson, CE Heckler, “Contribution Plots: a Missing Link in

Multivariate Quality Control155 ”, Applied Mathematics and Computer Science, 8 (4), 775-792, 1998.
(hard to obtain, but available on request, by email to kgdunn@[Link])

• Soft sensors: J.V. Kresta, T.E. Marlin, and J.F. MacGregor, “Development of Inferential Process
Models Using PLS156 ”. Computers and Chemical Engineering, 18, 597-611, 1994.

• Industrial applications: Ivan Miletic, Shannon Quinn, Michael Dudzic, Vit Vaculik and Marc
Champagne, “An Industrial Perspective on Implementing On-Line Applications of Multivariate
Statistics157 ”, Journal of Process Control, 14, p. 821-836, 2004.

• Batch modelling and monitoring: S. Wold, N. Kettaneh-Wold, J.F. MacGregor, K.G. Dunn, “Batch
Process Modeling and MSPC158 ”. Comprehensive Chemometrics, 2, 163-197, 2009. (available from the
author on request, by email to kgdunn@[Link])

• Image analysis: M. Bharati, and J.F. MacGregor “Multivariate Image Analysis for Real Time Process
Monitoring and Control159 ”, Industrial and Engineering Chemistry Research, 37, 4715-4724, 1998
148 [Link]
149 [Link]
150 [Link]
151 [Link]
152 [Link]
153 [Link]
154 [Link]
155 [Link]
156 [Link]
157 [Link]
158 [Link]
159 [Link]

412 Chapter 6. Latent Variable Modelling

 Release 547b16

6.3 Extracting value from data

There are five main areas where engineers use large quantities of data.

1. Improved process understanding

This is an implicit goal in any data analysis: either we confirm what we know about the process, or
we see something unusual show up and learn from it. Plots that show, in one go, how a complex set
of variables interact and relate to each other are required for this step.

2. Troubleshooting process problems

Troubleshooting occurs after a problem has occurred. There are many potential sources that could
have caused the problem. Screening tools are required that will help isolate the variables most
related to the problem. These variables, combined with our engineering knowledge, are then used
to troubleshoot why the problem occurred.

3. Improving, optimizing and controlling processes

We have already introduced the concept of designed experiments and response surface methods
(page 319). These are excellent tools to intentionally manipulate your process so that you can find a
more optimal operating point, or even develop a new product. We will show how latent variable
tools can be used on a large historical data set to improve process operation, and to move to a new
operating point. There are also tools for applying process control in the latent variable space.

4. Predictive modelling (inferential sensors)

The section on least squares modelling (page 209) provided you with a tool for making predictions.
We will show some powerful examples of how a “difficult-to-measure” variable can be predicted in
real-time, using other easy-to-obtain process data. Least squares modelling is a good tool, but it
lacks some of the advantages that latent variable methods provide, such as the ability to handle
highly collinear data, and data with missing values.

5. Process monitoring

Once a process is running, we require monitoring tools to ensure that it maintains and stays at
optimal performance. We have already considered process monitoring charts (page 149) for univariate
process monitoring. In this section we extend that concept to monitoring multiple variables.

6.3.1 The types of data engineers deal with now

When industrial manufacturing and chemical engineering started to develop around the 1920’s to
1950’s, data collected from a process were, at most, just a handful of columns. These data were
collected manually and often at considerable expense.

The “classical” tools required to visualize and understand these datasets are scatter plots (page 12),
time-series plots (page 2), Shewhart charts (page 153) and EWMA charts (page 164) for process
monitoring, and multiple linear regression (page 253) (MLR) least-squares models; all the tools which we
have already learned about so far.

We will represent any data set as a matrix, called X, where each row in X contains values taken from
an object of some sort. These rows, or observations could be a collection of measurements at a particular
point in time, various properties on a sample of final product, or a sample of raw material from a
supplier. The columns in X are the values recorded for each observation. We call these the variables
and there are 𝐾 of them.

6.3. Extracting value from data 413

Process Improvement Using Data

These data sets from the 1950’s frequently had many more rows than columns, because it was
expensive and time-consuming to measure additional columns. The choice of which columns to
measure was carefully thought out, so that they didn’t unnecessarily duplicate the same measurement.
As a result:

• the columns of X were often independent, with little or no overlapping information

• the variables were measured in a controlled environment, with a low amount of error

These data sets meet all the assumptions required to use the so-called “classical” tools, especially least
squares modelling. Data sets that engineers currently deal with though can be of any configuration
with both large and small 𝑁 and large and small 𝐾, but more likely we have many columns for each
observation.

Small N and small K

These cases are mostly for when we have expensive measurements, and they are hard to obtain
frequently. Classical methods to visualize and analyze these data always work well: scatterplots,
linear regression, etc.

Small N and large K

This case is common for laboratory instrumentation, particularly spectroscopic devices. In recent
years we are routinely collecting large quantities of data. A typical example is with near-infrared
probes embedded at-line. These probes record a spectral response at around 1000 to 2000
different wavelengths. The data are represented in X using one wavelength per column and each
sample appears in a row. The illustration here shows data from 𝑁 = 460 samples, with data
recorded every 2 nm (𝐾 = 650).

Obviously not all the columns in this matrix are important; some regions are more useful than
others, and columns immediately adjacent to each other are extremely similar (non-independent).

An ordinary least squares regression model, where we would like to predict some 𝑦-variable
from these spectral data, cannot be calculated when 𝐾 > 𝑁 , since we are then estimating more
unknowns than we have observations for. A common strategy used to deal with

414 Chapter 6. Latent Variable Modelling

 Release 547b16

non-independence is to select only a few columns (wavelengths in the spectral example) so that
𝐾 < 𝑁 . The choice of columns is subjective, so a better approach is required, such as projection to
latent structures (page 472).

Large N and small K

A current-day chemical refinery easily records about 2 observations (rows) per second on around
2000 to 5000 variables (called tags); generating in the region of 50 to 100 Mb of data per second.

For example, a modest size distillation column would have about 35 temperature measurements,
5 to 10 flow rates, 10 or so pressure measurements, and then about 5 more measurements derived
from these recorded values.

An industrial distillation example160 is given on the data set website with 𝐾 = 27, from a small
column in Canada.

N approximately equal to K

The case of squarish matrices mostly occurs by chance: we just happen to have roughly the same
number of variables as observations.

X and Y matrices

This situation arises when we would like to predict one or more variables from another group of
variables. We have already seen this data structure in the least squares section (page 253) where
𝑀 = 1, but more generally we would like to predict several 𝑦-values from the same data in X.

The “classical” solution to this problem is to build and maintain 𝑀 different least squares
models. We will see in the section on projection to latent structures (page 472) that we can build a
single regression model. The sections on principal component regression (page 468) also
investigates the above data structure, but for single 𝑦-variables.

3D data sets and higher dimensions

These data tables are becoming very common, especially since 2000 onwards. A typical example
is for image data from digital cameras. In this illustration a single image is taken at a point in
time. The camera records the response at 6 different wavelengths, and the 𝑥 − 𝑦 spatial directions
(top-to-bottom and left-to-right). These values are recorded in a 3D data cube.
160 [Link]

6.3. Extracting value from data 415

Process Improvement Using Data

A fourth dimension can be added to this data if we start recording images over time. Such
systems generate between 1 and 5 Mb of data per second. As with the spectral data set
mentioned earlier, these camera systems generate large quantities of redundant data, because
neighbouring pixels, both in time and spatially, are so similar. It is a case of high noise and little
real information.

Batch data sets

Batch systems are common with high-value products: pharmaceuticals, fine-chemicals, and
polymers. The Z matrix below contains data that describes how the batch is prepared and also
contains data that is constant over the duration of the whole batch. The X matrix contains the
recorded values for each variable over the duration of the batch. For example, temperature
ramp-up and ramp-down, flow rates of coolant, agitator speeds and so on. The final product
properties, recorded at the end of the batch, are collected in matrix Y.

An example of batch trajectory data, in matrix X, where there are 4 variables, recorded at 80
times points, on about 20 batches is shown here:

416 Chapter 6. Latent Variable Modelling

 Release 547b16

Data fusion

This is a recent buzz-word that simply means we collect and use data from multiple sources.
Imagine the batch system above: we already have data in Z recorded by manual entry, data in X
recorded by sensors on the process, and then Y, typically from lab measurements. We might
even have a near infrared probe in the reactor that provides a complete spectrum (a vector) at
each point in time. The process of combining these data sets together is called data fusion. Each
data set is often referred to as a block. We prefer to use the term multiblock data analysis when
dealing with combined data sets.

6.3.2 Issues faced with engineering data

Size of the data

The most outstanding feature of the above data sets is their large size, both in terms of the
number of rows and columns. This is primarily because data acquisition and data storage has
become cheap.

The number of rows isn’t too big of a deal: we can sub-sample the data, use parallel processors
on our computers or distributed computing (a.k.a. cloud computing) to deal with this. The
bigger problem is the number of columns in the data arrays. A data set with 𝐾 columns can be
visualized using 𝐾(𝐾 − 1)/2 pairs of scatterplots (page 422); this is manageable for 𝐾 < 8, but the
quadratic number of combinations prevents us from using scatterplot matrices to visualize this
data, especially when 𝐾 > 10.

The need here is for a tool that deals with large 𝐾.

Lack of independence

The lack of independence is a big factor in modern data sets - it is problematic for example with
MLR where the X′ X becomes singular as the data become more dependent. Sometimes we can

6.3. Extracting value from data 417

Process Improvement Using Data

make our data more independent by selecting a reduced number of columns, but this requires
good knowledge of the system being investigated, is time-consuming, and we risk omitting
important variables.

Low signal to noise ratio

Engineering systems are usually kept as stable as possible: the ideal being a flat line. Data from
such systems have very little signal and high noise. Even though we might record 50 Mb per
second from various sensors, computer systems can, and actually do, “throw away” much of the
data. This is not advisable from a multivariate data analysis perspective, but the reasoning
behind it is hard to fault: much of the data we collect is not very informative. A lot of it is just
from constant operation, noise, slow drift or error.

Finding the interesting signals in these routine data (also known as happenstance data), is a
challenge.

Non-causal data

This happenstance data is also non-causal. The opposite case is when one runs a designed
experiment; this intentionally adds variability into a process, allowing us to conclude
cause-and-effect relationships, if we properly block and randomize.

But happenstance data just allows us to draw inference based on correlation effects. Since
correlation is a prerequisite for causality, we can often learn a good deal from the correlation
patterns in the data. Then we use our engineering knowledge to validate any correlations, and
we can go on to truly verify causality with a randomized designed experiment, if it is an
important effect to verify.

Errors in the data

Tools, such as least squares analysis, assume the recorded data has no error. But most
engineering systems have error in their measurements, some of it quite large, since much of the
data is collected by automated systems under non-ideal conditions.

So we require tools that relax the assumption that measurements have no error.

Missing data

Missing data are very common in engineering applications. Sensors go off-line, are damaged, or
it is simply not possible to record all the variables (attributes) on each observation. Classical
approaches are to throw away rows or columns with incomplete information, which might be
acceptable when we have large quantities of data, but could lead to omitting important
information in many cases.

In conclusion, we require methods that:

• are able to rapidly extract the relevant information from a large quantity of data

• deal with missing data

• deal with 3-D and higher dimensional data sets

• be able to combine data on the same object, that is stored in different data tables

• handle collinearity in the data (low signal to noise ratio)

• assume measurement error in all the recorded data.

Latent variable methods are a suitable tool that meet these requirements.

418 Chapter 6. Latent Variable Modelling

 Release 547b16

6.4 What is a latent variable?

We will take a look at what a latent variable is conceptually, geometrically, and mathematically.

6.4.1 Your health

Your overall health is a latent variable. But there isn’t a single measurement of “health” that can be
measured - it is a rather abstract concept. Instead we measure physical properties from our bodies,
such as blood pressure, cholesterol level, weight, various distances (waist, hips, chest), blood sugar,
temperature, and a variety of other measurements. These separate measurements can be used by a
trained person to judge your health, based on their experience of seeing these values from a variety of
healthy and unhealthy patients.

In this example, your health is a latent, or hidden variable. If we had a sensor for health, we could
measure and use that variable, but since we don’t, we use other measurements which all contribute in
some way to assessing health.

6.4.2 Room temperature

Conceptually

Imagine the room you are in has 4 temperature probes that sample and record the local temperature
every 30 minutes. Here is an example of what the four measurements might look like over 3 days.

In table form, the first few measurements are:

6.4. What is a latent variable? 419

Process Improvement Using Data

Date 𝑥1 𝑥2 𝑥3 𝑥4
Friday 11:00 295.2 297.0 295.8 296.3
Friday 11:30 296.2 296.4 296.2 296.3
Friday 12:00 297.3 297.5 296.7 297.1
Friday 12:30 295.9 296.7 297.4 297.0
Friday 13:00 297.2 296.5 297.6 297.4
Friday 13:30 296.6 297.7 296.7 296.5

The general up and down fluctuations are due to the daily change in the room’s temperature. The
single, physical phenomenon being recorded in these four measurements is just the variation in room
temperature.

If we added two more thermometers in the middle of the room, we would expect these new
measurements to show the same pattern as the other four. In that regard we can add as many
thermometers as we like to the room, but we won’t be recording some new, independent piece of
information with each thermometer. There is only one true variable that drives all the temperature
readings up and down: it is a latent variable.

Notice that we don’t necessarily have to know what causes the latent variable to move up and down (it
could be the amount of sunlight on the building; it could be the air-conditioner’s settings). All we
know is that these temperature measurements just reflect the underlying phenomenon that drives the
up-and-down movements in temperature; they are correlated with the latent variable.

Notice also the sharp spike recorded at the back-left corner of the room could be due to an error in the
temperature sensor. And the front part of the room showed a dip, maybe because the door was left
open for an extended period; but not long enough to affect the other temperature readings. These two
events go against the general trend of the data, so we expect these periods of time to stand out in some
way, so that we can detect them.

Mathematically

If we wanted to summarize the events taking place in the room we might just use the average of the
recorded temperatures. Let’s call this new, average variable 𝑡1 , which summarizes the other four
original temperature measurements 𝑥1 , 𝑥2 , 𝑥3 and 𝑥4 .
⎡ ⎤
𝑝1,1
[︁ ]︁ ⎢𝑝 ⎥
⎢ 2,1 ⎥
𝑡1 = 𝑥1 𝑥2 𝑥3 𝑥4 ⎢ ⎥ = 𝑥1 𝑝1,1 + 𝑥2 𝑝2,1 + 𝑥3 𝑝3,1 + 𝑥4 𝑝4,1
⎣𝑝3,1 ⎦
𝑝4,1

and suitable values for each of the weights are 𝑝1,1 = 𝑝2,1 = 𝑝3,1 = 𝑝4,1 = 1/4.

Mathematically the correct way to say this is that 𝑡1 is a linear combination of the raw measurements
(𝑥1 , 𝑥2 , 𝑥3 and 𝑥4 ) given by the weights (𝑝1,1 , 𝑝2,1 , 𝑝3,1 , 𝑝4,1 ).

Geometrically

We can visualize the data from this system in several ways, but we will simply show a 3-D
representation of the first 3 temperatures: 𝑥1 , 𝑥2 , 𝑥3 .

420 Chapter 6. Latent Variable Modelling

 Release 547b16

The 3 plots show the same set of data, just from different points of view. Each observation is a single
dot, the location of which is determined by the recorded values of temperature, 𝑥1 , 𝑥2 and 𝑥3 . We will
use this representation in the next section again.

Note how correlated the data appear: forming a diagonal line across the cube’s interior, with a few
outliers (described above) that don’t obey this trend.

The main points from this section are:

• Latent variables capture, in some way, an underlying phenomenon in the system being investigated.

• After calculating the latent variables in a system, we can use these fewer number of variables,
instead of the 𝐾 columns of raw data. This is because the actual measurements are correlated with
the latent variable.

The examples given so far showed what a single latent variables is. In practice we usually obtain
several latent variables for a data array. At this stage you likely have more questions, such as “how
many latent variables are there in a matrix” and “how are the values in P chosen”, and “how do we know these
latent variables are a good summary of the original data”?

We address these issues more formally in the next section on principal component analysis (page 421).

6.5 Principal Component Analysis (PCA)

Principal component analysis, PCA, builds a model for a matrix of data.

A model is always an approximation of the system from where the data came. The objectives for
which we use that model can be varied (page 413).

In this section we will start by visualizing the data as well as consider a simplified, geometric view of
what a PCA model looks like. A mathematical analysis of PCA is also required to get a deeper
understanding of PCA, so we go into some detail on that point, however it can be skipped on first
reading.

The first part of this section emphasizes the general interpretation of a PCA model, since this is a
required step that any modeller will have to perform. We leave to the second half of this section
(page 448) the important details of how to preprocess the raw data, how to actually calculate the PCA
model, and how to validate and test it. This “reverse” order may be unsatisfying for some, but it is
helpful to see how to use the model first, before going into details on its calculation.

6.5. Principal Component Analysis (PCA) 421

Process Improvement Using Data

6.5.1 Visualizing multivariate data

The data, collected in a matrix X, contains rows that represent an object of some sort. We usually call
each row an observation. The observations in X could be a collection of measurements from a chemical
process at a particular point in time, various properties of a final product, or properties from a sample
of raw material. The columns in X are the values recorded for each observation. We call these the
variables.

Which variables should you use, and how many observations do you require? We address this issue
later. For now though we consider that you have your data organized in this manner:

Consider the case of 2 variables, 𝐾 = 2 (left) and 𝐾 = 3 variables (right) for the room thermometers
example from earlier (page 419):

Each point in the plot represents one object, also called an observation. There are about 150 observations
in each plot here. We sometimes call these plots data swarms, but they are really just ordinary
scatterplots that we saw in the visualization section (page 1). Notice how the variables are correlated
with each other, there is a definite trend. If we want to explain this trend, we could draw a line
through the cloud swarm that best explains the data. This line now represents our best summary and
estimate of what the data points are describing. If we wanted to describe that relationship to our
colleagues we could just give them the equation of the best-fit line.

Another effective way to visualize small multivariate data sets is to use a scatterplot matrix. Below is
an example for 𝐾 = 5 measurements on 𝑁 = 50 observations. Scatterplot matrices require 𝐾(𝐾 − 1)/2
plots and can be enhanced with univariate histograms (on the diagonal plots), and linear regressions
and loess smoothers on the off-diagonals to indicate the level of correlation between any two variables.

422 Chapter 6. Latent Variable Modelling

 Release 547b16

6.5.2 Geometric explanation of PCA

We refer to a 𝐾-dimensional space when referring to the data in X. We will start by looking at the
geometric interpretation of PCA when X has 3 columns, in other words a 3-dimensional space, using
measurements: [𝑥1 , 𝑥2 , 𝑥3 ].

The raw data in the cloud swarm show how the 3 variables move together. The first step in PCA is to
move the data to the center of the coordinate system. This is called mean-centering and removes the
arbitrary bias from measurements that we don’t wish to model. We also scale the data, usually to
unit-variance. This removes the fact that the variables are in different units of measurement.
Additional discussion on centering and scaling is in the section on data preprocessing (page 448).

After centering and scaling we have moved our raw data to the center of the coordinate system and
each variable has equal scaling.

The best-fit line is drawn through the swarm of points. The more correlated the original data, the
better this line will explain the actual values of the observed measurements. This best-fit line will best

6.5. Principal Component Analysis (PCA) 423

Process Improvement Using Data

explain all the observations with minimum residual error. Another, but equivalent, way of expressing
this is that the line goes in the direction of maximum variance of the projections onto the line. Let’s take a
look at what that phrase means.

When the direction of the best-fit line is found we can mark the location of each observation along the
line. We find the 90 degree projection of each observation onto the line (see the next illustration). The
distance from the origin to this projected point along the line is called the score. Each observation gets
its own score value. When we say the best-fit line is in the direction of maximum variance, what we
are saying is that the variance of these scores will be maximal. (There is one score for each observation,
so there are 𝑁 score values; the variance of these 𝑁 values is at a maximum). Notice that some score
values will be positive and others negative.

After we have added this best-fit line to the data, we have calculated the first principal component,
also called the first latent variable. Each principal component consists of two parts:

• The direction vector that defines the best-fit line. This is a 𝐾-dimensional vector that tells us which
direction that best-fit line points, in the 𝐾-dimensional coordinate system. We call this direction
vector p1 , it is a 𝐾 × 1 vector. This vector starts at the origin and moves along the best-fit line. Since
vectors have both magnitude and direction, we chose to rescale this vector so that it has magnitude
of exactly 1, making it a unit-vector.

• The collection of 𝑁 score values along this line. We call this our score vector, t1 , and it is an 𝑁 × 1
vector.

• The subscript of “1” emphasizes that this is the first latent variable.

424 Chapter 6. Latent Variable Modelling

 Release 547b16

This first principal component is fixed and we now add a second component to the system. We find
the second component so that it is perpendicular to the first component’s direction. Notice that this
vector also starts at the origin, and can point in any direction as long as it remains perpendicular to the
first component. We keep rotating the second component’s direction vector around until we find a
direction that gives the greatest variance in the score values when projected on this new direction
vector.

What that means is that once we have settled on a direction for the second component, we calculate
the scores values by perpendicularly projecting each observation towards this second direction vector.
The score values for the second component are the locations along this line. As before, there will be
some positive and some negative score values. This completes our second component:

• This second direction vector, called p2 , is also a 𝐾 × 1 vector. It is a unit vector that points in the
direction of next-greatest variation.

• The scores (distances), collected in the vector called t2 , are found by taking a perpendicular
projection from each observation onto the p2 vector.

Notice that the p1 and p2 vectors jointly define a plane. This plane is the latent variable model with two
components. With one component the latent variable model is just a line, with two components, the
model is a plane, and with 3 or more components, the model is defined by a hyperplane. We will use
the letter 𝑎 to identify the number of components. The PCA model is said to have 𝐴 components, or 𝐴
latent variables, where 𝑎 = 1, 2, 3, . . . 𝐴.

This hyperplane is really just the best approximation we can make of the original data. The
perpendicular distance from each point onto the plane is called the residual distance or residual error. So
what a principal component model does is break down our raw data into two parts:

1. a latent variable model (given by vectors p and t), and

2. a residual error.

A principal component model is one type of latent variable model. A PCA model is computed in such
a way that the latent variables are oriented in the direction that gives greatest variance of the scores. There
are other latent variable models, but they are computed with different objectives.

6.5. Principal Component Analysis (PCA) 425

Process Improvement Using Data

6.5.3 Mathematical derivation for PCA

Geometrically, when finding the best-fit line for the swarm of points, our objective was to minimize the
error, i.e. the residual distances from each point to the best-fit line is the smallest possible. This is also
mathematically equivalent to maximizing the variance of the scores, t𝑎 .

We briefly review here what that means. Let x′𝑖 be a row from our data, so x′𝑖 is a 1 × 𝐾 vector. We
defined the score value for this observation as the distance from the origin, along the direction vector,
p1 , to the point where we find the perpendicular projection onto p1 . This is illustrated below, where
the score value for observation x𝑖 has a value of 𝑡𝑖,1 .

Recall from geometry that the cosine of an angle in a right-angled triangle is the ratio of the adjacent
side to the hypotenuse. But the cosine of an angle is also used in linear algebra to define the
dot-product. Mathematically:

adjacent length 𝑡𝑖,1 x′𝑖 p1

cos 𝜃 = = and also cos 𝜃 =
hypotenuse ‖x𝑖 ‖ ‖x𝑖 ‖‖p1 ‖
𝑡𝑖,1 x′𝑖 p1
=
‖x𝑖 ‖ ‖x𝑖 ‖‖p1 ‖
𝑡𝑖,1 = x′𝑖 p1
(1 × 1) = (1 × 𝐾)(𝐾 × 1)

where ‖ · ‖ indicates the length of the enclosed vector, and the length of the direction vector, p1 is 1.0,
by definition.

Note that 𝑡𝑖,1 = x′𝑖 p1 represents a linear combination (page 420)

𝑡𝑖,1 = 𝑥𝑖,1 𝑝1,1 + 𝑥𝑖,2 𝑝2,1 + . . . + 𝑥𝑖,𝑘 𝑝𝑘,1 + . . . + 𝑥𝑖,𝐾 𝑝𝐾,1

So 𝑡𝑖,1 is the score value for the 𝑖th observation along the first component, and is a linear combination
of the 𝑖th row of data, x𝑖 and the direction vector p1 . Notice that there are 𝐾 terms in the linear
combination: each of the 𝐾 variables contributes to the overall score.

We can calculate the second score value for the 𝑖th observation in a similar way:

𝑡𝑖,2 = 𝑥𝑖,1 𝑝1,2 + 𝑥𝑖,2 𝑝2,2 + . . . + 𝑥𝑖,𝑘 𝑝𝑘,2 + . . . + 𝑥𝑖,𝐾 𝑝𝐾,2

And so on, for the third and subsequent components. We can compactly write in matrix form for the
𝑖th observation that:

t′𝑖 = x′𝑖 P
(1 × 𝐴) = (1 × 𝐾)(𝐾 × 𝐴)

which calculates all 𝐴 score values for that observation in one go. This is exactly what we derived earlier
(page 420) in the example with the 4 thermometers in the room.

426 Chapter 6. Latent Variable Modelling

 Release 547b16

Finally, for an entire matrix of data, X, we can calculate all scores, for all observations:

T = XP
(6.1)
(𝑁 × 𝐴) = (𝑁 × 𝐾)(𝐾 × 𝐴)

6.5.4 More about the direction vectors (loadings)

The direction vectors p1 , p2 and so on, are each 𝐾 × 1 unit vectors. These are vectors in the original
coordinate space (the 𝐾-dimensional real-world) where the observations are recorded.

But these direction vectors are also our link to the latent-variable coordinate system. These direction
vectors create a (hyper)plane that is embedded inside the 𝐾-dimensional space of the 𝐾 original
variables. You will see the terminology of loadings - this is just another name for these direction vectors:
[︁ ]︁
Loadings, a 𝐾 × 𝐴 matrix: P = p1 p2 . . . p𝐴

Once this hyperplane is mapped out, then we start to consider how each of the observations lie on this
hyperplane. We start to be more and more interested in this reduced dimensional plane, because it is
an 𝐴-dimensional plane, where 𝐴 is often much smaller than 𝐾. Returning back to the case of the
thermometers in a room: we had 4 thermometers (𝐾 = 4), but only one latent variable, 𝐴 = 1. Rather
than concern ourself with the original 4 measurements, we only focus on the single column of score
values, since this single variables is the best summary possible of the 4 original variables.

How do we get the score value(s)? We use the equation from the prior section (page 426) (repeated here).
It is the multiplication of the pre-processed data by the loadings vectors:

T = XP
(𝑁 × 𝐴) = (𝑁 × 𝐾)(𝐾 × 𝐴)
and it shows how the loadings are our link from the 𝐾-dimensional, real-world, coordinate system to
the 𝐴-dimensional, latent variable-world, coordinates.

Let’s return to the example of the 4 temperatures (page 419). We derived there that a plausible summary
of the 4 temperatures could be found from:
⎡ ⎤ ⎡ ⎤
𝑝1,1 0.25
[︁ ]︁ ⎢𝑝 ⎥ [︁ ]︁ ⎢0.25⎥
⎢ 2,1 ⎥
𝑡1 = 𝑥1 𝑥2 𝑥3 𝑥4 ⎢ ⎥ = 𝑥1 𝑥2 𝑥3 𝑥4 ⎢ ⎥ = x𝑖 p1
⎢ ⎥
⎣𝑝3,1 ⎦ ⎣0.25⎦
𝑝4,1 0.25
So the loading vector for this example points in the direction p′1 = [0.25, 0.25, 0.25, 0.25]. This isn’t a
unit vector though; but we can make it one:
√
• Current magnitude of vector = 0.252 + 0.252 + 0.252 + 0.252 = 0.50
1
• Divide the vector by current magnitude: p1 = · [0.25, 0.25, 0.25, 0.25]
0.5
• New, unit vector = p1 = [0.5, 0.5, 0.5, 0.5]
√
• Check new magnitude = 0.52 + 0.52 + 0.52 + 0.52 = 1.0

What would be the entries in the p1 loading vector if we had 6 thermometers? (Ans = 0.41; in general,
√
for 𝐾 thermometers, 1/ 𝐾).

This is very useful, because now instead of dealing with 𝐾 thermometers we can reduce the columns
of data down to just a single, average temperature. This isn’t a particularly interesting case though;
you would have likely done this anyway as an engineer facing this problem. But the next food texture
example (page 428) will illustrate a more realistic case.

6.5. Principal Component Analysis (PCA) 427

Process Improvement Using Data

6.5.5 PCA example: Food texture analysis

Let’s take a look at an example to consolidate and extend the ideas introduced so far. This data set is
from a food manufacturer161 making a pastry product. Each sample (row) in the data set is taken from
a batch of product where 5 quality attributes are measured:

1. Percentage oil in the pastry

2. The product’s density (the higher the number, the more dense the product)

3. A crispiness measurement, on a scale from 7 to 15, with 15 being more crispy.

4. The product’s fracturability: the angle, in degrees, through which the pasty can be slowly bent
before it fractures.

5. Hardness: a sharp point is used to measure the amount of force required before breakage occurs.

A scatter plot matrix of these 𝐾 = 5 measurements is shown for the 𝑁 = 50 observations.

We can get by with this visualization of the data because 𝐾 is small in this case. This is also a good
starting example, because you can refer back to these scatterplots to confirm your findings.

Preprocessing the data

The first step with PCA is to center and scale the data. The box plots show how the raw data are
located at different levels and have arbitrary units.
161 [Link]

428 Chapter 6. Latent Variable Modelling

 Release 547b16

Centering removes any bias terms from the data by subtracting the mean value from each column in
the matrix X. For the 𝑘 th column:

x𝑘,center = x𝑘,raw − mean (x𝑘,raw )

Scaling removes the fact that the raw data could be in diverse units:
x𝑘,center
x𝑘 =
standard deviation (x𝑘,center )

Then each column x𝑘 is collected back to form matrix X. This preprocessing is so common it is called
autoscaling: center each column to zero mean and then scale it to have unit variance. After this
preprocessing each column will have a mean of 0.0 and a variance of 1.0. (Note the box plots don’t
quite show this final result, because they use the median instead of the mean, and show the
interquartile range instead of the standard deviation).

Centering and scaling does not alter the overall interpretation of the data: if two variables were
strongly correlated before preprocessing they will still be strongly correlated after preprocessing.

For reference, the mean and standard deviation of each variable is recorded below. In the last 3
columns we show the raw data for observation 33, the raw data after centering, and the raw data after
centering and scaling:

Variable Mean Standard deviation Raw data After centering After autoscaling
Oil 17.2 1.59 15.5 -1.702 -1.069
Density 2857.6 124.5 3125 267.4 +2.148
Crispy 11.52 1.78 7 -4.52 -2.546
Fracture 20.86 5.47 33 12.14 +2.221
Hardness 128.18 31.13 92 -36.18 -1.162

Loadings: p1

We will discuss how to determine the number of components to use in a future section (page 456), and
how to compute them (page 450), but for now we accept there are two important components, p1 and p2 .
They are:
⎡ ⎤ ⎡ ⎤
+0.46 −0.37
⎢−0.47⎥ ⎢+0.36⎥
⎢ ⎥ ⎢ ⎥
p1 = ⎢+0.53⎥ and p2 = ⎢+0.20⎥
⎢ ⎥ ⎢ ⎥
⎢ ⎥ ⎢ ⎥
⎣−0.50⎦ ⎣−0.22⎦
+0.15 +0.80

6.5. Principal Component Analysis (PCA) 429

Process Improvement Using Data

Where we might visualize that first component by a bar plot:

This plot shows the first

component. All variables, except for hardness have large values in p1 . If we write out the equation for
𝑡1 for an observation 𝑖:

𝑡𝑖,1 = 0.46 𝑥oil − 0.47 𝑥density + 0.53 𝑥crispy − 0.50 𝑥fracture + 0.15 𝑥hardness (6.2)

Once we have centered and scaled the data, remember that a negative 𝑥-value is a value below the
average, and that a positive 𝑥-value lies above the average.

For a pastry product to have a high 𝑡1 value would require it to have some combination of
above-average oil level, low density, and/or be more crispy and/or only have a small angle by which it
can be bent before it fractures, i.e. low fracturability. So pastry observations with high 𝑡1 values sound
like they are brittle, flaky and light. Conversely, a product with low 𝑡1 value would have the opposite
sort of conditions: it would be a heavier, more chewy pastry (higher fracture angle) and less crispy.

Scores: t1

Let’s examine the score values calculated. As shown in equation (6.2), the score value is a linear
combination of the data, x, given by the weights in the loadings matrix, P. For the first component,
t1 = Xp1 . The plot here shows the values in vector t1 (an 𝑁 × 1 vector) as a sequence plot

The samples appear to be evenly spread, some high and some low on the 𝑡1 scale. Sample 33 has a 𝑡1
value of -4.2, indicating it was much denser than the other pastries, and had a high fracture angle (it
could be bent more than others). In fact, if we refer to the raw data162 we can confirm these findings:
x𝑖=33 = [15.5, 3125, 7, 33, 92]. Also refer back to the scatterplot matrix and mark the point which has
162 [Link]

430 Chapter 6. Latent Variable Modelling

 Release 547b16

density of 3125, and fracture angle of 33. This pastry also has a low oil percentage (15.5%) and low
crispy value (7).

We can also investigate sample 36, with a 𝑡1 value of 3.6. The raw data again confirm that this pastry
follows the trends of other, high 𝑡1 value pastries. It has a high oil level, low density, high crispiness,
and a low fracture angle: 𝑥36 = [21.2, 2570, 14, 13, 105]. Locate again on the scatterplot matrices
sample 36 where oil level is 21.2 and the crispiness is 14. Also mark the point where density = 2570
and the fracture value = 13 for this sample.

We note here that this component explains 61% of the original variability in the data. It’s hard to say
whether this is high or low, because we are unsure of the degree of error in the raw data, but the point
is that a single variable summarizes about 60% of the variability from all 5 columns of raw data.

Loadings: p2

The second loading vector is shown as a bar plot:

This direction is aligned mainly with the hardness variable: all other variables have a small coefficient
in p2 . A high 𝑡2 value is straightforward to interpret: it would imply the pastry has a high value on the
hardness scale. Also, this component explains an additional 26% of the variability in the dataset.

Because this component is orthogonal to the first component, we can be sure that this hardness
variation is independent of the first component. One valuable way to interpret and use this
information is that you can adjust the variables in p2 , i.e. the process conditions that affect the pastry’s
hardness, without affecting the other pastry properties, i.e the variables described in p1 .

6.5.6 Interpreting score plots

Before summarizing some points about how to interpret a score plot, let’s quickly repeat what a score
value is. There is one score value for each observation (row) in the data set, so there are are 𝑁 score
values for the first component, another 𝑁 for the second component, and so on.

The score value for an observation, for say the first component, is the distance from the origin, along
the direction (loading vector) of the first component, up to the point where that observation projects
onto the direction vector. We repeat an earlier figure here (page 423), which shows the projected values
for 2 of the observations.

6.5. Principal Component Analysis (PCA) 431

Process Improvement Using Data

We used geometric concepts in another section (page 426) that showed we can write: T = XP to get all
the scores value in one go. In this section we are plotting values from the columns of T. In particular,
for a single observation, for the 𝑎th component:

𝑡𝑖,𝑎 = 𝑥𝑖,1 𝑝1,𝑎 + 𝑥𝑖,2 𝑝2,𝑎 + . . . + 𝑥𝑖,𝑘 𝑝𝑘,𝑎 + . . . + 𝑥𝑖,𝐾 𝑝𝐾,𝑎

The first score vector, t1 ,explains the greatest variation in the data; it is considered the most important
score from that point of view, at least when we look at a data set for the first time. (After that we may
find other scores that are more interesting). Then we look at the second score, which explains the next
greatest amount of variation in the data, then the third score, and so on. Most often we will plot:

• time-series plots of the scores, or sequence order plots, depending on how the rows of X are ordered

• scatter plots of one score against another score

An important point with PCA is that because the matrix P is orthonormal (see the later section on PCA
properties (page 458)), any relationships that were present in X are still present in T. We can see this
quite easily using the previous equation. Imagine two observations taken from a process at different
points in time. It would be quite hard to identify those similar points by looking at the 𝐾 columns of
raw data, especially when the two rows are not close to each other. But with PCA, these two similar
rows are multiplied by the same coefficients in P and will therefore give similar values of 𝑡. So score
plots allow us to rapidly locate similar observations.

When investigating score plots we look for clustering, outliers, time-based patterns. We can also
colour-code our plots to be more informative. Let’s take a look at each of these.

Clustering

We usually start by looking at the (t1 , t2 ) scatterplot of the scores, the two directions of greatest
variation in the data. As just previously explained, observations in the rows of X that are similar will
fall close to each other, i.e. they cluster together, in these score plots. Here is an example of a score plot,
calculated from data from a fluidized catalytic cracking (FCC) process [Taken from the Masters thesis
of Carol Slama (McMaster University, p 78, 1991)].

432 Chapter 6. Latent Variable Modelling

 Release 547b16

It shows how the process was operating in region A, then moved to region B and finally region C. This
provides a 2-dimensional window into the movements from the 𝐾 = 147 original variables.

Outliers

Outliers are readily detected in a score plot, and using the equation below we can see why. Recall that
the data in X have been centered and scaled, so the 𝑥-value for a variable that is operating at the mean
level will be roughtly zero. An observation that is at the mean value for all 𝐾 variables will have a
score vector of t𝑖 = [0, 0, . . . , 0]. An observation where many of the variables have values far from their
average level is called a multivariate outlier. It will have one or more score values that are far from
zero, and will show up on the outer edges of the score scatterplots.

Sometimes all it takes is for one variable, 𝑥𝑖,𝑘 to be far away from its average to cause 𝑡𝑖,𝑎 to be large:

𝑡𝑖,𝑎 = 𝑥𝑖,1 𝑝1,𝑎 + 𝑥𝑖,2 𝑝2,𝑎 + . . . + 𝑥𝑖,𝑘 𝑝𝑘,𝑎 + . . . + 𝑥𝑖,𝐾 𝑝𝐾,𝑎

But usually it is a combination of more than one 𝑥-variable. There are 𝐾 terms in this equation, each of
which contribute to the score value. A bar plot of each of these 𝐾 terms, 𝑥𝑖,𝑘 𝑝𝑘,𝑎 , is called a
contribution plot. It shows which variable(s) most contribute to the large score value.

As an example from the food texture data (page 428) from earlier, we saw that observation 33 had a large
negative t1 value. From that prior equation (page 430):

𝑡33,1 = 0.46 𝑥oil − 0.47 𝑥density + 0.53 𝑥crispy − 0.50 𝑥fracture + 0.15 𝑥hardness
𝑡33,1 = 0.46 × −1.069 − 0.47 × +2.148 + 0.53 × −2.546 − 0.50 × 2.221 + 0.15 × −1.162
𝑡33,1 = −4.2

The 𝐾 = 5 terms that contribute to this value are illustrated as a bar plot, where the sum of the bar
heights add up to −4.2:

6.5. Principal Component Analysis (PCA) 433

Process Improvement Using Data

This gives a more accurate indication of exactly how the low 𝑡𝑖 value was achieved. Previously we had
said that pastry 33 was denser than the other pastries, and had a higher fracture angle; now we can see
the relative contributions from each variable more clearly.

In the figure from the FCC process (in the preceding subsection on clustering (page 432)), the cluster
marked C was far from the origin, relative to the other observations. This indicates problematic
process behaviour around that time. Normal process operation is expected to be in the center of the
score plot. These outlying observations can be investigated as to why they are unusual by constructing
contribution bar plots for a few of the points in cluster C.

Time-based or sequence-based trends

Any strong and consistent time-based or sequence-order trends in the raw data will be reflected in the
scores also. Visual observation of each score vector may show interesting phenomena such as
oscillations, spikes or other patterns of interest. As just described, contribution plots can be used to see
which of the original variables in X are most related with these phenomena.

Colour-coding

Plotting any two score variables on a scatter plot provides good insight into the relationship between
those independent variables. Additional information can be provided by colour-coding the points on the
plot (page 15) by some other, 3rd variable of interest. For example, a binary colour scheme could
denote success of failure of each observation.

A continuous 3rd variable can be implied using a varying colour scheme, going from reds to oranges to
yellows to greens and then blue, together with an accompanying legend. For example profitability of
operation at that point, or some other process variable. A 4th dimension could be inferred by plotting
smaller or larger points. We saw an example of these high-density visualizations (page 15) earlier.

Summary

• Points close the average appear at the origin of the score plot.

• Scores further out are either outliers or naturally extreme observations.

• We can infer, in general, why a point is at the outer edge of a score plot by cross-referencing with the
loadings. This is because the scores are a linear combination of the data in X as given by the
coefficients in P.

• We can determine exactly why a point is at the outer edge of a score plot by constructing a
contribution plot to see which of the original variables in X are most related with a particular score.
This provides a more precise indication of exactly why a score is at its given position.

• Original observations in X that are similar to each other will be similar in the score plot, while
observations much further apart are dissimilar. This comes from the way the scores are computed:

434 Chapter 6. Latent Variable Modelling

 Release 547b16

they are found so that span the greatest variance possible. But it is much easier to detect this
similarity in an 𝐴-dimensional space than the original 𝐾-dimensional space.

6.5.7 Interpreting loading plots

Recall that the loadings plot is a plot of the direction vectors that define the model. Returning back to a
previous illustration:

In this system the first component, p1 , is oriented primarily in the 𝑥2 direction, with smaller amounts
in the other directions. A loadings plot would show a large coefficient (negative or positive) for the 𝑥2
variable and smaller coefficients for the others. Imagine this were the only component in the model,
i.e. it is a one-component model. We would then correctly conclude the other variables measured have
little importance or relevance in understanding the total variability in the system. Say these 3 variables
represented the quality of our product, and we had been getting complaints about the variability of it.
This model indicates we should focus on whatever aspect causes in variance in 𝑥2 , rather than other
variables.

Let’s consider another visual example where two variables, 𝑥1 and 𝑥2 , are the predominant directions
in which the observations vary; the 𝑥3 variable is only “noise”. Further, let the relationship between 𝑥1
and 𝑥2 have a negative correlation.

A model of such a system would have a loading vector with roughly equal weight in the +𝑥1 direction
as it has in the −𝑥2 direction. The direction could be represented as 𝑝1 = [+1, −1, 0], or rescaled as a

6.5. Principal Component Analysis (PCA) 435

Process Improvement Using Data

unit vector: 𝑝1 = [+0.707, −0.707, 0]. An equivalent representation, with exactly the same
interpretation, could be 𝑝1 = [−0.707, +0.707, 0].

This illustrates two points:

• Variables which have little contribution to a direction have almost zero weight in that loading.

• Strongly correlated variables, will have approximately the same weight value when they are
positively correlated. In a loadings plot of 𝑝𝑖 vs 𝑝𝑗 they will appear near each other, while
negatively correlated variables will appear diagonally opposite each other.

• Signs of the loading variables are useful to compare within a direction vector; but these vectors can
be rotated by 180° and still have the same interpretation.

This is why they are called loadings: they show how the original variables load, (contribute), to
creating the component.

Another issue to consider is the case when one has many highly correlated variables. Consider the
room temperature example (page 419) where the four temperatures are highly correlated with each other.
The first component from the PCA model is shown here:

Notice how the model spreads the weights out evenly over all the correlated variables. Each variable is
individually important. The model could well have assigned a weight of 1.0 to one of the variables
and 0.0 to the others. This is a common feature in latent variable models: variables which have
roughly equal influence on defining a direction are correlated with each other and will have roughly
equal numeric weights.

Finally, one way to locate unimportant variables in the model is by finding which variables have small
weights in all components. These variables can generally be removed, as they show no correlation to
any of the components or with other variables.

6.5.8 Interpreting loadings and scores together

It is helpful to visualize any two score vectors, e.g. t1 vs t2 , in a scatterplot: the 𝑁 points in the
scatterplot are the projection of the raw data onto the model plane described by the two loadings
vectors, p1 and p2 .

Any two loadings can also be shown in a scatterplot and interpreted by recalling that each loading
direction is orthogonal and independent of the other direction.

436 Chapter 6. Latent Variable Modelling

 Release 547b16

Side-by-side, these 2 plots very helpfully characterize all the observations in the data set. Recall
observation 33 had a large, negative 𝑡1 value. It had an above average fracture angle, an above average
density, a below average crispiness value of 7, and below average oil level of 15.5.

It is no coincidence that we can mentally superimpose these two plots and come to exactly the same
conclusions, using only the plots. This result comes from the fact that the scores (left) are just a linear
combination of the raw data, with weighting given by the loadings (right).

Use these two plots to characterize what values the 5 measurements would have been for these
observations:

• sample 8:

• sample 20:

• sample 35:

• sample 42:

6.5.9 Predicted values for each observation

An interesting aspect of a PCA model is that it provides an estimate of each observation in the data set.
Recall the latent variable model was oriented to create the best-fit plane to the data. This plane was
oriented to minimize the errors, which implies the best estimate of each observation is its perpendicular
projection onto the model plane.

Referring to the illustration and assume we have a PCA model with a single component, the best
estimate of observation x𝑖 is the point along the direction vector, p1 , where the original observation is
projected. Recall that the distance along that direction vector was 𝑡𝑖,1 , but the actual point along p1 is a
vector, and it is our best estimate of the original observation. We will call that estimate x
̂︀𝑖,1 , indicating
that it is an estimate of x𝑖 along the first component.

6.5. Principal Component Analysis (PCA) 437

Process Improvement Using Data

Since x
̂︀𝑖,1 is a vector, we can write it as the product of a magnitude value and a direction vector. The
magnitude of x ̂︀𝑖 is 𝑡𝑖 in the direction of p1 , which is a unit vector, then mathematically we can write:

̂︀′𝑖,1 = 𝑡𝑖,1 p′1

x
(1 × 𝐾) = (1 × 1)(1 × 𝐾)

This is the best prediction of the original observation using one component. If we added a second
component to our model, then our estimate improves:

̂︀′𝑖,2 = 𝑡𝑖,1 p′1 + 𝑡𝑖,2 p′2

x
(1 × 𝐾) = (1 × 𝐾) + (1 × 𝐾)

With multiple components, we write:

[︁ ]︁
̂︀′𝑖,𝐴 = 𝑡𝑖,1
x 𝑡𝑖,2 , . . . , 𝑡𝑖,𝐴 P′
̂︀′𝑖,𝐴 = t′𝑖 P′
x
(1 × 𝐾) = (1 × 𝐴)(𝐴 × 𝐾)

̂︀𝑖,𝐴 is a prediction of every variable in the 𝑖th observation. We only require the
This is interesting: x
score values for that 𝑖th observation in order to get this prediction. We multiply the scores t𝑖 by the
direction vectors in matrix P to get the prediction.

The preceding equation can be written in a way that handles the entire matrix X:
̂︀ = TP′
X
(6.3)
(𝑁 × 𝐾) = (𝑁 × 𝐴)(𝐴 × 𝐾)

Once we have the predicted value for an observation, we are also interested in the residual vector
between the actual and predicted observation:

e′𝑖,𝐴 = x′𝑖 − x
̂︀′𝑖,𝐴
(1 × 𝐾) = (1 × 𝐾) − (1 × 𝐾)

The residual length or distance is the sum of squares of this residual, then we take the square root to
form a distance. Technically the squared prediction error (SPE) is just the sum of squares for each
observation, but often we refer to the square root of this quantity as the SPE as well. Some software
packages will scale the root of the SPE by some value; you will see this referred to as the DModX,
distance to the model plane for X.

√︁
SPE𝑖 = e′𝑖,𝐴 e𝑖,𝐴

(1 × 1) = (1 × 𝐾)(𝐾 × 1)

where e𝑖,𝐴 is the residual vector of the 𝑖th observation using 𝐴 components.

438 Chapter 6. Latent Variable Modelling

 Release 547b16

6.5.10 Interpreting the residuals

We consider three types of residuals: residuals within each row of X, called squared prediction errors
(SPE); residuals for each column of X, called 𝑅𝑘2 for each column, and finally residuals for the entire
matrix X, usually just called 𝑅2 for the model.

Residuals for each observation: the square prediction error

We have already introduced the squared prediction error geometrically (page 437). We showed in that
section that the residual distance from the actual observation to the model plane is given by:

e′𝑖,𝐴 = x′𝑖 − x
̂︀′𝑖,𝐴
e′𝑖,𝐴 = x′𝑖 − t′𝑖 P′

Turning this last equation around we have:

x′𝑖 = t′𝑖 P′ + e′𝑖,𝐴

(1 × 𝐾) = (1 × 𝐴)(𝐴 × 𝐾) + (1 × 𝐾)

Or in general, for the whole data set

X = TP′ + E = X
̂︀ + E
(𝑁 × 𝐾) = (𝑁 × 𝐴)(𝐴 × 𝐾) + (𝑁 × 𝐾)

This shows that each observation (row in X) can be split and interpreted in two portions: a vector
on-the-plane, t′𝑖 P′ , and a vector perpendicular to the plane, e′𝑖,𝐴 . This residual portion, a vector, can be
reduced to a single number, a distance value called SPE, as previously described (page 437).

An observation in X that has SPE𝑖 = 0 is exactly on the plane and follows the model structure exactly;
this is the smallest SPE value possible. For a given data set we have a distribution of SPE values. We
can calculate a confidence limit below which we expect to find a certain fraction of the data, e.g. a 95%
confidence limit. We won’t go into how this limit is derived, suffice to say that most software packages
will compute it and show it.

The most convenient way to visualize these SPE values is as a sequence plot, or a line plot, where the
𝑦-axis has a lower limit of 0.0, and the 95% and/or 99% SPE limit is also shown. Remember that we
would expect 5 out of 100 points to naturally fall above the 95% limit.

If we find an observation that has a large squared prediction error, i.e. the observation is far off the
model plane, then we say this observation is inconsistent with the model. For example, if you have data
from a chemical process, taken over several days, your first 300 observations show SPE values below
the limit. Then on the 4th day you notice a persistent trend upwards in SPE values: this indicates that
those observations are inconsistent with the model, indicating a problem with the process, as reflected
in the data captured during that time.

We would like to know why, specifically which variable(s) in X, are most related with this deviation
off the model plane. As we did in the section on interpreting scores (page 431), we can generate a

6.5. Principal Component Analysis (PCA) 439

Process Improvement Using Data

contribution plot.

e′𝑖,𝐴 = x′𝑖 − x
̂︀′𝑖,𝐴

Dropping the 𝐴 subscript for convenience we can write the 1 × 𝐾 vector as:

e′𝑖 = x′𝑖 − x
̂︀′𝑖
[︁ ]︁
(1 × 𝐾) = (𝑥𝑖,1 − 𝑥 ^𝑖,1 ) (𝑥𝑖,2 − 𝑥
^𝑖,2 ) . . . (𝑥𝑖,𝑘 − 𝑥
^𝑖,𝑘 ) . . . (𝑥𝑖,𝐾 − 𝑥
^𝑖,𝐾 )

The SPE is just the sum of the squares of these 𝐾 terms, so a residual contribution plot, most
conveniently shown as a bar chart of these 𝐾 terms, indicates which of the original 𝐾 variable(s) are
most associated with the deviation off the model plane. We say that the correlation structure among these
variables has been broken. This is because PCA provides a model of the correlation structure in the data
table. When an observation has a large residual, then that observation is said to break the correlation
structure, and is inconsistent with the model.

Looking back at the room-temperature example (page 419): if we fit a model with one component, then
the residual distance, shown with the 95% limit, appears as follows:

Using the raw data for this example163 , shown below, can you explain why we see those unusual
points in the SPE plot around time 50 to 60?
163 [Link]

440 Chapter 6. Latent Variable Modelling

 Release 547b16

Finally, the SPE value is a complete summary of the residual vector. As such, it is sometimes used to
colour-code score plots, as we mentioned back in the section on score plots (page 431). Another
interesting way people sometimes display SPE is to plot a 3D data cloud, with t1 and t2 , and use the
SPE values on the third axis. This gives a fairly complete picture of the major dimensions in the model:
the explained variation on-the-plane, given by t1 and t2 , and the residual distance off-the-plane,
summarized by SPE.

Residuals for each column

Using the residual matrix E = X − TP′ = X − X,̂︀ we can calculate the residuals for each column in the
2
original matrix. This is summarized by the 𝑅 value for each column in X and gives an indication of
how well the PCA model describes the data from that column.

In the section on least squares modelling (page 209), the 𝑅2 number was shown to be the ratio between
the variance remaining in the residuals over the total variances we started off with, subtracted from
1.0. Using the notation in the previous illustration:

Var(x𝑘 − x
̂︀𝑘 ) Var(e𝑘 )
𝑅𝑘2 = 1 − =1−
Var(x𝑘 ) Var(x𝑘 )

The 𝑅𝑘2 value for each variable will increase with every component that is added to the model. The
minimum value is 0.0 when there are no components (since x ̂︀𝑘 = 0), and the maximum value is 1.0,

6.5. Principal Component Analysis (PCA) 441

Process Improvement Using Data

when the maximum number of components have been added (and x ̂︀𝑘 = x𝑘 , or e𝑘 = 0). This latter
extreme is usually not reached, because such a model would be fitting the noise inherent in x𝑘 as well.

The 𝑅2 values for each column can be visualized as a bar plot for dissimilar variables (chemical
process data), or as a line plot if there are many similar variables that have a logical left-to-right
relationship, such as the case with spectral variables (page 442) (wavelengths).

Residuals for the whole matrix X

Finally, we can calculate an 𝑅2 value for the entire matrix X. This is the ratio between the variance of
X we can explain with the model over the ratio of variance initially present in X.

Var(X − X)
̂︀ Var(E)
𝑅2 = 1 − =1−
Var(X) Var(X)

The variance of a general matrix, G, is taken as the sum of squares of every element in G. The example
in the next section illustrates how to interpret these residuals. The smallest value of 𝑅2 value is
2 2
𝑅𝑎=0 = 0.0 when there are no components. After the first component is added we can calculate 𝑅𝑎=1 .
2
Then after fitting a second component we get 𝑅𝑎=2 . Since each component is extracting new
2 2 2 2
information from X, we know that 𝑅𝑎=0 < 𝑅𝑎=1 < 𝑅𝑎=2 < . . . < 𝑅𝑎=𝐴 = 1.0.

6.5.11 PCA example: analysis of spectral data

A data set, available on the dataset website164 , contains data on 460 tablets, measured at 650 different
wavelengths.

This code will calculate principal components for this data:

import pandas as pd
from process_improve.multivariate import PCA, MCUVScaler

# Read large data file

file = "[Link]
spectra = pd.read_csv(file, header=None, index_col=0)

# Only extract 4 components, but center and

# scale the data before fitting. MCUVScaler is
# mean-centring and unit-variance scaling.
spectra_mcuv = MCUVScaler().fit_transform(spectra)
model_pca = PCA(n_components=4).fit(spectra_mcuv)

# Proportion of variance per component:

print(model_pca.r2_per_component_)
(continues on next page)

164 [Link]

442 Chapter 6. Latent Variable Modelling

 Release 547b16

(continued from previous page)

# Cumulative proportion:
print(model_pca.r2_cumulative_)

# Read large data file

file <- '[Link]
spectra <- [Link](file, header = FALSE, [Link] = 1)

# Only extract 4 components, but

# center and scale the data before
# calculation the components
[Link] <- prcomp(spectra,
center = TRUE,
scale =TRUE,
rank. = 4)
summary([Link])

which gives this output:

Importance of first k=4 (out of 460) components:

PC1 PC2 PC3 PC4
Standard deviation 21.8835 10.9748 3.60075 3.27081
Proportion of Variance 0.7368 0.1853 0.01995 0.01646
Cumulative Proportion 0.7368 0.9221 0.94200 0.95846

The 𝑅𝑎2 (Cumulative Proportion) values shows the first component explains 73.7% of the
variability in X, the second explains an additional 18.5% for a cumulative total of 92.2%, and the third
component explains an additional 1.99%. These three components together explain 94.2% of all the
variation in X. This means we have reduced X from a 460 × 650 matrix to a 460 × 3 matrix of scores,
T, and a 650 × 3 matrix of loadings, P. This is a large reduction in data size, with a minimal loss of
information.

Let’s visually show what the 𝑅2 values are for each column. Shown below are these values for the first
3 components. The first component (green, thin line) explains certain regions of the spectra very well,
particularly the region around 1100nm. Wavelengths beyond 1800 nm are not well explained at all.
The second component is primarily responsible for explaining additional variability in the 700 to
1100nm region. The third component only seems to explain the additional variability from 1700 to
1800nm. Fitting a fourth component is only going to start fitting the noisy regions of the spectrum on
the very right. For these data we could use 2 components for most applications, or perhaps 3 if the
region between 1700 and 1800nm was also important.

6.5. Principal Component Analysis (PCA) 443

Process Improvement Using Data

Finally, we can show the SPE plot for each observation. SPE values for each tablet become smaller and
smaller as each successive component is added. Since each new component explains additional
variance, the size of SPE must decrease. There don’t appear to be any major outliers off the model’s
plane after the first component.

The code for the above plots is:

import numpy as np
import plotly.graph_objects as go
import pandas as pd
from [Link] import make_subplots
from process_improve.multivariate import PCA, MCUVScaler

file = "[Link]
spectra = pd.read_csv(file, header=None, index_col=0)

# Center and scale the data before fitting.

spectra_mcuv = MCUVScaler().fit_transform(spectra)
model_pca = PCA(n_components=4).fit(spectra_mcuv)

# The fitted PCA model exposes the per-variable R^2

# (cumulative up to each component) and the per-row
# SPE values directly, both indexed by component.
# r2_per_variable_ is K-by-A; spe_ is N-by-A.
(continues on next page)

444 Chapter 6. Latent Variable Modelling

 Release 547b16

(continued from previous page)

wavelengths = [Link](600, 1900, 2)
colors = {1: "darkgreen", 2: "black", 3: "blue"}
widths = {1: 2, 2: 4, 3: 6}

r2_fig = [Link]()
for a in (1, 2, 3):
r2_fig.add_trace([Link](
x=wavelengths,
y=model_pca.r2_per_variable_[a],
mode="lines",
name=f"R^2: component {a}",
line=dict(color=colors[a], width=widths[a]),
))
r2_fig.update_layout(
xaxis_title_text="Wavelengths",
yaxis_title_text="R^2 per component (wavelength)",
yaxis=dict(range=[0, 1]),
)
r2_fig.show()

# SPE plot: 3 stacked panels (A=1, A=2, A=3).

N = [Link][0]
spe_fig = make_subplots(
rows=3, cols=1, shared_xaxes=True,
subplot_titles=("SPE: A=1", "SPE: A=2", "SPE: A=3"),
)
for k, a in enumerate((1, 2, 3), start=1):
spe_fig.add_trace(
[Link](x=[Link](1, N + 1),
y=model_pca.spe_[a],
mode="lines",
line=dict(color=colors[a], width=2),
showlegend=False),
row=k, col=1,
)
spe_fig.update_xaxes(title_text="Tablet number",
row=3, col=1)
spe_fig.show()

file <- '[Link]

spectra <- [Link](file, header = FALSE, [Link] = 1)

# Only extract 4 components, but

# center and scale the data before
# calculation the components
[Link] <- prcomp(spectra,
center = TRUE,
scale =TRUE,
rank. = 4)
spectra.P <- [Link]$rotation
spectra.T <- [Link]$x

# Baseline: mean and standard deviation per column

[Link] <- apply(spectra, 2, mean, [Link]=TRUE)
[Link] <- apply(spectra, 2, sd, [Link]=TRUE)

# Remove the calculated mean from each column (margin=2)

# by using the subtract function (FUN argument)
[Link] <- sweep(spectra, 2, [Link], FUN='-')

# Scale each column, dividing by the standard deviation

[Link] <- sweep([Link], 2, [Link], FUN='/')

# Baseline variance
(continues on next page)

6.5. Principal Component Analysis (PCA) 445

Process Improvement Using Data

(continued from previous page)

spectra.X2 <- [Link] * [Link]

# A = 1
#------
a = 1
[Link].a <- spectra.T[,seq(1,a)] %*% t(spectra.P[,seq(1,a)])
spectra.E <- [Link] - [Link].a
spectra.E2 <- spectra.E * spectra.E
[Link].a.2 <- [Link].a * [Link].a

SPE.1 <- sqrt(apply(spectra.E2, 1, sum))

R2.k.a <- apply([Link].a.2, 2, sum) / apply(spectra.X2, 2, sum)

wavelengths <- seq(600, 1898, 2)

plot(wavelengths, R2.k.a, col='darkgreen',
type='l', lwd=a*2, ylim=c(0,1),
ylab=expression("R"^2*" per component (wavelength)"),
xlab="Wavelengths")

# A = 2
#------
a = 2
[Link].a <- spectra.T[,seq(1,a)] %*% t(spectra.P[,seq(1,a)])
spectra.E <- [Link] - [Link].a

# mean for each row

[Link] <- apply(spectra.E, 1, mean, [Link]=TRUE)
spectra.E2 <- spectra.E * spectra.E
[Link].a.2 <- [Link].a * [Link].a

SPE.2 <- sqrt(apply(spectra.E2, 1, sum))

R2.k.a <- apply([Link].a.2, 2, sum) / apply(spectra.X2, 2, sum)

lines(wavelengths, R2.k.a, col='black', type='l', lwd=a*2)

# A = 3
#------
a = 3
[Link].a <- spectra.T[,seq(1,a)] %*% t(spectra.P[,seq(1,a)])
spectra.E <- [Link] - [Link].a
spectra.E2 <- spectra.E * spectra.E
[Link].a.2 <- [Link].a * [Link].a

SPE.3 <- sqrt(apply(spectra.E2, 1, sum))

R2.k.a <- apply([Link].a.2, 2, sum) / apply(spectra.X2, 2, sum)

lines(wavelengths, R2.k.a, col='blue', type='l', lwd=a*2)

legend(x=650, y=0.35,
legend=c(expression("R"^2*": 1st component"),
expression("R"^2*": 2nd component"),
expression("R"^2*": 3rd component")),
col=c("darkgreen", "black", "blue"),
lty=c(1, 1, 1), lwd=c(2,4,6), cex=1.0)

# SPE plot
N <- dim(spectra)[1]
layout(matrix(c(1,2,3), 3, 1))
plot(seq(1, N), SPE.1, col='darkgreen',
type='l', lwd=2, ylab="SPE: A=1",
ylim=c(0, max(SPE.1)))
plot(seq(1, N), SPE.2, col='black',
type='l', lwd=2, ylab="SPE: A=2",
ylim=c(0, max(SPE.2)))
plot(seq(1, N), SPE.3, col='blue',
(continues on next page)

446 Chapter 6. Latent Variable Modelling

 Release 547b16

(continued from previous page)

type='l', lwd=2, ylab="SPE: A=3",
xlab="Tablet number", ylim=c(0, max(SPE.3)))

6.5.12 Hotelling’s T2
The final quantity from a PCA model that we need to consider is called Hotelling’s 𝑇 2 value. Some
PCA models will have many components, 𝐴, so an initial screening of these components using score
scatterplots will require reviewing 𝐴(𝐴 − 1)/2 scatterplots. The 𝑇 2 value for the 𝑖th observation is
defined as:
𝑎=𝐴
∑︁ (︂ 𝑡𝑖,𝑎 )︂2
2
𝑇 =
𝑎=1
𝑠𝑎

where the 𝑠2𝑎 values are constants, and are the variances of each component. The easiest interpretation
is that 𝑇 2 is a scalar number that summarizes all the score values. Some other properties regarding 𝑇 2 :

• It is a positive number, greater than or equal to zero.

• It is the distance from the center of the (hyper)plane to the projection of the observation onto the
(hyper)plane.

• An observation that projects onto the model’s center (usually the observation where every value is
at the mean), has 𝑇 2 = 0.

• The 𝑇 2 statistic is distributed according to the 𝐹 -distribution and is calculated by the multivariate
software package being used. For example, we can calculate the 95% confidence limit for 𝑇 2 , below
which we expect, under normal conditions, to locate 95% of the observations.

• It is useful to consider the case when 𝐴 = 2, and fix the 𝑇 2 value at its 95% limit, for example, call
2
that 𝑇𝐴=2,𝛼=0.95 . Using the definition for 𝑇 2 :

𝑡21 𝑡2
2
𝑇𝐴=2,𝛼=0.95 = 2 + 22
𝑠1 𝑠2
On a scatterplot of 𝑡1 vs 𝑡2 for all observations, this would be the equation of an ellipse, centered at
the origin. You will often see this ellipse shown on 𝑡𝑖 vs 𝑡𝑗 scatterplots of the scores. Points inside
this elliptical region are within the 95% confidence limit for 𝑇 2 .

6.5. Principal Component Analysis (PCA) 447

Process Improvement Using Data

• The same principle holds for 𝐴 > 2, except the ellipse is called a hyper-ellipse (think of a rugby-ball
shaped object for 𝐴 = 3). The general interpretation is that if a point is within this ellipse, then it is
also below the 𝑇 2 limit, if 𝑇 2 were to be plotted on a line.

6.5.13 Preprocessing the data before building a model

The previous sections of this chapter considered the interpretation of a PCA latent variable model.
From this section onwards we return to filling important gaps in our knowledge. There are 3 major
steps to building any latent variable models:

1. Preprocessing the data

2. Building the latent variable model in the algorithms section (page 450)

3. Testing the model (page 455), including testing for the number of components to use.

We discuss the first step in this section, and the next two steps after that.

There are a number of possibilities for data preprocessing. We mainly discuss centering and scaling in
this section, but outline a few other tools first. These steps are usually univariate, i.e. they are applied
separately to each column in the raw data matrix Xraw . We call the matrix of preprocessed data X, this
is the matrix that is then presented to the algorithm to build the latent variable model. Latent variable
algorithms seldom work on the raw data.

Transformations

The columns in Xraw can be transformed: log, square-root and various powers (-1, -0.5, 0.5, 2) are
popular options. These are used to reduce the effect of extreme measurements (e.g. log
transforms), or because the transformed variable is known to be more correlated with the other
variables. An example of this is in a distillation column: the inverse temperature is known to
more correlated to the vapour pressure, which we know from first-principles modelling. Using
the untransformed variable will lead to an adequate model, but the transformed variable, e.g.
using the inverse temperature, can lead to a better model.

448 Chapter 6. Latent Variable Modelling

 Release 547b16

The tools we considered earlier on visualization and univariate distributions (histograms) can
help assess which variables require transformation. But one’s knowledge of the system is the
most useful guide for knowing which transformations to apply. Note: latent variable models do
not require each column in Xraw to be normally distributed: any type of quantitative variable
may be used.

Expanding the X-matrix

Additional columns can and should be added to the X-matrix. This is frequently done in
engineering systems where we can augment Xraw with columns containing heat, mass, and
energy balances. It might be useful to add certain dimensionless numbers or other quantities that
can be derived from the raw data.

Another step that is applied, usually to experimental data, is to add square and cross terms. For
example, if 3 of the columns in Xraw were from a factorial designed experiment with center
points, then augment Xraw with columns containing interaction terms: 𝑥1 𝑥2 , 𝑥1 𝑥3 , 𝑥2 𝑥3 . If face
points or axial points (such as from a central composite design) were used, then also add the
square terms to estimate the quadratic effects: 𝑥21 , 𝑥22 , 𝑥23 . When studying experimental data
with latent variable methods (PCA or PLS), also add columns related to measured disturbance
variables, often called covariates, and blocking variables - you won’t know if they are important
if they are not included.

The general rule is: add as many columns into Xraw as possible for the initial analysis. You can
always prune out the columns later on if they are shown to be uninformative.

Dealing with outliers

Users often go through a phase of pruning outliers prior to building a latent variable model.
There are often uninteresting outliers, for example when a temperature sensor goes off-line and
provides a default reading of 0.0 instead of its usual values in the range of 300 to 400K. The
automated tools used to do this are known by names such as trimming and winsorizing. These
tools remove the upper and lower 𝛼 percent of the column’s tails on the histogram. But care
should be taken with these automated approaches, since the most interesting observations are
often in the outliers.

The course of action when removing outliers is to always mark their values as missing just for
that variable in Xraw , rather than removing the entire row in Xraw . We do this because we can use
the algorithms to calculate the latent variable model when missing data are present within a row.

Centering

Centering moves the coordinate system to a new reference point, usually the origin of the
coordinate system in 𝐾 variables (i.e. in 𝐾-dimensional space). Mean centering is effective and
commonly used: after mean centering the mean of every column in Xraw will be exactly 0.0. An
example of mean centering was given in the food texture example (page 428).

As we learned in the section on univariate data analysis (page 37), the mean has a low resistance to
outliers: any large outlier will distort the value of the mean. So users often resort to trimming
their data and then mean centering. In this regard, centering each column around its median is a
better choice. We recommend median centering as it avoids the trimming step, and
simultaneously highlights any outliers.

In the paper by Bro and Smilde on centering and scaling165 they show how centering is far more
influential on the model than scaling. Centering can be seen as adding a new principal
165 [Link]

6.5. Principal Component Analysis (PCA) 449

Process Improvement Using Data

component to the model, while scaling has much less of an effect.

Scaling

Scaling is an important step in latent variable modelling. Scaling can be seen as a way of
assigning weights, or relative importance, to each column in Xraw . If we don’t know much about
our data, then it is common to assign an equal weight to each column. We can do this by simply
dividing each column by its standard deviation. After this scaling each column will have
variance (and standard deviation) of exactly 1.0. This allows each column an equal opportunity
of contributing to the model.

This sort of scaling is called unit-variance scaling. When combined with mean centering you will
see the terminology that the data have been autoscaled.

Imagine a variable that is mostly constant, just noise. It will have a small standard deviation.
When dividing by the standard deviation we artificially inflate its variance to the level of the
other, truly-varying variables. These noisy, uninformative variables can be removed from Xraw ,
or they can simply be multiplied by a smaller weight, so that their variance after preprocessing is
less than 1.0. Such variables will also have small loading coefficients in all components, so they
will be discovered during model investigation, if not sooner.

One could use the median absolute deviation (MAD) instead of the standard deviation to scale
the columns, but it most cases, any approximate scaling vector will work adequately (see the Bro
and Smilde paper referenced earlier).

6.5.14 Algorithms to calculate (build) PCA models

The different algorithms used to build a PCA model provide a different insight into the model’s
structure and how to interpret it. These algorithms are a reflection of how PCA has been used in
different disciplines: PCA is called by different names in each area.

Eigenvalue decomposition

ò Note

The purpose of this section is not the theoretical details, but rather the interesting interpretation of
the PCA model that we obtain from an eigenvalue decomposition.

Recall that the latent variable directions (the loading vectors) were oriented so that the variance of the
scores in that direction were maximal. We can cast this as an optimization problem. For the first
component:

max 𝜑 = t′1 t1 = p′1 X′ Xp1

s.t. p′1 p1 = 1

This is equivalent to max 𝜑 = p′1 X′ Xp1 − 𝜆 (p′1 p1 − 1), because we can move the constraint into the
objective function with a Lagrange multiplier, 𝜆1 .

The maximum value must occur when the partial derivatives with respect to p1 , our search variable,

450 Chapter 6. Latent Variable Modelling

 Release 547b16

are zero:
𝜕𝜑
= 0 = p′1 X′ Xp1 − 𝜆1 (p′1 p1 − 1)
𝜕p1
0 = 2X′ Xp1 − 2𝜆1 p1
0 = (X′ X − 𝜆1 𝐼𝐾×𝐾 )p1
X′ Xp1 = 𝜆1 p1
which is just the eigenvalue equation, indicating that p1 is the eigenvector of X′ X and 𝜆1 is the
eigenvalue. One can show that 𝜆1 = t′1 t1 , which is proportional to the variance of the first component.

In a similar manner we can calculate the second eigenvalue, but this time we add the additional
constraint that p1 ⊥ p2 . Writing out this objective function and taking partial derivatives leads to
showing that X′ Xp2 = 𝜆2 p2 .

From this we learn that:

• The loadings are the eigenvectors of X′ X.

• Sorting the eigenvalues in order from largest to smallest gives the order of the corresponding
eigenvectors, the loadings.

• We know from the theory of eigenvalues that if there are distinct eigenvalues, then their
eigenvectors are linearly independent (orthogonal).

• We also know the eigenvalues of X′ X must be real values and positive; this matches with the
interpretation that the eigenvalues are proportional to the variance of each score vector.

• Also, the sum of the eigenvalues must add up to sum of the diagonal entries of X′ X, which
represents of the total variance of the X matrix, if all eigenvectors are extracted. So plotting the
eigenvalues is equivalent to showing the proportion of variance explained in X by each component.
This is not necessarily a good way to judge the number of components to use, but it is a rough
guide: use a Pareto plot of the eigenvalues (though in the context of eigenvalue problems, this plot
is called a scree plot).

The general approach to using the eigenvalue decomposition would be:

1. Preprocess the raw data, particularly centering and scaling, to create a matrix X.

2. Calculate the correlation matrix X′ X.

3. Calculate the eigenvectors and eigenvalues of this square matrix and sort the results from largest to
smallest eigenvalue.

6.5. Principal Component Analysis (PCA) 451

Process Improvement Using Data

4. A rough guide is to retain only the first 𝐴 eigenvectors (loadings), using a Scree plot of the
eigenvalues as a guide. Alternative methods to determine the number of components are described
in the section on cross-validation and randomization.

However, we should note that calculating the latent variable model using an eigenvalue algorithm is
usually not recommended, since it calculates all eigenvectors (loadings), even though only the first few
will be used. The maximum number of components possible is 𝐴max = min(𝑁, 𝐾). Also, the default
eigenvalue algorithms in software packages cannot handle missing data.

Singular value decomposition

The singular value decomposition (SVD), in general, decomposes a given matrix X into three other
matrices:

X = UΣV′

Matrices U and V are orthonormal (each column has unit length and each column is orthogonal to the
others), while Σ is a diagonal matrix. The relationship to principal component analysis is that:

X = TP′

where matrix P is also orthonormal. So taking the SVD on our preprocessed matrix X allows us to get
the PCA model by setting P = V, and T = UΣ. The diagonal terms in Σ are related to the variances
of each principal component and can be plotted as a scree plot, as was done for the eigenvalue
decomposition (page 450).

Like the eigenvalue method, the SVD method calculates all principal components possible,
𝐴 = min(𝑁, 𝐾), and also cannot handle missing data by default.

Non-linear iterative partial least-squares (NIPALS)

The non-linear iterative partial least squares (NIPALS) algorithm is a sequential method of computing
the principal components. The calculation may be terminated early, when the user deems that enough
components have been computed. Most computer packages tend to use the NIPALS algorithm as it
has two main advantages: it handles missing data and calculates the components sequentially.

The purpose of considering this algorithm here is three-fold: it gives additional insight into what the
loadings and scores mean; it shows how each component is independent of (orthogonal to) the other
components, and it shows how the algorithm can handle missing data.

The algorithm extracts each component sequentially, starting with the first component, direction of
greatest variance, and then the second component, and so on.

We will show the algorithm here for the 𝑎th component, where 𝑎 = 1 for the first component. The
matrix X that we deal with below is the preprocessed (page 448), usually centered and scaled matrix,
not the raw data.

1. The NIPALS algorithm starts by arbitrarily creating an initial column for t𝑎 . You can use a column
of random numbers, or some people use a column from the X matrix; anything can be used as long
as it is not a column of zeros.

2. Then we take every column in X, call it X𝑘 , and regress it onto this initial column t𝑎 ; store the
regression coefficient as the entry in 𝑝𝑘,𝑎 . What this means, and it is illustrated below, is that we
perform an ordinary least squares regression (y = 𝛽x), except our x-variable is this column of t𝑎
values, and our y-variable is the particular column from X.

452 Chapter 6. Latent Variable Modelling

 Release 547b16

For ordinary least squares, you will remember that the solution for this regression problem is
x′ y
𝛽̂︀ = ′ . Using our notation, this means:
xx
t′ X𝑘
𝑝𝑘,𝑎 = 𝑎′
t𝑎 t𝑎
This is repeated for each column in X until we fill the entire vector p𝑘 . This is shown in the
illustration where each column from X is regressed, one at a time, on t𝑎 , to calculate the loading
entry, 𝑝𝑘,𝑎 In practice we don’t do this one column at time; we can regress all columns in X in go:
1
p′𝑎 = ′ · t′ X𝑎 , where t𝑎 is an 𝑁 × 1 column vector, and X𝑎 is an 𝑁 × 𝐾 matrix, explained more
t𝑎 t𝑎 𝑎
clearly in step 6.

3. The loading vector p′𝑎 won’t have unit length (magnitude) yet. So we simply rescale it to have
magnitude of 1.0:
1
p′𝑎 = √︀ · p′𝑎
p′𝑎 p𝑎

4. The next step is to regress every row in X onto this normalized loadings vector. As illustrated
below, in our linear regression the rows in X are our y-variable each time, while the loadings vector
is our x-variable. The regression coefficient becomes the score value for that 𝑖th row:

x′𝑖 p𝑎
𝑡𝑖,𝑎 =
p′𝑎 p𝑎

6.5. Principal Component Analysis (PCA) 453

Process Improvement Using Data

where x′𝑖 is an 𝐾 × 1 column vector. We can combine these 𝑁 separate least-squares models and
1
calculate them in one go to get the entire vector, t𝑎 = ′ · Xp𝑎 , where p𝑎 is a 𝐾 × 1 column
p𝑎 p𝑎
vector.

5. We keep iterating steps 2, 3 and 4 until the change in vector t𝑎 from one iteration to the next is small
(usually around 1 × 10−6 to 1 × 10−9 ). Most data sets require no more than 200 iterations before
achieving convergence.

6. On convergence, the score vector and the loading vectors, t𝑎 and p𝑎 are stored as the 𝑎th column in
matrix T and P respectively. We then deflate the X matrix. This crucial step removes the variability
captured in this component (t𝑎 and p𝑎 ) from X:

E𝑎 = X𝑎 − t𝑎 p′𝑎
X𝑎+1 = E𝑎

For the first component, X𝑎 is just the preprocessed raw data. So we can see that the second
component is actually calculated on the residuals E1 , obtained after extracting the first component.

This is called deflation, and nicely shows why each component is orthogonal to the others. Each
subsequent component is only seeing variation remaining after removing all the others; there is no
possibility that two components can explain the same type of variability.

After deflation we go back to step 1 and repeat the entire process for the next component. Just
before accepting the new component we can use a test, such as a randomization test, or
cross-validation (page 456), to decide whether to keep that component or not.

The final reason for outlining the NIPALS algorithm is to show one way in which missing data can be
handled. All that step 2 and step 4 are doing is a series of regressions. Let’s take step 2 to illustrate, but
the same idea holds for step 4. In step 2, we were regressing columns from X onto the score t𝑎 . We can
visualize this for a hypothetical system below

There are 3 missing observations (open circles), but despite this, the regression’s slope can still be
adequately determined. The slope is unlikely to change by very much if we did have the missing
values. In practice though we have no idea where these open circles would fall, but the principle is the
same: we calculate the slope coefficient just ignoring any missing entries.

In summary:

• The NIPALS algorithm computes one component at a time. The first component computed is
equivalent to the t1 and p1 vectors that would have been found from an eigenvalue or singular
value decomposition.

454 Chapter 6. Latent Variable Modelling

 Release 547b16

• The algorithm can handle missing data in X.

• The algorithm always converges, but the convergence can sometimes be slow.

• It is also known as the Power algorithm to calculate eigenvectors and eigenvalues.

• It works well for very large data sets.

• It is used by most software packages, especially those that handle missing data.

• Of interest: it is well known that Google used this algorithm for the early versions of their search
engine, called PageRank166 .

6.5.15 Testing the PCA model

As mentioned previously there are 3 major steps to building a PCA model for engineering
applications. We have already considered the first two steps in the preceding sections.

1. Preprocessing the data

2. Building the latent variable model

3. Testing the model, including testing for the number of components to use

The last step of testing, interpreting and using the model is where one will spend the most time.
Preparing the data can be time-consuming the first time, but generally the first two steps are less
time-consuming. In this section we investigate how to determine the number of components that
should be used in the model and how to use an existing latent variable model. The issue of
interpreting a model has been addressed in the section on interpreting scores (page 431) and interpreting
loadings (page 435).

Using an existing PCA model

In this section we outline the process required to use an existing PCA model. What this means is that
you have already calculated the model and validated its usefulness. Now you would like to use the
model on a new observation, which we call x′new, raw . The method described below can be efficiently
applied to many new rows of observations by converting the row vector notation to matrix notation.

1. Preprocess your vector of new data in the same way as you did when you built the model. For
example, if you took the log transform of a certain variable, then you must do so for the
corresponding entry in x′new, raw . Also apply mean centering and scaling, using the mean centering
and scaling information you calculated when you originally built the model.

2. Call this preprocessed vector xnew now; it has size 𝐾 × 1, so x′new is a 1 × 𝐾 row vector.

3. Calculate the location, on the model (hyper)plane, where the new observation would project. In
other words, we are calculating the scores:

t′new = x′new P

where P is the 𝐾 × 𝐴 matrix of loadings calculated when building the model, and t′new is a 1 × 𝐴
vector of scores for the new observation.

̂︀′new , the
4. Calculate the residual distance off the model plane. To do this, we require the vector called x
point on the plane, a 1 × 𝐾 vector:

̂︀′new = t′new P′
x
166 [Link]

6.5. Principal Component Analysis (PCA) 455

Process Improvement Using Data

5. The residual vector is the difference between the actual observation and its projection onto the
plane. The 𝐾 individual entries inside this residual vector are also the called the contributions to the
error.

e′new = x′new − x
̂︀′new

6. And the residual distance is the sum of squares of the entries in the residual vector, followed by
taking a square root.
√︀
SPEnew = e′new enew

This is called the squared prediction error, SPE, even though it is more accurately a distance.

7. Another quantity of interest is Hotelling’s 𝑇 2 value for the new observation:

𝑎=𝐴
∑︁ (︂ )︂2
2 𝑡new,𝑎
𝑇new =
𝑎=1
𝑠𝑎

where the 𝑠𝑎 values are the standard deviations for each component’s scores, calculated when the
model was built.

The above outline is for the case when there is no missing data in a new observation. When there are
missing data present in x′new , then we require a method to estimate the score vector, tnew in step 3.
Methods for doing this are outlined and compared in the paper by Nelson, Taylor and MacGregor167
and the paper by Arteaga and Ferrer168 . After that, the remaining steps are the same, except of course
that missing values do not contribute to the residual vector and the SPE.

6.5.16 Determining the number of components to use in the model with cross-validation
Cross-validation is a general tool that helps to avoid over-fitting - it can be applied to any model, not
just latent variable models.

As we add successive components to a model we are increasing the size of the model, 𝐴, and we are
explaining the model-building data, X, better and better. (The equivalent in least squares models
would be to add additional X-variable terms to the model.) The model’s 𝑅2 value will increase with
every component. As the following equation shows, the variance of the X ̂︀ matrix increases with every
component, while the residual variance in matrix E must decrease.

X = TP′ + E
X=X ̂︀ + E
𝒱(X) = 𝒱(X)
̂︀ + 𝒱(E)

This holds for any model where the X ̂︀ and E matrices are completely orthogonal to each other:
′
X E = 0 (a matrix of zeros), such as in PCA, PLS and least squares models.
̂︀

There comes a point for any real data set where the number of components, 𝐴 = the number of
columns in T and P, extracts all systematic variance from X, leaving unstructured residual variance in
E. Fitting any further components will start to fit this noise, and unstructured variance, in E.

Cross-validation for multivariate data sets was described by Svante Wold in his paper on
Cross-validatory estimation of the number of components in factor and principal components
models169 , in Technometrics, 20, 397-405, 1978.
167 [Link]
168 [Link]
169 [Link]

456 Chapter 6. Latent Variable Modelling

 Release 547b16

The general idea is to divide the matrix X into 𝐺 groups of rows. These rows should be selected
randomly, but are often selected in order: row 1 goes in group 1, row 2 goes in group 2, and so on. We
can collect the rows belonging to the first group into a new matrix called X(1) , and leave behind all the
other rows from all other groups, which we will call group X(−1) . So in general, for the 𝑔 th group, we
can split matrix X into X(𝑔) and X(−𝑔) .

Wold’s cross-validation procedure asks to build the PCA model on the data in X(−1) using 𝐴
components. Then use data in X(1) as new, testing data. In other words, we preprocess the X(1) rows,
̂︀ (1) = T(1) P′ , and their
calculate their score values, T(1) = X(1) P, calculate their predicted values, X
residuals, E(1) = X(1) − X̂︀ (1) . We repeat this process, building the model on X(−2) and testing it with
X(2) , to eventually obtain E(2) .

After repeating this on 𝐺 groups, we gather up E1 , E2 , . . . , E𝐺 and assemble a type of residual matrix,
E𝐴,CV , where the 𝐴 represents the number of components used in each of the 𝐺 PCA models. The CV
subscript indicates that this is not the usual error matrix, E. From this we can calculate a type of 𝑅2
value. We don’t call this 𝑅2 , but it follows the same definition for an 𝑅2 value. We will call it 𝑄2𝐴
instead, where 𝐴 is the number of components used to fit the 𝐺 models.

Var(E𝐴,CV )
𝑄2𝐴 = 1 −
Var(X)

We also calculate the usual PCA model on all the rows of X using 𝐴 components, then calculate the
usual residual matrix, E𝐴 . This model’s 𝑅2 value is:

2 Var(E𝐴 )
𝑅𝐴 =1−
Var(X)

The 𝑄2𝐴 behaves exactly as 𝑅2 , but with two important differences. Like 𝑅2 , it is a number less than 1.0
that indicates how well the testing data, in this case testing data that was generated by the
cross-validation procedure, are explained by the model. The first difference is that 𝑄2𝐴 is always less
than the 𝑅2 value. The other difference is that 𝑄2𝐴 will not keep increasing with each successive
component, it will, after a certain number of components, start to decrease. This decrease in 𝑄2𝐴
indicates the new component just added is not systematic: it is unable to explain the cross-validated
testing data. We often see plots such as this one:

This is for a real data set, so the actual cut off for the number of components could be either 𝐴 = 2 or
𝐴 = 3, depending on what the 3rd component shows to the user and how interested they are in that
component. Likely the 4th component, while boosting the 𝑅2 value from 66% to 75%, is not really
fitting any systematic variation. The 𝑄2 value drops from 32% to 25% when going from component 3
to 4. The fifth component shows 𝑄2 increasing again. Whether this is fitting actual variability in the
data or noise is for the modeller to determine, by investigating that 5th component. These plots show
that for this data set we would use between 2 and 5 components, but not more.

6.5. Principal Component Analysis (PCA) 457

Process Improvement Using Data

Cross-validation, as this example shows is never a precise answer to the number of components that
should be retained when trying to learn more about a dataset. Many studies try to find the “true” or
“best” number of components. This is a fruitless exercise; each data set means something different to
the modeller and the objective for which the model was intended to assist.

The number of components to use should be judged by the relevance of each component. Use
cross-validation as guide, and always look at a few extra components and step back a few
components; then make a judgement that is relevant to your intended use of the model.

However, cross-validation’s objective is useful for predictive models, such as PLS, so we avoid
over-fitting components. Models where we intend to learn from, or optimize, or monitor a process
may well benefit from fewer or more components than suggested by cross-validation.

6.5.17 Some properties of PCA models

We summarize various properties of the PCA model, most have been described in the previous
sections. Some are only of theoretical interest, but others are more practical.

• The model is defined by the direction vectors, or loadings vectors, called p1 , p2 , . . . , p𝐴 ; each are a
𝐾 × 1 vector, and can be collected into a single matrix, P, a 𝐾 × 𝐴 loadings matrix.

• These vectors form a line for one component, a plane for 2 components, and a hyperplane for 3 or
more components. This line, plane or hyperplane define the latent variable model.

• An equivalent interpretation of the model plane is that these direction vectors are oriented in such a
way that the scores have maximal variance for that component. No other directions of the loading
vector (i.e. no other hyperplane) will give a greater variance.

• This plane is calculated with respect to a given data set, X, an 𝑁 × 𝐾 matrix, so that the direction
vectors best-fit the data. We can say then that with one component, the best estimate of the original
matrix X is:

X
̂︀ 1 = t1 p1 or equivalently: X1 = t1 p1 + E1

where E1 is the residual matrix after fitting one component. The estimate for X will have smaller
residuals if we fit a second component:

X
̂︀ 2 = t1 p1 + t2 p2 or equivalently: X2 = t1 p1 + t2 p2 + E2

In general we can illustrate this:

√︀
• The loadings vectors are of unit length: ‖p𝑎 ‖ = p′𝑎 p𝑎 = 1.0

458 Chapter 6. Latent Variable Modelling

 Release 547b16

• The loading vectors are independent or orthogonal to one another: p′𝑖 p𝑗 = 0.0 for 𝑖 ̸= 𝑗; in other
words p𝑖 ⊥ p𝑗 .

• Orthonormal matrices have the property that P′ P = I𝐴 , an identity matrix of size 𝐴 × 𝐴.

• These last 3 properties imply that P is an orthonormal matrix. From matrix algebra and geometry
you will recall that this means P is a rigid rotation matrix. We are rotating our real-world data in X
to a new set of values, scores, using the rotation matrix P. But a rigid rotation implies that distances
and angles between observations are preserved. Practically, this means that by looking at our data
in the score space, points which are close together in the original 𝐾 variables will be close to each
other in the scores, T, now reduced to 𝐴 variables.

• The variance of the t1 vector must be greater than the variance of the t2 vector. This is because we
intentionally find the components in this manner. In our notation: 𝑠1 > 𝑠2 > . . . > 𝑠𝐴 , where 𝑠𝑎 is
the standard deviation of the 𝑎th score.

• The maximum number of components that can be extracted is the smaller of 𝑁 or 𝐾; but usually we
will extract only 𝐴 ≪ 𝐾 number of components. If we do extract all components, 𝐴* = min(𝑁, 𝐾),
then our loadings matrix, P, merely rotates our original coordinate system to a new system without
error.

• The eigenvalue decomposition of X′ X gives the loadings, P, as the eigenvectors, and the
eigenvalue for each eigenvector is the variance of the score vector.

• The singular value decomposition of X is given by X = UΣV′ , so V′ = P′ and UΣ = T, showing

the equivalence between PCA and this method.

• If there are no missing values in X, then the mean of each score vector is 0.0, which allows us to
calculate the variance of each score simply from t′𝑎 t𝑎 .

• Notice that some score values are positive and others negative. Each loading direction, p𝑎 , must
point in the direction that best explains the data; but this direction is not unique, since −p𝑎 also
meets this criterion. If we did select −p𝑎 as the direction, then the scores would just be −t𝑎 instead.
This does not matter too much, because (−t𝑎 )(−p′𝑎 ) = t𝑎 p′𝑎 , which is used to calculate the predicted
X and the residuals. But this phenomena can lead to a confusing situation for newcomers when
different computer packages give different-looking loading plots and score plots for the same data
set.

6.5.18 Latent variable contribution plots

We have previously seen (page 426) how contribution plots are constructed for a score value, for the SPE
and for 𝑇 2 . We breakdown the value, such as SPE, into its individual terms, one from each variable.
Then we plot these 𝐾 contribution values as a bar chart.

There are 𝐾 contribution terms for a score: 𝑡𝑖,𝑎 = x𝑖 p𝑎 :

[︁ ]︁
𝑥𝑖,1 𝑝1,𝑎 𝑥𝑖,2 𝑝2,𝑎 . . . 𝑥𝑖,𝑘 𝑝𝑘,𝑎 ... 𝑥𝑖,𝐾 𝑝𝐾,𝑎

The contribution to 𝑇 2 is similar to a score contribution, except we calculate the weighted summation
over all the scores, 𝑡𝑖,𝑎 , where the weights are the variances of the 𝑎th score.

For SPE = e′𝑖 e𝑖 , where e′𝑖 = x′𝑖 − x̂︀′𝑖 , the bars in the contribution plots are:
√︀

[︁ ]︁
(𝑥𝑖,1 − 𝑥^𝑖,1 ) (𝑥𝑖,2 − 𝑥 ^𝑖,2 ) . . . (𝑥𝑖,𝑘 − 𝑥
^𝑖,𝑘 ) . . . (𝑥𝑖,𝐾 − 𝑥
^𝑖,𝐾 )

6.5. Principal Component Analysis (PCA) 459

Process Improvement Using Data

The SPE contributions are usually shown as the square of the values in brackets, accounting for the
^𝑖,𝑘 ), and then plot each bar: sign(𝑒𝑖,𝑘 ) × 𝑒2𝑖,𝑘 . The squared values are more
sign, as in 𝑒𝑖,𝑘 = (𝑥𝑖,𝑘 − 𝑥
realistic indicators of the contributions, while the sign information might be informative in some cases.

The other point to mention here is that contributions are calculated from one point to another point.
Most often, the from point is the model center or the model plane. So for SPE, the contributions are
from the model plane to the 𝑖th observation off the model plane. The score contributions are from the
model center to the observation’s projection on the (hyper)plane.

But sometimes we would like to know, as in the figure below, what are the contribution from one point
to another. And these start and end points need not be an actual point; for a group of points we can
use a suitable average of the points in the cluster. So there are point-to-point, point-to-group,
group-to-point, and group-to-group contributions in the scores.

The calculation procedure is actually the same in all cases: for a group of points, collapse it down to
the center point in the group, then calculate the point-to-point contribution. If the starting point is not
specified, then the contribution will be from the model center, i.e. (𝑡𝑖 , 𝑡𝑗 ) = (0, 0) to the point.

6.5.19 Using indicator variables in a latent variable model

Indicator variables, also called dummy variables, are most often binary variables that indicate the
presence or absence of a certain effect. For example, a variable that shows if reactor A or reactor B was
used. Its value is either a 0 or a 1 in the data matrix X. It’s valid to include these sort of variables in a
principal component analysis model where they are used and interpreted as any other continuous
variable.

Sometimes these variables are imported into the computer software, but not used in the model. They are
only used in the display of results, where the indicator variable is shown in a different colour or with a
different marker shape. We will see an example of this for process troubleshooting (page 491), to help
isolate causes for poor yield from a process:

460 Chapter 6. Latent Variable Modelling

 Release 547b16

If the variable is included in the model then it is centered and scaled (preprocessed) like any other
variable. Care must be taken to make sure this variable is reasonably balanced. There is no guide as to
how balanced it needs to be, but there should be a good number of observations of both zeros and
ones. The extreme case is where there are 𝑁 observations, and only 1 of them is a zero or a one, and
the other 𝑁 − 1 observations are the rest. You are not likely to learn much from this variable in any
case; furthermore, the scaling for this variable will be poor (the variance will be small, so dividing by
this small variance will inflate that variable’s variance).

Interpreting these sort of variables in a loading plot is also no different; strong correlations with this
variable are interpreted in the usual way.

6.5.20 Visualization latent variable models with linking and brushing

Linking is when the same data point(s), are highlighted in two or more plots. This is used to highlight
outliers or interesting points in a multivariate data set. The points could be highlighted in terms of
colour and/or shape.

Brushing is the same as linking, except it is done in real-time as the user moves a mouse over a plot.
This concept was described by Becker and Cleveland in their original article called Brushing
Scatterplots170 , Technometrics, 29, 127-142, 1987.
170 [Link]

6.5. Principal Component Analysis (PCA) 461

Process Improvement Using Data

In this illustration we are considering the well-known iris data set, a multivariate data set consisting of
the 4 length measurements taken on 3 species of iris. There are 150 observations (50 for each species).
Linking is used to mark each iris species with a different marker shape (a different colour could also
have been used). Brushing cannot be illustrated, but as shown in the paper by Becker and Cleveland, it
would amount to dynamically changing the marker shape or colour of points in one plot, while the
user selects those same observations in another plot.

This concept is very powerful to learn from, and to interrogate a latent variable model. For example,
when we see interesting observations in the score plot, we can brush through the scores, while having
a time series plot of the raw data open alongside. This would highlight what that score feature means
in the context of the raw data.

6.5.21 PCA Exercises

Each exercise introduces a new topic or highlights some interesting aspect of PCA.

Room temperature data

• 𝑁 = 144

• 𝐾 = 4 + 1 column containing the date and time at which the 4 temperatures were recorded

• Web address: [Link]

• Description: Temperature measurements from 4 corners of a room

462 Chapter 6. Latent Variable Modelling

 Release 547b16

Objectives

Before even fitting the model:

1. How many latent variables do you expect to use in this model? Why?.

2. What do you expect the first loading vector to look like?

Now build a PCA model using any software package.

1. How much variation was explained by the first and second latent variables? Is this result
surprising, given the earlier description (page 419) of the dataset?

2. Plot a time series plot (also called a line plot) of 𝑡1 . Did this match your expectations? Why/why
not?

3. Plot a bar plot of the loadings for the second component. Given this bar plot, what are the
characteristics of an observation with a large, positive value of 𝑡2 ; and a large, negative 𝑡2 value?

4. Now plot the time series plot for 𝑡2 . Again, does this plot match your expectations?

Now use the concept of brushing (page 461) to interrogate and learn from the model.

1. Plot a score plot of 𝑡1 against 𝑡2 .

2. Also plot the time series plot of the raw data.

3. Select a cluster of interest in the score plot and see the brushed values in the raw data. Are these the
values you expected to be highlighted?

4. Next plot the Hotelling’s 𝑇 2 line plot, as described earlier (page 447). Does the 95% limit in the
Hotelling’s 𝑇 2 line plot correspond to the 95% limit in the score plot?

5. Also plot the SPE line plot. Brush the outlier in the SPE plot and find its location in the score plot.

6. Why does this point have a large SPE value?

6.5. Principal Component Analysis (PCA) 463

Process Improvement Using Data

7. Describe how a 3-D scatter plot would look with 𝑡1 and 𝑡2 as the (𝑥, 𝑦) axes, and SPE as the 𝑧-axis.

What have we learned?

• Interpreted that a latent variable is often a true driving force in the system under investigation.

• How to interpret a loadings vector and its corresponding score vector.

• Brushing multivariate and raw data plots to confirm our understanding of the model.

• Learned about Hotelling’s 𝑇 2 , whether we plot it as a line plot, or as an ellipse on a scatter plot.

• We have confirmed how the scores are on the model plane, and the SPE is the distance from the
model plane to the actual observation.

Food texture data set

• 𝑁 = 50

• 𝐾 = 5 + 1 column containing the labels for each batch

• Web address: [Link]

• Description: Data from a food manufacturer making a pastry product (page 428). Each row contains the
5 quality attributes of a batch of product.

1. Fit a PCA model.

2. Report the 𝑅2 values for the overall model and the 𝑅2 values for each variable, on a per-component
basis for components 1, 2, and 3. Comment on what each latent variable is explaining and by how
much.

3. Plot the loadings plot as a bar plot for 𝑝1 . Does this match the values given earlier (page 428)?
Interpret what kind of pastry would have a large positive 𝑡1 value?

464 Chapter 6. Latent Variable Modelling

 Release 547b16

4. What feature(s) of the raw data does the second component explain? Plot sequence-ordered plots of
the raw data to confirm your answer.

5. Look for any observations that are unusual. Are there any unusual scores? SPE values? Plot
contribution plots for the unusual observations and interpret them.

Food consumption data set

This data set has become a classic data set when learning about multivariate data analysis. It consists
of

• 𝑁 = 16 countries in the European area

• 𝐾 = 20 food items

• Missing data: yes

• Web address: [Link]

• Description: The data table lists for each country the relative consumption of certain food items,
such as tea, jam, coffee, yoghurt, and others.

1. Fit a PCA model to the data using 2 components.

2. Plot a loadings plot of 𝑝1 against 𝑝2 . Which are the important variables in the first component? And
the second component?

3. Since each column represents food consumption, how would you interpret a country with a high
(positive or negative) 𝑡1 value? Find countries that meet this criterion. Verify that this country does
indeed have this interpretation (hint: use a contribution plot and examine the raw data in the table).

4. Now plot SPE after 2 components (don’t plot the default SPE, make sure it is the SPE only after two
components). Use a contribution plot to interpret any interesting outliers.

5. Now add a third component and plot SPE after 3 components. What has happened to the
observations you identified in the previous question? Investigate the loadings plot for the third
component now (as a bar plot) and see which variables are heavily loaded in the 3rd component.

6. Also plot the 𝑅2 values for each variable, after two components, and after 3 components. Which
variables are modelled by the 3rd component? Does this match with your interpretation of the
loadings bar plot in the previous question?

7. Now plot a score plot of the 3rd component against the 1st component. Generate a contribution plot
in the score from the interesting observation(s) you selected in part 4. Does this match up with your
interpretation of what the 3rd component is modelling?

What we learned:

6.5. Principal Component Analysis (PCA) 465

Process Improvement Using Data

• Further practice of our skills in interpreting score plots and loading plots.

• How to relate contribution plots to the loadings and the 𝑅2 values for a particular component.

Silicon wafer thickness

• 𝑁 = 184

• 𝐾=9

• Web address: [Link]

• Description: These are nine thickness measurements recorded from various batches of silicon
wafers. One wafer is removed from each batch and the thickness of the wafer is measured at the
nine locations, as shown in the illustration.

1. Build a PCA model on all the data.

2. Plot the scores for the first two components. What do you notice? Investigate the outliers, and the
raw data for each of these unusual observations. What do you conclude about those observations?

3. Exclude the unusual observations and refit the model.

4. Now plot the scores plot again; do things look better? Record the 𝑅2 and 𝑄2 values (from
cross-validation) for the first three components. Are the 𝑅2 and 𝑄2 values close to each other; what
does this mean?

5. Plot a loadings plot for the first component. What is your interpretation of 𝑝1 ? Given the 𝑅2 and 𝑄2
values for this first component (previous question), what is your interpretation about the variability
in this process?

6. And the interpretation of 𝑝2 ? From a quality control perspective, if you could remove the variability
due to 𝑝2 , how much of the variability would you be removing from the process?

7. Also plot the corresponding time series plot for 𝑡1 . What do you notice in the sequence of score
values?

8. Repeat the above question for the second component.

9. Finally, plot both the 𝑡1 and 𝑡2 series overlaid on the same plot, in time-order, to see the smaller
variance that 𝑡2 explains.

What we learned:

466 Chapter 6. Latent Variable Modelling

 Release 547b16

• Identifying outliers; removing them and refitting the model.

• Variability in a process can very often be interpreted. The 𝑅2 and 𝑄2 values for each component
show which part of the variability in the system is due the particular phenomenon modelled by that
component.

Process troubleshooting

Recent trends show that the yield of your company’s flagship product is declining. You are uncertain if
the supplier of a key raw material is to blame, or if it is due to a change in your process conditions.
You begin by investigating the raw material supplier.

The data available has:

• 𝑁 = 24

• 𝐾 = 6 + 1 designation of process outcome

• Web address: [Link]

• Description: 3 of the 6 measurements are size values for the plastic pellets, while the other 3 are the
outputs from thermogravimetric analysis (TGA), differential scanning calorimetry (DSC) and
thermomechanical analysis (TMA), measured in a laboratory. These 6 measurements are thought to
adequately characterize the raw material. Also provided is a designation Adequate or Poor that
reflects the process engineer’s opinion of the yield from that lot of materials.

Import the data, and set the Outcome variable as a secondary identifier for each observation, as shown
in the illustration below. The observation’s primary identifier is its batch number.

1. Build a latent variable model for all observations and use auto-fit to determine the number of
components. If your software does not have and auto-fit features (cross-validation), then use a
Pareto plot of the eigenvalues to decide on the number of components.

2. Interpret component 1, 2 and 3 separately (using the loadings bar plot).

3. Now plot the score plot for components 1 and 2, and colour code the score plot with the Outcome
variable. Interpret why observations with Poor outcome are at their locations in the score plot (use
a contribution plot).

6.5. Principal Component Analysis (PCA) 467

Process Improvement Using Data

4. What would be your recommendations to your manager to get more of your batches classified as
Adequate rather than Poor?

5. Now build a model only on the observations marked as Adequate in the Outcome variable.

6. Re-interpret the loadings plot for 𝑝1 and 𝑝2 . Is there a substantial difference between this new
loadings plot and the previous one?

What we learned:

• How to use an indicator variable in the model to learn more from our score plot.

• How to build a data set, and bring in new observations as testing data.

6.6 Principal Component Regression (PCR)

Principal component regression (PCR) is an alternative to multiple linear regression (MLR) and has
many advantages over MLR.

In multiple linear regression (page 253) we have two matrices (blocks): X, an 𝑁 × 𝐾 matrix whose
columns we relate to the single vector, y, an 𝑁 × 1 vector, using a model of the form: y = Xb. The
−1
solution vector b is found by solving b = (X′ X) X′ y. The variance of the estimated solution is given
−1
by 𝒱(b) = (X′ X) 𝑆𝐸 2
.

In the section on factorial experiments (page 330) we intentionally set our process to generate a matrix X
that has independent columns. This means that each column is orthogonal to the others, you cannot
express one column in terms of the other, and it results in a diagonal X′ X matrix.

On most data sets though the columns in X are correlated. Correlated columns are not too serious if
they are mildly correlated. But the illustration here shows the problem with strongly correlated
variables, in this example 𝑥1 and 𝑥2 are strongly, positively correlated. Both variables are used to
create a predictive model for 𝑦. The model plane, 𝑦^ = 𝑏0 + 𝑏1 𝑥1 + 𝑏2 𝑥2 is found so that it minimizes the
residual error. There is a unique minimum for the sum of squares of the residual error (i.e. the
objective function). But very small changes in the raw 𝑥-data lead to almost no change in the objective
function, but will show large fluctuations in the solution for b as the plane rotates around the axis of
correlation. This can be visualized in this illustration.

468 Chapter 6. Latent Variable Modelling

 Release 547b16

The plane will rotate around the axial, dashed line if we make small changes in the raw data. At each
new rotation we will get very different values of 𝑏1 and 𝑏2 , even changing in sign(!), but the objective
function’s minimum value does not change very much. This phenomena shows up in the least squares
solution as wide confidence intervals for the coefficients, since the off-diagonal elements in X′ X will
be large. This has important consequences when you are trying to learn about your process from this
model: you have to use caution. A model with low or uncorrelated variables is well-supported by the
data, and cannot be arbitrarily rotated.

The common “solution” to this problem of collinearity is to revert to variable selection. In the above
example the modeller would select either 𝑥1 or 𝑥2 . In general, the modeller must select a subset of
uncorrelated columns from the 𝐾 columns in X rather than using the full matrix. When 𝐾 is large,
then this becomes a large computational burden. Further, it is not clear what the trade-offs are, and
how many columns should be in the subset. When is a correlation too large to be problematic?

We face another problem with MLR: the assumption that the variables in X are measured without
error, which we know to be untrue in many practical engineering situations and is exactly what leads
to the instability of the rotating plane. Furthermore, MLR cannot handle missing data. To summarize,
the shortcomings of multiple linear regression are that:

• it cannot handle strongly correlated columns in X

• it assumes X is noise-free, which it almost never is in practice

• cannot handle missing values in X

• MLR requires that 𝑁 > 𝐾, which can be impractical in many circumstances, which leads to

• variable selection to meet the 𝑁 > 𝐾 requirement, and to gain independence between columns of
X, but that selection process is non-obvious, and may lead to suboptimal predictions.

The main idea with principal component regression is to replace the 𝐾 columns in X with their
uncorrelated 𝐴 score vectors from PCA.

6.6. Principal Component Regression (PCR) 469

Process Improvement Using Data

In other words, we replace the 𝑁 × 𝐾 matrix of raw data with a smaller 𝑁 × 𝐴 matrix of data that
summarizes the original X matrix. Then we relate these 𝐴 scores to the y variable. Mathematically it is
a two-step process:

1. T = XP from the PCA model

−1
2. y
̂︀ = Tb and can be solved as b = (T′ T) T′ y

This has a number of advantages:

1. The columns in T, the scores from PCA, are orthogonal to each other, obtaining independence for
the least-squares step.

2. These T scores can be calculated even if there are missing data in X.

3. We have reduced the assumption of errors in X, since X ̂︀ = TP′ + E. We have replaced it with the
assumption that there is no error in T, a more realistic assumption, since PCA separates the noise
from the systematic variation in X. The T’s are expected to have much less noise than the X’s.

4. The relationship of each score column in T to vector y can be interpreted independently of each
other.

5. Using MLR requires that 𝑁 > 𝐾, but with PCR this changes to 𝑁 > 𝐴; an assumption that is more
easily met for short and wide X matrices with many correlated columns.

6. There is much less need to resort to selecting variables from X; the general approach is to use the
entire X matrix to fit the PCA model. We actually use the correlated columns in X to stabilize the
PCA solution, much in the same way that extra data improves the estimate of a mean (recall the
central limit theorem).

7. But by far one of the greatest advantages of PCR though is the free consistency check that one gets
on the raw data, which you don’t have for MLR. Always check the SPE and Hotelling’s 𝑇 2 value for
a new observation during the first step. If SPE is close to the model plane, and 𝑇 2 is within the
range of the previous 𝑇 2 values, then the prediction from the second step should be reasonable.

Illustrated as follows we see the misleading strategy that is regularly seen with MLR. The modeller has
build a least squares model relating 𝑥1 and 𝑥2 to 𝑦, over the given ranges of 𝑥. The closed circles
represent the actual data, while the open circles are the projections of the 𝑥1 and 𝑥2 values on the
𝑥1 − 𝑥2 plane. The predictive model works adequately.

470 Chapter 6. Latent Variable Modelling

 Release 547b16

But the misleading strategy often used by engineers is to say that the model is valid as long as
−5 ≤ 𝑥1 ≤ +6 and −2 ≤ 𝑥2 ≤ +1. If the engineer wants to use the model at the points marked with *,
the results will be uncertain, even though those marked points obey the given constraints. The
problem is that the engineer has not taken the correlation between the variables into account. With
PCR we would immediately detect this: the points marked as * would have large SPE values from the
PCA step, indicating they are not consistent with the model.

Here then is the procedure for building a principal component regression model.

1. Collect the X and y data required for the model.

2. Build a PCA model on the data in X, fitting 𝐴 components. We usually set 𝐴 by cross-validation,
but often components beyond this will be useful. Iterate back to this point after the initial model to
assess if 𝐴 should be changed.

3. Examine the SPE and 𝑇 2 plots from the PCA model to ensure the model is not biased by unusual
outliers.

4. Use the columns in T from PCA as your data source for the usual multiple linear regression model
(i.e. they are now the X-variables in an MLR model).
−1
5. Solve for the MLR model parameters, b = (T′ T) T′ y, an 𝐴 × 1 vector, with each coefficient entry
in b corresponding to each score.

Using the principal component regression model for a new observation:

1. Obtain your vector of new data, x′new, raw , a 1 × 𝐾 vector.

2. Preprocess this vector in the same way that was done when building the PCA model (usually just
mean centering and scaling) to obtain x′new

3. Calculate the scores for this new observation: t′new = x′new P.

̂︀′new = t′new P′ .
4. Find the predicted value of this observation: x

5. Calculate the residual vector: e′new = x′new − x

̂︀′new .
√︀
6. Then compute the residual distance from the model plane: SPEnew = e′new enew

6.6. Principal Component Regression (PCR) 471

Process Improvement Using Data

𝑎=𝐴
∑︁ (︂ )︂2
𝑡new,𝑎
7. And the Hotelling’s 𝑇 2 value for the new observation: 𝑇new
2
= .
𝑎=1
𝑠𝑎
2
8. Before calculating the prediction from the PCR model, first check if the SPEnew and 𝑇new values are
below their 95% or 99% limits. If the new observation is below these limits, then go on to calculate
the prediction: 𝑦̂︀new = t′new b, where b was from the

9. If either of the SPE or 𝑇 2 limits were exceeded, then one should investigate the contributions to SPE,
𝑇 2 or the individuals scores to see why the new observation is unusual.

Predictions of 𝑦̂︀new when a point is above either limit, especially the SPE limit, are not to be trusted.

Multiple linear regression, though relatively simpler to implement, has no such consistency check on
the new observation’s 𝑥-values. It simply calculates a direct prediction for 𝑦̂︀new , no matter what the
values are in xnew .

One of the main applications in engineering for PCR is in the use of software sensors, also called
inferential sensors (page 495). The method of PLS has some distinct advantages over PCR, so we prefer
to use that method instead, as described next.

6.7 Introduction to Projection to Latent Structures (PLS)

Projection to Latent Structures (PLS) is the first step we will take to extending latent variable methods
to using more than one block of data. In the PLS method we divide our variables (columns) into two
blocks: called X and Y.

Learning how to choose which variables go in each block will become apparent later, but for now you
may use the rule of thumb that says X takes the variables which are always available when using the
model, while Y takes the variables that are not always available. Both X and Y must be available when
building the model, but later, when using the model, only X is required. As you can guess, one of the
major uses of PLS is for predicting variables in Y using variables in X, but this is not its only purpose
as a model. It is a very good model for process understanding and troubleshooting.

PLS can be used for process monitoring and for optimizing the performance of a process. It is also
widely used for new product development, or for improving existing products. In all these cases the Y
block most often contains the outcome, or quality properties.

However, PLS is most commonly used for prediction. And this is also a good way to introduce PLS. In
(chemical) engineering processes we use it to develop software sensors (also known as inferential
sensors) that predict time-consuming lab measurement in real-time, using the on-line data from our
processes. In laboratories we use spectral data (e.g. NIR spectra) to predict the composition of a liquid;
this is known as the calibration problem; once calibrated with samples of known composition we can
predict the composition of future samples.

But why use the PLS method at all?

6.7.1 Advantages of the projection to latent structures (PLS) method

So for predictive uses, a PLS model is very similar to principal component regression (page 468) (PCR)
models. And PCR models were a big improvement over using multiple linear regression (MLR). In
brief, PCR was shown to have these advantages (page 470):

• It handles the correlation among variables in X by building a PCA model first, then using those
orthogonal scores, T, instead of X in an ordinary multiple linear regression. This prevents us from
having to resort to variable selection.

472 Chapter 6. Latent Variable Modelling

 Release 547b16

• It extracts these scores T even if there are missing values in X.

• We reduce, but don’t remove, the severity of the assumption in MLR that the predictor’s, T in this
case, are noise-free. This is because the PCA scores are less noisy than the raw data X.

• With MLR we require that 𝑁 > 𝐾 (number of observations is greater than the number of variables),
but with PCR this is reduced to 𝑁 > 𝐴, and since 𝐴 ≪ 𝐾 this requirement is often true, especially
for spectral data sets.

• We get the great benefit of a consistency check on the raw data, using SPE and 𝑇 2 from PCA, before
moving to the second prediction step.

An important point is that PCR is a two-step process:

In other words, we replace the 𝑁 × 𝐾 matrix of raw data with a smaller 𝑁 × 𝐴 matrix of data that
summarizes the original X matrix. Then we relate these scores to the y variable. Mathematically it is a
two-step process:

1. T = XP
−1
2. y
̂︀ = Tb and can be solved as b = (T′ T) T′ y

The PLS model goes a bit further and introduces some additional advantages over PCR:

• A single PLS model can be built for multiple, correlated Y variables. This eliminates having to
build 𝑀 PCR models, one for each column in Y.

• The PLS model directly assumes that there is error in X and Y. We will return to this important
point of an X-space model later on.

• PLS is more efficient than PCR in two ways: with PCR, one or more of the score columns in T may
only have a small correlation with Y, so these scores are needlessly calculated. Or as is more
common, we have to extract many PCA components, going beyond the level of what would
normally be calculated (essentially over fitting the PCA model), in order to capture sufficient
predictive columns in T. This augments the size of the PCR model, and makes interpretation
harder, which is already strained by the two-step modelling required for PCR.

Similar to PCA, the basis for PCR, we have that PLS also extracts sequential components, but it does so
using the data in both X and Y. So it can be seen to be very similar to PCR, but that it calculates the
model in one go. From the last point just mentioned, it is not surprising that PLS often requires fewer
components than PCR to achieve the same level of prediction. In fact when compared to several
regression methods, MLR, ridge regression and PCR, a PLS model is often the most “compact” model.

We will get into the details shortly, but as a starting approximation, you can visualize PLS as a method
that extracts a single set of scores, T, from both X and Y simultaneously.

6.7. Introduction to Projection to Latent Structures (PLS) 473

Process Improvement Using Data

From an engineering point of view this is quite a satisfying interpretation. After all, the variables we
chose to be in X and in Y come from the same system. That system is driven (moved around) by the
same underlying latent variables.

6.7.2 A conceptual explanation of PLS

Now that you are comfortable with the concept of a latent variable using PCA and PCR, you can
interpret PLS as a latent variable model, but one that has a different objective function. In PCA the
objective function was to calculate each latent variable so that it best explains the available variance in
X𝑎 , where the subscript 𝐴 refers to the matrix X before extracting the 𝑎th component.

In PLS, we also find these latent variables, but we find them so they best explain X𝑎 and best explain
Y𝑎 , and so that these latent variables have the strongest possible relationship between X𝑎 and Y𝑎 .

In other words, there are three simultaneous objectives with PLS:

1. The best explanation of the X-space.

2. The best explanation of the Y-space.

3. The greatest relationship between the X- and Y-space.

6.7.3 A mathematical/statistical interpretation of PLS

We will get back to the mathematical details later on (page 479), but we will consider our conceptual
explanation above in terms of mathematical symbols.

In PCA, the objective was to best explain X𝑎 . To do this we calculated scores, T, and loadings P, so
that each component, t𝑎 , had the greatest variance, while keeping the loading direction, p𝑎 ,
constrained to a unit vector.

max : t′𝑎 t𝑎 subject to p′𝑎 p𝑎 = 1.0

The above was shown to be a concise mathematical way to state that these scores and loadings best
explain X; no other loading direction will have greater variance of t′𝑎 . (The scores have mean of zero,
so their variance is proportional to t′𝑎 t𝑎 ).

For PCA, for the 𝑎th component, we can calculate the scores as follows (we are projecting the values in
X𝑎 onto the loading direction p𝑎 ):

t𝑎 = X𝑎 p𝑎

Now let’s look at PLS. Earlier we said that PLS extracts a single set of scores, T, from X and Y
simultaneously. That wasn’t quite true, but it is still an accurate statement! PLS actually extracts two

474 Chapter 6. Latent Variable Modelling

 Release 547b16

sets of scores, one set for X and another set for Y. We write these scores for each space as:

t 𝑎 = X 𝑎 w𝑎 for the X-space

u𝑎 = Y𝑎 c𝑎 for the Y-space

The objective of PLS is to extract these scores so that they have maximal covariance. Let’s take a look at
this. Covariance was shown (page 210) to be:
{︀ }︀
Cov (t𝑎 , u𝑎 ) = ℰ (t𝑎 − t𝑎 )(u𝑎 − u𝑎 )

Using the fact that these scores have mean of zero, the covariance is proportional (with a constant
scaling factor of 𝑁 ) to t′𝑎 u𝑎 . So in summary, each component in PLS is maximizing that covariance, or
the dot product: t′𝑎 u𝑎 .

Now covariance is a hard number to interpret; about all we can say with a covariance number is that
the larger it is, the greater the relationship, or correlation, between two vectors. So it is actually more
informative to rewrite covariance in terms of correlations (page 213) and variances:
√︀ √︀
Cov (t𝑎 , u𝑎 ) = Correlation (t𝑎 , u𝑎 ) × Var (t𝑎 ) × Var (u𝑎 )
√︀ √︀
Cov (t𝑎 , u𝑎 ) = Correlation (t𝑎 , u𝑎 ) × t′𝑎 t𝑎 × u′𝑎 u𝑎

As this shows then, maximizing the covariance between t′𝑎 and u𝑎 is actually maximizing the 3
simultaneous objectives mentioned earlier:

1. The best explanation of the X-space: given by t′𝑎 t𝑎

2. The best explanation of the Y-space. given by u′𝑎 u𝑎

3. The greatest relationship between the X- and Y-space: given by correlation (t𝑎 , u𝑎 )

These scores, t′𝑎 and u𝑎 , are found subject to the constraints that wa′ wa = 1.0 and c′a ca = 1.0. This is
similar to PCA, where the loadings p𝑎 were constrained to unit length. In PLS we constrain the
loadings for X, called w𝑎 , and the loadings for Y, called c𝑎 , to unit length.

The above is a description of one variant of PLS, known as SIMPLS171 (simple PLS).

6.7.4 A geometric interpretation of PLS

As we did with PCA (page 423), let’s take a geometric look at the PLS model space. In the illustration
below we happen to have 𝐾 = 3 variables in X, and 𝑀 = 3 variables in Y. (In general 𝐾 ̸= 𝑀 , but
𝐾 = 𝑀 = 3 make explanation in the figures easier.) Once the data are centered and scaled we have
just shifted our coordinate system to the origin. Notice that there is one dot in X for each dot in Y.
Each dot represents a row from the corresponding X and Y matrix.
171 [Link]

6.7. Introduction to Projection to Latent Structures (PLS) 475

Process Improvement Using Data

We assume here that you understand how the scores are the perpendicular projection of each data
point onto each direction vector (if not, please review the relevant section (page 423) in the PCA notes).
In PLS though, the direction vectors, w1 and c1 , are found and each observation is projected onto the
direction. The point at which each observation lands is called the X-space score, 𝑡𝑖 , or the Y-space
score, 𝑢𝑖 . These scores are found so that the covariance between the 𝑡-values and 𝑢-values is
maximized.

As explained above (page 474), this means that the latent variable directions are oriented so that they
best explain X, and best explain Y, and have the greatest possible relationship between X and Y.

The second component is then found so that it is orthogonal to the first component in the X space (the
second component is not necessarily orthogonal in the Y-space, though it often is close to orthogonal).

476 Chapter 6. Latent Variable Modelling

 Release 547b16

6.7.5 Interpreting the scores in PLS

Like in PCA, our scores in PLS are a summary of the data from both blocks. The reason for saying that,
even though there are two sets of scores, T and U, for each of X and Y respectively, is that they have
maximal covariance. We can interpret one set of them. In this regard, the T scores are more readily
interpretable, since they are always available. The U scores are not available until Y is known. We
have the U scores during model-building, but when we use the model on new data (e.g. when making
predictions using PLS), then we only have the T scores.

The scores for PLS are interpreted in exactly the same way as for PCA (page 431). Particularly, we look
for clusters, outliers and interesting patterns in the line plots of the scores.

The only difference that must be remembered is that these scores have a different orientation to the
PCA scores. As illustrated below, the PCA scores are found so that they only explain the variance in X;
the PLS scores are calculated so that they also explain Y and have a maximum relationship between X
and Y. Most time these directions will be close together, but not identical.

6.7. Introduction to Projection to Latent Structures (PLS) 477

Process Improvement Using Data

6.7.6 Interpreting the loadings in PLS

Like with the loadings from PCA (page 435), p𝑎 ,we interpret the loadings w𝑎 from PLS in the same way.
Highly correlated variables have similar weights in the loading vectors and appear close together in
the loading plots of all dimensions.

We tend to refer to the PLS loadings, w𝑎 , as weights; this is for reasons that will be explained soon.

There are two important differences though when plotting the weights. The first is that we
superimpose the loadings plots for the X and Y space simultaneously. This is very powerful, because
we not only see the relationship between the X variables (from the w vectors), we also see the
relationship between the Y variables (from the c vectors), and even more usefully, the relationship
between all these variables.

This agrees again with our (engineering) intuition that the X and Y variables are from the same
system; they have been, somewhat arbitrarily, put into different blocks. The variables in Y could just
have easily been in X, but they are usually not available due to time delays, expense of measuring
them frequently, etc. So it makes sense to consider the w𝑎 and c𝑎 weights simultaneously.

The second important difference is that we don’t actually look at the w vectors directly, we consider
rather what is called the r vector, though much of the literature refers to it as the w* vector (w-star).
The reason for the change of notation from existing literature is that w* is confusingly similar to the
multiplication operator (e.g. w * c: is frequently confused by newcomers, whereas r : c would be
cleaner). The w* notation gets especially messy when adding other superscript and subscript elements
to it. Further, some of the newer literature on PLS, particularly SIMPLS, uses the r notation.

The r vectors show the effect of each of the original variables, in undeflated form, rather that using the
w vectors which are the deflated vectors. This is explained next.

478 Chapter 6. Latent Variable Modelling

 Release 547b16

6.7.7 How the PLS model is calculated

This section assumes that you are comfortable with the NIPALS algorithm for calculating a PCA model
(page 452) from X. The NIPALS algorithm proceeds in exactly the same way for PLS, except we iterate
through both blocks of X and Y.

The algorithm starts by selecting a column from Y𝑎 as our initial estimate for u𝑎 . The X𝑎 and Y𝑎
matrices are just the preprocessed version of the raw data when 𝑎 = 1.

Arrow 1
Perform 𝐾 regressions, regressing each column from X𝑎 onto the vector u𝑎 . The slope
coefficients from the regressions are stored as the entries in w𝑎 . Columns in X𝑎 which are
strongly correlated with u𝑎 will have large weights in w𝑎 , while unrelated columns will have
small, close to zero, weights. We can perform these regression in one go:
1
w𝑎 = · X′𝑎 u𝑎
u′𝑎 u𝑎

Step 2
w𝑎
Normalize the weight vector to unit length: w𝑎 = √︀ .
w𝑎′ w𝑎

Arrow 3
Regress every row in X𝑎 onto the weight vector. The slope coefficients are stored as entries in
t𝑎 . This means that rows in X𝑎 that have a similar pattern to that described by the weight
vector will have large values in t𝑎 . Observations that are totally different to w𝑎 will have
near-zero score values. These 𝑁 regressions can be performed in one go:

1
t𝑎 = · X 𝑎 w𝑎
w𝑎′ w𝑎

Arrow 4
Next, regress every column in Y𝑎 onto this score vector, t𝑎 . The slope coefficients are stored in

6.7. Introduction to Projection to Latent Structures (PLS) 479

Process Improvement Using Data

c𝑎 . We can calculate all 𝑀 slope coefficients:

1
c𝑎 = · Y𝑎′ t𝑎
t′𝑎 t𝑎

Arrow 5
Finally, regress each of the 𝑁 rows in Y𝑎 onto this weight vector, c𝑎 . Observations in Y𝑎 that
are strongly related to c𝑎 will have large positive or negative slope coefficients in vector u𝑎 :

1
u𝑎 = · Y 𝑎 c𝑎
c′𝑎 c𝑎

This is one round of the NIPALS algorithm. We iterate through these 4 arrow steps until the u𝑎 vector
does not change much. On convergence, we store these 4 vectors: w𝑎 , t𝑎 , c𝑎 , and u𝑎 , which jointly
define the 𝑎th component.

Then we deflate. Deflation removes variability already explained from X𝑎 and Y𝑎 . Deflation proceeds
as follows:

Step 1: Calculate a loadings vector for the X space

We calculate the loadings for the X space, called p𝑎 , using the X-space scores:
1
p𝑎 = ′ · X′𝑎 t𝑎 . This loading vector contains the regression slope of every column in X𝑎
t𝑎 t𝑎
onto the scores, t𝑎 . In this regression the x-variable is the score vector, and the y variable is the
column from X𝑎 . If we want to use this regression model in the usual least squares way, we
would need a score vector (our x-variable) and predict the column from X𝑎 as our y-variable.

If this is your first time reading through the notes, you should probably skip ahead to the next
step in deflation. Come back to this section after reading about how to use a PLS model on
new data, then it will make more sense.

Because it is a regression, it means that if we have a vector of scores, t𝑎 , in the future, we can
predict each column in X𝑎 using the corresponding slope coefficient in p𝑎 . So for the 𝑘 th
column, our prediction of column X𝑘 is the product of the slope coefficient, 𝑝𝑘,𝑎 , and the score
vector, t𝑎 . Or, we can simply predict the entire matrix in one operation: X ̂︀ = t𝑎 p′ .
𝑎

Notice that the loading vector p𝑎 was calculated after convergence of the 4-arrow steps. In
other words, these regression coefficients in p𝑎 are not really part of the PLS model, they are
merely calculated to later predict the values in the X-space. But why can’t we use the w𝑎
vectors to predict the X𝑎 matrix? Because after all, in arrow step 1 we were regressing
columns of X𝑎 onto u𝑎 in order to calculate regression coefficients w𝑎 . That would imply that
a good prediction of X𝑎 would be X ̂︀ 𝑎 = u𝑎 w′ .
𝑎

That would require us to know the scores u𝑎 . How can we calculate these? We get them from
1
u𝑎 = ′ · Y𝑎 c𝑎 . And there’s the problem: the values in Y𝑎 are not available when the PLS
c𝑎 c𝑎
model is being used in the future, on new data. In the future we will only have the new values
of X. This is why we would rather predict X𝑎 using the t𝑎 scores, since those t-scores are
available in the future when we apply the model to new data.

This whole discussion might also leave you asking why we even bother to have predictions of
the X. We do this primarily to ensure orthogonality among the t-scores, by removing
everything from X𝑎 that those scores explain (see the next deflation step).

These predictions of X
̂︀ are also used to calculate the squared prediction error, a very important
consistency check when using the PLS model on new data.

480 Chapter 6. Latent Variable Modelling

 Release 547b16

Step 2: Remove the predicted variability from X and Y

Using the loadings, p𝑎 just calculated above, we remove from X𝑎 the best prediction of X𝑎 , in
other words, remove everything we can explain about it.

̂︀ 𝑎 = t𝑎 p′
X 𝑎
̂︀ 𝑎 = X𝑎 − t𝑎 p′
E𝑎 = X𝑎 − X 𝑎

X𝑎+1 = E𝑎

For the first component, the X𝑎=1 matrix contains the preprocessed raw X-data. By
convention, E𝑎=0 is the residual matrix before fitting the first component and is just the same
matrix as X𝑎=1 , i.e. the data used to fit the first component.

We also remove any variance explained from Y𝑎 :

̂︀ 𝑎 = t𝑎 c′
Y 𝑎
̂︀ 𝑎 = Y𝑎 − t𝑎 c′
F𝑎 = Y 𝑎 − Y 𝑎

Y𝑎+1 = F𝑎

For the first component, the Y𝑎=1 matrix contains the preprocessed raw data. By convention,
F𝑎=0 is the residual matrix before fitting the first component and is just the same matrix as
Y𝑎=1 .

Notice how in both deflation steps we only use the scores, t𝑎 , to deflate. The scores, u𝑎 , are not
used for the reason described above: when applying the PLS model to new data in the future,
we won’t have the actual y-values, which means we also don’t know the u𝑎 values.

The algorithm repeats all over again using the deflated matrices for the subsequent iterations.

What is the difference between W and R?

After reading about the NIPALS algorithm for PLS (page 479) you should be aware that we deflate the X
matrix after every component is extracted. This means that w1 are the weights that best predict the t1
score values, our summary of the data in X𝑎=1 (the preprocessed raw data). Mathematically we can
write the following:

t1 = X𝑎=1 w1 = X1 w1

The problem comes once we deflate. The w2 vector is calculated from the deflated matrix X𝑎=2 , so
interpreting these scores is a quite a bit harder.

t2 = X2 w2 = (X1 − t1 p′1 ) w2
= (X1 − X1 w1 p1 ) w2

The w2 is not really giving us insight into the relationships between the score, t2 , and the data, X, but
rather between the score and the deflated data, X2 .

Ideally we would like a set of vectors we can interpret directly; something like:

t𝑎 = Xr𝑎

One can show, using repeated substitution, that a matrix R, whose columns contain r𝑎 , can be found
−1
from: R = W (P′ W) . The first column, r1 = w1 .

So our preference is to interpret the R weights (often called W* in some literature), rather than the W
weights when investigating the relationships in a PLS model.

6.7. Introduction to Projection to Latent Structures (PLS) 481

Process Improvement Using Data

6.7.8 Variability explained with each component

We can calculate 𝑅2 values, since PLS explains both the X-space and the Y-space. We use the E𝑎
matrix to calculate the cumulative variance explained for the X-space.

2 Var(E𝑎 )
𝑅X,𝑎,cum =1−
Var(X𝑎=1 )
2
Before the first component is extracted we have 𝑅X,𝑎=0 = 0.0, since E𝑎=0 = X𝑎=1 . After the second
2
component, the residuals, E𝑎=1 , will have decreased, so 𝑅X,𝑎 would have increased.

We can construct similar 𝑅2 values for the Y-space using the Y𝑎 and F𝑎 matrices. Furthermore, we
construct in an analogous manner the 𝑅2 values for each column of X𝑎 and Y𝑎 , exactly as we did for
PCA (page 441).

These 𝑅2 values help us understand which components best explain different sources of variation. Bar
plots of the 𝑅2 values for each column in X and Y, after a certain number of 𝐴 components are one of
the best ways to visualize this information.

6.7.9 Coefficient plots in PLS

After building an initial PLS model one of the most informative plots to investigate are plots of the
r : c vectors: using either bar plots or scatter plots. (The notation r : c implies we superimpose a plot
of r on a plot of c.) These plots show the relationship between variables in X, between variables in Y,
as well as the latent variable relationship between these two spaces. The number of latent variables, 𝐴,
is much smaller number than the original variables, 𝐾 + 𝑀 , effectively compressing the data into a
small number of informative plots.

There are models where the number of components is of moderate size, around 𝐴 = 4 to 8, in which
case there are several combinations of r : c plots to view. If we truly want to understand how all the X
and Y variables are related, then we must spend time investigating all these plots. However, the
coefficient plot can be a useful compromise if one wants to learn, in a single plot,how the X variables
are related to the Y variables using all 𝐴 components.

Caution using the coefficients

It is not recommended that PLS be implemented in practice as described here. In other words, do
not try make PLS like multiple linear regression and go directly from the X’s to the Y’s using
′
y
̂︀new = x′new 𝛽.

Instead, one of the major benefits of a PLS model is that we first calculate the scores, then verify 𝑇 2
and SPE are below their critical limits, e.g. 95% limits. If so, then we go ahead and calculate the
predictions of Y. Direct calculation of Y bypasses this helpful information. Furthermore, using the
𝛽 coefficients directly means that we cannot handle missing data.

Only use the coefficients to learn about your system. Do not use them for prediction.

The coefficient plot is derived as follows. First preprocess the new observation, xnew,raw , to obtain xnew .

• Project the new observation onto the model to get scores: t′new = x′new R.
′
• Calculate the predicted y
̂︀new = t′new C′ using these scores.

482 Chapter 6. Latent Variable Modelling

 Release 547b16

• Now combine these steps:

′
y
̂︀new = t′new C′
′
y
̂︀new = x′new RC′
′
y
̂︀new = x′new 𝛽

where the matrix 𝛽 is a 𝐾 × 𝑀 matrix: each column in 𝛽 contains the regression coefficients for all
𝐾 of the X variables, showing how they are related to each of the 𝑀 Y-variables.

From this derivation we see these regression coefficients are a function of all the latent variables in the
−1
model, since R = W (P′ W) as shown in an earlier section of these notes (page 481).

In the example below there were 𝐴 = 6 components, and 𝐾 = 14 and 𝑀 = 5. Investigating all 6 of the
r : c vectors is informative, but the coefficient plot provides an efficient way to understand how the X
variables are related to this particular Y variable across all the components in the model.

In this example the Tin, z2, Tcin2 and Tmax2, Fi2, Fi1, Tmax1, and Press variables are all related
to conversion, the y variable. This does not imply a cause and effect relationships, rather it just shows
they are strongly correlated.

6.7.10 Analysis of designed experiments using PLS models

Data from a designed experiment, particularly factorial experiments, will have independent columns
in X. These data tables are adequately analyzed using multiple linear regression (page 253) (MLR) least
squares models.

These factorial and fractional factorial data are also well suited to analysis with PLS. Since factorial
models support interaction terms, these additional interactions should be added to the X matrix. For
example, a full factorial design with variables A, B and C will also support the AB, AC, BC and ABC
interactions. These four columns should be added to the X matrix so that the loadings for these
variables are also estimated. If a central composite design (page 370), or some other design that supports
quadratic terms has been performed, then these columns should also be added to X, e.g.: A2 , B2 and
C2 .

The PLS loadings plots from analyzing these DOE data are interpreted in the usual manner; and the
coefficient plot is informative if 𝐴 > 2.

6.7. Introduction to Projection to Latent Structures (PLS) 483

Process Improvement Using Data

There are some other advantages of using and interpreting a PLS model built from DOE data, rather
than using the MLR approach:

• If additional data (not the main factors) are captured during the experiments, particularly measurable
disturbances, then these additional columns can, and should, be included in X. These extra data are
called covariates in other software packages. These additional columns will remove some of the
orthogonality in X, but this is why a PLS model would be more suitable.

• If multiple Y measurements were recored as the response, and particularly if these Y variables are
correlated, then a PLS model would be better suited than building 𝐾 separate MLR models. A good
example is where the response variable from the experiment is a complete spectrum of
measurements, such as from a NIR probe.

One other point to note when analyzing DOE data with PLS is that the 𝑄2 values from cross-validation
are often very small. This makes intuitive sense: if the factorial levels are suitably spaced, then each
experiment is at a point in the process that provides new information. It is unlikely that
cross-validation, when leaving out one or more experiments, is able to accurately predict each corner
in the factorial.

Lastly, models built from DOE data allow a much stronger interpretation of the loading vectors, R : C.
This time we can infer cause-and-effect behaviour; normally in PLS models the best we can say is that
the variables in X and Y are correlated. Experimental studies that are run in a factorial manner will
break happenstance correlation structures; so if any correlation that is present, then this truly is causal
in nature.

6.7.11 PLS Exercises

The taste of cheddar cheese

• 𝑁 = 30

• 𝐾=3

• 𝑀 =1

• Link to cheese data172

• Description: This very simple case study considers the taste of mature cheddar cheese. There are 3
measurements taken on each cheese: lactic acid, acetic acid and H2 S.

1. Import the data into R: cheese <-

[Link]('[Link]

2. Use the car library and plot a scatter plot matrix of the raw data:

• library(car)

• scatterplotMatrix(cheese[,2:5])
172 [Link]

484 Chapter 6. Latent Variable Modelling

 Release 547b16

import pandas as pd
import [Link] as px

filename = "[Link]
cheese = pd.read_csv(filename)
[Link]()

fig = px.scatter_matrix([Link][:, 1:5])

[Link]()

filename <- '[Link]

cheese <- [Link](filename)
summary(cheese)

library(car)
scatterplotMatrix(cheese[, 2:5],
col=c(1,1,1),
smooth=FALSE)

3. Using this figure, how many components do you expect to have in a PCA model on the 3 X
variables: Acetic, H2S and Lactic?

6.7. Introduction to Projection to Latent Structures (PLS) 485

Process Improvement Using Data

4. What would the loadings look like?

5. Build a PCA model now to verify your answers.

import pandas as pd
from process_improve.multivariate import PCA, MCUVScaler

filename = "[Link]
cheese = pd.read_csv(filename)
[Link]()

X = MCUVScaler().fit_transform([Link][:, 1:5])
model_pca = PCA(n_components=4).fit(X)
print(model_pca.r2_cumulative_)
loadings_P = model_pca.loadings_
scores_T = model_pca.scores_

filename <- '[Link]

cheese <- [Link](filename)
summary(cheese)

[Link] <- prcomp(cheese[, 2:5],

scale=TRUE)
summary([Link])
loadings.P <- [Link]$rotation
scores.T <- [Link]$x

6. Before building the PLS model, how many components would you expect? And what would the
weights look like (r1 , and c1 )?

7. Build a PLS model and plot the r : c1 bar plot. Interpret it.

8. Now plot the SPE plot; these are the SPE values for the projections onto the X-space. Any outliers
apparent?

9. In R, build a least squares model that regresses the Taste variable on to the other 3 X variables.

• [Link] <- lm(Taste ~ Acetic + H2S + Lactic, data=cheese)

• Report each coefficient ±2𝑆𝐸 (𝑏𝑖 ). Which coefficients does R find significant in MLR? (You can
use the confint([Link]) function too.)

𝛽Acetic = ±
𝛽H2S = ±
𝛽Lactic = ±

• Report the standard error and the 𝑅𝑦2 value for this model.

• Compare this to the PLS model’s 𝑅𝑦2 value.

import numpy as np
import pandas as pd
import [Link] as sm

cheese = pd.read_csv(
"[Link]
[Link]()

(continues on next page)

486 Chapter 6. Latent Variable Modelling

 Release 547b16

(continued from previous page)

# Least squares model:
X = sm.add_constant(
cheese[["Acetic", "H2S", "Lactic"]])
model_lm = [Link](cheese["Taste"], X).fit()
resid = model_lm.resid
resid_ssq = (resid ** 2).sum()
standard_error = [Link](
resid_ssq / model_lm.df_resid)
ssq_total = ((cheese["Taste"]
- cheese["Taste"].mean()) ** 2).sum()
R2_value = 1 - resid_ssq / ssq_total
print(f"Least squares SE = "
f"{round(standard_error, 2)}")
print(f"Least squares R^2 = "
f"{round(R2_value * 100, 2)}%")

cheese <- [Link]('[Link]

summary(cheese)

# Least squares model:

[Link] <- lm(Taste ~ Acetic + H2S +
Lactic, data=cheese)
resid = residuals([Link])
[Link] = sum(resid**2)
[Link] = sqrt( [Link] /
[Link]$[Link] )
[Link] = sum((cheese$Taste -
mean(cheese$Taste)) ** 2)
[Link] = 1 - [Link] / [Link]
paste0('Least squares SE = ',
round([Link], 2))
paste0('Least squares R^2 = ',
round([Link]*100, 2), '%')

10. Now build a PCR model in R using only 1 component, then using 2 components. Again calculate
the standard error and 𝑅𝑦2 values.

import numpy as np
import pandas as pd
import [Link] as sm
from process_improve.multivariate import PCA, MCUVScaler

cheese = pd.read_csv(
"[Link]
[Link]()

# PCA model with only 2 components

X_scaled = MCUVScaler().fit_transform(
[Link][:, 1:4])
model_pca = PCA(n_components=2).fit(X_scaled)
scores_T = model_pca.scores_.to_numpy()

# PCR model using only PC 1

y = cheese["Taste"]
pcr_1 = [Link](
y, sm.add_constant(scores_T[:, [0]])).fit()

# PCR model using PC 1 and PC 2

pcr_2 = [Link](
y, sm.add_constant(scores_T)).fit()

SE_1 = [Link](
(continues on next page)

6.7. Introduction to Projection to Latent Structures (PLS) 487

 Process Improvement Using Data

(continued from previous page)

(pcr_1.resid ** 2).sum() / pcr_1.df_resid)
SE_2 = [Link](
(pcr_2.resid ** 2).sum() / pcr_2.df_resid)

print(f"SE for PCR with 1 component: "

f"{round(SE_1, 2)}")
print(f"SE for PCR with 2 components: "
f"{round(SE_2, 2)}")

cheese <- [Link]('[Link]

summary(cheese)

# PCA model with only 2 components

[Link] <- prcomp(cheese[,2:4],
scale = TRUE,
rank. = 2)

scores.T <- [Link]$x

# PCR model using only PC 1

pcr.1 <- lm(cheese$Taste ~ scores.T[,1])

# PCR model using PC 1 and PC 2

pcr.2 <- lm(cheese$Taste ~ scores.T[,1:2])

SE.1 <- sqrt( sum( residuals(pcr.1)^2 ) /

pcr.1$[Link] )
SE.2 <- sqrt( sum( residuals(pcr.2)^2 ) /
pcr.2$[Link] )

paste0('SE for PCR with 1 component: ',

round(SE.1, 2))
paste0('SE for PCR with 2 components: ',
round(SE.2, 2))

11. Plot the observed y values against the predicted y values for the PLS model.

12. PLS models do not have a standard error, since the degrees of freedom√︂
are not as easily defined. But
e′ e
you can calculate the RMSEE (root mean square error of estimation) = . Compare the RMSEE
𝑁
values for all the models just built.

Obviously the best way to test the model is to retain a certain amount of testing data (e.g. 10
observations), then calculate the root mean square error of prediction (RMSEP) on those testing data.

Comparing the loadings from a PCA model to a PLS model

PLS explains both the X and Y spaces, as well as building a predictive model between the two spaces.
In this question we explore two models: a PCA model and a PLS model on the same data set.

The data are from the plastic pellets troubleshooting example (page 467).

• 𝑁 = 24

• 𝐾 = 6 + 1 designation of process outcome

• Link to raw materials data173

173 [Link]

488 Chapter 6. Latent Variable Modelling

 Release 547b16

• Description: 3 of the 6 measurements are size values for the plastic pellets, while the other 3 are the
outputs from thermogravimetric analysis (TGA), differential scanning calorimetry (DSC) and
thermomechanical analysis (TMA), measured in a laboratory. These 6 measurements are thought to
adequately characterize the raw material. Also provided is a designation Adequate or Poor that
reflects the process engineer’s opinion of the yield from that lot of materials.

1. Build a PCA model on all seven variables, including the 0-1 process outcome variable in the X
space. Previously we omitted that variable from the model, this time include it.

2. How do the loadings look for the first, second and third components?

3. Now build a PLS model, where the Y-variable is the 0-1 process outcome variable. In the previous
PCA model the loadings were oriented in the directions of greatest variance. For the PLS model the
loadings must be oriented so that they also explain the Y variable and the relationship between X
and Y. Interpret the PLS loadings in light of this fact.

4. How many components were required by cross-validation for the PLS model?

5. Explain why the PLS loadings are different to the PCA loadings.

Predicting final quality from on-line process data: LDPE system

• 𝑁 = 54

• 𝐾 = 14

• 𝐾=5

• Link to dataset website174 and description of the data.

1. Build a PCA model on the 14 X-variables and the first 50 observations.

2. Build a PCA model on the 5 Y-variables: Conv, Mn, Mw, LCB, and SCB. Use only the first 50
observations

3. Build a PLS model relating the X variables to the Y variables (using 𝑁 = 50). How many
components are required for each of these 3 models?

4. Compare the loadings plot from PCA on the Y space to the weights plot (c1 vs c2 ) from the PLS
model.
2
5. What is the 𝑅𝑋 (not for Y) for the first few components?

6. Now let’s look at the interpretation between the X and Y space. Which plot would you use?

• Which variable(s) in X are strongly related to the conversion of the product (Conv)? In other
words, as an engineer, which of the 14 X variables would you consider adjusting to improve
conversion.

• Would these adjustments affect any other quality variables? How would they affect the other
quality variables?

• How would you adjust the quality variable called Mw (the weight average molecular weight)?
174 [Link]

6.7. Introduction to Projection to Latent Structures (PLS) 489

Process Improvement Using Data

6.8 Applications of Latent Variable Models

6.8.1 Improved process understanding

Interpreting the loadings plot (page 435) from a model is well worth the time spent. At the very least, one
will confirm what you already know about the process, but sometimes there are unexpected insights
that are revealed. Guide your interpretation of the loadings plot with contributions in the scores, and
cross-referencing with the raw data, to verify your interpretation.

There are 𝐴 loadings and score plots. In many cases this is far fewer than the 𝐾 number of original
variables. Furthermore, these 𝐴 variables have a much higher signal and lower noise than the original
data. They can also be calculated if there are missing data present in the original variables.

In the example shown here the company was interested in how their product performed against that
of their competitor. Six variables called A to F were measured on all the product samples, (codes are
used, because the actual variables measured are proprietary). The loadings for these 6 variables are
labelled below, while the remaining points are the scores. The scores have been scaled and
superimposed on the loadings plot, to create what is called a biplot. The square, green points were the
competitor’s product, while the smaller purple squares were their own product.

From this single figure the company learned that:

• The quality characteristics of this material is not six-dimensional; it is two-dimensional. This means
that based on the data used to create this model, there is no apparent way to independently
manipulate the 6 quality variables. Products produced in the past land up on a 2-dimensional latent
variable plane, rather than a 6-dimensional space.

490 Chapter 6. Latent Variable Modelling

 Release 547b16

• Variables D, E and F in particular are very highly correlated, while variable C is also somewhat
correlated with them. Variable A and C are negatively correlated, as are variables B and C. Variables
A and B are positively correlated with each other.

• This company’ competitor was able to manufacture the product with much lower variability than
they were: there is greater spread in their own product, while the competitor’s product is tightly
clustered.

• The competitors product is characterized as having lower values of C, D, E, and F, while the values
of A and B are around the average.

• The company had produced product similar to their competitor’s only very infrequently, but since
their product is spread around the competitor’s, it indicates that they could manufacture product of
similar characteristics to their competitor. They could go query the score values close those of those
of the competitors and using their company records, locate the machine and other process settings
in use at that time.

However, it might not just be how they operate the process, but also which raw materials and their
consistency, and the control of outside disturbances on the process. These all factor into the final
product’s variability.

It it is not shown here, but the competitor’s product points are close to the model plane (low SPE
values), so this comparison is valid. This analysis was tremendously insightful, and easier to complete
on this single plot, rather than using plots of the original variables.

6.8.2 Troubleshooting process problems

We already saw a troubleshooting example in the section on interpreting scores (page 431). In general,
troubleshooting with latent variable methods uses this approach:

1. Collect data from all relevant parts of the process: do not exclude variables that you think might be
unimportant; often the problems are due to unexpected sources. Include information on operators,
weather, equipment age (e.g. days since pump replacement), raw material properties being
processed at that time, raw material supplier (indicator variable). Because the PCA model
disregards unimportant or noisy variables, these can later be pruned out, but they should be kept in
for the initial analysis. (Note: this does not mean the uninformative variables are not important -
they might only be uninformative during the period of data under observation).

2. Structure the data so that the majority of the data is from normal, common-cause operation. The
reason is that the PCA model plane should be oriented in the directions of normal operation. The
rest of the X matrix should be from when the problem occurs and develops.

6.8. Applications of Latent Variable Models 491

Process Improvement Using Data

3. Given the wealth of data present on many processes these days, it is helpful to prune the X matrix
so that it is only several hundred rows in length. Simply subsample, or using averages of time; e.g.
hourly averages. Later we can come back and look at a higher resolution. Even as few as 50 rows
can often work well.

4. Build the PCA model. You should observe the abnormal operation appearing as outliers in the score
plots and SPE plots. If not, use colours or different markers to highlight the regions of poor
operation in the scores: they might be clustered in a region of the score plot, but not appear as
obvious outliers.

5. Interrogate and think about the model. Use the loadings plots to understand the general trends
between the variables. Use contribution plots to learn why clusters of observations are different
from others. Use contribution plots to isolate the variables related to large SPE values.

6. It should be clear that this is all iterative work; the engineer has to be using her/his brain to
formulate hypotheses, and then verify them in the data. The latent variable models help to reduce
the size of the problem down, but they do not remove the requirement to think about the data and
interpret the results.

Here is an example where the yield of a company’s product was declining. They suspected that their
raw material was changing in some way, since no major changes had occurred on their process. They
measured 6 characteristic values on each lot (batch) of raw materials: 3 of them were a size
measurement on the plastic pellets, while the other 3 were the outputs from thermogravimetric
analysis (TGA), differential scanning calorimetry (DSC) and thermomechanical analysis (TMA),
measured in a laboratory. Also provided was an indication of the yield: “Adequate” or “Poor”. There
were 24 samples in total, 17 batches of adequate yield and the rest the had poor yield.

The score plot (left) and loadings plot (right) help isolate potential reasons for the reduced yield.
Batches with reduced yield have high, positive 𝑡2 values and low, negative 𝑡1 values. What factors lead
to batches having score values with this combination of 𝑡1 and 𝑡2 ? It would take batches with a
combination of low values of TGA and TMA, and/or above average size5, size10 and size15 levels,
and/or high DSC values to get these sort of score values. These would be the generally expected trends,
based on an interpretation of the scores and loadings.

492 Chapter 6. Latent Variable Modelling

 Release 547b16

We can investigate specific batches and look at the contribution of each variable to the score values.
Let’s look at the contributions for batch 8 for both the 𝑡1 and 𝑡2 scores.

𝑡8,𝑎=1 = 𝑥s5 𝑝s5,1 + 𝑥s10 𝑝s10,1 + 𝑥s15 𝑝s15,1 + 𝑥TGA 𝑝TGA,1 + 𝑥DSC 𝑝DSC,1 + 𝑥TMA 𝑝TMA,1
𝑡8,𝑎=1 = −0.85 − 0.74 − 0.62 + 0.27 + 0.12 + 0.10
𝑡8,𝑎=2 = 𝑥s5 𝑝s5,2 + 𝑥s10 𝑝s10,2 + 𝑥s15 𝑝s15,2 + 𝑥TGA 𝑝TGA,2 + 𝑥DSC 𝑝DSC,2 + 𝑥TMA 𝑝TMA,2
𝑡8,𝑎=2 = 0.39 + 0.44 + 0.14 + 0.57 + 0.37 + 0.24

Batch 8 is at its location in the score plot due to the low values of the 3 size variables (they have strong
negative contributions to 𝑡1 , and strong positive contributions to 𝑡2 ); and also because of its very large
DSC value (the 0.57 contribution in 𝑡2 ).

Batch 22 on the other hand had very low values of TGA and TMA, even though its size values were
below average. Let’s take a look at the 𝑡2 value for batch 22 to see where we get this interpretation:

𝑡22,𝑎=2 = 𝑥s5 𝑝s5,2 + 𝑥s10 𝑝s10,2 + 𝑥s15 𝑝s15,2 + 𝑥TGA 𝑝TGA,2 + 𝑥DSC 𝑝DSC,2 + 𝑥TMA 𝑝TMA,2
𝑡22,𝑎=2 = −0.29 − 0.17 − 0.08 + 0.84 − 0.05 + 1.10

This illustrates that the actual contribution values are a more precise diagnostic tool that just
interpreting the loadings.

6.8.3 Optimizing: new operating point and/or new product development

This application area is rapidly growing in importance. Fortunately it is fairly straightforward to get
an impression of how powerful this tool is. Let’s return back to the food texture example considered
previously (page 428), where data from a biscuit/pastry product was considered. These 5
measurements were used:

1. Percentage oil in the pastry

2. The product’s density (the higher the number, the more dense the product)

3. A crispiness measurement, on a scale from 7 to 15, with 15 being more crispy.

4. The product’s fracturability: the angle, in degrees, through which the pasty can be slowly bent
before it fractures.

5. Hardness: a sharp point is used to measure the amount of force required before breakage occurs.

The scores and loadings plot are repeated here again:

6.8. Applications of Latent Variable Models 493

Process Improvement Using Data

Process optimization follows the principle that certain regions of operation are more desirable than
others. For example, if all the pastry batches produced on the score plot are of acceptable quality, there
might be regions in the plot which are more economically profitable than others.

For example, pastries produced in the lower right quadrant of the score plot (high values of 𝑡1 and low
values of 𝑡2 ), require more oil, but might require a lower cooking time, due to the decreased product
density. Economically, the additional oil cost is offset by the lower energy costs. All other things being
equal, we can optimize the process by moving production conditions so that we consistently produce
pastries in this region of the score plot. We could cross-reference the machine settings for the days
when batches 17, 49, 36, 37 and 30 were produced and ensure we always operate at those conditions.

New product development follows a similar line of thought, but uses more of a “what-if” scenario. If
market research or customer requests show that a pastry product with lower oil, but still with high
crispiness is required, we can initially guess from the loadings plot that this is not possible: oil
percentage and crispiness are positively correlated, not negatively correlated.

But if our manager asks, can we readily produce a pastry with the 5 variables set at [Oil=14%,
Density=2600, Crispy=14, Fracture can be any value, Hardness=100]. We can treat this as a new
observation, and following the steps described in the earlier section on using a PCA model (page 455), we
will find that e = [2.50, 1.57, −1.10, −0.18, 0.67], and the SPE value is 10.4. This is well above the 95%
limit of SPE, indicating that such a pastry is not consistent with how we have run our process in the
past. So there isn’t a quick solution.

Fortunately, there are systematic tools to move on from this step, but they are beyond the level of this
introductory material. They involve the inversion and optimization of latent variable models. This
paper is a good starting point if you are interested in more information: Christiane Jaeckle and John
MacGregor, “Product design through multivariate statistical analysis of process data175 ”. AIChE
Journal, 44, 1105-1118, 1998.

The general principle in model inversion problems is to manipulate the any degrees of freedom in the
process (variables that can be manipulated in a process control sense) to obtain a product as close as
possible to the required specification, but with low SPE in the model. A PLS model built with these
manipulated variables, and other process measurements in X, and collecting the required product
specifications in Y can be used for these model inversion problems.
175 [Link]

494 Chapter 6. Latent Variable Modelling

 Release 547b16

6.8.4 Predictive modelling (inferential sensors)

This section will be expanded soon, but we give an outline here of what inferential sensors are, and
how they are built. These sensors also go by the names of software sensors or just soft sensors.

The intention of an inferential sensor is to infer a hard-to-measure property, usually a lab measurement
or an expensive measurement, using a combination of process data and software-implemented
algorithms.

Consider a distillation column where various automatic measurements are used to predict the vapour
pressure. The actual vapour pressure is a lab measurement, usually taken 3 or 4 times per week, and
takes several hours to complete. The soft sensor can predict the lab value from the real-time process
measurements with sufficient accuracy. This is a common soft sensor on distillation columns. The lab
values are used to build (train) the software sensor and to update in periodically.

Other interesting examples use camera images to predict hard-to-measure values. In the paper by
Honglu Yu, John MacGregor, Gabe Haarsma and Wilfred Bourg176 (Ind. Eng. Chem. Res., 42, 3036–3044,
2003), the authors describe how machine vision is used to predict, in real-time, the seasoning of
various snack-food products. This sensors uses the colour information of the snacks to infer the
amount of seasoning dispensed onto them. The dispenser is controlled via a feedback loop to ensure
the seasoning is at target.

Once validated, a soft sensor can also reduce costs of a process by allowing for rapid feedback control
of the inferred property, so that less off-specification product is produced. They also often have the
side-effect that reduced lab sampling is required; this saves on manpower costs.

Soft sensors using latent variables will almost always be PLS models. Once the model has been built, it
can be applied in real-time. The 𝑇 2 and SPE value for each new observation is checked for consistency
with the model before a prediction is made. Contribution plots are used to diagnose unusual
observations.

It is an indication that the predictive models need to be updated if the SPE and/or 𝑇 2 values are
consistently above the limits. This is a real advantage over using an MLR-based model, which has no
such consistency checks.

6.8.5 Process monitoring using latent variable methods

Any variable can be monitored using control charts, as we saw in the earlier section on process
monitoring (page 149). The main purpose of these charts is to rapidly distinguish between two types of
operation: in-control and out-of-control. We also aim to have a minimum number of false alarms (type
I error: we raise an alarm when one isn’t necessary) and the lowest number of false negatives possible
(type II error, when an alarm should be raised, but we don’t pick up the problem with the chart). We
used Shewhart charts, CUSUM and EWMA charts to achieve these goals.

Consider the case of two variables, called 𝑥1 and 𝑥2 , shown on the right, on the two horizontal axes.
These could be time-oriented data, or just measurements from various sequential batches of material.
The main point is that each variable’s 3𝜎 Shewhart control limits indicate that all observations are
within control. It may not be apparent, but these two variables are negatively correlated with each
other: as 𝑥1 increases, the 𝑥2 value decreases.
176 [Link]

6.8. Applications of Latent Variable Models 495

Process Improvement Using Data

Rearranging the axes at 90 degrees to each other, and plotting the joint scatter plot of the two variables
in the upper left corner reveals the negative correlation, if you didn’t notice it initially. Ignore the
ellipse for now. It is clear that sample 10 (green closed dot, if these notes are printed in colour) is very
different from the other samples. It is not an outlier from the perspective of 𝑥1 , nor of 𝑥2 , but jointly it
is an outlier. This particular batch of materials would result in very different process operation and
final product quality to the other samples. Yet a producer using separate control charts for 𝑥1 and 𝑥2
would not pick up this problem.

While using univariate control charts is necessary to pick up problems, univariate charts are not
sufficient to pick up all quality problems if the variables are correlated. The key point here is that
quality is a multivariate attribute. All our measurements on a system must be jointly within in the
limits of common operation. Using only univariate control charts will raise the type II error: an alarm
should be raised, but we don’t pick up the problem with the charts.

Let’s take a look at how process monitoring can be improved when dealing with many attributes (many
variables). We note here that the same charts are used: Shewhart, CUSUM and EWMA charts, the only
difference is that we replace the variables in the charts with variables from a latent variable model. We
monitor instead the:

• scores from the model, 𝑡1 , 𝑡2 , . . . , 𝑡𝐴

𝑎=𝐴
∑︁ (︂ 𝑡𝑎 )︂2
• Hotelling’s 𝑇 2 =
𝑎=1
𝑠𝑎

• SPE value

The last two values are particularly appealing: they measure the on-the-plane and off-the-plane
variation respectively, compressing 𝐾 measurements into 2 very compact summaries of the process.

496 Chapter 6. Latent Variable Modelling

 Release 547b16

There are a few other good reasons to use latent variables models:

• The scores are orthogonal, totally uncorrelated to each other. The scores are also unrelated to the
SPE: this means that we are not going to inflate our type II error rate, which happens when using
correlated variables.

• There are far fewer scores than original variables on the process, yet the scores capture all the
essential variation in the original data, leading to fewer monitoring charts on the operators’ screens.

• We can calculate the scores, 𝑇 2 and SPE values even if there are missing data present; conversely,
univariate charts have gaps when sensors go off-line.

• Rather than waiting for laboratory final quality checks, we can use the automated measurements
from our process. There are many more of these measurements, so they will be correlated – we have
to use latent variable tools. The process data are usually measured with greater accuracy than the
lab values, and they are measured at higher frequency (often once per second). Furthermore, if a
problem is detected in the lab values, then we would have to come back to these process data
anyway to uncover the reason for the problem.

• But by far, one of the most valuable attributes of the process data is the fact that they are measured
in real-time. The residence time in complex processes can be in the order of hours to days, going
from start to end. Having to wait till much later in time to detect problems, based on lab
measurements can lead to monetary losses as off-spec product must be discarded or reworked.
Conversely, having the large quantity of data available in real-time means we can detect faults as
they occur (making it much easier to decode what went wrong). But we need to use a tool that
handles these highly correlated measurements.

A paper that outlines the reasons for multivariate monitoring is by John MacGregor, “Using on-line
process data to improve quality: Challenges for statisticians177 ”, International Statistical Review, 65, p
309-323, 1997.

We will look at the steps for phase I (building the monitoring charts) and phase II (using the
monitoring charts).

Phase I: building the control chart

The procedure for building a multivariate monitoring chart, i.e. the phase I steps:

• Collect the relevant process data for the system being monitored. The preference is to collect the
measurements of all attributes that characterize the system being monitored. Some of these are
direct measurements, others might have to be calculated first.

• Assemble these measurements into a matrix X.

• As we did with univariate control charts, remove observations (rows) from X that are from out-of
control operation, then build a latent variable model (either PCA or PLS). The objective is to build a
model using only data that is from in-control operation.

• In all real cases the practitioner seldom knows which observations are from in-control operation, so
this is an iterative step.

– Prune out observations which have high 𝑇 2 and SPE (after verifying they are outliers).

– Prune out variables in X that have low 𝑅2 .

177 [Link]

6.8. Applications of Latent Variable Models 497

Process Improvement Using Data

• The observations that are pruned out are excellent testing data that can be set aside and used later
to verify the detection limits for the scores, 𝑇 2 and SPE.

• The control limits depend on the type of variable:

– Each score has variance of 𝑠2𝑎 , so this can be used to derive the Shewhart or EWMA control limits.
√
Recall that Shewhart limits are typically placed at ±3𝜎/ 𝑛, for subgroups of size 𝑛.

– Hotelling’s 𝑇 2 and SPE have limits provided by the software (we do not derive here how these
limits are calculated, though its not difficult).

However, do not feel that these control limits are fixed. Adjust them up or down, using your testing
data to find the desirable levels of type I and type II error.

• Keep in reserve some “known good” data to test what the type I error level is; also keep some
“known out-of-control” data to assess the type II error level.

Phase II: using the control chart

The phase II steps, when we now wish to apply this quality chart on-line, are similar to the phase II
steps for univariate control charts (page 152). Calculate the scores, SPE and Hotelling’s 𝑇 2 for the new
observation, x′new , as described in the section on using an existing PCA model (page 455). Then plot these
new quantities, rather than the original variables. The only other difference is how to deal with an
alarm.

The usual phase II approach when an alarm is raised is to investigate the variable that raised the
alarm, and use your engineering knowledge of the process to understand why it was raised. When
using scores, SPE and 𝑇 2 , we actually have a bit more information, but the approach is generally the
same: use your engineering knowledge, in conjunction with the relevant contribution plot.

• A score variable, e.g. 𝑡𝑎 raised the alarm. We derived earlier (page 431) that the contribution to each
score was 𝑡new,𝑎 = 𝑥new,1 𝑝1,𝑎 + 𝑥new,2 𝑝2,𝑎 + . . . + 𝑥new,𝑘 𝑝𝑘,𝑎 + . . . + 𝑥new,𝐾 𝑝𝐾,𝑎 . It indicates which
of the original 𝐾 variables contributed most to the very high or very low score value.

• SPE alarm. The contribution to SPE for a new observation was derived in an earlier section
(page 439) as well; it is conveniently shown using a barplot of the 𝐾 elements in the vector below.
These are the variables most associated with the broken correlation structure.

e′new = x′new − x
^ ′new = x′new − t′new P′
[︁ ]︁
= (𝑥new,1 − 𝑥 ^new,1 ) (𝑥new,2 − 𝑥 ^new,2 ) . . . (𝑥new,𝑘 − 𝑥
^new,𝑘 ) . . . (𝑥new,𝐾 − 𝑥
^new,𝐾 )

• 𝑇 2 alarm: an alarm in 𝑇 2 implies one or more scores are large. In many cases it is sufficient to go
2
investigate the score(s) that caused the value of 𝑇new to be large. Though as long as the SPE value is
below its alarm level, many practitioners will argue that a high 𝑇 2 value really isn’t an alarm at all;
it indicates that the observation is multivariately in-control (on the plane), but beyond the
boundaries of what has been observed when the model was built. My advice is to consider this
point tentative: investigate it further (it might well be an interesting operating point that still
produces good product).

6.8.6 Dealing with higher dimensional data structures

This section just gives a impression how 3-D and higher dimensional data sets are dealt with. Tools
such as PCA and PLS work on two-dimensional matrices. When we receive a 3-dimensional array,
such as an image, or a batch data set, then we must unfold that array into a (2D) matrix if we want to
use PCA and PLS in the usual manner.

498 Chapter 6. Latent Variable Modelling

 Release 547b16

The following illustration shows how we deal with an image, such as the one taken from a colour
camera. Imagine we have 𝐼 rows and 𝐽 columns of pixels, on 3 layers (red, green and blue
wavelengths). Each entry in this array is an intensity value, a number between 0 and 255. For example,
a pure red pixel is has the following 3 intensity values in layer 1, 2 and 3: (255, 0, 0), because layer 1
contains the intensity of the red wavelengths. A pure blue pixel would be (0, 0, 255), while a pure
green pixel would be (0, 255, 0) and a pure white pixel is (255, 255, 255). In other words, each pixel is
represented as a triplet of 3 intensity values.

In the unfolded matrix we have 𝐼𝐽 rows and 3 columns. In other words, each pixel in the image is
represented in its own row. A digital image with 768 rows and 1024 columns, would therefore be
unfolded into a matrix with 786,432 rows and 3 columns. If we perform PCA on this matrix we can
calculate score values and SPE values: one per pixel. Those scores can be refolded back into the
original shape of the image. It is useful to visualize those scores and SPE values in this way.

6.8. Applications of Latent Variable Models 499

Process Improvement Using Data

You can learn more about using PCA on image data in the manual that accompanies the interactive
software that is freely available from [Link]

500 Chapter 6. Latent Variable Modelling

CHAPTER 7
APPLICATIONS OF PROCESS IMPROVEMENT USING DATA

Over the years since 2010 when this online book has been available, there has been the request of case
studies and examples showing how the tools can be implemented. The focus of this chapter is not just
the direct implementation of the equations and tools, but a discussion about the thought process, the
details and issues that came up during the cases.

7.1 Product development and product improvement

Most “product development” is really product improvement: new products are seldom developed
from scratch. The usage examples below show what this looks like in practice, and they motivate the
rest of the chapter.

7.1.1 Usage examples

• Colleague: Our most high profile customer wants us to develop a product with similar, but different
specifications to the prior products we have made. Is it feasible to say ‘yes’ to them?

• You: we have an existing product, but a customer just wants to change one of the 7 specifications:
they want a slightly higher viscosity. Keep the ingredients and ratios the same, but which process
setting(s) do we change?

• Manager: Keep the specifications the same, but adjust the process to use less energy and reduce
emissions, even if we use slightly more expensive materials, with different ratios. Is this possible?

• Engineer: A constraint has changed (e.g. a new government regulation, we have to use a different
piece of equipment): how can we still get the same final product by adjusting the process
conditions, or materials used in the process?

• Financial controller: We can buy the raw ingredient from 4 different suppliers. Which suppliers do
we pick to most cost-effectively make the product, but still achieve the specifications?

• Engineer 2: Our current top line product is made with 6 different ingredients. Can we reduce this
number down by adjusting the ratios or the choices of ingredients?

As these examples show, the common case is not a request for entirely new specifications, but a change
to one of three things while the specifications stay the same:

• which ingredients (raw materials) we use,

501
Process Improvement Using Data

• the ratios in which we combine them, by mass fraction, and

• the process conditions used to make the product.

The methods in this chapter handle both situations — a wholly new product and the adaptation of an
existing one — because the underlying problem is the same: pick values for those three groups of
variables that produce the desired outcomes, using whatever historical data we have to guide the
choice.

7.1.2 The three degrees of freedom

The usage examples above already named the three things we can change. Each Design step in a
product-development cycle is a choice over these three groups, and the rest of the chapter keeps
coming back to them, so it is worth pinning the vocabulary down explicitly.

1. Select the ingredients. This is a discrete choice: either an ingredient is in the recipe, or it is not. The
candidate set is usually a catalogue or database of materials. In many of the usage examples above
this degree of freedom is actually fixed — regulatory constraints, validation cost, or the risk of
unexpected side-reactions mean we have to keep using what we already use.

2. Adjust the ratios of the ingredients. This is a continuous choice constrained to a simplex: the mass
fractions sum to 1, so reducing one ingredient forces another to increase. This sum-to-one structure
is exactly what mixture designs (page 378) are built to handle.

3. Choose the process conditions. Temperature, pH, residence time, addition order, and the on/off
state of optional steps. This is usually where the most degrees of freedom live, and where the
historical data has the most correlation: temperature and flow rate, for example, are rarely
independent in the historical record even when they are independent on paper.

7.1.3 Specifying the desired outcome

The desired outcome is the end goal: a vector of one or more specifications. A simple case is three
scalars — a viscosity, a melting point, a density — that jointly define what “good” means.

Some entries are inequalities rather than targets: an elongation of 15 or lower is acceptable, a shelf-life of
30 days or greater is acceptable. These are yes/no constraints, and they introduce a discontinuity into
the objective. Discontinuities are awkward for the optimizers later in the chapter, so we replace each
one with a smoothed indicator — a sigmoid or a Gompertz function178 — that approximates the cliff
but stays differentiable.

The desired outcome is sometimes a very long vector: a release-rate curve, a pH trajectory, an NIR
spectrum. The entries in such a vector are heavily correlated, so we do not work with them directly.
The first step is a principal component model (page 421) of the output space; the few scores that explain it
become the specification, and the rest of the methodology proceeds unchanged.

7.1.4 Product design uses every chapter of this book

The introduction to latent variable methods (page 413) listed the five main areas where engineers extract
value from process data:

1. Improved process understanding — confirming what we know, and seeing the unexpected, often
by inspecting score and loading plots (page 431).
178 [Link]

502 Chapter 7. Applications of Process Improvement using Data

 Release 547b16

2. Troubleshooting process problems — isolating which variables drove a deviation, using the
multivariate troubleshooting tools (page 491).

3. Improving, optimizing and controlling processes — moving the operating point on purpose,
using designed experiments (page 319) and response surface methods (page 365).

4. Predictive modelling — estimating a hard-to-measure quantity from the easy-to-measure ones,

either with least-squares models (page 209) or with inferential sensors (page 495) built on PLS.

5. Process monitoring — raising an alarm when the process drifts away from where it should be, with
univariate charts (page 149) or multivariate ones (page 495).

Product design and improvement is not a sixth area. It is what happens when we do all five at once, on
the same data set, in the same iteration. Each Design-Build-Test-Learn cycle (introduced below) draws
on every one of them:

• We start each campaign with a round of process understanding on the historical (F, Z, Y) data:
which materials, ratios and conditions drove the past quality outcomes? The score and loading
plots of a PCA (page 421) or PLS (page 472) model on these matrices answer that question directly.

• Each new experiment that misses the target is an act of troubleshooting: the same multivariate
contribution plots that diagnose a process upset tell us which input is responsible for a missed
quality target.

• The improvement and optimization step is what proposes the next recipe and conditions to try.
Sometimes this is a response-surface optimization (page 365) on top of a designed experiment;
sometimes it is mixture-design (page 378) reasoning on the ratios; often it is the inverse of a PLS
model, with constraints (page 375) from the operating window.

• Predictive models are how slow or expensive measurements — a taste panel, a 30-day shelf-life
test, a customer trial — are estimated from quick lab data so the cycle does not stall waiting for the
slowest test.

• Monitoring in the latent-variable space tells us whether a proposed recipe is still inside the region
where the historical model can be trusted, or whether the optimizer has pushed us into an
extrapolated region where the predictions are not reliable.

The chapter therefore does not introduce many new techniques: it shows how to assemble the
methods you already know into a single workflow that designs and improves products.

7.1.5 Why product development is difficult

Most product development is still done by intuition and trial-and-error. The methods later in this
chapter can speed that work up considerably, but only if they address the things that make the
problem difficult in the first place. The challenges below appear in almost every campaign, and a tool
that ignores them will not improve on a good engineer’s intuition.

We frame each iteration as a Design-Build-Test-Learn (DBTL) cycle, a framing widely used in the
biotechnology and autonomous-experimentation literature:

• Design: pick the values for the three groups of degrees of freedom (ingredients, ratios, conditions).

• Build: run one or more experiments with that design.

• Test: measure the outcomes on the resulting product.

• Learn: compare the outcomes against the targets, update the model, decide on the next iteration.

7.1. Product development and product improvement 503

Process Improvement Using Data

We do not get the product right on the first cycle, so we iterate. Even when you work by eye from plots
you are running an implicit model of the system; the methods in this chapter make that model explicit,
so it can be reused, criticized and improved. Six broad areas of difficulty recur — one for each stage of
the cycle, plus a cross-cutting set that affects the iteration as a whole.

Problems with the specifications

• The targets are usually correlated. As one increases, so does another. A specification that treats
them as independent (or constrains one and lets the other float) ignores this structure and will
mislead the optimization.

• Specifications are often asymmetric. A weight of “ideally above 4 g” is a soft target, while “must not
exceed 7 g” is a hard limit. Casting both into a single objective requires care, often using a smoothed
indicator such as a sigmoid or Gompertz function179 .

• Each DBTL cycle is solved as an optimization, so we have to capture several targets in a single
objective. Sum of squared deviations? Absolute deviations? Equal weights, or a ranking? And how
do we scale a target measured in 1000s so it does not swamp one measured in 0.01s?

• Profit is often suggested as the objective, but reliable estimates of selling price, raw-material costs,
emissions, labour and rework costs are rarely all available at the same time.

• Closeness to a lab-scale specification is not the same as robustness in production or in the

customer’s hands. An optimum found in the lab can perform poorly at full scale.

Problems in the Design step

• The first iteration has little or no data, so the search direction has to come from prior knowledge or
screening designs (page 357).

• When experiments can run in parallel (e.g. on a robotic platform), how many points, where in the
input space, and with how many replicates?

• Step size matters. Moving from 𝑥 = 0.5 to 𝑥 = 0.499 is wasteful; moving to 𝑥 = 0.2 may be too far.
The right step balances information against cost, and we would like that choice to be automatic in a
self-driving DBTL loop.

• Bringing in new candidate ingredients is a combinatorial problem. With 100 candidate plastics it
helps to rank them on a continuous scale (e.g. thermal stability) so the choice becomes a numeric
variable rather than a 100-way switch.

• On/off effects: the mere presence of an ingredient at small amounts can flip the system’s behaviour,
creating strong nonlinearities that are hard to model.

• Process conditions are sometimes discrete (stirrer at off / low / high), and the order of
manufacturing steps matters.

• Model inversion is non-unique. Solving 𝑥 + 𝑦 = 4 has infinitely many solutions, and product design
almost always has more inputs (manipulated variables) than outputs (targets), so the inverse
problem is underdetermined.
179 [Link]

504 Chapter 7. Applications of Process Improvement using Data

 Release 547b16

Problems in the Build step

• How repeatable can you run the same recipe? Without good control here an apparent improvement
may just be experimental noise, and a move to a new operating region cannot be distinguished from
drift.

• Even when a recipe is reproduced perfectly, there are block effects (page 362) between iterations.
Including references and controls in each cycle lets us correct for these.

• Is the lab system a faithful proxy for how the customer actually uses the product? A product that is
robust on the bench and fails in customer hands has not really been improved.

Problems in the Test step

• Measurement reproducibility. Outputs sometimes shift between iterations because of uncontrolled

factors. We would like to eliminate those factors, or at least correct for them.

• Outputs interact. Viscosity readings depend on pH, for example, so the test result is itself a function
of more than one quality.

• Sensory and slow lab measurements cannot be done on every experiment. A taste panel, or a 5-day
shelf-life test, will not keep up with a fast iteration loop. Inferential sensors (page 495) and other
surrogate measurements speed up the cycle, but they introduce their own error.

Problems in the Learn step

• Which way do we go next: explore an unexamined region, or exploit the neighbourhood of the
current best result? And how should the balance shift as we accumulate cycles?

• Which model family should we use for the forward (inputs to outputs) prediction: Gaussian
processes, polynomial response surfaces, splines, latent variable models? Which can be inverted
analytically, and which require an optimization to invert?

• When do we drop old data; when is a surprising point an outlier (page 262) and not the next ah-ha
result; and how do we use the model’s predictive uncertainty to decide where to sample next?

Problems with the cycle itself

• When do we stop? The obvious case is when the goal is reached, but we also need to detect
diminishing returns, and to recognize when no feasible solution exists.

• An optimum sits in a nonlinear region by definition. How do we know we have found the global
peak rather than a local one?

• Storage and provenance. How do we name experiments, record ratios and units, and capture
covariates (ambient humidity, operator, lot numbers, raw-material amounts that were “constant”)
that we did not think mattered until the day they did? Model versioning, input-data lineage and a
clear audit trail are part of this discipline.

What we ask of an ideal framework

The difficulties above tell us what an ideal framework needs to do. Most of those properties —
handling correlated and high-dimensional outputs, tolerating missing data, working from small
experimental sets, balancing exploration against exploitation, supporting model inversion — are
addressed directly by the methods later in this chapter and the cross-references above. Four further
properties cut across the whole cycle and are worth stating explicitly:

7.1. Product development and product improvement 505

Process Improvement Using Data

• Guided by the subject matter expert. The tool accelerates the expert; it does not replace them. The
SME specifies constraints, vetoes infeasible suggestions, and chooses between alternative solutions
when the inversion is non-unique.

• Inductive, not transductive. The model must generalize to new ingredients, conditions and
properties — not just interpolate within the training set. This is why we represent the recipe
through a property database (the D matrix in the next section), rather than as a categorical choice
between named materials. See the transduction180 entry for the contrast with the inductive case.

• Interpretable. The results must be understandable to the expert: they should confirm prior
knowledge where it exists, and surface new insight where it does not. Score and loading plots from
PCA (page 421) and PLS (page 472) are the workhorses here, which is one reason we lean on latent
variable methods (page 411) for the rest of the chapter.

• Reports an operating window. When constraints are active at a solution, the framework names
them, so we learn the limits of the system. When constraints are inactive, the directions in which we
can move without losing the solution are reported as an operating window, and ideally the solution
is parameterised so the SME can fine-tune within that window. Multivariate troubleshooting tools
(page 491) give us the same diagnostic in the latent-variable space.

The remainder of this chapter shows how the methods already developed in the book — designed
experiments (page 319), response surface methods (page 365) and latent variable models (page 411), together
with standard and Bayesian optimization — deliver against these properties.

Three further goals are sometimes asked of a product-development framework but are not treated
rigorously here: handling large data sets, transfer learning across manufacturing sites, and a formal
proof of permutation invariance. The methods we use cope with each of them informally —
latent-variable models scale to large 𝑁 , mean-centring and scaling are insensitive to row order in
practice, and a model trained on one site can be re-fit at another — but a careful treatment is left to the
literature.

7.1.6 Data needed for product development

The methods later in this chapter operate on four data tables that fit together. We refer to them by
capital letters: D, F, Z and Y. Each table answers a different question and each one normally comes
from a different source. Setting them up correctly — and getting the alignment between them right —
is most of what makes the rest of the workflow possible.

• D: a property database, one column per candidate building block, one row per measured property.

• F: a recipe table, one row per experiment, columns aligned with D, entries are mass fractions that
sum to 1.

• Z: a process-conditions table, one row per experiment, columns are the conditions used to make
that row’s product.

• Y: a quality-outcomes table, one row per experiment, columns are the KPIs that the customer (or
our specification) cares about.

The matrix D shares columns with F. The matrices F, Z and Y share rows: row 𝑖 of each describes the
same experiment.
180 [Link]

506 Chapter 7. Applications of Process Improvement using Data

 Release 547b16

The property database D

This is the most heterogeneous of the four. Each column of D is a candidate building block — an oil, a
fat, a binder, a filler, a polymer — and the rows are properties: molecular weight, melting point,
viscosity at a given shear rate, surface tension, NIR absorbance at each wavelength, and so on. Some
practitioners prefer the transposed layout (one material per row, properties as columns); either is fine,
as long as you stay consistent.

D is assembled from internal lab measurements, supplier data sheets, public databases, and
computational property estimators. It is worth investing in: a well-populated D is reused across many
product-development campaigns. Collect more rows and more columns than you strictly need for the
current project.

A few practical points:

• Do not break “compound” ingredients into their pure constituents. If you use milk as an
ingredient, store the properties of milk; do not try to split it into water, fat, protein and lactose. The
same goes for any pre-mix or proprietary blend. The columns of D should match the units in which
you actually purchase, weigh and add the materials — the same units that appear in F.

• Properties are usually blocked. Some rows are only meaningful for solids, others only for liquids,
others only for spectroscopic samples. The methods in this chapter handle missing values explicitly,
as long as you mark them as missing rather than filling with zeros.

• The result must not depend on the order of rows or columns. We may group rows — for example,
all NIR wavelengths together — so that block-wise preprocessing is possible, but inside a group the
order is arbitrary. This is one of the desiderata (item M) in the previous section.

• Vector-valued properties belong in :math:`mathbf{D}` as consecutive rows: a particle-size

distribution, a thermogravimetric trace, an NIR spectrum. Such blocks are highly correlated and are
a natural fit for a principal component model (page 421).

The recipe F

Each row of F is one experiment or one product, and each column is a building block, aligned with the
columns of D. The entries in a row are mass fractions and typically sum to 1.

The rows in F need not be products that you sell. Intermediate blends, side experiments and customer
trials all belong, as long as each row has a corresponding outcome in Y. It is also useful to split F into
sub-blocks — a binders block, a fats block, a starches block — so that the model can later assess the
effect of each material family separately.

The process conditions Z

The matrix Z has the same number of rows as F and one column per process condition: temperature,
pressure, mixing speed, residence time, addition order. Discrete settings (a stirrer that is off, low or
high) are one-hot encoded (page 460). Recipe steps that may or may not be applied are stored as 0/1
indicators.

It is tempting to leave Z out when only the recipe varies during a campaign. Resist that temptation.
Conditions that look constant in a campaign — ambient humidity, operator, raw-material lot number,
the calibration date of the analyser — routinely turn out, after the fact, to have driven a result. If they
were never recorded, that diagnosis is impossible. (When such effects are expected and we want to
remove them by design, we can plan the campaign with blocking (page 362).) The correct rule is: store

7.1. Product development and product improvement 507

Process Improvement Using Data

the suspected covariates as well as the obvious controlled variables, even if you do not plan to vary
them.

The quality outcomes Y

Each row of Y is the same experiment as the corresponding row of F and Z. The columns are the key
performance indicators (KPIs): viscosity, melting point, shelf life, taste-panel score, and so on. Vector
outcomes (such as a release-rate curve) can also be stored in Y as consecutive columns; if they are
highly correlated, summarize them with a few principal components first.

The crucial design rule for Y is to capture, at minimum, the same metrics the customer specifies. The
whole point of the methodology is to invert the relationship (D, F, Z) → Y so that, given a target row
ynew , we can prescribe the recipe and conditions (fnew , znew ) that achieve it. If the customer’s target
metrics are not among the columns of Y, the inversion is solving the wrong problem.

As with D, capture provenance for every entry: who measured the value, where, when, in which
units, and with which protocol. Two viscosity numbers measured with different geometries are not the
same measurement.

How the four tables fit together

• D and F share columns. Every ingredient appearing in any recipe must have a property column in
D. A new, unseen ingredient can later be added to D and then used in F, provided its properties lie
within the correlation structure of the existing materials.

• F, Z and Y share rows. Row 𝑖 of each describes the same experiment, so they can be concatenated
horizontally for a multi-block PLS (page 472) analysis. Standard preprocessing (page 448)
(mean-centring, scaling, optional block scaling) applies before the model is built.

• The forward model (F, Z) → Y is augmented by the property information in D, so that the model is
expressed in terms of what the materials do (their physical and chemical properties), not just which
materials were chosen. This is what allows the model to generalize to new ingredients — the
inductive property (item D) of the desiderata.

• Inversion of this model is generally underdetermined: there are typically more inputs than outputs,
so a target ynew corresponds to a region of feasible (fnew , znew ) rather than a single point. The
dimension of that region is the difference between the rank of the input space and the rank of the
output space (the “rank” idea in the previous subsection); the inactive directions are the operating
window described by item J of the desiderata.

7.1.7 References
The four-table organization is drawn from a sequence of papers, several of whose authors I have had
the pleasure of working with over the past 25 years:

• S. Garcia-Munoz, “Two Novel Methods to Analyze the Combined Effect of Multiple Raw-Materials
and Processing Conditions on the Product’s Final Attributes: JRPLS and TPLS.” Chemometrics and
Intelligent Laboratory Systems, 133, 2014, [Link]

• K. Muteki, J. F. MacGregor, and T. Ueda. “Mixture Designs and Models for the Simultaneous
Selection of Ingredients and Their Ratios.” Chemometrics and Intelligent Laboratory Systems, 86,
2007, [Link]

• C.M. Jaeckle and J. F. MacGregor. “Product Design through Multivariate Statistical Analysis of
Process Data.” AIChE Journal, 44, 1998, [Link]

508 Chapter 7. Applications of Process Improvement using Data

 Release 547b16

• E. Tomba, M. Barolo, and S. García-Muñoz. “General Framework for Latent Variable Model
Inversion for the Design and Manufacturing of New Products.” Industrial & Engineering
Chemistry Research, 51, 2012, [Link]

7.1. Product development and product improvement 509

Process Improvement Using Data

510 Chapter 7. Applications of Process Improvement using Data

 INDEX

Symbols B
2^(k-p) design, 346 bar chart, see bar plot, 21
2^k design, 330 bar plot
2fi, see two-factor interaction category axis, 6
3fi, see three-factor interaction value axis, 6
6-sigma, see six-sigma process visualization, 6
batch data analysis
A latent variable modelling, 498
A-optimal design Bernoulli distribution, 50
optimal designs, 376 bin, in a histogram, 42
action limits, 161 binary distribution, 50
adjusted R-squared, 228 biplot, 490
algorithms Bisgaard, Søren, 360
principal component analysis, 450 bivariate plot
aliasing visualization, 12
experiments, 349 block (data set), 417
analysis of variance, 225 blocking, 449
ANOVA, 226 experiments, 363
applications in experimental design, 88
latent variable modelling, 490 bootstrapping, 268
applications of latent variable box plot
models five-number summary, 9
references and readings, 411 interquartile range, 9
aspect ratio outlier, 11
visualization, 18 visualization, 9
assignable cause, 153 whisker, 11
assumptions for Box, George E. P., 322
least squares, 228 breakdown point, 49
autocorrelation, 84, 201, 245 brushing
Durbin-Watson test, 246 latent variable modelling, 461
autoscaled, 450 business intelligence, 171
autoscaling, 429
average, 47 C
average run length, 160 candidate set
axes, data frame, 18 optimal designs, 377
axial points capability of a process, 169, 171
experiments, 370 categorical factor
experiments, 320

511
Process Improvement Using Data

categorical variable, 42 experiments, 375

category axis consumer's risk, see type II error
bar plot, 6 continuous variable, 42
cause and effect, 322 contour plot
cell, in a data table, 15 response surface, 365
center line, 160 contribution plot, 433
center point contribution plots, 459
experiments, 373 control charts, 151
centered process, 169 controlling for another variable, 257
centering, 449 convex optimization
centering, about median, 449 least squares, 220
central composite designs, 370 Cook's D-statistic, 263
Central limit theorem, 53 Cook's distance, see Cook's D-statistic
change one single variable at a time, correlation, 213
see COST approach COST approach
chartjunk, 2 experiments, 328
cheddar cheese example covariance, 210
projection to latent structures covariates, 449
(PLS), 484 Cp, 169
chemometrics, 442 Cpk, 171
chi-squared distribution, 93 critical value, 77
Cleveland, William, 2 cross-validation, 268, 456
coded units cube plot, 331
experiments, 365 cumulative distribution, 61
coefficient of determination, see cumulative sum, see CUSUM
R-squared curvature
coefficient plot, 482 response surface, 370
projection to latent structures CUSUM
(PLS), 482 process monitoring, 162
collinearity, 347, 468
colour D
visualization, 18 D matrix
colour-blindness, 18 product development, 506
common cause, 153 D-optimal design
complementary half-fraction optimal designs, 376
experiments, 350 data density
confidence interval, 66, 77 visualization, 5, 19
for proportions, 94 data frame
for variance, 93 visualization, 18
interpreting, 77 data fusion, 417
known variance, 79 data historian, 160
least squares, 228 data organization
ratio of variances, 94 product development, 506
unknown variance, 80 data table
confounded, 348 visualization, 15
confounding, 87, 363 data visualization, 1
confounding pattern data-ink ratio
experiments, 349 visualization, 8, 19
constant error variance, 228 DBTL cycle, 503
constraints dealiasing

512 Index
 Release 547b16

experiments, 360 exercises

defects per million, 171 experiments, 379
defining relationship latent variable modelling, 462
experiments, 349, 353 least squares, 269
definitive screening design process monitoring, 175
experiments, 377 projection to latent structures
deflation, 454 (PLS), 484
projection to latent structures univariate data, 97
(PLS), 479 visualizing data, 19
degrees of freedom, 48 expected value, 47
in product development, 502 experiments
loss of, 93 aliasing, 349
density, 44 axial points, 370
derivation blocking, 363
least squares, 218 categorical factor, 320
Design of experiments, see experiments center point, 373
design resolution coded units, 365
experiments, 355 complementary half-fraction, 350
Design-Build-Test-Learn cycle confounding pattern, 349
product development, 503 constraints, 375
designed experiments COST approach, 328
projection to latent structures dealiasing, 360
(PLS), 483 defining relationship, 349, 353
deviation variables, 211 definitive screening design, 377
direction vector, see loadings (P matrix), 427 design resolution, 355
discrepancy disturbance, 362
least squares, 262 exercises, 379
distance to model, see DModX factor, 320
disturbance factorial design, 330
experiments, 362 foldover, 360
DModX, 438 fractional factorial, 346
Draper, Norman, 375 full factorial, 330
dummy variable, see integer variables, see generating relationship, 349
indicator variable, 460 half fraction, 346
Durbin-Watson test interaction effect, 333
autocorrelation, 246 main effect, 331
mixture design, 378
E numeric factor, 320
eigenvalue, 450 objective, 320
eigenvalue decomposition, 450 one-factor-at-a-time, 328
eigenvector, 450 optimal design, 376
enterprise resource planning, 171 outcome, 320
error pairing, 326
statistical, 38 Plackett-Burman designs, 360
evolutionary operation (EVOP), 373 projectivity, 361
EVOP, see evolutionary operation (EVOP) qualitative factor, 320
EWMA quarter fraction, 346
process monitoring, 164 randomization, 326
EWMV chart references and readings, 321
process monitoring, 168 replicates, 341

Index 513
Process Improvement Using Data

response, 320 frequency distribution, 42

response surface methods, 365 frequency, relative, 44
run, 320 full factorial
saturated design, 357 experiments, 330
screening designs, 346
sequential experiments, 373 G
three-factor interaction, 338 G-optimal design
two-factor interaction, 333 optimal designs, 376
usage examples, 321 GapMinder, 15
variable, 320 generating relationship
words, 353 experiments, 349
explore-exploit tradeoff, 505 Gompertz function, 502
exponentially weighted moving grand mean, see mean
average, see EWMA grayscale, 18
exponentially weighted moving grid lines, 15
variance, see EWMV chart
extracting value from data
H
latent variable modelling, 413 half fraction
product development, 502 experiments, 346
happenstance data, 323, 418
F hat value, see leverage
F matrix histogram
product development, 506 visualization, 19, 21
F-distribution, 94 histograms, 42
F-statistic homoscedasticity, see constant error variance
least squares, 225 horizontal bar chart
factor visualization, 9
experiments, 320 Hotelling's T-squared statistic, 447
factorial design principal component analysis, 447
experiments, 330 hyperplane, 423
false acceptance rate, see type II error hypothesis test, 82
false alarm, see type I error one-sided, 82
false negative, see type II error
false positive, see type I error
I
false rejection rate, see type I error image unfolding, 498
feedback control, 39, 151, 323 in-control, 153
Few, Stephen, 2, 16, 19 out-of-control, 153
Fisher, R. A., 328 independence, 55, 86
fitted value lack of, 85
least squares, 222 independence in least squares, 245
five uses of data indicator variable, see integer variables, 460
product development, 502 industrial practice
five-number summary process monitoring, 171
box plot, 9 inferential sensor, see soft sensor
foldover inflation
experiments, 360 adjusting for, 5
food texture example inflection point, 58
latent variable modelling, 428 influence
fractional factorial least squares, 263
experiments, 346 inner relation

514 Index
 Release 547b16

projection to latent structures least squares

(PLS), 474 assumptions for, 228
inspection costs, 40 confidence interval, 228
integer variables convex optimization, 220
least squares, 258 derivation, 218
interaction effect discrepancy, 262
experiments, 333 exercises, 269
interaction plot, 333 F-statistic, 225
intercept fitted value, 222
least squares, 218 influence, 263
interpret score plot integer variables, 258
latent variable modelling, 431 intercept, 218
interquartile range leverage, 261
box plot, 9 mean-centering, 253
inverse approach, see model inversion multiple linear regression (MLR),
inverse cumulative distribution, 61 253
objective function, 221
K outliers, 260, 265
key performance indicator, 171, 505 references and readings, 210
KPI, see key performance indicator slope, 218
summary of steps, 250
L testing data, 267
lack-of-fit, 230 training data, 268
Lagrange multiplier, 450 usage examples, 209
latent variable, 419 Lenth, Russel, 373
latent variable modelling, 419 leverage
applications, 490 least squares, 261
batch data analysis, 498 line plot
brushing, 461 visualization, 2
exercises, 462 linear combination, 420
extracting value from data, 413 linking
food texture example, 428 latent variable modelling, 461
interpret score plot, 431 loading plot, 435
linking, 461 loading vector
loadings plot, interpretation of, principal component analysis, 423
435 loadings
missing data, 413 projection to latent structures
number of components, 456 (PLS), 478
preprocessing, 448 loadings (P matrix), 427
principal component analysis, 423 loadings plot, interpretation of
principal component regression latent variable modelling, 435
(PCR), 468 locally weighted regression, see LOESS
process monitoring, 490 location, 47, 49
projection to latent structures location (process monitoring), 153
(PLS), 472 LOESS
references and readings, 411 nonparametric modelling, 264
signal-to-noise ratio, 413 logistic modelling, see logistic regression
spectral data example, 442 logistic regression, 266
troubleshooting, 490 long-term reference set, 82
what is a latent variable, 419 lower control limit, 154, 156

Index 515
Process Improvement Using Data

lower specification limit, 169 np chart

process monitoring, 168
M null hypothesis, 92
MAD, 49 number of components
main effect, 331 latent variable modelling, 456
experiments, 331 numeric factor
marker size experiments, 320
scatter plot, 14
mean, 47 O
mean-centering objective
least squares, 253 experiments, 320
median, 49 objective function
median absolute deviation, see MAD least squares, 221
median, in a box plot, 9 OFAT, see one-factor-at-a-time
missing data off-specification product, 40
latent variable modelling, 413 OLS, see ordinary least squares (OLS)
mixture design one-factor-at-a-time
experiments, 378 experiments, 328
model inversion, 490, 504 operating window, 505, 506
product development, 504 optimal design
monitoring chart assessment, 158 experiments, 376
monitoring charts, 151 optimal designs
moving average, 164 A-optimal design, 376
MSPC, see multivariate statistical process control candidate set, 377
(MSPC) D-optimal design, 376
multi-objective optimization, 505 G-optimal design, 376
multiblock, 417 V-optimal design, 376
multiple linear regression (MLR) ordinary least squares (OLS), 218
least squares, 253 orthogonality, 337
multivariate calibration, 472 out-of-control, 153
multivariate image analysis (MIA), 498 in-control, 153
multivariate outlier, 433 outcome
multivariate statistical process experiments, 320
control (MSPC), 490 outlier, 48, 156, 243
box plot, 11
N outliers
new product development, 490 least squares, 260, 265
NIPALS algorithm, 452 over-control, 188
principal component analysis, 452 over-fitting, 456
projection to latent structures
(PLS), 479 P
non-constant error variance, 244 p chart
nonparametric modelling, 263 process monitoring, 168
LOESS, 264 paired test, 92
scatterplot smoother, 264 pairing
normal distribution experiments, 326
check if, 61 parallel axes
formal definition, 56 visualization, 4
standard, 59 parameter
table for, 94 population, 47

516 Index
 Release 547b16

Pareto plot, see Pareto plot of effects probability distribution, 42

Pareto plot of effects, 341 process capability
partial least squares, see projection to process monitoring, 169
latent structures process capability ratio, 169
path of steepest ascent, see steepest process monitoring
ascent CUSUM, 162
PCA, see principal component analysis EWMA, 164
PCR, see principal component regression (PCR) EWMV chart, 168
percentile, 9 exercises, 175
phase 1 (monitoring charts), 152, 155, 155 industrial practice, 171
phase 2 (monitoring charts), 152 latent variable modelling, 490
pie chart multivariate, 413
visualization, 16, 35 np chart, 168
Plackett-Burman designs p chart, 168
experiments, 360 process capability, 169
PLS, see projection to latent structures R chart, 168
Poisson distribution, 75 references and readings, 150
pooled variance, 90 S chart, 168
pooled variances, 94 Shewhart chart, 153
population usage examples, 149
parameter, 47 process width, 169
power algorithm, see NIPALS algorithm, 452 producer's risk, see type I error
predicted value, see fitted value product development, 501
prediction error interval, 234 data organization, 506
prediction interval, 234, 235 Design-Build-Test-Learn cycle, 503
predictor block extracting value from data, 502
projection to latent structures five uses of data, 502
(PLS), 472 model inversion, 504
preprocessing product improvement, 501
latent variable modelling, 448 subject matter expert, 505
transformations, 448 usage examples, 501
principal component, 423 product improvement, 501
principal component analysis product development, 501
algorithms, 450 product quality, 151
Hotelling's T-squared statistic, projection to latent structures
447 references and readings, 411
latent variable modelling, 423 projection to latent structures (PLS),
loading vector, 423 472
NIPALS algorithm, 452 cheddar cheese example, 484
R-squared, 439 coefficient plot, 482
references and readings, 411 deflation, 479
residual error, 423 designed experiments, 483
residuals, 439 exercises, 484
score, 423 inner relation, 474
using model on new data, 455 latent variable modelling, 472
principal component analysis (PCA), 423 loadings, 478
principal component regression (PCR), NIPALS algorithm, 479
468 predictor block, 472
latent variable modelling, 468 response block, 472
probability, 46 scores (t, u), 475

Index 517
Process Improvement Using Data

weight vector (w, c), 475 replicates

X-space, 474 experiments, 341
Y-space, 474 residual error
projectivity principal component analysis, 423
experiments, 361 residual plots, 242
residuals
Q principal component analysis, 439
q-q plot q-q plot, 243
residuals, 243 resolution III design, 355
Q-squared statistic, 456 resolution IV design, 355
qualitative factor resolution V design, 355
experiments, 320 response
quantile-quantile plot (q-q plot), 61 experiments, 320
quantitative factor, see numeric factor response block
quantitative plots, 1 projection to latent structures
quarter fraction (PLS), 472
experiments, 346 response surface
quartile, 9 contour plot, 365
curvature, 370
R steepest ascent, 365
R chart response surface methods
process monitoring, 168 experiments, 365
r weights (W*), 478 RMSEE, 268
R2 , see R-squared RMSEP, 268
R-squared, 227 robust least squares, 265
principal component analysis, 439 robust monitoring chart, 160
Randomization, 88 robust statistics, 49
randomization robustness
experiments, 326 example, 100
range chart, see R chart Rosling, Hans, 19
rare events, 75 RSM, see response surface methods
rate parameter run
Poisson distribution, 75 experiments, 320
raw material variability, 40
real-world units, see coded units S
references and readings S chart
applications of latent variable process monitoring, 168
models, 411 sample, 46
experiments, 321 saturated design
latent variable modelling, 411 experiments, 357
least squares, 210 scaling, 450
principal component analysis, 411 scatter plot
process monitoring, 150 regression line, 14
projection to latent structures, visualization, 12
411 scatterplot matrix
univariate data, 38 visualization, 19, 24
visualization, 2 scatterplot smoother
regression line nonparametric modelling, 264
scatter plot, 14 score
relative frequency, 44 principal component analysis, 423

518 Index
 Release 547b16

score plot, 431 subject matter expert

scores (t, u) product development, 505
projection to latent structures summary of steps
(PLS), 475 least squares, 250
scree plot, 451 SVD, see singular value decomposition (SVD)
screening designs system failures, 75
experiments, 346 systematic error, 92
sequential experiments
experiments, 373 T
Shewhart chart t-distribution, 69
process monitoring, 153 table for, 94
Shewhart, Walter, 153 T-squared statistic, see Hotelling's
sigmoid function, 502 T-squared statistic
signal-to-noise ratio table, see data table
latent variable modelling, 413 tail, in a histogram, 58
significant difference, see tests for testing data
differences least squares, 267
simplex (mixture design), 378 testing least squares models, 267
SIMPLS, 475 tests for differences, 80
singular value decomposition (SVD), 452 three-factor interaction
six-sigma process, 171 experiments, 338
slope time-series plots
least squares, 218 visualization, 2
small multiples tolerance limits, see specification limits
visualization, 19 training data
soft sensor, 490, 495 least squares, 268
sparklines transformations, 249
visualization, 5 preprocessing, 448
SPE, see squared prediction error (SPE) trimming, 449
special cause, 153 troubleshooting, 323, 413
specification limits, 161 latent variable modelling, 490
spectral data example Tufte, Edward, 2, 8, 13, 15, 18
latent variable modelling, 442 two treatments, 92
spectroscopy, 442 two-factor interaction
spread, 47, 49 experiments, 333
squared prediction error (SPE), 438 type I error, 158
stable operation, see in-control type II error, 158
standard deviation, 48 type II error, 158
standard error, 226, 244 type I error, 158
standard form, 59, 86
standard order, see Yates order
U
standardize a variable, 59 unbiased estimator, 48
statistic, 47 uncentered process capability
statistical process control, 151 process monitoring, 171
statistical tables, 94 unconstrained optimization, 220
steepest ascent uniform distribution, 51
response surface, 365 continuous, 52
Stryjewsk, Lisa, 9 discrete, 51
studentized residuals, 262 unit-variance scaling, see autoscaling, 448
subgroups (monitoring charts), 154 univariate data

Index 519
Process Improvement Using Data

exercises, 97 violin plot, 12

references and readings, 38 visualizing data
usage examples, 37 exercises, 19
unstable operation, see out-of-control
upper control limit, 154, 156 W
upper specification limit, 169 W-star, see r weights (W*)
usage examples warning limits, 161
experiments, 321 weight vector (w, c)
least squares, 209 projection to latent structures
process monitoring, 149 (PLS), 475
product development, 501 weighted least squares
univariate data, 37 WLS, 245
using model on new data Western Electric rules, 160
principal component analysis, 455 what is a latent variable
latent variable modelling, 419
V whisker
V-mask, 164 box plot, 11
V-optimal design Wickham, Hadley, 9
optimal designs, 376 winsorizing, 449
value axis WLS, see weighted least squares
bar plot, 6 weighted least squares, 245
variability, 38 words
cost of, 40 experiments, 353
in raw materials, 40
variable X
experiments, 320 X-block, 472
variance, 47 X-space
violin plot projection to latent structures
visualization, 12 (PLS), 474
visualization
aspect ratio, 18
Y
bar plot, 6 Y matrix
bivariate plot, 12 product development, 506
box plot, 9 Y-block, 472
colour, 18 Y-space
data density, 5, 19 projection to latent structures
data frame, 18 (PLS), 474
data table, 15 Yates order, 330
data-ink ratio, 8, 19
Z
histogram, 19, 21
horizontal bar chart, 9 Z matrix
line plot, 2 product development, 506
parallel axes, 4 z-value, 59
pie chart, 16, 35
references and readings, 2
scatter plot, 12
scatterplot matrix, 19, 24
small multiples, 19
sparklines, 5
time-series plots, 2

520 Index