MSA

MEASUREMENT SYSTEM
ANALYSIS (MSA)

Workshop on Core Tools - MSA 2

 Measurement System Analysis (MSA)

What is It?
An MSA is a statistical tool used to
determine if a measurement system
is capable of precise measurement.

Objective or Purpose
• To determine how much error is in
the measurement due to the
When to Use It measurement process itself.
• On the critical inputs and outputs • Quantifies the variability added by
prior to collecting data for analysis. the measurement system.
• For any new or modified process in • Applicable to attribute data and
order to ensure the quality of the variable data.
data.
IMPORTANT!
Who Should be Involved Measurement System Analysis is
an analysis of the measurement
Everyone that measures and makes process, not an analysis of the
decisions about these measurements people!!
should be involved in the MSA.

Workshop on Core Tools - MSA 3

Attribute and Variable MSA

– Attribute Data Examples:

 Count, Pass/fail, yes/no, red/green/yellow,
timekeeping buckets

– Variable Data Examples:

 Physical measurement (length, width, area, …)
 Physical conditions (temperature, pressure…)
 Physical properties (strength, load, strain…)
 Continuous or non-ending

Unless
Unless approved
approved by
by an
an NCR
NCR SQE,
SQE, attribute
attribute data
data
is
is not
not acceptable
acceptable for
for PPAP
PPAP submission
submission

Workshop on Core Tools - MSA 4

Measurement System Analysis (MSA)

Measuremen
Measuremen
The observed variation in tt
process output System
System
measurements is not Variation
Variation
simply the variation in the
process itself; it is the
variation in the process
Observed
Observed
plus the variation in Variation
Variation
measurement that results
from an inadequate
measurement system.
Process
Process
Variation
Variation

Conducting an MSA reduces the likelihood of

passing a bad part or rejecting a good part

Workshop on Core Tools - MSA 5

Measurement System Analysis (MSA)

Observed Variation

Measuremen
Measuremen
tt
System
System
Variation
Variation

Observed
Observed
Variation
Variation The output of the process
measured by:

Process • Cycle time

Process
Variation • Dimensional data
Variation
• Number of defects
and others

Workshop on Core Tools - MSA 6

Measurement System Analysis (MSA)

Observed Variation
Resolution
Resolution
Precision Repeatabilit
Repeatabilit
Precision
(Variability)
(Variability)
yy
Reproducibility
Reproducibility
Measuremen
Measuremen
tt
System
System
Variation
Variation
Linearity
Linearity
Observed Accuracy
Accuracy
Observed (Central Bias
Bias
(Central
Variation
Variation Location)
Location)
Stability
Stability

Process
Process
Variation
Variation Calibration addresses accuracy

Workshop on Core Tools - MSA 7

Measurement System Analysis (MSA)

Observed Variation
Resolution
Resolution

Precision
Precision Repeatability
Repeatability
(Variability)
(Variability)

Measuremen Reproducibility
Reproducibility
Measuremen
tt
System
System
Variation
Variation
Linearity
Linearity
Accuracy
Accuracy
Observed
Observed (Central Bias
Bias
(Central
Variation
Variation Location)
Location)
Stability
Stability

Process Calibration Addresses Accuracy

Process
Variation
Variation Let’s take a closer look at
Precision
Workshop on Core Tools - MSA 8
Measurement System Analysis (MSA)

Resolution
Error in Resolution
The inability to detect small
changes.

Possible Cause
Wrong measurement device
selected - divisions on scale
not fine enough to detect
changes.

Workshop on Core Tools - MSA 9

Measurement System Analysis (MSA)

Repeatability
Error in Repeatability
The inability to get the same
answer from repeated
measurements made of the
same item under absolutely
identical conditions.

Possible Cause
Lack of standard operating
procedures (SOP), lack of
training, measuring system
variablilty.
Equipment
Variation

Workshop on Core Tools - MSA 10

 Measurement System Analysis (MSA)

Reproducibility
Error in Reproducibility
The inability to get the same
answer from repeated
measurements made under
various conditions from
different inspectors.

Possible Cause
Lack of SOP, lack of training.

Appraiser
Variation

Workshop on Core Tools - MSA 11

 Variable MSA – Gage R&R Study

•Gage R&R is the combined estimate of measurement

system Repeatability and Reproducibility

•Typically, a 3-person study is performed

Each person randomly measures 10 marked parts per trial

Each person can perform up to 3 trials

•There are 3 key indicators

EV or Equipment Variation

AV or Appraiser Variation

Overall % GRR

Workshop on Core Tools - MSA 12

Variable MSA – NCR’s Gage R&R Form

Gage Repeatability and Reproducibility

This spreadsheet is designed for up to three operators, three trials, and ten samples.
Enter data ONLY in gray shaded cells.

Number of operators Upper specification limit

Number of trials Lower specification limit
Number of samples

Data Operator 1 Operator 2 Operator 3

Trial Trial Trial
Sample # 1 2 3 Range 1 2 3 Range 1 2 3 Range
1 0.000 0.000 0.000
2 0.000 0.000 0.000
3 0.000 0.000 0.000
4 0.000 Automatically calculates
0.000 0.000
5 0.000 0.000 0.000
6 0.000 EV, AV, and % GRR!
0.000 0.000
7 0.000 0.000 0.000
8 0.000 0.000 0.000
9 0.000 0.000 0.000
10 0.000 0.000 0.000
Range average 0.000 0.000 0.000
Sample average #DIV/0! #DIV/0! #DIV/0!

Tolerance analysis
Average range 0.000 Repeatability (EV) #N/A #N/A
X-bar range #DIV/0! Reproducibility (AV) #DIV/0! #DIV/0!
Repeatability and Reproducibility (R&R) #N/A #N/A
Control limit for individual ranges #N/A
Note: any ranges beyond this limit may be the result
of assignable causes. Identify and correct. Discard
values and recompute statistics.

Do not modify this table

Trials D4 K1 K2 Included in PPAP Playbook!
2 3.27 4.56 3.65
3 2.58 3.05 2.7

Workshop on Core Tools - MSA 13

 Variable MSA – Gage R&R Steps

Step 1 Step 2 Step 3 Step 4 Step 5 Step 6 Step 7 Step 8 Step 9 Step 10

1. Select 10 items that represent the full range of long-term process variation.
2. Identify the appraisers.
3. If appropriate, calibrate the gage or verify that the last calibration date is
valid.
4. Open the Gage R&R worksheet in the PPAP Playbook to record data.
5. Have each appraiser assess each part 3 times (trials – first in order, second
in reverse order, third random).
6. Input data into the Gage R&R worksheet.
7. Enter the number of operators, trials, samples and specification limits
8. Analyze data in the Gage R&R worksheet.
9. Assess MSA trust level.
10. Take actions for improvement if necessary.

Workshop on Core Tools - MSA 14

Steps 1 and 2: Variable MSA - Gage R&R

Select 10 items that represent

Step 1 the full range of long-term process
variation.

Step 2 Identify the appraisers.

– Should use individuals that actually do the

process being tested.
– Can also include other appraisers
(supervisors, etc.).
– Should have a minimum of 3 appraisers.

Workshop on Core Tools - MSA 15

Steps 3 and 4: Variable MSA – Gage R&R

If appropriate, calibrate the gage

Step 3 or verify that the last calibration

date is valid.

Step 4 Open the Gage R&R worksheet in the

PPAP Playbook to record the data

Workshop on Core Tools - MSA 16

Step 5: Variable MSA – Gage R&R

Have each appraiser assess each item 3

Step 5 times.
– Each appraiser has to work
independently.
– Items should be evaluated in random
order.
– After each appraiser completes the first
evaluation of all items – repeat the
process at least 2 more times.
– Do not let the appraisers see any of the
data during the test !!

Workshop on Core Tools - MSA 17

Steps 6 and 7: Variable MSA – Gage R&R

Step 6 Input data into the Gage R&R

worksheet

Step 7 Enter the number of operators, trials,

samples and specification limits

Workshop on Core Tools - MSA 18

Steps 8 and 9: Variable MSA – Gage R&R

Step 8 Analyze data in the Gage R&R worksheet

Step 9 Assess MSA Trust Level.

% Tolerance*
– Red: > 30% (fail)
30%
– Yellow: 10-30% (marginal)
10%
– Green: < 10% (pass)

Workshop on Core Tools - MSA 19

Step 10: Variable MSA – Gage R&R

If the Measurement System needs improvement:

Step 10
– Brainstorm with the team for improvement solutions.
– Determine best “practical solution” (may require some experimentation).
– Pilot the best solution (PDSA)
– Implement best solution – train employees.
– Re-run the study to verify the improvement.

Workshop on Core Tools - MSA 20

Variable MSA – Gage R&R Example

Problem Statement
• The sulfuric acid concentration in process tank 8 is measured at least
once per day
• Additions/deletions of chemicals and decisions to shut down the
process are dependent on these results.
• Based on current data, we need to do an MSA.

MSA Process
A Gage R&R was conducted in order to validate
the process.

MSA Parameters
» (3) Operators
» (3) Trials
» (10) Samples

Workshop on Core Tools - MSA 21

 Variable MSA – Gage R&R Example

Entered upper and

Entered the number lower specification
of operators, trials, limits
and samples Gage Repeatability and operators
Chose 3 to
Reproducibility
be appraisers
This spreadsheet is designed for up to three operators, three trials, and ten samples. Enter data ONLY in gray shaded cells.

Number of operators 3 Upper specification limit 27

Number of trials 3 Lower specification limit 22
Number of samples 10

Data Operator 1 Operator 2 Operator 3

Trial Trial Trial
Sample # 1 2 3 Range 1 2 3
Had each appraiser
Range 1 2 3 Range
1 25.250 measure each
25.540 25.390 0.290 25.390 25.390 25.390 0.000 25.680 26.330 26.520 0.840
2 25.420 25.540 24.960 0.580 25.540 25.520 25.470 0.070 26.380 26.130 25.960 0.420
3 22.370 sample 3 times
22.370 22.670 0.300 22.220 22.180 22.220 0.040 22.670 22.810 22.670 0.140
4 24.960 25.250 25.110 0.290 25.110 25.680 25.920 0.810 25.680 26.200 25.540 0.660
5 25.540 25.540 25.820 0.280 25.540 25.680 25.680 0.140 25.730 25.960 25.890 0.230
6 24.610 Results calculated
25.110 24.820 0.500 24.960 26.700 25.110 1.740 25.390 25.540 25.960 0.570
7 24.440 24.260 24.110 0.330 25.250 25.540 25.420 0.290 25.250 25.830 25.390 0.580
8 26.100 automatically
26.800 26.100 0.700 25.540 25.540 25.820 0.280 25.110 25.680 25.920 0.810
9 25.390 25.390 25.390 0.000 26.380 26.660 26.220 0.440 25.250 25.540 25.390 0.290
10 25.680 25.820 25.540 0.280 23.900 25.250 25.110 1.350 24.200 23.870 22.960 1.240
Range average 0.355 0.516 0.578
Sample average 25.043 25.211 25.248
Selected 10 samples
to be measured Tolerance analysis
Average range 0.483 Repeatability (EV) 1.4732 29.46%
X-bar range 0.205 Reproducibility (AV) 0.4827 9.65%
Repeatability and Reproducibility (R&R) 1.5502 31.00%
Control limit for individual ranges 1.2461
Note: any ranges beyond this limit may be the result
of assignable causes. Identify and correct. Discard
values and recompute statistics.

Workshop on Core Tools - MSA 22

 Variable MSA – Gage R&R Example

Gage Repeatability and Reproducibility

This spreadsheet is designed for up to three operators, three trials, and ten samples.
Enter data ONLY in gray shaded cells.

Number of operators 3 Upper specification limit 27

Number of trials 3 Lower specification limit 22
Number of samples 10

Data Operator 1 Operator 2 Operator 3

Trial Trial Trial
Sample # 1 2 3 Range 1 2 3 Range 1 2 3 Range
1 25.250 25.540 25.390 0.290 25.390 25.390 25.390 0.000 25.680 26.330 26.520 0.840
2 25.420 25.540 24.960 0.580 25.540 25.520 25.470 0.070 26.380 26.130 25.960 0.420
3 22.370 22.370 22.670 0.300 22.220 22.180 22.220 0.040 22.670 22.810 22.670 0.140
4 24.960 25.250 25.110 0.290 25.110 25.680 25.920 0.810 25.680 26.200 25.540 0.660
5 25.540 25.540 25.820 0.280 25.540 25.680 25.680 0.140 25.730 25.960 25.890 0.230
6 24.610 25.110 24.820 0.500 24.960 26.700 25.110 1.740 25.390 25.540 25.960 0.570
7 24.440 24.260 24.110 0.330 25.250 25.540 25.420 0.290 25.250 25.830 25.390 0.580
8 26.100 26.800 26.100 0.700 25.540 25.540 25.820 0.280 25.110 25.680 25.920 0.810
9 25.390 25.390 25.390 0.000 26.380 26.660 26.220 0.440 25.250 25.540 25.390 0.290
10 25.680 25.820 25.540 0.280 23.900 25.250 25.110 1.350 24.200 23.870 22.960 1.240
Range average 0.355 0.516 0.578
Sample average 25.043 25.211 25.248

Repeatability
Average range = (EV)
Tolerance analysis % Tolerance*
Repeatability
Reproducibility =
= (EV)
0.483
(AV)
Repeatability (EV) 1.4732 29.46%
Reproducibility
X-bar
Equipment
Repeatability
range = (AV) Repeatability and Reproducibility (R&R) 0.4827
0.205
Variation
&
Reproducibility (AV) 9.65%
Equipment
Repeatability
Appraiser Variation
&
Variation 1.5502 31.00%
Appraiser Variation
Reproducibility
Reproducibility = R&R = R&R Note:
Control limit for individual ranges
any ranges beyond this limit may be the
1.2461
result 30%
% Tolerance is > 30%
of assignable causes. Identify and correct. Discard

10%
values and recompute statistics.
MSA fails!

Workshop on Core Tools - MSA 23

Gage R&R Exercise - Setup Instructions

•Divide into teams

•Distribute stars (10 per team), measurement devices (1
per team), and markers (1 per team).
•Number the stars from 1-10.
•Mark the 2 points to be measured on each star (see
diagram on next page)
•Determine and document the measurement process.
•Be sure everyone has a clear understanding of the
process.
•Determine roles.
−(3) inspectors, (1) data recorder, (1) customer

Workshop on Core Tools - MSA 24

Gage R&R Exercise - Dimensional Information

Each star will be measured as shown.

Mark the 2 points

to be measured

Dimension

Workshop on Core Tools - MSA 25

Gage R&R Exercise – Inspection Instructions

1. All inspectors need to wait outside the room when it is not their
turn to evaluate the stars.
2. Open the PPAP Training [Link] file, then select the Gage
R&R worksheet to record the data.
3. Round 1
Have the 1st inspector come in the room and measure all 10 stars
in order. Data collector record the data in the Gage R&R
worksheet.
» Do not give any additional information to the inspector
4. Repeat Step 3 with the 2nd inspector
5. Repeat Step 3 with the 3rd inspector
6. Round 2
Change the inspection to reverse order and repeat.
7. Round 3
Change the inspection to random order and repeat.

Workshop on Core Tools - MSA 26

Gage R&R Exercise - Analysis Instructions

1. Complete the top section of the Gage R&R worksheet

− Enter the number of operators, trials, and samples
− Enter the upper and lower specification limit

2. Assess MSA Trust Level. % Tolerance*

– Red: > 30% (fail)

30%
– Yellow: 10-30% (marginal)
10%
– Green: < 10% (pass)

3. Interpret results - are improvements required?

Workshop on Core Tools - MSA 27

Tips and Lessons Learned

Important: An MSA is an analysis of the process, not an

analysis of the people. If an MSA fails, the process failed.
A Variable MSA provides more analysis capability than an
Attribute MSA. For this and other reasons, always use variable
data if possible.
The involvement of people is the key to success.
Involve the people that actually work the process
Involve the supervision
Involve the suppliers and customers of the process

An MSA primarily addresses precision with limited accuracy

information.

Workshop on Core Tools - MSA 28

 MSA

Reviewer’s Checklist
If the gage/inspection affects quality, then conduct a
Gage R&R
Make sure the study is recent - less than 1 year
Compare the control plan gages against the Gage
R&Rs
If you question that gage, then
−Question the technique and part sampling
−Ask for additional studies

Workshop on Core Tools - MSA 29