Gage R&R

Sep 24-25, 2014

WHY MEASURE?

“The most savage controversies are about those matters as to

which there is no good evidence either way.”
- Bertrand Russell

“Nothing has really happened until it has been recorded.”

- Virginia Woolf

“That which gets measured gets attention.”

- Ian Littman

2
TWO KINDS OF DATA
Continuous Data
✓ Also called variables data or measurement data.
✓ Measured along a continuous scale of possible values:
length, weight, flow, pressure, temperature, voltage, etc.

Discrete Data
✓ Also called attribute, counting, classification or categorical data.
✓ Examples: Defective parts (good/bad), number of defects/part,
leaks (yes/no), etc.

No leak Leak
Count

R G B Y P OK

Color
Cracked

3
CAN WE TRUST THE MEASUREMENT?
The question applies to meters,
scales, calipers, etc.
It also applies to less obvious
situations, such as inspections,
classifications, or ratings.

4
TRUSTING THE MEASUREMENT SYSTEM MEANS:
⚫ Is the measurement system variation small enough, compared to the variation
in the thing being measured, such that we can:
✓ use the measurement results to separate good quality from bad?
✓ use the measurement results to detect changes in process performance?

⚫ Is the measurement system accurate?

Does the average of a large number of repeated measurements come close to the ‘true value?’

⚫ Is the measurement system precise?

Do a large number of repeated measurements cluster together tightly?

⚫ Is the measurement system repeatable?

Does the measurement device return consistent results when used by the same technician to
repeatedly measure the same part?

⚫ Is the measurement system reproducible?

Does the measurement method return consistent results when used by the different technicians
to repeatedly measure the same part?

⚫ Is the measurement system unbiased?

Is the measurement system accurate, or does it have constant offset?

⚫ Is the measurement system stable over its useable range?

Does the measurement system’s accuracy and precision remain constant across the full range
of the gauge?

⚫ Is the measurement system stable over time?

Does the measurement system’s accuracy and precision remain constant over time?

5
MEASUREMENT SYSTEM VARIATION
KC
A KC of this part is it’s length.
2. Equipment

1.217
Here is a measurement system:
1. Method
How to measure a Part
First, take the part and hold it in your left hand. Then pick up
the gauge in your right hand and use the thumbwheel to roll
the traveling anvil back sufficiently.
4. Something to
be measured
Place the part between the anvils, holding one end of the part

Suppose Technician 1 against the stationary anvil. Use the thumbwheel to bring the
traveling anvil toward the part until it just touches the part.
Read the resulting length from the digital scale on the gauge, 3. People
measures the same part
and record the result in the data acquisition spreadsheet.
If you are really reading this, then you have way too much
time on your hands. My favorite flavor of ice cream is
chocolate. What’s yours?

multiple times…

Tech 1: Tech 2:
1.217 … and Technician 2 1.210
1.209 does the same thing… 1.211
1.210 1.204
1.220 1.218
1.119 Operator 1 1.210 Operator 2
etc. Measurement System etc. Measurement System
Variability Variability
6
WHY DO WE CHECK THE MEASUREMENT SYSTEM?

Operator 1 Operator 2
Measurement System Measurement System
Variability Variability

Engineering
Does each operator add specification
Is each operator getting the
about the same amount of same result, on average? How much of the
variability to the The true length of
measurement system? Are the operators getting the specification is used up by
that part
correct result, on average? the measurement system?

7
GAUGE RESOLUTION
The resolution of this gauge is 0.05 units.

0 1 1.5 2

GENERAL GUIDELINE: The resolution of the gauge should be at least one

order of magnitude smaller than the total tolerance of the part being measured.

8
HOW TO CONDUCT AN MSA* FOR MEASURED KCs
1) Identify the specification limits for the Key Characteristic of the part.
2) Select between 5 and 20 samples of the part being measured. Larger samples are
preferred. Use smaller samples only if the measurement is costly or very time-consuming.
These samples should span the range of the KC feature being measured, as produced by the
process.
PLANNING 3) Number the samples such that the operators making the measurement will not know which
part they are measuring.
4) Identify the gauge normally used to measure the key characteristic of the part.
5) Check the resolution of the gauge.
6) Check the calibration of the gauge.
7) Identify at least two technicians or operators whose job it is to measure these parts during
normal operations.
8) Plan to have the operators make the measurements using the gauge in the normal work
environment.

9) Randomly select a part and ask Operator 1 to make the measurement in the usual method.
Be sure the other operators are not present.
EXECUTION 10) Continue, until Operator 1 has measured, in a random order, all 10 parts one time.
11) Repeat steps (9) and (10) for Operator 2 (and also Operator 3, if a third operator is used in
the study).
12) Repeat steps (9), (10), and (11) so that each operator has measured each part at least twice.

13) Enter the data into an appropriate analysis program.

ANALYSIS 14) Assess the results.

* Also referred to as Gauge Repeatability and Reproducibility studies, or GR&R studies.

9
MSA GR&R WORKSHEET (Long form)

10
TYPICAL MSA DATA COLLECTION
Measurement data

1.250 ± 0.010

10 samples
3 technicians
2 measurements
10 x 3 x 2 = 60 observations

11
WHAT THE DATA TELLS US

3 sources of variation: part-to-part, reproducibility, and repeatability.

Ranges can be used to assess the amount of variation from each
of these sources.

12
GRAPHICAL GR&R RESULTS

13
ANALYTICAL GR&R RESULTS

Three critical numbers:

⚫ % Study Variation

⚫ % Tolerance

⚫ Distinct categories

14
% STUDY VARIATION
⚫ Compares the measurement system variation
to the total variation observed within the
sample parts.
⚫ Indicates how much of the total variation
observed in the study is consumed by the
measurement system variation.
⚫ If the parts used in the study span the full
range of the process’s output, then the study Meas. Sys.
variability
variation is an estimate of the total process
(6Meas)
variation.
⚫ Used to decide if the measurement system is Process variability
good enough to detect changes in the (6Total)
process.
⚫ If %SV is too large (>20%), then changes in
the process (i.e. deliberate adjustments,
process improvements, process drift or shift,
etc.) may not be detectable.

sMeasure 0.001034 "

% SV = = = 0.1145, or 11.45%
sTotal 0.009027 "

15
% TOLERANCE
⚫ Compares the measurement system variation LSL USL
to the engineering tolerance.

⚫ Also called the Precision-to-tolerance (P/T)

ratio.

⚫ Indicates how much of the engineering

tolerance is consumed by variation in the
measurement system.
Meas. Sys.
variability
⚫ Used to decide if the measurement system is
(6Meas)
good enough to assess the quality of the parts,
compared to the specifications. Engineering tolerance

⚫ If the % tolerance is too large (>20%), then

assessing the quality of the parts may be
difficult.

6  sMeasure 6  0.001034 "

%Tolerance = = = 0.3101, or 31.01%
Tolerance 0.0200 "

16
NUMBER OF DISTINCT CATEGORIES (ndc)

sParts 0.008968 " ⚫ This is a measure of discrimination,

ndc = 2= 1.41 = 12.269, or 12 and represents the number of
sMeasure 0.001034 " distinct groups of parts within the
range the sample part variation that
can be distinguished from one
6  sParts another.
width of each category = = 0.0045 "
ndc ⚫ So if the difference between two
measured parts is greater than
0.0045 inches, we can claim the
parts to be truly different from one
another.
⚫ If the difference between two
measured parts is less than 0.0045
inches, then we cannot tell if this is
a true part-to-part difference, or due
to the noise in the measurement
system.
( 1 )( 2 )( 3 )( 4 )( 5 )( 6 )( 7 )( 8 )( 9 )( 10 )( 11 )( 12 )
⚫ Look for the number of distinct
Part-to-part variability (6Parts)
categories to be 5 or greater.
divided into 12 distinct categories

17
NUMBER OF DISTINCT CATEGORIES
Example: ndc= 10 (good discrimination)
True part – to – part variation, and a sample of 10 measured parts

ndc = 10

Discrimination:
1 3 7 9
4 6
2 5 8 10

⚫ Number of distinct categories = 10, and there is good

discrimination between parts.
⚫ Part 6 (for example) is truly different from any of the other nine
parts, since there are no red cells to the right or above sample 6
in the Discrimination Plot.
⚫ Part 4 (for example) cannot be claimed as different from either
parts 3 or 5, as indicated by the red cells.

18
NUMBER OF DISTINCT CATEGORIES
Example: ndc= 5 (adequate discrimination)
True part – to – part variation, and a sample of 10 measured parts

ndc = 5

Discrimination:
1 3 7
2 6 9
4 8 10
5

⚫ Number of distinct categories = 5, and there is still adequate

discrimination between parts.
⚫ No parts can be claimed to be uniquely different from all of the
others. Worst case is that a part cannot be resolved from its 4
nearest neighbors.
⚫ Part 4 (for example) cannot be claimed to be different from parts
2, 3, 5 & 6, but is different from parts 1, 7, 8, 9, & 10.

19
NUMBER OF DISTINCT CATEGORIES
Example: ndc= 2 (poor discrimination)
True part – to – part variation, and a sample of 10 measured parts

ndc = 2

Discrimination:
1 7
2 8
3 9
4 10
5
6

⚫ Number of distinct categories = 2. Discrimination between parts is

very poor.
⚫ At best, this measurement system can reliably discriminate
between parts that are at the extremes of the part-to-part
distribution.
⚫ Parts in the middle of the range (see part 6, for example) cannot
be claimed as different from any of the other parts (no
discrimination).
20
WHAT MAKES A “GOOD” GAUGE?
LSL USL

Meas. Sys.
variability
(6GR&R)

Process
variability
(6Total)

Engineering Tolerance

< 10% Excellent gauge performance

% Tol 10% to 20% Adequate gauge performance
and =
20% to 30% Marginal gauge performance, may be unacceptable
% SV
> 30% Poor gauge performance, generally unacceptable

21
USABLE TOLERANCE--SPECIAL CASE

To be sure the part is good,

Measurement system could only accept parts within the Measurement system could
report a measured value usable tolerance report a measured value
anywhere in this range anywhere in this range

Usable
tolerance
Meas. system Meas. system
variability variability

A true bad part just LSLUseable USLUsable A true bad part just
below the LSL above the USL
Stated Tolerance
LSL USL

LSLUsable = LSL + 3sMeasure

USLUsable = USL − 3sMeasure

22
GOOD GAUGE, CENTERED & CAPABLE PROCESS

Process:
Meas. sys. variation Cpk = Cp = 1.33
%Tolerance = 20%

Usable tolerance
LSLUsable USLUsable
Stated Tolerance
LSL USL

A centered and capable process fits within the usable tolerance of a

measurement system with 20% GR&R.

23
GOOD PROCESS, GOOD GAGE: LOW ERROR RATE

Process capability: Cp = 1.33.

Measurement system is good (but

not perfect). GR&R = 20%.

If process is centered, risk of

measurement error* is virtually 0%.

As process average moves away

from target, risk of measurement
error* grows to 8.6%.

Mistake in Mistake in Mistake in

measurement* measurement* measurement*
≈0.00% 1.1% 8.6%

*Measurement error: Good part measured as bad, or bad part measured as good.

24
GOOD PROCESS, BAD GAGE: HIGH ERROR RATE

Process capability: Cp = 1.33.

Measurement system is very poor:

GR&R = 120%.

If process is centered, risk of

measurement error* is about 3%.

As process average moves away

from target, risk of measurement
error* grows to 33%.

Mistake in Mistake in
measurement* measurement* Mistake in
3.2% 14.0% measurement*
32.7%

*Measurement error: Good part measured as bad, or bad part measured as good.

25
MSA GR&R WORKSHEET (Short form)
⚫ Used for a quick
assessment of the
measurement system.
⚫ Repeatability and
reproducibility cannot
be separated.
⚫ If the overall result is
not satisfactory, we
cannot tell if the
problem is in the
gauge itself or the
people using the
gauge.

26
MSA IN MINITAB: SETUP
 Typical Minitab data entry for GR&R study

 Set up for GR&R analysis

Minitab 16 path:
Stat > Quality Tools > Gage Study > Gage R&R Study
(Crossed)…

Enter engineering tolerance information under Options

button.

27
MSA IN MINITAB: RESULTS

⚫ % Study Variation

⚫ % Tolerance

⚫ Distinct categories

28
IDEAS TO IMPROVE MEASUREMENT GAUGES

⚫ Add stability
Fixtures and jigs to hold the parts being measured and/or the gauge being used.

⚫ Minimize the human element

Can the operator’s role be reduced? No manual manipulation of the parts or
gauges. Load the part and push a button.

⚫ Reorganize the work area

Ergonomics. Lighting. Noise and distractions.

⚫ Standard work. Operator certification. Audits.

⚫ Automation.
Can the measurement be made automatically, by the process itself?

⚫ Gauge tune-ups, calibration, and maintenance.

⚫ Report the average of multiple readings.
Can the central limit theorem be made to work in our favor?

29
MSA FOR GO/NO-GO INSPECTED KCS

⚫ Itis just as important to check inspection systems where the

decision is simply a classification of something as good or bad.

⚫ The questions we are interested in are:

● If the same assessor inspects the same part twice, will he agree with himself
and get the same result both times?
● If two assessors independently inspect the same part, will they both agree?
● If an assessor inspects a standard judged by an “expert” as either good or
bad, will he come to the same conclusion?

⚫ The tool for checking this kind of measurement system is an

Agreement-between-Assessors (AbA) study.

30
AGREEMENT-BETWEEN-ASSESSORS SET-UP
⚫ Variables:
– Assessors: Usually 2 inspectors.
– Expert: 1 person with expert knowledge. (Optional) Attribute measures:
– Parts: Typically 30 - 50 parts. – Attribute measurements often rely on
subjective judgement. (i.e. Inspection
⚫ Trials: against a reference standard.)
– Each assessor assesses each part at least twice.
– Parts are presented randomly, and “blindly.”

⚫ Part Selection:
– Some parts in the AbA study should clearly be
good.
– Some parts should obviously be bad.
– The majority of the parts (about 2/3 of them)
should be “borderline.” – Pass/fail, go/no-go gauges make the
decision-making process easier.
⚫ Before conducting the AbA study:
– Have a clear definition of what is “good” and
what is “bad.”
– Once the samples are selected, have them
classified by an “expert.” (Optional.)

31
CONDUCTING AN AbA FOR INSPECTION SYSTEMS
To conduct An Agreement-between-Assessors (AbA) study
1) Identify the Key Characteristic of the part.
2) Clearly define “good” and “bad.”
3) Select 30 to 50 representative samples of the part being inspected.
4) Number them as 1 to N, such that the operators making the measurement will not know which
part they are measuring.
PLANNING
5) Identify the go/no-go gauge (if there is one) normally used to inspect the key characteristic of
the part. Check the calibration of the gauge.
6) (Optional) Have an “expert” inspect each part and to determine which are truly good and bad.
7) Identify at least two technicians or operators whose job it is to inspect these parts during
normal operations.
8) Plan to have the operators perform the inspections in the normal work environment.

9) Randomly select a part and ask Operator 1 to make the inspection in the usual method. Be
sure the other operators are not present.
EXECUTION 10) Continue, until Operator 1 has inspected, in a random order, all of the parts one time.
11) Repeat steps (9) and (10) for Operator 2.
12) Repeat steps (9), (10), and (11) so that each operator has measured each part twice.

13) Enter the data into an appropriate analysis program.

ANALYSIS 14) Assess the results.

32
ATTRIBUTE AbA WORKSHEET

33
Too light
OK
Too dark
TYPICAL AbA DATA COLLECTION
⚫ 30 parts
⚫ 2 Assessors
⚫ Each assessor inspects
each part twice.
⚫ 1 Expert “gold standard.”
⚫ If inspected part is good,
enter a ‘1’. If it is bad,
enter a ‘0’.

34
TYPICAL AbA RESULTS

35
AbA ATTRIBUTE RUN PLOTS
1 1
Perfect agreement: Assessors agree with themselves, the two
assessors agree with each other, and also with the expert.
0 0
A1 A2 E B1 B2 A1 A2 E B1 B2

1 1
Expert misalignment: Assessors agree with each other, but not
with the expert.
0 0
A1 A2 E B1 B2 A1 A2 E B1 B2

1 1 Between assessor misalignment: Assessors disagree with each

other.
0 0
A1 A2 E B1 B2 A1 A2 E B1 B2

1 1
Within assessor misalignment: Assessor disagrees with him or
herself.
0 0
A1 A2 E B1 B2 A1 A2 E B1 B2

Within, Between, and Expert

percent agreement should all be
high. (in the high 90’s).

Otherwise, work to improve the

attribute inspection system.

36
SUMMARY OF AbA RESULTS

⚫ Individual performance:
– Assessor 1 was repeatable within himself about 83% of the time. (Should be better)
– Assessor 2 was repeatable within himself about 63% of the time. (Needs improvement.)
– Neither assessor performed well against the Expert, but B was especially poor.

⚫ The assessors only agreed with each other only about 47% of the time.

37
ATTRIBUTE MSA IN MINITAB: SETUP
Typical Minitab data entry for AbA study Set up for AbA analysis
Minitab 16 path to create Gage Run Chart:
Stat > Quality Tools > Gage Study > Gage Run Chart…

Minitab 16 path to create Gage Run Chart:

Stat > Quality Tools > Attribute Agreement Analysis…

38
ATTRIBUTE MSA IN MINITAB: RESULTS

39
AbA IMPROVEMENT CONSIDERATIONS
⚫ Sense Multipliers
Devices to improve human senses.

⚫ Masks / Templates
Block out unimportant information.

⚫ Checklists

⚫ Jigs and fixtures

⚫ Automation

⚫ Reorganization of Work Area

⚫ Visual Aids

⚫ Go /no-go tools and gauges

40
KEY LEARNING POINTS

Trustworthy measurement is critical to good process

certification.

We are concerned with both measured features and attribute

features, and MSA can be done for both.

Measurement system errors can come from the gauge

hardware (repeatability) or the people using it (reproducibility).

41
CASE STUDY: GAGE R&R
Perform Gage R&R for your assigned KC– Review the
results

In your team:

• Analyze the variable gage R&R

• Evaluate whether the measurement system is acceptable in
each case
• Document your Gage R&R results on slides
• Use flip chart to summarize results
• Prepare to review results with the class

TIME: 60 minutes

42