Statistical Methods for Medical Devices
Statistical Methods for Medical Devices
0 Purpose
This procedure defines the methods for.
1.1 To identify statistical methods used for sampling, interpretation and analysis of
data in order to draw valid statistically based conclusions supporting business and
engineering decisions.
2.0 Scope
2.1 This procedure applies to all functional areas and provides reference and guidance
for using statistical methods, sampling plans for use during development and
manufacturing and supporting criteria with the required statistical justification.
Other approaches may be used with the appropriate statistical justification. Specific
applications include but are not limited to the Product Development Process,
Clinical studies, Design Verification and Validation, Performance Validation and
Process Improvement and Optimization Studies. Incoming Inspection, In-process
inspection, Final inspection or any other test where data driven product or process
evaluation and acceptance is the objective. In practice, the final choices made for
each application are to be documented, reviewed and approved per quality system
requirements. Lot inspection sampling procedures are addressed separately in
800403001, WI Sampling Plan.
4.0 Definitions
4.1 Descriptive Statistics: The term given to the analysis of data that helps describe,
show or summarize data in a meaningful way such that, for example, patterns
might emerge from the data. Examples of such measures are mean, mode,
median, standard deviation, range.
4.2 Inferential Statistics: Modeling techniques that allow us to use sampling to draw
conclusions about the populations from which the samples are drawn, e.g.
capability studies, hypothesis testing.
4.3 Type I Error (α): Rejecting a hypothesis when it is true. Its probability is called the
level of significance or producer’s risk and denoted by α.
4.4 Type II Error (β): Accepting a hypothesis when it is false. Its probability is called
the consumer’s risk and denoted by β.
4.5 Power: Rejecting a hypothesis when it is false. Its probability is denoted by (1- β).
4.6 Normality or Normal Distribution: The normal distribution is a commonly occurring
distribution that appears as a symmetrical bell-shaped curve. Tables of the normal
distribution are commonly available. Numerous statistical procedures have been
developed assuming the data being analyzed follows the normal distribution.
Many of these procedures are robust to this assumption and work well even for
non-normal data. However, variables sampling plans based on the normal
distribution and related procedures (normal tolerance intervals and confidence
intervals for Ppk) are quite sensitive to departures from this assumption and require
verification that the data fits or is well approximated by the normal distribution.
4.7 Process Capability Indices: Process capability indices compare the performance of
an in-control (stable) process to the specification limits. Ppk and Cpk are the most
commonly used process capability indices. The indices are calculated by forming
the ratio of the spread between the process specifications (the specification
"width") to the spread of the process values, as measured by 6 process standard
deviation units (the process "width"). Ppk is based on the standard deviation
across all subgroups whereas Cpk is based on the standard deviation within each
of the subgroups.
4.8 Operating Characteristic (OC) Curve: The protection provided by a sampling plan
is summarized by its operating characteristic curve or OC curve. The OC curve for
the single sampling plan for proportion conforming with sample size n=300 and
accept number a=0 is shown below:
OC Curve - n=300, a=0
Probability Accept 1
0.8
0.6
0.4
0.2
0
0 0.25 0.5 0.75 1
Percent Nonconforming
The bottom axis represents possible values for the quality level. The quality level
can be percent nonconforming units, nonconformities per quantity, average,
standard deviation, difference between two averages, ratio of two standard
deviations, etc. In this example it is in terms of percent nonconforming. The left axis
gives the corresponding probability that the sampling plan will accept or pass such
a lot. For example, a 0.5% nonconforming lot has a 20% chance of passing
(probability of 0.2).
4.9 Acceptable Quality Level (AQL): The AQL of a sampling plan is a level of quality
that is routinely passed by a sampling plan. Products or processes at or better
than the AQL are passed at least 95% of the time. Typically, the quality level can
be in terms of percent conforming or nonconformities per quantity. The AQL
describes the risk associated with failing a good product or process. However,
even more important is the Rejectable Quality Level (RQL) defined below. The
RQL describes the risk associated with passing a bad product or process.
Sampling plans should be selected considering both risks.
The AQL is determined from the OC curve. In the example below the quality level
on the bottom axis is in terms of percent nonconforming. The AQL is the quality
level or percent nonconforming that corresponds to a 95% chance of passing on the
left axis. As shown below, the AQL of the single sampling plan n=300 and
acceptance criterion, a=0 is 0.018% nonconforming.
OC Curve - n=300, a=0
1
Probability Accept
0.8
0.6
0.4
0.2
0
0 0.25 0.5 0.75 1
Percent Nonconforming
AQL = 0.018%
4.10 Rejectable Quality Level (RQL): The RQL of a sampling plan is a level of quality
that is routinely failed by a sampling plan. Products or processes at or worse than
the RQL fail most of the time. The quality level can be percent nonconforming
units, nonconformities per quantity, average, standard deviation, difference
between two averages, ratio of two standard deviations, etc. Two RQLs are
commonly used, which are denoted RQL.10 and RQL.05. They are determined from
the OC curve.
In the example below, the quality level on the bottom axis is in terms of percent
nonconforming. RQL.10 is that quality level or percent nonconforming on the bottom
axis that corresponds to a 10% chance of passing. As shown below, the RQL.10 of
the single sampling plan n=300 and a=0 is 0.76% nonconforming.
RQL.05 is that quality level or percent nonconforming on the bottom axis that
corresponds to a 5% chance of passing. As shown above, the RQL.05 of the single
sampling plan n=300 and a=0 is 1% nonconforming.
0.8
0.6
0.4
0.2
0
0 0.25 0.5 0.75 1
Percent Nonconforming
RQL.05 = 1%
RQL.10 = 0.76%
Associated with the RQLs are confidence statements that can be made. Passing
results with 90% confidence that the quality level is better than the RQL.10.
Demonstrating that a specified quality level is met with 90% confidence requires the
use of a sampling plan whose RQL.10 is equal to the specified quality level. Passing
also results with 95% confidence that the quality level is better than the RQL.05.
Demonstrating that a specified quality level is met with 95% confidence requires the
use of a sampling plan whose RQL.05 is equal to the specified quality level. The RQL
is also referred to as the Lot Tolerance Percent Defective (LTPD).
4.11 Representative Sample: A representative sample of a lot means a sample that is
spread out across the lot to maximize the chance of finding a run or cluster of
nonconforming units. A random sample is the benchmark method but a stratified
or periodic sample that spreads the samples equally across the lot is generally
preferred. A representative sample relative to a validation study also requires that
the lots produced be manufactured under what the regulations call “anticipated
conditions” so these lots are representative of future production.
4.12 Nonconformance: A nonconformance is a unit of production which fails acceptance
criteria.
4.13 Critical-to-Quality Characteristic (CTQ): CTQs are the key measurable
characteristics of a product or process whose specification limits and quality levels
must be met in order to satisfy the customer and stakeholder(s). They align
improvement or design efforts with customer and stakeholder requirements.
4.14 Confidence/Reliability Statement: This statement is of the form that with some
chosen % confidence (e.g. C = 95%), more than R% of units conform to
requirements. R is chosen based on risk assessment.
4.15 Statistical Process Control (SPC): A method of monitoring, controlling and, ideally,
improving a process through statistical analysis. Its four basic steps include
measuring the process, eliminating variances in the process to make it consistent,
monitoring the process, and improving the process to its best target value.
4.16 Common Cause or Random Variation: The common cause variation arises from a
multitude of small factors that invariably affect any process and will conform to a
normal distribution, or a distribution that is closely related to the normal distribution.
4.17 Assignable Cause: Also known as special cause variation, which arises from
specific factors that have an identifiable effect on the process.
4.18 Inspection: A conformity evaluation by observation and judgment accompanied as
appropriate by measurement, testing, or gauging.
4.19 Sub-System: Includes any type of System sub-elements including disposables, but
does not include sub-system components.
5.0 Responsibilities
5.1 Quality Engineering: Quality Engineering is responsible for establishing
appropriate statistical techniques that assure adequate levels of quality and
confirm conformance to stated requirements. Appropriate selection and use of
statistical techniques is reviewed and assured by qualified individuals.
5.2 Each department with specified or implied responsibility is required to use
statistical methods which conform to this procedure.
[Link] Points outside the Control Limits: Any points that fall outside the
Upper Control Limit (UCL) or Lower Control Limit (LCL) indicate
that some assignable cause variation may be present. These data
points should be investigated for root cause and corrected.
[Link] Shifts: Eight or more consecutive points on one side of the
centerline indicate a special cause has influenced the process.
Points on the centerline are not included; they neither break the
string nor add to it.
[Link] Trends: Six consecutive jumps in the same direction indicate
that a special cause is acting on the process to cause a trend. Flat
line segments are not included either to break a trend or to count
towards it.
[Link] Patterns: A pattern that recurs eight or more times in a row is a
good indication to look for a special cause.
Capability of a process output is considered only after the process has been
established as “In Statistical Control”. That is to say all assignable or
special causes of variation have been identified and eliminated.
.
7.5 Recommended Statistical Methods:
Chart B – Statistical Analysis Process Flow
No
Improve
Measurement
Solution or
Use alternate
Measurement
Equipment
Yes
Implement
Corrective
Actions and
Collect data
again
Yes
ANALYZE
DATA
For Exceptional
CAPABILITY
Situations consult
STUDIES
with QE
ATTRIBUTE VARIABLES
DATA DATA
ANALYSIS ANALYSIS
MEASUREMENT MEASUREMENT
HYPOTHESIS STATISTICAL HYPOTHESIS STATISTICAL
SYSTEMS DESIGN OF SYSTEMS DESIGN OF
TESTING FOR ANALYSIS PROCESS TESTING FOR ANALYSIS PROCESS
EXPERIMENTS EXPERIMENTS
ATTRIBUTE (GAGE R & R CONTROL VARIABLE (GAGE R & R CONTROL
(DOE) (DOE)
DATA STUDY) (SPC) DATA STUDY) (SPC)
Chart D – Determine Sampling Method to be Used
Sampling
Methods
Based on For
Based on For
Confidence levels Systems
Inspection Disposable
and Precision and Sub-
Level, Lot Size Products
(Unacceptable Systems
and AQL
difference)
For Both
Variable and Based on Based on
Variable Attribute Used for
Attribute data, Confidence/ Confidence/
Sampling: Use Sampling: Use Equivalency
Tables Based on Reliability only Reliability
ANSI/ASQ ANSI/ASQ Testing, i.e.
Confidence/ (Variable (Attribute
Z1.9 Z1.4 Hypothesis Testing
Reliability and Sampling only) Sampling only)
AQL Levels
This section shall be used as a model for when sampling considerations are applicable
within the scope of design assessments such as Design Verification or Validation. The
following flowcharts provide a process for rationalizing where statistical methods are
appropriate and where other analytical approaches may be used. When statistical
methods are applied, sampling decisions must be made.
A first step is to identify the level at which requirements will be verified.
System/Sub-System Verification Planning
Verification Planning
for each Requirement:
No
Develop
Document
Sub-System
Level Plan
Done
Once it is known at which level requirements will be verified, Sample Sizes and
Acceptance Criteria can be developed for Verification and Validation Test Plans as
shown in the chart following.
Verification and Validation Sample Size Planning
System/Sub-System
Design Assessment
Yes
Document sample
size rationale in No No
Test Plan No
Yes
Will 100%
Choose another
inspection be No
Method Is Test
ongoing? Is there a validated Performance Create Model/ Method
No
Model/Simulation for this requirement? Simulation Validation
Yes required?
Document Rationale
in Test Plan
Note 1: The Confidence/Reliability criteria in Table I are then used to select the appropriate table
for sample size selection from the Variable and Attribute tables provided in the Appendix I. These
tables provide equivalent optional sampling plans for each C/R level and RQL. Known non-
conformance rates must be used to determine AQL levels at or above the non-conformance rate,
since the AQL for a sampling plan is a level of quality that is passed by that sampling plan 95% of
the time. For attribute sampling plans used in manufacturing sampling, the tables provide additional
options of single and double sampling, the latter providing an additional step to accept the lot if the
first sample is rejected. Single sampling plans do not present that option and therefore start with a
higher sample size.
For design validation and process validation, AQL levels are chosen based on the Severity level of
the characteristic being evaluated. The AQL chosen will be lower if Severity level is higher.
Note 2: For S-4 and S-5 severity process outputs, attribute data should only be used when it is not
feasible to obtain variable data.
Additional Notes:
a. Where FDA requirements may differ from the above table; FDA requirements take
precedence, e.g., Balloon Fatigue (90% reliability/95% confidence) and Rated Burst Pressure
(99.9% reliability/95% confidence).
b. For process validations involving subassembly components being supplied by an OEM,
the defect severity must be determined/provided by the OEM customer. It is recognized that
external customers may not follow SOP, Risk Management Process 102-0100.03 and will
need to determine component failure severity or reliability and confidence requirements by
other means.
8.0 Documentation
8.1 Retain records per SOP, Control of Quality Records, 102-0100.04.
9.0 Training
9.1 Review and complete the applicable training requirements per SOP,
Employee Training Program QD000117145 before performing this task.
10.0 Appendices
10.1 Appendix I: Contains Attribute and Variable Sampling Tables for different
Confidence/Reliability Levels
10.2 Appendix II: Contains Two-sided and One-Sided Tolerance Limit Factors
for a Normal distribution for different Confidence/Reliability Levels:
10.2.1 Table A: Two-sided Statistical Tolerance Interval Factors for a
Normal Distribution
10.2.2 Table B: One-sided Statistical Tolerance Bound Factors for a Normal
Distribution
.
APPENDIX I
90 / 80 Attribute
RQL0.10 = 20%
95 / 80 Attribute
RQL0.05 = 20%
Parameters AQL
n=15, Ppk=0.50 1.7% corresponding to Ppk=0.70
n=20, Ppk=0.46 3.0% corresponding to Ppk=0.63
n=30, Ppk=0.42 4.9% corresponding to Ppk=0.55
n=40, Ppk=0.40 6.2% corresponding to Ppk=0.51
n=50, Ppk=0.39 7.1% corresponding to Ppk=0.49
n=60, Ppk=0.38 7.9% corresponding to Ppk=0.47
n=80, Ppk=0.36 9.5% corresponding to Ppk=0.44
n=100, Ppk=0.35 10.5% corresponding to Ppk=0.42
Parameters AQL
n=15, Ppk=0.51, Pp=0.60 1.6% corresponding to Ppk=0.72
n=20, Ppk=0.48, Pp=0.58 2.5% corresponding to Ppk=0.65
n=30, Ppk=0.43, Pp=0.54 4.5% corresponding to Ppk=0.56
n=40, Ppk=0.41, Pp=0.53 5.8% corresponding to Ppk=0.52
n=50, Ppk=0.39, Pp=0.51 7.1% corresponding to Ppk=0.49
n=60, Ppk=0.38, Pp=0.51 7.9% corresponding to Ppk=0.47
n=80, Ppk=0.37, Pp=0.50 9.0% corresponding to Ppk=0.45
n=100, Ppk=0.36, Pp=0.49 9.9% corresponding to Ppk=0.43
90 / 90 Attribute
RQL0.10 = 10%
95 / 90 Attribute
RQL0.05 = 10%
Parameters AQL
n=15, Ppk=0.70 0.22% corresponding to Ppk=0.95
n=20, Ppk=0.65 0.51% corresponding to Ppk=0.86
n=30, Ppk=0.60 1.1% corresponding to Ppk=0.76
n=40, Ppk=0.57 1.7% corresponding to Ppk=0.70
n=50, Ppk=0.55 2.3% corresponding to Ppk=0.67
n=60, Ppk=0.54 2.6% corresponding to Ppk=0.65
n=80, Ppk=0.52 3.4% corresponding to Ppk=0.61
n=100, Ppk=0.51 3.8% corresponding to Ppk=0.59
Parameters AQL
n=15, Ppk=0.71, Pp=0.77 0.19% corresponding to Ppk=0.96
n=20, Ppk=0.66, Pp=0.73 0.46% corresponding to Ppk=0.87
n=30, Ppk=0.61, Pp=0.70 1.0% corresponding to Ppk=0.77
n=40, Ppk=0.58, Pp=0.67 1.6% corresponding to Ppk=0.72
n=50, Ppk=0.56, Pp=0.66 2.1% corresponding to Ppk=0.68
n=60, Ppk=0.55, Pp=0.65 2.4% corresponding to Ppk=0.66
n=80, Ppk=0.53, Pp=0.63 3.1% corresponding to Ppk=0.62
n=100, Ppk=0.52, Pp=0.63 3.6% corresponding to Ppk=0.60
95 / 95 Attribute
RQL0.05 = 5%
Parameters AQL
n=15, Ppk=0.86 0.027% corresponding to Ppk=1.15
n=20, Ppk=0.80 0.091% corresponding to Ppk=1.04
n=30, Ppk=0.74 0.28% corresponding to Ppk=0.92
n=40, Ppk=0.71 0.47% corresponding to Ppk=0.87
n=50, Ppk=0.69 0.66% corresponding to Ppk=0.83
n=60, Ppk=0.68 0.80% corresponding to Ppk=0.80
n=80, Ppk=0.66 1.09% corresponding to Ppk=0.76
n=100, Ppk=0.65 1.29% corresponding to Ppk=0.74
Parameters AQL
n=15, Ppk=0.87, Pp=0.91 0.024% corresponding to Ppk=1.16
n=20, Ppk=0.82, Pp=0.87 0.071% corresponding to Ppk=1.06
n=30, Ppk=0.76, Pp=0.83 0.22% corresponding to Ppk=0.95
n=40, Ppk=0.73, Pp=0.81 0.39% corresponding to Ppk=0.89
n=50, Ppk=0.71, Pp=0.79 0.54% corresponding to Ppk=0.85
n=60, Ppk=0.69, Pp=0.77 0.73% corresponding to Ppk=0.81
n=80, Ppk=0.67, Pp=0.76 1.00% corresponding to Ppk=0.78
n=100, Ppk=0.65, Pp=0.74 1.29% corresponding to Ppk=0.74
95 / 99 Attribute
RQL0.05 = 1%
Parameters AQL
n=15, Ppk=1.18 0.00014% corresponding to Ppk=1.56
n=20, Ppk=1.10 0.0012% corresponding to Ppk=1.41
n=30, Ppk=1.03 0.0071% corresponding to Ppk=1.27
n=40, Ppk=0.99 0.018% corresponding to Ppk=1.19
n=50, Ppk=0.96 0.033% corresponding to Ppk=1.13
n=60, Ppk=0.94 0.050% corresponding to Ppk=1.10
n=80, Ppk=0.92 0.079% corresponding to Ppk=1.05
n=100, Ppk=0.90 0.11% corresponding to Ppk=1.02
Parameters AQL
n=15, Ppk=1.18, Pp=1.18 0.00013% corresponding to Ppk=1.57
n=20, Ppk=1.11, Pp=1.13 0.0010% corresponding to Ppk=1.42
n=30, Ppk=1.04, Pp=1.08 0.0061% corresponding to Ppk=1.28
n=40, Ppk=1.00, Pp=1.05 0.016% corresponding to Ppk=1.20
n=50, Ppk=0.97, Pp=1.03 0.029% corresponding to Ppk=1.15
n=60, Ppk=0.95, Pp=1.01 0.044% corresponding to Ppk=1.11
n=80, Ppk=0.93, Pp=1.00 0.070% corresponding to Ppk=1.06
n=100, Ppk=0. 91, Pp=0.98 0.10% corresponding to Ppk=1.03
Factors k such that the confidence level is , that at least a proportion P (reliability level) of the Normal distribution,
will be less than X̅ + k*s and greater than X̅ – k*s for a two-sided limit, where X̅ and s are estimates of the mean
and standard deviation computed from a sample size n from a Normal distribution. Two-sided intervals cover X̅ ± k*s.
Note: To demonstrate a two-sided specification is met with specified confidence and reliability, then both X̅ + k*s ≤ USL and X̅ – k*s ≥ LSL
Factors k such that the confidence level is , that at least a proportion P (reliability level) of the Normal distribution
will be less than X̅ + k*s (or greater than X̅ – k*s) for a one-sided limit, where X̅ and s are estimates of the mean
and standard deviation computed from a sample of size n from a Normal distribution.
Note: To demonstrate that a one-sided specification is met with specified confidence and reliability, then either X̅ + k*s ≤ USL or X̅ – k*s ≥ LSL









