Chapter 5

Quality control
W. Greg Miller1 and James H. Nichols2
1
Department of Pathology, Virginia Commonwealth University; Richmond, VA, United States, 2Department of Pathology, Microbiology and
Immunology, Vanderbilt University Medical Center; Nashville, TN, United States

Learning objectives QC samples that have been stored appropriately before

After reviewing this chapter, the reader will be able to: and during use. Process control involves quality manage-
G Define the purpose of quality control for laboratory testing. ment of preanalytical, analytical, and postanalytical pro-
G Describe the principles of statistical process control. cesses with documentation and review of results.
G Identify the attributes and limitations of quality control Additional aspects of a quality management system
materials. should include information management (document and
G Explain the role of risk management and individualized qual- results control and record retention), occurrence manage-
ity control plans (IQCPs) for defining potential hazards and ment (incident response and resolution), assessment of
reducing errors in the laboratory testing process.
quality benchmarks and process improvement, determina-
G Discuss the utility of proficiency testing.
tion of customer service and satisfaction, and facilities for
test performance in a safe manner.
Introduction QC is a part of the process control portion of the qual-
The total error of a laboratory test must be small enough ity management system. An SOP should include all
to reflect changes in the biological condition in order to aspects of the analysis including the selection of QC
have value for clinical decision-making. The total error of materials and other monitoring parameters, how fre-
a result is influenced by a number of factors including quently to sample the measurement process, how to deter-
biological (physiological) variability within an individual; mine statistical parameters to describe a measuring
preanalytical variability in sample collection, transporta- system’s performance, the QC plan and the acceptance
tion, processing, and storage; analytical variability in test criteria for QC results, corrective action when problems
performance; and interfering substances such as drugs or are identified, and the documentation and review pro-
metabolic components. cesses. The SOP should also include who is authorized to
This chapter addresses quality control (QC) of the ana- establish acceptance criteria for the QC plan and the
lytical measurement process to ensure analytical variabil- acceptance of results, who should review performance
ity meets accuracy and precision requirements that have parameters including statistical QC results, and who can
been established for a measurement procedure and are authorize exceptions to or modify an established QC pol-
considered appropriate for patient care. QC of the analyti- icy or procedure.
cal measurement process is a component of an integrated The essential components of statistical process control
quality management, or quality assurance, system, as are also shown in Fig. 5.1. The measurement system
shown in Fig. 5.1. The quality management system inte- should be periodically evaluated by sampling its ability to
grates good laboratory practices to assure correct results produce the expected results for QC samples. The
for patient care. Written standard operating procedures expected results are predefined and reflect the measure-
(SOPs) should be available to describe test methods that ment variability when the system is operating correctly.
meet the organization’s requirements for patient care, and If the QC results indicate a stable measurement process,
SOPs should be followed by well-trained and competent then the patient results have a high probability of being
personnel. Samples must be analyzed on well-maintained correct. If the QC results fail evaluation criteria, then the
equipment that has been verified to meet manufacturer patient results may not be reliable for clinical use. In the
performance specifications, using validated reagents and latter case, corrective action must be taken to fix the

Contemporary Practice in Clinical Chemistry. DOI: [Link]

© 2020 Elsevier Inc. All rights reserved. 77
78 Contemporary Practice in Clinical Chemistry

Quality management system FIGURE 5.1 Overview of statistical process

control (quality control) and its integration
Documents and records
Sample the into a quality management system.
Organization
measurement Personnel
process Equipment
Purchasing and inventory
Process control
- Preanalytical
Repeat - Analytical
patients * Calibration
* Maintenance
* Quality control
* PT/EQA
- Postanalytical
Information management
Occurrence management
NO YES Report Assessment
Corrective
Stable? patient Process improvement
action
results Service and satisfaction
Facilities and safety

analytical process and patient samples must be reanalyzed to a clinically acceptable level and ensure quality of test
for the interval since the last acceptable QC results. Good results.
laboratory practice requires verification that a measuring
system is performing correctly anytime patient samples
are analyzed and results released to clinicians for manage- Selection of quality control materials
ment decisions.
Generally, two different concentrations of QC materials
are necessary for adequate process control and are the
minimum required by the Clinical Laboratory
Implementing statistical process control Improvement Amendments of 1988 (CLIA) regulations in
Traditional statistical process control uses stable QC the United States. For quantitative methods, QC materials
materials as surrogates for patient samples in a measure- should be selected to provide analyte concentrations that
ment process. Reliance on periodic analysis of QC sam- adequately monitor the analytical measurement range
ples does not require a comprehensive knowledge or (also called the measuring interval) of the method. Most
assessment of potential hazards in the testing process, quantitative assays have a linear response over the analyt-
since this approach evaluates the aggregate effect of all ical measurement range, and one can be confident that the
components of the measurement system on the test result. performance is acceptable if the results near the lower
Evaluation of the performance of QC samples provides a and upper assay limits are acceptable. In the case of non-
good means of detecting systematic errors that persist linear method response, it may be necessary to use addi-
over time but may not detect random, sporadic errors in tional controls at intermediate concentrations. Critical
test performance. A complementary approach, discussed concentrations for clinical decisions may also warrant QC
in a later section, assesses the potential hazards or weak monitoring. In the case of analytes that have poor preci-
points in each step of the testing process. Frequently sion at low to normal concentrations, such as bilirubin,
occurring hazardous conditions or those with sufficient the concentration should be chosen to provide an adequate
potential for patient harm require specific controls (that standard deviation (SD) for practical evaluation. For pro-
may be in addition to periodic measurement of QC sam- cedures with extraction or pretreatment, at least one QC
ples) or actions to mitigate risk to the patient. Some material should be employed to detect errors in the
devices have built-in checks or controls that monitor por- extraction or pretreatment step.
tions of the analytic process that may address specific This chapter is primarily focused on QC procedures
potential hazards. Some of these internal built-in checks for quantitative methods. However, the principles can be
are performed with each test and may provide a better adapted to most qualitative procedures. For qualitative
means of detecting random errors than reliance only on tests, for example, urine drugs of abuse testing, both a
surrogate QC sample results. The compilation of all of the negative and positive control are necessary, and controls
risk mitigation strategies, both built-in and external con- with concentrations near the threshold are recommended
trol processes, constitutes a QC plan for a specific mea- to adequately control the method’s ability to discriminate
surement device that can reduce the residual risk of harm between negative and positive. For tests based on
 Quality control Chapter | 5 79

qualitative interpretation of quantitative measurements, Repeated freeze/thaw cycles should be avoided. Users
for example, drugs of abuse where a concentration is con- should limit the time spent at room temperature and the
verted into a positive or negative result, it is also neces- time spent uncapped with the potential for evaporation.
sary to monitor near the threshold/cutoff concentrations to An expiration time after opening should be established for
ensure appropriate discrimination between negative and each QC material and may be different for different ana-
positive responses. For qualitative procedures with graded lytes in the same control product. For QC materials recon-
responses, for example, dipstick urinalysis, a negative, stituted by adding a diluent, the vial-to-vial variability
and at least one positive with a value in the intermediate can be minimized by standardizing the pipetting proce-
graded response region are appropriate. For qualitative dure, for example, use the same pipette or filling device
tests based on other properties, for example, electropho- (preferably an automated device) and have the same per-
retic procedures and immunofluorescence, the QC proce- son prepare the controls whenever practical.
dure should appropriately verify the method’s ability to Analyte concentrations in multiconstituent control
discriminate normal from pathologic conditions. materials may not be at optimal concentration levels for
The QC materials selected must be manufactured to all assays. This limitation is caused by solubility consid-
provide a stable product that can be used for an extended erations and potential interactions between different
time period, preferably 1 or more years. Use of a single constituents, particularly at higher concentrations.
lot for an extended period allows reliable interpretive cri- Supplementary QC materials may be needed to ade-
teria to be established that will permit efficient detection quately monitor the entire analytical measurement range
of an assay problem, avoid false alerts due to poorly when using a multiconstituent QC material to control sev-
defined acceptance ranges, and minimize the influence of eral analytes at the same time.
reagent lot changes on the QC matrix and need for adjust-
ment of target values.
Frequency to assay quality control samples
The frequency to assay QC samples is a function of sev-
Limitations of quality control materials
eral parameters:
There are limitations inherent in currently available QC G the analytical stability of the method,
materials. One limitation is that QC materials are not G how much error can be tolerated without impacting
always “commutable” with native clinical samples. A
patient care,
commutable QC material (or other reference material) is G the number of results produced in a period of time,
one that reacts in a measuring system to give a result
and
that would be comparable to that expected for a native G the need to verify and document the reliability of clini-
patient sample containing the same amount of analyte.
cal results at the time they are reported.
Commercially available QC materials are typically
noncommutable with native patient samples because the The stability of the measurement system is the funda-
serum or other biological fluid matrix has been altered mental determinant of how frequently a QC sample needs
from that of a native patient sample [1]. The matrix alter- to be assayed. The more stable the system, the less fre-
ation is due to processing of the biological fluid during quently a statistical process control needs to be performed.
product manufacturing, use of partially purified human Minimum laboratory practice, consistent with CLIA
and nonhuman analyte additives to achieve the desired Regulations Section 493.1256 [3], is to assay controls at
concentrations, and various stabilization processes that least once per 24 hours, or more frequently if specified by
can alter proteins, cells, and other components. The the method manufacturer or if the laboratory determines
impact of matrix alteration on the recovery of an analyte that more frequent QC analysis is necessary to maintain
in a measuring system is not predictable, is different for the performance characteristics of a method. Some tests
different lots of QC material, is different for different ana- have more stringent requirements. For example, CLIA
lytical methods, and can be different for different lots of Regulations Section 493.1267 requires that blood gas
reagent within the same analytical method [2]. measurements analyze at least one control every 8 hours
A second limitation of QC materials is possible deteri- that includes both high and low levels in the course of
oration of the analyte during storage. Analyte stability 24 hours; in addition, a control must be run with each
during unopened refrigerated or frozen storage is gener- patient sample unless the instrument automatically cali-
ally excellent, but slow deterioration eventually limits the brates at least every 30 minutes. Methods that have auto-
shelf life of a product and can introduce a gradual drift in mated built-in control procedures may use less frequent
control results. Analyte stability after reconstitution, thaw- analysis of external QC materials (discussed in a later
ing, vial opening, or exposure to light or air can be impor- section).
tant sources of variability in QC results, and stability can The need to verify the clinical acceptability of results
vary substantially among analytes in the same vial. may support more frequent QC sampling than that based
80 Contemporary Practice in Clinical Chemistry

TABLE 5.1 Common sources of measurement variability.

Source Time interval for fluctuation Likely statistical distribution

Pipette volume Short Gaussian
Instrument temperature control Short or long Gaussian or other
Electronic noise in the measuring system Short Gaussian
Calibration cycles Short to long Gaussian or shift (periodic step)
Reagent deterioration in storage Long Drift
Reagent deterioration after opening Intermediate Cyclic, periodic drift/step
Calibrator deterioration in storage Long Drift
Calibrator deterioration after opening Intermediate Cyclic, periodic drift/step
Control material deterioration in storage Long Drift
Control material deterioration after opening Intermediate Cyclic, periodic drift/step
Environmental temperature and humidity Variable Variable
Reagent lot changes Intermediate to long Shift (random step)
Calibrator lot changes Intermediate to long Shift (random step)
Instrument maintenance Variable Gaussian, cyclic, or shift (periodic step)
Deterioration of instrument components Variable Variable

strictly on method stability characteristics or on regulatory detect a problem while minimizing the frequency of false
minimum requirements. More frequent QC sampling is alerts. Prior to establishing QC values and the statistical
appropriate with automated systems that release results parameters used to evaluate QC results, the measuring
continuously to minimize the probability to release incor- system must be correctly calibrated and operating within
rect results that could initiate inappropriate clinical action acceptable performance specifications. Some sources of
before an instrument problem is detected. For example, measurement variability are listed in Table 5.1.
QC scheduled on a 24-hour cycle might be performed at Measurement variability can be affected by factors with
09:00 hours. If the QC results indicate a method problem, short time-interval frequencies, many of which can be
the erroneous condition could have started at any time described by Gaussian error distributions, as well as inter-
during the previous 24 hours. If the problem had occurred mediate and long time-interval factors that can cause
at 15:00 hours the previous day, out-of-control results cyclic fluctuations over several days or weeks, gradual
would have been reported for 18 hours, and the risk of drift over weeks or months, and more abrupt shifts in
harm to a patient from decisions based on those erroneous results. Acceptance criteria for QC results must ade-
results must be considered. The cost of a medical error, or quately account for all sources of expected variability in
simply the cost of repeating the questionable patient sam- results that are likely to occur when the measuring system
ples, could be more expensive than more frequent QC is performing to specifications.
analysis. Analytes, such as whole blood glucose and A QC material must have a reliable target value for an
serum carbon dioxide, are not stable over 24 hours, and analyte. This requires adequate statistical sampling of
more frequent QC may be necessary to ensure sample via- both the QC material’s vial-to-vial and open-vial stability,
bility should a specimen need to be repeated. as well as the measurement variability during a period of
time when the measuring system is calibrated correctly
and is exhibiting the expected imprecision associated with
Establishing quality control target values and
a stable measurement condition. The objective to estab-
acceptance ranges that represent a lishing a reliable QC target value is to include all sources
stable measurement operating condition of variability in the measurement process that will ensure
QC target values and acceptable performance ranges are a representative mean concentration. This objective is
established to optimize the probability of detecting a mea- rarely met due to long-term variability components and
surement error that is large enough to impact clinical the practical inability to account for all influences at the
care. QC acceptable ranges should be sensitive enough to time of QC target value assignment. A Clinical and
 Quality control Chapter | 5 81

Laboratory Standards Institute (CLSI) guideline for QC method exists, the SD or range of QC results for
recommends the mean value from assaying a QC material stable measurement performance can be established from
on a minimum of 10 different days for an initial target the cumulative SD over a 6- to 12-month period, which
value [4]. Note that it is preferable to use a single bottle will include most sources of expected variation. Note that
of QC material for the expected duration of its open-vial the cumulative SD is not established from the average of
stability in order to account for analyte stability effects monthly QC statistical summaries. Different sources of
rather than a fresh bottle on each day of testing. If a 10- long-term variability, which occur at different times dur-
day protocol is not practical, provisional, or temporary, ing the use of a measuring system, may not be represented
target values can be established with less data but should in a monthly SD for QC results. However, the cumulative
be updated when additional replicate results are available. SD includes contributions from most sources of variabil-
When applicable, more than one assay calibration event ity as they occur and as they are reflected in the individ-
should be represented in the 10-day period to include the ual QC results. Consequently, the cumulative SD will
variability associated with the calibration process. The typically be larger than the monthly values and better rep-
target value should be reassessed after additional QC data resent the actual variability of the measuring system. If
are available that include more sources of variability than the imprecision expected during stable operation is under-
can be included in 10 days. estimated, then the acceptable range for QC results will
Some QC materials are provided with manufacturer- be too narrow and the false-alert rate will be unacceptably
assigned target values and ranges that can be used to con- high. If the imprecision is overestimated, then the
firm that a method meets the manufacturer’s specifica- acceptable range for QC results will be too large and the
tions. Such assigned values can be used initially by the QC process will be less sensitive to detect a measurement
laboratory, but both the target value and ranges should be error.
reevaluated and adjusted by the laboratory after adequate It is important to include all valid QC results in the
replicate results have been obtained. Laboratory-defined calculation of a QC range. A valid QC result is one
target values and SDs ensure that the acceptance criteria that was, or would have been, used to verify acceptable
used by a laboratory adequately reflect performance for measuring system performance and reporting of patient
the method in that laboratory. The acceptability limits results. Only QC results that would have been responsi-
suggested by a manufacturer may reflect sources of vari- ble for holding the release of patient results should be
ability, such as between instruments and between reagent deleted (with documentation) from summary calcula-
and calibrator lots, or may represent best-case or worst- tions. Selective editing of QC results underestimates the
case performance that may not be typical for that QC true method of SD, so QC ranges will be too narrow and
material in an individual laboratory. QC materials with will generate increased false QC alerts with concomitant
assigned target values are also available from third-party reduction in the effectiveness of the statistical process
manufacturers (i.e., manufacturers not affiliated with the control.
method manufacturer). Caution should be used when When a method has been established in a laboratory,
employing assigned values for third-party QC materials, and a new lot of QC material is being introduced, the tar-
since the target values and ranges may have been estab- get value for the new lot should be used with the cumula-
lished using protocols that do not adequately reflect spe- tive SD from the previous lot to establish an
cific measuring system performance. Noncommutability acceptable range for the new lot of QC material. This
may further limit applicability of third-party QC values practice is appropriate because the measurement impreci-
for some methods. sion is a characteristic of the measuring system. The mea-
Once a target value has been assigned to a QC mate- suring system imprecision is unlikely to change with a
rial, a range or SD must be established that represents the different lot of QC material. If the target values for the
typical imprecision of the measuring system when it is old and new lots are substantially different, there may be
performing according to its manufacturer’s specifications. a different imprecision observed, and an adjustment to the
Although there are non-Gaussian components of variabil- QC SD may be necessary as additional experience with
ity in QC results, an SD is the conventional way to the new lot of QC material is accumulated.
express method variability because the statistical packages When a new method is introduced for which there is
in instrument and laboratory computer systems are no historical performance information, then the SD for
designed for QC data analysis based on mean and SD stable performance must be established using data from
parameters. An SD based on the data from a 10-day target the method validation and target value assignment of the
value assignment, or a 30-day monthly summary, has a QC materials. Additional information on expected method
large uncertainty and cannot reflect all sources of vari- performance can be obtained from the method manufac-
ability that need to be accommodated in the QC result turer, the package insert, and the method’s performance
evaluation process [4]. When previous experience with a in interlaboratory QC and/or proficiency testing (PT)
82 Contemporary Practice in Clinical Chemistry

programs. For a new method, the initial SD and QC range of falsely indicating an error condition when there is zero
evaluation criteria will need to be monitored closely, and systematic bias. Trend detection techniques such as cumu-
adjustments made after more data have been collected to lative sum or exponentially weighted moving averages
include intermediate and long-term sources of method are more powerful than rules based on the number of
variability. results that exceed a particular SDI and are recommended
when computer systems can support their use [5,8].
A combination of interpretive rules applied simulta-
Establishing acceptance criteria to evaluate neously to a set of QC results will optimally detect errors
quality control results with the lowest false-alert rates [6,7,9]. For example, a
Operating conditions must be established for each mea- 13S/22S/R4S multirule applied to two QC results identifies
suring system before suitable QC interpretive criteria or an error condition if one control exceeds 6 3 SD from
rules can be selected. Establishing acceptance criteria to the target value, if two controls exceed 6 2 SD in the
evaluate QC results is a balance between the sensitivity of same direction from the target value, or if the range
detecting a significant analytical error condition and the between two controls exceeds 4 SD. In this multirule
frequency of false alerts. Various statistical models and example, the 13S component is sensitive to large system-
assumptions regarding distribution of errors have been atic bias, the 22S component is sensitive to systematic
used to establish interpretive rules for QC results [5 7]. bias, and the R4S component is sensitive to imprecision.
The probability for any interpretive rule to detect a mea- This multirule has a low false-alert rate and a better prob-
surement error increases as the magnitude of the error ability to detect an error condition than the individual
increases. When evaluating interpretive rules, it is com- components alone. A number of multirule combinations
mon practice to express the magnitude of an error in mul- can be used as appropriate for the performance character-
tiples of the SD for the method (i.e., SD intervals, SDIs, istics and clinical requirements of a measuring system.
or z-score). A challenge when selecting QC interpretive rules is
A conventional way to express QC interpretive rules is that all sources of variability listed in Table 5.1 occur in
with an abbreviation nomenclature summarized in most contemporary automated measuring systems and
Table 5.2. Note that fractional SDIs are permissible, as in introduce long-term cyclic and step fluctuations in perfor-
the 22.5S rule. For example, consider a 12S interpretive mance. These sources of variability are not adequately
rule that would be violated if a single QC result was described by Gaussian-based QC interpretive rules. Since
.2 SD from the target value. A power function graph [7] almost all instrument and laboratory computer system QC
shows that a 12S rule has a 35% probability to detect a software is based on SDI (z-score) interpretation, it is nec-
systematic error that is 1 SD in magnitude and a 78% or essary to estimate an SD that approximates the long-term
98% probability to detect systematic errors that are 2 or variability from Gaussian and non-Gaussian contributions.
3 SD in magnitude, respectively. However, it is not Thus data collected over a significant time period are nec-
recommended to use a 12S rule because, due to the inher- essary to ensure the SD and QC ranges represent typical
ent imprecision, a 12S rule has an almost 10% probability

TABLE 5.2 Abbreviation nomenclature to express quality control (QC) rules assuming Gaussian (normal) distribution
of imprecision.

Rule Meaning Detects

12S One observation exceeds 2 SD from the target value Imprecision or systematic bias
13S One observation exceeds 3 SD from the target value Imprecision or systematic bias
22s Two sequential observations, or observations for two QC samples in the same run, exceed Systematic bias
2 SD from the target value in the same direction
22.5s Two sequential observations, or observations for two QC samples in the same run, exceed Systematic bias
2.5 SD from the target value in the same direction
R4S Range between two observations in the same run exceeds 4 SD Imprecision
10m Ten sequential observations are on the same side of the target value (mean) Systematic bias and trend (not
recommended)
81S Eight sequential observations for the same material exceed 1 SD in the same direction from Systematic bias and trend
the target value
 Quality control Chapter | 5 83

FIGURE 5.2 Levey Jennings plot of quality

control results (n 5 1232) for a single lot of qual-
ity control material used for a 10-month period.
The mean and standard deviation are cumulative
values from all data.

variability when the method is stable and performing measuring system performance, and the need to identify
correctly. potential measurement issues with a false-alert rate ,1%.
Fig. 5.2 shows results for a single lot of QC material This multirule had 0.6% false alerts when applied to the
over a 10-month period for an automated glucose measur- data in Fig. 5.2 with a target value assigned as the mean
ing system. The glucose method stability and performance value from the November-to-January period and the SD
over the 10 months were considered acceptable for clini- for the 10-month period to represent overall imprecision.
cal use. If acceptance criteria were based on the SD from If a 22S rule had been used instead of a 22.5S rule (sequen-
the initial 30-day period, the subsequent long-term QC tial observations in this example), the false-alert rate
result fluctuations would have caused a large number of would have increased from 0.6% to 1.8%. An 81.5S rule
false alerts with significant effort and expense for trouble- was selected to provide detection of bias trends because it
shooting what was actually inherent variability during had zero false alerts compared to 0.5% false alerts for an
stable method performance. The SD lines shown in 81S rule. The 81.5S rule value was small enough that a bias
Fig. 5.2 were determined from the entire 10 months of trend of this magnitude would not adversely affect clinical
data. Examination of these data shows several fluctuations interpretation of patient results. A rule that evaluates
that cannot be described by a Gaussian statistical model. sequential results on one side of the mean can have a high
The first reagent lot change caused a step shift to higher false-alert rate when a measuring system’s actual error
values, but the shift was small enough to not require a tar- distributions are not perfectly Gaussian. For example, a
get value adjustment (see later section). The second 10m rule for the data in Fig. 5.2 would have increased the
reagent lot change had no effect on the QC results. false-alert rate by 10.6%, which is why a 10m rule is not
Between March and April, there was a transition to lower recommended. Many contemporary analyzers are very
results that did not correspond to any reagent lot change, stable and may produce QC results on one side of the
maintenance, or calibration events. Throughout the 10- mean for extended periods of time; however, if the mag-
month period, there are intervals of several weeks dura- nitude of the difference from the target value is small, it
tion when the imprecision was either better or worse than may not indicate a problem with clinical interpretation of
other time periods emphasizing the importance to use a the patients’ results.
cumulative SD over a sufficient time interval that In practice, empirical judgment is frequently necessary
includes most sources of variability. to set QC rules that fit the actual data accumulated over a
An empirical approach is frequently necessary to long enough time period to adequately describe the vari-
establish QC interpretive rules that will adequately ability observed when a measuring system is working cor-
encompass the observed variability in a measuring sys- rectly. Caution should be used when selecting QC
tem. An example of an empirically developed multirule interpretive rules based only on Gaussian models of
based on the data in Fig. 5.2 is 13S/22.5S/R4S/81.5S, in imprecision because the rules may not accommodate all
which 13S detects imprecision or bias, 22.5S detects sys- types of variability observed for many measuring systems.
tematic bias, R4S detects imprecision, and 81.5S detects a Whatever statistical approach is used, the balance
trend in bias based on eight consecutive results that between false alerts and the probability of error detection
exceed 1.5 SD. This multirule was selected based on the is improved when multiple rules are used in combination.
clinical requirements for patient care, observed long-term When establishing rules to interpret QC results, statistical
84 Contemporary Practice in Clinical Chemistry

process control can verify that a measuring system is pro- Corrective action when a quality control result
ducing results within its expected variability only at a indicates a measurement problem
point in time compared to when the system was properly
calibrated and in a stable operating condition. QC rules A QC alert occurs when a QC result fails an evaluation
are intended to detect bias and imprecision that is large rule, indicating that an analytical problem may exist. A
enough to require correction before patient results are QC alert should occur only when there is a high probabil-
reported. Random events, for example, a temporary clot ity the measuring system is producing results that are
in a pipette, that do not persist until the next QC sample unreliable for patient care. When this condition occurs,
is measured will not be detected by external surrogate QC corrective action should be taken to investigate the cause
sample evaluation. for the QC alert. Fig. 5.3 presents a generalized trouble-
A laboratory director may determine in the process of shooting sequence. When a QC alert occurs, the first
reviewing statistical parameters for QC data that a mea- action is to discontinue assaying patient samples.
suring system’s variability is too large to meet medical Repeating the QC measurement on the same sample is
requirements. If the measuring system is performing in a not recommended, because, with properly designed accep-
stable condition within its manufacturer’s specifications, tance criteria, it is more likely that a measurement system
the observed variability is an inherent property of the cur- problem exists than the QC result was a statistical outlier.
rent technology. In this case, the only solutions are to However, QC materials can deteriorate after opening due
improve the measuring system performance or to use a to improper handling and storage or due to labile analytes.
different method. If the measuring system performance Thus repeating the measurement on a new vial of the QC
cannot be improved and a better method is not available, material may establish that the alert was caused by a dete-
the laboratory must accept the limitations and communi- riorated QC material rather than a method problem. In
cate them to patient care providers. In this circumstance, this situation, if the QC alert is resolved, testing of patient
the QC ranges or interpretive rules should not be made samples can resume. One caution is to consider a devel-
artificially stringent. This incorrect approach will not oping trend. If the repeat result is very near the accept-
improve measuring system performance but will increase ability limits, it means the measuring system’s results are
the QC false-alert rate and decrease the efficiency and close to unacceptability and may indicate an impending
practicality of the statistical QC process. problem that should be investigated as soon as possible.
It is important that the multirule sequence include a

FIGURE 5.3 Generalized troubleshooting sequence

Stop reporting patient following a quality control alert.
results

Assay a new vial of

QC

Yes Does QC
QC result suggest
OK a trend toward
the limits No
No

• Identify and correct Yes

Continue patient
issue
sample testing
• Repeat QC

• Repeat patient
QC samples
OK • Continue patient
sample testing
No

Further technical
investigation
 Quality control Chapter | 5 85

FIGURE 5.4 Levey Jennings graph showing impact of a reagent

lot change on matrix interaction with quality control materials.

trend-detecting QC rule, such as the 81.5S (in the previous representative assessment of the interval when there was a
example), an exponentially weighted moving average, or measurement defect. If the repeat results for this sample
a cumulative sum. group are within established performance criteria for
When repeat testing of a new QC sample does not repeat testing of patient specimens, the original results for
resolve the alert situation, the instrument and reagents the QNS samples can be reported. Otherwise, the original
should be inspected for component deterioration, mechan- results for the QNS samples are considered erroneous,
ical problems, or other issues. When the problem is iden- and a corrected report should indicate no results and that
tified and corrected, controls should be measured to the original results were incorrect.
verify assay performance has acceptably returned to
stable conditions. In many cases, the method will require
Verifying quality control evaluation parameters
recalibration before QC results can confirm correct
method performance. After confirmation of correction, following a reagent lot change
patient samples since the time of the last acceptable QC Changing reagent lots can have an unexpected impact on
results should be repeated. The laboratory director should QC results. Careful cross-over evaluation of QC target
establish acceptable performance criteria for results from values is necessary. Because the matrix interaction
repeat testing of patient samples in order to determine between a QC material and a reagent can change with a
whether the original test results can be reported or different reagent lot, the QC results may not be a reliable
whether a corrected result should be reported. The criteria indicator of a method’s performance for patient samples
for acceptability of repeated tests are based on measuring following a reagent lot change [2]. In the example in
system performance, the patient population, and the risk Fig. 5.4, the QC values for the high-concentration control
of harm to a patient. shifted following the change to a new lot of reagents, but
In some cases, there may not be adequate volume left there was no change in results for the low-concentration
in patient samples (quantity not sufficient; QNS) for control. A comparison of results for native patient sam-
repeat testing. In these situations, no results can be ples assayed using the old and new reagent lots demon-
reported unless the impact of the error on the original strated equivalent patient results using either lot of
results was small enough to have an insignificant effect reagents. Consequently, the change in QC values was due
on clinical interpretation. One way to evaluate the clinical to a difference in matrix interaction between the high-
impact of a measuring system problem is to repeat those concentration QC material and the two reagent lots.
patient samples that have adequate volume. The repeated When changing reagent lots, laboratories should be
samples must represent the concentration range of the aware that the target value for a QC material may
QNS samples, must represent the time span since the pre- be affected. Target values for QC materials should be
vious acceptable QC results, and should include a adjusted when a different matrix interaction is observed
suitable number of samples originally assayed while the with a new reagent lot. First, patient sample results mea-
method was in the unacceptable condition to ensure sured using the old and new reagent lots should be
86 Contemporary Practice in Clinical Chemistry

compared to verify equivalent results are produced. If a shift when a new kit is opened. Native patient samples
problem is identified, the calibration of the new reagent should be analyzed with the old and new kits to verify
lot should be investigated and corrected, or the possibility consistency of patient results. When possible, QC materi-
of defective reagents considered. The native patient sam- als that are independent of the kit lot are recommended.
ple results, not the QC results, are the basis to verify that Laboratories should also avoid changing lots of QC mate-
results using the new reagent lot are consistent with those rial at the same time as changing lots of reagent or cali-
using the previous reagent lot. CLSI document EP26 pro- brators. The analysis of native patient samples, except for
vides guidance for verifying reagent lot changes and unstable analytes, provides a reliable approach to verify
recommends a minimum of three patient samples or more the consistency of results following changes in lots of
depending on the number of critical clinical decision reagents or calibrators.
values and the imprecision of a measuring system [10].
EP26 also includes guidelines for acceptance criteria
Calibration issues in quality control
related to the magnitude of a difference that is clinically
meaningful and the imprecision of a measuring system. Calibration of the analytical measurement system is a key
CLSI document C24 also includes information on the component to achieve quality results. The principal reason
influence of reagent lot and other changes on QC target to perform statistical process control is to verify that the
values [4]. calibration and the variability of the analytical system
Once patient results are shown to be equivalent within remain within limits expected for stable measuring system
clinically acceptable limits, the results from QC samples performance. Specific techniques for calibration are
should be evaluated to determine whether the target unique to individual measuring systems and will not be
values have changed with the new lot of reagent. If a QC covered here. However, some general principles for
target value has changed, that value should be adjusted to implementing calibration procedures, and for verifying
reflect the new value to be used with the new lot of calibration uniformity among multiple methods, can con-
reagents to keep the QC interpretive rules centered on the tribute to stability and clinical reliability of laboratory
appropriate QC target value. This target value adjustment results.
will allow the same QC interpretive rules to be utilized. Calibration is most often performed by the laboratory
In making this adjustment, the laboratory is compensating using calibrator materials provided by the measuring sys-
for an altered matrix interaction of the QC material with a tem manufacturer. In some cases, for example, point-of-
different reagent lot. The SD used to evaluate QC results care devices, measuring systems are calibrated during the
will not typically change when a new lot of reagent is put manufacturing process, and the laboratory performs a ver-
into service. The SD represents the expected variability ification of that calibration. In either situation, traceability
when the method is stable and performing according to of result accuracy to the highest order reference system is
specifications. In most cases, the variability of a method provided by the measuring system manufacturer. The
will be the same with any lot of reagent(s); however, measuring system manufacturer’s calibrator material and
there may be occasional exceptions requiring adjustment assigned target values are designed to produce accurate
of the SD, for example, if the reagent is reformulated and results with native patient samples assayed using that par-
performance changes. ticular manufacturer’s measuring system. One manufac-
turer’s product calibrator is not intended to be used with
other measuring system, and laboratories should not use
Verifying method performance following use of a calibrator materials intended for one measuring system
with any other measuring system because the matrix inter-
new lot of calibrator
action with different reagents will produce a different
When implementing a new lot of calibrator, with no response. Use of a calibrator with a measuring system for
change in reagent, there should be no change in matrix which it was not specifically intended can produce misca-
interaction between the QC material and the reagent. In libration and erroneous patient results.
this case, the QC results provide a reliable indication of National and international reference materials are
calibration status with the new lot of calibrator. If the QC available for some analytes. In many cases, these refer-
results indicate a bias following use of a new lot of cali- ence materials are intended for use with higher order ref-
brator, the problem is caused by the new lot of calibrator erence measurement procedures and may not be
and needs to be addressed to ensure consistent results for suitable to use directly with routine clinical laboratory
patient samples. measuring systems. Laboratories should not use national
Some manufacturers package kits that include or international reference materials to directly calibrate a
reagents, calibrators, and QC materials together. In this clinical laboratory measuring system (or to verify the cali-
case, the QC results could fail to identify a calibration bration of a clinical laboratory measuring system) unless
 Quality control Chapter | 5 87

commutability with native patient samples has been veri- package insert and cannot be used with any other measur-
fied for that specific reference material and clinical labo- ing systems. These calibration verification materials may
ratory measuring system (see Chapter 17: Harmonization have target values certified by the manufacturer to be
of results among laboratories, for additional information suitable for all reagent lots, or they may be specific for
on calibration traceability and commutable reference stated reagent lots. Third-party QC materials are usually
materials). The reference material certificate of analysis not suitable for calibration verification. These materials
should be reviewed for commutability documentation. If are typically not validated for commutability with native
the reference material is noncommutable, the clinical lab- clinical specimens for use with different measuring sys-
oratory measuring system could actually be miscalibrated tems and do not have target values that are traceable to
and produce erroneous results for patient samples [11,12]. reference systems.
Optimal long-term stability of patient results is
achieved if recalibration is performed only when the rela-
tionship between the analytical measuring system Development of a quality control plan
response and specimen concentration has changed.
Performing a recalibration when the calibration relation-
based on risk management
ship between measurement signal and concentration has Historically, the quality of laboratory results was ensured
not changed can be a source of imprecision in results. by periodic testing of QC samples to verify stability of
This imprecision occurs because the new calibration will measuring system performance. QC samples act as surro-
produce a slightly different relationship between analyti- gates for patient samples and have target values that
cal system response and specimen concentration. CLIA when recovered upon periodic analysis verify
Regulations Section 493.1255 requires calibration or cali- acceptable measuring system performance. Many regula-
bration verification at least every 6 months or more fre- tory agencies have required periodic analysis of QC sam-
quently if recommended by the measuring system ples as a means of ensuring quality. For instance, CLIA
manufacturer [3]. Recalibration should be performed requires the analysis of at least two levels of controls
when necessitated by drift (which can be detected by each day of patient testing (more frequently for blood gas
shifts in QC results or other monitoring parameters), by and coagulation devices). This QC strategy is sensitive to
changes in reagent lots, or after significant instrument changes in the measuring system that persist in time (sys-
maintenance. When there has been no change in measur- tematic errors). But, external QC samples do a poor job at
ing system performance, verification of the current cali- detecting random errors that occur sporadically, at one
bration may be preferred rather than performing a point in time, or for a single patient sample (i.e., hemoly-
recalibration. sis, clots, and drug interferences).
One common procedure to verify calibration is to ana- Newer laboratory instrumentation utilizes a variety of
lyze the calibrator materials as “unknowns.” Recovery of control processes to ensure the quality of patient results.
the target values indicates the measuring system calibra- Some of these controls are built-in checks performed by
tion has not changed. There is no reason to perform a the device to verify instrument function, such as electrical
recalibration because the same relationship would be rees- checks, optical checks, and biosensor checks. For exam-
tablished between measurement signal and the concentra- ple, the control line which develops on a lateral flow
tion of analyte in the calibrators. The laboratory must pregnancy or drug test evaluates kit storage viability,
establish criteria for calibration verification (i.e., agree- appropriate sample volume, sample application, and ade-
ment between the assayed result and the calibrator target quate timing. Other devices store on-board QC materials
value). Conservative criteria for agreement, such as that are analyzed periodically by the measuring system to
6 1 SD from the target value, should be considered to automate QC sample measurement. Still other devices
avoid misinterpretation of the calibration status of a require the operator to perform an action, like verifying
method. Calibration can also be verified by performing a the integrity of a temperature check (a simple device that
method comparison using native patient samples mea- changes color if frozen or heated during transit) when
sured with another measuring system known to be cor- receiving shipments of reagents and calibrators. Some of
rectly calibrated. these control processes are conducted with each analysis,
Another approach to verify calibration is to analyze providing a better means of detecting certain types of ran-
control materials that are specifically intended for this dom errors than periodic analysis of external QC samples.
purpose. Some manufacturers produce control materials, Each control process is directed at minimizing the proba-
called trueness controls, designed for calibration verifica- bility of certain errors, and devices have different risks
tion. Such manufacturer-provided materials typically have for different types of errors. Thus each device may have
matrix characteristics and target values specifically different control processes to mitigate risk. Consequently,
designed for the measuring systems claimed in the no single QC plan is totally adequate for all devices given
88 Contemporary Practice in Clinical Chemistry

the complexity and variety of laboratory instrumentation device or measuring system. The laboratory can never pre-
that is available on the market today. dict every possible hazard that could occur with a labora-
Use of risk management principles allows the develop- tory device, so it is never possible to entirely eliminate
ment of a QC plan that is customized to an individual risk. There will always be some residual risk that the labo-
device, laboratory setting, and clinical application. CLSI ratory should manage through its control processes to min-
guideline EP23 uses risk management to develop a labora- imize errors to a clinically acceptable level. Once
tory QC plan [13]. EP23 is intended for clinical laboratories implemented, the QC plan is monitored for the occurrence
and provides guidance on using the manufacturer-provided of unexpected errors or complaints. When errors or trends
information in the package insert, instructions for use are identified, new control processes are defined to further
and owner’s manual, the risk of errors when testing with mitigate and minimize risk, and the original QC plan is
the device, and engineered control processes available modified. In this manner, the laboratory responds to issues
on the instrumentation to develop custom, individualized with corrective and preventive actions and cycles of con-
QC plans (IQCPs). tinuous quality improvement.
Fig. 5.5 shows the process for developing a QC plan The QC plan provides documentation of the reasoning
[13]. The laboratory starts by collecting information about for laboratory QC processes for a specific device. The QC
the medical requirements of the test, local and regulatory plan defines what potential hazards the laboratory consid-
QC requirements, the measuring system, and the heath ered, the mitigations that will reduce risk of those hazards
care and laboratory settings. This information is processed to a clinically acceptable level, the frequency of QC pro-
using a risk assessment to develop a QC plan that will cesses, and how the QC plan will be monitored (baseline
meet patient care needs and local QC regulations. measures for improvement). The QC plan provides a writ-
Potential hazards or weaknesses in each step of analysis ten outline of the laboratory’s strategy for controlling the
are identified. These hazards are assessed for frequency of quality of a laboratory method and provides the rationale
occurrence and potential harm to the patient. Mitigation or for control process improvement. Regulatory inspectors
control processes are required for those hazards that have will find the QC plan sets laboratory policy for a measur-
greatest risk of harm to the patient or highest frequency of ing system’s quality by defining how the laboratory will
occurrence in order to reduce the residual risk of an error manage hazards, and inspectors can look to specific SOPs
to a clinically acceptable level. The collection of all the to determine whether the laboratory is following the ele-
control processes that are executed by the laboratory (sur- ments of the QC plan.
rogate QC sample testing plus other risk mitigation proce- Under the CLIA interpretive guidelines implemented
dures) becomes the laboratory specific QC plan for that on January 1, 2016, by the Centers for Medicare and

FIGURE 5.5 Process to develop and continually improve a quality control plan. Reproduced from the EP-23-A Document with permission from the
Clinical and Laboratory Standards Institute.
 Quality control Chapter | 5 89

Medicaid Services (CMS), laboratories in the United specimen collection has a much lower probability of spec-
States have two options: (1) perform a minimum of two imen mix-up and the chance of human error during
levels of QC for each test daily; or (2) develop an IQCP labeling.
[14]. CMS is initially requiring an IQCP for CLIA non- Minor variations in preanalytical practices can occur
waived tests. IQCPs will allow laboratories to find the throughout the testing process. Individual operator tech-
right balance of surrogate sample QC frequency with nique, specimen processing, aliquotting, dilutions, reagent
manufacturer built-in QC and control processes engi- preparation, and calibration all present a potential for
neered into the device. Laboratories will be able to utilize error. QC specimens, if handled in the same manner as
the device control processes to reduce the frequency of patient specimens, may detect some of these operational
surrogate sample QC for measuring systems that have variations, especially if the technique leads to errors or
minimal risk. In other situations, laboratories may find increased variability in results. However, QC is often han-
that they will need to increase the frequency of surrogate dled differently than patient samples and thus may not
sample QC as well as implement additional control pro- identify all sources of potential hazards. The management
cesses for high-risk measuring systems to achieve a clini- and handling of QC should therefore be considered in
cally acceptable level of risk. Inspectors will be reviewing developing or revising a QC plan.
three items as part of the IQCP inspection process: (1) the Review of the QC plan and discussion of the testing
laboratory’s risk assessment; (2) the IQCP summary; and process with those involved is also an opportunity to
(3) the laboratory’s quality assessment (e.g., the labora- uncover QC efficiencies that may not have been consid-
tory’s quality benchmarks monitored to prove the effec- ered. Many point-of-care devices act as test readers and
tiveness of the IQCP). As the philosophy of risk are simple voltmeters or light photometers with no mov-
management is more widely adopted, laboratories can ing parts. The chemistry of these devices is in the car-
expect to see additional changes to regulations that will tridges or test strips. QC can be performed on every
allow a variety of QC processes to be utilized in various device and a representative test cartridge every 24 hours,
combinations to manage measuring system quality and but with dozens of devices and locations to manage, this
more reliance on laboratory specific IQCPs to define a frequency may be excessive given the manufacturer con-
laboratory’s QC strategy. trol processes that are engineered into each test cartridge/
strip. A urine pregnancy test, for example, has a control
line that develops with each test. Analyzing a QC sample
Reviewing the quality control plan on these devices verifies the viability of that lot of test
cartridges/strips during storage but can be duplicative to
A QC plan should be reviewed at least annually or when- the internal control on that cartridge/strip. A QC plan may
ever troubleshooting of QC errors or trends indicates the therefore consider utilizing the internal control on each
need to change the QC strategy. To develop a QC plan test as the process for verifying chemical viability while
requires review of the many steps of the testing process employing the external QC samples as an event-driven
looking for potential hazards or weaknesses where errors process for confirming viability of new shipments, com-
may occur. In a central laboratory with the analyzers all parability of new lots (together with patient sample corre-
located in the same space, the analytical process may be lations), and detection of any deterioration during storage
more uniform than a point-of-care testing program with (periodic weekly or monthly analysis).
dozens of devices spread throughout an institution and Testing surrogate sample QC materials cannot be
hundreds of staff conducting testing. Discussion of the billed to a patient or reimbursed by insurance but is
testing process with representative staff can uncover var- required to manage the quality of the testing process.
iations that may present the potential for errors in one Large institutions that share a common lot of cartridges/
location that may be less likely in other locations. strips might consider performing QC on a subset of analy-
Consider specimen labeling, for instance. Printing multi- zers, since the QC is detecting the chemistry of the test
ple specimen labels in advance before specimen collec- while the devices are internally verifying the voltage and
tion may seem to be efficient but also presents the photometer readouts when devices are turned on.
potential to mix-up labels between patient samples com- Table 5.3 shows that an institution that has 20 readers
pared to on-demand printing of the label at the time of receives quarterly shipments and conducts monthly QC
specimen collection. This difference in process requires on two lots of cartridges/strips could achieve significant
an extra control step where staff must verify the correct efficiencies versus performing QC for every device and
specimen label matches the right patient and the right col- shipment. Such considerations are the heart of strategies
lection container with suitable preservative when needed. for developing a robust QC plan that meets clinical needs
This verification step may be overlooked if printing but is also cost-effective and optimizes available
occurs in advance. Printing labels on-demand during resources while ensuring high-quality test results.
90 Contemporary Practice in Clinical Chemistry

TABLE 5.3 Example comparing performing quality control (QC) for every point-of-care device versus a subset of
devicesa.
QC for every device
20 readers 3 4 quarterly shipments 3 2 levels of QC 5 160 QC tests
20 readers 3 12 monthly QC 3 2 levels of QC 3 2 lots cartridges/strips 5 960 QC tests
Total annual 5 1120 QC tests
QC for a subset of devices
3 readers 3 4 quarterly shipments 3 2 levels of QC 5 24 QC tests
3 readers 3 12 monthly QC 3 2 levels of QC 3 2 lots cartridges/strips 5 144 QC tests
Total annual 5 168 QC tests
Savings of 952 QC tests or 85% reduction QC annually
a
These estimates assume a shared source of reagents throughout the institution, four quarterly shipments, and monthly two-level QC of no more than two
lots of cartridges/strips on a point-of-care reader or simple photometer where the chemistry of the test is in the disposable cartridge/strip, the cartridge/strip
includes internal control for the chemical reaction, and the point-of-care reader includes built-in checks of its electronic performance. This example shows
one approach that may be utilized to develop a QC Plan that saves analysis of external QC in lieu of internal control processes engineered by the
manufacturer into the test system. The actual approach chosen will depend on the device, the manufacturer QC recommendations, the local regulations,
and, most importantly, the medical director approval that the QC plan covers the primary risks in the testing process to appropriately meet medical needs.

Using patient data in quality control for the delta parameter must be sufficiently large to avoid
procedures excessive numbers of false alerts; however, as the differ-
ence threshold becomes larger, the number of potential
Results from patient samples are used in four principal problems missed also increases. Kazmierczak [15] has
ways to support the QC processes in the laboratory: reviewed delta check and other patient data-based QC pro-
G to verify consistency of patient results when changing cedures. CLSI guideline EP33 describes various approaches
lots of reagent or calibrator (discussed in a previous to using delta checks for quality management [16].
section),
G to perform a delta check with a previous patient result,
G to verify consistency of patient results when an analyte Verify consistency between more than one
is measured using more than one instrument or method instrument or method
in a healthcare system, and Another common use of patient results in a QC process is
G to use a statistical process control scheme that tracks to verify consistency of patient results when an analyte is
the mean (or median) of patient results to monitor measured using more than one measuring system within
method performance. the same health system. Good laboratory practice requires
that multiple measuring systems (whether identical or
based on different method principles) for the same analyte
Delta check with a previous result for a patient
be calibrated to give clinically equivalent patient results
Some types of laboratory errors can be identified by com- whenever possible. It may be necessary to modify the cal-
paring a patient’s current test result to a previous result ibration settings of one measurement system to match
for the same analyte. This comparison is called a “delta another system’s results. This strategy allows a common
check.” Mislabeled samples and samples altered by dilu- reference interval to be used, provides continuity of
tion with IV fluid or collected with an incorrect preserva- results between patient encounters at different locations,
tive are examples of errors that can be detected using and avoids clinical confusion regarding interpretation of
delta checks. The time period between two patient results laboratory results.
must be small enough that a significant physiological One method or instrument should be chosen to repre-
change in the analyte is unlikely for a delta check to be sent the primary comparative method to which others will
effective to identify mislabeled or improperly collected be adjusted to achieve equivalent results. The primary
samples. This limitation restricts the analytes that can be method should be chosen based on quality and reliability
monitored with a delta check to those with small within- of results with consideration of calibration traceability to
individual variability. The difference between results used national or international standards, performance stability,
 Quality control Chapter | 5 91

selectivity for the analyte, and influence of interfering obtain homogeneous subgroups. Results may be trimmed
substances. CLSI guideline EP31 suggests procedures to to remove extreme values, so the remaining results are
evaluate the equivalence of patient results within a health- more reflective of patients without disease conditions that
care system [17]. The number of samples to use when influence a particular analyte. However, trimming may
verifying equivalence of patient sample results can be as remove results that are indicative of a calibration drift
few as one or more depending on the frequency of the reducing the effectiveness of the procedure. The median
assessment, the magnitude of a difference that is clinically is minimally influenced by extreme values and is recom-
meaningful, and the imprecision of the measuring sys- mended when the computer system can make the calcula-
tems. The laboratory will need to establish the frequency tion. The mean (or preferably median) for a group of
of evaluation and number of samples based on the stabil- results is tracked to monitor trends in method perfor-
ity of the methods, the frequency of reagent and calibrator mance using statistical procedures such as cumulative
lot changes, and the clinical requirements of the health sum or exponentially weighted moving average [5,8].
system. Common practices include splitting one or more These approaches can be a useful supplement to tradi-
individual patient samples, or a small pool from several tional surrogate QC materials when monitoring a single
samples, on a weekly basis for high-volume methods, and measuring system’s stability and the calibration unifor-
a monthly, quarterly, or semiannual basis for lower vol- mity between multiple measuring systems in one or sev-
ume or very stable methods. A larger number of samples eral distributed laboratories. Patient result-based
are recommended if the monitoring is performed less fre- algorithms are useful for high-volume settings but have
quently. When establishing interpretation criteria, the labo- not been widely adopted due to lack of consensus guide-
ratory needs to consider the limited statistical power for lines for their use and lack of computer support from
the number of results available and utilize trend-monitoring instrument manufacturers and laboratory information
techniques. For example, if one patient sample is tested systems.
weekly, the evaluation criteria for agreement will need to
allow for the expected imprecision of the methods and rely
more on trends to identify a measuring system that may be
Proficiency testing
performing differently than others in the group.
Results for QC materials should not be used for veri- Evaluation of method performance by an external entity
fying equivalence of patient sample results assayed using is referred to as PT or external quality assessment (EQA).
different measuring systems. Noncommutability of QC PT/EQA allows a laboratory to verify that its results are
materials with patient specimens is a common occurrence. consistent with other laboratories using the same or simi-
Even when more than one measuring system from the lar measuring systems for an analyte and thus confirms
same manufacturer is used, QC results may show differ- the measuring system is being used correctly [19]. PT/
ences between different reagent lots and instrument EQA providers circulate a set of samples among a group
models. of laboratories. Each laboratory includes the PT/EQA
samples along with patient specimens in the usual assay
process. The results for the PT/EQA samples are reported
Using patient data for statistical process control to the PT/EQA provider for evaluation.
For a sufficiently large number of patient results, the Because PT/EQA samples are typically not
mean (or median) value can be stable enough to be used commutable with native patient specimens, each reported
as an indicator of measuring system consistency over result is compared to the mean result from all laboratories
time. This approach can be used on a periodic basis by using the same measuring system or closely related ones
extracting data for a time period, for example, 1 month, (called a “peer group”). The report also includes the SD
calculating the mean and SD for the distribution of for the peer group distribution of results, the number of
results, and comparing one time period to another to laboratories in the peer group, and the SDI or z-score,
determine if any changes have occurred. Automated which expresses the reported result as the number of SD
approaches have been described to determine the mean from the mean value. The limits of acceptability may be
(or median) for groups of sequential patient results for use established by regulation or by the PT/EQA provider for
as a process control parameter. These methods are called analytes without regulatory criteria. The evaluation crite-
“average of normals” (AON) or “moving average” techni- ria are defined as a number of SD, a fixed percent, or a
ques and are suitable for use in higher volume assays in fixed concentration from the peer group mean value.
chemistry and hematology [18]. In general, these Many QC manufacturers also provide a data analysis ser-
approaches evaluate sequential patient results over time vice that calculates summary statistics for measuring sys-
intervals such as several hours to one or more days. The tem peer groups similar to those for PT/EQA providers.
patients may be grouped by age, gender, and ethnicity to This type of interlaboratory QC data allows a laboratory
92 Contemporary Practice in Clinical Chemistry

to verify that it is producing results that are consistent

with those of other labs using the same measuring system.
PT/EQA is designed to evaluate the total error of a
single measurement. The acceptability limits for PT/EQA
include laboratory bias and imprecision components plus
other error components that are unique to PT/EQA sam-
ples such as between laboratory variation in calibration,
variable matrix interaction with different lots of reagent
within a peer group, filling imprecision of the PT/EQA
material vials, stability variability in storage, shipping,
and after reconstitution or opening of PT/EQC materials
in the laboratory. Consequently, the acceptability limits
for PT/EQA specimens are frequently larger than what
might be expected for clinically acceptable total error
with patient specimens.
PT/EQA is not offered for some analytes either
because the test is new to the clinical laboratory or the
analyte stability makes it difficult to include in a PT/EQA
material. In these situations, the laboratory should use an
alternate approach to verify acceptable performance of
FIGURE 5.6 Example of noncommutability between proficiency test-
the method. CLSI guideline GP27 provides several ing materials and native patient sample pools for a specific method.
approaches to assess measuring system performance when Adapted with permission from H.K. Naito, Y.S. Kwak, J.L. Hartfield, J.K.
formal PT/EQA is not available [20]. Park, E.M. Travers, G.L. Myers, et al. Matrix effects on proficiency test-
ing materials: impact on accuracy of cholesterol measurement in labora-
tories in the nation’s largest hospital system. Arch. Pathol. Lab. Med.
117 (1993) 345 351.
Noncommutability of proficiency testing/
external quality assessment materials and peer laboratories using the Dimension method because the matrix
group grading interference was uniform within this peer group. Thus if an
PT/EQA results are usually evaluated through peer group individual laboratory’s results agreed with those of the peer
comparison because of the noncommutability of the mate- group, the individual laboratory could conclude the measur-
rials typically used as PT/EQA specimens. Matrix inter- ing system was performing as expected. However, the accu-
ference is unique to each combination of measuring racy of the Dimension method for patient specimens could
system and processed PT/EQA sample (just as for QC or not be evaluated based on the PT/EQA results.
reference materials) and can be quantitatively different
for different method, instrument, and material combina- Reporting proficiency testing/external quality
tions for the same analyte. Several investigations have assessment results when one method is adjusted
reported .60% incidence of noncommutable materials
to agree with another method
for different analytes [1,11].
Fig. 5.6 illustrates the effect of noncommutable materials Good laboratory practice that is consistent with CLIA
on interpretation of PT/EQA results. In this example, pooled Regulations Section 493.1281 recommends adjusting the
native patient sera and PT/EQA samples were assayed by calibration of different measuring systems for the same
the DuPont Dimension analyzer and by the Abell Kendall analyte, within a health system, so the results for patient
reference method for cholesterol [21]. The patient samples specimens are comparable irrespective of analytical
showed excellent agreement between the two methods (aver- method. In this situation, PT/EQA results must be
age bias 5 0.2%). However, the PT/EQA materials had a reported correctly so they can be properly evaluated
large negative bias (29.5%) with this measuring system that against the appropriate peer group. The peer group target
was due to noncommutability between the PT/EQA samples value reflects the method calibration established by the
and the patient specimens with the Dimension measuring measuring system manufacturer. For an individual labora-
system. If the apparent accuracy of the routine method was tory’s method with adjusted calibration, PT/EQA results
compared against the reference method based on PT/EQA must be reported to the PT/EQA provider with any user-
results, and the calibration was erroneously adjusted, the applied calibration adjustment removed so the reported
results for patient specimens would then be incorrect. PT/ result can be compared to the manufacturer’s calibration.
EQA results were still valuable to judge the performance of The most convenient way to remove a calibration
 Quality control Chapter | 5 93

adjustment is to first assay the PT/EQA samples with the measuring system. These practices support identification
calibration adjustment applied to the measuring system as of potential problems before they progress to more serious
would be the usual assay process for patient samples. situations.
After analysis, the PT/EQA results are corrected by math- PT/EQA results are always received several weeks or
ematically removing the calibration adjustment. One more after the date of testing. Consequently, a review of
should not recalibrate the instrument with a new set of QC, reagent, calibration, and maintenance records for the
calibrators for the purpose of analyzing PT/EQA speci- date of the testing and the preceding several weeks or
mens because this practice would violate regulations months is necessary. If review of these records suggests a
requiring the PT/EQA material to be assayed in the same stable operating condition, and review of the PT/EQA
manner as patient specimens. material handling and documentation does not identify a
For example, a laboratory has performed a native cause for the erroneous PT/EQA result, it may be con-
patient specimen comparison between Method A used in cluded the PT/EQA failure was a random event. The
the main lab and Method B used in a satellite laboratory. investigative steps, data reviewed, and conclusions from
Method B consistently generates results that are 10% the review must be documented in a written report of the
higher than Method A, that is, a slope of 1.10 with negli- unacceptable PT/EQA result and reviewed by the labora-
gible intercept. Method B can be adjusted to agree with tory director.
Method A by automatically multiplying Method B’s PT/EQA providers also include a summary report that
results by 1/1.10 5 0.91 to lower the results by 10% includes the mean and SD/range for all the peer groups
before reporting (typically performed by the instrument’s represented by the participants’ results. Similar reports
computer as a post measurement adjustment). When are available from interlaboratory QC programs.
reporting PT/EQA results from Method B, it is necessary Summary reports are useful but must be interpreted
to remove the 0.91 factor to allow the reported result to with consideration of the limitations of noncommutable
be evaluated against the appropriate Method B peer samples. The peer group mean and SD can be used to
group mean that will reflect the calibration established evaluate the uniformity of results between laboratories in
by the method manufacturer and allow the laboratory to the same peer group, to confirm an individual laboratory
assess its performance versus the peer group. Removing is using a measuring system in conformance to the manu-
the 0.91 factor is accomplished by multiplying the facturer’s specifications, and to evaluate the consistency
reported PT/EQA result from Method B by the factor 1/ of an individual laboratory’s performance relative to the
0.91 5 1.10 to increase its numeric value by 10%. The peer group over extended periods of time from one
numeric result reported to the PT/EQA provider reflects PT/EQA event to the next (trend monitoring). The sum-
the actual measured result using the manufacturer’s mary information also allows evaluation of the impreci-
recommended calibration settings. This process allows sion of different types of measuring systems, and the
the PT/EQA sample to be assayed in the same manner as number of users of each peer group reflects the market
the patient specimens and the PT/EQA result to be share for different measuring systems.
appropriately evaluated by comparison to its peer group. Because PT/EQA samples are commonly
noncommutable with patient samples, PT/EQA summary
reports cannot be used to compare: (1) the agreement or
Interpretation of proficiency testing/external disagreement of patient results among different peer
groups; (2) peer group means for different measuring sys-
quality assessment results
tems to each other; or (3) an individual measuring system
When an unacceptable PT/EQA grade is received from a peer group mean to a value assigned by a reference
PT program, the measuring system should be investigated method. Noncommutability also prevents any inference of
for possible causes and corrective action taken as neces- the agreement for patient results among different measur-
sary to correct any problems. Even when a PT/EQA grade ing systems.
is within acceptable limits, good laboratory practice
recommends investigating any PT/EQA results that are
greater than B2.5 SDI from the peer group mean. When Accuracy-based proficiency testing/external
the SDI is 2.5, there is only a 0.6% probability of the
quality assessment programs
result being within the expected distribution for the peer
group; consequently, there is a reasonable probability of a PT/EQA providers offer programs that use commutable
measuring system problem that needs to be investigated. samples for some analytes. Commutable samples are typi-
In addition, PT/EQA results that have been near the fail- cally prepared by pooling native clinical samples with mini-
ure limit for more than one PT/EQA event, even if the mal processing or additives to avoid any alteration of the
results have passed the PT/EQA acceptance criteria, sample matrix. To achieve samples with abnormal concen-
should initiate a review for systematic problems with the trations of analytes, donors can be identified with known
94 Contemporary Practice in Clinical Chemistry

pathologic conditions, or blood and serum units from a gen- [8] T.P. Ryan, Statistical Methods for Quality Control, Wiley, New
eral donor population can be prescreened for selected ana- York, 1989, pp. 82 178.
lytes. When commutable PT/EQA samples can be prepared, [9] J.O. Westgard, P.L. Barry, M.R. Hunt, A multi-rule Shewhart
then the results from all laboratories reflect results that chart for quality control in clinical chemistry, Clin. Chem. 27
(1981) 493 501.
would be expected if native patient specimens were sent to
[10] CLSI, User Evaluation of Between-Reagent Lot Variation:
each of the different laboratories. Thus agreement between
Approved Guideline EP26-A, Clinical and Laboratory Standards
laboratories (called “harmonization”) and between different Institute, Wayne, PA, 2013.
types of measuring systems can be correctly evaluated. The [11] W.G. Miller, G.L. Myers, Commutability still matters, Clin.
agreement between an individual laboratory result and a ref- Chem. 59 (2013) 1291 1293.
erence measurement result (accuracy), and the agreement [12] W.G. Miller, J.R. Tate, J.H. Barth, G.R. Jones, Harmonization: the
between a measuring system mean and a reference method sample, the measurement and the report, Ann. Lab. Med. 34
result (called trueness), can be evaluated when a reference (2014) 187 197.
method is available for an analyte. [13] CLSI, Laboratory Quality Control Based on Risk Management:
For example, the College of American Pathologists Approved Guideline EP23-A, Clinical and Laboratory Standards
Glycohemoglobin Survey has for many years used fresh, Institute, Wayne, PA, 2011.
[14] Department of Health and Human Services. Centers for Medicare
pooled whole blood from both normal and diabetic donors.
& Medicaid Services, IQCP announcement letter for CLIA CoC
The target values for the pooled blood are assigned by refer-
and PPM laboratories, September 30, 2013. ,[Link]
ence measurement procedures for hemoglobin A1c. In this gov/Regulations-and-Guidance/Legislation/CLIA/
survey, the accuracy of individual laboratory results, and Individualized_Quality_Control_Plan_IQCP.html. (accessed
the trueness of measuring system mean values, can be eval- April 2018).
uated because the PT/EQA materials are commutable with [15] S.C. Kazmierczak, Laboratory quality control: using patient data
native patient specimens. The measuring system trueness to assess analytical performance, Clin. Chem. Lab. Med. 41
can be used by the respective manufacturers to monitor the (2003) 617 627.
effectiveness of their calibration processes. [16] CLSI, Use of Delta Checks in the Medical Laboratory: Approved
PT/EQA programs have included commutable samples Guideline EP33-A, Clinical and Laboratory Standards Institute,
on occasion to evaluate individual laboratories and mea- Wayne, PA, 2016.
[17] CLSI, Verification of Comparability of Patient Results Within
suring systems groups for agreement with reference meth-
One Healthcare System, Approved Guideline (Interim Revision)
ods and for harmonization of results between laboratories
EP31-A-IR, Clinical and Laboratory Standards Institute, Wayne,
and measuring systems groups [19]. Use of commutable PA, 2012.
materials adds substantial value to the information avail- [18] D. Ng, F.A. Polito, M.A. Cervinski, Optimization of a moving
able from the results of PT/EQA programs. averages program using a simulated annealing algorithm: the goal
is to monitor the process not the patients, Clin. Chem. 62 (2016)
1361 1371.
References [19] W.G. Miller, G.R.D. Jones, G.L. Horowitz, C. Weykamp,
[1] W.G. Miller, Specimen materials, target values and commutability Proficiency testing/external quality assessment: current challenges
for external quality assessment (proficiency testing) schemes, and future directions, Clin. Chem. 57 (2011) 1670 1680.
Clin. Chim. Acta. 327 (2003) 25 37. [20] CLSI, Using Proficiency Testing to Improve the Clinical Laboratory:
[2] W.G. Miller, A. Erek, T.D. Cunningham, O. Oladipo, M.G. Scott, Approved Guideline GP27-A3, Clinical and Laboratory Standards
R.E. Johnson, Commutability limitations influence quality control Institute, Wayne, PA, 2016.
results with different reagent lots, Clin. Chem. 57 (2011) 76 83. [21] H.K. Naito, Y.S. Kwak, J.L. Hartfiel, J.K. Park, E.M. Travers, G.
[3] US Department of Health and Human Services, Medicare, L. Myers, et al., Matrix effects on proficiency testing materials:
Medicaid, and CLIA programs; laboratory requirements relating impact on accuracy of cholesterol measurement in laboratories in
to quality systems and certain personnel qualifications; final rule. the nation’s largest hospital system, Arch. Pathol. Lab. Med. 117
42 CFR Part 493, Fed. Regist. 68 (2003) 3639 3714. (1993) 345 351.
[4] CLSI, Statistical Quality Control for Quantitative Measurement
Procedures: Principles and definitions: Approved Guideline C24-
A4, Clinical and Laboratory Standards Institute, Wayne, PA, 2016.
[5] A.S. Neuhauer, The EWMA control chart: properties and compari-
Further reading
son with other quality-control procedures by computer simulation, U.S. Department of Health and Human Services, Centers for Medicare
Clin. Chem. 43 (1997) 594 601. and Medicaid Services. Appendix C, Survey procedures and inter-
[6] K. Linnet, Choosing quality-control systems to detect maximum pretive guidelines for laboratories and laboratory services, Medicare,
clinically allowable errors, Clin. Chem. 35 (1989) 284 288. Medicaid, and CLIA programs; laboratory requirements relating to
[7] C.A. Parvin, A.M. Gronowski, Effect of analytical run length on quality systems and certain personnel qualifications; final rule on
quality-control (QC) performance and the QC planning process, January 24, 2003. CMS-2226-F: 42 CFR 493. ,[Link]
Clin. Chem. 43 (1997) 2149 2154. [Link]/clia/[Link]. (accessed April 2018).
 Quality control Chapter | 5 95

Self-assessment questions 6. Which of the following criteria should be used to

select rules for evaluation of QC results?
1. What is the primary purpose of QC (statistical pro- a. frequency of false alerts
cess control)? b. probability to detect significant bias
a. to determine that a method is adequate for clini- c. probability to detect significant imprecision
cal requirements d. probability to detect trends in bias
b. to verify that a method is performing as expected 7. Which of the following are considerations for how
for its stable operating condition frequently to run QC samples?
c. to establish the uncertainty in a laboratory result a. stability of the measurement system
d. to ensure that regulatory requirements are met b. use of built-in controls
2. Which of the following criteria should be used to c. clinical impact of reporting an incorrect result
select materials to use for QC? d. cost to repeat questionable results following a
a. analyte stability after opening QC alert
b. analyte concentrations at low and high values rel- 8. Why might it be necessary to adjust QC target values
ative to the analytical measurement range following a reagent lot change?
c. a matrix similar to that of clinical samples a. The calibration could be incorrect.
d. number of analytes in the same vial b. The matrix interaction between the QC material
3. How should the target value for a QC material be and the new reagent could be different than with
assigned? the old reagent.
a. by assaying one bottle 10 times over 5 days c. The method could have changed.
b. by assaying 10 bottles on each of 10 days d. The original target value could have been
c. by assaying several bottles on each of 10 days incorrect.
while allowing the open-vial stability to age 9. Why are QC or PT results not used to evaluate
d. by using the manufacturer’s assigned value agreement between different methods?
4. How is the expected variability for a stable measure- a. The concentrations are different in different
ment system established? samples.
a. by calculating the SD after a full month of QC b. The within-peer group SD and CV include cali-
results bration variability between laboratories.
b. by calculating the cumulative SD over a 6-month c. The peer groups may not have enough partici-
interval pants for reliable statistics.
c. by averaging the monthly SD over 6 months d. The specimens used are frequently noncommutable
d. by calculating the cumulative SD over a long with native clinical samples.
enough time period that all expected sources of 10. Which of the following components of the QC pro-
variability have been included in the data gram are documented in the SOP and review logs?
5. Which of the following sources of measurement vari- a. the specimen target values and SDs used for
ability need to be included in the SD used to evaluate evaluation
a method’s performance? b. the results for each measurement
a. instrument maintenance cycles c. troubleshooting performed to resolve a QC alert
b. reagent and calibrator lot changes d. the acceptance criteria used to evaluate the QC
c. QC material open-vial stability results
d. instrument parameters such as pipette stability,
temperature stability, and dirt accumulation

Answers
1. b
2. a, b, c, d
3. c
4. d
5. a, b, c, d
6. a, b, c, d
7. a, b, c, d
8. b
9. d
10. a, b, c, d