Chapter-2
laboratory method selection, validation and
verification
Introduction
• ISO/IEC 17025 requires the use of appropriate test methods and
has requirements for selection, development, and validation
• Clients and laboratory staff must have a clear understanding of
what performance can be expected from a test
• Many laboratories will therefore need to demonstrate competence
in the development, adaptation and validation of test methods.
• Many laboratories develop or modify methods , the others use
non-standard methods, or a combined
• Some laboratories even use in-house verifications, optimization,
and/or validation generally must be done to ensure valid results
Method validation vs verification
❖ Method validation is a comprehensive process to prove that a
new test method is accurate, precise, reliable, and fit for its
intended clinical purpose
➢ It is done typically for new methods or new instruments that
have never been used in the laboratory
➢ Approve method for routine use
❖ Method verification is a confirmatory process to ensure that
an already validated or commercially available method
performs as expected in our specific laboratory setting
➢ It is done for methods/instruments already validated by the
manufacturer but new to our lab
➢ Confirm reliability locally
Method Selection
❖ The introduction of new or revised methods is a common
occurrence in the clinical laboratory
❖ A new or revised method: mainly based on changes in
principle, procedure, technology, reagents, or
performance characteristics
❖ New method involves introducing a new analytical
principle, technology, or test system
❖ Revised method refers to refers to a previously validated
method that has undergone changes(like modified reagent,
instrument, or procedure) that may influence performance
❖ New methods require validation to establish performance,
while revised methods require verification to confirm
continued reliability
Cont’d…
Why new/ revised method need to be required?
➢ To improve quality
➢ to reduce costs
➢ To improve efficiency
➢ To satisfy clients
Cont’d…
Process to introduce new method
1. Pre purchase assessment
2. Installation and calibration
3. Familiarization period
4. Method verification
5. Introduction in to routine service
Cont’d…
Pre purchase assessment
➢ Performance characteristics (with regard to quality
requirements)
✓ is the method well validated?
✓ How is good when compared to others?
➢Cost
✓ Is a cost of a new method being affordable?
Cont’d…
Pre purchase assessment
➢System and supplies
✓Is there a need of power supply ? How much ?
✓Are accessories available?
✓Is closed or open system?
✓ are reagents easily available?
Cont’d…
Pre purchase assessment
➢Maintenance service and spare part
availability
✓Does the manufacturer of the method will
provide maintenance service?
✓can we easily make maintenance service
during failure?
✓Are spare parts available?
Cont’d…
Pre purchase assessment
➢Environmental requirement
✓ the required working temperature, humidity ?
✓The space required for storage/ physical setup
▪ Laboratory methods vary widely in
complexity: Simple, intermediate, complex
Cont’d…
Pre purchase assessment
➢Level of technical expert needed
✓ Can the manufacturer will provide training?
✓Is it easy to use with minimum training ?
Cont’d…
2. Installation and calibration
➢ Set up the method in the physical location
identified for routine operation
➢ Usually service engineer will perform installation,
system checks and calibration
➢ Calibration is the process of configuring an
instrument to ensure that its measurements are
accurate and consistent with a known standard
➢ When calibration? After installation, regular
maintenance, QC fails, environmental changes
Cont’d…
➢ System calibration refers to the comprehensive process of
adjusting and verifying an entire laboratory testing system
Software Calibration:
• Verifies that the software accurately interprets and
processes data from the instrument
Instrument Calibration:
• Aligns individual equipment, such as analyzers,
spectrophotometers, or pipettes, with a reference standard
➢ Retain all installation and calibration paper works
(documentation)
Cont’d…
3. Familiarization period
➢Staff must be familiar with:
✓Operation
✓Maintenance procedure
✓Method of sample preparation, IQC,
calibration and monitoring functions
Cont’d…
4. Method verification
➢ Once the instrument or the test kit has been selected
or installed, the next step is method verification
➢ Method verification is applied to assess analytical
errors for the method which has been validated by
developers of the method
➢ Unlike method validation, which is broader and
required for developing a method from scratch,
verification focuses on ensuring that a pre-established
method works correctly under local conditions
Cont’d…
Why method verification is important
➢Differences may happen due to:
✓Climate conditions
✓Shipping and storage condition
✓Skill level of the operators
✓Changes in instrument or system components
Cont’d…
Method verification steps
➢ Define quality requirement(TE of lab < allowable error)
➢ Select the appropriate types of experiment(verification
method) to reveal analytical errors
➢ Collect experimental data
➢ Use statistical tools on the data to estimate size of
analytical errors
➢ Compare the observed errors with the predefined
allowable error
➢ Judge the acceptability observed performance
Cont’d…
➢ Method verification is done through two ways :
➢ in the absence of a comparative method
➢ with the presence of comparative method
Method verification in the absence of a comparative
method
1. Linearity check
❖Linearity defined as ability of analytical procedure
(within a given linearity range) to obtain results
that are directly proportional to the concentration
of analyte in the sample
❖Linearity check confirm that the new test method
produces results that are directly proportional to
the analyte concentration over the expected
measuring range — showing that the instrument or
method responds accurately across low to high
levels
❖Used to asses the constant and proportional
systematic errors
Cont’d…
Procedure
➢ Prepare or obtain samples with known analyte concentrations
spanning the full expected range (from low to high)
✓ These can be made by serially diluting a high-concentration sample
with a blank or low-level sample
➢ Testing: Measure each standard using the method or instrument
➢ Plot Results: Place the expected value on the X-axis and the obtained
value on the Y-axis
• If the method is linear the line should have a slope of one and a Y-
intercept zero.
• If the method is not linear over its published range stop the analysis
and investigate the problem
• Accept the method’s linearity if the line of agreement has a slope of
1.00 ±0.05 and a y- intercept of zero
Linearity Example (Cholesterol)
600.00
500.00
400.00
Measured Value
300.00
200.00
100.00
y = 1.001x - 0.289
0.00
0 100 200 300 400 500 600
Bottle Value
Decision Levels 1 2 3 4
Cholesterol (mg/dL) 90 240 260 350
Cont’d…
➢ Linearity can be assessed through regression
analysis, where the correlation coefficient (R²)
measures the strength and direction of the linear
relationship between two variables — expected
concentration and measured concentration
▪ R² values range from 0 to 1
▪ R² = 1 indicates a perfect fit
▪ R² = 0 indicates no relationship
▪ For most analytical methods, an R² value of ≥ 0.99
is considered excellent and indicates a very good
linear relationship between the analyte
concentration and the measured response
2. Replicate experiment (study)
▪ Replicating an experiment means performing the same
analysis multiple times to evaluate the consistency of results
• replicate experiments are used to assess the precision and
reliability of an analytical method
➢ Within-run precision(short term): also called repeatability
✓ Repeated measurements on the same instrument during a
single analytical run
✓ Evaluates instrument stability and operator consistency over
a short period
➢ Between-run precision(long term):also called reproducibility
✓ Repeated measurements on different days or across different
batches or runs
✓ Evaluates method stability over time, including effects of
reagent changes, calibration shifts, or environmental
differences
Cont’d…
• replicate experiment also can be done on three types of
concentration levels
low concentrations
normal concentrations
high concentration
❖ Perform multiple measurements (at least 20-30) on the same
sample using the method being verified, under the same
operational conditions
Cont’d…
Statistical analysis
– Mean = ∑xi
n
– SD or s = √∑(xi-mean)2
N–1
– CV = s X 100
mean
• Determine whether short-term and long-term imprecision is acceptable based
on total allowable error
• within-run standard deviation should be ¼ or less of the defined allowable
total error
• For long-term imprecision, the total standard deviation should be 1/3 or less
of the defined TE
• Between-run precision is better indicator of the methods over all precision
Cont’d…
Interpretation
❖ The greater the standard deviation, the greater the
imprecision of the method
❖ In normal condition 95% of the values should fall with in the
±2SD
❖ Large Co-efficient of variation indicates: the method lacks
precision
Precision Example – Cholesterol
Within-run Control A
Mean 199.2
Std. Dev. 5.84
CV 2.93
• Cholesterol Target value(allowable total error)= ± 10%
• Within-run coefficient of variation limits = 10%/4 = 2.5%
• Within-run standard deviation limits= (199.2*10%)/4= 4.98
• Is this method acceptable based on the within-run standard deviation limits?
3. Recovery Studies
❖ A recovery study is an experiment used to evaluate the
accuracy of an analytical method — that is, how well
the method can measure a known amount of analyte
added (spiked) to a sample
❖ In simple terms: it checks how much of the added
substance the method can “recover”
❖ Recovery studies help determine how well the
analytical method is able to measure the true amount
of an analyte in a sample, by comparing the observed
concentration to the known amount of analyte that
was added
❖ A method is considered accurate if it provides results
close to the true value of the analyte
cont’d…
Procedure (basic steps):
• Take a sample with a known or low analyte
concentration.
• Add (spike) a known quantity of the analyte to
the sample
• Measure the analyte concentration before
and after spiking.
• Calculate recovery (%)
Cont’d…
Recovery(%)=(Measured after spike−Original)/added amount×100
Interpretation:
Recovery (%)= 95–105%- Excellent accuracy
Recovery (%)= 90–110%- Acceptable for many clinical methods
Recovery (%) =<90% or >110%- Indicates potential method error
Cont’d…
• Sample 1:
2.0 mL serum +
0.1 mL H2O
• Sample 2:
2.0 mL serum +
0.1 mL 20 mg/dL Ca Std.
• Sample 3:
2.0 mL serum +
0.1 mL 50 mg/dL Ca Std.
Cont’d…
• Concentration added =
std concentration x mL standard
mL std + mL serum
• Example
Conc. added =
20 mg/dL x 0.1 mL standard
0.1 mL std + 2.0 mL serum
= 0.95 mg/dL
Result Concentration Concentration Concentration % recovered
measured added recovered
Sample 1 7.2 mg/dl 0.0 mg/dl 0.0 mg/dl 0
Sample 2 9.0 mg/dl 2.0mg/dl 1.8 mg/dl 90 %
Sample 3 9.9 mg/dl 3.0 mg/dl 2.7 mg/ dl 90%
Calculation of ca recovered= Concentration measured in the test sample
─concentration measured in the base line.
For sample number 2 =9.0mg/dl─7.2mg/dl=1.8 mg /dl
For sample number 3 =9.9─7.2 mg/ dl =2.7 mg/dl
Calculation of % recovered =Concentration recovered X100%
Concentration added
For sample number 2= 1.8 mg/dl X100 % =90 %
2 mg/dl
For sample number 3 = 2.7 mg/dl X100 % =90%
3 mg /dl
Cont’d…
• Error=0.2 mg /dl and 0.3mg/ dl for sample
number 2 & 3 respectively.
• Under this experiment the method has 90 %
recovery rate or 10% proportional systematic
error
• Acceptable recovery or accuracy is based on
Westgard rule limit of acceptability and this
occurs when the percentage of error is less than
the determined allowable analytical error (95%
of the limit of allowable error) at the medical
decision level
4. Interference study
❖ Interferences can falsely increase or decrease the measured
analyte concentration.
❖ Interference study helps to evaluate how various substances,
either present in the sample matrix or introduced externally,
affect the accuracy and performance of the analytical method
❖ The interference study helps to ensure that the method is
specific to the analyte of interest
❖ It identifies substances that may alter the method
Cont’d…
Procedure
❖Add suspected interfering material to a sample
containing the analyte
– Eg. add hemoglobin or bilirubin
❖Analyze the sample by the test method
❖Determine difference
Cont’d…
➢ The percentage of analyte recovered in the presence of the
interferent can be calculated and compared to the acceptable
range
➢ If the recovery is significantly lower (e.g., <80% or >120%) than
expected, it may indicate interference
➢ Percent Recovery=(Measured analyte concentration)/
(Expected concentration)×100
➢ Typically, a method is considered acceptable if interference does
not alter the analyte concentration by more than 10-20%
Interference Example
Ca measured Mg++ added Interference
Sample 1 9.80 mg/dL
Sample 2 10.53 mg/dL 0.91 mg/dL 0.73 mg/dL
Sample 3 11.48 mg/dL 1.81 mg/dL Answer key
Average Interference Answer key
CLIA’s Acceptable Level = Target value ± 1.0 mg/dL
Is the performance of this method acceptable? Answer key
Method verification in the presence of comparative
methods
❖Comparative/reference method is a high-
accuracy, well-established method with known
precision and accuracy, often traceable to a
national or international standard
❖Reference method used as the benchmark for
evaluating the performance of the new method
Cont’d…
Procedure:
➢ Choose patient samples or control materials that
span the full analytical measurement range (AMR)
of the new method
➢ Use at least 40–100 samples for robust statistical
comparison
➢ Test the same samples using both the new method
and the reference method
➢ Use appropriate statistical methods to evaluate
agreement between the new and reference
methods: Linear regression analysis
Cont’d…
➢ Linear regression analysis is a key statistical tool used in
method verification to compare a new analytical method
against a reference method
➢ Obtain paired results by testing the same samples with
both the new and reference methods
➢ Plot the results of the new method (y) against the
reference method (x) on a scatterplot
➢ Fit a straight line(regression line) to the data: y=mx+b,
where m= slope of the line, b = y-intercept (value of y
when x=0)
➢ The key regression parameters: slope(m), intercept(b)
and Coefficient of Determination (R²) can be calculated
with statistical software like SPSS or others
• Here is a graph illustrating the Coefficient of Determination (R²)
• Blue points: The observed data values
• Red line: The regression line (best-fit line predicted by the model)
• R² : The value displayed in the legend quantifies how well the
regression line explains the variability in the data. In this example,
R² ≈0.99, indicating an excellent fit
Cont’d…
R2 = 1- (SSres/ SStot)
where, SSres (Residual Sum of Squares): The sum of squared differences
between observed values (yi) and predicted values (y^i)
SSres=∑(yi−y^i)2
SStot (Total Sum of Squares): The sum of squared differences between observed
values (yi) and their (yˉ)
SStot=∑(yi−yˉ)2
Cont’d…
Parameter Ideal Value Interpretation
Indicates proportional
Slope (m) ≈1
agreement
Intercept (b) ≈0 No constant bias
High correlation between
R2 ≥0.95
method
4. Interference study
❖ Interference study helps to evaluate how various substances,
either present in the sample matrix or introduced externally,
affect the accuracy and performance of the analytical method
❖ The interference study helps to ensure that the method is
specific to the analyte of interest and that other components in
the sample do not interfere with its accurate measurement
❖ It identifies substances that may alter the method’s response,
allowing laboratory personnel to address these issues before the
method is used routinely
❖ An interference study is a vital step in method verification to ensure that the analytical method is accurate
and specific
Cont’d…
Procedure
❖Add suspected interfering material to a sample
containing the analyte
– Eg. add hemoglobin or bilirubin
❖Analyze the sample by the test method
❖Determine difference
Cont’d…
➢ The percentage of analyte recovered in the presence of the
interferent can be calculated and compared to the acceptable
range
➢ If the recovery is significantly lower (e.g., <80% or >120%) than
expected, it may indicate interference
➢ Percent Recovery=(Measured analyte concentration)/
(Expected concentration)×100
➢ Typically, a method is considered acceptable if interference does
not alter the analyte concentration by more than 10-20%
Interference Example
• Usually use a target value of the upper reference
interval x acceptable error
• Published Acceptable error for Calcium = Target value
± 1.0 mg/dL
• Upper Reference Interval = URI
• URI = Ca URI = 8.5 - 9
• Is the performance of this method acceptable?
Example of Published Total Allowable
Error Limit
Routine Chemistry tests
Test or Analyte Acceptable Performance
Alanine aminotransferase Target value ± 20%
Alkaline phosphatase Target value ± 30%
Aspartate aminotransferase (AST) Target value ± 20%
Bilirubin, total Target value ± 0.4 mg/dL or ± 20% (greater)
Calcium, total Target value ± 1.0 mg/dL
Cholesterol, total Target value ± 10%
Creatinine Target value ± 0.3 mg/dL or ± 15% (greater)
Glucose Target value ± 6 mg/dL or ± 10% (greater)
Urea Nitrogen Target value ± 2 mg/dL or ± 9% (greater)
CLIA’s Acceptable Level
Interference Study Example
Ca measured Mg added Measured
Interference
Sample 1 9.80 mg/dL 0 0
(baseline)
Sample 2 10.53 mg/dL 0.91 mg/dL 0.73
Sample 3 11.48 mg/dL 1.81 mg/dL
Interference = 10.53 mg/dL - 9.80 mg/dL
= 0.73 mg/dL
Example of Published Total Allowable
Error Limit
Routine Chemistry tests
Test or Analyte Acceptable Performance
Alanine aminotransferase Target value ± 20%
Alkaline phosphatase Target value ± 30%
Aspartate aminotransferase (AST) Target value ± 20%
Bilirubin, total Target value ± 0.4 mg/dL or ± 20% (greater)
Calcium, total Target value ± 1.0 mg/dL
Cholesterol, total Target value ± 10%
Creatinine Target value ± 0.3 mg/dL or ± 15% (greater)
Glucose Target value ± 6 mg/dL or ± 10% (greater)
Urea Nitrogen Target value ± 2 mg/dL or ± 9% (greater)
CLIA’s Acceptable Level
Interference Study Example
Ca measured Mg added Measured
Interference
Sample 1 9.80 mg/dL 0 0
(baseline)
Sample 2 10.53 mg/dL 0.91 mg/dL 0.73 mg/dL
Sample 3 11.48 mg/dL 1.81 mg/dL ??
Average Interference ??
Method verification in the presence of comparative
methods
❖Comparative/reference method is a high-
accuracy, well-established method with known
precision and accuracy, often traceable to a
national or international standard
❖Reference method used as the benchmark for
evaluating the performance of the new method
Cont’d…
Procedure:
➢ Choose patient samples or control materials that
span the full analytical measurement range (AMR)
of the new method
➢ Use at least 40–100 samples for robust statistical
comparison
➢ Test the same samples using both the new method
and the reference method
➢ Use appropriate statistical methods to evaluate
agreement between the new and reference
methods: Linear regression analysis
Cont’d…
➢ Linear regression analysis is a key statistical tool used in
method verification to compare a new analytical method
against a reference method
➢ Obtain paired results by testing the same samples with
both the new and reference methods
➢ Plot the results of the new method (y) against the
reference method (x) on a scatterplot
➢ Fit a straight line(regression line) to the data: y=mx+b,
where m= slope of the line, b = y-intercept (value of y
when x=0)
➢ The key regression parameters: slope(m), intercept(b)
and Coefficient of Determination (R²) can be calculated
with statistical software like SPSS or others
• Here is a graph illustrating the Coefficient of Determination (R²)
• Blue points: The observed data values
• Red line: The regression line (best-fit line predicted by the model)
• R² : The value displayed in the legend quantifies how well the
regression line explains the variability in the data. In this example,
R² ≈0.99, indicating an excellent fit
Cont’d…
R2 = 1- (SSres/ SStot)
where, SSres (Residual Sum of Squares): The sum of squared differences
between observed values (yi) and predicted values (y^i)
SSres=∑(yi−y^i)2
SStot (Total Sum of Squares): The sum of squared differences between observed
values (yi) and their (yˉ)
SStot=∑(yi−yˉ)2
Cont’d…
Parameter Ideal Value Interpretation
Indicates proportional
Slope (m) ≈1
agreement
Intercept (b) ≈0 No constant bias
High correlation between
R2 ≥0.95
method
Kappa analysis
➢ Kappa analysis (or Cohen's Kappa coefficient) is a statistical
measure used to assess the agreement between two raters or
methods for categorical data
➢ It corrects for the agreement that might occur by chance,
providing a more reliable measure of agreement than simple
percentage agreement.
➢ Kappa is particularly useful when you are comparing results
from two different methods or observers that produce
categorical (nominal or ordinal) data, such as diagnostic
classifications (positive/negative, normal/abnormal)
Cont’d.
• It compares the agreement against that which might be
expected by chance
• The Cohen's Kappa coefficient is calculated as:
kappa = Observed agreement - Chance Agreement
Total observed-Chance agreement
• Or observed-expected/1-expected
Hypothetical Example:
29 patients are examined by two independent doctors (see Table). 'Yes' denotes the patient is diagnosed
with disease X by a doctor. 'No' denotes the patient is classified as no disease X by a doctor.
Doctor A
No Yes Total
No 10 (34.5%) 7 (24.1%) 17
(58.6%)
Doctor B
Yes 0 (0.0%) 12 (41.4%) 12
(41.4%)
Total 10 (34.5%) 19 (65.5%) 29
Observed agreement = (10 + 12)/29 = 0.76
Chance agreement = 0.586 * 0.345 + 0.655 * 0.414 = 0.474
Kappa = (0.76 - 0.474)/(1 - 0.474) = 0.54
Two laboratory technologists determine the hemoglobin values of 120
patients using hemocue methods. The following table showed the individual
results.
LAB
Personnel I
Anemic Anemic Non anemic Total
LAB Anemic 40 60 100
Personnel II
Nonanemic 50 50 100
Total 90 110 200
❖What is the kappa value of the two readers to diagnose anemia using hemocue
methods.
Show all the steps to calculate the kappa values and interpret the level of anemia in the
two readers/ laboratory technologists
Cont’d…
Interpretation of kappa values (K)
< 0 = No agreement
0.0-0.19 = Poor agreement
0.20-0.39 = Fair agreement
0.40-0.59 = Moderate agreement
0.60-0.79 = Substantial agreement
0.80-1.00 = Almost perfect agreement
References
1. Lawrence Kaplan, Clinical Chemistry, theory, analysis,
correlation, Fourth edition, Mosby Inc., 2003.
2. Endris Mokonnen, Health Laboratory Management and
quality assurance lecture note, carter centre, the Ethiopian
public heath initiative, 2004.