Introduction

• Method: Detailed (defined) procedure of a

technical operation for performing an analysis.

Major classification of methods

– Quantitative Method
Numerically derived test results
– Qualitative Method
– Express as presence/absence, positive/negative
reactive/non-reactive, etc
 Introduction
• Type of Methods
– Standard Method
– Peer review, published. National/international etc
– Verification is required

– Non Standard/
• Modified standard method
• in house method
• Validation is required
 Validation & Verification
• Validation: is conformation through provision of
objective evidence, that the requirements for specific
intended use or application have been fulfilled
– Non Standard/modified/in house developed method

• Verification: is conformation through provision of

objective evidence, that the specified requirements
have been fulfilled
– Standard method
 Verification of Qualitative Method
• Diagnostic Sensitivity (Se): is defined as the ability of a test to
correctly identify the true Positive/Present/Reactive result
• Analytical sensitivity: Analytical sensitivity of a diagnostic assay
can be assessed by quantifying the least amount of analyte that
is detectable in the sample
• Diagnostic Specificity (Sp): is defined as the ability of the test to
correctly identify the true Negative/Absence/Non-Reactive
result
• The selectivity: selectivity of a method is the accuracy of its
measurement in the presence of interferences such as
competing non-target microorganisms, impurities, degradents
and matrix components.
– The terms selectivity and specificity have often been used
interchangeably.
 Verification of Qualitative Method
• Accuracy: The closeness of the measured result to the true
value
– In Qualitative Method,
• Accuracy is the proportion of true results (both true positives and
true negatives) in the given population
• Precision: Reproducibility or closeness of results to each other
• Predictive value PV(Positive and Negative)
– PPV is the proportion of pts with positive test results who
are correctly diagnosed.
– NPV is the proportion of pts with negative test results who
are correctly diagnosed
 Verification Sensitivity & Specificity
(Accuracy)
• Choose source of materials or samples
• Specimen, Reference material, PT
– PT sample should be similar to actual sample
• Number of specimen to be tested is CLSI recommends 50
for each positive and negative
• Number of days 10 to 20 to evaluate the method to
verify
• Calibrate equipments to be used for verification
• Run quality control samples;
– the IQC must be passed, otherwise repeat it
Sensitivity & Specificity (Accuracy)
Test Out Come Condition/Reference
or the method Total
to be verified Material Result
Positive Negative

Positive True Positive False Positive TP+FP

TP FP
Negative False Negative True Negative FN+TN
FN TN
TOTAL TP+FN FP+TN TP+TN+FP+FN

Sensitivity Specificity Accuracy

 Sensitivity Verification Instruction

• Analyze at least 50 specimens from different subjects

known to be positive (similar to the actual sample)
• Record result/data of each sample on worksheet.
• Calculate the of the results accordingly
Specificity Verification Instruction
• Analyze at least 50 specimens from different
subjects known to be negative (similar to the
actual sample)
• Record result/data of each sample on
worksheet.
• Calculate the of the results accordingly as
follow
 Acceptance Criteria Verification of
Sensitivity & Specificity
• CLSI recommends different type of Stastics
Model such as binomial proportion
• Calculate confidence intervals using exact binomial
confidence intervals
 Acceptance Criteria Verification
• Get manufacture’s claim results i.e. check kit insert or
operating procedure or manufacture validation etc.

• Verify the calculated results for Se, and SP against manufacture’s claim
results
– If manufacture’s claim results with in the interval, the method is
verified
– If manufacture’s claim results out of the interval, the method is
not verified
– contact the manufacture
Note
• If the method is not verified, increasing the sample size will
helps us to verify manufacturer claimed value
 Example Procedure for Verification of Qualitative
Method

•• The manufacture claimed that sensitivity and

specificity results are 99% & 99.7% respectively and
you run samples for verification and got the
following results. is the method verified?
Method to be Golden/Standard Method RF Total
verified
Positive Negative
Positive 47 2 49
Negative 3 48 51
TOTAL 50 50 100
Sens= 47/50=94% Spec=48/50= 96%
 Example Procedure for Verification of Qualitative
Method Cont’d…
• Calculate the confidence interval of sensitivity using at 95%

• Sensitivity =94% = 0.94, =0.94,

0.94-0.0658= 0.874, 0.94+0.658= 1.006

• confidence interval for the specificity (0.874,
1.006) 87.4%-100%
Example Procedure for Verification of Qualitative
Method Cont’d…

Calculate the confidence interval of specificity using at 95%

Sensitivity = 96% = 0.96 = 0.96

0.96- 0.054 = 0.906 0.96+0.054 = 1.014

Confidence interval for the specificity (0.906, 1.014)

90.6%-100%
 Example Procedure for Verification of Qualitative
Method Cont’d…

• we are 95% confident that the sensitivity of

manufacture’s claim results is between 87.4% &
100.6 % (0.874 & 1.006)

• we are 95% confident that the specificity of

manufacture’s claim results is between 90.6% and
101.4%. (0.906 and 1.014)
 Precision Verification Instruction
• Precision studies for qualitative tests provide an estimate of the
imprecision of the method

• Cutoff value , the threshold above which the result is reported as

positive and below which the result is reported as negative( yield
50% positive and 50% negative results).
•
• Cut-off concentration (or threshold): The concentration of a
analyte in a specimen used to determine whether the specimen is
considered positive or negative.

• In some circumstances it is recommended that the cut-off

concentration should be set equal to the limit of detection. See also
threshold.
 Precision Verification Instruction
• Choose source of materials or samples
• Specimen
• Analyze at least 40 specimens (CLSI Recommended)
from different subjects known to be
 +20 of concentration cutoff point
 -20 of concentration cutoff point
• Calculate the of the results accordingly
• Calibrate equipments to be used for verification
• Run quality control samples;
– the IQC must be passed, otherwise repeat it
 Precision Verification Instruction
• Prepare a sample with concentration cutoff
value( C50),
• Prepare two samples, one each with
concentration 20% above and 20% below the
cutoff value (C50),
• These sample should be prepared in sufficient
volume to allow up to 40 or more replicates
tests on the same sample (40 replicates from
concentration 20% above and 40 replicates from
20% below the cutoff value( C50))
 How to prepare Cutoff value when
there is no system
• Identify positive fresh sample
• dilute it with different concentration and test
it and take the last positive dilution result
• Run or test it 40 times and
• If you get 14 to 26 of 40(35 to 65%) samples
positive result , you can conclude that the
Cutoff sample is accepted.
• If it out of the above range make dilution
correctly
 Precision Verification Instruction
• Check all equipments calibration
• Run quality control samples according to the
procedures (refer QC SOPs);
– the IQC must be passed, otherwise repeat the procedure.
• Analyze 40 replicates specimens from
concentration 20% above and 40 replicates from
20% below the cutoff value)
• Record result/data
• Calculate the positivity and negative rate follow:
Precision Verification Instruction
Calculate the positivity and negative rate follow:
 % of Positive with +20 of concentration cutoff point
 % of Negative with -20 of concentration cutoff point

%Pos= %Neg=
 Acceptance Criteria Verification
• CLSI recommend that the result of both positive and
negative percent should be >90% if the sample size
is greater or equality 40 for each (>40 replicate
sample for the +20 % concentration cutoff point and
> 40 replicate sample the -20% of concentration
cutoff point) to verify the method.
 Acceptance Criteria Verification
• In addition there are different type of statistical
model for Verification therefore the labs should
verify their methods based on validation procedure