An American National Standard

Designation: D 7372 – 07

Standard Guide for

Analysis and Interpretation of Proficiency Test Program
Results1
This standard is issued under the fixed designation D 7372; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (e) indicates an editorial change since the last revision or reapproval.

1. Scope 3.1.2 assignable cause, n—factor that contributes to varia-

1.1 This guide covers the analysis and interpretation of tion and that is feasible to detect and identify. E 456
proficiency test (PT) program results. For participants in 3.1.3 bias, n—systematic error that contributes to the dif-
interlaboratory proficiency test (or crosscheck, check scheme, ference between a population mean of the measurements or test
etc.) programs, this guide describes procedures for assessing results and an accepted reference or true value. E 177, E 456
participants’ results relative to the PT program results and 3.1.4 control limits, n—limits on a control chart that are
potentially improving the laboratory’s testing performance used as criteria for signaling the need for action or for judging
based on the assessment findings and insights (see 6.1). For the whether a set of data does or does not indicate a state of
committees responsible for the test methods included in statistical control. E 456
interlaboratory proficiency testing programs, this guide de- 3.1.5 in-statistical-control, adj—process, analytical mea-
scribes procedures for assessing the industry’s ability to surement system, or function that exhibits variations that can
perform test methods, and for potentially identifying needs for only be attributable to common cause. D 6299
test method improvement (see 6.2). 3.1.6 proficiency testing, n—determination of a laboratory’s
1.2 This standard does not purport to address all of the testing capability by participation in an interlaboratory cross-
safety concerns, if any, associated with its use. It is the check program D 6299
responsibility of the user of this standard to establish appro- 3.1.7 Z-score, n—standardized and dimensionless measure
priate safety and health practices and determine the applica- of the difference between an individual result in a data set and
bility of regulatory limitations prior to use. the arithmetic mean of the dataset, re-expressed in units of
standard deviation of the dataset (by dividing the actual
2. Referenced Documents difference from the mean by the standard deviation for the data
2.1 ASTM Standards: 2 set). D 6299
D 6299 Practice for Applying Statistical Quality Assurance 3.1.8 Z8-score, n—measure similar to the Z-score except
Techniques to Evaluate Analytical Measurement System that the PT program standard deviation is replaced with one
Performance that takes into account the site precision of the laboratory. Z8 is
D 6792 Guide for Quality System in Petroleum Products a valid approach when the laboratory’s site precision standard
and Lubricants Testing Laboratories deviation is less than the PT program (that is, these data
E 177 Practice for Use of the Terms Precision and Bias in standard deviation) or stated otherwise when the TPI > 1.
ASTM Test Methods ~ Xi – X !
Z8 5
E 456 Terminology Relating to Quality and Statistics
3. Terminology
ŒS ~s8!2 1 S 2
sthese
n
data
DD
3.1 Definitions: where:
3.1.1 accuracy, n—closeness of agreement between an Z8 =
site precision adjusted Z-Score,
observed value and an accepted reference value. E 177, Xi =
laboratory’s result,
E 456 X =
PT average value,
s’ =
site precision standard deviation estimate,
and
1
This guide is under the jurisdiction of ASTM Committee D02 on Petroleum sthese data = PT Program standard deviation estimate.
Products and Lubricants and is the direct responsibility of Subcommittee D02.94 on
Coordinating Subcommittee on Quality Assurance and Statistics.
3.2 Definitions of Terms Specific to This Standard:
Current edition approved July 1, 2007. Published July 2007. 3.2.1 common (chance, random) cause, n—for quality as-
2
For referenced ASTM standards, visit the ASTM website, [Link], or surance programs, one of generally numerous factors, individu-
contact ASTM Customer Service at service@[Link]. For Annual Book of ASTM ally of relatively small importance, that contributes to varia-
Standards volume information, refer to the standard’s Document Summary page on
the ASTM website. tion, and that is not feasible to detect or control. D 6299

Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.

1
 D 7372 – 07
3.2.2 these test data, n—term used by the ASTM Interna- its PT records. Verify that the units for the data reported for
tional D02 PT program to identify statistical results calculated your laboratory are the same as that requested by the PT
from the data submitted by program participants. program. Report discrepancies to the PT program contacts.
3.2.3 site precision (R’), n—value below which the absolute Investigate to determine the root cause of the problem.
difference between two individual test results obtained under 6.1.2 Missing Data—If data and corresponding results are
site precision conditions may be expected to occur with a not present when they are clearly expected, then investigate to
probability of approximately 0.95 (95 %). It is defined as 2.77 determine the cause. In some cases it could be an error within
times the standard deviation of results obtained under site the PT program data entry system, or it could be an omission
precision conditions. D 6299 on the part of the laboratory.
3.2.4 site precision conditions, n—conditions under which 6.1.3 Rejected Data—Perform an investigation for each
test results are obtained by one or more operators in a single instance where laboratory data are rejected by the PT program
site location practicing the same test method on a single data treatment process. Attempt to determine the root cause and
measurement system which may comprise multiple instru- take corrective actions as needed. Document all such investi-
ments, using test specimens taken at random from the same gations and outcomes. Causes should be shared with the
sample of material, over an extended period of time spanning laboratory staff performing the testing. Guidelines on conduct-
at least a 15-day interval. D 6299 ing these types of investigations are available in Practice
3.3 Symbols: D 6299.
3.3.1 I—individual observation (as in I-chart). 6.1.4 Warnings/Alerts on Data—The ASTM International
3.3.2 QC—quality control. D02 PT programs provide comments (that is, Notes 1 to 3 in
3.3.3 R’—site precision. each Table of Results) that warn participants when their result
is:
4. Summary of Guide
Note 1—outside 3-sigma range for these test data
4.1 Petroleum and petroleum product samples are regularly Note 2—outside 3-sigma range for ASTM reproducibility
Note 3—When the Z-score is outside the range -2 to 2
analyzed by specified standard test methods as part of a
proficiency test program. This guide provides a laboratory with Investigations should also be conducted when any of these
the tools and procedures for evaluating their results from the warning situations occur. The priority for conducting investi-
PT program. Techniques are presented to screen, plot, and gations should be for Note 1 > Note 2 > Note 3. Note 1
interpret test results in accordance with industry-accepted indicates that the laboratory is out-of-control with respect to
practices. the data set (with the rejected data removed), which is a
potentially serious situation with respect to the quality control
5. Significance and Use performance of the corresponding standard test method. A
5.1 This guide can be used to evaluate the performance of a similar argument could also be made for Note 2. Note 3 is a
laboratory or group of laboratories participating in an inter- less severe situation, but should be investigated from a con-
laboratory proficiency test (PT) program involving petroleum tinuous improvement standpoint.
and petroleum products.
NOTE 1—If the user notices that the majority of the laboratories have
5.2 Data accrued, using the techniques included in this been cited with a Note 2, then an investigation may not produce any
guide, provide the ability to monitor analytical measurement meaningful corrective actions. This occurrence may be the result of the
system precision and bias. These data are useful for updating precision statement not accurately reflecting the variability of the test
standard test methods, as well as for indicating areas of method and should be addressed by the subcommittee responsible for the
potential measurement system improvement for action by the method. Also, if the Anderson-Darling statistic is >1.3, then the “Note 2”
laboratory. flag may not be valid.
5.3 Reference is made in this standard to the ASTM 6.1.5 Z-score—The Z-score calculated for each datum sub-
International Interlaboratory Cross-Check Program on Petro- mitted by the laboratory should be reviewed with respect to the
leum Products and Lubricants. Program reports containing following:
similarly displayed results and statistical treatments may be [Link] Sign and Magnitude of Z-score—The sign (“+” or
available in other PT programs. “-”) of the statistic reflects the relative bias of the individual
result versus the mean of the sample group. Z-score values
6. Procedure falling in the ranges of 60-1, 1 to 2, 2 to 3, and >3 can be
6.1 Analysis and Interpretation by the Participating compared to control chart values falling in the ranges between
Laboratory—The laboratory should review the results pub- the mean and 1-sigma, 1 to 2-sigma, 2 to 3-sigma, and >
lished for each proficiency test program and for each test 3-sigma. For normally distributed data, there is an expectation
method or parameter for which the laboratory submitted data. that about 68% of the data will lie in the -1 sigma to +1 sigma
This section covers the evaluations and analyses that the range, about 95% in the -2 sigma to +2 sigma range, and 99%
laboratory should consider during their review of proficiency in the -3 to +3 sigma range. The further a laboratory’s Z-score
test results. is from zero, the greater the relative bias and lower the
6.1.1 Reported versus Submitted Data—Check to verify probability that the data is considered within statistical control.
that the values ascribed to the laboratory in the Proficiency Test Conduct investigations to determine the cause of any perceived
(PT) report agree with the values recorded by the laboratory in bias as needed.

2
 D 7372 – 07
[Link] Trend of Z-scores from Previous Rounds—Record 6.2 Analysis and Interpretation by Control Group—This set
the Z-score values for each test method (parameter) for covers the analysis and interpretation of proficiency test data
successive PT program rounds on a control chart to show the by a committee or working group charged with determining the
trend over time. The lab can use the run rules promulgated in overall implications that the published results have with respect
Practice D 6299 to evaluate any observed trends. Conduct to the corresponding test method or to the working group of
investigations to determine causes as needed. participants as a whole. This section covers the evaluations and
[Link] Average Z-score—Calculate the average Z-score for analyses that the working group should consider during their
a series over a selected time period. The sign and magnitude of review.
this result is an indication of the long-term relative bias. 6.2.1 TPI and Precision Trends—Compare precisions ob-
Conduct investigations to determine the cause of any perceived tained over a reasonable number of rounds for a given PT
bias as needed. program test method (or parameter). Such data series could be
6.1.6 Z8-score—The analysis of any Z8 calculated by the plotted to more clearly show trends. The precision estimates
laboratory should be evaluated as described in [Link] for the followed may include TPI (Industry), standard deviations, or
Z-score. relative standard deviations (sigma/mean).
6.1.7 TPI (Industry)—Consider the TPI (Industry) value 6.2.2 Influence of Uncontrolled Variables on Robust Stan-
reported for the data set along with the corresponding Z-score dard Deviations—Use auxiliary information or data to create
for the laboratory’s result (reference Guide D 6792). subsets of the PT data set and recalculate precisions and other
[Link] Broad Implications—Consider the following for statistics for each subset. Evaluate these results with the
interpreting the TPI (Industry): expectation of identifying root causes and potential corrective
> 1.2 The performance of the group providing data is probably satisfac- action steps.
tory relative to the corresponding ASTM published precision. 6.2.3 Normality Evaluations for Historical Sequence—Plot
0.8 to 1.2 The performance of the group providing data may be marginal. the PT results using Q-Q Chart and consider the corresponding
Each laboratory should consider reviewing the test method proce-
dures to identify opportunities for improvement. Anderson-Darling statistic. Observe similar plots for the his-
< 0.8 The performance of the test method as practiced by the group is torical data sets for a given test method (parameter). Investi-
not consistent with the ASTM published precision and laboratory gate situations of non-normal data.
method performance. Improvements should be investigated by all
laboratories. 6.2.4 Contribution of Individual Laboratory Bias to Poor
Reproducibility—Identify the laboratories that are contributing
[Link] Specific Implications Considering TPI (Industry) to poor reproducibility (for example, those laboratories with
and Z-score—A TPI (Industry) <0.8 coupled with a Z-score >3 Z-score > 63), and evaluate the factors that may be contrib-
(or < -3) implies that the laboratory is likely a significant uting to this performance. This may involve targeting these
contributor to the group’s poor performance. This situation laboratories with questionnaires to gather appropriate informa-
warrants an investigation to look for potential causes of the tion. Consultation with test method experts is generally helpful
apparent bias. When the TPI (industry) < 0.8 and the Z-score is in interpreting results from these investigations.
between 2 and 3 (or -2 and -3), then the laboratory should
consider the situation a warning and consider an investigation 7. Report
to find the root cause. 7.1 Laboratories and working groups should document their
6.1.8 Precision—Compare the standard deviation for the PT investigations. In the spirit of continuous improvement, labo-
results versus the site precision value derived from the labo- ratories and working groups are encouraged to share their
ratory’s corresponding quality control chart. The expectation is findings from their investigations and analyses.
that in most cases the site precision value should be less than
the PT program standard deviation. If the laboratory’s site 8. Keywords
precision is greater than the PT standard deviation, then the 8.1 proficiency testing; quality control; test performance
laboratory should investigate to determine the cause. index; Z-score

APPENDIX

(Nonmandatory Information)

X1. CHECKLIST FOR INVESTIGATING THE ROOT CAUSE OF UNSATISFACTORY ANALYTICAL PERFORMANCE

X1.1 To identify why a laboratory’s data may have been X1.1.2 Reanalyze the sample; compare to site precision, or,
considered a statistical outlier or to improve the precision, or if not available, test method repeatability.
both, the following action items (not necessarily in the order of X1.1.3 Check the sample for homogeneity, contamination,
preference) are suggested. There may be additional ways to or that a representative sample has been analyzed.
improve the performance. X1.1.4 Review the test method, and ensure that the latest
X1.1.1 Check the results for typos, calculation errors, and version of the ASTM test method is being used. Check the
transcription errors. procedure step-by-step with the analyst.

3
 D 7372 – 07
X1.1.5 Check the instrument calibration. X1.1.10 After the problem has been resolved, analyze a
X1.1.6 Check the statistical quality control chart to see if the certified reference material, if one is available, or the labora-
problem developed earlier. tory quality control sample, to ascertain that the analytical
X1.1.7 Check the quality of the reagents and standards used operation is under control.
and whether or not they are expired or contaminated. X1.1.11 Provide training to new analysts and, if necessary,
X1.1.8 Check the equipment for proper operation against refresher training to experienced analysts.
the vendor’s operating manual.
X1.1.9 Perform maintenance or repairs, or both, on the X1.1.12 Document the incident and the learnings for use in
equipment following guidelines established by the vendor. the future if a similar problem occurs.

ASTM International takes no position respecting the validity of any patent rights asserted in connection with any item mentioned
in this standard. Users of this standard are expressly advised that determination of the validity of any such patent rights, and the risk
of infringement of such rights, are entirely their own responsibility.

This standard is subject to revision at any time by the responsible technical committee and must be reviewed every five years and
if not revised, either reapproved or withdrawn. Your comments are invited either for revision of this standard or for additional standards
and should be addressed to ASTM International Headquarters. Your comments will receive careful consideration at a meeting of the
responsible technical committee, which you may attend. If you feel that your comments have not received a fair hearing you should
make your views known to the ASTM Committee on Standards, at the address shown below.

This standard is copyrighted by ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959,
United States. Individual reprints (single or multiple copies) of this standard may be obtained by contacting ASTM at the above
address or at 610-832-9585 (phone), 610-832-9555 (fax), or service@[Link] (e-mail); or through the ASTM website
([Link]).