Designation: D6300 − 08 An American National Standard

Standard Practice for

Determination of Precision and Bias Data for Use in Test
Methods for Petroleum Products and Lubricants1
This standard is issued under the fixed designation D6300; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.

INTRODUCTION

Both Research Report RR:D02-1007,2 Manual on Determining Precision Data for ASTM Methods
on Petroleum Products and Lubricants2 and the ISO 4259, benefitted greatly from more than 50 years
of collaboration between ASTM and the Institute of Petroleum (IP) in the UK. The more recent work
was documented by the IP and has become ISO 4259.
ISO 4259 encompasses both the determination of precision and the application of such precision
data. In effect, it combines the type of information in RR:D02-10072 regarding the determination of
the precision estimates and the type of information in Practice D3244 for the utilization of test data.
The following practice, intended to replace RR:D02-1007,2 differs slightly from related portions of the
ISO standard.

1. Scope D3244 Practice for Utilization of Test Data to Determine

1.1 This practice covers the necessary preparations and Conformance with Specifications
planning for the conduct of interlaboratory programs for the E29 Practice for Using Significant Digits in Test Data to
development of estimates of precision (determinability, repeat- Determine Conformance with Specifications
ability, and reproducibility) and of bias (absolute and relative), E456 Terminology Relating to Quality and Statistics
and further presents the standard phraseology for incorporating E691 Practice for Conducting an Interlaboratory Study to
such information into standard test methods. Determine the Precision of a Test Method
2.2 ISO Standards:
1.2 This practice is generally limited to homogeneous prod-
ISO 4259 Petroleum Products-Determination and Applica-
ucts with which serious sampling problems do not normally
tion of Precision Data in Relation to Methods of Test4
arise.
1.3 This practice may not be suitable for solid or semisolid 3. Terminology
products such as petroleum coke, industrial pitches, paraffin 3.1 Definitions:
waxes, greases, or solid lubricants when the heterogeneous 3.1.1 analysis of variance (ANOVA), n—a procedure for
properties of the substances create sampling problems. In such dividing the total variation of a set of data into two or more
instances, use Practice E691 or consult a trained statistician. parts, one of which estimates the error due to selecting and
2. Referenced Documents testing specimens and the other part(s) possible sources of
added variation. D123
2.1 ASTM Standards:3
D123 Terminology Relating to Textiles 3.1.2 bias, n—the difference between the population mean
of the test results and an accepted reference value. E456
1
This practice is under the jurisdiction of ASTM Committee D02 on Petroleum
3.1.3 bias, relative, n—the difference between the popula-
Products and Lubricantsand is the direct responsibility of Subcommittee D02.94 on tion mean of the test results and an accepted reference value,
Coordinating Subcommittee on Quality Assurance and Statistics. which is the agreed upon value obtained using an accepted
Current edition approved Dec. 15, 2008. Published February 2009. Originally reference method for measuring the same property.
approved in 1998. Last previous edition approved in 2007 as D6300–07. DOI:
10.1520/D6300-08. 3.1.4 degrees of freedom, n—the divisor used in the calcu-
2
Supporting data have been filed at ASTM International Headquarters and may lation of variance.
be obtained by requesting Research Report RR:D02-1007.
3
For referenced ASTM standards, visit the ASTM website, [Link], or
contact ASTM Customer Service at service@[Link]. For Annual Book of ASTM
4
Standards volume information, refer to the standard’s Document Summary page on Available from International Organization for Standardization, 1 rue de
the ASTM website. Varembé, Case postale 56, CH-1211 Geneva 20, Switzerland.

Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States

Copyright by ASTM Int'l (all rights reserved); 1

 D6300 − 08
[Link] Discussion—This definition applies strictly only in [Link] Discussion—Interpret as the value equal to or
the simplest cases. Complete definitions are beyond the scope below which the absolute difference between two single test
of this practice. ISO 4259 results obtained in the above conditions may expect to lie with
3.1.5 determinability, n—a quantitative measure of the vari- a probability of 95 %. ISO 4259
ability associated with the same operator in a given laboratory [Link] Discussion—The difference is related to the repeat-
obtaining successive determined values using the same appa- ability standard deviation but it is not the standard deviation or
ratus for a series of operations leading to a single result; it is its estimate. RR:D02–10072
defined as that difference between two such single determined 3.1.12 reproducibility, n—a quantitative expression of the
values as would be exceeded in the long run in only one case random error associated with different operators from different
in 20 in the normal and correct operation of the test method. laboratories using different apparatus, each obtaining a single
[Link] Discussion—This definition implies that two deter- result by applying the same test method on an identical test
mined values, obtained under determinability conditions, sample. It is defined as the 95 % confidence limit for the
which differ by more than the determinability value should be difference between two such single and independent results.
considered suspect. If an operator obtains more than two
[Link] Discussion—Interpret as the value equal to or
determinations, then it would usually be satisfactory to check
below which the absolute difference between two single test
the most discordant determination against the mean of the
results on identical material obtained by operators in different
remainder, using determinability as the critical difference (1).5
laboratories, using the standardized test, may be expected to lie
3.1.6 mean square, n— in analysis of variance, a contrac- with a probability of 95 %. ISO 4259
tion of the expression “mean of the squared deviations from the [Link] Discussion—The difference is related to the repro-
appropriate average(s)” where the divisor of each sum of ducibility standard deviation but is not the standard deviation
squares is the appropriate degrees of freedom. D123 or its estimate. RR:D02–10072
3.1.7 normal distribution, n—the distribution that has the [Link] Discussion—In those cases where the normal use
probability function: of the test method does not involve sending a sample to a
f ~ x ! 5 ~ 1/s !~ 2p ! 21/2 exp@ 2 ~ x 2 µ ! 2 /2s 2 # (1) testing laboratory, either because it is an in-line test method or
because of serious sample instabilities or similar reasons, the
where: precision test for obtaining reproducibility may allow for the
x = a random variate, use of apparatus from the participating laboratories at a
µ = the mean distribution, and common site (several common sites, if feasible). The statistical
s = the standard deviation of the distribution. analysis is not affected thereby. However, the interpretation of
(Syn. Gaussian distribution, law of error) D123 the reproducibility value will be affected, and therefore, the
3.1.8 outlier, n—a result far enough in magnitude from other precision statement shall, in this case, state the conditions to
results to be considered not a part of the set. RR:D02–10072 which the reproducibility value applies.
3.1.9 precision, n—the degree of agreement between two or 3.1.13 standard deviation, n—the most usual measure of the
more results on the same property of identical test material. In dispersion of observed values or results expressed as the
this practice, precision statements are framed in terms of positive square root of the variance. E456
repeatability and reproducibility of the test method. 3.1.14 sum of squares, n—in analysis of variance, a con-
[Link] Discussion—The testing conditions represented by traction of the expression “sum of the squared deviations from
repeatability and reproducibility should reflect the normal the appropriate average(s)” where the average(s) of interest
extremes of variability under which the test is commonly used. may be the average(s) of specific subset(s) of data or of the
Repeatability conditions are those showing the least variation; entire set of data. D123
reproducibility, the usual maximum degree of variability. Refer
to the definitions of each of these terms for greater detail. 3.1.15 variance, n—a measure of the dispersion of a series
RR:D02–10072 of accepted results about their average. It is equal to the sum of
3.1.10 random error, n—the chance variation encountered in the squares of the deviation of each result from the average,
all test work despite the closest control of variables. divided by the number of degrees of freedom. RR:D02–10072
RR:D02–10072 3.1.16 variance, between-laboratory, n—that component of
3.1.11 repeatability, n—the quantitative expression of the the overall variance due to the difference in the mean values
random error associated with the same operator in a given obtained by different laboratories. ISO 4259
laboratory obtaining repetitive results by applying the same test [Link] Discussion—When results obtained by more than
method with the same apparatus under constant operating one laboratory are compared, the scatter is usually wider than
conditions on identical test material within short intervals of when the same number of tests are carried out by a single
time. It is defined as the difference between two such results at laboratory, and there is some variation between means obtained
the 95 % confidence level. RR:D02–10072 by different laboratories. Differences in operator technique,
instrumentation, environment, and sample “as received” are
among the factors that can affect the between laboratory
5
The bold numbers in parentheses refers to the list of references at the end of this variance. There is a corresponding definition for between-
standard. operator variance.

Copyright by ASTM Int'l (all rights reserved); 2

 D6300 − 08
[Link] Discussion—The term “between-laboratory” is of- addressing precision and bias are required in ASTM test
ten shortened to “laboratory” when used to qualify represen- methods. These then give the user an idea of the precision of
tative parameters of the dispersion of the population of results, the resulting data and its relationship to an accepted reference
for example as “laboratory variance.” material or source (if available). Statements addressing deter-
3.2 Definitions of Terms Specific to This Standard: minability are sometimes required as part of the test method
3.2.1 determination, n—the process of carrying out a series procedure in order to provide early warning of a significant
of operations specified in the test method whereby a single degradation of testing quality while processing any series of
value is obtained. samples.
3.2.2 operator, n—a person who carries out a particular test. 5.2 Repeatability and reproducibility are defined in the
precision section of every Committee D02 test method. Deter-
3.2.3 probability density function, n—function which yields
minability is defined above in Section 3. The relationship
the probability that the random variable takes on any one of its
among the three measures of precision can be tabulated in
admissible values; here, we are interested only in the normal
terms of their different sources of variation (see Table 1).
probability.
5.2.1 When used, determinability is a mandatory part of the
3.2.4 result, n—the final value obtained by following the Procedure section. It will allow operators to check their
complete set of instructions in the test method. technique for the sequence of operations specified. It also
[Link] Discussion—It may be obtained from a single deter- ensures that a result based on the set of determined values is
mination or from several determinations, depending on the not subject to excessive variability from that source.
instructions in the method. When rounding off results, the
procedures described in Practice E29 shall be used. 5.3 A bias statement furnishes guidelines on the relationship
between a set of test results and a related set of accepted
4. Summary of Practice reference values. When the bias of a test method is known, a
4.1 A draft of the test method is prepared and a pilot compensating adjustment can be incorporated in the test
program can be conducted to verify details of the procedure method.
and to estimate roughly the precision of the test method. 5.4 This practice is intended for use by D02 subcommittees
4.2 A plan is developed for the interlaboratory study using in determining precision estimates and bias statements to be
the number of participating laboratories to determine the used in D02 test methods. Its procedures correspond with ISO
number of samples needed to provide the necessary degrees of 4259 and are the basis for the Committee D02 computer
freedom. Samples are acquired and distributed. The interlabo- software, Calculation if Precision Data: Petroleum Test Meth-
ratory study is then conducted on an agreed draft of the test ods. The use of this practice replaces that of Research Report
method. RR:D02-1007.2

4.3 The data are summarized and analyzed. Any depen- 5.5 Standard practices for the calculation of precision have
dence of precision on the level of test result is removed by been written by many committees with emphasis on their
transformation. The resulting data are inspected for uniformity particular product area. One developed by Committee E11 on
and for outliers. Any missing and rejected data are estimated. Statistics is Practice E691. Practice E691 and this practice
The transformation is confirmed. Finally, an analysis of vari- differ as outlined in Table 2.
ance is performed, followed by calculation of repeatability,
reproducibility, and bias. When it forms a necessary part of the 6. Stages in Planning of an Interlaboratory Test Program
test procedure, the determinability is also calculated. for the Determination of the Precision of a Test
Method
5. Significance and Use 6.1 The stages in planning an interlaboratory test program
5.1 ASTM test methods are frequently intended for use in are: preparing a draft method of test (see 6.2), planning and
the manufacture, selling, and buying of materials in accordance executing a pilot program with at least two laboratories
with specifications and therefore should provide such precision (optional but recommended for new test methods) (see 6.3),
that when the test is properly performed by a competent planning the interlaboratory program (see 6.4), and executing
operator, the results will be found satisfactory for judging the the interlaboratory program (see 6.5). The four stages are
compliance of the material with the specification. Statements described in turn.

TABLE 1 Sources of Variation

Method Apparatus Operator Laboratory Time
Reproducibility Complete Different Different Different Specified
(Result)
Repeatability Complete Same Same Same Almost same
(Result)
Determinability Incomplete Same Same Same Almost same
(Part result)

Copyright by ASTM Int'l (all rights reserved); 3

 D6300 − 08
TABLE 2 Differences in Calculation of Precision in Practices practice. If any are considered to be too large for the technical
D6300 and E691 application, then consider alterations to the test method.
Element This Practice Practice E691
Applicability Limited in general to homoge- Permits heterogeneous
6.4 Planning the Interlaboratory Program:
neous samples for which seri- samples. 6.4.1 There shall be at least five participating laboratories,
ous sampling problems do but it is preferable to exceed this number in order to reduce the
not normally arise.
number of samples required and to make the precision state-
Number of duplicates Two Any number ment as representative as possible of the qualified user popu-
lation. In the absence of pilot test program information to
Precision is written Test method Each sample
for permit use of Fig. 1 (see 6.4.3) to determine the number of
laboratories, the minimum number of laboratories shall be six.
Outlier tests: Sequential Simultaneous 6.4.2 The number of samples shall be sufficient to cover the
Within laboratories Cochran test k-value
Between laborato- Hawkins test h-value range of the property measured, and to give reliability to the
ries precision estimates. If any variation of precision with level was
Outliers Rejected, subject to subcom- Rejected if many laborato-
observed in the results of the pilot program, then at least five
mittee approval. ries or for cause such as samples shall be used in the interlaboratory program. In any
blunder or not following case, it is necessary to obtain at least 30 degrees of freedom in
method.
both repeatability and reproducibility. For repeatability, this
Retesting not generally per- Laboratory may retest means obtaining a total of at least 30 pairs of results in the
mitted. sample having rejected program. In the absence of pilot test program information to
data.
permit use of Fig. 1 (see 6.4.3) to determine the number of
Rejection limit 20 % 5% samples, the number of samples shall be greater than five, and
chosen such that the number of laboratories times the number
Analysis of variance Two-way, applied globally One-way, applied to each
to all the remaining data sample separately.
of samples is greater than or equal to 42.
at once. 6.4.3 For reproducibility, Fig. 1 gives the minimum number
of samples required in terms of L, P, and Q, where L is the
Precision multiplier tœ2 , where t is the2.851.96 œ 2
number of participating laboratories, and P and Q are the ratios
two-tailed Student’s t
for 95 % probability. of variance component estimates (see 8.3.1) obtained from the
pilot program. Specifically, P is the ratio of the interaction
Increases with decreasing Constant. component to the repeats component, and Q is the ratio of the
laboratories × samples par-
ticularly below 12. laboratories component to the repeats component.

Variation of precision Minimized by data transfor- User may assess from in- NOTE 1—Appendix X1 gives the derivation of the equation used. If Q
with level mation. Equations dividual sample precisions. is much larger than P, then 30 degrees of freedom cannot be achieved; the
for repeatability and reproduc- blank entries in Fig. 1 correspond to this situation or the approach of it
ibility are generated in the (that is, when more than 20 samples are required). For these cases, there
retransformation process. is likely to be a significant bias between laboratories. The program
organizer shall be informed; further standardization of the test method
may be necessary.
6.5 Executing the Interlaboratory Program:
6.2 Preparing a Draft Method of Test—This shall contain all 6.5.1 One person shall oversee the entire program, from the
the necessary details for carrying out the test and reporting the distribution of the texts and samples to the final appraisal of the
results. Any condition which could alter the results shall be results. He or she shall be familiar with the test method, but
specified. The section on precision will be included at this stage should not personally take part in the actual running of the
only as a heading. tests.
6.3 Planning and Executing a Pilot Program with at Least 6.5.2 The text of the test method shall be distributed to all
Two Laboratories: the laboratories in time to raise any queries before the tests
6.3.1 A pilot program is recommended to be used with new begin. If any laboratory wants to practice the test method in
test methods for the following reasons: (1) to verify the details advance, this shall be done with samples other than those used
in the operation of the test; (2) to find out how well operators in the program.
can follow the instructions of the test method; (3) to check the 6.5.3 The samples shall be accumulated, subdivided, and
precautions regarding sample handling and storage; and (4) to distributed by the organizer, who shall also keep a reserve of
estimate roughly the precision of the test. each sample for emergencies. It is most important that the
6.3.2 At least two samples are required, covering the range individual laboratory portions be homogeneous. Instructions to
of results to which the test is intended to apply; however, each laboratory shall include the following:
include at least 12 laboratory-sample combinations. Test each [Link] The agreed draft method of test;
sample twice by each laboratory under repeatability conditions. [Link] Material Safety Data Sheets, where applicable, and
If any omissions or inaccuracies in the draft method are the handling and storage requirements for the samples;
revealed, they shall now be corrected. Analyze the results for [Link] The order in which the samples are to be tested (a
precision, bias, and determinability (if applicable) using this different random order for each laboratory);

Copyright by ASTM Int'l (all rights reserved); 4

 D6300 − 08

FIG. 1 Determination of Number of Samples Required (see 6.4.3)

[Link] The statement that two test results are to be obtained carried out in a short period of time (preferably the same day).
in the shortest practical period of time on each sample by the The term blind fashion means that the operator does not know
same operator with the same apparatus. For statistical reasons that the sample is a duplicate of any previous run.
it is imperative that the two results are obtained independently [Link] The period of time during which repeated results are
of each other, that is, that the second result is not biased by to be obtained and the period of time during which all the
knowledge of the first. If this is regarded as impossible to samples are to be tested;
achieve with the operator concerned, then the pairs of results [Link] A blank form for reporting the results. For each
shall be obtained in a blind fashion, but ensuring that they are sample, there shall be space for the date of testing, the two

Copyright by ASTM Int'l (all rights reserved); 5

 D6300 − 08
results, and any unusual occurrences. The unit of accuracy for 7.2.3 The relationships D = f1(m) and d = f2( m) will not in
reporting the results shall be specified. This should be, if general be identical. It is frequently the case, however, that the
possible, more digits reported than will be used in the final test d
ratios u j 5 Dj are approximately the same for all mj, in which
method, in order to avoid having rounding unduly affect the j

estimated precision values. case f1 is approximately proportional to f2 and a single

[Link] When it is required to estimate the determinability, transformation will be adequate for both repeatability and
the report form must include space for each of the determined reproducibility. The statistical procedures of this practice are
values as well as the test results. greatly facilitated when a single transformation can be used.
[Link] A statement that the test shall be carried out under For this reason, unless the uj clearly vary with property level,
normal conditions, using operators with good experience but the two relationships are combined into a single dependency
not exceptional knowledge; and that the duration of the test relationship D = f(m) (where D now includes d) by including
shall be the same as normal. a dummy variable T. This will take account of the difference
6.5.4 The pilot program operators may take part in the between the relationships, if one exists, and will provide a
interlaboratory program. If their extra experience in testing a means of testing for this difference (see A4.1).
few more samples produces a noticeable effect, it will serve as 7.2.4 In the event that the rations uj do vary with level
a warning that the test method is not satisfactory. They shall be (mean, mj), as confirmed with a regression of uj on mj, or
identified in the report of the results so that any such effect may log(uj) on log(mj), follow the instructions in Annex A5.
be noted. Otherwise, continue with 7.2.5.
6.5.5 It can not be overemphasized that the statement of 7.2.5 The single relationship D = f(m) is best estimated by
precision in the test method is to apply to test results obtained weighted linear regression analysis. Strictly speaking, an
by running the agreed procedure exactly as written. Therefore, iteratively weighted regression should be used, but in most
the test method must not be significantly altered after its cases even an unweighted regression will give a satisfactory
precision statement is written. approximation. The derivation of weights is described in A4.2,
and the computational procedure for the regression analysis is
7. Inspection of Interlaboratory Results for Uniformity described in A4.3. Typical forms of dependence D = f(m) are
and for Outliers given in A3.1. These are all expressed in terms of at most two
7.1 Introduction: (2) transformation parameters, B and B0.
7.1.1 This section specifies procedures for examining the 7.2.6 The typical forms of dependence, the transformations
results reported in a statistically designed interlaboratory they give rise to, and the regressions to be performed in order
program (see Section 6) to establish: to estimate the transformation parameters B, are all summa-
[Link] The independence or dependence of precision and rized in A3.2. This includes statistical tests for the significance
the level of results; of the regression (that is, is the relationship D = f(m) parallel
[Link] The uniformity of precision from laboratory to to the m-axis), and for the difference between the repeatability
laboratory, and to detect the presence of outliers. and reproducibility relationships, based at the 5 % significance
level. If such a difference is found to exist, follow the
NOTE 2—The procedures are described in mathematical terms based on procedures in Annex A5.
the notation of Annex A1 and illustrated with reference to the example
data (calculation of bromine number) set out in Annex A2. Throughout
7.2.7 If it has been shown at the 5 % significance level that
this section (and Section 8), the procedures to be used are first specified there is a significant regression of the form D = f(m), then the
and then illustrated by a worked example using data given in Annex A2. appropriate transformation y = F(x), where x is the reported
NOTE 3—It is assumed throughout this section that all the deviations are result, is given by the equation
either from a single normal distribution or capable of being transformed
into such a distribution (see 7.2). Other cases (which are rare) would
require different treatment that is beyond the scope of this practice. Also,
F~x! 5 K * f ~dxx ! (2)
see (2) for a statistical test of normality. where K = a constant. In that event, all results shall be
7.2 Transformation of Data: transformed accordingly and the remainder of the analysis
carried out in terms of the transformed results. Typical trans-
7.2.1 In many test methods the precision depends on the formations are given in A3.1.
level of the test result, and thus the variability of the reported
results is different from sample to sample. The method of 7.2.8 The choice of transformation is difficult to make the
analysis outlined in this practice requires that this shall not be subject of formalized rules. Qualified statistical assistance may
so and the position is rectified, if necessary, by a transforma- be required in particular cases. The presence of outliers may
tion. affect judgement as to the type of transformation required, if
7.2.2 The laboratories’ standard deviations Dj, and the any (see 7.7).
repeats standard deviations dj (see Annex A1) are calculated 7.2.9 Worked Example:
and plotted separately against the sample means mj . If the [Link] Table 3 lists the values of m, D, and d for the eight
points so plotted may be considered as lying about a pair of samples in the example given in Annex A2, correct to three
lines parallel to the m-axis, then no transformation is necessary. significant digits. Corresponding degrees of freedom are in
If, however, the plotted points describe non-horizontal straight parentheses. Inspection of the values in Table 3 shows that both
lines or curves of the form D = f1(m) and d = f2(m), then a D and d increase with m, the rate of increase diminishing as m
transformation will be necessary. increases. A plot of these figures on log-log paper (that is, a

Copyright by ASTM Int'l (all rights reserved); 6

 D6300 − 08
TABLE 3 Computed from Bromine Example Showing Dependence of Precision on Level
Sample Number 3 8 1 4 5 6 2 7
m 0.756 1.22 2.15 3.64 10.9 48.2 65.4 114
D 0.0669 (14) 0.159 (9) 0.729 (8) 0.211 (11) 0.291 (9) 1.50 (9) 2.22 (9) 2.93 (9)
d 0.0500 (9) 0.0572 (9) 0.127 (9) 0.116 (9) 0.0943 (9) 0.527 (9) 0.818 (9) 0.935 (9)

graph of log D and log d against log m) shows that the points TABLE 4 Absolute Differences Between Transformed Repeat
Results: Bromine Example
may reasonably be considered as lying about two straight lines
(see Fig. A4.1 in Annex A4). From the example calculations Laboratory Sample
1 2 3 4 5 6 7 8
given in A4.4, the gradients of these lines are shown to be the A 42 21 7 13 7 10 8 0
same, with an estimated value of 0.638. Bearing in mind the B 23 12 12 0 7 9 3 0
errors in this estimated value, the gradient may for convenience C 0 6 0 0 7 8 4 0
D 14 6 0 13 0 8 9 32
be taken as 2/3. E 65 4 0 0 14 5 7 28
F 23 20 34 29 20 30 43 0
*x 2
2
3
1
dx 5 3x 3 (3) G 62 4 78 0 0 16 18 56
H 44 20 29 44 0 27 4 32
[Link] Hence, the same transformation is appropriate both J 0 59 0 40 0 30 26 0
for repeatability and reproducibility, and is given by the
equation. Since the constant multiplier may be ignored, the
transformation thus reduces to that of taking the cube roots of 0.0782
the reported bromine numbers. This yields the transformed 0.0439
5 0.138 (4)
data shown in Table A1.3, in which the cube roots are quoted where 0.138 is the result obtained by electronic calculation of
correct to three decimal places. unrounded factors in the expression. There are 72 ranges and
7.3 Tests for Outliers: as, from Table A2.2, the criterion for 80 ranges is 0.1709, this
ratio is not significant.
7.3.1 The reported data or, if it has been decided that a
transformation is necessary, the transformed results shall be 7.3.4 Uniformity of Reproducibility :
inspected for outliers. These are the values which are so [Link] The following outlier tests are concerned with es-
different from the remainder that it can only be concluded that tablishing uniformity in the reproducibility estimate, and are
they have arisen from some fault in the application of the test designed to detect either a discordant pair of results from a
method or from testing a wrong sample. Many possible tests laboratory on a particular sample or a discordant set of results
may be used and the associated significance levels varied, but from a laboratory on all samples. For both purposes, the
those that are specified in the following subsections have been Hawkins’ test (4) is appropriate.
found to be appropriate in this practice. These outlier tests all [Link] This involves forming for each sample, and finally
assume a normal distribution of errors. for the overall laboratory averages (see 7.6), the ratio of the
7.3.2 Uniformity of Repeatability —The first outlier test is largest absolute deviation of laboratory mean from sample (or
concerned with detecting a discordant result in a pair of repeat overall) mean to the square root of certain sums of squares
results. This test (3) involves calculating the eij2 over all the (A1.6).
laboratory/sample combinations. Cochran’s criterion at the 1 % [Link] The ratio corresponding to the largest absolute
significance level is then used to test the ratio of the largest of deviation shall be compared with the critical 1 % values given
these values over their sum (see A1.5). If its value exceeds the in Table A1.5, where n is the number of laboratory/sample cells
value given in Table A2.2, corresponding to one degree of in the sample (or the number of overall laboratory means)
freedom, n being the number of pairs available for comparison, concerned and where v is the degrees of freedom for the sum
then the member of the pair farthest from the sample mean of squares which is additional to that corresponding to the
shall be rejected and the process repeated, reducing n by 1, sample in question. In the test for laboratory/sample cells v will
until no more rejections are called for. In certain cases, refer to other samples, but will be zero in the test for overall
specifically when the number of digits used in reporting results laboratory averages.
leads to a large number of repeat ties, this test can lead to an [Link] If a significant value is encountered for individual
unacceptably large proportion of rejections, for example, more samples the corresponding extreme values shall be omitted and
than 10 %. If this is so, this rejection test shall be abandoned the process repeated. If any extreme values are found in the
and some or all of the rejected results shall be retained. A laboratory totals, then all the results from that laboratory shall
decision based on judgement will be necessary in this case. be rejected.
7.3.3 Worked Example—In the case of the example given in [Link] If the test leads to an unacceptably large proportion
Annex A2, the absolute differences (ranges) between trans- of rejections, for example, more than 10 %, then this rejection
formed repeat results, that is, of the pairs of numbers in Table test shall be abandoned and some or all of the rejected results
A1.3, in units of the third decimal place, are shown in Table 4. shall be retained. A decision based on judgement will be
The largest range is 0.078 for Laboratory G on Sample 3. The necessary in this case.
sum of squares of all the ranges is 7.3.5 Worked Example:
0.0422 + 0.0212 + . . . + 0.0262 + 02 = 0.0439. [Link] The application of Hawkins’ test to cell means
Thus, the ratio to be compared with Cochran’s criterion is within samples is shown below.

Copyright by ASTM Int'l (all rights reserved); 7

 D6300 − 08
[Link] The first step is to calculate the deviations of cell event, care should be taken that the extreme standard deviation
means from respective sample means over the whole array. is not due to the application of an inappropriate transformation
These are shown in Table 5, in units of the third decimal place. (see 7.1), or undetected outliers.
The sum of squares of the deviations are then calculated for 7.4.4 There is no optimal test when standard deviations are
each sample. These are also shown in Table 5 in units of the based on different degrees of freedom. However, the ratio of
third decimal place. the largest variance to that pooled from the remaining samples
[Link] The cell to be tested is the one with the most extreme follows an F-distribution with v1 and v2 degrees of freedom
deviation. This was obtained by Laboratory D from Sample 1. (see A1.7). Here v1 is the degrees of freedom of the variance in
The appropriate Hawkins’ test ratio is therefore: question and v2 is the degrees of freedom from the remaining
0.314 samples. If the ratio is greater than the critical value given in
B* 5 5 0.7281 (5) A2.6, corresponding to a significance level of 0.01/S where S is
=0.11710.0151. . .10.017 the number of samples, then results from the sample in
[Link] The critical value, corresponding to n = 9 cells in question shall be rejected.
sample 1 and v = 56 extra degrees of freedom from the other 7.4.5 Worked Example:
samples is interpolated from Table A1.5 as 0.3729. The test [Link] The standard deviations of the transformed results,
value is greater than the critical value, and so the results from after the rejection of the pair of results by Laboratory D on
Laboratory D on Sample 1 are rejected. Sample 1, are given in Table 6 in ascending order of sample
[Link] As there has been a rejection, the mean value, mean, correct to three significant digits. Corresponding degrees
deviations, and sum of squares are recalculated for Sample 1, of freedom are in parentheses.
and the procedure is repeated. The next cell to be tested will be [Link] Inspection shows that there is no outlying sample
that obtained by Laboratory F from Sample 2. The Hawkins’ among these. It will be noted that the standard deviations are
test ratio for this cell is: now independent of the sample means, which was the purpose
0.097 of transforming the results.
B* 5 5 0.3542 (6)
=0.00610.0151. . .10.017 [Link] The values in Table 7, taken from a test program on
bromine numbers over 100, will illustrate the case of a sample
[Link] The critical value corresponding to n = 9 cells in rejection.
Sample 2 and v = 55 extra degrees of freedom is interpolated
[Link] It is clear, by inspection, that the laboratories stan-
from Table A1.5 as 0.3756. As the test ratio is less than the
dard deviation of Sample 93 at 15.76 is far greater than the
critical value there will be no further rejections.
others. It is noted that the repeats standard deviation in this
7.4 Rejection of Complete Data from a Sample: sample is correspondingly large.
7.4.1 The laboratories standard deviation and repeats stan- [Link] Since laboratory degrees of freedom are not the
dard deviation shall be examined for any outlying samples. If same over all samples, the variance ratio test is used. The
a transformation has been carried out or any rejection made, variance pooled from all samples, excluding Sample 93, is the
new standard deviations shall be calculated. sum of the sums of squares divided by the total degrees of
7.4.2 If the standard deviation for any sample is excessively freedom, that is
large, it shall be examined with a view to rejecting the results
from that sample. ~ 8 3 5.102 19 3 4.202 1…18 3 3.852 !
5 19.96 (7)
7.4.3 Cochran’s criterion at the 1 % level can be used when ~ 8191…18 !
the standard deviations are based on the same number of [Link] The variance ratio is then calculated as
degrees of freedom. This involves calculating the ratio of the
15.262
largest of the corresponding sums of squares (laboratories or 5 11.66 (8)
19.96
repeats, as appropriate) to their total (see A1.5). If the ratio where 11.66 is the result obtained by electronic calculation
exceeds the critical value given in Table A2.2, with n as the without rounding the factors in the expression.
number of samples and v the degrees of freedom, then all the
results from the sample in question shall be rejected. In such an [Link] From Table A1.8 the critical value corresponding to
a significance level of 0.01/8 = 0.00125, on 8 and 63 degrees
of freedom, is approximately 4. The test ratio greatly exceeds
TABLE 5 Deviations of Cell Means from Respective Sample
Means: Transformed Bromine Example
this and results from Sample 93 shall therefore be rejected.
Sample
[Link] Turning to repeats standard deviations, it is noted
Laboratory 1 2 3 4 5 6 7 8 that degrees of freedom are identical for each sample and that
A 20 8 14 15 10 48 6 3 Cochran’s test can therefore be applied. Cochran’s criterion
B 75 7 20 9 10 47 6 3
C 64 35 3 20 30 4 22 25
will be the ratio of the largest sum of squares (Sample 93) to
D 314 33 18 42 7 39 80 50 the sum of all the sums of squares, that is
E 32 32 30 9 7 18 18 39
F 75 97 31 20 30 8 74 53 2.972 / ~ 1.132 10.992 1…11.36 2 ! 5 0.510 (9)
G 10 34 32 20 20 61 9 62 This is greater than the critical value of 0.352 corresponding to
H 42 13 4 42 13 21 8 50 n = 8 and v = 8 (see Table A2.2), and confirms that results from
J 1 28 22 29 14 8 10 53 Sample 93 shall be rejected.
Sum of Squares 117 15 4 6 3 11 13 17
7.5 Estimating Missing or Rejected Values:

Copyright by ASTM Int'l (all rights reserved); 8

 D6300 − 08
TABLE 6 Standard Deviations of Transformed Results: Bromine Example
Sample number 3 8 1 4 5 6 2 7
m 0.9100 1.066 1.240 1.538 2.217 3.639 4.028 4.851
D 0.0278 0.0473 0.0354 0.0297 0.0197 0.0378 0.0450 0.0416
(14) (9) (13) (11) (9) (9) (9) (9)
d 0.0214 0.0182 0.028 0.0164 0.0063 0.0132 0.0166 0.0130
(9) (9) (8) (9) (9) (9) (9) (9)

TABLE 7 Example Statistics Indicating Need to Reject an Entire Sample

Sample number 90 89 93 92 91 94 95 96
m 96.1 99.8 119.3 125.4 126.0 139.9 139.4 159.5
D 5.10 4.20 15.26 4.40 4.09 4.87 4.74 3.85
(8) (9) (8) (11) (10) (8) (9) (8)
d 1.13 0.99 2.97 0.91 0.73 1.32 1.12 1.36
(8) (8) (8) (8) (8) (8) (8) (8)

7.5.1 One of the Two Repeat Values Missing or Rejected—If Total of remaining results in Laboratory 4 = 36.354
one of a pair of repeats (Yij1 or Yij2) is missing or rejected, this Total of remaining results in Sample 1 = 19.845
Total of all the results except a41 = 348.358
shall be considered to have the same value as the other repeat Also S' = 8 and L = 9.
in accordance with the least squares method.
Hence, the estimate of a41 is given by
7.5.2 Both Repeat Values Missing or Rejected:
[Link] If both the repeat values are missing, estimates of aij 1
~9 2 1! ~8 2 1! @~
a ij 5 9 3 36.354! 1 ~ 8 3 19.845! 2 348.358#
(= Yij1 + Yij2) shall be made by forming the laboratories ×
samples interaction sum of squares (see Eq 17), including the (11)
missing values of the totals of the laboratories/samples pairs of Therefore,
results as unknown variables. Any laboratory or sample from 137.588
which all the results were rejected shall be ignored and new a ij 5 5 2.457 (12)
56
values of L and S used. The estimates of the missing or rejected
7.6 Rejection Test for Outlying Laboratories:
values shall be those that minimize the interaction sum of
7.6.1 At this stage, one further rejection test remains to be
squares.
carried out. This determines whether it is necessary to reject the
[Link] If the value of single pair sum aij has to be estimated,
complete set of results from any particular laboratory. It could
the estimate is given by the equation:
not be carried out at an earlier stage, except in the case where
1 no individual results or pairs are missing or rejected. The
~ L 2 1 ! ~ S'21 ! ~ 1
a ij 5 LL 1S'S 1 2 T 1 ! (10)
procedure again consists of Hawkins’ test (see 7.3.4), applied
where: to the laboratory averages over all samples, with any estimated
results included. If any laboratories are rejected on all samples,
L1 = total of remaining pairs in the ith laboratory,
S1 = total of remaining pairs in the jth sample, new estimates shall be calculated for any remaining missing
S' = S – number of samples rejected in 7.4, and values (see 7.5).
T1 = total of all pairs except aij. 7.6.2 Worked Example:
[Link] The procedure on the laboratory averages shown in
[Link] If more estimates are to be made, the technique of Table 8 follows exactly that specified in 7.3.4. The deviations
successive approximation can be used. In this, each pair sum is of laboratory averages from the overall mean are given in Table
estimated in turn from Eq 10, using L1, S1, and T1, values, 9 in units of the third decimal place, together with the sum of
which contain the latest estimates of the other missing pairs. squares. Hawkins’ test ratio is therefore:
Initial values for estimates can be based on the appropriate
sample mean, and the process usually converges to the required B* 5 0.026/ =0.00222 5 0.5518 (13)
level of accuracy within three complete iterations (5). Comparison with the value tabulated in Table A1.5, for n = 9
7.5.3 Worked Example: and v = 0, shows that this ratio is not significant and therefore
[Link] The two results from Laboratory D on Sample 1 no complete laboratory rejections are necessary.
were rejected (see 7.3.4) and thus a41 has to be estimated. 7.7 Confirmation of Selected Transformation:

TABLE 8 Averages of All Transformed Results from Each Laboratory

Grand
Laboratory A B C D E F G H J
Average
Average 2.437 2.439 2.424 2.426A 2.444 2.458 2.410 2.428 2.462 2.436
A
Including estimated value.

Copyright by ASTM Int'l (all rights reserved); 9

 D6300 − 08
TABLE 9 Absolute Deviations of Laboratory Averages from Grand Average × 1000
Sum of
Laboratory A B C D E F G H J
Squares
Deviation 1 3 12 10 8 22 26 8 26 2.22

7.7.1 At this stage it is necessary to check that the rejections no estimated values, the above analysis of variance is exact and
carried out have not invalidated the transformation used. If paragraph 8.2.2 shall be disregarded.
necessary, the procedure from 7.2 shall be repeated with the [Link] Worked Example:
outliers replaced, and if a new transformation is selected, 350.8152
outlier tests shall be reapplied with the replacement values Mean correction 5 (19)
144
reestimated, based on the new transformation.
7.7.2 Worked Example: 5854.6605
[Link] It was not considered necessary in this case to repeat where 854.6605 is the result obtained by electronic calculation
the calculations from 7.2 with the outlying pair deleted. without rounding the factors in the expression.
Samples sum of squares (20)
8. Analysis of Variance and Calculation of Precision
Estimates 22.3022 172.512 2 1…119.1922
5 2 854.6605
8.1 After the data have been inspected for uniformity, a 18
transformation has been performed, if necessary, and any 5 293.5409
outliers have been rejected (see Section 7), an analysis of
variance shall be carried out. First an analysis of variance table
Laboratories sum of squares (21)
shall be constructed, and finally the precision estimates de-
rived. 38.9922 139.0202 1…139.3872
5
8.2 Analysis of Variance: 16
8.2.1 Forming the Sums of Squares for the Laboratories ×
Samples Interaction Sum of Squares—The estimated values, if 2 854.6605
any, shall be put in the array and an approximate analysis of 5 0.0356
variance performed.
M 5 mean correction 5 T 2 /2L'S' (14) Pairs sum of squares 5 ~ 1/2 ! ~ 2.5202 18.0412 1…12.2382 !
where: 2 854.6605 (22)
L' = L – number of laboratories rejected in 7.6 – number of
5293.6908
laboratories with no remaining results after rejections in
7.3.4,
S' = total of remaining pairs in the jth sample, and Repeats sum of squares 5 ~ 1/2 ! ~ 0.0422 10.0212 1…10 2 ! (23)
T = the total of all duplicate test results.
50.0219

Samples sum of squares 5 F( ~

S'

j51
2
g /2L' ! 2 M
j G (15) Table 10 can then be derived.
8.2.2 Forming the Sum of Squares for the Exact Analysis of
where gj is the sum of sample j test results.
Variance:
Laboratories sum of squares 5 F( ~
i51
L'

G
h i2 /2S' ! 2 M (16) [Link] In this subsection, all the estimated pairs are disre-
garded and new values of gj are calculated. The following sums
where hi is the sum of laboratory i test results. of squares for the exact analysis of variance (6) are formed.
Pairs sum of squares 5 ~ 1/2 ! F( ( GL'

i51 j51
S'

a ij2 2 M (17)
Uncorrected sample sum of squares 5 (
j51
S'
g j2
Sj
(24)

I = Laboratories × samples interaction sum of squares where:

= (pairs sum of squares) – (laboratories sum of squares)
Sj = 2(L' – number of missing pairs in that sample).
– (sample sum of squares)
Ignoring any pairs in which there are estimated values,
repeats sum of squares,
L' S'
TABLE 10 Sums of Squares: Bromine Example
E 5 ~ 1/2 ! ( (e
i51 j51
2
ij (18)
Sources of Variation Sum of Squares
Samples 293.5409
The purpose of performing this approximate analysis of Laboratories 0.0356
variance is to obtain the minimized laboratories × samples Laboratories × samples interaction 0.1143
Pairs 293.6908
interaction sum of squares, I. This is then used as indicated in Repeats 0.0219
8.2.2, to obtain the laboratories sum of squares. If there were

Copyright by ASTM Int'l (all rights reserved); 10

 D6300 − 08
L' S' TABLE 11 Analysis of Variance Table
Uncorrected pairs sum of squares 5 ~ 1/2 ! ((a
i51 j51
2
ij (25)
Sources of Variation Degrees of Freedom Sum of Squares
Mean
Square
The laboratories sum of squares is equal to (pairs sum of Laboratories L' − 1 Laboratories sum of ML
squares
squares) – (samples sum of squares) – (the minimized labora-
tories × samples interaction sum of squares) Laboratories × (L' − 1) (S' − 1) − number of I MLS
samples estimated pairs

5 ~ 1/2 ! F( ( G F( G
L'

i51 j51
S'

a ij2 2
S'

j51
g j2
Sj
2I (26) Repeats L'S' − number of pairs in
which one or both values
E Mr

are estimated
[Link] Worked Example:
Uncorrected samples sum of squares (27)

19.8452 72.5122 19.1922

5 1 1…1
16 18 18 [Link] Worked Example—The analysis of variance is shown
in Table 12. The ratio ML/MLS = 0.0044/0.002078 has a value
5 1145.1834
2.117. This is greater than the 5 % critical value obtained from
Table A1.6, indicating bias between laboratories.
2.5202 8.0412 2.2382
Uncorrected pairs sum of squares 5
2
1
2
1…1
2 8.3 Expectation of Mean Squares and Calculation of Preci-
sion Estimates:
(28)
8.3.1 Expectation of Mean Squares with No Estimated
51145.3329 Values—For a complete array with no estimated values, the
expectations of mean squares are
Therefore, laboratories sum of squares (29) Laboratories: so2 + 2s12 + 2S' s2 2
Laboratories × samples: so2 + 2s12
Repeats: so 2
5 1145.3329 2 1145.183410.1143
where:
5 0.0352
s12 = the component of variance due to interaction between
8.2.3 Degrees of Freedom: laboratories and samples, and
s22 = the component of variance due to differences between
[Link] The degrees of freedom for the laboratories are
laboratories.
(L'–1). The degrees of freedom for laboratories × samples
interaction are (L' –1)(S'–1) for a complete array and are 8.3.2 Expectation of Mean Squares with Estimated Values:
reduced by one for each pair which is estimated. The degrees [Link] The coefficients of s12 and s22 in the expectation of
of freedom for repeats are (L'S' ) and are reduced by one for mean squares are altered in the cases where there are estimated
each pair in which one or both values are estimated. values. The expectations of mean squares then become
[Link] Worked Example—There are eight samples and nine Laboratories: aso 2 + 2s12 + b s22
Laboratories × samples:gso2 + 2s1 2
laboratories in this example. As no complete laboratories or Repeats: so 2
samples were rejected, then S' = 8 and L' = 9. where:
Laboratories degrees of freedom = L – 1 = 8.
K 2 S'
b52 (30)
Laboratories × samples interaction degrees of freedom if there L'21'
had been no estimates, would have been (9 – 1)(8 – 1) = 56. where:
But one pair was estimated, hence laboratories × samples K = the number of laboratory × sample cells containing at
interaction degrees of freedom = 55. Repeats degrees of least one result, and a and g are computed as in [Link]
freedom would have been 72 if there had been no estimates. In
this case one pair was estimated, hence repeats degrees of [Link] If there are no cells with only a single estimated
freedom = 71. result, then a = g = 1.
8.2.4 Mean Squares and Analysis of Variance:
[Link] The mean square in each case is the sum of squares
divided by the corresponding degrees of freedom. This leads to TABLE 12 Analysis of Variance Table: Transformed Benzene
the analysis of variance shown in Table 11. The ratio ML/MLS Example
is distributed as F with the corresponding laboratories and Source of Variation
Sum of Degrees of
Mean Square F
interaction degrees of freedom (see A1.7). If this ratio exceeds Squares Freedom
Laboratories 0.0352 8 0.004400 2.117
the 5 % critical value given in Table A1.6, then serious bias
between the laboratories is implied and the program organizer Laboratories × 0.1143 55 0.002078
shall be informed (see 6.5); further standardization of the test samples

method may be necessary, for example, by using a certified Repeats 0.0219 71 0.000308 ...
reference material.

Copyright by ASTM Int'l (all rights reserved); 11

 D6300 − 08
[Link] If there are no empty cells (that is, every lab has [Link] Worked Example:
tested every sample at least once, and K = L'× S'), then a and Repeatability variance 5 2s o 2 (37)
g are both one plus the proportion of cells with only a single
result. 50.000616
[Link] If there are both empty cells and cells with only one
result, then, for each lab, compute the proportion of samples Repeatability of y 5 t 71=0.000616
tested for which there is only one result, pi, and the sum of
these proportions over all labs, P. For each sample, compute 51.994 x 0.0428
the proportion of labs that have tested the sample for which
50.0495
there is only one result on it, qj, and the sum of these
proportions over samples, Q. Compute the total number of cells Repeatability of x 5 3x 2/3 3 0.0495
with only one result, W, and the proportion of these among all
nonempty cells, W/K. Then 50.148x 2/3

a 5 11
P 2 W/K
(31)
[Link] Reproducibility—Reproducibility variance = 2 (so2
L'21 + s12 + s22) and can be calculated using Eq 38.
and Reproducibility variance (38)

g 5 11
W 2 P 2 Q1W/K
K 2 L'2S'11
(32)
NOTE 4—These subsections are based upon the assumptions that both
5
2
S
M 1 12
b L
2
b D S
M LS1 2 2 g1
2
b D
~g 2 a! Mr

samples and laboratories are random effects. where the symbols are as set out in 8.2.4 and 8.3.2. The
[Link] Worked Example—For the example, which has eight reproducibility estimate is the product of the reproducibility
samples and nine laboratories, one cell is empty (Laboratory D standard deviation and the “t-value” with appropriate degrees
on Sample 1), so K = 71 and of freedom (see Table A2.3), corresponding to a two-sided
probability of 95 %. An approximation (7) to the degrees of
71 2 8
b52 5 15.75 (33) freedom of the reproducibility variance is given by Eq 39.
~9 2 1!
~ Reproducibility variance! 2
None of the nonempty cells has only one result, so a = g = v5 (39)
r 12 r 22 r 32
1. To make the example more interesting, assume that only one 1 1
L'21 v LS v r
result remains from Laboratory A on Sample 1. Then W = 1, p1
= 1⁄8 , p2 = p3 = ... = p9 = 0, and P = 0.125. We compute q1 where:
= 1⁄8 (we don’t count Laboratory D in the denominator), q2 = r1, r2, and r3 = the three successive terms in Eq 38,
q3 =...= q8 = 0, and Q = 0.125. Consequently, vLS = the degrees of freedom for laboratories ×
0.125 2 1/71
samples, and
a 5 11 5 1.014 (34) vr = the degrees of freedom for repeats.
921
(1) Round calculated estimates of reproducibility in accor-
and dance with Practice E29, specifically paragraph 7.6 of that
1 2 0.125 2 0.12511/71 practice.
g 5 11 5 1.014 (35)
55 (2) Substantial bias between laboratories will result in a loss
8.3.3 Calculation of Precision Estimates: of degrees of freedom estimated by Eq 39. If reproducibility
degrees of freedom are less than 30, then the program organizer
[Link] Repeatability—The repeatability variance is twice
shall be informed (see 6.5); further standardization of the test
the mean square for repeats. The repeatability estimate is the
method may be necessary.
product of the repeatability standard deviation and the “t-
value” with appropriate degrees of freedom (see Table A2.3) [Link] Worked Example—Recalling that a = g = 1 (not
corresponding to a two-sided probability of 95 %. Round 1.014, as shown in Eq 34 and 35):
calculated estimates of repeatability in accordance with Prac- Reproducibility variance (40)
tice E29, specifically paragraph 7.6 of that practice. Note that
if a transformation y = f(x) has been used, then 5 S 2
15.75 DS
3 0.00440 1
13.75
15.75 D
3 0.002078 10.000308

r~x!' U U
dx
dy
r~y! (36)
50.00055910.00181410.000308
where r(x), r(y) are the corresponding repeatability functions
(seeA3.1 ). A similar relationship applies to the reproducibility 50.002681
functions R(x), R(y).

Copyright by ASTM Int'l (all rights reserved); 12

 D6300 − 08
0.0026812 [Link] Because there will always be at least some bias
v5 (41)
0.000559 0.0018142 0.0003082
2
because of the inherent variability of test results, it is recom-
1 1
8 55 71 mended to test the bias value by applying Student’s t test using
the number of laboratories degrees of freedom for the sample
572 made available during the calculation of precision. When the
calculated t is less than the critical value at the 5 % confidence
Reproducibility of y 5 t 72= 0.002681 (42) level, the bias should be reported as not significant.
8.4 Precision and Bias Section for a Test Method—When
50.1034
the precision of a test method has been determined, in
Reproducibility of x 5 0.310x 2/3 accordance with the procedures set out in this practice, it shall
be included in the test method as illustrated in these examples:
[Link] Determinability—When determinability is relevant, 8.4.1 Precision—The precision of this test method, which
it shall be calculated by the same procedure as is used to was determined by statistical examination of interlaboratory
calculate repeatability except that pairs of determined values results using Practice D6300, is as follows.
replace test results. This will as much as double the number of [Link] Repeatability—The difference between repetitive re-
“laboratories” for the purposes of this calculation. sults obtained by the same operator in a given laboratory
8.3.4 Examination of Precision-to-mean Ratio: applying the same test method with the same apparatus under
[Link] For test methods that are intended to quantitate constant operating conditions on identical test material within
analyte(s), for each sample, calculate the following precision- short intervals of time would in the long run, in the normal and
to-mean ratio: correct operation of the test method, exceed the following
@ standard deviation under repeatability conditions# values only in one case in 20.
10 3 (43)
@ sample mean# Repeatability 5 0.148 x 2/3 (44)
where x is the average of two results.
[Link] Remove all results for samples with the precision-
to-mean ratio (Eq 43) that are greater than 1, and repeat all [Link] Reproducibility—The difference between two single
precision calculation procedures using this reduced dataset. and independent results obtained by different operators apply-
[Link] If the precision versus level relationship established ing the same test method in different laboratories using
using the reduced dataset (described in [Link]) is significantly different apparatus on identical test material would, in the long
different than that calculated using the original dataset, report run, in the normal and correct operation of the test method,
the precision for the test method established from the reduced exceed the following values only in one case in 20.
dataset in lieu of the precision established from the original Reproducibility 5 0.310 x 2/3 (45)
dataset. Examples of significantly different relationships can
be, but are not limited to, different functional forms of the where x is the average of two results.
transformation, or parameter values that are highly divergent [Link] If determinability is relevant, it shall precede repeat-
numerically. ability in the statement above. The unit of measurement shall
be specified when it differs from that of the test result:
NOTE 5—It is highly recommended that the decision of including or [Link] Determinability—The difference between the pair of
excluding samples with precision-to-mean ratio greater than 1 is made determined values averaged to obtain a test result would, in the
under the guidance of qualified statistical assistance.
long run, in the normal and correct operation of the test
8.3.5 Bias: method, exceed the following value in only one case in 20.
[Link] Bias equals average sample test result minus its When this occurs, the operator must take corrective action:
accepted reference value. In the ideal case, average 30 or more
test results, measured independently by processes in a state of Determinability 5 0.59=m (46)
statistical control, for each of several relatively uniform mate- where m is the mean of the two determined values in mL.
rials, the reference values for which have been established by 8.4.2 A graph or table may be used instead of, or in addition
one of the following alternatives, and subtract the reference to, the equation format shown above. In any event, it is helpful
values. In practice, the bias of the test method, for a specific to include a table of typical values like Table 13.
material, may be calculated by comparing the sample average 8.4.3 The wording to be used for test methods where the
with the accepted reference value. statistical treatment applied is unknown is: “The precision of
[Link] Accepted reference values may be one of the fol- this test is not known to have been obtained in accordance with
lowing: an assigned value for a Standard Reference Material, a
consensus value based on collaborative experimental work
under the guidance of a scientific or engineering organization, TABLE 13 Typical Precision Values: Bromine Example
an agreed upon value obtained using an accepted reference Average Value Repeatability Reproducibility
Bromine Numbers Bromine Numbers Bromine Numbers
method, or a theoretical value. 1.0 0.15 0.31
[Link] Where possible, one or more materials with ac- 2.0 0.23 0.49
cepted reference values shall be included in the interlaboratory 10.0 0.69 1.44
20.0 1.09 2.28
program. In this way sample averages free of outliers will 100.0 3.19 6.68
become available for use in determining bias.

Copyright by ASTM Int'l (all rights reserved); 13

 D6300 − 08
currently accepted guidelines (for example, in Committee D02, 8.5 Data Storage:
Practice D6300).” The existing statement of precision would 8.5.1 The interlaboratory program data should be preserved
then follow. for general reference. Prepare a research report containing
8.4.4 Insuffıcient Degrees of Freedom: details of the test program, including description of the
[Link] In the event that the degrees of freedom associated samples, the raw data, and the calculations described herein.
with either the repeatability estimate or the reproducibility Send the file to ASTM Headquarters and request a File
estimate are less than the minimum requirement of 30, but are Reference Number.
greater than 15, report the actual degrees of freedom associated
with the respective repeatability or reproducibility results as 8.5.2 Use the following footnote style in the precision
follows: section of the test method. “The results of the cooperative test
program, from which these values have been derived, are filed
The degrees of freedom associated with the repeatability/
reproducibility estimate from this round robin study are XXX. Since at ASTM Headquarters as RR:D02–XXXX.”
the minimum requirement of 30 (in accordance with Practice D6300)
is not met, users are cautioned that the actual repeatability/
reproducibility may be significantly different than these estimates. 9. Keywords
[Link] This practice does not recommend publishing preci- 9.1 interlaboratory; precision; repeatability; reproducibility;
sion estimates with degrees of freedom less than 15 as the round robin
reliability of such estimates is highly questionable.

ANNEXES

(Mandatory Information)

A1. NOTATION AND TESTS

A1.1 Notation Used Throughout TABLE A1.1 Typical Layout of Data from Round Robin
Sample
a =the sum of duplicate test results, Laboratory 1 2 j S
e =the difference between duplicate test results, 1 x111 x121 x1j1 x1S1
x112 x122 x1j2 x1S2
g =the sum of sample test results,
h =the sum of laboratory test results, 2 x211 x221 x2j1 x2S1
i =the suffix denoting laboratory number, x212 x222 x2j2 x2S2
j =the suffix denoting sample number,
i xi11 xi21 xij1 xiS1
S =the number of samples, xi12 xi22 xij2 xiS2
T =the total of all duplicate test results,
L =the number of laboratories, L xL11 xL21 xLj1 xLS1
m =the mean of sample test results, xL12 xL22 xLj2 xLS2
x =the mean of a pair of test results in repeatability and Total g1 g2 gj gs
reproducibility statements,
x... = an individual test result, Mean m1 m2 mj ms
y... = a transformed value of x..., and
v = the degrees of freedom.
A1.4 Sums of Squares and Variances (7.2)
A1.2 Array of Duplicate Results from Each of L Labora- A1.4.1 Repeats Variance for Sample j:
tories on S Samples and Corresponding Means mj L

A1.2.1 See Table A1.1. 2

(e
i51
2
ij
d 5
j (A1.1)
2L
NOTE A1.1—If a transformation y = F(x) of the reported data is
necessary (see 7.2), then corresponding symbols yij1 and yij2 are used in where:
place of xij1 and xij2. L = the repeats degrees of freedom for Sample j, one degree
of freedom for each laboratory pair. If either or both of
A1.3 Array of Sums of Duplicate Results, of Laboratory a laboratory/sample pair of results is missing, the
Totals hi and Sample Totals gj corresponding term in the numerator is omitted and the
A1.3.1 See Table A1.2. factor L is reduced by one.

A1.3.2 If any results are missing from the complete array, A1.4.2 Between Cells Variance for Sample j:
then the divisor in the expression for mj will be correspond-
ingly reduced.
C j2 5 F( L

i51
a ij2
n ij
2
g j2
Sj G
/~L 2 1! (A1.2)

Copyright by ASTM Int'l (all rights reserved); 14

 D6300 − 08
TABLE A1.2 Typical Layout of Sums of Duplicate ResultsA ~ K j D j2 ! 2
vj 5 (A1.5)
Sample ~ C ! @ ~ K j 2 1 ! d j2 # 2
2 2
j
Laboratory 1 2 j S Total 1
1 a11 a12 a1j aiS h1
L21 L
2 a21 a22 a2j a2S h2
i a i1 ai2 aij ai1 hi
(rounded to the nearest integer)
L a L1 aL2 aLj aLS hL A1.4.5 If either or both of a laboratory/sample pair of results
Total g1 g2 gj gS T is missing, the factor L is reduced by one.
A
aij = xij1 + xij2 (or aij = yij1 + y ij2, if a transformation has been used) A1.4.6 If both of a laboratory/sample pair of results is
eij = x ij1 – x ij2 (or aij = yij1 – yij2, if a transformation has been used) missing, the factor (L – 1) is reduced by one.

L S A1.5 Cochran’s Test

gj 5 oa
i51
ij hi 5
j51
oa ij
A1.5.1 The largest sum of squares, SSk, out of a set of n
mutually independent sums of squares each based on v degrees
L S
of freedom, can be tested for conformity in accordance with:
m j 5 g j /2L
T5 oh
i51
i 5 o
j51
gj SSk
Cochran’s criterion 5 n (A1.6)
(
i51
SSi

A1.5.2 The test ratio is identical if sum of squares values are

A1.4.3 Laboratories Variance for Sample j: replaced by mean squares (variance estimates). If the calcu-
1 lated ratio exceeds the critical value given in Table A1.3, then
D j2 5 @ C 2 1 ~ K j 2 1 ! d j2 # (A1.3) the sum of squares in question, SSk, is significantly greater than
Kj j
the others with a probability of 99 %. Examples of SSi include
where: eij2 and dj2 (Eq A1.1).

K j 5 S j2 2S ( n D /@S
L

i51
2
ij j ~L 2 1!# (A1.4) A1.6 Hawkins’ Test
A1.6.1 An extreme value in a data set can be tested as an
nij = number of results obtained by Laboratory i from outlier by comparing its deviation from the mean value of the
Sample j, data set to the square root of the sum of squares of all such
Sj = total number of results obtained from Sample j, and deviations. This is done in the form of a ratio. Extra informa-
L = number of cells in Sample j containing at least one tion on variability can be provided by including independent
result. sums of squares into the calculations. These will be based on v
A1.4.4 Laboratories degrees of freedom for Sample j is degrees of freedom and will have the same population variance
given approximately (6) by: as the data set in question. Table A1.4 shows the values that are

TABLE A1.3 Cube Root of Bromine Number for Low Boiling Samples
Sample
Laboratory 1 2 3 4 5 6 7 8
A 1.239 4.010 0.928 1.547 2.224 3.586 4.860 1.063
1.281 4.031 0.921 1.560 2.231 3.596 4.852 1.063

B 1.193 4.029 0.884 1.547 2.231 3.691 4.856 1.063

1.216 4.041 0.896 1.547 2.224 3.682 4.853 1.063

C 1.216 3.990 0.913 1.518 2.183 3.647 4.826 1.091

1.216 3.996 0.913 1.518 2.190 3.639 4.830 1.091

D 1.601 3.992 0.928 1.587 2.210 3.674 4.774 1.000

1.578 3.998 0.928 1.574 2.210 3.682 4.765 1.032

E 1.281 3.998 0.940 1.547 2.217 3.619 4.871 1.091

1.216 3.994 0.940 1.547 2.231 3.624 4.864 1.119

F 1.216 4.135 0.896 1.504 2.257 3.662 4.946 1.119

1.193 4.115 0.862 1.533 2.237 3.632 4.903 1.119

G 1.239 3.996 0.917 1.518 2.197 3.586 4.850 1.032

1.301 3.992 0.839 1.518 2.197 3.570 4.832 0.976

H 1.260 4.051 0.921 1.474 2.204 3.674 4.860 1.032

1.216 4.031 0.892 1.518 2.204 3.647 4.856 1.000

J 1.281 4.086 0.932 1.587 2.231 3.662 4.873 1.119

1.281 4.027 0.932 1.547 2.231 3.632 4.847 1.119

Copyright by ASTM Int'l (all rights reserved); 15

 D6300 − 08
TABLE A1.4 Calculations for Hawkins’ Test for OutliersA S

Sample v5 ( ~n
j51
j 2 1 ! , jfik. (A1.9)
1 2 j S
No. of cells n1 n2 nj ns A1.6.1.5 If B* exceeds the critical value, reject results from
Sample mean m1 m2 mj ms
Sum of squares SS1 SS2 SSj SSs
the cell in question (Sample k, Laboratory i), modify nk, mk ,
A
and SSk values accordingly, and repeat from A1.6.1.1.
nj = the number of cells in Sample j which contains at least one result,
mj = the mean of Sample j, and NOTE A1.2—Hawkins’ test applies theoretically to the detection of only
SSj = the sum of squares of deviations of cell means aij /nij from sample mean a single outlier laboratory in a sample. The technique of repeated tests for
mj , and is given by
a single outlier, in the order of maximum deviation from sample mean,
SS j 5 s L 2 1 d C j2 implies that the critical values in Table A1.5 will not refer exactly to the
1 % significance level. It has been shown by Hawkins, however, that if n
(L–1) is the between cells (laboratories) degrees of freedom, and shall be
$ 5 and the total degrees of freedom (n + v) are greater than 20, then this
reduced by 1 for every cell in Sample j which does not contain a result.
effect is negligible, as are the effects of masking (one outlier hiding
another) and swamping (the rejection of one outlier leading to the
rejection of others).
required to apply Hawkins’ test to individual samples. The test A1.6.1.6 When the test is applied to laboratories averaged
procedure is as follows: over all samples, Table A1.4 will reduce to a single column
A1.6.1.1 Identify the sample k and cell mean aik/nik, which containing:
has the most extreme absolute deviation ? a ik/n ik2m k ? . The cell n = number of laboratories = L,
identified will be the candidate for the outlier test, be it high or m = overall mean = T/N, where N is the total number of results
low. in the array, and
A1.6.1.2 Calculate the total sum of squares of deviations SS = sum of squares of deviations of laboratory means from the
S
overall mean, and is given by

( Sn D
SS 5 ( SS
i51
j (A1.7)
SS 5
L
hi
2m
2
(A1.10)
i51 i
A1.6.1.3 Calculate the test ratio
where:
B* 5
?a ik/n ik 2 m k ? (A1.8) ni = the number of results in Laboratory i.
=SS In the test procedure, therefore, identify the laboratory mean
A1.6.1.4 Compare the test ratio with the critical value from hi/ni which differs most from the overall mean, m. The
Table A1.5, for n = nk and extra degrees of freedom v where corresponding test ratio then becomes:

TABLE A1.5 Critical Values of Hawkins’ 1 % Outlier Test for n = 3 to 50 and y = 0 to 200
Degrees of Freedom y
n 0 5 10 15 20 30 40 50 70 100 150 200
3 0.8165 0.7240 0.6100 0.5328 0.4781 0.4049 0.3574 0.3233 0.2769 0.2340 0.1926 0.1674
4 0.8639 0.7505 0.6405 0.5644 0.5094 0.4345 0.3850 0.3492 0.3000 0.2541 0.2096 0.1824
5 0.8818 0.7573 0.6530 0.5796 0.5258 0.4510 0.4012 0.3647 0.3142 0.2668 0.2204 0.1920
6 0.8823 0.7554 0.6571 0.5869 0.5347 0.4612 0.4115 0.3749 0.3238 0.2755 0.2280 0.1988
7 0.8733 0.7493 0.6567 0.5898 0.5394 0.4676 0.4184 0.3819 0.3307 0.2819 0.2337 0.2039
8 0.8596 0.7409 0.6538 0.5901 0.5415 0.4715 0.4231 0.3869 0.3358 0.2868 0.2381 0.2079
9 0.8439 0.7314 0.6493 0.5886 0.5418 0.4738 0.4262 0.3905 0.3396 0.2906 0.2416 0.2112
10 0.8274 0.7213 0.6439 0.5861 0.5411 0.4750 0.4283 0.3930 0.3426 0.2936 0.2445 0.2139
11 0.8108 0.7111 0.6380 0.5828 0.5394 0.4753 0.4295 0.3948 0.3448 0.2961 0.2469 0.2162
12 0.7947 0.7010 0.6318 0.5790 0.5373 0.4750 0.4302 0.3960 0.3466 0.2981 0.2489 0.2181
13 0.7791 0.6910 0.6254 0.5749 0.5347 0.4742 0.4304 0.3968 0.3479 0.2997 0.2507 0.2198
14 0.7642 0.6812 0.6189 0.5706 0.5319 0.4731 0.4302 0.3972 0.3489 0.3011 0.2521 0.2212
15 0.7500 0.6717 0.6125 0.5662 0.5288 0.4717 0.4298 0.3973 0.3496 0.3021 0.2534 0.2225
16 0.7364 0.6625 0.6061 0.5617 0.5256 0.4701 0.4291 0.3972 0.3501 0.3030 0.2544 0.2236
17 0.7235 0.6535 0.5998 0.5571 0.5223 0.4683 0.4282 0.3968 0.3504 0.3037 0.2554 0.2246
18 0.7112 0.6449 0.5936 0.5526 0.5189 0.4665 0.4272 0.3964 0.3505 0.3043 0.2562 0.2254
19 0.6996 0.6365 0.5876 0.5480 0.5155 0.4645 0.4260 0.3958 0.3506 0.3047 0.2569 0.2262
20 0.6884 0.6286 0.5816 0.5436 0.5120 0.4624 0.4248 0.3951 0.3505 0.3051 0.2575 0.2269
21 0.6778 0.6209 0.5758 0.5392 0.5086 0.4603 0.4235 0.3942 0.3503 0.3053 0.2580 0.2275
22 0.6677 0.6134 0.5702 0.5348 0.5052 0.4581 0.4221 0.3934 0.3500 0.3055 0.2584 0.2280
23 0.6581 0.6062 0.5647 0.5305 0.5018 0.4559 0.4206 0.3924 0.3496 0.3056 0.2588 0.2285
24 0.6488 0.5993 0.5593 0.5263 0.4984 0.4537 0.4191 0.3914 0.3492 0.3056 0.2591 0.2289
25 0.6400 0.5925 0.5540 0.5221 0.4951 0.4515 0.4176 0.3904 0.3488 0.3056 0.2594 0.2293
26 0.6315 0.5861 0.5490 0.5180 0.4918 0.4492 0.4160 0.3893 0.3482 0.3054 0.2596 0.2296
27 0.6234 0.5798 0.5440 0.5140 0.4885 0.4470 0.4145 0.3881 0.3477 0.3053 0.2597 0.2299
28 0.6156 0.5737 0.5392 0.5101 0.4853 0.4447 0.4129 0.3870 0.3471 0.3051 0.2599 0.2302
29 0.6081 0.5678 0.5345 0.5063 0.4821 0.4425 0.4113 0.3858 0.3464 0.3049 0.2600 0.2304
30 0.6009 0.5621 0.5299 0.5025 0.4790 0.4403 0.4097 0.3846 0.3458 0.3047 0.2600 0.2306
35 0.5686 0.5361 0.5086 0.4848 0.4641 0.4294 0.4016 0.3785 0.3421 0.3031 0.2600 0.2312
40 0.5413 0.5136 0.4897 0.4688 0.4504 0.4191 0.3936 0.3722 0.3382 0.3010 0.2594 0.2314
45 0.5179 0.4939 0.4728 0.4542 0.4377 0.4094 0.3859 0.3660 0.3340 0.2987 0.2586 0.2312
50 0.4975 0.4764 0.4577 0.4410 0.4260 0.4002 0.3785 0.3600 0.3299 0.2962 0.2575 0.2308

Copyright by ASTM Int'l (all rights reserved); 16

 D6300 − 08

B* 5
? h /n
i i 2m ? (A1.11) F5
V1
(A1.12)
= SS V2

A1.6.1.7 This shall be compared with the critical value from A1.7.2 If the ratio exceeds the appropriate critical value
Table A1.5 as before, but now with extra degrees of freedom v given in Tables A1.6-A1.9, where v1 corresponds to the
= 0. If a laboratory is rejected, adjust the values of n, m, and SS numerator and v2 corresponds to the denominator, then V1 is
accordingly and repeat the calculations. greater than V2 at the chosen level of significance.
A1.7 Variance Ratio Test (F-Test)
A1.7.1 A variance estimate V1, based on v1 degrees of
freedom, can be compared with a second estimate V2, based on
v2 degrees of freedom, by calculating the ratio
TABLE A1.6 Critical 5 % Values of F
y1
3 4 5 6 7 8 9 10 15 20 30 50 100 200 500 `
3 9.28 9.12 9.01 8.94 8.89 8.85 8.81 8.79 8.70 8.66 8.62 8.58 8.55 8.54 8.53 8.53
4 6.59 6.39 6.26 6.16 6.09 6.04 6.00 5.96 5.86 5.80 5.75 5.70 5.66 5.65 5.64 5.63
5 5.41 5.19 5.05 4.95 4.88 4.82 4.77 4.74 4.62 4.56 4.50 4.44 4.41 4.39 4.37 4.37
6 4.76 4.53 4.39 4.28 4.21 4.15 4.10 4.06 3.94 3.87 3.81 3.75 3.71 3.69 3.68 3.67
7 4.35 4.12 3.97 3.87 3.79 3.73 3.68 3.64 3.51 3.44 3.38 3.32 3.27 3.25 3.24 3.23
8 4.07 3.84 3.69 3.58 3.50 3.44 3.39 3.35 3.22 3.15 3.08 3.02 2.97 2.95 2.94 2.93
9 3.86 3.63 3.48 3.37 3.29 3.23 3.18 3.14 3.01 2.94 2.86 2.80 2.76 2.73 2.72 2.71
10 3.71 3.48 3.33 3.22 3.14 3.07 3.02 2.98 2.85 2.77 2.70 2.64 2.59 2.56 2.55 2.54
y 2
15 3.29 3.06 2.90 2.79 2.71 2.64 2.59 2.54 2.40 2.33 2.25 2.18 2.12 2.10 2.08 2.07
20 3.10 2.87 2.71 2.60 2.51 2.45 2.39 2.35 2.20 2.12 2.04 1.97 1.91 1.88 1.86 1.84
30 2.92 2.69 2.53 2.42 2.33 2.27 2.21 2.16 2.01 1.93 1.84 1.76 1.70 1.66 1.64 1.62
50 2.79 2.56 2.40 2.29 2.20 2.13 2.07 2.03 1.87 1.78 1.69 1.60 1.52 1.48 1.46 1.44
100 2.70 2.46 2.31 2.19 2.10 2.03 1.97 1.93 1.77 1.68 1.57 1.48 1.39 1.34 1.31 1.28
200 2.65 2.42 2.26 2.14 2.06 1.98 1.93 1.88 1.72 1.62 1.52 1.41 1.32 1.26 1.22 1.19
500 2.62 2.39 2.23 2.12 2.03 1.96 1.90 1.85 1.69 1.59 1.48 1.38 1.28 1.21 1.16 1.11
` 2.60 2.37 2.21 2.10 2.01 1.94 1.88 1.83 1.67 1.57 1.46 1.35 1.24 1.17 1.11 1.00

Copyright by ASTM Int'l (all rights reserved); 17

 D6300 − 08
TABLE A1.7 Critical 1 % Values of F
y1
3 4 5 6 7 8 9 10 15 20 30 50 100 200 500 `
3 29.5 28.7 28.2 27.9 27.7 27.5 27.3 27.2 26.9 26.7 26.5 26.4 26.2 26.2 26.1 26.1
4 16.7 16.0 15.5 15.2 15.0 14.8 14.7 14.5 14.2 14.0 13.8 13.7 13.6 13.5 13.5 13.5
5 12.1 11.4 11.0 10.7 10.5 10.3 10.2 10.1 9.72 9.55 9.38 9.24 9.13 9.08 9.04 9.02
6 9.78 9.15 8.75 8.47 8.26 8.10 7.98 7.87 7.56 7.40 7.23 7.09 6.99 6.93 6.90 6.88
7 8.45 7.85 7.46 7.19 6.99 6.84 6.72 6.62 6.31 6.16 5.99 5.86 5.75 5.70 5.67 5.65
8 7.59 7.01 6.63 6.37 6.18 6.03 5.91 5.81 5.52 5.36 5.20 5.07 4.96 4.91 4.88 4.86
9 6.99 6.42 6.06 5.80 5.61 5.47 5.35 5.26 4.96 4.81 4.65 4.52 4.42 4.36 4.33 4.31
10 6.55 5.99 5.64 5.39 5.20 5.06 4.94 4.85 4.56 4.41 4.25 4.12 4.01 3.96 3.93 3.91
y 2
15 5.42 4.89 4.56 4.32 4.14 4.00 3.89 3.80 3.52 3.37 3.21 3.08 2.98 2.92 2.89 2.87
20 4.94 4.43 4.10 3.87 3.70 3.56 3.46 3.37 3.09 2.94 2.78 2.64 2.54 2.48 2.44 2.42
30 4.51 4.02 3.70 3.47 3.30 3.17 3.07 2.98 2.70 2.55 2.39 2.25 2.13 2.07 2.03 2.01
50 4.20 3.72 3.41 3.19 3.02 2.89 2.79 2.70 2.42 2.27 2.10 1.95 1.82 1.76 1.71 1.68
100 3.98 3.51 3.21 2.99 2.82 2.69 2.59 2.50 2.22 2.07 1.89 1.73 1.60 1.52 1.47 1.43
200 3.88 3.41 3.11 2.89 2.73 2.60 2.50 2.41 2.13 1.97 1.79 1.63 1.48 1.39 1.33 1.28
500 3.82 3.36 3.05 2.84 2.68 2.55 2.44 2.36 2.07 1.92 1.74 1.56 1.41 1.31 1.23 1.16
` 3.78 3.32 3.02 2.80 2.64 2.51 2.41 2.32 2.04 1.88 1.70 1.52 1.36 1.25 1.15 1.00

TABLE A1.8 Critical 0.1 % Values of F

y1
3 4 5 6 7 8 9 10 15 20 30 50 100 200 500 `
3 141 137 135 133 132 131 130 129 127 126 125 125 124 124 124 124
4 56.2 53.4 51.7 50.5 49.7 49.0 48.5 48.0 46.8 46.1 45.4 44.9 44.5 44.3 44.1 44.0
5 33.2 31.1 29.8 28.8 28.2 27.6 27.2 26.9 25.9 25.4 24.9 24.4 24.1 23.9 23.8 23.8
6 23.7 21.9 20.8 20.0 19.5 19.0 18.7 18.4 17.6 17.1 16.7 16.3 16.0 15.9 15.8 15.8
7 18.8 17.2 16.2 15.5 15.0 14.6 14.3 14.1 13.3 12.9 12.5 12.2 11.9 11.8 11.7 11.7
8 15.8 14.4 13.5 12.9 12.4 12.0 11.8 11.5 10.8 10.5 10.1 9.80 9.57 9.46 9.39 9.34
9 13.9 12.6 11.7 11.1 10.7 10.4 10.1 9.89 9.24 8.90 8.55 8.26 8.04 7.93 7.86 7.81
10 12.6 11.3 10.5 9.92 9.52 9.20 8.96 8.75 8.13 7.80 7.47 7.19 6.98 6.87 6.81 6.76
y 2
15 9.34 8.25 7.57 7.09 6.74 6.47 6.26 6.08 5.53 5.25 4.95 4.70 4.51 4.41 4.35 4.31
20 8.10 7.10 6.46 6.02 5.69 5.44 5.24 5.08 4.56 4.29 4.01 3.77 3.58 3.48 3.42 3.38
30 7.05 6.12 5.53 5.12 4.82 4.58 4.39 4.24 3.75 3.49 3.22 2.98 2.79 2.69 2.63 2.59
50 6.34 5.46 4.90 4.51 4.22 4.00 3.82 3.67 3.20 2.95 2.68 2.44 2.24 2.14 2.07 2.03
100 5.85 5.01 4.48 4.11 3.83 3.61 3.44 3.30 2.84 2.59 2.32 2.07 1.87 1.75 1.68 1.62
200 5.64 4.81 4.29 3.92 3.65 3.43 3.26 3.12 2.67 2.42 2.15 1.90 1.68 1.55 1.46 1.39
500 5.51 4.69 4.18 3.82 3.54 3.33 3.16 3.02 2.58 2.33 2.05 1.80 1.57 1.43 1.32 1.23
` 5.42 4.62 4.10 3.74 3.47 3.27 3.10 2.96 2.51 2.27 1.99 1.73 1.49 1.34 1.21 1.00

TABLE A1.9 Critical 0.05 % Values of F

y1
3 4 5 6 7 8 9 10 15 20 30 50 100 200 500 `
3 225 218 214 211 209 208 207 206 203 201 199 198 197 197 196 196
4 80.1 76.1 73.6 71.9 70.6 69.7 68.9 68.3 66.5 65.5 64.6 63.8 63.2 62.9 62.7 62.6
5 44.4 41.5 39.7 38.5 37.6 36.9 36.4 35.9 34.6 33.9 33.1 32.5 32.1 31.8 31.7 31.6
6 30.4 28.1 26.6 25.6 24.9 24.3 23.9 23.5 22.4 21.9 21.4 20.9 20.5 20.3 20.2 20.1
7 23.5 21.4 20.2 19.3 18.7 18.2 17.8 17.5 16.5 16.0 15.5 15.1 14.7 14.6 14.5 14.4
8 19.4 17.6 16.4 15.7 15.1 14.6 14.3 14.0 13.1 12.7 12.2 11.8 11.6 11.4 11.4 11.3
9 16.8 15.1 14.1 13.3 12.8 12.4 12.1 11.8 11.0 10.6 10.2 9.80 9.53 9.40 9.32 9.26
10 15.0 13.4 12.4 11.8 11.3 10.9 10.6 10.3 9.56 9.16 8.75 8.42 8.16 8.04 7.96 7.90
y 2
15 10.8 9.48 8.66 8.10 7.68 7.36 7.11 6.91 6.27 5.93 5.58 5.29 5.06 4.94 4.87 4.83
20 9.20 8.02 7.28 6.76 6.38 6.08 5.85 5.66 5.07 4.75 4.42 4.15 3.93 3.82 3.75 3.70
30 7.90 6.82 6.14 5.66 5.31 5.04 4.82 4.65 4.10 3.80 3.48 3.22 3.00 2.89 2.82 2.78
50 7.01 6.01 5.37 4.93 4.60 4.34 4.14 3.98 3.45 3.16 2.86 2.59 2.37 2.25 2.17 2.13
100 6.43 5.47 4.87 4.44 4.13 3.89 3.70 3.54 3.03 2.75 2.44 2.18 1.95 1.82 1.74 1.67
200 6.16 5.23 4.64 4.23 3.92 3.68 3.49 3.34 2.83 2.56 2.25 1.98 1.74 1.60 1.50 1.42
500 6.01 5.09 4.51 4.10 3.80 3.56 3.36 3.21 2.72 2.45 2.14 1.87 1.61 1.46 1.34 1.24
` 5.91 5.00 4.42 4.02 3.72 3.48 3.30 3.14 2.65 2.37 2.07 1.79 1.53 1.36 1.22 1.00

Copyright by ASTM Int'l (all rights reserved); 18

 D6300 − 08

A2. EXAMPLE RESULTS OF TEST FOR DETERMINATION OF BROMINE NUMBER AND STATISTICAL TABLES

A2.1 Bromine Number for Low Boiling Samples A2.5 Critical Values of t
A2.1.1 See Table A2.1. A2.5.1 See Table A2.3.

A2.2 Cube Root of Bromine Number for Low Boiling A2.6 Critical Values of F6
Samples
A2.6.1 Critical 5 % Values of F—See Table A1.6.
A2.2.1 See Table A1.3.
A2.6.2 Critical 1 % Values of F—See Table A1.7.
A2.3 Critical 1 % Values of Cochran’s Criterion for n A2.6.3 Critical 0.1 % Values of F—See Table A1.8.
Variance Estimates and v Degrees of Freedom
A2.6.4 Critical 0.05 % Values of F—See Table A1.9.
A2.3.1 See Table A2.2.
A2.6.5 Approximate Formula for Critical Values of
A2.4 Critical Values of Hawkins’ 1 % Outlier Test for n F—Critical values of F for untabulated values of v1, and v2 may
= 3 to 50 and v = 0 to 200 be approximated by second order interpolation from the tables.
Critical values of F corresponding to v1 > 30 and v2 > 30
A2.4.1 See Table A1.5. degrees of freedom and significance level 100 (1–P) %, where
A2.4.2 The critical values in the table are correct to the P is the probability, can also be approximated from the formula
fourth decimal place in the range n = 3 to 30 and v = 0, 5, 15,
and 30 (3). Other values were derived from the Bonferroni
inequality as
log10 ~ F ! 5
A~P!
= b 2 B~P!
2 C~P! S 1 1
1
v1 v2 D (A2.2)

where:
F !G
1
~n 2 1! 2
B* 5 t
n ~ n1v 2 21t 2

where t is the upper 0.005/ n fractile of a t-variate with n +

(A2.1)
b 5 2/ S 1 1
1
v1 v2 D (A2.3)

v – 2 degrees of freedom. The values so computed are only A2.6.5.1 Values of A(P), B(P), and C(P) are given in Table
slightly conservative, and have a maximum error of approxi- A2.4 for typical values of significance level 100 (1 – P) %.
mately 0.0002 above the true value. If critical values are
required for intermediate values of n and v, they may be A2.7 Critical Values of the Normal Distribution (see Table
estimated by second order interpolation using the square of the A2.5):
reciprocals of the tabulated values. Similarly, second order
extrapolation can be used to estimate values beyond n = 50 and
v = 200. 6
See Ref (8) for the source of these tables.

TABLE A2.1 Bromine Number for Low Boiling Samples

Sample
Laboratory 1 2 3 4 5 6 7 8
A 1.9 64.5 0.80 3.7 11.0 46.1 114.8 1.2
2.1 65.5 0.78 3.8 11.1 46.5 114.2 1.2

B 1.7 65.4 0.69 3.7 11.1 50.3 114.5 1.2

1.8 66.0 0.72 3.7 11.0 49.9 114.3 1.2

C 1.8 63.5 0.76 3.5 10.4 48.5 112.4 1.3

1.8 63.8 0.76 3.5 10.5 48.2 112.7 1.3

D 4.1 63.6 0.80 4.0 10.8 49.6 108.8 1.0

4.0 63.9 0.80 3.9 10.8 49.9 108.2 1.1

E 2.1 63.9 0.83 3.7 10.9 47.4 115.6 1.3

1.8 63.7 0.83 3.7 11.1 47.6 115.1 1.4

F 1.8 70.7 0.72 3.4 11.5 49.1 121.0 1.4

1.7 69.7 0.64 3.6 11.2 47.9 117.9 1.4

G 1.9 63.8 0.77 3.5 10.6 46.1 114.1 1.1

2.2 63.6 0.59 3.5 10.6 45.5 112.8 0.93

H 2.0 66.5 0.78 3.2 10.7 49.6 114.8 1.1

1.8 65.5 0.71 3.5 10.7 48.5 114.5 1.0

J 2.1 68.2 0.81 4.0 11.1 49.1 115.7 1.4

2.1 65.3 0.81 3.7 11.1 47.9 113.9 1.4

Copyright by ASTM Int'l (all rights reserved); 19

 D6300 − 08
TABLE A2.2 Critical 1 % Values of Cochran’s Criterion for n Variance Estimates and y Degrees of FreedomA
Degrees of Freedom y
n 1 2 3 4 5 10 15 20 30 50
3 0.9933 0.9423 0.8831 0.8335 0.7933 0.6743 0.6145 0.5775 0.5327 0.4872
4 0.9676 0.8643 0.7814 0.7212 0.6761 0.5536 0.4964 0.4620 0.4213 0.3808
5 0.9279 0.7885 0.6957 0.6329 0.5875 0.4697 0.4168 0.3855 0.3489 0.3131
6 0.8828 0.7218 0.6258 0.5635 0.5195 0.4084 0.3597 0.3312 0.2982 0.2661
7 0.8376 0.6644 0.5685 0.5080 0.4659 0.3616 0.3167 0.2907 0.2606 0.2316
8 0.7945 0.6152 0.5209 0.4627 0.4227 0.3248 0.2832 0.2592 0.2316 0.2052
9 0.7544 0.5727 0.4810 0.4251 0.3870 0.2950 0.2563 0.2340 0.2086 0.1842
10 0.7175 0.5358 0.4469 0.3934 0.3572 0.2704 0.2342 0.2135 0.1898 0.1673
11 0.6837 0.5036 0.4175 0.3663 0.3318 0.2497 0.2157 0.1963 0.1742 0.1532
12 0.6528 0.4751 0.3919 0.3428 0.3099 0.2321 0.2000 0.1818 0.1611 0.1414
13 0.6245 0.4498 0.3695 0.3223 0.2909 0.2169 0.1865 0.1693 0.1498 0.1313
14 0.5985 0.4272 0.3495 0.3043 0.2741 0.2036 0.1748 0.1585 0.1400 0.1226
15 0.5747 0.4069 0.3318 0.2882 0.2593 0.1919 0.1645 0.1490 0.1315 0.1150
20 0.4799 0.3297 0.2654 0.2288 0.2048 0.1496 0.1274 0.1150 0.1010 0.0879
25 0.4130 0.2782 0.2220 0.1904 0.1699 0.1230 0.1043 0.0939 0.0822 0.0713
30 0.3632 0.2412 0.1914 0.1635 0.1455 0.1046 0.0885 0.0794 0.0694 0.0600
35 0.3247 0.2134 0.1685 0.1435 0.1274 0.0912 0.0769 0.0690 0.0601 0.0519
40 0.2940 0.1916 0.1507 0.1281 0.1136 0.0809 0.0681 0.0610 0.0531 0.0457
45 0.2690 0.1740 0.1364 0.1158 0.1025 0.0727 0.0611 0.0547 0.0475 0.0409
50 0.2481 0.1596 0.1248 0.1057 0.0935 0.0661 0.0555 0.0496 0.0431 0.0370
60 0.2151 0.1371 0.1068 0.0902 0.0796 0.0561 0.0469 0.0419 0.0363 0.0311
70 0.1903 0.1204 0.0935 0.0788 0.0695 0.0487 0.0407 0.0363 0.0314 0.0269
80 0.1709 0.1075 0.0832 0.0701 0.0617 0.0431 0.0360 0.0320 0.0277 0.0236
90 0.1553 0.0972 0.0751 0.0631 0.0555 0.0387 0.0322 0.0287 0.0248 0.0211
100 0.1424 0.0888 0.0685 0.0575 0.0505 0.0351 0.0292 0.0260 0.0224 0.0191
A
These values are slightly conservative approximations calculated via Bonferroni’s inequality (3) as the upper 0.01/n fractile of the beta distribution. If intermediate values
are required along the n-axis, they may be obtained by linear interpolation of the reciprocals of the tabulated values. If intermediate values are required along the v-axis,
they may be obtained by second order interpolation of the reciprocals of the tabulated values.

TABLE A2.3 Critical Values of t

Double-Sided % Significance Level
Degrees of Freedom
50 40 30 20 10 5 1
1 1.000 1.376 1.963 3.078 6.314 12.706 63.657
2 0.816 1.061 1.386 1.886 2.920 4.303 9.925
3 0.765 0.978 1.250 1.638 2.353 3.182 5.841
4 0.741 0.941 1.190 1.533 2.132 2.776 4.604
5 0.727 0.920 1.156 1.476 2.015 2.571 4.032
6 0.718 0.906 1.134 1.440 1.943 2.447 3.707
7 0.711 0.896 1.119 1.415 1.895 2.365 3.499
8 0.706 0.889 1.108 1.397 1.860 2.306 3.355
9 0.703 0.883 1.100 1.383 1.833 2.262 3.250
10 0.700 0.879 1.093 1.372 1.812 2.228 3.165
11 0.697 0.876 1.088 1.363 1.796 2.201 3.106
12 0.695 0.873 1.083 1.356 1.782 2.179 3.055
13 0.694 0.870 1.079 1.350 1.771 2.160 3.012
14 0.692 0.868 1.076 1.345 1.761 2.145 2.977
15 0.691 0.866 1.074 1.341 1.753 2.131 2.947
16 0.690 0.865 1.071 1.337 1.746 2.120 2.921
17 0.689 0.863 1.069 1.333 1.740 2.110 2.898
18 0.688 0.862 1.067 1.330 1.734 2.101 2.878
19 0.688 0.861 1.066 1.328 1.729 2.093 2.861
20 0.687 0.860 1.064 1.325 1.725 2.086 2.845
21 0.686 0.859 1.063 1.323 1.721 2.080 2.831
22 0.686 0.858 1.061 1.321 1.717 2.074 2.819
23 0.685 0.858 1.060 1.319 1.714 2.069 2.807
24 0.685 0.857 1.059 1.318 1.711 2.064 2.797
25 0.684 0.856 1.058 1.316 1.708 2.060 2.787
26 0.684 0.856 1.058 1.315 1.706 2.056 2.779
27 0.684 0.855 1.057 1.314 1.703 2.052 2.771
28 0.683 0.855 1.056 1.313 1.701 2.048 2.763
29 0.683 0.854 1.055 1.311 1.699 2.045 2.756
30 0.683 0.854 1.055 1.310 1.697 2.042 2.750
40 0.681 0.851 1.050 1.303 1.684 2.021 2.704
50 0.680 0.849 1.048 1.299 1.676 2.008 2.678
60 0.679 0.848 1.046 1.296 1.671 2.000 2.660
120 0.677 0.845 1.041 1.289 1.658 1.980 2.617
` 0.674 0.842 1.036 1.282 1.645 1.960 2.576

Copyright by ASTM Int'l (all rights reserved); 20

 D6300 − 08
TABLE A2.4 Constants for Approximating Critical Values of FA TABLE A2.5 Critical Values of the Normal DistributionA
100 (1 – P) % A(P) B(P) C(P) P 0.70 0.80 0.90 0.95 0.975 0.99 0.995
10.0 % 1.1131 0.77 0.527 Z 0.524 0.842 1.282 1.645 1.960 2.326 2.576
5.0 % 1.4287 0.95 0.681 2(1 – P) 0.60 0.40 0.20 0.10 0.05 0.02 0.01
2.5 % 1.7023 1.14 0.846 A
When P is less than 0.5 the appropriate critical value is the negative of the value
1.0 % 2.0206 1.40 1.073
corresponding to a probability (1 – P).
0.5 % 2.2373 1.61 1.250
0.1 % 2.6841 2.09 1.672
0.05 % 2.8580 2.30 1.857
A
For values of P not given above, critical values of F may be obtained by second
order interpolation/extrapolation of log (F) (either tabulated or estimated from the
formula) against log (1 – P).
x2µ
Z5 (A2.4)
s
A2.7.1 Critical values Z corresponding to a single-sided
and where µ and s are the mean and standard deviation
probability P, or to a double-sided significance level 2 (1 – P)
respectively of the normal distribution.
are given below in terms of the “standard normal deviate,”
where

A3. TYPES OF DEPENDENCE AND CORRESPONDING TRANSFORMATIONS (7.2)

A3.1 Types of Dependence The values and significances of all the constants are determined
A3.1.1 See Table A3.1. simultaneously as part of the simplex minimization. For
detailed discussion of simplex minimization consult a trained
A3.2 Transformation Procedure statistician.
A3.2.1 The following steps shall be taken in identifying the A3.2.1.4 In order to confirm the selected transformation
correct type of transformation and its parameters, B or B0, or type, and to estimate the parameter B in the case of the power
both. transformation, fit the line specified in Table A3.1, correspond-
A3.2.1.1 Plot laboratories standard deviations, D, and re- ing to the transformation in question, in accordance with the
peats standard deviations, d, against sample means in the form computational procedure in A4.3. For the power transforma-
of scatter diagrams. Refer to Figs. A3.1-A3.6 and identify the tion, coefficient B, shall differ significantly from zero and shall
type of transformation to be applied (if any). be rounded to a meaningful value. For the arcsin transforma-
A3.2.1.2 With the exception of the power transformation tion, b1 shall have a value not significantly different from 0.5.
(Type 2 in Table A3.1), the transformation parameter is either Similarly, b1 shall not significantly differ from a value of one
known in advance or estimated from the scatter diagrams. For for the logistic, log, and arctan transformations. In every case
the arcsin (Type 3) and logistic (Type 4) transformations, B will the test specified in Table A3.1 shall be applied at the 5 %
be the upper limit of the rating scale or “score” that defines significance level. Failure of this test implies either that the
results. For the log (Type 1) transformation, calculate B0 from type of transformation or its parameter B is incorrect. Similarly,
the intercept and slope (B0 = intercept/slope), estimated from coefficient b3 shall in every case be tested as zero. Failure in
the scatter diagrams. Similarly, estimate B from the intercept in this case implies that the transformation is different for
the case of the arctan (Type 5) transformation. In every case, B repeatability and reproducibility, and the procedures of Annex
or B0, or both, shall be rounded to give a meaningful value that A5 shall be applied. In some cases the presence of outliers (see
satisfies the plots for both the laboratories and repeats standard 7.3) can give rise to this difference, so the adequacy of a single
deviations. transformation should be reassessed after removing outlying
A3.2.1.3 In the case of the power transform, B and B0 = 0 observations, if any.
will be estimated as part of the line fitting procedure described A3.2.1.5 If the tests applied above were satisfactory, trans-
in the next section (A3.2.1.4). A non-zero B0 may be estimated form all the results accordingly, recalculate means and standard
by minimizing the sum of squared residuals from the fitted line. deviations using transformed results, and create new scatter
Function minimization using a simplex procedure due to diagrams as in A3.2.1. These will now show a uniform level for
Nelder and Mead (9) has been found satisfactory. This is laboratories standard deviation, and a uniform (but not neces-
applied to the functional form of the line shown in Table A3.1 sarily the same) level for repeats standard deviation. A statis-
using the calculated sample means and standard deviations. tical test for uniformity is given in 7.4.

Copyright by ASTM Int'l (all rights reserved); 21

 D6300 − 08
TABLE A3.1 Types of DependenceA
Form of Dependence Transformations Form of Line to be Fitted dx/dy Remarks
D = K(m + B0) y = log(x + B0) log(D) = bo + (x + B0) Care must be taken if (x + B0) is small, as
m + B0 > 0 Type 1 – “log” + b1log(m + B0) + b2 T + b3Tlog(m + B0) rounding becomes critical

Test: b1 = 1, b3 = 0

B 1–B
D = K(m + B0) y =( x + B0) log(D) = bo + Blog(m + B0) + b2T + (x + B0)B/(1 - B) B = 1⁄2 or 2 are common cases.
m + B0 > 0, Type 2 – “power” b3Tlog(m + B0) If B is not different from 1, use log
Bfi1 Test: B fi 1, b3 = 0 transform 1 above. The fitted line may pass
through the origin.

1/2
D=K[(m/B) (1 - m/B)] y=arcsin(x/B)1/2 log(D) = bo + b1log[m (B - m)] + b2T + 2[x (B - x)]1/2 This case often arises when results are
0#m#B Type 3 – “arcsin” b3Tlog[ m (B – m)] reported as percentages or qualitatively as
Test: b1 = 1/2, b3 = 0 “scores.” If x is always small compared to
B, the transformation reduces to y=(x)1/2, a
special case of 2 above.

D=K[ (m/B)(1- m/B)] y=log[x/(B-x)] log(D)= bo + b1log[m (B - m)] + b2 T + x (B - x)/B This case arises when results are reported
b3Tlog[m (B – m)] on a scale of 0 to B. If x is always small
0#m#B Type 4 – “logistic” compared to B, then the transformation
Test: b1 = 1, b3 = 0 reduces to y = log(x) a special case of 1
above.

D=K[(m2 + B2)/B] y = arctan(x/B) log(D)= bo + b1log(m2+B2) + b2T + (x2 + B 2)/ B The fitted line does not pass through the
b3Tlog(m2 + B2) origin. If B is small, the transformation
B>0 Type 5 – “arctan” reduces to y = 1/ x, a special case of 2
Test: b1 = 1, b3 = 0 above.
A
The forms of dependence above are shown graphically in the corresponding Figs. A3.1-A3.6. In all cases, K can be any positive constant, and “log” refers to natural
logarithms. The form of line to be fitted includes a dummy variable T (see A4.1) by which it is possible to test for a difference in the transformation as applied to repeatability
and reproducibility.

FIG. A3.1 Type 1, log FIG. A3.2 Type 2, power

Copyright by ASTM Int'l (all rights reserved); 22

 D6300 − 08

FIG. A3.3 Type 2, power

FIG. A3.4 Type 3, arcsin

FIG. A3.5 Type 4, logistic

Copyright by ASTM Int'l (all rights reserved); 23

 D6300 − 08

FIG. A3.6 Type 5, arctan

A4. WEIGHTED LINEAR REGRESSION ANALYSIS (7.2)

A4.1 Explanation for Use of a Dummy Variable 2:1 in the favor of reproducibility shall be applied by setting T1
A4.1.1 Two different variables Y1 and Y2, when plotted = 1 and T2 = –2, where T1 refers to the plot of laboratories
against the same independent variable X, will in general give standard deviation and T2 refers to the repeats standard devia-
different linear relationships of the form tion.
Y 1 5 b 10 1b 11X (A4.1) A4.2 Derivation of Weights Used in Regression Analysis
A4.2.1 In order to account for the relative precision of fitted
Y 2 5 b 20 1b 21X
variables in a regression analysis, weights shall be used that are
where the coefficients bij are estimated by regression analy- inversely proportional to the variances of the fitted variables.
sis. In order to compare the two relationships, a dummy A4.2.1.1 For a variable D, which is an estimate of popula-
variable T can be defined such that tion standard deviation s, based on v(D) degrees of freedom,
T = T1, a constant value for every observation of Y1, the variance of D is given by
T = T2, a constant value for every observation of Y2, and
Var ~ D ! 5 s 2 /2v ~ D ! (A4.6)
T 1 fi T2
2 2
A4.1.2 Letting Y represent the combination of Y1 and Y2, A4.2.1.2 Replacing s by its estimate D , the weight for this
plot a single relationship variable will be approximated by
Y 5 b 0 1b 1 X1b 2 T1b 3 TX (A4.2) w ~ D ! 5 2v ~ D ! /D 2 (A4.7)

where, as before, the coefficients bi are estimated by regres- A4.2.1.3 It is clear that as standard deviation D increases, so
sion analysis. By comparing Eq A4.1 and Eq A4.2), it is will the weight decrease. For this reason the fitted variable in
evident that the weighted regression shall instead be a function of standard
deviation, which yields weights independent of the fitted
b 10 5 b 0 1b 2 T 1 (A4.3)
variable.
b 20 5 b 0 1b 2 T 2 A4.2.1.4 In cases where a function g(D) is fitted, rather than
D itself, the variance formula becomes
and that therefore
1 1 s2
b 10 2 b 20 5 b 2 ~ T 1 2 T 2 ! (A4.4) Var @ log~ D ! # 5 Var ~ D ! 5 2 (A4.8)
D2 D 2v ~ D !
A4.1.3 Similarly, A4.2.1.5 Once again replacing s2 by its estimate D2, the
b 11 2 b 21 5 b 3 ~ T 1 2 T 2 ! (A4.5) weight for log(D) will be approximated by
A4.1.4 In order to test for a difference between b10 and b20 w @ log~ D ! # 5 2v ~ D ! (A4.9)
therefore, it is only necessary to test for a non-zero coefficient A4.2.1.6 In relation to laboratories standard deviation D and
b2. Similarly, to test for a difference between b11 and b21, test repeats standard deviation d, therefore, it is necessary to
for a non-zero coefficient b3. perform regression analysis in terms of log(D) and log(d),
A4.1.5 Any non-zero values can be chosen for T1 and T2. since weighting will then take account only of the amount of
However, since reproducibility is the basis of tests for quality data on which the standard deviation was based. A relationship
control against specifications, weighting shall reflect this in the estimated in this way will be less dependent on samples which
estimation of precision relationships. An “importance ratio” of have a high proportion of missing results.

Copyright by ASTM Int'l (all rights reserved); 24

 D6300 − 08
A4.2.1.7 Denoting degrees of freedom as v(D) for labora- a y3 5 a 31b 1 1a 32b 2 1a 33b 3
tory standard deviations D and v(d) for repeats standard
A4.3.1.5 Examples of aij and ayi elements, in terms of
deviations d, formulae for calculating weights then become
weighted means x̄i, are as follows
w @ log~ D ! # 5 2v ~ D ! (A4.10)
a 22 5 (w i ~ x 2i 2 x̄ 2 ! 2 a 23 5 (w i ~ x 2i 2 x̄ 2 ! ~ x 3i 2 x̄ 3 !
w @ log~ d ! # 5 2v ~ d ! (A4.11) (A4.16)
NOTE A4.1—Unweighted regression corresponds to weighted regres-
sion in which all the weights have a constant value 1. a y2 5 (w i ~ y i 2 ȳ ! ~ x 2i 2 x̄ 2 ! a yy 5 (w i ~ y i 2 ȳ ! 2
A4.3 Computational Procedure for Regression Analysis A4.3.1.6 Having solved the equations for b1, b2, and b3,
calculate the intercept from the weighted means of the vari-
A4.3.1 The following technique gives the best fitting ables as
straight line of the form of Eq A4.2.
A4.3.1.1 First draw up a table (see Table A4.1) giving b 0 5 ȳ 2 b 1 x̄ 1 2 b 2 x̄ 2 2 b 3 x̄ 3 (A4.17)
values of the variables to be plotted in the regression, together A4.3.1.7 Coefficient estimates bi can be summarized in
with corresponding weights. Functions g1 and g2 will always be tabular form, together with test statistics, as in Table A4.2.
natural logarithms corresponding to the transformation in A4.3.1.8 In order to complete the table, it is necessary to
question, as specified in A3.2. calculate the standard deviation of the observed y values about
A4.3.1.2 Using the symbols defined in Table A4.1, the line the estimated line. This is called the residual standard devia-
to be fitted (Eq A4.2) becomes tion, and is given by

Œ
y 5 b 0 1b 1 x 1 1b 2 x 2 1b 3 x 3 (A4.12)
1
A4.3.1.3 The intercept b0 can be eliminated by rewriting
s5 ~ a 2 b 1 a y1 2 b 2 a y2 2 b 3 a y3 !
n 2 4 yy
(A4.18)
this as
A4.3.1.9 Standard errors of the estimates then become
~ y 2 ȳ ! 5 b 1 ~ x 1 2 x̄ 1 ! 1b 2 ~ x 2 2 x̄ 2 ! 1b 3 ~ x 3 2 x̄ 3 !
e i 5 s =c ii, for i 5 1 to 3 (A4.19)
(A4.13)

where y, x1, x2, and x3 are weighted mean values, for and
example e0 5 (A4.20)
n

x̄ 2 5
(
i21
n
w i x 2i
(A4.14)
s Π1
n
1c 11x̄ 1 2 1c 22x̄ 2 2 1c 33x̄ 3 2 12c 12x̄ 1 x̄ 2 12c 13x̄ 1 x̄ 3 12c 23x̄ 2 x̄ 3

(w
i51
i where the elements cjj correspond to the inverse of the matrix
containing elements ajj.
and where n is the number of points (twice the number of A4.3.1.10 The t-ratios are the ratios (bi–K)/ ej, where K is a
samples) to be plotted. constant, and by comparing these to the critical values of t in
A4.3.1.4 The least squares solution of Eq A4.14 requires the Table A2.3, it is possible to test if coefficient bi differs from K.
solution of the set of simultaneous equations of the form If ti is greater than the critical value corresponding to 5 %
a y1 5 a 11b 1 1a 12b 2 1a 13b 3 (A4.15) significance and (n – 4) degrees of freedom, then the coefficient
can be regarded as differing from K. In particular, t1 will
a y2 5 a 21b 1 1a 22b 2 1a 23b 3
identify an inappropriate slope b1 and t3 will indicate whether
the slope is different for laboratories and repeats standard
TABLE A4.1 Arrangement of Variables for Regression Analysis
deviations. Since laboratories standard deviation will generally
Standard
Deviation
Sample Mean be larger than repeats standard deviation at the same level of
Sample Function Dummy T Tg2 Weight sample mean, t2 will in general indicate a non-zero coefficient
Function
g2
g1 b2.
1 g1 (D1) g2 (m1) 1 g2 (m1) 2y (D1)
2 g1 (D2) g2 (m2) 1 g2 (m2) 2y (D2) A4.4 Worked Example
3 g1 (D3) g2 (m3) 1 g2 (m3) 2y (D3)
· · · · · · A4.4.1 This section describes the fitting of a power function
· · · · · ·
· · · · · ·
(Type 2 of Table A3.1) using weighted linear regression
S g1 (Ds) g2 (ms) 1 g2 (ms) 2y (Ds) according to the procedure of A3.2. Rounded sample means
1 g1 (d1) g2 (m1) –2 –2g2 (m1) 2y (d1)
2 g1 (d2) g2 (m2) –2 –2g2 (m2) 2y (d2) TABLE A4.2 Presentation of Estimates from Regression Analysis
3 g1 (d3) g2 (m3) –2 –2g2 (m3) 2y (d3)
· · · · · · Fitted Coefficient Standard Error of
t-Ratio
· · · · · · Variable Estimate Estimate
· · · · · · Intercept b0 e0 t0
S g1 (ds) g2 (ms) –2 –2g2 (ms) 2y (ds) Sample Mean b1 e1 t1
Dummy b2 e2 t2
Symbol yi x1j x2i x3 i wi Dummy × mean b3 e3 t3

Copyright by ASTM Int'l (all rights reserved); 25

 D6300 − 08
and standard deviations are given in Table 3, 7.2, based on the TABLE A4.3 Arrangement of Variables for Sample Data
bromine number data in A2.1. Logarithm of
Logarithm of Dummy × log
A4.4.1.1 Scatter diagrams identified the power transforma- Sample Standard Dummy T Weight
Sample Mean (mean)
Deviation
tion as appropriate, as indicated by the log-log plot shown in
1 –0.3158 0.7655 1 0.7655 16
Fig. A4.1. 2 0.7969 4.1804 1 4.1804 18
A4.4.1.2 Transformation parameter B need not be estimated 3 –2.7046 –0.2802 1 –0.2802 28
from Fig. A4.1, since it will be given in the regression analysis 4 –1.5568 1.2932 1 1.2932 22
5 –1.2358 2.3888 1 2.3888 18
that follows. 6 0.4029 3.8755 1 3.8755 18
A4.4.1.3 The form of the line to be fitted (Table A3.1) is 7 1.0762 4.7378 1 4.7378 18
8 –1.8401 0.1975 1 0.1975 18
log~ D ! 5 b 0 1b 1 log~ m ! 1b 2 T1b 3 Tlog~ m ! (A4.21)
1 –2.0644 0.7655 –2 –1.5309 18
A4.4.1.4 The table of values to be fitted (see Table A4.1) is 2 –0.2015 4.1804 –2 –8.3609 18
shown in Table A4.3. 3 –2.9957 –0.2802 –2 0.5605 18
4 –2.1585 1.2932 –2 –2.5864 18
A4.4.1.5 Least squares regression requires the solution of 5 –2.3613 2.3888 –2 −4.7775 18
the simultaneous equations 6 –0.6415 3.8755 –2 –7.7510 18
7 −0.0674 4.7378 –2 −9.4756 18
614.671 5 999.894b 1 2 35.8524b 2 2 493.045b 3 (A4.22) 8 –2.8612 0.1975 –2 −0.3949 18

188.526 5 35.8524b 1 1673.920b 2 11409.58b 3 Symbol yi x1i x2 i x3i wi

195.477 5 2493.045b 1 11409.58b 2 15362.27b 3

TABLE A4.4 Presentation of Estimates from Sample Data
A4.4.1.6 Also required are
Coefficient Estimate Standard Error of
a yy 5 505.668 (A4.23) Fitted Variable t-Ratio
bi Estimate
Intercept –2.4064
s 5 2.23868 Log (mean) 0.63773 0.07359 8.67
Dummy 0.25496 0.13052 1.95
A4.4.1.7 The solution is summarized in Table A4.4 (see Dummy × log (mean) 0.02808 0.04731 0.59
Table A4.2):
A4.4.1.8 Comparing the t-ratios with the critical 5 % values
for 12 degrees of freedom (namely 2.179) given in Table A2.3,
it can be seen that the slope is significantly non-zero (b1 =
0.638), confirming that a transformation was required. Further-
more, since coefficient b3 does not significantly differ from
zero, the slope (and resulting transformation) is the same for
both laboratories and repeats standard deviations.
A4.4.1.9 As the slope b1 = 0.638 has a standard error of
0.074, the approximate 66 % confidence region of 0.638 6
0.074 will contain the value 2/3. Rounding to this value is
therefore reasonable, and leads to the convenient transforma-
tion
y 5 x 1/3 (A4.24)

A4.4.1.10 Having applied this transformation and recalcu-

lated sample means and standard deviations, corresponding
scatter diagrams are shown in Fig. A4.2. These show uniform
levels for both laboratories and repeats standard deviations for
all samples except Sample 1. In the case of the latter sample,
FIG. A4.1 Precisions Vary with Level the extreme point is due to outliers.

Copyright by ASTM Int'l (all rights reserved); 26

 D6300 − 08

FIG. A4.2 After Transforming, Precisions Do Not Vary with Level

A5. DIFFERENT (TWO) TRANSFORMATIONS FOR REPEATABILITY AND REPRODUCIBILITY

A5.1 Introduction (4) Having removed repeatability outliers, select a trans-

form suitable for the Dj. Using this second transformation, do
A5.1.1 Occasionally a single transformation cannot be
the complete ANOVA, except do not remove any additional
found that eliminates simultaneously the dependence of both
outliers for repeatability.
repeatability and reproducibility on property level. When this (5) After removing reproducibility outliers, go back to step
happens, it is an indication that the sources of variation 1. If a single transformation works now, then use it. Otherwise
contributing to repeatability and reproducibility are of a very continue with step 6.
different nature. At the same time, reproducibility may be very (6) Estimate R from the ANOVA in Part 4.
much larger than repeatability for almost all materials tested.
This may occur if the repeatability conditions are not correctly A5.1.2 If a single transformation cannot be found, separate
identified, and/or if all steps of the method are not “repeated” transformations must be applied to dj and Dj of A1.1 and A1.2.
independently. Alternatively, there may be single large con- The transformations of Table A3.1 apply, but there will be no
tributor to inter-lab variation (a laboratory bias, for example) dummy variable T in the models, and no parameters b3 to test.
that needs to be identified and eliminated. It is advisable to The computational methods of Annex A4 still apply, but
investigate these possibilities diligently before making use of without the complicating dummy variable.
separate transformations for repeatability and reproducibility. A5.1.3 Although separate models are to be fit to the dj and
A5.1.1.1 Outline of main steps involved in a two-transform Dj, efforts should be made to make them as alike as possible,
process: without sacrificing significantly the quality of the fits. For
(1) A single transformation should be used whenever (R/r) example, if power function transformations, “Type 2,” are
does not vary with level. The feasibility of a single transform fitted to both, it would be desirable that one or the other of the
should be assessed using regression plots of log(R/r) on mean pairs of parameters b2 and B0 take on the same value for both
values and, separately on log(mean values). It is strongly models. (If both are alike, then a single transformation could
recommended that a single transformation be used whenever and should be used.) If a common value for either parameter
data does not overwhelmingly suggest otherwise. If separate pair can be found so that the fit of neither model is significantly
transformations are indicated, then continue to step 2. degraded, then that common value should be used.
(2) Choose a transformation suitable for dj only, as is A5.1.4 The identification and removal of outliers can affect
described in Annex A4, only no dummy variables are required. the choice of transformations—the process is an iterative one.
Examine transformed data for repeatability outliers (see 7.3.1 As outliers are removed, the necessity of separate transforma-
and 7.3.2), and iterate transformation selection as necessary. tions should be reexamined.
Compute estimate of r.
(3) Having removed repeatability outliers, re-compute cell A5.2 Example Data
averages, sample averages, dj, Cj, and Dj from the remaining A5.2.1 Table A5.1 contains data from a round robin on
(untransformed) data. If a single transformation works now, Derived Cetane Number (DCN – D6890) on diesel fuels. These
then use it. Otherwise continue with step 4. data will be used as an example in the following sections. The

Copyright by ASTM Int'l (all rights reserved); 27

 D6300 − 08
TABLE A5.1 Derived Cetane Numbers
Lab Repeat D1 D2 D3 D4 D5 D6 D7 D8 D9 D10 D11 D12 D13 D14 D15
Lab 1 1 51.2 33.6 43.3 46.8 52.2 53.9 36.8 60.6 50.1 42.1 57.1 60.4 50.5 56.5 45.2
2 51.4 33.6 43.3 46.5 51.7 53.2 37.1 60.3 50.1 42.1 56.9 60.4 51.4 56.6 45.8
Lab 2 1 52.1 33.7 42.7 46.0 52.2 54.3 37.4 60.7 50.9 42.6 57.2 60.6 51.5 57.1 45.6
2 52.0 34.3 43.0 46.0 52.3 54.6 36.8 60.4 50.6 43.1 57.4 61.0 51.5 57.3 45.8
Lab 3 1 53.3 35.1 44.9 48.0 54.3 55.4 38.4 62.2 52.2 43.4 58.1 63.4 52.8 58.1 47.2
2 53.8 35.6 44.5 47.4 54.6 55.1 38.3 62.3 51.8 43.3 58.2 63.1 54.1 58.6 47.6
Lab 4 1 51.9 34.8 44.2 47.8 53.8 55.0 37.9 60.5 51.3 43.6 57.6 61.8 51.8 57.4 46.2
2 51.7 34.9 43.9 47.9 54.1 54.9 38.2 60.9 51.1 43.3 57.5 61.8 51.4 57.7 46.5
Lab 5 1 50.8 34.9 43.3 45.4 52.4 53.8 37.8 60.6 50.2 42.5 56.8 61.4 50.8 56.6 46.0
2 51.4 34.6 43.6 46.3 53.2 54.3 37.8 60.8 50.1 42.4 56.9 61.7 51.0 56.8 46.0
Lab 6 1 51.7 33.5 42.6 45.8 52.0 53.3 36.9 59.5 50.0 42.3 57.1 61.0 50.7 56.6 45.7
2 51.3 33.5 42.6 46.2 52.5 53.8 36.9 60.0 49.8 41.9 56.5 60.9 51.0 55.9 45.2
Lab 7 1 52.5 35.6 44.8 47.5 53.6 56.0 38.7 62.1 51.9 43.0 59.5 63.7 52.9 58.8 47.1
2 52.5 35.4 44.9 46.8 54.7 55.2 39.9 61.8 52.3 43.4 59.2 63.5 52.8 58.9 47.4
Lab 8 1 50.7 33.2 42.5 45.2 51.7 52.4 36.9 59.3 49.8 41.3 56.0 59.5 49.9 55.8 46.3
2 50.9 34.1 42.5 45.8 51.6 52.8 36.8 59.2 49.3 41.9 56.3 59.5 49.9 55.8 44.7
Lab 9 1 50.5 33.8 42.6 45.8 50.9 51.9 36.6 58.1 50.2 42.0 54.5 59.2 50.1 55.8 45.0
2 50.6 34.3 42.5 45.0 50.6 52.1 36.8 58.1 49.9 41.4 55.2 59.8 50.0 55.9 45.3
Lab 10 1 51.5 34.5 42.9 47.8 52.0 53.0 38.2 60.8 50.8 41.9 56.2 61.7 52.2 57.4 45.6
2 52.3 34.4 42.4 47.8 52.3 53.4 37.8 61.0 50.6 41.3 56.8 62.0 52.5 57.1 45.6
Mean mj 51.7 34.4 43.3 46.6 52.6 53.9 37.6 60.5 50.7 42.4 57.1 61.3 51.4 57.0 46.0
dj 0.2768 0.3105 0.1820 0.3833 0.3756 0.3298 0.3249 0.2094 0.2060 0.2957 0.2846 0.2099 0.3855 0.2431 0.4245
C j2 1.502 1.056 1.602 1.856 2.891 2.708 1.467 2.991 1.574 1.094 2.881 3.892 2.661 2.074 1.248
Dj 0.888 0.759 0.904 1.001 1.231 1.187 0.887 1.232 0.899 0.769 1.217 1.403 1.185 1.033 0.845
Ratio uj 3.21 2.44 4.97 2.61 3.28 3.60 2.73 5.88 4.36 2.60 4.28 6.68 3.07 4.25 1.99

means, repeats standard deviation and laboratories standard yields a slope of 0.117, with standard error 0.033, which
deviation have been computed and are shown in the table, as confirms that the uj vary with concentration.
well as the ratios uj. Fig. A5.1 shows that the uj appear to vary
with concentration, mj. Regression of the uj on the means, mj,

FIG. A5.1 Precision Ratio Increases with Mean DCN

Copyright by ASTM Int'l (all rights reserved); 28

 D6300 − 08
A5.3 Repeatability Transformation and Outlier Rejection TABLE A5.2 Quantiles of the Standard Normal Probability
Distribution
A5.3.1 Following Annex A4, perform a weighted linear p F–1(p ) p F–1(p ) p F–1(p )
regression of the logarithms of the repeats standard deviations, 0.01 -2.33 0.34 -0.41 0.67 0.44
dj, on the logarithms of the sample mean concentrations, mj. 0.02 -2.05 0.35 -0.39 0.68 0.47
0.03 -1.88 0.36 -0.36 0.69 0.50
Alternatively, use log(mj + B0) as the regressor variable, where 0.04 -1.75 0.37 -0.33 0.70 0.52
B 0 > – min(mj) is chosen to minimize the sum of weighted 0.05 -1.64 0.38 -0.31 0.71 0.55
squared residuals. This leads us to a model of Type 2 in Table 0.06 -1.55 0.39 -0.28 0.72 0.58
0.07 -1.48 0.40 -0.25 0.73 0.61
A3.1, but with no dummy variable: 0.08 -1.41 0.41 -0.23 0.74 0.64
log~ d ! 5 b 0 1Blog~ m1B 0 ! (A5.1) 0.09 -1.34 0.42 -0.20 0.75 0.67
0.10 -1.28 0.43 -0.18 0.76 0.71
A5.3.2 The parameter B should be rounded to carry as few 0.11 -1.23 0.44 -0.15 0.77 0.74
0.12 -1.17 0.45 -0.13 0.78 0.77
digits as possible, provided the rounded result does not differ 0.13 -1.13 0.46 -0.10 0.79 0.81
from the weighted least squares solution by more than twice its 0.14 -1.08 0.47 -0.08 0.80 0.84
standard error (Eq A4.19). If |B| itself is less than twice this 0.15 -1.04 0.48 -0.05 0.81 0.88
0.16 -0.99 0.49 -0.03 0.82 0.92
standard error, then B should be rounded to zero, as this implies 0.17 -0.95 0.50 0.00 0.83 0.95
that no transformation is necessary. If B cannot be rounded to 0.18 -0.92 0.51 0.03 0.84 0.99
zero, then B0 should be rounded to carry no more than two 0.19 -0.88 0.52 0.05 0.85 1.04
0.20 -0.84 0.53 0.08 0.86 1.08
significant digits. 0.21 -0.81 0.54 0.10 0.87 1.13
A5.3.3 In rare cases, it may be necessary to fit a model of 0.22 -0.77 0.55 0.13 0.88 1.17
0.23 -0.74 0.56 0.15 0.89 1.23
Types 3, 4, or 5. Use Table A3.1 to guide such an endeavor. 0.24 -0.71 0.57 0.18 0.90 1.28
0.25 -0.67 0.58 0.20 0.91 1.34
A5.3.4 Based on the regression model of Eq A5.1, trans- 0.26 -0.64 0.59 0.23 0.92 1.41
form every response using the appropriate transformation: 0.27 -0.61 0.60 0.25 0.93 1.48
yijk = (xijk + B0)1-B, for a Type 2 model with B fi 1, yijk = 0.28 -0.58 0.61 0.28 0.94 1.55
0.29 -0.55 0.62 0.31 0.95 1.64
log(xijk + B0) for a Type 1 model (that is, a Type 2 model with 0.30 -0.52 0.63 0.33 0.96 1.75
B = 1), or as guided by Table A3.1 for a model of a different 0.31 -0.50 0.64 0.36 0.97 1.88
type. Re-compute the cell differences from the transformed 0.32 -0.47 0.65 0.39 0.98 2.05
0.33 -0.44 0.66 0.41 0.99 2.33
results: eij = yij1 – yij2.
A5.3.5 Test for Uniformity of Repeatability:
A5.3.5.1 Apply Cochran’s criterion to compare the maxi-
the sum of squared differences, ( e ij2 , is 27.9. Cochran’s ratio is
mum or the eij2 to the sum of squared differences, ( e ij, as
2
i,j
i,j
0.0921, which is less than the critical 1 % value obtained from
described in 7.3.2 and A1.5. an extended version of Table A2.3 (0.1130).
A5.3.5.2 Test repeats standard deviation for outlying A5.3.6.3 As no data are missing or removed at this point,
samples, as in 7.4. Cochran’s test may be applied to:
A5.3.5.3 A half-normal plot of remaining absolute differ-
ences, |eij|, may be produced as follows: Rank the absolute
maxj (e
i
2
ij
maxj ~ d j2 !
5 5 0.1292 (A5.2)
differences from smallest to largest. If n is the number of (e
i,j
2
ij (dj
2
j
differences remaining, and the rank a specific |eij | is k, then plot where the maximum occurs on fuel S15. Table A2.3, for n = 10
|eij| against F-1(n +k⁄2n+1 ), where F-1 is the inverse of the degrees of freedom and n = 15 variances, is 0.1919, so there is
standard normal distribution function. F-1(n +k⁄2n+1 ), is tabu- no reason to reject any fuel for excessive repeatability varia-
lated in Table A5.2. In the event that the half-normal plot does tion.
not approximate a straight line, especially for the largest |eij|,
A5.3.6.4 Fig. A5.2 is the half-normal plot of the 150
then additional outliers may remain. Repeat Cochran’s test
absolute differences. The trace is not linear, indicating that the
(7.4.3) with significance level 2 %. (Due to rounding, an
distribution is not normal. Returning to Cochran’s test with
excessive number of |eij| may be zero. Then the half-normal
significance level 2 %, the critical value for A5.3 is 0.1815,
plot may fail to approximate a line for small values of |eij|. This
which still does not suggest rejection of the largest pair
is not a reason to suspect additional outliers.)
difference.
A5.3.6 Worked Example:
A5.3.6.1 As no data are missing, an unweighted regression A5.4 Reproducibility Transformation and Outlier Rejec-
of log(dj) on log(mj) is performed. The estimated slope is tion
–0.445 with standard error 0.447, so is not significantly A5.4.1 Returning now to the data before transformation,
different from zero. Trial regressions of log(dj) on log(mj + B0 remove the results identified as repeat outliers in A5.3.5, if any.
) fail to yield a significant slope for any value of B0 . Thus we If outliers have been removed, re-compute the means, mj,
conclude that repeatability does not differ significantly with repeats variances, dj2, and laboratories variance, Dj2 (see
concentration, and no transformation is required in A5.3.4. A1.4.1-A1.4.3), for each sample, and reassess the necessity of
A5.3.6.2 The largest of the eij2 (fuel D15, Lab 8) is 2.57 and separate transformations (see 7.2.3).

Copyright by ASTM Int'l (all rights reserved); 29

 D6300 − 08

FIG. A5.2 Half-Normal Plot of Absolute Pair Differences

A5.4.2 If a single, suitable transformation can now be A5.4.8.1 Following 7.3.4, test the yijk for uniformity of
found, return to 7.2.5. reproducibility (outliers).
A5.4.3 If a single, suitable transformation still cannot be A5.4.8.2 Following 7.4, test laboratories standard deviation
found, follow Annex A4 to perform a weighted linear regres- for outlying samples.
sion of the logarithms of the laboratories standard deviations,
A5.4.9 Estimating Missing or Rejected Values—If data are
Dj, on the logarithms of the sample mean concentrations, mj.
missing, or if outliers have been removed in either A5.3.5 or
Weight each observation log(Dj) by Lj, the number of labs that
A5.4.8, estimate the missing yijk in accordance with 7.5.
have measured sample j. Alternatively, regress the log(Dj) on
log(mj + B'0), where B'0 > – min(mi) is chosen to minimize the A5.4.10 Rejection Test for Outlying Laboratories—
sum of weighted squared residuals. This leads us to a model of Following 7.6, and using the data as transformed in A5.4.6, test
Type 2 in Table A3.1, but with no dummy variable: the laboratory means for outliers using Hawkins’ test.
D 5 K ~ m1B' 0 ! B' (A5.3)
A5.4.11 Confirmation of Selected Transformations—If any
A5.4.4 The parameter B' should be rounded to the nearest outliers have been removed in A5.4.8 or A5.4.10, check to see
1/10. B'0 should be rounded to carry no more than two that these rejections have not invalidated the transformation of
significant digits. A5.4.6, and reassess again the need for separate transforma-
A5.4.5 In rare cases, it may be necessary to fit a model of tions before continuing to A5.5.
Types 3, 4, or 5. Use Table A3.1 to guide such an endeavor.
A5.4.12 Worked Example:
A5.4.6 Based on the regression model of Eq A5.3, trans- A5.4.12.1 As no repeats data have been removed, go
form every response using the appropriate transformation: directly to A5.4.3. The means and variances do not need to be
yijk = (x + B'0) 1-B' for a Type 2 model with B' fi 1, yijk = recomputed and remain as shown in Table A5.1.
log(xijk + B'0) for a Type 1 model (that is, a Type 2 model with
A5.4.12.2 Regressing the log(Dj) on log(mj + B'0), for a
B' = 1), or as guided by Table A3.1 for a model of a different
type. number of choices of B'0, with constant weights, we find that
the sum of squared residuals takes its minimum value of
A5.4.7 Do not test for the uniformity of repeatability (see 0.1575 when B'0 is very large—greater than 106. But when
7.3.3). Do not reject any additional data as repeat outliers. B'0 = 4, the sum of squared residuals is 0.1691, an increase of
A5.4.8 Test for Uniformity of Reproducibility: less than 1/12 = 1/(S–3). This penalty is reasonable for

Copyright by ASTM Int'l (all rights reserved); 30

 D6300 − 08
constraining one parameter, and the specific choice and results A5.5.1.4 The repeatability variance is one-half the mean
in a slope B very close to 1. These are the values that have been square for repeatability. The repeatability standard deviation is
selected. the square root of the repeatability variance.
A5.4.12.3 The transformation appropriate to the parameters A5.5.1.5 The estimate of repeatability for results as trans-
B'0 = 4 and B' = 1 is: formed according toA5.3.4 is the product of the square root of
y ijk 5 ln~ x ijk14 ! (A5.4) the mean square for repeatability and the “ t-value” with
The cell sums were computed as aij = yij1 + yij2 . These are degrees of freedom for repeats. Use the t-value corresponding
shown in Table A5.3. to a two-sided probability of 95 %. (See Table A2.5.) Round
A5.4.12.4 The largest difference between a cell sum and its the calculated estimate in accordance with Practice E29.
corresponding sample mean is 0.0898, from Lab 7, fuel D7. A5.5.1.6 The estimate for repeatability for untransformed
The root sum of squares of such differences is 0.4704, resulting (raw) results is given by:
in a Hawkins ratio of 0.1908. We enter Table A1.5, n = 9 and
n = 135. As the tabled value for n = 9 and n = 150, 0.2416, is r~x! 5 U U~!
dx
dy
r y (A5.5)
larger than our ratio, no cell sums are identified as outliers. where |dx/dy | is the absolute value of the reciprocal of the
A5.4.12.5 The fuel with the largest sum of squared devia- derivative of the transformation of A5.3.4.
tions of cell totals from their means is D5, and that sum is
A5.5.2 Worked Example:
0.0187. The total for all fuels is 0.2213. The degrees of
freedom for every fuel’s sum of squares is the same, 9. The A5.5.2.1 From A5.3.6.2, E5 ( e ij2 527.90. There are 150
ratio of the largest sum to the total is 0.0844, while the value i,j
from Table A2.2 for n = 15 and n = 9, would have to be larger pairs, so the mean square for repeats is 27.9/150 = 0.1860. The
than the value for n = 15 and n = 10, namely 0.1919. Thus there repeatability variance is 0.1860/2 = 0.0930, and the repeatabil-
is no indication that the between labs variation is larger for ity standard deviation is the square root, 0.3040. The t critical
some fuels than for others. value, from an extended version of Table A2.5, with 135
A5.4.12.6 The means of the cell totals across fuels are given degrees of freedom, is 1.978, so the repeatability estimate is
in Table A5.3. The largest deviation from the grand mean is 1.978=0.186050.8530. As there was no transformation for
from Lab 7, at 0.0577. The root sum of squared deviations is repeatability, this value is appropriate for all concentrations
0.1108, and Hawkins ratio is 0.0577/0.1108 = 0.5212. Entering within the range the fuels tested. In conformance with Practice
Table A1.5 with n = 10 and n = 1, we see that so long as this E29, the repeatability should be reported as:
ratio is less than 0.7175, there is no indication that any lab
r x 5 0.85 numbers (A5.6)
mean is an outlier.
A5.5.3 Analysis of Variance and Estimate of Reproducibil-
A5.5 Analysis of Variance and Calculation of Precision ity:
Estimates A5.5.3.1 Using the transformed yijk from A5.4.6, carry out
A5.5.1 Repeatability Estimate: an analysis of variance as in Section 8.
A5.5.1.1 Using the eij as produced from the transformation A5.5.3.2 Plot the absolute residuals from the ANOVA on a
in A5.3.4, but eliminating any differences from samples half-normal plot as was done with the pair differences in
rejected in A5.4.8.2, and from labs rejected in A5.4.10, A5.3.5.3.
calculate the sum of squares for repeats, E5 ( ( e ij2 . A5.5.3.3 The reproducibility variance, degrees of freedom
i j
for reproducibility, and the reproducibility estimate for results
A5.5.1.2 The degrees of freedom for repeats is the number as transformed according toA5.4.6 are given exactly as in
remaining differences, that is, the number of terms in the sum [Link]. Round the reproducibility estimate in accordance with
of the previous paragraph. Practice E29.
A5.5.1.3 The mean square for repeats is the sum of squares A5.5.3.4 The estimate for reproducibility for untransformed
for repeats divided by the degrees of freedom for repeats. (raw) results is given by:

TABLE A5.3 Sums of Transformed Results

Fuel D1 D2 D3 D4 D5 D6 D7 D8 D9 D10 D11 D12 D13 D14 D15 Mean
Lab 1 7.90 7.05 7.51 7.66 7.91 7.99 7.22 8.21 7.83 7.48 8.09 8.20 7.86 8.07 7.64 7.77
Lab 2 7.96 7.13 7.60 7.73 8.00 8.02 7.29 8.26 7.85 7.52 8.10 8.22 7.88 8.09 7.64 7.78
Lab 3 7.89 7.10 7.57 7.73 7.98 8.01 7.28 8.21 7.90 7.54 8.13 8.30 7.96 8.13 7.72 7.84
Lab 4 7.87 7.10 7.54 7.65 7.93 7.98 7.27 8.21 7.87 7.54 8.11 8.25 7.89 8.11 7.67 7.81
Lab 5 7.88 7.02 7.50 7.66 7.91 7.96 7.22 8.18 7.83 7.50 8.08 8.24 7.86 8.07 7.66 7.79
Lab 6 7.92 7.14 7.61 7.71 7.98 8.04 7.34 8.25 7.82 7.48 8.08 8.22 7.86 8.06 7.63 7.77
Lab 7 7.86 7.03 7.50 7.64 7.89 7.93 7.21 8.16 7.91 7.53 8.17 8.31 7.93 8.15 7.71 7.85
Lab 8 7.85 7.05 7.50 7.63 7.85 7.90 7.21 8.12 7.81 7.46 8.06 8.17 7.82 8.04 7.63 7.75
Lab 9 7.90 7.08 7.51 7.73 7.91 7.95 7.28 8.22 7.83 7.46 8.01 8.17 7.83 8.04 7.62 7.74
Lab 10 7.90 7.05 7.51 7.66 7.91 7.99 7.22 8.21 7.85 7.46 8.07 8.25 7.92 8.09 7.64 7.79
Mean 2mj 7.89 7.07 7.54 7.68 7.93 7.97 7.25 8.20 7.85 7.50 8.09 8.23 7.88 8.09 7.66 7.79

Copyright by ASTM Int'l (all rights reserved); 31

 D6300 − 08

r~x! 5 U U~!
dx
dy
r y (A5.7)
A5.5.4.3 By 8.3, the expected mean square for labs is s02 +
2s12 + 30s22. The expect mean square for interactions is s02 +
where |dx/dy | is the absolute value of the reciprocal of the 2s12, and the expect mean square for repeats is s02. Thus, to
derivative of the transformation of A5.4.6. estimate reproducibility variance, s02 + s12 + s22, we take 1⁄30
A5.5.4 Worked Example: (0.00875) + 14⁄30 (0.000125) + 1⁄2 (0.000035) = 0.000368. The
A5.5.4.1 ANOVA applied to the complete Table A5.3 re- degrees of freedom for this variance is approximately 14
sults in Table A5.4. Note that the statistics for repeats in this (Warning! Less than 30!), so the t-value we use is 2.145. The
table have all been computed from differences of transformed estimate of reproducibility of the transformed results is:
results.
A5.5.4.2 The residuals from the ANOVA are plotted in Fig. R y 5 2.145=2 3 0.000368 5 0.0582 (A5.8)
A5.3. The plot appears very straight, indicating normal residu-
als. A5.5.4.4 Reproducibility in terms of sulfur concentration is
given by:
TABLE A5.4 Analysis of Variance for Transformed

Source
Sulfur Concentrations
Sum Sq df Mean Sq
Rx 5 U U
dx
dy
Ry (A5.9)
As y = ln(x + 4), ⁄
dx dy = (x + 4) and
Samples 6.76 14
Labs 0.0787 9 0.00875 R x 5 ~ x14 ! R y 5 0.0582~ x14 !
Interaction 0.0158 126 0.000125
Repeats 0.00532 150 0.000035
Total 6.86 299

Copyright by ASTM Int'l (all rights reserved); 32

 D6300 − 08

FIG. A5.3 Half-Normal Plot of Residuals from ANOVA

APPENDIX

(Nonmandatory Information)

X1. DERIVATION OF FORMULA FOR CALCULATING THE NUMBER OF SAMPLES REQUIRED (see 6.4.3)

X1.1 An analysis of variance is carried out on the results of Q = s22 /s02,

the pilot program. Setting the three expressions in 8.3.1 equal v = reproducibility degrees of freedom,
to the corresponding mean squares and solving yields rough L = number of laboratories, and
estimates of the three components of variance, namely: S = number of samples.
s02 for repeats,
X1.3 The formula rearranges into the form
s12 for laboratories × samples interaction, and
s22 for laboratories. aS1b 5 0 (X1.2)

X1.2 Substituting the above in Eq 39 ([Link]) for calculat- where:

ing the reproducibility degrees of freedom, this becomes a = vQ2 – (1 + P + Q) 2(L – 1), and
b = v[(2Q + 1/2 + P) (1/2 + P) + 0.25 (L – 1) / L].
~ 11P1Q ! 2 @ ~ 1/21P ! /S1Q # 2 ~ S 2 1 ! ~ 1/21P ! 2 1
5 1 1 X1.3.1 Therefore S = –b/a gives the values of S for given
v ~L 2 1! S 2~ L 2 1 ! 4LS
(X1.1)
values of L, P, Q, and v.

where: X1.4 Fig. 1 is based on v = 30 degrees of freedom. For

non-integral values of P and Q, S can be estimated by second
P = s12/s0 2,
order interpolation from the table.

Copyright by ASTM Int'l (all rights reserved); 33

 D6300 − 08

REFERENCES

(1) Standard Methods for Analysis and Testing of Petroleum and Related 2nd ed., 1963, Example 6B.1, pp. 236-238.
Products, The Institute of Petroleum, London, England, 1993, Appen- (6) Kolodziejczyk, S., Biometrika, Vol 27, 1935, pp. 161-190.
dix E. (7) Welch, B. L., Biometrika, Vol 29, 1938, pp. 350-362.
(2) Shapiro, S. S., and Wilks, M. B., Biometrika, Vol 52, 1965, pp. (8) Merrington, M., and Thompson, C. M., Biometrika, Vol 33, 1943, pp.
591-611. 73-88.
(3) Cochran, W. G., Ann. Eugen., Vol 11, 1941, pp. 47-52. (9) Nelder, J. A., and Mead, R., Computer Journal, Vol 7, 1965, pp.
(4) Hawkins, D. M., Identification of Outliers, 1980, pp. 136-138. 308-313.
(5) Davies, O. L., et al, Design and Analysis of Industrial Experiments,

ASTM International takes no position respecting the validity of any patent rights asserted in connection with any item mentioned
in this standard. Users of this standard are expressly advised that determination of the validity of any such patent rights, and the risk
of infringement of such rights, are entirely their own responsibility.

This standard is subject to revision at any time by the responsible technical committee and must be reviewed every five years and
if not revised, either reapproved or withdrawn. Your comments are invited either for revision of this standard or for additional standards
and should be addressed to ASTM International Headquarters. Your comments will receive careful consideration at a meeting of the
responsible technical committee, which you may attend. If you feel that your comments have not received a fair hearing you should
make your views known to the ASTM Committee on Standards, at the address shown below.

This standard is copyrighted by ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959,
United States. Individual reprints (single or multiple copies) of this standard may be obtained by contacting ASTM at the above
address or at 610-832-9585 (phone), 610-832-9555 (fax), or service@[Link] (e-mail); or through the ASTM website
([Link]). Permission rights to photocopy the standard may also be secured from the ASTM website ([Link]/
COPYRIGHT/).

Copyright by ASTM Int'l (all rights reserved); 34