30/05/2016

6BASICSTATISTICALTOOLS

Producedby:NaturalResources
ManagementandEnvironment
Department
Title:Guidelinesforqualitymanagementinsoilandplantlaboratories.(FAO
Soils...

6BASICSTATISTICALTOOLS
Therearelies,damnlies,andstatistics......
(Anon.)
6.1Introduction
6.2Definitions
6.3BasicStatistics
6.4Statisticaltests

6.1Introduction
Intheprecedingchaptersbasicelementsfortheproperexecutionofanalyticalwork
suchaspersonnel,laboratoryfacilities,equipment,andreagentswerediscussed.
Beforeembarkingupontheactualanalyticalwork,however,onemoretoolforthe
qualityassuranceoftheworkmustbedealtwith:thestatisticaloperationsnecessaryto
controlandverifytheanalyticalprocedures(Chapter7)aswellastheresultingdata
(Chapter8).
[Link]
reasonwhyacomplexsystemofprecautionstopreventerrorsandtrapstodetect
[Link]
[Link]
performanceoftheanalysisasawholeandalsooftheinstrumentsandoperators
[Link],
statisticaltreatmentofdataisindispensable.
Amultitudeofdifferentstatisticaltoolsisavailable,someofthemsimple,some
complicated,[Link],themost
importantcommonoperationisthecomparisonofdata,orsetsofdata,toquantify
accuracy(bias)[Link],withafewsimpleconvenientstatisticaltools
mostoftheinformationneededinregularlaboratoryworkcanbeobtained:the"ttest,
the"Ftest",[Link],examplesofthesewillbegiveninthe
ensuingpages.
Clearly,statisticsareatool,[Link],withoutstatistical
treatment,byanexperiencedanddedicatedanalystmaybejustasusefulasstatistical
[Link]
simplifyingdata,topermitsomeobjectiveestimateshowingthatananalysisisunder
[Link]
statisticalproceduresarerecordedandcanberetrieved.

6.2Definitions
[Link]

1/29

30/05/2016

6BASICSTATISTICALTOOLS

6.2.1Error
6.2.2Accuracy
6.2.3Precision
6.2.4Bias
DiscussingQualityControlimpliestheuseofseveraltermsandconceptswitha
specific(andsometimesconfusing)[Link],someofthemostimportant
conceptswillbedefinedfirst.

6.2.1Error
Erroristhecollectivenounforanydepartureoftheresultfromthe"true"value*.
Analyticalerrorscanbe:
[Link],quantifiedwith
the"standarddeviation".
[Link]"true"value,
quantifiedas"meandifference"([Link]
andthemeanofreplicatedeterminations).
[Link],unrelatedtotheconcentrationofthesubstanceanalyzed(the
analyte).
[Link],[Link].
*The"true"valueofanattributeisbynatureindeterminate
[Link]
scienceforseveralattributesthereisnosuchthingasthetrue
valueasanyvalueobtainedismethoddependent([Link]
exchangecapacity).Obviously,thisdoesnotmeanthatno
[Link],
however,emphasizetheneedfortheestablishmentof
standardreferencemethodsandtheimportanceofexternal
QC(seeChapter9).

6.2.2Accuracy
The"trueness"ortheclosenessoftheanalyticalresulttothe"true"[Link]
constitutedbyacombinationofrandomandsystematicerrors(precisionandbias)and
[Link]
accuratedeterminationproducesa"true"quantitativevalue,[Link]
bias.

6.2.3Precision
[Link]
measureofdispersionorscatteringaroundthemeanvalueandusuallyexpressedin
termsofstandarddeviation,standarderrororarange(differencebetweenthehighest
andthelowestresult).

6.2.4Bias
[Link]

2/29

30/05/2016

6BASICSTATISTICALTOOLS

Theconsistentdeviationofanalyticalresultsfromthe"true"valuecausedbysystematic
[Link]"trueness"which
istheagreementofthemeanofanalyticalresultswiththetruevalue,[Link]
[Link]
contributingtobias:
[Link]
Thedifferencebetweenthe(mean)testresultobtainedfromanumberof
[Link]
methodbiasmaydependontheanalytelevel.
[Link]
Thedifferencebetweenthe(mean)testresultfromaparticularlaboratory
andtheacceptedreferencevalue.
[Link]
Thedifferencebetweenthemeanofreplicatetestresultsofasampleand
the("true")valueofthetargetpopulationfromwhichthesamplewas
[Link],foralaboratorythisrefersmainlytosamplepreparation,
[Link]
representativeforthepopulationinthefieldisanextremelyimportant
aspectbutusuallyfallsoutsidetheresponsibilityofthelaboratory(insome
caseslaboratorieshavetheirownfieldsamplingpersonnel).
Therelationshipbetweentheseconceptscanbeexpressedinthefollowingequation:
Figure

ThetypesoferrorsareillustratedinFig.61.
[Link].(Notethatthe
qualificationsapplytothemeanofresults:incthemeanisaccuratebutsome
individualresultsareinaccurate)

[Link]

3/29

30/05/2016

6BASICSTATISTICALTOOLS

6.3BasicStatistics
6.3.1Mean
6.3.2Standarddeviation
[Link]
6.3.4Confidencelimitsofameasurement
6.3.5Propagationoferrors
InthediscussionsofChapters7and8basicstatisticaltreatmentofdatawillbe
[Link],someunderstandingofthesestatisticsisessentialandtheywill
brieflybediscussedhere.
Thebasicassumptiontobemadeisthatasetofdata,obtainedbyrepeatedanalysisof
thesameanalyteinthesamesampleunderthesameconditions,hasanormalor
Gaussiandistribution.(Whenthedistributionisskewedstatisticaltreatmentismore
complicated).Theprimaryparametersusedarethemean(oraverage)andthe
standarddeviation(seeFig.62)andthemaintoolstheFtest,thettest,and
regressionandcorrelationanalysis.
[Link](approx.)
68%ofthedatafallintherangexs,95%intherangex2s,and99.7%in
therangex3s.
[Link]

4/29

30/05/2016

6BASICSTATISTICALTOOLS

6.3.1Mean
Theaverageofasetofndataxi:
(6.1)

6.3.2Standarddeviation
Thisisthemostcommonlyusedmeasureofthespreadordispersionofdataaround
[Link](V).The
varianceisdefinedasthesumofthesquareddeviationsfromthemean,dividedbyn
[Link],thereareseveralwaysofcalculation:
(6.1)

or
(6.3)

or
(6.4)

Thecalculationofthemeanandthestandarddeviationcaneasilybedoneona
calculatorbutmostconvenientlyonaPCwithcomputerprogramssuchasdBASE,
Lotus123,QuattroPro,Excel,andothers,whichhavesimplereadytousefunctions.
(Warning:someprogramsusenratherthann1!).

[Link]
Althoughthestandarddeviationofanalyticaldatamaynotvarymuchoverlimited
rangesofsuchdata,itusuallydependsonthemagnitudeofsuchdata:thelargerthe
figures,[Link],forcomparisonofvariations([Link])itisoften
moreconvenienttousetherelativestandarddeviation(RSD)thanthestandard
[Link],butmoreusuallyasapercentage
andisthencalledcoefficientofvariation(CV).Often,however,thesetermsare
confused.
(6.56.6)

[Link]([Link],seeEq.6.11)thevariancecan,of
course,becalculatedbysquaringthestandarddeviation:
V=s2 (6.7)
[Link]

5/29

30/05/2016

6BASICSTATISTICALTOOLS

6.3.4Confidencelimitsofameasurement
Themoreananalysisormeasurementisreplicated,thecloserthemeanxofthe
resultswillapproachthe"true"value,oftheanalytecontent(assumingabsenceof
bias).
Asingleanalysisofatestsamplecanberegardedasliterallysamplingtheimaginary
[Link]
subsamplingisexpressedby
(6.8)

where
="true"value(meanoflargesetofreplicates)
x=meanofsubsamples
t=astatisticalvaluewhichdependsonthenumberofdataandthe
requiredconfidence(usually95%).
s=standarddeviationofmeanofsubsamples
n=numberofsubsamples
(Theterm

isalsoknownasthestandarderrorofthemean.)

ThecriticalvaluesfortaretabulatedinAppendix1(theyare,therefore,herereferred
toasttab).Tofindtheapplicablevalue,thenumberofdegreesoffreedomhastobe
establishedby:df=n1(seealsoSection6.4.2).
Example
Forthedeterminationoftheclaycontentintheparticlesizeanalysis,asemiautomatic
[Link]
[Link],thepipettehastobe
calibrated,[Link](trueness)andprecisionhavetobeestablished.
Atenfoldmeasurementofthevolumeyieldedthefollowingsetofdata(inmL):
19.941 19.812 19.829 19.828 19.742
19.797 19.937 19.847 19.885 19.804

[Link]
1forn=10isttab=2.26(df=9)andusingEq.(6.8)thiscalibrationyields:
pipettevolume=19.8422.26(0.0627/

)=19.840.04mL

(Notethatthepipettehasasystematicdeviationfrom20mLasthisisoutsidethe
[Link]).
Inroutineanalyticalwork,resultsareusuallysinglevaluesobtainedinbatchesof
severaltestsamples.Nolaboratorywillanalyzeatestsample50timestobeconfident
[Link],thestatisticalparametershavetobeobtainedin
[Link](seeChapter7)and/orby
keepingcontrolcharts,whichisbasicallythecollectionofanalyticalresultsfromoneor
[Link]

6/29

30/05/2016

6BASICSTATISTICALTOOLS

morecontrolsamplesineachbatch(seeChapter8).Equation(6.8)isthenreducedto
(6.9)

where
="true"value
x=singlemeasurement
t=applicablettab(Appendix1)
s=standarddeviationofsetofpreviousmeasurements.
InAppendix1canbeseenthatifthesetofreplicatedmeasurementsislarge(say>
30),[Link],the(95%)confidenceoftheresultxofasingletest
sample(n=1inEq.6.8)isapproximatedbythecommonlyusedandwellknown
expression
(6.10)

whereSisthepreviouslydeterminedstandarddeviationofthelargesetofreplicates
(seealsoFig.62).
Note:This"methods"orsofacontrolsampleisnotaconstantandmay
varyfordifferenttestmaterials,analytelevels,andwithanalytical
conditions.
Runningduplicateswill,accordingtoEquation(6.8),increasetheconfidenceofthe
(mean)resultbyafactor

where
x=meanofduplicates
s=knownstandarddeviationoflargeset
Similarly,triplicateanalysiswillincreasetheconfidencebyafactor
arefurtherdiscussedinSection8.3.3.

,[Link]

Thus,insummary,Equation(6.8)canbeappliedinvariouswaystodeterminethesize
oferrors(confidence)inanalyticalworkormeasurements:singledeterminationsin
routinework,determinationsforwhichnopreviousdataexist,certaincalibrations,etc.

6.3.5Propagationoferrors
[Link].Propagationofrandomerrors
6.3.5.2Propagationofsystematicerrors
Thefinalresultofananalysisisoftencalculatedfromseveralmeasurements
performedduringtheprocedure(weighing,calibration,dilution,titration,instrument
readings,moisturecorrection,etc.).AswasindicatedinSection6.2,thetotalerrorin
[Link]
[Link]

7/29

30/05/2016

6BASICSTATISTICALTOOLS

dailypractice,thebiasandprecisionofthewholemethodareusuallythemostrelevant
parameters(obtainedfromvalidation,Chapter7orfromcontrolcharts,Chapter8).
However,sometimesitisusefultogetaninsightinthecontributionsofthe
subprocedures(andthenthesehavetobedeterminedseparately).Forinstanceifone
wantstochange(partof)themethod.
Becausethe"addingup"oferrorsisusuallynotasimplesummation,thiswillbe
[Link](precision)and
systematicerrors(bias).
[Link].Propagationofrandomerrors
Inestimatingthetotalrandomerrorfromfactorsinafinalcalculation,thetreatmentof
summationorsubtractionoffactorsisdifferentfromthatofmultiplicationordivision.
[Link]
Ifthefinalresultxisobtainedfromthesum(ordifference)of(sub)measurementsa,b,
c,etc.:
x=a+b+c+...
thenthetotalprecisionisexpressedbythestandarddeviationobtainedbytakingthe
squarerootofthesumofindividualvariances(squaresofstandarddeviation):

Ifa(sub)measurementhasaconstantmultiplicationfactororcoefficient(suchasan
extradilution),thenthisisincludedtocalculatetheeffectofthevarianceconcerned,
e.g.(2b)2
Example
TheEffectiveCationExchangeCapacityofsoils(ECEC)isobtainedbysummationof
theexchangeablecations:
ECEC=Exch.(Ca+Mg+Na+K+H+Al)
StandarddeviationsexperimentallyobtainedforexchangeableCa,Mg,Na,Kand(H+
Al)onacertainsample,[Link],are:0.30,0.25,0.15,0.15,and0.60
cmolc/[Link]:

Itcanbeseenthatthetotalstandarddeviationislargerthanthehighestindividual
standarddeviation,but(much)[Link]
reducethetotalstandarddeviation,qualitativelythebestresultcanbeexpectedfrom
reducingthelargestindividualcontribution,inthiscasetheexchangeableacidity.
[Link]
Ifthefinalresultxisobtainedfrommultiplication(orsubtraction)of(sub)measurements
accordingto

[Link]

8/29

30/05/2016

6BASICSTATISTICALTOOLS

thenthetotalerrorisexpressedbythestandarddeviationobtainedbytakingthe
squarerootofthesumoftheindividualrelativestandarddeviations(RSDorCV,asa
fractionoraspercentage,seeEqs.6.6and6.7):

Ifa(sub)measurementhasaconstantmultiplicationfactororcoefficient,thenthisis
includedtocalculatetheeffectoftheRSDconcerned,e.g.(2RSDb)2.
Example
ThecalculationofKjeldahlnitrogenmaybeasfollows:

where
a=mlHClrequiredfortitrationsample
b=mlHClrequiredfortitrationblank
s=airdrysampleweightingram
M=molarityofHCl
1.4=14103100%(14=atomicweightofN)
mcf=moisturecorrectionfactor
Notethatinadditiontomultiplications,thiscalculationcontainsasubtractionalso
(often,calculationscontainbothsummationsandmultiplications.)
Firstly,thestandarddeviationofthetitration(ab)isdeterminedasindicatedinSection
[Link](6.5)or(6.6).Thenthe
[Link]
foundRSDsare,forinstance:
distillation:0.8%,
titration:0.5%,
molarity:0.2%,
sampleweight:0.2%,
mcf:0.2%.
Thetotalcalculatedprecisionis:

Hereagain,thehighestRSD(ofdistillation)[Link],
theprecisionoftheKjeldahlmethodisusuallyconsiderablyworse(2.5%)probably
[Link]
takethatintoaccount.Itwouldimplythat2.5%1.0%=1.5%or3/5ofthetotal
randomerrorisduetosampleheterogeneity(orotheroverlookedcause).Thisimplies
thatpainstakingeffortstoimprovesubproceduressuchasthetitrationorthe
[Link],however,payto
improvethehomogeneityofthesample,[Link]
preparatorystage.
[Link]
[Link],theinfluenceofthisfactoronthefinalresultisusually
[Link]

9/29

30/05/2016

6BASICSTATISTICALTOOLS

verysmall.
6.3.5.2Propagationofsystematicerrors
Systematicerrorsof(sub)measurementscontributedirectlytothetotalbiasofthe
resultsincetheindividualparametersinthecalculationofthefinalresulteachcarry
[Link],thesystematicerrorinabalancewillcauseasystematic
errorinthesampleweight(aswellasinthemoisturedetermination).Notethatsome
systematicerrorsmaycancelout,[Link]
abiasedbalance.
Theonlywaytodetectoravoidsystematicerrorsisbycomparison(calibration)with
independentstandardsandoutsidereferenceorcontrolsamples.

6.4Statisticaltests
[Link]
6.4.2Ftestforprecision
6.4.3tTestsforbias
6.4.4Linearcorrelationandregression
6.4.5Analysisofvariance(ANOVA)
Inanalyticalworkafrequentlyrecurringoperationistheverificationofperformanceby
[Link]:
performanceoftwoinstruments,
performanceoftwomethods,
performanceofaprocedureindifferentperiods,
performanceoftwoanalystsorlaboratories,
resultsobtainedforareferenceorcontrolsamplewiththe"true","target"
or"assigned"valueofthissample.
Someofthemostcommonandconvenientstatisticaltoolstoquantifysuch
comparisonsaretheFtest,thettests,andregressionanalysis.
BecausetheFtestandthettestsarethemostbasicteststheywillbediscussedfirst.
Thesetestsexamineiftwosetsofnormallydistributeddataaresimilarordissimilar
(belongornotbelongtothesame"population")bycomparingtheirstandarddeviations
[Link].63.
[Link](n1=n2).A.
Differentmean(bias),[Link](nobias),different
[Link].(Thefourthcase,identical
sets,hasnotbeendrawn).

[Link]

10/29

30/05/2016

6BASICSTATISTICALTOOLS

[Link]

11/29

30/05/2016

6BASICSTATISTICALTOOLS

[Link]
Thesetestsforcomparison,forinstancebetweenmethodsAandB,arebasedonthe
assumptionthatthereisnosignificantdifference(the"nullhypothesis").Inotherwords,
whenthedifferenceissosmallthatatabulatedcriticalvalueofFortisnotexceeded,
wecanbeconfident(usuallyat95%level)[Link]
fundamentallydifferentquestionscanbeaskedconcerningboththecomparisonofthe
standarddeviationss1ands2withtheFtest,andofthemeansx1,andx2,withthet
test:
[Link]?(twosidedtest)
[Link](orlower)thanB?(onesidedtest).
Thisdistinctionhasanimportantpracticalimplicationasstatisticallytheprobabilitiesfor
thetwosituationsaredifferent:thechancethatAandBareonlydifferent("itcango
twoways")istwiceaslargeasthechancethatAishigher(orlower)thanB("itcango
onlyoneway").Themostcommoncaseisthetwosided(alsocalledtwotailed)test:
therearenoparticularreasonstoexpectthatthemeansorthestandarddeviationsof
[Link]
withthepreviousone(see8.3).However,whenitisexpectedorsuspectedthatthe
meanand/orthestandarddeviationwillgoonlyoneway,[Link]
analyticalprocedure,theonesided(oronetailed)[Link]
probabilitythatitgoestheotherwaythanexpectedisassumedtobezeroand,
therefore,[Link],morecorrectly,
theuncertaintyinthetwowaytestof5%(ortheprobabilityof5%thatthecriticalvalue
isexceeded)isdividedoverthetwotailsoftheGaussiancurve(seeFig.62),i.e.2.5%
attheendofeachtailbeyond2s.Ifweperformtheonesidedtestwith5%uncertainty,
weactuallyincreasethis2.5%to5%attheendofonetail.(Notethatforthewhole
gaussiancurve,whichissymmetrical,thisisthenequivalenttoanuncertaintyof10%in
twoways!)
Thisdifferenceinprobabilityinthetestsisexpressedintheuseoftwotablesofcritical
[Link],theonesidedtableat95%confidencelevelis
equivalenttothetwosidedtableat90%confidencelevel.
Itisemphasizedthattheonesidedtestisonlyappropriatewhenadifferenceinone
[Link]
resultsshowaclear(unexpected)[Link],however,thenatwotimeshigher
[Link]
thiswayevencontradictoryconclusionsmayarise:ifinanexperimentcalculated
valuesofFandtarefoundwithintherangebetweenthetwosidedandonesided
valuesofFtab,andttab,thetwosidedtestindicatesnosignificantdifference,whereas
theonesidedtestsaysthattheresultofAissignificantlyhigher(orlower)thanthatof
B.Whatactuallyhappensisthatinthefirstcasethe2.5%boundaryinthetailwasjust
notexceeded,andthen,subsequently,this2.5%boundaryisrelaxedto5%whichis
[Link]
strictnessandthatforproperinterpretationofresultsinreports,thestatistical
techniquesused,includingtheconfidencelimitsorprobability,shouldalwaysbe
specified.
[Link]

12/29

30/05/2016

6BASICSTATISTICALTOOLS

6.4.2Ftestforprecision
BecausetheresultoftheFtestmaybeneededtochoosebetweentheStudent'sttest
andtheCochranvariant(seenextsection),theFtestisdiscussedfirst.
TheFtest(orFisher'stest)isacomparisonofthespreadoftwosetsofdatatotestif
thesetsbelongtothesamepopulation,inotherwordsiftheprecisionsaresimilaror
dissimilar.
Thetestmakesuseoftheratioofthetwovariances:
(6.11)

[Link]
verydifferent,thentheestimatess1,ands2,donotdiffermuchandtheirratio(andthat
oftheirsquares)[Link],thecalculatedFis
comparedwiththeapplicableFvalueintheFtable(alsocalledthecriticalvalue,see
Appendix2).Toreadthetableitisnecessarytoknowtheapplicablenumberof
degreesoffreedomfors1,[Link]:
df1=n11
df2=n21
IfFcalFtabonecanconcludewith95%confidencethatthereisnosignificant
differenceinprecision(the"nullhypothesis"thats1,=s,isaccepted).Thus,thereis
stilla5%[Link]
maybeneeded,thena99%confidencetablecanbeused,whichcanbefoundin
statisticaltextbooks.
ExampleI(twosidedtest)
Table61givesthedatasetsobtainedbytwoanalystsforthecationexchangecapacity
(CEC)[Link](6.11)[Link]
hadnoparticularreasontoexpectthattheanalystswouldperformdifferently,weuse
theFtableforthetwosidedtestandfindFtab=4.03(Appendix2,df1,=df2=9).This
exceedsthecalculatedvalueandthenullhypothesis(nodifference)[Link]
beconcludedwith95%confidencethatthereisnosignificantdifferenceinprecision
betweentheworkofAnalyst1and2.
[Link](incmolc/kg)ofacontrolsampledeterminedbytwoanalysts.
1

10.2 9.7
10.7 9.0
10.5 10.2
9.9 10.3
9.0 10.8
11.2 11.1
11.5 9.4
10.9 9.2
8.9 9.8

[Link]

13/29

30/05/2016

6BASICSTATISTICALTOOLS

8.9 9.8
10.6 10.2
x:
s:
n:
Fcal=1.62

10.34
9.97
0.819
0.644
10
10
tcal=1.12

Ftab=4.03 ttab=2.10

Example2(onesidedtest)
ThedeterminationofthecalciumcarbonatecontentwiththeScheiblerstandard
methodiscomparedwiththesimpleandmorerapid"acidneutralization"methodusing
[Link]
therapidmethodwesuspectittoproducealowerprecisionthenobtainedwiththe
Scheiblermethodandwecan,therefore,[Link]
Ftab=3.07(App.2,df1,=12,df2=9)whichislowerthanFcal(=18.3)andthenull
hypothesis(nodifference)[Link](with95%confidence)that
forthisonesampletheprecisionoftherapidtitrationmethodissignificantlyworsethan
thatoftheScheiblermethod.
Table62.ContentsofCaCO3(inmass/mass%)inasoilsampledeterminedwiththe
Scheiblermethod(A)andtherapidtitrationmethod(B).
A

2.5 1.7
2.4 1.9
2.5 2.3
2.6 2.3
2.5 2.8
2.5 2.5
2.4 1.6
2.6 1.9
2.7 2.6
2.4 1.7
2.4
2.2
2.6
x:
s:
n:
Fcal=18.3

2.51
2.13
0.099
0.424
10
13
tcal=3.12

Ftab=3.07 ttab*=2.18

(ttab*=Cochran's"alternative"ttab)

6.4.3tTestsforbias
[Link]

14/29

30/05/2016

6BASICSTATISTICALTOOLS

[Link].Student'sttest
6.4.3.2Cochran'sttest
6.4.3.3tTestforlargedatasets(n30)
6.4.3.4Pairedttest
Dependingonthenatureoftwosetsofdata(n,s,samplingnature),themeansofthe
[Link]
commontypeswillbediscussed:
[Link]'sttestforcomparisonoftwoindependentsetsofdatawith
verysimilarstandarddeviations
[Link]
independentsetsdiffersignificantly
[Link].
Basically,forthettestsEquation(6.8)isusedbutwritteninadifferentway:
(6.12)

where
x=meanoftestresultsofasample
="true"orreferencevalue
s=standarddeviationoftestresults
n=numberoftestresultsofthesample.
Tocomparethemeanofadatasetwithareferencevaluenormallythe"twosidedt
tableofcriticalvalues"isused(Appendix1).Theapplicablenumberofdegreesof
freedomhereis:
df=n1
IfavaluefortcalculatedwithEquation(6.12)doesnotexceedthecriticalvalueinthe
table,thedataaretakentobelongtothesamepopulation:thereisnodifferenceand
the"nullhypothesis"isaccepted(withtheapplicableprobability,usually95%).
AswiththeFtest,whenitisexpectedorsuspectedthattheobtainedresultsarehigher
orlowerthanthatofthereferencevalue,theonesidedttestcanbeperformed:iftcal
>ttab,thentheresultsaresignificantlyhigher(orlower)thanthereferencevalue.
Morecommonly,however,the"true"valueofproperreferencesamplesis
accompaniedbytheassociatedstandarddeviationandnumberofreplicatesusedto
[Link]
themeansoftwodatasets:the"true"valueinEquation(6.12)isthenreplacedbythe
[Link].63,totestiftwodatasetsbelongto
[Link]
words,[Link].
Similarityornonsimilarityofstandarddeviations
[Link]

15/29

30/05/2016

6BASICSTATISTICALTOOLS

Whenusingthettestfortwosmallsetsofdata(n1and/orn2<30),achoiceofthetype
oftestmustbemadedependingonthesimilarity(ornonsimilarity)ofthestandard
[Link]
"pooled"[Link]
sufficientlysimilaranalternativeprocedureforthettestmustbefollowedinwhichthe
[Link]
[Link](see
6.4.2),thatis,[Link],forsmalldata
sets,theFtestshouldprecedethettest.
Fordealingwithlargedatasets(n1,n2,30)the"normal"ttestisused(seeSection
6.4.3.3andApp.3).
[Link].Student'sttest
(Tobeappliedtosmalldatasets(n1,n2<30)wheres1,ands2aresimilaraccording
toFtest.
Whencomparingtwosetsofdata,Equation(6.12)isrewrittenas:
(6.13)

where
x1=meanofdataset1
x2=meanofdataset2
sp="pooled"standarddeviationofthesets
n1=numberofdatainset1
n2=numberofdatainset2.
Thepooledstandarddeviationspiscalculatedby:
6.14

where
s1=standarddeviationofdataset1
s2=standarddeviationofdataset2
n1=numberofdatainset1
n2=numberofdatainset2.
Toperformthettest,thecriticalttabhastobefoundinthetable(Appendix1)the
applicablenumberofdegreesoffreedomdfisherecalculatedby:
df=n1+n22
Example
ThetwodatasetsofTable61canbeused:WithEquations(6.13)and(6.14)tcal,is
[Link]

16/29

30/05/2016

6BASICSTATISTICALTOOLS

calculatedas1.12whichislowerthanthecriticalvaluettabof2.10(App.1,df=18,
twosided),hencethenullhypothesis(nodifference)isacceptedandthetwodatasets
areassumedtobelongtothesamepopulation:thereisnosignificantdifference
betweenthemeanresultsofthetwoanalysts(with95%confidence).
[Link]
thedifferencebetweenthemeansfallswithinoroutsidetherangewhere
[Link],ifthisdifference
islessthantheleastsignificantdifference(lsd).Thiscanbederivedfrom
Equation(6.13):
6.15

InthepresentexampleofTable61,thecalculationyieldslsd=[Link]
differencebetweenthemeansis10.349.97=0.37whichissmallerthanthelsd
indicatingthatthereisnosignificantdifferencebetweentheperformanceofthe
analysts.
Inaddition,inthisapproachthe95%confidencelimitsofthedifferencebetweenthe
meanscanbecalculated(cf.Equation6.8):
confidencelimits=0.370.69=0.32and1.06
Notethatthevalue0forthedifferenceissituatedwithinthisconfidenceintervalwhich
agreeswiththenullhypothesisofx1=x2(nodifference)havingbeenaccepted.
6.4.3.2Cochran'sttest
Tobeappliedtosmalldatasets(n1,n2,<30)wheres1ands2,aredissimilar
accordingtoFtest.
Calculatetwith:
6.16

Thendeterminean"alternative"criticaltvalue:
6.17

where
t1=ttabatn11degreesoffreedom
t2=ttabatn21degreesoffreedom
Nowthettestcanbeperformedasusual:iftcal<ttab*thenthenullhypothesisthatthe
[Link]

17/29

30/05/2016

6BASICSTATISTICALTOOLS

meansdonotsignificantlydifferisaccepted.
Example
ThetwodatasetsofTable62canbeused.
AccordingtotheFtest,thestandarddeviationsdiffersignificantlysothattheCochran
[Link],incontrasttoourexpectationthattheprecisionof
therapidtestwouldbeinferior,wehavenoideaaboutthebiasandthereforethetwo
[Link]=3.12andttab*=2.18meaningthat
tcalexceedsttab*whichimpliesthatthenullhypothesis(nodifference)isrejectedand
thatthemeanoftherapidanalysisdeviatessignificantlyfromthatofthestandard
analysis(with95%confidence,andforthissampleonly).Furtherinvestigationofthe
rapidmethodwouldhavetoincludetheuseofmoredifferentsamplesandthen
comparisonwiththeonesidedttestwouldbejustified(see6.4.3.4,Example1).
6.4.3.3tTestforlargedatasets(n30)
Intheexampleabove([Link])theconclusionhappenstohavebeenthesameifthe
Student'[Link]
factthatthedifferenceinresultoftheStudentandCochranvariantsofthettestis
largestwhensmallsetsofdataarecompared,anddecreaseswithincreasingnumber
[Link],withincreasingnumberofdataabetterestimateofthereal
distributionofthepopulationisobtained(theflattertdistributionconvergesthentothe
standardizednormaldistribution).Whenn30forbothsets,[Link]
ControlCharts(see8.3),forallpracticalpurposesthedifferencebetweentheStudent
[Link]"normal"ttest
bysimplycalculatingtcalwithEq.(6.16)andcomparingthiswithttabatdf=n1+n22.
(NoteinApp.1thatthetwosidedttabisnowcloseto2).
Theproperchoiceofthettestasdiscussedaboveissummarizedinaflowdiagramin
Appendix3.
6.4.3.4Pairedttest
Whentwodatasetsarenotindependent,thepairedttestcanbeabettertoolfor
comparisonthanthe"normal"[Link]
instancethecasewhentwomethodsarecomparedbythesameanalystusingthe
samesample(s).Itcould,infact,alsobeappliedtotheexampleofTable61ifthetwo
analystsusedthesameanalyticalmethodat(about)thesametime.
Asstatedpreviously,comparisonoftwomethodsusingdifferentlevelsofanalytegives
[Link]
[Link]
pairedttest,however,allowsfordifferentlevelsprovidedtheconcentrationrangeis
[Link],therangeofresultsshouldbewithinthesame
[Link],[Link],regression
analysismustbeconsidered(seeSection6.4.4).Inintermediatecases,either
techniquemaybechosen.
Thenullhypothesisisthatthereisnodifferencebetweenthedatasets,sothetestisto
seeifthemeanofthedifferencesbetweenthedatadeviatessignificantlyfromzeroor
not(twosidedtest).Ifitisexpectedthatonesetissystematicallyhigher(orlower)than
theotherset,thentheonesidedtestisappropriate.
Example1
[Link]

18/29

30/05/2016

6BASICSTATISTICALTOOLS

The"promising"rapidsingleextractionmethodforthedeterminationofthecation
exchangecapacityofsoilsusingthesilverthioureacomplex(AgTU,bufferedatpH7)
wascomparedwiththetraditionalammoniumacetatemethod(NH4OAc,pH7).
Althoughforcertainsoiltypesthedifferenceinresultsappearedinsignificant,forother
[Link](oxic)
properties([Link]).InTable63theresultsoften
soilswiththesepropertiesaregroupedtotestiftheCECmethodsgivedifferent
[Link]
testaregivenalso.
[Link](incmolc/kg)obtainedbytheNH4OAcandAgTUmethods
(bothatpH7)fortensoilswithferralicproperties.
Sample NH4OAc AgTU

7.1

6.5

0.6

4.6

5.6

+1.0

10.6

14.5 +3.9

2.3

5.6

25.2

23.8 1.4

4.4

10.4 +6.0

7.8

8.4

+0.6

2.7

5.5

+2.8

14.3

19.2 +4.9

10

13.6

15.0 +1.4

+3.3

d=+2.19 tcal=2.89
sd=2.395 ttab=2.26

UsingEquation(6.12)andnotingthatd=0(hypothesisvalueofthedifferences,i.e.
nodifference),thetvaluecanbecalculatedas:

where
=meanofdifferenceswithineachpairofdata
sd=standarddeviationofthemeanofdifferences
n=numberofpairsofdata
Thecalculatedtvalue(=2.89)exceedsthecriticalvalueof1.83(App.1,df=n1=9,
onesided),hencethenullhypothesisthatthemethodsdonotdifferisrejectedanditis
concludedthatthesilverthioureamethodgivessignificantlyhigherresultsascompared
withtheammoniumacetatemethodwhenappliedtosuchhighlyweatheredsoils.
[Link],the"normal"
ttestwhichcomparesmeansofsetscannotbeusedhere(themeansdo
notconstituteafairrepresentationofthesets).Forthesamereasonno
informationabouttheprecisionofthetwomethodscanbeobtained,nor
[Link],replicate
[Link]

19/29

30/05/2016

6BASICSTATISTICALTOOLS

determinationsareneeded.
Example2
Table64showsthedataoftotalPinfourplanttissuesamplesobtainedbya
laboratoryLandthemedianvaluesobtainedby123laboratoriesinaproficiency
(roundrobin)test.
[Link](inmmol/kg)ofplanttissueasdeterminedby123
laboratories(Median)andLaboratoryL.
Sample Median LabL d
1

93.0

85.2 7.8

201

224

78.9

84.5 5.6

175

185

d=7.70

23
10

tcal=1.21

sd=12.702 ttab=3.18

Toverifytheperformanceofthelaboratoryapairedttestcanbeperformed:
UsingEq.(6.12)andnotingthatd=0(hypothesisvalueofthedifferences,[Link]
difference),thetvaluecanbecalculatedas:

Thecalculatedtvalueisbelowthecriticalvalueof3.18(Appendix1,df=n1=3,
twosided),hencethenullhypothesisthatthelaboratorydoesnotsignificantlydiffer
fromthegroupoflaboratoriesisaccepted,andtheresultsofLaboratoryLseemto
agreewiththoseof"therestoftheworld"(thisisasocalledthirdlinecontrol).

6.4.4Linearcorrelationandregression
6.4.4.1Constructionofcalibrationgraph
6.4.4.2Comparingtwosetsofdatausingmanysamplesatdifferent
analytelevels
Thesealsobelongtothemostcommonusefulstatisticaltoolstocompareeffectsand
[Link],there
isafundamentaldifferenceinconcept:correlationanalysisisappliedtoindependent
factors:ifXincreases,whatwillYdo(increase,decrease,orperhapsnotchangeat
all)?Inregressionanalysisaunilateralresponseisassumed:changesinXresultin
changesinY,butchangesinYdonotresultinchangesinX.
Forexample,inanalyticalwork,correlationanalysiscanbeusedforcomparing
methodsorlaboratories,whereasregressionanalysiscanbeusedtoconstruct
[Link],however,comparisonoflaboratoriesormethodsis
[Link]
(programmed)calculatorormoreconvenientlyonaPCusingahomemadeprogram.
[Link]

20/29

30/05/2016

6BASICSTATISTICALTOOLS

Evenmoreconvenientaretheregressionprogramsincludedinstatisticalpackages
suchasStatistix,Mathcad,Eureka,Genstat,Statcal,SPSS,[Link],most
spreadsheetprogramssuchasLotus123,Excel,andQuattroProhavefunctionsfor
this.
[Link],forregression
analysisonefactorhastobetheindependentor"constant"factor([Link]
method,orthefactorwiththesmalleststandarddeviation).Thisfactorisbyconvention
designatedX,whereastheotherfactoristhenthedependentfactorY(thus,wespeak
of"regressionofYonX").
AswasdiscussedinSection6.4.3,suchcomparisonscanoftenbeendonewiththe
Student/[Link],correlationanalysisisindicated:
[Link],bothrandom
andsystematic,arenotindependent(whichistheassumptionforthet
tests).Thisisoftenthecasewhereconcentrationrangesofseveral
magnitudesareinvolved.
[Link],notablyalongtime
spanbetweenthetwoanalyses(sampleaging,changeinlaboratory
conditions,etc.).
Theprincipleistoestablishastatisticallinearrelationshipbetweentwosetsof
correspondingdatabyfittingthedatatoastraightlinebymeansofthe"leastsquares"
[Link],forexample,analyticalresultsoftwomethodsappliedtothe
samesamples(correlation),ortheresponseofaninstrumenttoaseriesofstandard
solutions(regression).
Note:Naturally,nonlinearhigherorderrelationshipsarealsopossible,but
sincethesearelesscommoninanalyticalworkandmorecomplexto
handlemathematically,[Link],to
avoidmisinterpretation,alwaysinspectthekindofrelationshipbyplotting
thedata,eitheronpaperoronthecomputermonitor.
Theresultinglinetakesthegeneralform:
y=bx+a (6.18)

where
a=interceptofthelinewiththeyaxis
b=slope(tangent)
Inlaboratoryworkideally,whenthereisperfectpositivecorrelationwithoutbias,the
intercepta=0andtheslope=[Link]"1:1line"passingthroughthe
origin(dashedlineinFig.65).
Iftheintercepta0thenthereisasystematicdiscrepancy(bias,error)betweenXand
Ywhenb1thenthereisaproportionalresponseordifferencebetweenXandY.
ThecorrelationbetweenXandYisexpressedbythecorrelationcoefficientrwhichcan
becalculatedwiththefollowingequation:
6.19

[Link]

21/29

30/05/2016

6BASICSTATISTICALTOOLS

where
xi=dataX
x=meanofdataX
yi=dataY
y=meanofdataY
Itcanbeshownthatrcanvaryfrom1to1:
r=1perfectpositivelinearcorrelation
r=0nolinearcorrelation(maybeothercorrelation)
r=1perfectnegativelinearcorrelation
Often,thecorrelationcoefficientrisexpressedasr2:thecoefficientofdeterminationor
coefficientofvariance.Theadvantageofr2isthat,whenmultipliedby100,itindicates
[Link],forexample,when
r=0.71about50%(r2=0.504)ofthevariationinYisduetothevariationinX.
Thelineparametersbandaarecalculatedwiththefollowingequations:
6.20

and
a=ybx 6.21

Itisworthtonotethatrisindependentofthechoicewhichfactoristheindependent
[Link],theregressionparametersaanddo
dependonthischoiceastheregressionlineswillbedifferent(exceptwhenthereis
ideal1:1correlation).
6.4.4.1Constructionofcalibrationgraph
Asanexample,wetakeastandardseriesofP(01.0mg/L)forthespectrophotometric
determinationofphosphateinaBrayIextract("availableP"),readinginabsorbance
[Link]
[Link].64.
[Link]
calculationofthecorrelationcoefficientrwithEquation(6.19)yieldsavalueof0.997(r2
=0.995).[Link]
closeto1(say,below0.98)thismustbetakenasawarninganditmightthenbe
[Link]([Link]
pipetting)[Link],ahighr
[Link],toverifythis,
thecalibrationgraphshouldalwaysbeplotted,eitheronpaperoroncomputermonitor.
UsingEquations(6.20and(6.21)weobtain:

[Link]

22/29

30/05/2016

6BASICSTATISTICALTOOLS

and
a=0.3500.313=0.037
Thus,theequationofthecalibrationlineis:
y=0.626x+0.037 (6.22)

[Link].64.
xi

yi

x1x (xix)2 yiy (yiy)2 (x1x)(yiy)

0.0

0.05

0.5

0.25

0.30 0.090

0.150

0.2

0.14

0.3

0.09

0.21 0.044

0.063

0.4

0.29

0.1

0.01

0.06 0.004

0.006

0.6

0.43

0.1

0.01

0.08 0.006

0.008

0.8

0.52

0.3

0.09

0.17 0.029

0.051

1.0

0.67

0.5

0.25

0.32 0.102

0.160

3.0

2.10

0.70

0.2754

0.438

x=0.5 y=0.35

[Link]
delineatethe95%[Link]
highestatthecentroidofthegraph.

[Link]

23/29

30/05/2016

6BASICSTATISTICALTOOLS

Duringcalculation,themaximumnumberofdecimalsisused,roundingofftothelast
significantfigureisdoneattheend(seeinstructionforroundingoffinSection8.2).
Oncethecalibrationgraphisestablished,itsuseissimple:foreachyvaluemeasured
thecorrespondingconcentrationxcanbedeterminedeitherbydirectreadingorby
calculationusingEquation(6.22).Theuseofcalibrationgraphsisfurtherdiscussedin
Section7.2.2.
[Link].
6.4.4.2Comparingtwosetsofdatausingmanysamplesatdifferentanalyte
levels
Althoughregressionanalysisassumesthatonefactor(onthexaxis)isconstant,when
certainconditionsaremetthetechniquecanalsosuccessfullybeappliedtocomparing
[Link]:
Themostprecisedatasetisplottedonthexaxis
Atleast6,butpreferablymorethan10differentsamplesareanalyzed
Thesamplesshouldratheruniformlycovertheanalytelevelrangeof
interest.
Todecidewhichlaboratoryormethodisthemostprecise,multireplicateresultshave
tobeusedtocalculatestandarddeviations(see6.4.2).Ifthesearenotavailablethen
thestandarddeviationsofthepresentsetscouldbecompared(notethatwearenow
notdealingwithnormallydistributedsetsofreplicateresults).Anotherconvenientway
[Link]

24/29

30/05/2016

6BASICSTATISTICALTOOLS

istoruntheregressionanalysisonthecomputer,reversethevariablesandrunthe
[Link](orstandard
erroroftheintercepta,bothgivenbythecomputer)andthenusetheresultsofthe
regressionanalysiswherethisvariablewasplottedonthexaxis.
Iftheanalytelevelrangeisincomplete,onemighthavetoresorttospikingorstandard
additions,withtheinherentdrawbackthattheoriginalanalytesamplecombinationmay
notadequatelybereflected.
Example
Intheframeworkofaperformanceverificationprogramme,alargenumberofsoil
sampleswereanalyzedbytwolaboratoriesXandY(aformof"thirdlinecontrol",see
Chapter9)andthedatacomparedbyregression.(Inthisparticularcase,thepairedt
testmighthavebeenconsideredalso).Theregressionlineofacommonattribute,the
pH,isshownhereasanillustration.Figure65showsthesocalled"scatterplot"of
[Link]
[Link](=1.03)indicatesthattheregressionlineis
onlyslightlysteeperthanthe1:[Link],however,isthe
[Link]
unitlowerthanlaboratoryXatthelowendofthepHrange(theintercept1.18isat
pHx=0)whichdifferencedecreasestoabout0.8unitatthehighend.
[Link]:regressionline
dashedline:1:1idealregressionline.

[Link]

25/29

30/05/2016

6BASICSTATISTICALTOOLS

Thettestforsignificanceisasfollows:
Forintercepta:a=0(nullhypothesis:nobiasidealinterceptisthenzero),standard
error=0.14(calculatedbythecomputer),andusingEquation(6.12)weobtain:

Here,ttab=1.98(App.1,twosided,df=n2=122(n2becauseanextradegreeof
freedomislostasthedataareusedforbothaandb)hence,thelaboratorieshavea
significantmutualbias.
Forslope:b=1(idealslope:nullhypothesisisnodifference),standarderror=0.02
(givenbycomputer),andagainusingEquation(6.12)weobtain:

Again,ttab=1.98(App.1twosided,df=122),hence,thedifferencebetweenthe
laboratoriesisnotsignificantlyproportional(or:thelaboratoriesdonothavea
significantdifferenceinsensitivity).Theseresultssuggestthatinspiteofthegood
correlation,thetwolaboratorieswouldhavetolookintothecauseofthebias.
[Link]

26/29

30/05/2016

6BASICSTATISTICALTOOLS

[Link],thescatteringofthepointsaroundthe
[Link]
indicatesthattheprecisionoflaboratoryYdoesnotchangeverymuch
[Link]
suchcases,weightedregression(notdiscussedhere)ismoreappropriate
thantheunweightedregressionasusedhere.
Validationofamethod(seeSection7.5)mayrevealthatprecisioncan
changesignificantlywiththelevelofanalyte(andwithotherfactorssuch
assamplematrix).

6.4.5Analysisofvariance(ANOVA)
Whenresultsoflaboratoriesormethodsarecomparedwheremorethanonefactor
canbeofinfluenceandmustbedistinguishedfromrandomeffects,thenANOVAisa
[Link]:differentanalysts,
sampleswithdifferentpretreatments,differentanalytelevels,differentmethodswithin
oneofthelaboratories).MoststatisticalpackagesforthePCcanperformthisanalysis.
AsatreatiseofANOVAisbeyondthescopeofthepresentGuidelines,forfurther
discussionthereaderisreferredtostatisticaltextbooks,someofwhicharegiveninthe
listofLiterature.
Errororuncertaintyintheregressionline
The"fitting"ofthecalibrationgraphisnecessarybecausetheresponsepointsyi,
[Link],randomerrorsareimplied.
Thisisexpressedbyanuncertaintyabouttheslopeandinterceptbandadefiningthe
[Link]
[Link]
illustratetheprocedure,theexampleofthecalibrationgraphinSection6.4.3.1is
elaboratedhere.
Apracticalquantificationoftheuncertaintyisobtainedbycalculatingthestandard
deviationofthepointsonthelinethe"residualstandarddeviation"or"standarderror
oftheyestimate",whichweassumedtobeconstant(butwhichisonlyapproximately
so,seeFig.64):
(6.23)

where
="fitted"yvalueforeachxi,(readfromgraphorcalculatedwithEq.
6.22).Thus,
line.

isthe(vertical)deviationofthefoundyvaluesfromthe

n=numberofcalibrationpoints.
Note:[Link]
[Link],ofcourse,
onlythecaseifthestandardsareveryaccuratelyprepared.
Nowthestandarddeviationsfortheinterceptaandslopebcanbecalculatedwith:
[Link]

27/29

30/05/2016

6BASICSTATISTICALTOOLS

6.24

and
6.25

Tomakethisprocedureclear,theparametersinvolvedarelistedinTable66.
Theuncertaintyabouttheregressionlineisexpressedbytheconfidencelimitsofaand
baccordingtoEq.(6.9):[Link]
[Link](usealsofigures
ofTable65).
xi

yi

0 0.05 0.037 0.013

0.0002

0.2 0.14 0.162 0.022

0.0005

0.4 0.29 0.287 0.003

0.0000

0.6 0.43 0.413 0.017

0.0003

0.8 0.52 0.538 0.018

0.0003

1.0 0.67 0.663 0.007

0.0001
0.001364

Inthepresentexample,usingEq.(6.23),wecalculate

and,usingEq.(6.24)andTable65:

and,usingEq.(6.25)andTable65:

Theapplicablettabis2.78(App.1,twosided,df=n1=4)hence,usingEq.(6.9):
a=0.0372.780.0132=0.0370.037
and
b=0.6262.780.0219=0.6260.061
Notethatifsaislargeenough,anegativevalueforaispossible,[Link]
fortheblankorzerostandard.(Foradiscussionabouttheerrorinxresultingfroma
[Link]

28/29

30/05/2016

6BASICSTATISTICALTOOLS

readinginy,whichisparticularlyrelevantforreadingacalibrationgraph,seeSection
7.2.3)
Theuncertaintyaboutthelineissomewhatdecreasedbyusingmorecalibrationpoints
(assumingsyhasnotincreased):onemorepointreducesttabfrom2.78to2.57(see
Appendix1).

[Link]

29/29