1.

Probabilities for events

P (A B) = P (A) + P (B) P (A B)

For events A, B, and C

More generally

P(

Ai ) =

P (Ai Aj ) +

P (Ai Aj Ak )

The odds in favour of A

P (A) / P (A)

Conditional probability

P (A B) =

Chain rule

P (A B C) = P (A) P (B A) P (C A B)

Bayes rule

P (A B) =

A and B are independent if

P (B A) = P (B)

A, B, and C are independent if

P (A B C) = P (A)P (B)P (C) , and

P (A B) = P (A)P (B) ,

2.

P (Ai )

P (A B)
P (B)

provided that P (B) > 0

P (A) P (B A)
P (A) P (B A) + P (A) P (B A)

P (B C) = P (B)P (C) , P (C A) = P (C)P (A)

Probability distribution, expectation and variance

The probability distribution for a discrete random variable X is the complete set of

Expectation

E(X) = =

probabilities {px } = {P (X = x)}

xpx

1 X
xk
n k

Sample mean x =

estimates from random sample x1 , x2 , . . . , xn

Variance var (X) = 2 = E{(X )2 } = E(X 2 ) 2 , where E(X 2 ) =

Sample variance
Standard deviation

1
s2 =
n1

X

1
x2k
n

sd (X) =

If value y is observed with frequency ny

n=

X
y

For function g(x) of x,

ny ,

xk =

E{g(X)} =

yny ,

X

xj

2 

x2k =

estimates 2

y 2 ny

g(x)px

X 3



X 4
= E
3

Skewness

1 = E

Kurtosis



1 X xi x 3
n1
s


1 X xi x 4
is estimated by
3
n1
s

is estimated by

x2 px

Sample median xe . If the sample values x1 , . . . , xn are ordered x(1) x(2) x(n)
xe = x( n+1 ) if n is odd, and xe =
2

1
2

(x( n2 ) + x( n+2 ) ) if n is even.

2

-quantile Q() is such that P (X Q()) =

b
b
Sample -quantile Q()
is the sample value for which the proportion of values Q()
is
(using linear interpolation between values on either side)

The sample median xe estimates the population median Q(0.5).

3.

Probability distribution for a continuous random variable

The cumulative distribution function (cdf)

F (x) = P (X x) =

The probability density function (pdf)

E(X) = =

x f (x)dx ,

f (x) =

dF (x)
dx

x0 =

f (x0 )dx0

var (X) = 2 = E(X 2 ) 2 ,

2

where E(X ) =

4.

Z x

x2 f (x)dx

Discrete probability distributions

Discrete Uniform Uniform (n)
1
(x = 1, 2, . . . , n)
px =
n

= 12 (n + 1) , 2 =

1
12

(n2 1)

Binomial distribution Binomial (n, )

px =

n
x

x (1 )nx (x = 0, 1, 2, . . . , n)

= n , 2 = n(1 )

Poisson distribution Poisson ()

px =

x e
x!

(x = 0, 1, 2, . . .) (with > 0)

= , 2 =

Geometric distribution Geometric ()

px = (1 )x1 (x = 1, 2, 3, . . .)

5.

1
1
, 2 =

Continuous probability distributions

Uniform distribution Uniform (, )
f (x) =

( < x < ),
(otherwise).

= 12 ( + ) , 2 =

1
12

( )2

Exponential distribution Exponential ()

f (x) =

ex

(0 < x < ),

= 1/ ,

( < x 0).

2 = 1/2 .

Normal distribution N (, 2 )
f (x) =

1
exp
2 2

1 x
2

2 

( < x < )

E(X) = , var (X) = 2

Standard normal distribution N (0, 1)

If X is N (, 2 ), then Y =

6.

X
is N (0, 1)

Reliability
For a device in continuous operation with failure time random variable T having pdf
f (t) (t > 0)

7.

The reliability function at time t

R(t)

= P (T > t)

The failure rate or hazard function

h(t)

= f (t)/R(t)

The cumulative hazard

H(t)

The Weibull(, ) distribution has

H(t)

Z t
0

h(t0 ) dt0 =

ln{R(t)}

= t

System reliability
For a system of k devices, which operate independently, let
Ri = P (Di ) = P (device i operates)
The system reliability, R, is the probability of a path of operating devices
A system of devices in series operates only if every device operates
R = P (D1 D2 Dk ) = R1 R2 Rk
A system of devices in parallel operates if any device operates
R = P (D1 D2 Dk ) = 1 (1 R1 )(1 R2 ) (1 Rk )

8.

Covariance and correlation

The covariance of X and Y

cov (X, Y ) = E(XY ) {E(X)}{E(Y )}

From pairs of observations (x1 , y1 ), . . . , (xn , yn )

Sxy =

X
k

Sxx =

X
k

9.

x2k

1
(
n

xi )2 ,

Syy =

X
k

1
Sxy
n1

yk2

Sample covariance

sxy =

Correlation coefficient

= corr (X, Y ) =

Sample correlation coefficient

Sxy
Sxx Syy

xk yk

X
1 X
(
xi )(
yj )
n i
j

1 X 2
(
yj )
n j

estimates cov (X, Y )

cov (X, Y )
sd (X) sd (Y )

estimates

Sums of random variables

E(X + Y ) = E(X) + E(Y )
var (X + Y ) = var (X) + var (Y ) + 2 cov (X, Y )
cov (aX + bY, cX + dY ) = (ac) var (X) + (bd) var (Y ) + (ad + bc) cov (X, Y )
If X is N (1 , 12 ), Y is N (2 , 22 ), and cov (X, Y ) = c,
then X + Y is N (1 + 2 , 12 + 22 + 2c)

10.

Bias, standard error, mean square error

If t estimates (with random variable T giving t)
Bias of t

bias(t)

= E(T )

Standard error of t

se (t)

= sd (T )

= E{(T )2 } = {se (t)}2 + {bias(t)}2

If x estimates , then bias(x) = 0 , se (x) = / n , MSE(x) = 2 /n , sc

e (x) = s/ n
Mean square error of t

Central limit property

11.

MSE(t)

if n is fairly large, x is from N (, 2 /n) approximately

Likelihood
The likelihood is the joint probability as a function of the unknown parameter .
For a random sample x1 , x2 , . . . , xn
`(; x1 , x2 , . . . , xn ) = P (X1 = x1 ) P (Xn = xn ) (discrete distribution)
`(; x1 , x2 , . . . , xn ) = f (x1 )f (x2 ) f (xn ) (continuous distribution)

The maximum likelihood estimator (MLE) is b for which the likelihood is a maximum.

12.

Confidence intervals
If x1 , x2 , . . . , xn are a random sample from N (, 2 ) and 2 is known, then

the 95% confidence interval for is (x 1.96 , x + 1.96 )

n
n
If 2 is estimated, then from the Student t table for tn1 we find t0 = tn1,0.05
s
s
The 95% confidence interval for is (x t0 , x + t0 )
n
n

13.

14.

Standard normal table

Values of pdf (y) = f (y) and cdf (y) = F (y)

y (y) (y)

(y) (y)

(y)

.9 .266

.816 1.8 .079 .964

2.8

.997

.1 .397 .540

1.0 .242

.841 1.9 .066 .971

3.0

.998

.2 .391 .579

1.1 .218

.864 2.0 .054 .977

0.841 .8

.3 .381 .618

1.2 .194

.885 2.1 .044 .982

1.282 .9

.4 .368 .655

1.3 .171

.903 2.2 .035 .986

1.645 .95

.5 .352 .691

1.4 .150

.919 2.3 .028 .989

1.96

.6 .333 .726

1.5 .130

.933 2.4 .022 .992

2.326 .99

.7 .312 .758

1.6 .111

.945 2.5 .018 .994

2.576 .995

.8 .290 .788

1.7 .094

.955 2.6 .014 .995

3.09

.399 .5

Student t table
p .10

(y) (y)

.975

.999

Values tm,p of x for which P (|X| > x) = p , when X is tm

.05

.02

0.01

m 1 6.31 12.71 31.82 63.66 m

.10

.05

.02

0.01

1.83

2.26 2.82

3.25

2 2.92 4.30

6.96

9.92

10 1.81

2.23 2.76

3.17

3 2.35 3.18

4.54

5.84

12 1.78

2.18 2.68

3.05

4 2.13 2.78

3.75

4.60

15 1.75

2.13 2.60

2.95

5 2.02 2.57

3.36

4.03

20 1.72

2.09 2.53

2.85

6 1.94 2.45

3.14

3.71

25 1.71

2.06 2.48

2.78

7 1.89 2.36

3.00

3.50

40 1.68

2.02 2.42

2.70

8 1.86 2.31

2.90

3.36

1.645 1.96 2.326 2.576

15.

Chi-squared table
Values 2k,p of x for which P (X > x) = p , when X is 2k and p = .995, .975, etc

.995 .975

.05

.025

.01

.005

.995

.975

.05

.025

.01

.005

.000 .001

3.84

5.02

6.63

7.88

18

6.26

8.23

28.87 31.53 34.81 37.16

.010 .051

5.99

7.38

9.21

10.60

20

7.43

9.59

31.42 34.17 37.57 40.00

.072 .216

7.81

9.35

11.34 12.84

22

8.64

10.98 33.92 36.78 40.29 42.80

.207 .484

9.49

11.14 13.28 14.86

24

9.89

12.40 36.42 39.36 42.98 45.56

.412 .831 11.07 12.83 15.09 16.75

26

11.16 13.84 38.89 41.92 45.64 48.29

.676 1.24 12.59 14.45 16.81 18.55

28

12.46 15.31 41.34 44.46 48.28 50.99

.990 1.69 14.07 16.01 18.48 20.28

30

13.79 16.79 43.77 46.98 50.89 53.67

1.34 2.18 15.51 17.53 20.09 21.95

40

20.71 24.43 55.76 59.34 63.69 66.77

1.73 2.70 16.92 19.02 21.67 23.59

50

27.99 32.36 67.50 71.41 76.15 79.49

10 2.16 3.25 13.31 20.48 23.21 25.19

60

35.53 40.48 79.08 83.30 88.38 91.95

12 3.07 4.40 21.03 23.34 26.22 28.30

70

43.28 48.76 90.53 95.02 100.4 104.2

14 4.07 5.63 23.68 26.12 29.14 31.32

80

51.17 57.15 101.9 106.6 112.3 116.3

16 5.14 6.91 26.30 28.85 32.00 34.27 100 67.33 74.22 124.3 129.6 135.8 140.2

16.

The chi-squared goodness-of-fit test

b y for every group exceeds
The frequencies ny are grouped so that the fitted frequency n
about 5.
X (ny n
b y )2
X2 =
is referred to the table of 2k with significance point p,
b
ny
y
where k is the number of terms summed, less one for each constraint, eg matching total
frequency, and matching x with .

17.

Joint probability distributions

Discrete distribution {pxy }, where pxy = P ({X = x} {Y = y}) .

Let
px

px = P (X = x), and py = P (Y = y), then

X
pxy
=
pxy , and P (X = x Y = y) =
py
y

Continuous distribution
Z x

Z y

f (x, y) =

d2 F (x, y)
dx dy

fX (x) =

f (x, y0 ) dy0

F (x, y) = P ({X x} {Y y}) =

Joint cdf
Joint pdf

Marginal pdf of X
Conditional pdf of X given Y = y

18.

fX|Y (x|y) =

x0 =

y0 =

f (x, y)
fY (y)

f (x0 , y0 ) dx0 dy0

(provided fY (y) > 0)

Linear regression
b
b + x
from observations
To fit the linear regression model y = + x by ybx =
(x1 , y1 ), . . . , (xn , yn ) , the least squares fit is

The residual sum of squares

c2 =

b
var ()

ybx =

b
b + x

,
b
sc
e ()

19.

RSS

b = Sxy /Sxx

= Syy

2
Sxy
Sxx

n 2 c2
is from 2n2
2

RSS
,
n2

b = ,
E()

b = y xb ,

b = ,
E()

P 2
x
i

n Sxx

2 ,

b
var ()
=

E(ybx ) = + x ,

b
,
b
sc
e ()

ybx x
sc
e (ybx )

Design matrix for factorial experiments

X =

1
1
1
1
1
1
1
1

1
1
1
1
1
1
1
1

1
1
1
1
1
1
1
1

2
,
Sxx

b
b )
cov (,
=

var (ybx ) =

x 2

Sxx

1 (x x)2
+
n
Sxx

are each from tn2

With 3 factors each at 2 levels

1
1
1
1
1
1
1
1

1
1
1
1
1
1
1
1

1
1
1
1
1
1
1
1

1
1
1
1
1
1
1
1