KAN-CFIVO1001U Quantitative Methods Autumn 2021

TABLE OF CONTENTS
1 Basic Algebra 4

2 Functions of One Variable 5

2.1 Linear Function 5

2.2 Quadratic Function 5

2.3 Power Function, Exponential Function and Logarithmic Function 6

3 Continuity and Derivatives 7

3.1 Limits 7

3.2 Continuity and Differentiability 8

3.3 Rules for Differentiation 8

3.4 Increasing and Decreasing Functions 9

3.5 Concave and Convex Functions 9

3.6 Linear Approximations 9

3.7 Newton’s Method 10

3.8 Bisection Method 11

4 Single Variable Optimisation 12

4.1 Finding Extreme points 12

4.2 Local Extreme Points 13

4.3 Inflection Points 13

5 Integration 14
5.1 Introduction to Integration 14

5.2 Indefinite Integrals 15

5.3 Definite Integrals 16

5.4 Integration Methods 17

5.5 Interest Rates and Present Value 18

6 Functions of Multiple Variables 22

6.1 Partial Derivatives 22
6.1.1 The Chain Rule 23

6.2 Surfaces and Distance 23

6.3 Homogeneous and Homothetic Functions 24

6.4 Linear Approximations 24

6.5 Differentials 24

Page 1 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

6.6 Systems of Equations 25

7 Continuous Optimisation 26
7.1 Linear Regression as a Minimisation Problem 26

7.2 Multivariate Optimisation 26

7.2.1 Local Extreme Points 27

7.2.2 Extreme Value Theorem 28

7.3 Constrained Optimisation: Lagrange Multiplier 29

8 Linear Algebra 31
8.1 Vector and Matrix Algebra 31
8.1.1 Inverse of a Matrix 33

8.1.2 Positive and Negative Definite 33

8.1.3 Matrix Calculus 34

8.1.4 Vectors 34

8.1.5 Lines and Planes 35

8.2 Linear Equation Systems 35

8.2.1 Determinants 36

9 Probability 37
9.1 Sample Space and Algebra of Sets 37

9.2 The Probability Function 37

9.2.1 Discrete Random Variables 38

9.2.2 Continuous Random Variables 38

9.2.3 Expected Values and Variance 39

9.2.4 Higher Moments 40

9.3 Joint Distributions 40

9.3.1 Mean, Variance and Covariance 41

9.4 Important Distributions 41

9.5 Law of Large Numbers and Central Limit Theorem 45

10 Statistics 46
10.1 Estimation 46
10.1.1 Maximum Likelihood Estimation 46

10.1.2 Methods of Moments 47

10.2 Properties of Estimators 48

10.3 Hypothesis Testing 50

10.3.1 Normal, Known Variance 50

Page 2 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

10.3.2 Normal, Unknown Variance 51

10.3.3 Unknown distribution or non-normal data 51

10.3.4 Confidence Intervals 52

10.3.5 Type I and Type II Error 52

11 Linear Regression 53
11.1 Inference (Hypothesis Testing) 54

11.2 Multiple Linear Regression 55

12 Monte Carlo Methods 57

1 BASIC ALGEBRA
Carol (2008): Chapter 1.1-1.3 + Lecture 1
Essential Set Theory: A set, S, is a collection of distinct objects. The objects that make up a set
(elements or members) can be anything, e.g., numbers. The following notation applies

● x ∈ ( ∉ ) S : x is (not) an element of the set S

● A ⊂ B: A is a subset of B

● A ∪ B: The union is elements that belong to at least one of A and B ({x : x ∈ A∨x ∈ B })

● A ∩ B: The intersection is the elements that belong to both A and B ({x : x ∈ A∧x ∈ B })

● ∅ : The empty set

● If A ∩ B=∅ , then A and B are disjoing

c
● The complement of A in Ω is defined by A =Ω∖ A .

Integer Powers: a 0=1 for real a ≠ 0, as 0 0 is undefined. The following also holds true:

−n 1
a = n
r
a ∗a =a
s r +s s
( a r ) =a r∗s
a

A quantity K increasing by p % each period will have increased to K ( 1+ p /100 )t after t periods.

Quadratic Identities: Factoring an expression is simply to write it as a product of simpler factors

( a+ b )2=a2+ b2 +2 ab

( a−b )2=a2 +b 2−2 ab

2 2
a −b =(a+ b)(a−b)

Page 3 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

Properties of Fractions: Let a , b and c be any numbers, where b ≠ 0 and c ≠ 0 whenever they
appear in the numerator, then

a c ad +bc b ac +b
+ = a =
b d bd c c

a c ac a c a d ad
× = ÷ = =
b d bd b d b c bc

Fractional Powers: We have √ ❑. If a and b are nonnegative numbers, then √ ❑. If a ≥ 0 and b> 0,
then√ ❑. Every positive number a has two square roots ± √ ❑. √
n
a is the unique positive number that
n
raised to the n -th power gives a , that is ( √
n
a ) =a.

Rules for Summation: To following rules hold for additivity and homogeneity of summation:
n n n n n

∑ (a i+ bi)=∑ ai +∑ bi ∑ c a i=c ∑ ai
i=1 i=1 i=1 i=1 i=1

Page 4 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

2 FUNCTIONS OF ONE VARIABLE

Carol (2008): Chapter 1.1-1.3 + Lecture 1
Definition of Function: A function of a variable with domain D is a rule that assigns a unique
number to each x in D . The domain of a function is all possible x -values of the function, where the
range is all the possible y -values of the function. The particular element y=f (x ) is the image of x .
If each element of Y is the image of at most one element in X , the function is called one-to-one
(injective). A function has an inverse if it is one-to-one.

The inverse function of a function is not its reciprocal value at any x , but the reflection of the
function in the 45% line, i.e., f −1 ( x )=g ( x ) → f ( g ( x ) ) =x

2.1 Linear Function

A general linear function, f ( x )=ax+ b, is one whose graph is a straight line. The root of a function
is where it crosses the x-axis, i.e., where y=0. The slope of the straight line l is

y 2− y 1
a= ,x ≠x
x 2−x 1 1 2

The point-slope formula for a straight line passing through (x 1 , y 1 ) with slope a is

y− y 1=a(x− x1 )

2.2 Quadratic Function

A general quadratic function, f ( x )=a x2 +bx + c, has the following configuration

● If a> 0, it has a ∪ shape, while if a< 0 it has a ∩ shape. The magnitude of a determines the

steepness of the curve.

● b determines the horizontal location: If b> 0, the graph is shifted to the left of the y-axis at x=0 ,

otherwise it is shifted right. The magnitude of b is the extent of the shift.

● c determines the vertical location.

The roots of a quadratic equation ( y=0) is given as

−b ± √❑
x=
❑

The discriminant of the equation is d=b2−4 ac . If d <0 , there are no real roots, if d >0 there are
two real roots, and if d=0 there is one root. If x 1 and x 2 is the solution, then

2
a x +bx +c=a (x−x 1)(x−x 2 )

Page 5 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

Extrema of Quadratic Function: Can be found using:

If a> 0, then f ( x )=a x2 +bx + c has its minimum at x=−b /2 a

If a< 0, then f ( x )=a x2 +bx + c has its maximum at x=−b /2 a

2.3 Power Function, Exponential Function, and Logarithmic Function

The general polynomial of degree n with coefficients a n , a n−1 , … , a0 is given as

n n−1
P ( x )=a n x +an−1 x + …+a1 x+ a0

The general power function, where r and A are constants for x >0, is defined by the function
r
f ( x )= A x

The general exponential function with base a> 0 is

f ( x )= A a x

where a is the factor by which f ( x ) changes when x increases by 1. If a=1+ p/100, where p>0 and
A> 0, then f (x) will increase by p % for each unit increase.

An irrational number is a real number with a decimal expansion that continuous indefinitely. It
cannot be expressed as a fraction. e is irrational and transcendental. The natural exponential
function is

( )
n
x x
e =exp exp(x )= 1+
n

We have e 1=e and e 0=1. The graph of the exponential function has a slope equal to e x at any point
x . It also obeys the law of indicies, meaning:

exp exp ( x + y )=exp exp ( x ) exp exp ( y ) and exp exp ( x− y )=exp exp ( x ) /exp exp ( y )

The inverse of the exponential function is the natural logarithm function. It is only defined for
positive real numbers. We have ln ln ( 1 ) =0, ln ln (e)=1, ln ln ( e x )=x , and ln ln ( x ) <0 for 0< x <1.
An important property of the logarithmic function is that the log of a product is the sum of logs. Also:

ln ln (xy)=ln ln ( x ) +ln ln ( y ) p
ln ln ( x )= p ln ln ( x )

ln ln ( xy )=ln ln ( x )−ln ln ( y )

Page 6 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

3 DERIVATIVES AND CONTINUITY

Carol (2008): Chapter 1 + Lecture 1-2

Continuity: If the graph of a function has no breaks, then f (x) is a continuous function. A function
is continuous at x=a if f (x)=f ( a). Continuous functions are not always differentiable.

Differentiability: If the graph of a continuous function f (x) has no corners, then f (x) is a
differentiable function. A function is differentiable if the limit that defines the Newton quotient
exists. A differentiable function is always continuous.

3.1 Limits
When taking the limit, we let x tend towards a (but never equal to), making f (x) infinitely close to A

f (x)=A

For all real ε > 0, there exists a real δ >0 so for all x with 0<| x−a|< δ it holds |f ( x ) −A|< ε .

Rules for limits: Define f (x)=A and g(x )=B, then

● [ f ( x ) ± g ( x)]= A ± B

● [ f ( x )∗g ( x ) ]= A∗B

● [ ]
f (x)
g(x) B
A
= , b≠0

r
● [ f ( x ) ] =A r , if Ar is defined and r is a real number.

L’Hôpital’s Rule: We want to find the limit of a quotient when x tends to a , but both the numerator
and denominator are equal to zero, thus we use L’Hôpital’s rule. If f ( a )=g ( a )=0 and g ≠ 0 then:

'
f ( x ) f ( a)
= '
g ( x ) g (a )

3.2 Finding Derivatives

The derivative of a function measures the sensitivity to change of f (x) with respect to a change in x .
The first derivative at point x , is the slope of the tangent line at x . The second derivative is at point
x measures the sensitivity of the first derivative and is used to characterise point x .

Newton Quotient: The derivative of the function f at point a , denoted by f ' (a), is

' f ( a+ h )−f (a)

f ( a )=
h

Page 7 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

That is, we take the slope of the line connecting two points with distance Δ x and move them
infinitely close to one another. The equation for the tangent to the graph of y=f (x ) at the point
(a , f ( x ) ) is

y−f ( a ) =f '( a)( x−a)

If f ' is also differentiable at a , then we can find the second derivative f ' ' ( a) as:

'' f ' ( a+h )−f ' (a)

f ( a) =
h

Steps for computing the derivative:

i) Add h to a and compute f (a+ h)

ii) Compute the corresponding change in the function value f ( a+h ) −f (a).
iii) For h ≠ 0 form the Newton quotient
iv) Simplify the Newton quotient fraction. Try to cancel h from the numerator and denominatior
v) Then f ' (a) is the limit at h tends to 0 of the Newton quotient.

3.2.1 Rules for Differentiation

All the rules require that the function is differentiable at x

Additive constants: If f is a constant function, then its derivative is 0.

'
f ( x )=a ⟹ f ( x )=0

Sums and differences: The sum f ( x ) + g(x ) and the difference f (x)−g (x) are differentiable as:

' '
F ( x )=f ( x ) ± g ( x ) ⟹ F ( x ) =f ( x ) ± g' ( x )

Product rule: If we have a function as a product of two function F ( x )=f ( x ) g(x ), the derivative is

' ' '

F ( x )=f ( x ) g ( x ) ⟹ F ( x )=f ( x ) g ( x ) +f ( x ) g ( x)

Quotients: If g ( x ) ≠ 0, then F=f /g and the difference f −g is differentiable at x

'
f (x) f ( x ) g ( x )−f ( x ) g ' ( x )
F ( x )= ⟹ F ' ( x )=
g(x) [ g ( x )]
2

The chain rule: If the composite function F ( x )=f ( g ( x ) ) is differentiable as

F ( x )=f ( g ( x ) ) ⟹ F ( x )=f ( g ( x ) ) g ( x )
' ' '

Exponential function: Note we have e x =0 and e x =∞. The derivative is:

x ' x
f ( x )=e ⟹ f ( x )=e

Page 8 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

Logarithmic function: It may be useful to create a composite function y=ln ln ( h ( x ) ), impaling that
'
y =h ' (x)/h(x ), i.e., the relative growth of h( x). The derivative of the natural logarithmic function:

' 1
g ( x )=ln ln x ⟹ g ( x )=
x

3.3 Characterising a Function with Derivatives

Increasing and Decreasing Functions: If f is defined on I and x 1 and x 2 are numbers in I , then:

i) If f ( x 2 ) ≥ f (x 1 ) whenever x 2> x1 , then f is increasing in l

ii) If f ( x 2 ) > f ( x 1) whenever x 2> x1 , then f is strictly larger in l

iii) If f ( x 2 ) ≤ f (x 1 ) whenever x 2> x1 , then f is decreasing in l

iv) If f ( x 2 ) < f ( x 1) whenever x 2> x1 , then f is strictly decreasing in l

Using the sign of the derivative, we have the following conclusions

'
● f ( x ) ≥ 0 for all x in the interval I , f is increasing in l

'
● f ( x ) ≤ 0 for all x in the interval I , f is decreasing in l

'
● f ( x )=0 for all x in the interval I , f is constant in l

Concave and Convex Functions: Let f be continuous and twice differentiable in the interior of l :

'' ''
● f is strictly concave if f ( x )< 0 and concave if f ( x ) ≤ 0

'' ''
● f is strictly convex if f ( x )> 0 and convex if f ( x ) ≥ 0

A function is convex if for any x , y ∈l and for any α ∈ [ 0 ,1 ]:

f ( (1−α ) x +αy ) ≤ ( 1−α ) f ( x ) +αf ( y )

The function is concave if for any x , y ∈l and for any α ∈ [ 0 ,1 ]:

f ( (1−α ) x +αy ) ≥ ( 1−α ) f ( x ) +αf ( y )

3.4 Taylor Approximations

If a function f is differentiable at x=a , then we can approximate the function for x close to a by the
tangent line, which is the first order approximation of f (x) at x=a

f ( x ) ≈ f ( a )+ f '(a)( x−a)

Page 9 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

If a function is approximately linear in a neighborhood around x=a , then this may be a good
approximation, but for a highly non-linear function this may not be the case.

More generally, if a function y=f (x ) is n times differentiable at a it can be approximated by an n th

order Taylor approximation such that for x close to a

' '' (n )
f (a) f (a) f (0)
f ( x ) ≈ f ( a )+ ( x−a )+ ( x−a )2 + …+ ( x−a )n (3.6)
1! 2! n!

An example of an n th order approximation is the approximation of e x at x=0

2 n
x x x x
e ≈ 1+ + +…+
1! x ! n!

Taylor Formula: An exact solution is adding a remainder term to the Taylor approximation at x=0

' '' (n )
f f 2 f n
f ( x )=f ( 0 ) + x + x +…+ x + R n+1 (x ) (3.7)
1! 2! n!

It can be shown that if f is n+1 times differentiable in an interval that includes 0 and x , then

1 n+ 1
Rn +1 ( x )= f ( n+1) ( z ) x
( n+1 ) !

where z is a between 0 and x . It must hold that if f (n+ 1) ( x ) ≤ M in an interval, then in that interval

M
|R n+1 ( x )|≤ ( n+ 1 ) ! |x|
n+ 1

Intermediate Value Theorem: Let f be a function which is continuous in the interval [a , b]. Then it
takes on any given value between f (a) and f (b) at some point within the interval. More formally:

(1) If f (a) and f (b) have different signs, then there is at least one c in (a ,b) such that f ( c ) =0
(2) If f ( a ) ≠ f ( b ) then for every intermediate value y in the open interval between f (a) and
f (b) there is at least one c in (a ,b) such that f ( c ) = y .

Page 10 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

4 SINGLE VARIABLE OPTIMISATION

Carol (2008): Chapter 1 + Lecture 2
When we optimize with respect to one variable, we are identifying extreme points, where the domain
of a function reaches its largest (maximum) or its smallest values (minimum). If f (x) has domain D :

● c ∈ D is a maximum point for f , then f ( x ) ≤ f (c) for all x ∈ D

● d ∈ D is a minimum point for f , then f ( x ) ≥ f (c) for all x ∈ D

Interior of an interval: All points in the open interval (a ,b) are interior to the intervals [ a , b ], ¿ and
¿. For the intervals ¿ and (a ,b) , the end point b can by ∞ ; for the intervals ¿ and (a ,b) the end point
a can be −∞ .

Extreme Value Theorem: Suppose that f is continuous over a closed and bounded interval [ a , b ].
Then there exists a point d in [ a , b ] where f has a minimum, and a point c in [ a , b ] where f has a
maximum – that is, one has f ( d ) ≤ f ( x ) ≤ f (c ) for all x in [ a , b ].

Mean Value Theorem: If f is continuous in the closed and bounded interval [a , b], and
¿
differentiable in the open interval (a ,b), then there exists at least one point x in (a ,b) such that

f ( b )−f ( a )
f ' ( x¿) =
b−a

4.1 Finding Extreme points

Necessary first-order condition (FOC): A function f is differentiable in an interval l and c is an
interior point of l . For x=c to be a maximum or minimum, it must be a critical point or stationary
point for f , that is x=c is a solution of f ' ( x )=0 .

First-derivative test for extrema: If the function f (x) is differentiable in an interval including c :

' '
● If f ( x ) ≥ 0 for x ≤ c and f ( x ) ≤ 0 for x ≥ c , then x=c is a maximum point for f

' '
● If f ( x ) ≤ 0 for x ≤ c and f ( x ) ≥ 0 for x ≥ c , then x=c is a minimum point for f

Finding the extreme of bounded functions: To find the maximum and minimum values of a
differentiable function f defined on a closed, bounded interval [ a , b ], we need to

(1) Find all critical points of f in (a ,b)

(2) Evaluate f at the end points a and b of the interval and also at all critical points
(3) The largest function value found is the maximum and the smallest the minimum of f in [ a , b ]

Page 11 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

Extrema for concave and convex functions: Suppose the function f (x) is differentiable in an
interval l that c is a critical point for f in the interior of l

● If f is concave, then c is a maximum point for f

● If f is convex, then c is a minimum point for f

4.2 Local Extreme Points

The function f has a local maximum at c if there exists an interval (α , β ) about c such that
f ( x ) ≤ f (c) for all x in (α , β ) which are in the domain of f . It has a local minimum at c if there
exists an interval (α , β ) about c such that f ( x ) ≥ f (c) for all x in (α , β ) which are in the domain of f

First-derivative test for local extreme: Suppose c is a critical point for y=f (x )

(1) If f ' ( c ) ≥ 0 both throughout some interval (a ,c ) to the left of c and some interval (c , b) to the
right of c , then x=c is a local maximum
(2) If f ' ( c ) ≤0 both throughout some interval (a ,c ) to the left of c and f ' ( x ) ≤ 0 throughout some
interval (c , b) to the right of c , then x=c is a local minimum
(3) If f ' ( c ) ≤0 both throughout some interval (a ,c ) to the left of c and f ' ( x ) ≥ 0 throughout some
interval (c , b) to the right of c , then x=c is not a local extreme point for f . The same
conclusion holds if f ' ( x ) <0 on both sides of c .

Second-derivative test for local extreme: Let f be a twice differentiable and c be an interior point:

(1) If f ' ( c )=0 and f '' <0 , then x=c is a strict local maximum point

(2) If f ' ( c )=0 and f '' >0 , then x=c is a strict local minimum point

(3) If f ' ( c )=0 and f '' =0, then the character of x=c remains undetermined

Necessary second-order conditions: Let f be a twice differentiable and c be an interior point:

Point c is a local maximum for f , if f ' ' ( c ) ≤ 0

Point c is a local minimum for f , if f '' ( c ) ≥ 0

4.3 Inflection Points

If the function f is twice differentiable, the point c is an inflection point (saddle point) for f , if there
exists an interval (a ,b) about c such that either

(1) f '' ≥0 in (a ,c ) and f '' ≤0 in (c , b)

(2) f '' ≤0 in (a ,c ) and f '' ≥0 in (c , b)

Page 12 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

In other words, the point where a function changes from concave to convex, or vice versa.

Test for Inflection Points: Let f be a function with a continuous second derivative in an interval l
and let c be an interior point of l

(1) If c is an inflection point for f , then f '' =0

(2) If f ' '=0 and f ' ' changes sign at c , than c is an inflection point for f

5 NUMERICAL METHODS AND INTEGRATION

Carol (2008): Chapter 1 + Lecture 3

5.1 Newton’s Method

As long as f ' ≠ 0, Newton’s method generates the sequence of points given by the formula

f ( xn)
x n+1=x n − , n=0 , 1 , … (3.10)
f ' ( xn )

Usually, the sequence {x n } converges quickly to a zero of f

¿
Convergence of Newton’s Method: Suppose that f is twice differentiable with f ( x )=0 and

f ( x ) ≠ 0, and that there exists numbers K >0 and δ >0, with kδ <1 such that
' ¿

|f ( x ) f ' ' (x)|

≤ K |x −x|
¿
' 2
f (x)

for all x in an open interval l=¿ ). Then, provided that the sequence { x n } from Newton’s method starts

| |
at an x 0 in l , it will converge to x with an error |x n −x | that satisfies
¿ ¿

( δK )2 n
||x n −x¿||≤ K

Problems with Newton’s method: There are several problems with Newton’s method

● Sensitivity to starting point: Iteration formula is not a valid stational point (derivative is zero) or

starting point may enter a cycle

● Derivative issues: Derivative may not exist at the root or may be discontinuous

● Non-quadratic convergence: If first derivative is zero at root, then convergence is not quadratic.

Page 13 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

5.2 Bisection Method

The purpose of the bisection method is to find the root of a continuous function when we know two
values (points) with opposite signs. The input is a continuous function, f , an interval [ a , b ] and the
function values f (a) and f (b) which are of opposite sign. By the intermediate value theorem, there
must be a root in the interval. Each iteration of the method performs these steps

a+ b
(1) Calculate c , the midpoint of the interval c=
2
(2) Calculate the function value at the midpoint, f (c )
(3) If convergence is satisfactory, e.g. |f (c )| is sufficiently small, return c and stop iterating

(4) Examine the sign of f (c ) and replace either ( a , f ( a ) ) or ( b , f ( b ) ) with ( c , f ( c ) ) so that there is
zero crossing within the new interval.

The absolute error of the bisection method is halved at each step (linear convergence vs. quadratic
convergence for Newton). If c 1=(a+b)/2 is the midpoint of the initial interval and c n is the midpoint
of the interval in the n th step, then the difference between c n and a solution c is bounded by
|b−a|
|cn −c|< .
2n

5.3 Introduction to Integration

Integration is the opposite of differentiation. Differentiating a probability distribution gives the
density function, and integrating the density function gives the distribution function.

The integral of a function is the area between the curve and the x -axis. If the area is above the axis it
is positive, and if it is below the axis it is negative. When we place limits on the integral sign, the area
is calculated between these limits. This is a definite integral as the result is a number, as opposed to
indefinite integral, which is a function.

Integration is the opposite to differentiation, i.e.,

If f ( x )=F ' (x ) then F ( x )= ∫ f ( x ) dx

Rules for integration are the opposite of the rules for differentiation. Given a function f (x) of a real
variable x and an interval [ a , b ] of the real line, the definite integral is given as

∫ f ( x ) dx
a

where a is the lower limit and b is the upper limit of the range over which the area is calculated.

Page 14 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

Riemann Integral: Defined as the limit of a Riemann sum. An example of a Riemann sum is

∑ f ( a+ b−a N ) N
N
b−a b−a
n−0.5
n =1 N

The sum consists of N values which we can think of as rectangles with width (b−a)/ N and height
b−a b−a
equal to function f evaluated at the midpoint between (n−1) and n . The value of this
N N
sum will (generally) approach the area under the graph as N becomes larger.

Integration vs Differentiation: Let A(t) be the area under the curve y=f (x ) over the interval [a , t]
, whereas we are interested in the area under the curve over the interval[ a , b ]

b
A=∫ f ( x ) dx
a

We know A ( a )=0 and A ( b )=A . Consider the change in the area A(t), Δ a= A ( t + Δ t )− A (t). As
the function is decreasing on the interval from t to t+ Δ t , we have the inequality for all Δ t >0 :

f ( t ) Δt ≥ A ( t + Δt )− A ( t ) ≥ f (t + Δt ) Δt

As Δ t >0 , this implies that

A ( t+ Δt )− A ( t )
f (t ) ≥ ≥ f ( t+ Δt )
Δt

When Δ → 0 , we see that it must hold that A' ( t ) =f (t ). This argument holds for all t in [a , b].
Therefore, the derivative of the area function A(t) is the curve’s height function f (t)

5.4 Indefinite Integrals

We have two functions f (x) and F (x) defined for all x in some interval l with the property
f (x)=F ' . We define the indefinite integral

Page 15 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

∫ f (x )dx =F ( x )+ C
❑

where C is an arbitrary constant. We can thus see F (x) as an anti-derivative. We regard F ( x ) +C as

a class of functions that differs by a constant. We see that
❑
d
∫ f (x)dx=f (x)
dx ❑

Thus, the derivative of an indefinite integral equals the integrand. We also have
❑

∫ F ' (x )dx=F ( x )+ C
❑

Important Integrals: These follow immediately from the corresponding rules for differentiation

● Integration of power function: If a ≠ 1, then

❑
1 a+1
∫ x a dx= a+1 x +C
❑

● Integration of 1/ x : For x >0 and x <0, we have

∫ 1x dx=ln ln|x|+C
❑

ax
● Integration of e : If a ≠ 0 , then

∫ eax dx= 1a eax +C

❑

x
● Integration of a : If a> 0 and a ≠ 1, then

∫ ax dx= ln 1ln a a x + C
❑

General Basic Rules for Integration: If a ≠ 0 is a constant, then

❑ ❑

∫ af (x)dx=a∫ f ( x ) dx
❑ ❑

❑ ❑ ❑

∫ [f ( x )+ g ( x ) ]dx=∫ f (x )dx +∫ g ( x ) dx
❑ ❑ ❑

Page 16 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

5.5 Definite Integrals

Suppose that f is a continuous function in the interval [a , b] and suppose that F is also continuous in
[a , b] with derivative F ' ( x )=f (x) for all x in [a , b]. Then, we define the definite integral
b

∫ f ( x ) dx=F ( b ) −F (a)
a

where F is any indefinite integral of f over an interval containing both a and b .

b
If f ( x ) ≥ 0 over [a , b] then ∫ f (x )dx is the area below the graph of f over [a , b]. If f ( x ) ≤ 0 for all x
a

in [a , b], then graph of f and the line defined by the two points ( x , y ) =(a ,0) and ( x , y ) =( b , 0 ) still
encloses and area. This area is obtained by defining g ( x )=−f ( x) for which we can apply the theory
b
and get the area above the graph of f as ∫ (−f )(x )dx .
a

Properties of Definite Integrals: If f is a continuous function in an interval that contains the points a
, b and c , and α is an arbitrary number, then we have
b a

∫ f (x )dx =−∫ f ( x ) dx
a b

∫ f (x )dx =0
a

b b

∫ αf (x )dx=α ∫ f (x)dx
a a

b c b

∫ f (x )dx =∫ f (x )dx+∫ f (x )dx

a a c

Example: To get the area under f ( x )=x 2 over the interval [0 ,1] , we first get the anti-derivative
1 3
F ( x )= x . Then, we evaluate at the limits
3

[ ]
1
1 3 1 3 1 3 1
∫ x 2 dx= 3
x = 1− 0=
3 3 3
0

Numerical Evaluation: Refers to algorithms for calculating the numerical value of a definite integral.
First, the rectangle rule is defined as
b

( )
N
b−a b−a b−a
∫ f (x )dx ≈ ∑ f a+ N
n−0.5
N N
a n=1

Page 17 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

While the trapezoidal rule is

( )
b N −1

( )
N
b−a f ( a ) b−a f ( b )
∫ f (x )dx ≈ ∑ N 2 n=1
+ ∑ f a+n
N
+
2
a n=1

Differentiation wrt. The Limits of Integration: Suppose that the limits of integration a (t) and b (t)
depends on t and are differentiable and f (x) is continuous, then

b (t )
d
∫ f (x)dx=f ( b ( t ) ) b ' ( t )−f ( a ( t )) a' (t)
dt a (t )

This is a special case of Leibniz’s formula.

5.6 Integration Methods

Integration by Parts: Suppose that f (x) and g(x ) are continuously differentiable functions, then
❑ ❑

∫ f ( x ) g '(x ) dx=f ( x ) g ( x )−∫ f ' ( x ) g(x)dx

❑ ❑

This follows directly from using the product rule for differentiation on f (x) g(x )

Integration by Substitution: Suppose that g is continuously differentiable, and f (u) is continuous at

all points u belonging to the relevant range of g. Then,
❑ ❑

∫ f ( g ( x ) ) g ' (x )dx=∫ f (u)du

❑ ❑

where u=g (x) . If we have a definite integral, we can simply use

b g ( b)

∫ f ( g ( x ) ) g ' (x )dx= ∫ f (u) du

a g ( a)

❑
General method: In order to find ∫ G(x )dx :
❑

(1) Pick out a part of G(x ) and introduce this part as a new variable u=g (x)
(2) Compute du=g' ( x ) dx
❑
(3) Using substitution u=g (x) , du=f ( x ) dx, transform, if possible, ∫ G(x )dx to an integral of
'

❑
❑
the form ∫ f (u)du
❑
❑
(4) Find, if possible, ∫ f (u)du=F ( u ) +C
❑

(5) Replace u by g(x )

Page 18 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

Then, the final answer is

❑

∫ G(x )dx=F ( g ( x ) ) +C
❑

5.7 Interest Rates and Present Value

The annual rate/nominal rate is often the quoted rate. However, the interest period may not be equal
to a year. The periodic rate is obtained by dividing the annual rate, r , with the number of interest
periods during the year, i.e., r periodic =r /n .

Suppose we invest S0 at the annual rate r with n interest periods. After t years, we would have

( )
nt
r
St =S0 1+
n

When interest is added n times during the year at the rate r /n per period, then the effective yearly
rate, R , is defined as

( )
n
r
R= 1+ −1
n

Continuous Compounding: When the number of interest periods goes to infinity, n → ∞ , then

[( ) ] [( ) ]
n t m rt
r 1
St =S0 1+ =S 0 1+
n m

( )
m
1
where m=n/r . Since m →∞ as n → ∞ , and 1+ =e, then in the limit we have
m
rt
St =S0 e

Present Value: If an amount K is payable in t years, then the present value of this amount with
annual interest payment is
−t
K ( 1+r )

and with continuous compounding interest

−rt
Ke

We call ( 1+r )−t and e−rt discount factors.

Geometric Series: A geometric series with quotient k , where k ≠ 1, is defined by

n
n−2 n−1 k −1
sn=a+ak + …+a k +ak =a
k−1

Page 19 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

As n tends to infinity, and |k|<1, then the limit is a /(1−k ).

Total Present Value: The present value of n successive yearly payments a 1 , … , an at the rate of r is

a1 an
P N= + …+
1+r ( 1+ r )n

An annuity is a sequence of equal payments made at fixed periods of time over some time span, i.e.,
all payments in the above formula is equal to a . Applying the formula for a geometric series yields

[ ]
−n
a 1−( 1+r ) a 1
P n= −1
= 1− n
1+r 1− (1+ r ) r ( 1+r )

If an amount a is deposited in an account each period for n periods, earning interest at r per period,
then the future value of the annuity is given by
n−1
F n=a+ a ( 1+ r ) +…+ a (1+ r )

Applying the formula for a geometric series yields

n
1−( 1+r ) 1
F n=a = [ ( 1+r )n −1 ]
1−(1+ r) r

The present discounted value, at time t=0 of a continuous income stream at the rate of f (t) dollars
per year over the time interval [0 ,T ], with continuously compounded interest rate r per year is

T
PDV =∫ f ( t ) e
−rt
dt
0

The future discounted value, at time t=0 of a continuous income stream at the rate of f (t) dollars
per year over the time interval [0 ,T ], with continuously compounded interest rate r per year is

T
FDV =∫ f ( t ) e
r (T−t )
dt
0

The discounted value, at time t=0 of a continuous income stream at the rate of f (t) dollars per year
over the time interval [s ,T ], with continuously compounded interest rate r per year is

T
DV =∫ f ( t ) e
−r (t −s)
dt
0

Mortgage Repayments: Suppose that a loan of K is repaid as an annuity over n periods at the
interest rate r , where the first payment a is due after one period and the rest at equally spaced periods.
From the present value of an annuity, we have

Page 20 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

K=
a
r[1−
1
( 1+r ) n
]
a
= [ 1−( 1+r ) ]
r
−n

Solving for a yields

rk
a= −n
1−( 1+r )

The number of periods to pay off a loan of amount K

ln ln a−ln ln ( a−rK )
n≥
ln ln ( 1+r )

Internal Rate of Return: Consider the net returns a 0 , … , a n for n successive years from an
investment project. The net present value is

a1 a2 an
A=a0 + + 2
+…+
1+r ( 1+r ) ( 1+r )n

The internal rate of return is the interest rate r for which the present value is equal to zero, i.e.

a1 a2 an
a 0+ + 2
+…+ =0
1+r ( 1+ r ) ( 1+r )n

Returns: Let Pt be the price of an asset at time t . The net return from time t−1 to t is

Pt Pt−P t−1
Rt = −1=
P t−1 P t−1

Gross return from time t−1 to t is

Pt
=1+ Rt
P t−1

The gross return of the most recent k periods is the product of the single period goss return:

Pt
=( 1+ Rt )∗…∗( 1+ R t−k +1 )
P t−k

Log returns are also called continuously compounded returns given by

r t =ln ln (1+ Rt )=ln ln

( )
Pt
Pt−1
=ln ln Pt −ln ln Pt −1= p t− pt −1

The log return over k periods is simply the sum of single period log returns. It will hold that
ln ln (1+ R)≈ R for R close to zero.

Page 21 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

6 FUNCTIONS OF MULTIPLE VARIABLES

Carol (2008): Chapter 1 + Lecture 4

Denote any ordered collection of n numbers by the vector x=(x 1 … , x n). Given a set D of n -vectors,
a function f of n variables x 1 , … , x n with domain D is a rule that assigns a specified number

f ( x )=f (x 1 , … , x n )

to each n -vector x=(x 1 , … , x n) in D .

The concept of continuity extends to the n -dimensional setting. Roughly speaking, a function of n
variables is continuous if small changes in the independent variables induce smalle changes in the
function value.

6.1 Partial Derivatives

If z=f ( x ) =f (x 1 , … , x n ) then

∂f
,i=1 , 2 , … , n
∂ xi

means that the partial derivative of f (x 1 , … , x n) wrt. x i when all the other variables x j for j ≠ i are
held constant. Formally, the partial derivative is the limit

∂ f f ( x 1 ,… , x i +h , … , xn ) −f ( x 1 , … , x i , … , x n)
=
∂ xi h

provided that the limit exists. If the limit does not exist then the function is not differentiable at this
point. With n=2, we have:

∂ f (x , y) f ( x +h , y )−f (x , y )
=
∂x h

Partial Derivatives: Gradient. We denote the vector of first order partial derivatives as the gradient:

Df ( x )=( f '1 ( x ) f '2 ( x ) ⋮ f 'n ( x ) )

Partial Derivatives: Hessain Matrix. Each of the n first order partial derivatives of f has n second-
order partials, i.e.,

( )
2
∂ ∂f ∂ f ''
= =z ij
∂ x j ∂ x i ∂ x j ∂ xi

The second order derivatives are gathered in the Hessain matrix:

Young’s Theorem: States that all two cross-partial derivatives usually are equal

Page 22 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

∂ ∂f
( ) ( )
=
∂ ∂f
∂ x j ∂ xi ∂ xi ∂ x j

Suppose that all the m th-order partial derivatives of the function f (x 1 , … , x n) are continuous. If any
two of them involve differentiating wrt. each of the variables the same number of times, then they are
necessarily equal.

Geometric Representation: To understand partial derivatives, consider the point (x 0 , y 0). The

partial derivative f 'x ( x 0 , y 0) is the derivative of z=f (x , y 0 ) wrt. x . Thus, we can think of the partial

derivative as the tangent at the point x 0 on the curve defined by ( x , y 0 , f ( x , y 0 ) ).

6.1.1 The Chain Rule

Consider z=f (x , y ) where x=g(t) and y=h(t), so z=F (t )=f ( g ( t ) , h ( t ) ) . The total derivative
with respect to t is then given by

dz ' dx ' dy
=f 1 ( x , y ) + f 1 ( x , y )
dt dt dt

Instead assume that z=F (t )=f ( g ( t , s ) , h ( t , s ) ). The total derivative with respect to t is then given

∂z ' ∂x ' ∂y
=f 1 ( x , y ) + f 2( x , y )
∂t ∂t ∂t

∂z ' ∂x ' ∂y
=f 1 ( x , y ) + f 2( x , y )
∂s ∂s ∂s

The general chain rule: If z=f (x 1 , … , x n) is continuously differentiable, and x i=g(t 1 ,… ,t m ) for
each i=1 , 2 ,… , n are all differentiable then

∂ z ∂ z ∂ x1 ∂ z ∂ xn
= + …+
∂ t j ∂ x1 ∂ t j ∂ xn ∂ t j

6.2 Surfaces and Distance

A general 3-dimensional plan is defined by the equation

ax +by + cz=d

with a , b , and c not all zero. The distance between the two point (x 1 , y 1 , z 1 ) and (x 2 , y 2 , z2 ) is

d= √ ❑

The equation for a sphere with center (a ,b ,c ) and radius r is

( x−a )2+ ( y−b )2+ ( z−c )2=r 2

Page 23 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

6.3 Homogeneous and Homothetic Functions

Suppose that f is a function of n variables defined in domain D , which is a cone. Thus, if
( x 1 ,… , x n ) ∈ D then for t >0 ( t x 1 , … ,t x n) ∈ D. Then f is homogeneous of degree k if

k
f ( t x 1 , … , t x n )=t f ( x 1 ,… , x n )c

Euler’s Theorem: Suppose that f is a differentiable function of n variables, defined in an open cone
D . Then, f is homogeneous of degree k if the following holds for all x in D :
n

∑ x i f 'i ( x )=kf ( x )
i=1

Homothetic Functions: Let f be a function of n variables x=(x 1 , … , x n) defined in a cone K . Then

f is called homothetic if

x , y ∈ K , f ( x )=f ( y ) ,t >0 ⟹ f ( tx ) =f (ty)

Relationship between Homogeneous and Homothetic Functions: Suppose that the function F can
be written as the composition of functions H and f , so that F ( x )=H ( f ( x ) ) . If H is strictly increasing
and f is homogeneous of any degree, then F is homothetic. It also holds that homogeneous function f
of any degree k is homothetic.

6.4 Linear Approximations

0 0 0
The linear approximation of f ( x )=f (x 1 , … , x n ) about x =( x 1 , … , x n) is given by

f ( x ) ≈ f ( x 0 ) + f '1 ( x 0 ) ( x 1−x 01 ) + …+ f 'n ( x 0 ) ( x n−x 0n )

Example: Consider the function f ( x , y )=−( x 2 + y 2 ) the linear approximation is then given by

f ( x , y )=f ( x 0 , y 0 ) + f 'x ( x 0 , y 0 ) ( x−x 0 ) + f 'y ( x 0 , y 0 ) ( y− y 0 )=−( x20 + y20) −2 x 0 ( x −x 0) −2 y 0 ( y− y 0 )

6.5 Differentials
The differential of a function z=f (x 1 , x 2 ,… , x n ) of n variables is defined by

' '
dz=df =f 1 d x 1 +…+ f n d x n

Rules for differentials: Let f and g be differentiable functions of x , and let a and b be constants.
Then the following rules hold true

d ( af +bg )=adf + bdg

d ( fg ) =gdf + fdg

Page 24 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

and, if g ≠ 0

d ( fg )= gdf g−fdg
2

Chain rule for Differentials:

z=g ( f ( x ) ) → dz=g ( f ( x ) ) df
'

6.6 Systems of Equations

A system of m equations in n unknowns can be written as

f 1 ( x 1 , x 2 , … , x n ) =0 f 2 (x 1 , x 2 , … , x n )=0 ⋮ ⋮ ⋮ f m ( x 1 , x 2 ,… , x n )=0

To find the degree of freedom for a system of equations, count the number of variables, n , and the
number of independent equations m . In general, if n> m, there are n−m degrees of freedom in the
system. If n< m, there is no solution to the system. A system of equations in n variables is said to
have k degrees of freedom if there is a set of k variables that can be freely chosen, while the
remaining n−k variables are uniquely determined once k free variables have been assigned specific
values.

Page 25 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

7 CONTINUOUS OPTIMISATION
Carol (2008): Chapter 1 + Lecture 5

7.1 Linear Regression as a Minimisation Problem

A linear model can be written as

y i=α + β x i +e i ,i=1 , … , N

where α and β generally are unknows and therefore chosen to make the error term minimal. To that
end, we define the loss function we want to minimise
N N
1 1
L ( α , β )=
N
∑ e 2i = ∑ ( y −α −β x i )2
N i=1 i
i=1

By finding the FOC, one can show that the solution to the minimisation problem is
N

∑ ( y i− y)( xi −x), α^ = y−β x

^β= i=1
❑
Non-Linear Regression as a Minimisation: If the relationship between y and x is non-linear, we
generally do not have a closed form solution

y i=f ( x i ; θ ) + ei , i=1 , … , N

where f (x) is some real valued function and θ is chosen to make the error term as small as possible.
Again, we define the loss function we wish to minimise
N N
1 1
L ( θ )= ∑ e2i = ∑ ( y i−f ( x i ; θ ) )
2

N i=1 N i=1

7.2 Multivariate Optimisation

Single-variable Case: A differentiable function can only have a max or min at an interior point of its
domain if it is a critical point.

Two-variable Case: A differentiable function z=f ( x , y ) can have a maximum or minimum at an

interior point ( x 0 , y 0 ) of its domain only if it is a critical point, that is, if the point ( x , y ) =( x 0 , y 0 )
satisfies the two (necessary) first order conditions (FOCs) :
'
f 1 ( x , y )=0
'
f 2 ( x , y )=0

Page 26 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

The intuition is that if we fix y at y 0, then the critical point for the function z=g ( x )=f (x , y 0 ) is
' '
defined where g x ( x )=f x (x , y 0). If we fix x=x 0, then the critical point for the function

z=h ( y ) =f (x 0 , y) is defined where h'y ( y )=f 'y ( x 0 , y ) =0.

First order conditions are necessary, but not sufficient!

● Single-variable case: A sufficient condition for min or max at a critical point in an interval l is

that the function was respectively convex or concave on this interval.

Sufficient Conditions for a Maximum or Minimum: Suppose that (x 0 , y 0) is an interior point and
that f ( x , y ) is a two times differentiable function defined on a convex set S in R2.

(a) If for all ( x , y ) in S, one has

''
f 11 ( x , y ) ≤0
''
f 22 ( x , y ) ≤ 0
2
f ''11 ( x , y ) f '22' ( x , y )−[ f '12' ( x , y ) ] ≥ 0

then (x 0 , y 0) is a maximum point for f (x , y ) in S

(b) If for all ( x , y ) in S, one has

''
f 11 ( x , y ) ≥0
''
f 22 ( x , y ) ≥ 0
2
f ''11 ( x , y ) f '22' ( x , y )−[ f '12' ( x , y ) ] ≥ 0

then (x 0 , y 0) is a minimum point for f (x , y ) in S

7.2.1 Local Extreme Points

The point (x 0 , y 0) is said to be …

… a local maximum point of f if f ( x , y ) ≤ f (x 0 , y 0 ) for all pairs in S sufficient close to ( x 0 , y 0 )

… a local minimum point of f if f ( x , y ) ≥ f (x 0 , y 0 ) for all pairs in S sufficient close to ( x 0 , y 0 )

We talk about a strict local max or min if we have a strict inequality for ( x , y ) ≠(x 0 , y 0). Any local
extreme point in the interior of the domain of a differentiable function must be critical. However, at
critical point does not have to be a max or min, but can also be a saddle point.

Page 27 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

A saddle point (x 0 , y 0) is a critical point with the property that there exist points ( x , y ) arbitrarily

close to ( x 0 , y 0 ) with f ( x , y )< f ( x0 , y 0 ) and there also exists points with f ( x , y )> f ( x0 , y 0 ).

Second-Derivative Test for Local Extrema: Suppose that f (x 0 , y 0 ) is a C 2 function in a domain S,

and let (x 0 , y 0) be an interior critical point of S. Define

'' '' ''

A=f 11 (x 0 , y 0 ), B=f 12 (x 0 , y 0 ),and C=f 22(x 0 , y 0)

(a) If A< 0 and AC−B2 >0 , then (x 0 , y 0) is a strict local maximum point
(b) If A> 0 and AC−B2 >0 , then (x 0 , y 0) is a strict local minimum point
(c) If AC−B2 <0 , then (x 0 , y 0) is a saddle point
(d) If AC−B2=0, then (x 0 , y 0) could be a local maximum, a local minimum or a saddle point.

7.2.2 Extreme Value Theorem

Suppose that the function f ( x , y ) is continuous throughout a nonempty closed and bounded set S in
the plane. Then there exist both a point ( a , b ) in S where f has a minimum and a point ( c , d ) in S
where it has a maximum, that is for all ( x , y ) in S

f ( a , b) ≤ f ( x , y )≤ f ( c , d )

Finding Maxima and Minima: In order to find the maximum and minimum values of a
differentiable function f (x , y ) defined on the closed, bounded set S in the plane:

(1) Find all critical points of f in the interior of S

(2) Find the largest value and the smallest value of f on the boundary of S, along with associated
points. If it is convenient, subdivide the boundary into several pieces and find the larges and
smallest value on each piece.
(3) Compute the values of the function at all the points found in (1) and (2). The largest function
value is the maximum value of f and the smallest one is the minimum value of f in S.

Page 28 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

The General Case: Necessary Conditions for Interior Extrema: Suppose that f is defined in a set
S in Rn and let c=(c 1 , … ,c n ) be an interior point in S where f is differentiable. A necessary
condition for c to be a maximum or minimum point of f is that c is a critical point for f , i.e., x=c
satisfies the n first-order conditions stating that for each i=1 , … , n

f i ( x )=0

The General Case: The Extreme Value Theorem: Suppose that the function f is continuous
throughout a nonempty, closed and bounded set S in Rn . Then there exist both a point a in S where f
has a minimum and a point b in S where f has a maximum – that is, for all x in S, we have

f ( a ) ≤ f ( x ) ≤ f (b)

7.3 Constrained Optimisation: Lagrange Multiplier

The Lagrange multiplier method is useful for constrained optimization. Assume we have

f (x , y ), s . t g ( x , y )=c

To deal with this problem, we use the Lagrangian function, L:

L ( x , y )=f ( x , y )−λ [ g ( x , y )−c ]

where λ is the Lagrange multiplier. This can be solved as follows

(1) Write down the Lagrangian function, where λ is a constant

(2) Differentiate L w.r.t. x and y , and equate the partial derivatives to 0
(3) The two equations in (ii), along with the constraint, yield the three first-order conditions:
' ' ' ' ' '
L1 ( x , y )=f 1 ( x , y )−λ g1 ( x , y )=0 L2 ( x , y )=f 2 ( x , y )−λ g2 ( x , y )=0 g ( x , y )=c

(4) Solve the three equations simultaneously for x , y , and λ . These triplets (x , y , λ) are the
solution candidates, at least one of which solves the respective problem if it has a solutions

Importantly, if g'1 (x , y ) and g'2 (x , y) both vanish, the method might fail. Generally, the solution

( x ¿ , y ¿ ) =( x ¿ ( c ) , y ¿ ( c ) ) will be a function of c and so will the associated function value.

Assume that these solutions are differentiable functions of c . Then, we can write the value function as

f ¿ ( c )=f ( x¿ ( c ) , y ¿ ( c ) )

Given some regularity conditions, we have

¿
df
= λ(c )
dc

Page 29 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

Thus, λ is the rate at which the optimal value of the objective function changes with respect to
changes in the constrained constant c .

Concave/Convex Lagrangian: Consider a problem of the type

and suppose that (x 0 , y 0) is a critical point for the Lagrangian L.

(a) If the Lagrangian is concave, then ( x 0 , y 0 ) solves the maximization problem

(b) If the Lagrangian is convex, then ( x 0 , y 0 ) solves the minimization problem

Local Second-Order Conditions: Consider a problem of the type

and suppose that (x 0 , y 0) satisfies the first order conditions. Define

' 2 ' 2
D ( x , y , λ )=( f 11−λ g11 )( g2 ) −2 ( f 12−λ g 12) g1 g 2+ ( f 22−λ g22)( g1 )
'' '' '' '' ' ' '' ''

Then

(a) If D ( x 0 , y 0 , λ ) < 0 then (x 0 , y 0) solves the maximization problem

(b) If D ( x 0 , y 0 , λ ) > 0then ( x 0 , y 0 ) solves the minimization problem

Multiple constraints: If we have a problem with more variables and constraints, the Lagrangian is
m
L ( x ) =f ( x )−∑ λ j [ g j ( x )−c j ]
j=1

The first order conditions for each i=1 , … , n are the following (along with the constraints)

∂L ∂f (x)
m
∂ g j( x )
= −∑ λ j =0
∂ xi ∂ xi j=1 ∂ xi

Page 30 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

8 LINEAR ALGEBRA
Carol (2008): Chapter 2 + Lecture 6

8.1 Vector and Matrix Algebra

Matrices are always written with bold. A m -by-n matrix has m rows and n columns. A matrix with
only one row is called a row vector, while a matrix with only one column is a column vector.

Matrix Addition and Multiplication by a Scalar: If A=( aij )m × n and B=( bij )m ×n are two matrices of

the same order, we define the sum of A and B as the m ×n matrix ( a ij +b ij ) m ×n:

A+ B= ( aij )m × n+ ( bij )m × n=( aij +b ij )m × n

If α is a real number, we define α A by

α A=α ( aij )m × n=( α aij )m ×n

Rules for matrix addition and multiplication by scalars: Let A , B, and C be arbitrary m ×n
matrixes, and let α and β be real numbers. Let 0 denote the m ×n matrix consisting only of zeros,
called the zero matrix. Then

(a) ( A+ B ) +C= A+( B+C)

(b) A+ B=B+ A
(c) A+0= A
(d) A+ (− A )=0
(e) (α + β) A=α A+ β A
(f) α ( A +B )=α A+ α B

Matrix Multiplication: If A=( aij )m × n and B=( bij )n × p, then the product C= AB is the m × p product

matrix C=( c ij )m × p whose element in the i th row and the j th column is the inner product.

n
c ij =∑ a ir b rj =ai 1 b1 j + ai 2 b 2 j +…+a ¿ bnj
r =1

Of the ith row of A and the j th columns of B.

Rules for matrix multiplication: If A , B, and C are matrices whose dimensions are such that the
specified multiplication operations are defined, and If α is an arbitrary scalar.

(a) ( AB ) C= A (BC )
(b) A ( B+c )= AB+ AC
(c) ( A+ B ) C=AC +BC
(d) ( α A ) B=A ( α B )=α ( AB)

Page 31 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

Note that AB≠ BA !

Example: Two matrices multiplied below – you multiply rows by columns in each element.

A=[ 1 0 2 0 ] B=[ 1 11 2 ]

AB=[ 1∗1+0∗1 1∗1+0∗2 2∗1+0∗1 2∗1+0∗2 ]= [ 1 12 2 ]

The Identity Matrix: The identity matrix of order n , denoted by I , is the n × n matrix having entries
1 along the main diagonal and 0 elsewhere, that is

I n=( 1 0 ⋯0 0 1 ⋯ 0 ⋮ ⋮ ⋱⋮ 0 0⋯ 1 )

The identity matrix has the properties that if A is a m ×n matrix, then A I n= A . If B is a n × m

matrix, then I n B=B . If A is a n × n matrix then A I n=I n A=A .

The Transpose: Consider the m ×n matrix A . We denote the transpose of A by A ' or AT , which is
defined as the n × m matrix whose first columns is the first row of A , and so on. Thus
'
A=( a11 a12 ⋯ a1 n a21 a22 ⋯ a2 n ⋮ ⋮ ⋱ ⋮ am 1 am 2 ⋯ amn ) ⟹ A =( a 11 a21 ⋯ a m1 a12 a22 ⋯ a m 2 ⋮ ⋮ ⋱ ⋮ a 1 n a 2 n ⋯ amn )

This means we have A =( aij ) =a ji.

' '

Rules for transposition: Given matrices A and B suitable for the following operations, and given
any scalar α :
'
( A' ) = A

( A+ B )' = A ' + B'

( α A )' =α A '

( AB )' =B ' A '

Example: A=[ 23 1 4 0 0 ] ⟹ A ' =[ 2 10 3 4 0 ]

Symmetric Matrices: A square matrix, say A=( aij )n × n, with the property that it is symmetric about
the main diagonal is called symmetric, e.g.,

A=( 2−1 5−1−32 5 2 8 )

A symmetric matrix is equal to its own transpose, i.e., A=A ' . Hence, the matrix A=( aij )n × n is

symmetric if a ij=a ji for all i , j.

8.1.1 Inverse of a Matrix

If we have a real number α ≠ 0 , then we have α −1 such that

Page 32 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

−1 −1
α α =α α =1

Here, α −1 is the inverse of α . Assume that A is a n × n matrix, then A−1 is the inverse if

−1 −1
A A =A A=I

If A has an inverse, it is called invertible. We see that A must be the inverse of A−1 and that A−1 and
A must be square matrices of the same order.

The inverse will exist if its determinant is different from zero, that is A−1 exists if | A|≠ 0 . A square
matrix is singular if | A|=0 and non-singular if | A|≠ 0 . This means that a matrix has an inverse if it
is non-singular. Further, a matrix can have only one inverse.

Properties of the Inverse: Let A and B be invertible n × n matrices. Then

−1
(a) A−1 is invertible and ( A−1 ) = A

(b) AB is invertible and ( AB )−1=B−1 A−1

−1
(c) The transpose A ' is invertible ( A' ) =( A−1)'

(d) ( cA )−1=c−1 A−1 whenever c is a number different from zero.

Inverse of a Matrix: 2x2 Provided that | A|=ad −bc ≠ 0 then for

A=( a b c d )

we have

−1 1
A = ( d −b−c a )
ad−bc

Solving System of Equations with the Inverse: Provided that | A|≠ 0 , one has the solution to Ax=B
given by x= A−1 b .

8.1.2 Positive and Negative Definite

A square and symmetric n × n matrix A is said to be positive definite if the quadratic form is positive
for all n ×1 vectors v ≠ 0
'
v Av >0

It is positive semi-definite if
'
v Av ≥ 0

The condition for negative definite is found by inverting the inequalities. This concept is important
for SOC for minimum/maximum with n variables. E.g., if the Hessain is negative (positive) definite,
the function is concave (convex), and a critical point will be a local or global maximum (minimum).

Page 33 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

8.1.3 Matrix Calculus

'
Assume that f (x)=[ f 1 ( x ) , … , f n ( x ) ] and let x=[ x 1 ,… , x k ] . Then
'

∂ xi ∂ xi[
∂ f ( x ) ∂ f 1 (x ) ∂ f n (x )
= ⋮
∂ xi ]
,i=1, … , k

And

∂x [
∂ f i ( x ) ∂ f i (x ) ∂ f i( x )
=
∂ x1
⋮
∂ xk ]
,i=1 , … , n

Combining the two results, we have

∂ x' [
∂ f (x ) ∂ f 1 ( x ) ∂ f n ( x )
=
∂x '
⋮
∂ x' ]
Some important results for linear and quadratic forms:
' ' '
∂x a ∂a x ∂ x Ax
=a =a ' =2 Ax
∂x ∂x ∂x

Example: Consider the linear regression model Y = Xβ+ ε . We want to minimise the loss function
' ' ' ' ' '
SR=( Y −X β ) ( Y − X β ) =Y Y −2 β X Y + β X X β

Taking the derivative with respect to β gives us the FOC

∂ SR ' '
=−2 X Y +2 X Xβ=0
∂β
−1
Thus, β=( X ' X ) X'Y

8.1.4 Vectors
A matrix with one row is a row vector, which is written as a=(a1 , … , an) . A matrix with only one
column is a column vector, which is written as

a=( a 1 a 2 ⋮ an )

The numbers a 1 , … , an are called components or coordinates.

Inner Product: of the pair of n -vectors a=(a1 , … , an) and b=(b1 ,… ,b n) is defined by

n
a∗b=a ' b=a1 b1 +a 2 b 2+ …+an b n=∑ ai bi
i=1

Length of Vectors: The length or norm of a vector a=(a1 , … , an) is denoted by ||a|| and defined

Page 34 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

||a||=√❑
Further, the Cauchy-Schwarz inequality states that |a∗b|≤||a||∗||b||

Orthogonality: Two vectors are orthogonal if the angel between them is 90 degrees. This can only be
the case if the inner product of the vectors are zero, i.e., a ⊥ b ⟹ a∗b=0. In economics, this
concept corresponds to the requirement that two variables are uncorrelated.

8.1.5 Lines and Planes

The line L in Rn through the two distinct points a=(a1 , … , an) and b=(b1 ,… ,b n) is the set of all
x=(x 1 , … , x n) satisfying x=ta+(1−t)b for some real number t .

Hyperplane in n -space: The hyperplane H in Rn through a=(a1 , … , an) which is orthogonal to the

nonzero vector p=( p1 , … , pn ) is the set of all points x=( x 1 , … , x n ) satisfying p∗( x−a ) =0.

8.2 Linear Equation Systems

A system of m equations in n unknown is written in the following way

a 11 x 1 +a 12 x 2+ …+a1 n x n=b1 a21 x 1 +a 22 x 2+ …+a2 n x n=b2 ⋮ ⋮ ⋮ am 1 x 1 +a m 2 x 2+ …+a mn x n=b m

Ax=b

where a 11 , … , amn are the coefficients and x 1 , … , x n are the unknowns. A solution (s1 , … , s n) solves
all the equations simultaneously. A system with a solution is called consistent and a system with no
solution is called inconsistent.

When we write the system Ax=b, A must be (m ×n), x must be (n ×1), and b must be (m ×1)

Gaussian Elimination: When using this method to solve systems, we are allowed to use elementary
row operators to change a system of linear equations. This includes (1) interchanging any pair of
rows, (2) multiply any row by a scalar, and (3) add any multiple of one row to a different row. Thus,
in order to solve a system of linear equations:

(1) Make a staircase with 1 as the coefficient for each non-zero leading entry
(2) Produce zeros above each leading entry
(3) The general solution is found by expressing the unknowns that occur as leading entries in
terms of those unknowns that do not. The latter unknowns, if any, can be chosen freely

The number of unknowns that can be chose freely is the number of degrees of freedom.

8.2.1 Determinants
Consider the system of two equations in two unknowns

Page 35 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

a 11 x 1 +a 12 x 2=b1

a 21 x 1+ a22 x 2=b 2

This can be written as Ax=B , with

A=¿

The system of equations has the general solution

b 1 a 22−b2 a12
x 1=
a11 a 22−a21 a12

b2 a11−b1 a21
x 2=
a11 a 22−a21 a12

The denominators of the two fractions are the same, this number is called the determinant of A .

| A|=|a11 a12 a21 a22|=a 11 a22−a21 a12

If the determinator is zero, | A|=0 , then the general solution formula is not valid!

Determinants and the Hessain Matrix: Recall the Hessain given as

H=( f 'xx' (x . y )f 'xy' (x . y )f 'yx' (x , y)f ''yy ( x , y ) )

The determinant equals

2
|H|=f 'xx' ( x , y ) f 'yy' ( x , y )−[ f ''xy ( x , y ) ]

Page 36 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

The second order condition for a maximum or minimum can be stated using the determinant for a
function of two variables.

Page 37 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

9 PROBABILITY
Carol (2008): Chapter 3 + Lecture 6
A random variable is a variable whose value can be observed today and in the past, but whose future
value is unknown. A real continuous random variable can take any value within a defined range of
real numbers. By contrast, a discrete random variable can only take discrete real numbers.

9.1 Laws of Probability

A phenomenon whose outcome is uncertain is an experiment. The sample space Ω is the set of all
outcomes considered possible. An event, E , is a set of outcomes, i.e., a subset of the sample space.
The probability function, P , maps an event to a probability. The following must hold true:

● If P is defined over a sample space, Ω , then P ( Ω )=1

● Let E be any event defined over Ω . Then 0 ≤ P ( E ) ≤ 1

Definition of Sets: Note the following definitions and properties:

● The complement of A , denoted by A , is the set of all outcomes not in A .

● The intersection of events A and B, denoted by A ∩ B, is the set of all points in both A and B

● The union of event A and B, denoted by A ∪ B, is the set of outcomes contained either in A ,

B or both.

● Two events that do no intersect are disjoint or mutually exclusive.

● If A is included in B, A ⊂ B is every outcome in A is also in B. It holds that P ( A ) ≤ P ( B )

● ∅ is the empty set P ( ∅ )=0

Law 1: The probability of an event A is a number between 0 and 1:

P ( A ) ∈[ 0 , 1]

Law 2: If A and B are two mutually exclusive events, then the probability of either A or B occurring
(the union of events) is the sum of their probabilities

P ( A∨B ) =P ( A ∪ B )=P ( A )+ P(B)

The event that A does not occur is the complement of A denoted A . This means that

Page 38 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

P ( A ) + P ( A )=1

Law 3: The joint probability of two events A and B occurring is

P ( A∧B ) =P ( A ∩ B )=P ( B ) P(B)

Here P ( B ) is the conditional probability of the event A occurring, given that the event B occurs. If
this is zero the events are mutually exclusive. This law also implies

P ( A ∩ B)
P ( B )=
P (B )

This law is also the basis of Bayes’ theorem, given below. See more on Carol (2008) p. 74.

P ( E ) P(E)
P ( F)=
P(F)

Two events are independent if P ( B )=P( A) or equivalently P ( A )=P(B). If two events are
independent, then the probability they both occur is given as

P ( A∧B ) =P ( A ∩ B )=P ( A ) P(B)

Law 4: If A and B are any two events, then the probability of either A or B occurring is the sum of
probabilities less the probability they both occur. That is,

P ( A∨B ) =P ( A )+ P ( B )−P ( A∧B )

P ( A ∪ B )=P ( A ) + P ( B ) −P ( A ∩ B )

9.2 Density and Distribution Functions

The set of all outcomes and their associated probabilities is a probability measure. This may be
represented using a cumulative distribution function (cdf), which gives the probability that X takes
a value less than or equal to x . The probability measure may also be represented in a probability
density function (pdf). Generally, we need to differentiate between discrete and continuous
variables.

9.2.1 Discrete Random Variables

Probability Density Function (pdf): Let X be a discrete random variable. The probability density
function provides the probabilities P( X=x) for all values of x . This is given as

f X ( x )={ pi =P (X =xi )if x=x i 0 otherwise

We have the two properties that f ( x ) ≥ 0 for all x and ∑ f ( x )=1.

Page 39 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

Cumulative Distribution Function (cdf): Let X be a discrete random variable. The cumulative
distribution function provides the probability that X takes on a value X ≤ x . This is given as:
❑ ❑
F x ( x ) =P (X ≤ x )= ∑ f ( x i ) =∑ p ( x i )
xi ≤ x x i ≤x

Note the following to calculate probabilities:

b
P ( a ≤ X ≤ B )=F X ( b )−F X ( a )=∑ p i
i =a

9.2.2 Continuous Random Variables

Probability Density Function (pdf): Let X be a continuous random variable. The probability
density function is used to describe the variable, which has the following properties
b
P ( a ≤ X ≤ a )=∫ f X (x ) dx
a

∫ f X ( x ) dx=1
−∞

Cumulative Distribution Function: We can define the cumulative distribution function for
continuous variables by replacing the sum with an indefinite integral of the density function
x
F X ( x )=P ( X ≤ x )= ∫ f X ( y ) dy
−∞

This implies that

'
F x ( x ) =f X (x)

9.3 Moments of the Probability Distribution

9.3.1 Expected Value

The first moment of the probability distribution of a random variable X is the expected value, which
represents the centre of location of the distribution. Formally, if X is discrete random variable with
density function f X ( X), then

∞ ∞
μ=E ( X )=∑ xP ( X =x)=∑ x f X ( x )
−∞ −∞

Further, let g( X) be a function of X . The expected value of the random variable g( X) is given by

∞
E ( g ( X ) ) =∑ g ( x ) f X (x )
−∞

Page 40 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

If X is a continuous random variable with density function f X ( X), then

∞
μ=E ( X )= ∫ x f X ( x)dx
−∞

Further, let g( X) be a continuous function of X . The expected value of the random variable g(Y ) is

∞
E ( g ( X ) ) =∫ g ( x ) f X (x)dx
−∞

For any two random variables X and Y and constants a and b , we have

E ( aX + bY )=aE ( X ) +bE(Y )

E ( aX + b )=aE ( X ) +b

The sample mean is the arithmetic average of observations in the sample given by
n
x=n−1 ∑ x i
i=1

Note that this is an unbiased estimate of the expected value of the distribution E ( X )=E ( X )=μ

9.3.2 Variance
The second moment of the probability distribution of a random variable X is variance, which is the
dispersion about the centre of the density. Formally, if X is a discrete random variable with density
function f X ( X) , then:
❑ ❑
Var ( X )=σ =E [ ( X−μ ) ]=∑ ( x−μ ) P( X=x )=∑ ( x−μ ) f X (x )
2 2 2 2

x x

If X is a continuous random variable with density function f X ( X), then

∞
Var ( X )=σ =E [ ( X−μ ) ]= ∫ ( x−μ ) f X (x )dy
2 2 2

−∞

There is a simple relationship between variance and expectation

Var ( X )=E ( X 2 )−E ( X )

2

Let X be any random variable and a and b are constants. The variance of a constant is 0. Then

Var ( b+ aX )=a 2 Var ( X )

Suppose that X and Y are random variables and a and b are constants. Then
2 2
Var ( aX +bY )=a Var ( X ) +b Var ( Y ) +2 ab Cov( X ,Y )

Page 41 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

where Cov (X , Y ) is the covariance. For n random variables we have

(∑ )
n n n
Var ai W i=∑ a Var (W i)+2 ∑ a i a j Cov(W i , W j)
2
i
i =1 i=1 i< j

The sample variance can be calculated using

n
s2=(n−1)−1 ∑ ( x i−x )
2

i=1

Note that this is an unbiased estimate of the population variance of the distribution E ( s2 ) =σ 2

9.3.3 Skewness and Kurtosis

The k th central moment is defined by

μk =E ( [ X−μ ] )
k

'k' is the order of the moment (e.g., 2nd central moment for variance, 3rd central moment for
skewness, 4th central moment for kurtosis).

Skewness is the third standardised central moment. Long tail on the left side is a negative skewness,
while a long tail on the right side is a positive skewness. This is given as:

E [ ( E−μ )3 ]
γ 1= 3
σ

Kurtosis is the fourth standardised central moment defined below. A leptokurtic distribution is
more peaked around its mean and display fatter tails than a normal distribution (kurtosis > 3), while a
platykurtic distribution is flatter at the mean and display thinner tails (kurtosis < 3).

E [ ( E−μ )4 ]
γ 2= 4
σ

9.4 Univariate Distribution

Binomial Distribution: A Bernoulli distributed random variable takes the value 1 (success) with
probability p, f ( 1 ) =p , and the value 0 (failure) with probability 1− p , f ( 0 )=1− p . The binomial

Page 42 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

distribution is the distribution of the number of successes in n independent Bernoulli trials each with
the same probability p. The binomial density function is given as (that is, the probability of
obtaining k successes in n Bernoulli trails)
k n−k
f X ( k ) =P ( X=k )=( n k ) p ( 1− p ) , k=0 ,1 , … , n

The distribution has the mean and variance as

E ( Y )= p Var ( Y )= p(1− p)
n
Since X Binomial(n , p) can be written as X =∑ Y i we have that
i=1

E ( Y )=np Var ( Y )=np (1− p)

Then we have the following for large n (c.f. the central limit theorem, CLT)

P( X ≤ k )≈ Φ ( k√−np
❑ )

Uniform Distribution: The standard uniform distribution is the distribution of a random variable X
that can take any value between 0 and 1 with the same probability. We can define a continuous
unform variable X U (a , b). The density function is

1
f X ( x )={ if x ∈[a , b]0 otherwise
b−a

The mean and variance is given by

E ( X )=
b +a ( b−a )2
Var ( X )=
2 12

In Excel, use RAND(). Below you see the uniform PDF (left) and uniform CDF (right)

Normal Distribution: A random variable X has a normal distribution if it has the density function:

1
f X ( x )=ϕ ( x )=
σ √❑

Page 43 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

The mean and variance is given by

2
E ( X )=μ Var ( X )=σ

The standard normal distribution has μ=0 and σ =1. Any random variable can be transformed into a
standard normal variable using the standard normal distribution:

X−μ
Z=
σ

Below you see the normal PDF (left) and normal CDF (right)

Chi-Square Distribution: Let X i N (0 , 1) be standard normal for all i=1 , … , n and independent,
then the sum below will be chi-squared distributed with n degrees of freedom
n
Q=∑ X 2i χ 2 (n)
i=1

The mean and variance is given by

E ( Q )=n Var ( Q )=2 n

2 2
If we have two independent chi-square distributions Q 1 χ (n) and Q 2 χ (k ) then

2
Q1 +Q2 χ (n+k )

Below you see the chi-squared PDF (left) and chi-squared CDF (right)

Page 44 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

t-Distribution: The t -distibutionn is a symmetric curve that converges to the standard normal density
as the degrees of freedom increases. Assume that X N (0 , 1) and Q χ 2 (n), then the following is t-
distributed with n degrees of freedom:

X √❑
T=
❑
The mean and variance is given by

n
E ( T )=0 , n>0 Var ( T ) = ,n> 2
n−2

Below you see the t-distribution PDF (left) and t-distribution CDF (right)

F-Distribution: Assume that Q 1 χ 2 (n1 ), and Q 2 χ 2 (n2 ) are independent. Then the following is F-

distributed with n1 degrees of freedom in the numerator and n2 degrees of freedom in the
denominator. It has support (0 , ∞ ) will be chi-squared distributed with n degrees of freedom :

Q1 /n1
F= F(n1 ,n 2)
Q2 /n2

The mean and variance is given by

2
n2 2n 2(n1 +n2−2)
E ( F )= ,n > 2 Var ( F ) = 2
,n> 2
n2−2 2 n 1 ( n 2−2 ) (n2 −4)

Below you see the F-distribution PDF (left) and F-distribution CDF (right)

Page 45 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

Lognormal Distribution: A random variable is lognormally distributed when its logarithm is

normally distributed. The lognormal density function for 0< y< ∞ is:

1
f ( y )=
y √❑

μ and σ above are the expectation and variance of the associated normal density function. The mean
and variance of Y is given by

(1
E ( Y )=exp exp μ+ σ 2
2 ) Var ( Y )=exp exp ( 2 μ+ σ ) ( exp exp ( σ )−1 )
2 2

9.5 Multivariate Distributions

The bivariate distribution includes the joint distribution, the marginal distribution and the
conditional distribution of a pair of continuous random variables.

Discrete random variables: Suppose that Ω is a discrete sample space with two random variables X
and Y . The joint probability function of X and Y is denoted by f X , Y ( x , y) and is defined by

f X , Y ( x , y )=P ( X ( ω )=x∧Y ( ω )= y ) =P ( X=x , Y = y )

If we know the joint probability function of the discrete random variables X and Y , we can obtain
the marginal probability function as

Page 46 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

❑
f X ( x )=∑ F X , Y ( x , y)
y

❑
f Y ( y )=∑ F X ,Y (x , y )
x

If X and Y are discrete random variables, then

❑ ❑
E [ g( x , y) ] =∑ ∑ g ( x , y ) f X ,Y (x , y )
x y

Continuous random variables: The joint probability that X takes values in one range and Y takes
values in another range simultaneously is defined as the area under the surface defined by the ranges:
yb xb

P ( x a < X < x b , y a <Y < y b ) =∫ ∫ f (x , y)dxdy

ya xa

Note that any function f X , Y with f X , Y ( x , y ) ≥ 0 for all x and y , and where the following equation is
fulfilled, qualifies as a joint pdf.
∞ ∞

∫ ∫ f X ,Y ( x , y ) dxdy=1
−∞ −∞

The joint cumulative distribution function of two random variables X and Y is denoted F X ,Y (u , v)
:
y x
F X ,Y ( x , y )=P ( X < x , Y < y )= ∫ ∫ f X ,Y ( x , y ) dxdy
−∞ −∞

Note that
2
∂ F ( x , y)
f ( x , y )=
∂x∂ y

If X and Y are continuous random variables, then

∞ ∞
E [ g( x , y) ] = ∫ ∫ g ( x , y ) f X ,Y (x , y )dxdy
−∞ −∞

The marginal distributions can be derived from the join distribution as

H ( x )=F ( x , ∞ ) , G ( y )=F (∞ , y )

Marginal densities if they exist are given by

∞
f X ( x )=∫ f X ,Y (x , y ) dy
−∞

Page 47 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

The conditional distribution is given as

∂ F (x , y ) ∂F (x, y)
F ( y )= ∧F ( x )=
∂x ∂y

The conditional densities are given as

f (x , y) f (x , y)
f ( x )= ∧f ( y )=
h (x ) g( y)

Independent Random Variables: Two random variables are independent if their joint distribution
function is the product of the marginal distribution of X and the marginal distribution of Y . That is

F ( x , y )=H ( x ) G( y )

It follows from the density function that they are independent if

f ( x , y )=h( x)g ( y )

Covariance: Given any two random variables X and Y , define the covariance of X and Y as

Cov ( X , Y ) =E [ ( X −μ x ) ( Y −μ y ) ]=E ( XY )−E ( X ) E ( Y )

Correlation: The formal definition of correlation is

Cov ( X , Y )
Corr ( X ,Y )=
√❑
2 2
If we have two independent normal distributions X N (μ1 , σ 1 ) and X N (μ2 , σ 2 ), then

aX + bY N ( a μ 1+ b μ 2 , a 2 σ 21+ b2 σ 22 )

Page 48 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

10 STATISTICS
Carol (2008): Chapter 3 + Lecture 7

10.1 Central Limit Theorem and Law of Large Numbers

Law of Large Numbers (LLN): States that when sample size tends to infinity, the sample mean will
N
be equal to the population mean. Suppose you have a sample { x i }i=1 following any type of i.i.d.

function f (μ , σ ) (that is, following the same pdf). Define that sample mean

n
1
X n= ∑ X i
n i=1

Then can show that

2
E ( X n )=μ σ
Var ( X n ) =
n

This can be proved via (variance)

( )
N N N
1
E ( X n )=E ∑ x = 1 ∑ E(x i )= N1 ∑ μ
N i=1 i N i=1 i=1

( )
N N
1 1 1 σ2
Var ( X n ) =Var ∑ x = ∑
N i=1 i N 2 i=1
Var (x i )=
N2
N σ 2
=
N

The Law of Large Numbers (LLN) states that

X n=μ

P (| X n−μ|> ε ) =0

for all ε > 0. This is called convergence in probability.

Central Limit Theorem (CLT): States that when sample size tends to infinity, the sample mean will
be normally distributed. For a random sample of size n and if σ < ∞

P ( X n −μ
σ / √❑ )
where Φ (z) denotes the standard normal cdf. This applies to the Binomial distribution, see earlier.

Jensen’s Inequality: Suppose that f is a convex function, then it follows that

f ( E ( X )) ≤ E ( f ( X ))

Page 49 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

10.2 Estimation
We have a normally distribution random variable X N (μ , σ 2 ), where we know that E( X )=μ

and Var ( X )=σ 2. In the real world, we observe a realisation of random sample from the underlying
distribution, thus we do not know the exact values of μ and σ 2. We need to estimate the parameters,
which can be done using Maximum Likelihood of Method of Moments.

Important estimators: The sample covariance is

n
1
C X , Y= ∑ ( X −X )(Y i−Y )
n i=1 i

Estimate versus Estimator: An estimate is the value we obtain for a given sample, while an
n
1
estimator is a random variable. The sample mean given a realisation of the random sample ∑ x i is
n i=1
n
1
an estimate, while the estimator of the sample mean ∑ X is a random variable.
n i=1 i

10.2.1 Maximum Likelihood Estimation

Let x i ,… , x n be the realisation of a random sample of size n from the probability function (discrete)
or the probability density function f X ( x ; θ) (continuous) where θ is an unknown parameter. The
likelihood function, L(θ), is the product of f X (x ; θ) evaluated at n x i’s:

n
L ( θ )=∏ f X ( xi ; θ )
i=1

Let θ^ ML be the value that maximises the likelihood function L(θ), then it is called the maximum
likelihood estimate for θ . We often maximise the log-likelihood function:
n
log log L ( θ )=∑ log log f X ( x i ; θ)
i =1

Example: MLE of p in a Bernoulli Distribution: Assume we observe a realisation of the random

sample X 1 , … , X n where X i Bernoulli ( p), i=1 , … , n. The likelihood function is
n

n n
n ∑ (1−xi )
∑ x i (1− p) i=1

L ( p )=∏ f X (x i ; p)=∏ p (1−p )

xi 1−x i
=p i=1

i=1 i =1

To make it easier to maximise, we take the logarithm

n n
log log L( p)=log log ( p) ∑ x i +log log (1− p) ∑ (1−x i)
i=1 i=1

Page 50 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

The first-order condition is (first derivative with respect to p)

[ ]
n n
1
∑ x − 1 n−∑ x i=0
p i=1 i 1− p i=1

The solution, pe , is the maximum likelihood estimate

n
1
pe = ∑ x i
n i=1

This is the sample mean. The likelihood estimate could also be obtained numerically using Excel.

10.2.2 Methods of Moments

Suppose that Y is a continuous random variable where the pdf is a function of s unknown parameters
θ1 , … ,θ s . The first s moments, if they exist, can be calculated using
∞
E ( Y )= ∫ y f Y ( y ; θ1 , … , θs )dy , j=1 ,2 , … , s
j j

−∞

Generally, each E ( Y j ) will be a function of the s parameters

E ( Y j )=g j ( θ 1 , … , θ s )
n
1
This corresponds to each theoretical moment E ( Y j ) there is a sample moment ∑ y j. Setting
n i=1 i
these equal for all j gives us a system of s equations, which can be solves simultaneously to obtain
the parameter estimates:
∞ n ∞ n
1
∫ y f Y ( y ; θ1 ,… ,θ s ) dy= ∑ y ⋮ =⋮ ∫ y s f Y ( y ; θ 1 , … , θ s) dy= 1n ∑ y is
n i=1 i
−∞ −∞ i=1

Example 1: Method of Moments: Consider the normal distribution

1
f Y ( y ; μ , σ )=
2
√❑

It is possible to show that E ( Y )=μ and Var ( Y )=σ 2, which implies that

2
E ( Y ) =Var ( Y )+ [ E ( Y ) ] =σ + μ
2 2 2

Thus, we have the two moment conditions

n
1
μ= ∑y
n i=1 i

Page 51 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

n n
1 1
σ + μ = ∑ y 2i ⟹ σ 2= ∑ y 2i −μ 2
2 2
n i=1 n i=1

Example 2: Method of Moments: Consider the gamma distribution

r
λ r −1 λy
f Y ( y ; r , λ )= y e , y ≥0
Γ (r )

It is possible to show that E ( Y )=r / λ and Var ( Y )=r / λ2 , which implies that

()
r (r+ 1) 2
2 r r
E ( Y ) =Var ( Y )+ [ E ( Y ) ] = 2 +
2
= 2
λ λ λ

Thus, we have the two moment conditions

n
r 1
= ∑y
λ n i=1 i
n
r (r +1) 1
2
= ∑ y 2i
λ n i=1

The solution to this system of equations is

n

∑ yi
λ e = i=1
❑
And

λe n
re= ∑y
n i =1 i

10.3 Properties of Estimators

Estimators have three important properties, i.e., unbiasedness, efficiency and consistency, which the
last one being most important.

Unbiasedness: While estimators take different values depending on the sample, we want to on
average get the correct estimate. Suppose that Y 1 , … ,Y n is a random sample from the pdf f Y ( y ; θ),
^
where θ is an unknown parameter. Then an estimator is a function of the sample θ=h(Y 1 , … , Y n) .

The estimator is unbiased if the expected value is the actual value:

E ( θ^ ) =θ

The sample mean is an unbiased estimator of the expected value of the underlying distribution, since

Page 52 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

n
1
E ( Y )= ∑ E ( Y i )=E (Y )
n i=1

An unbiased estimator of sample variance is

n
1
S2 = ∑
n−1 i=1
( Y i−Y )
2

Efficiency: If we have two unbiased estimators for a parameter θ , θ^ 1 and θ^ 2 , then if

Var ( θ^ 1 ) < Var ( θ^ 2 )

we say that θ^ 1 is more efficient that θ^ 2. It will generally hold that if we have a random sample
Y 1 , … ,Y n from the pdf f Y ( y ; θ), where θ is an unknown parameter and the estimator is
θ^ n =h ( Y 1 ,… ,Y n ) , then

[ [( ) ]]
−1
2
∂ ln ln f Y ( Y ; θ )
Var ( θ^ n) ≥ nE =¿ ¿
∂θ

which is known as Cramér-Rao’s lower bound.

n
1
The maximum likelihood estimator for p in the Bernoulli distribution was ^p= ∑ Y i with E [ ^p ] = p
n i=1
and Var [ ^p ] =p (1− p)/n. We have that f Y ( Y i ; p )= p (1− p )
Yi 1−Y i
, which implies that

[[ ]]
2
∂ ln ln f Y ( Y ; θ ) −1 p (1−p )
E = ⟹ Var ( ^p ) ≥
∂θ p ( 1− p ) n

Consistency: An estimator θ^ n =h(Y 1 , … ,Y n) is said to be consistent for θ if it converges in

probability to θ , that is for all ε > 0

P(|θ^ n−θ|< ε )=1

For consistency, it is sufficient to show that

E( θ^ n)=θ

Var ( θ^ n)=0

We had that

n−1 2 2
E [ σ^ ] =
2
σ ≠σ
n

Page 53 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

so it is biased, but when n → ∞ it will converge to the true value – it is asymptotically unbiased. The
sample mean is consistent because it is unbiased and its variance converges to zero:
n
1
X = ∑ Xi
n i=1

10.4 Hypothesis Testing

We have two competing hypotheses, H 0 as the null hypothesis and H A as the alternative
hypothesis. It is assumed that H 0 is accepted unless the data argue overwhelmingly to the contrary.

When testing a two-sided hypothesis, we have

H 0 : μ=μ 0 , H A : μ ≠ μ 0

When testing a one-sided alternative, we have

H 0 : μ=μ 0 , H A : μ> μ0

10.4.1 Normal, Known Variance

Assume we have n realisations x 1 , … , x n from an independent and identically distributed (i.i.d.)

random sample X 1 , … . , X n. Assume that X i N ( μ , σ 2 ) ,i=1, … , n

n
1
The estimate ^μ= ∑ x is a realisation of the random variable.
n i=1 i
n
1
X= ∑ X N ( μ , σ 2 /n)
n i=1 i

The Z-score that follows a standard normal distribution under the null hypothesis is defined

X−μ0
Z=
√❑

A very small or very large value of z is seen as evidence against the null hypothesis.

The p-value is the probability that a realisation from the standard normal distribution will be more
extreme than the realised Z -score given that the null hypothesis is true. With a two-sided test, this is

p−value=P ( H 0 ) + P ( H 0 )=2 ( 1−Φ (|z|) )

where Φ is the cdf for the standard normal distribution. If we have a one-sided test, only large
positive values of the z -score will be evidence against the null, hence we define the p-value as
1−Φ (z ).

Page 54 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

Rejecting the Null Hypothesis: We define the significance level, α , which is the probability that we
will reject the null hypothesis even if it is true (usually 5%):

● Reject the null hypothesis H 0 if p-value ¿ α

● Fail to reject the null hypothesis H 0 if p-value ≥ α

We can also use the critical value, which is defined as

α =2 ( 1−Φ ( c ) ) ⟹ c=z 1−α /2=Φ (1−α /2)

−1

It is the 1−α /2 percentile of the standard normal distribution, i.e., Φ ( z 1−α /2 )=1−α /2. Thur

● Reject the null hypothesis H 0 if |z|> c

● Fail to reject the null hypothesis H 0 if |z|≤ c

10.4.2 Normal, Unknown Variance

If variance σ 2 is unknown, we must estimate it by using a consistent estimator such as:

n
1
S2 = ∑
n−1 i=1
( X i−X )
2

The test statistic follows a t -distribution under the null hypothesis, i.e.:

X−μ 0
t=
√❑

If we estimate the variance, we follow the same steps as with the case with known variance, only
using a different distribution!

10.4.3 Unknown distribution or non-normal data

Unknown distribution, n is large: If we do not know the underlying distribution, but the sample size
n is large, we can use the central limit theorem. For n large enough then the expression below holds:

X−μ 0
√❑

Now, we can use the same procedure as before when dealing with non-normal data.

Non-normal data, n is small: If the data cannot be assumed to be normally distributed and the
sample size n is too small to use the CLT approximation, we must use an alternative approach. If we
know that the data follows a certain distribution, then we can simply use it.

Page 55 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

10.4.4 Confidence Intervals

The confidence limits of an estimator define an interval. A 100 × ( 1−α ) % confidence interval for X
is the interval where, given H 0 is true, that 100 × ( 1−α ) % of the realisations of X will be in if we
sampled repeatedly. Thus,

(
1−α=P −z 1−α / 2<
X −μ0
√❑ )
With 100 × ( 1−α ) % confidence we say that μ0 is in the interval

μ0 ∈ (−z 1−α /2 √ ❑ )

10.4.5 Type I and Type II Error

Type I Error: Reject H 0, when H 0 is true (false negative) – determined by the significance level α .

Type II Error: Fail to reject H 0, when H 0 is false (false positive) – tricky to calculate (beta error)

The problem with calculating the type II error is that it depends on the unknown true value of μ. If we
assume the true value is μtrue , then we can calculate the probability of committing a type II error as:

X−μ 0
√❑

X−μ 0
√❑

P(Type II error∨μ=μtrue )=P ( μ + z❑ √❑ )

0 α/ 2

We can write P ( Type II error )=β . Then 1−β will denote the power of the test – the probability of
rejecting H 0 when H A is true. There is a relationship between the size of the test α and the power
1−β – if we increase α , then we will increase the power of the test.

Page 56 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

11 LINEAR REGRESSION
Carol (2008): Chapter 4 + Lecture 7
To conduct linear regression in Excel, there are three options: (1) use matrix algebra found in section
11.3, (2) use the linest() function (also includes standard errors etc.), or (3) use the regression
functionality under Data Analysis toolpack. Always check results in two ways!

11.1 Simple Linear Regression

The simple linear regression model is written as

Y ( x )=g ( x )+ ε=α + βx +ε

Y ( x )=Y ∨x is a random variable called the dependent (response) variable, which is a realisation of
random variable X called the explanatory (predictor) variable. Assume the error term, ε satisfies:

E ( ε )=0∧Var ( ε )=σ 2
2
This means ε i i. i . d .(0 ,σ ). We also have that E ( x )=g( x ) and Var ( x )=σ 2 .

The objective of the linear regression is estimating the regression function g(x ) using data. Therefore,
we need to estimate the intercept α and the slope β . We assume that the error term are realisations
from an independent random variable, then the realisations y i will also be outcomes of independent
random variables Y i. Thus, we have the model

Y i=α + β X i +ε i

Estimating parameters: To estimate the parameters, we need to define a measure of goodness of fit
chosen as the sum of squared errors. The estimates are then found as:
n
SSE ( α , β )=∑ ( y i−( α + β x i ) )
2

i=1

( α^ , ^β ) =arg arg SSE ( α , β )

Recall that the FOC for a minimum is defined by ∂ SSE/∂ α =0 and ∂ SSE/∂ β=0 . The estimates are

α^ =Y − ^β X
n

∑ ( X i−X )(Y i −Y )
^β= i=1
❑

^ )=α and E ( ^β )= β.
It is possible to show that the estimators are unbiased, i.e., E ( α

Variance of Estimators: The variance of the estimators conditional on x are

Page 57 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

( )
σ
2
1 x2
Var ( x )= Var ( x )=σ 2 + n
n
n
∑ ( x i−x )2 ∑ ( x i−x )2
i=1 i=1

If we do not condition on x we have

[ ] ( [ ])
2
1 1 X
Var ( ^β )=σ 2 E n Var ( α^ )=σ +E
2
n
n
∑ ( X i−X ) 2 ∑ ( X i −X )2
i=1 i=1

However, we rarely know σ 2, so it needs to be estimated. A consistent estimator is:

2 n 2 SSR
s= σ^ =
n−k−1 n−k −1

The square root of the expression above is the standard error of the regression.

Diagnostics: The total variation of the data can be measure by the total sum of squares (SST)
(proportional to variance):
n
SST =∑ ( y i− y )
2

i=1

The unexplained variation is the sum of squared residuals, when ^y i is the fitted value (n-k-1)?

n n
SSR=∑ ( y i−^y i ) =∑ ei=Var ( e )∗(n−k −1)
2

i=1 i=1

Then R-squared, R2, measures the proportion of the variation in the data that we can explain given:

2 SSE SSR
R= =1−
SSR SST

11.1.1 Inference in Simple Linear Regressions

Assume that we want to test the (two-sided) hypothesis

H 0 : β=β 0 , H A : β ≠ β0

Large n: This can easily be done by invoking the central limit theorem (CLT) (n must be large)

( )
^β∨x a σ2
N β, n

∑ ( x i−x )2
i=1

and therefore under H 0, we have the following:

( x ) a N (0 , 1)

Page 58 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

We can replace σ 2 with the estimator s2 since this is only an asymptotic result.

Small Sample Inference: Assuming the errors are normally distributed, we can make inference on
small samples as well, i.e., ε i N ( 0 , σ 2 ) , i=1 , … ,n , implying that

Y i∨x N ( α + β x i ,σ 2 ) ,i=1, … , n

^ and ^β are just linear combination of the Y i’s the

Since α

( ( )
2
2 1 x
α^ ∨x N α ,σ + n
n
∑ ( x i−x )2
i=1

( )
^β∨x N β , σ2
n

∑ ( x i−x )2
i=1

If we replace σ 2 with s2, then we obtain that under the null

( x ) t (n−2)

In the book, the t -test for coefficient parameters are

^β−β
0
t= t n−2
est . s . e . ( ^β )

where

s
est . s . e . ( ^β ) =
s X √❑
n

s=
2SSR ∑ ( x i−x )2
t−2 s2X = i=1
❑

11.2 Properties of OLS Estimators

The model coefficient β is not a random variable, but the estimator ^β is a random variable, because
different data give different estimates. This means the OLS estimators have sampling distributions.

There are two different types of random variables present in the context of regression:

● The error process, which is assumed to have a theoretical distribution (i.i.d. and normal)

● The coefficient estimators, which are random variables due to data sources.

Page 59 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

The properties of the estimator distributions follow from the error distribution. If the error process is
normal and i.i.d., then the OLS estimators will have normal distributions.

Unbiasedness: Means that the expected value of the estimator equals to true value of the parameter.

E [ ^β ]=β

Efficiency: Means that the variance of the estimator is as small as possible. That is, if we have

Var ( ^β 1 ) <Var ( β^ 2 )

we say that ^β 1 is more efficient that ^β 2.

Consistency: Means that the distribution of the estimator converges to the true value of the parameter
as the sample size tends to infinity. This means that we need to have:

E( θ^ n)=θ

Var ( θ^ n)=0

OLS estimators will always be consistent provided that the error is stationary.

Gauss-Markov Theorem: If the residuals are i.i.d. then the OLS estimators are the best linear
unbiased estimators (BLUE). This means…

● ‘Best’ because OLS estimators are more efficient than other linear unbiased estimators

● ‘Linear’ because the OLS formula is a linear function of the data

● ‘Unbiased’ because the expectation of the estimator is the true value of the coefficient

● ‘Estimator’ because it is a method for estimating parameters.

11.3 Multivariate Linear Regression

We will often have multiple explanatory variables giving the model

Y i=α + β 1 X 1 i+ β 2 X 2 i +…+ β k X ki + ε i

where X ji is either fixed or a random variable X ji for i=1 , … , m, j=1 ,… , k . In matrix form:

Y = Xβ+ ε

where ε t i . i. d . ( 0 , σ 2 ) and X is a (n × k ) matrix, X is a (n × k ) matrix, β is a (n ×1 ) matrix, Y is a (

n ×1) matrix, and ε is a (n ×1 ) matrix given as:

Page 60 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

X =( 1 X 11 ⋯ X k 1 1 X 12 ⋯ X k 2 ⋮ ⋮ ⋱ ⋮ 1 X 1 n ⋯ X kn ) , β=( α β 1 ⋮ β n) , Y =( Y 1 Y 2 ⋮ Y n ) , ε =( ε 1 ε 2 ⋮ ε n)

We assume that

E [ ε ∨X ] =0 ⟹ E [ ε ' X ]=0 Var [ X ] =σ I n

2
and

The OLS beta estimator is given as

^β=( X ' X )−1 X ' Y

X ' X must be non-singular (to be invertible, non-zero determinant), implying we cannot have two
perfectly correlated variables in the same model. The covariance matrix of the OLS estimator:
−1
Var ( X )=σ ( X X )
2 '

However, we usually do not know σ 2, so we use the unbiased estimator of σ 2

2 SSR
s=
n−k−1

This gives us the following covariance matrix of the OLS estimator as a ¿)-matrix with the estimated
variances of the estimators along the diagonal and the covariances in the off-diagonal elements:
−1
Var ( ^β )=s ( X X )
2 '

The total sum of squares (SST) is the sum of squared mean deviations of the dependent variable:
' 2
SST = y y−T Y

The residual sum of squares (SSR) is the sum of squared residuals:

SSR=e e= y y− β^ X ' y
' ' '

The explained sum of squares (SSE) is the amount of variation in Y captured by the model:

SSE=SST −SSR= β^ X ' y−T Y

' 2

The regression R-squared ( R2) is the proportion of variance the model an explain

2 SSE SSR
R= =1−
SST SST

Distribution of Estimator: Under the central limit theorem (CLT) we have that
−1
^β∨X a
N( β,σ (X X) )
2 '

11.3.1 Inference in Multivariate Linear Regressions

Testing Joint Significance of Parameters: In practise, if we want to test whether q of the
coefficients are equal to zero, we use

Page 61 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

(SS Rr −SS R ur )/q

F= ∨X F q ,n−k−1
SS Rur /(n−k−1)

where q is the number of restrictions. Thus, a large difference in the sum of squared residuals between
the restricted and unrestricted model is evidence against the null.

In matrix notation, under the null hypothesis H 0 : Rβ=r , where R is a q ×(k +1) matrix and r is (
q × 1) vector, the following holds true:
−1
R ^β−r ∨X N (0 , σ R ( X X ) R ')
a 2 '

See p. 166 in Carol (2008) for examples on how to set up Rβ=r . This implies that

1 −1
^ )' ( R ( X ' X )−1 R ' ) (R ^β−r )∨ X χ 2(q)
2
( R β−r
σ

Assuming errors are normal, these results hold in small samples. Assuming data is normal, we have

(Y −X ^β)'(Y −X ^β)
2
∨x χ 2 (n−k−1)
σ

Which finally gives us:

−1
( R ^β−r ) ( R ( X ' X ) R' ) (R β−r)/q
−1
'
^
∨X F (q , n−k−1)
(Y −X ^β)'(Y −X ^β)/(n−k−1)

See also problem set 6 exercise 5. Here, the following is used instead
−1
( R ^β−r ) ( R ( X ' X ) R' ) (R β−r)
−1
'
^
2
∨ X F (q ,n−k −1)
s ×q

Testing Individual Parameters: If we wish to test H 0 : β=β 0, the t -statistic is

^
β−β 0
t= t n−k−1
est . s . e .( β^ )

Remember the estimated standard errors of the estimators are found by taking the square root of the
diagonal element of the variance covariance matrix of the estimator!

Page 62 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

12 MONTE CARLO METHODS

Carol (2008): Chapter 5 + Lecture 11
All Monte Carlo methods involve simulations. It allows us to solve problems numerically. The
general pattern is:

● Define a domain of possible inputs

● Generate inputs randomly from a probability distribution over the domain

● Perform a deterministic computation on the inputs

● Aggerate results

12.1 Simulating Random Values via the Quantile Function

In Excel, random numbers can be generated using the Analysis Toolpak add-in. A more general
procedure is to use the Rand () function or the Rnd function in VBA that will give us a realization
from an uniform distribution.

Note we can generate random variables for other distributions by using the uniform random number
generator. For example, if U ∼U (0 ,1) then Φ−1 (U )∼ N (0 ,1) where Φ−1 denotes the quantile
function of a standard Normal random variable (Φ denotes the CDF of a standard normal RV)

We can draw numbers form a normal distribution with any mean and variance using
2
a+ bZ ∼ N (a , b )

We can also draw from a χ 2 distribution, since

2 2
X =Z ∼ χ (1)

We remember that the cumulative probability function of a continuous random variable X is written:
∞
P ( X ≤ x )=F X ( x )= ∫ f X (x )dx
−∞

Define the random variable

Y =F X (X )

Y can be shown to be uniformly distributed, Y ∼U (0 , 1). So, it is possible to use the inverse
distribution function (the quantile function) F−1
X to obtain a realisation of any continuous random

variable for which we know the inverse distribution function

Page 63 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

−1
F X ( X ) ∼ U ( 0 ,1 ) ⟹ F X (U)∼ S

12.2 Monte Carlo for Evaluating Integrals

We can use Monte Carlo methods to evaluate an integral. The basic idea is that
b b
1
∫ f (x )dx =( b−a )∫ f ( x ) b−a
dx
a a

such that
b

∫ f (x )dx =( b−a ) E [ f ( X ) ]
a

when X is assumed to be uniformly distributed on the interval [a , b]. If draw N random numbers x i,
i=1 , ..., N from X ∼U (a , b), we can approximate the integral of f (x) over the interval [a , b] by
b N

∫ f (x )dx ≈ b−a ∑ f (x i)
N i=1
a

The expected value of the Monte Carlo estimator is

N b
b−a
E[^
MC N ]= ∑ E [ f ( X i ) ]=( b−a ) E [ f ( X ) ]=∫ f (x)dx
N i=1 a

The variance of the Monte Carlo estimator is

( b−a )2 N ( b−a )2
Var [ ^
MC N ]= ∑
N 2 i=1
Var [ i ] N Var [ f ( X ) ]
f ( X ) =

which we can estimate with the sample variance.

Example: Monte Carlo for Evaluating Integrals: Consider the integral

1

∫ e x dx
0

The analytical solution is

1
1
∫ e x dx= [ e x ]0=e 1−e 0=e−1 ≈ 1.71828182845905
0

We can define the Monte Carlo estimator as

1 N

∫ e dx ≈ N1 ∑ e U
x i

0 i=1

with U i ∼U (0 , 1)

Page 64 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

Example: Monte Carlo for Evaluating Integrals: Consider the integral

2

∫ e x dx
0

The analytical solution is

2
2
∫ e x dx= [ e x ]0=e 2−e 0=e−1 ≈ 6.38905609893065
0

We can define the Monte Carlo estimator as

2 N

∫ e dx ≈ N2 ∑ e U
x i

0 i=1

with U i ∼U (0 , 2)

12.3 Simulating Stock Prices

In Finance, it is often assumed that the stock price follows a Geometric Brownian Motion
(stochastic differential equation), given as:

d S t =μ S t dt + σ S t d Z t

which implies that

(( ) )
2
σ
St =St −Δ t exp exp μ− Δ t +σ √❑
2

where Z t ∼ N (0 , 1). We say the stock prices follows a log-normal distribution and that log returns
follow a normal distribution

( )
2
σ
ln ln St =ln ln S t− Δt + μ− Δ t +σ √ ❑
2

and

(( ) )
2
σ 2
ln ln St −ln ln S t− Δt ∼ N μ− Δt ,σ Δt
2

Assume we want to simulate stock returns that are correlated and that returns are normally distributed
Then, we can simply generate two random numbers from a standard normal distribution Z1 and Z 2. Z1
and Z3 will be correlated with correlation coefficient ρ , if we define

Z3 =ρ Z 1 + √ ❑

Page 65 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

We can just add the expected value and multiply with the standard deviation to get the desired mean
and variance of the normal distributions

If we want to simulate multiple normal random variables with a given covariance matrix, then we can
easily do it by realising that we can write the covariance matrix, Σ , as (Cholesky decomposition)

Σ=BB '

Then,

Y =μ+ BZ ∼ N (μ , Σ)

where Z is a vector of independent normally distributed random variables

12.4 Option Pricing

We know that the Black-Scholes-Merton pricing formula for an European (can only be exercised at
maturity) call option is given by
−r ( T −t )
C ( S t , K , t ,T ,r , σ )=St × N ( d 1 )−K e × N (d 2)

1
N ( d )=
√❑

St
( )( )
2
σ
ln ln + r + ( T −t )
K 2
d 1=
σ √❑

( )(
St
)
2
σ
ln ln + r− ( T −t )
K 2
d 2=
σ √❑

where St is the price/level of the underlying at time t , σ is volatility, K is the strike price, T is the
maturity of the option, and r is the constant riskless short rate

Assume that we want to price the option using Monte Carlo simulations We apply the principle of
risk-neutral pricing, by…

● Simulate the model under the assumption that the expected return is equal to the risk-free rate

● Calculate the pay-off at maturity

● Discount the value at maturity to get the present value

This implies that we must set μ−σ 2 /2=r −σ 2 /2 in the log-normal distribution for the stock price,
since

Page 66 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

(( ))
2 2
σ σ
E ( ST )=S0 exp exp μ− + T =S 0 exp exp ( μT )
2 2

12.5 Assessing the Error

In most Monte Carlo experiments, we are looking at sums of independent and identically distributed
random variables. If we have a X 1 , … , X K independent and identically distributed with

E ( X i) =μ

2
Var ( X i ) =σ
K
1
for i=1 , ..., K , then the sample mean X K = ∑X
K i=1 i

√❑
❑
Assume that we want to ensure that the interval

(X K −a
σ
√❑ )
contains the expected value μ with probability 1−α . In the limit when K → ∞, we obtain this by
solving
−1
Φ ( a )−Φ (−a )=1−α ⟹ a=Φ (1−α /2)=z1−α /2

Therefore, the interval is

σ
X K ± z1−α /2
√❑

If we do not know σ , then we must replace it with a consistent estimator

sk =√ ❑

We see that the the error in the Monte Carlo experiment converges to zero at the rate of √ ❑. Thus, to
reduce the error to half its size, we need to increase K roughly 4 times

We can compute backwards to get the numbers of needed Monte Carlo simulations for a given error ε
(we still have α probability for a larger error). We can solve for K in

SK
z 1−α/ 2
√❑

and therefore

Page 67 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

( )
2
SK
K > z 1−α / 2
ε

12.6 Variance Reduction

An important research area focuses on how to make Monte Carlo techniques more efficient, i.e.,
reduce the variance of the Monte Carlo estimator

One technique is to use antithetic variates, which basically doubles the number of random draws by
adding the initial draws with opposite sign.

The idea is that if we want to estimate E( X ) using the sample mean. Assume that that we have two
samples with two unbiased estimators X 1 and X 2 , then the estimator

X 1+ X 2
X=
2

is unbiased and has the variance

Var ( X 1 ) +Var ( X 2) + 2Cov (X 1 , X 2 )

Var ( X )=
4

Other methods are stratification, control variates and importance sampling, among others

Assume that we want to use Monte Carlo methods to calculate the expectation E [X ] where
X ∼ N (5 , 4 ). The simple Monte Carlo approach would be to use
N
1
E [ X ] ≈ ∑ Xi
N i=1

An alternative approach is to use

N N
1 1
E[X]≈ ∑ X +1 1 ∑ X
2 N i=1 1 i 2 N i=1 2 i

where X i 1=5+ 2 Z i and X i 2=5−2 Z i with Zi ∼ N (0 , 1)

We want to calculate the integral

1

∫ 1+1 x dx=ln ln 2
0

using Monte Carlo integration. Again, the simple estimator is

1 N

∫ 1+1 x dx ≈ N1 ∑ 1+U
1
0 i=1 i

and, the more efficient estimator is

Page 68 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

1 N N

∫ 1+1 x dx ≈ 12 N1 ∑ 1+U
1
+
1 1
∑ 1
0 i=1 1 i 2 N i=1 1+U 2 i

where U i 1=U i and U i 2=1−U i with U i ∼ U (0 , 1). See Excel.

12.6.1 Importance sampling

Suppose that we want to calculate
❑ ❑
p(x)
∫❑ h ( x ) p (x)dx=∫❑ h ( x ) q ( x ) q ( x ) dx
If it is hard to sample the random variable with density p(x ), then we can sample the random variable
with density q (x) and estimate the integral as

❑
1
N
p ( Xi)
∫ h ( x ) p (x)dx= ∑
N i=1
h( Xi )
q ( Xi)
❑

This may also be relevant even if we can sample the random variable with density p(x ).

Suppose that we want to calculate the probability P( X> 5) for X ∼ N (0 ,1). Using simple Monte
Carlo integration would be hopeless, but we can choose q (x) to make life much easier. We let q (x)
be equal to the density of the exponential distribution truncated at 5

The simple Monte Carlo estimator is

N
1
N
∑ I ( X i >5 )
i=1

The efficient Monte Carlo estimator is

1
N
p (Y i ) 1
N
p ( Y i)
N
∑ I ( Y i> 5 ) q (Y i )
= ∑
N i=1 q ( Y i )
i=1

where Y i ∼ exp exp(1) with density

Page 69 of 70
KAN-CFIVO1001U Quantitative Methods Autumn 2021

f Y ( y )={exp exp (−( y −5 ) ) y ≥ 5 0 y <5

Page 70 of 70