Bayesian Machine Learning

Based on Machine Learning for Algorithmic Trading (Chapter 10)

Amirabbas Asadi

March 2022

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Outline

1 Bayesian Machine Learning 5 Discussion

2 Probabilistic Models
Probabilistic Graphical Models
Probabilistic Programming
3 Bayesian Inference
Exact Inference
Markov Chain Monte Carlo
Variational Inference
4 Practical Examples
Practical Examples

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

Can you guess the next number in the following sequence?

2, 4, 6, 8, 10, ?

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

Can you guess the next number in the following sequence?

2, 4, 6, 8, 10, ?

What function could have generated this sequence?

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

Can you guess the next number in the following sequence?

2, 4, 6, 8, 10, ?

What function could have generated this sequence?

f (n) = 2n

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

Can you guess the next number in the following sequence?

2, 4, 6, 8, 10, ?

What function could have generated this sequence?

f (n) = 2n

f (n) = 0.0167n5 − 0.25n4 + 1.4167n3 − 3.75n2 + 6.5667n − 2

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

Can you guess the next number in the following sequence?

2, 4, 6, 8, 10, ?

What function could have generated this sequence?

f (n) = 2n

f (n) = 0.0167n5 − 0.25n4 + 1.4167n3 − 3.75n2 + 6.5667n − 2

f (n) = 0.05n5 − 0.75n4 + 4.25n3 − 11.25n2 + 15.7n − 6

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

H1 : f (n) = 2n
H2 : f (n) = 0.0167n − 0.25n4 + 1.4167n3 − 3.75n2 + 6.5667n − 2
5

H3 : f (n) = 0.05n5 − 0.75n4 + 4.25n3 − 11.25n2 + 15.7n − 6

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

H1 : f (n) = 2n
H2 : f (n) = 0.0167n − 0.25n4 + 1.4167n3 − 3.75n2 + 6.5667n − 2
5

H3 : f (n) = 0.05n5 − 0.75n4 + 4.25n3 − 11.25n2 + 15.7n − 6

All functions reproduce the data exactly

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

H1 : f (n) = 2n
H2 : f (n) = 0.0167n − 0.25n4 + 1.4167n3 − 3.75n2 + 6.5667n − 2
5

H3 : f (n) = 0.05n5 − 0.75n4 + 4.25n3 − 11.25n2 + 15.7n − 6

All functions reproduce the data exactly

All have the same likelihood

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

H1 : f (n) = 2n
H2 : f (n) = 0.0167n − 0.25n4 + 1.4167n3 − 3.75n2 + 6.5667n − 2
5

H3 : f (n) = 0.05n5 − 0.75n4 + 4.25n3 − 11.25n2 + 15.7n − 6

All functions reproduce the data exactly

All have the same likelihood

p(D|H1 ) = p(D|H2 ) = p(D|H3 )

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

H1 : f (n) = 2n
H2 : f (n) = 0.0167n − 0.25n4 + 1.4167n3 − 3.75n2 + 6.5667n − 2
5

H3 : f (n) = 0.05n5 − 0.75n4 + 4.25n3 − 11.25n2 + 15.7n − 6

All functions reproduce the data exactly

All have the same likelihood

p(D|H1 ) = p(D|H2 ) = p(D|H3 )

Then why do people choose the first one for the same data???

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

H1 : f (n) = 2n
H2 : f (n) = 0.0167n − 0.25n4 + 1.4167n3 − 3.75n2 + 6.5667n − 2
5

H3 : f (n) = 0.05n5 − 0.75n4 + 4.25n3 − 11.25n2 + 15.7n − 6

All functions reproduce the data exactly

All have the same likelihood

p(D|H1 ) = p(D|H2 ) = p(D|H3 )

Then why do people choose the first one for the same data???

It seems people as a prior belief prefer H1 over other options

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

But How can we quantify and take into account a prior belief?

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

But How can we quantify and take into account a prior belief?

We can encode our prior belief p(H) as distribution over all hypotheses

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

But How can we quantify and take into account a prior belief?

We can encode our prior belief p(H) as distribution over all hypotheses

p(D|H)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

But How can we quantify and take into account a prior belief?

We can encode our prior belief p(H) as distribution over all hypotheses

p(D|H)p(H)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

But How can we quantify and take into account a prior belief?

We can encode our prior belief p(H) as distribution over all hypotheses

p(D|H)p(H)

To be a valid probability distribution, a normalization constant is needed

p(D|H)p(H)
p(H|D) =
p(D)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

But How can we quantify and take into account a prior belief?

We can encode our prior belief p(H) as distribution over all hypotheses

p(D|H)p(H)

To be a valid probability distribution, a normalization constant is needed

p(D|H)p(H)
p(H|D) =
p(D)

Bayes Theorem

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

What are the potential advantages of Bayesian Machine Learning?

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

What are the potential advantages of Bayesian Machine Learning?

Leveraging the knowledge of a human expert

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

What are the potential advantages of Bayesian Machine Learning?

Leveraging the knowledge of a human expert

Dealing with small data and rare phenomena

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

What are the potential advantages of Bayesian Machine Learning?

Leveraging the knowledge of a human expert

Dealing with small data and rare phenomena

Updating information by observing data (Bayesian Learning)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

What are the potential advantages of Bayesian Machine Learning?

Leveraging the knowledge of a human expert

Dealing with small data and rare phenomena

Updating information by observing data (Bayesian Learning)

Introducing and inferring latent variables

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

Trials: 0 - Success: 0 Trials: 1 - Success: 0 Trials: 3 - Success: 1

θMLE = 0.00 θMLE = 33.33

Posterior Probability

θMAP = 0.00 θMAP = 33.33

Trials: 5 - Success: 1 Trials: 10 - Success: 3 Trials: 25 - Success: 10

θMLE = 20.00 θMLE = 30.00 θMLE = 40.00

Posterior Probability

θMAP = 20.20 θMAP = 30.30 θMAP = 40.40

Trials: 50 - Success: 23 Trials: 100 - Success: 49 Trials: 500 - Success: 249

θMLE = 46.00 θMLE = 49.00 θMLE = 49.80

Posterior Probability

θMAP = 46.46 θMAP = 49.49 θMAP = 49.49

0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100

p, Success Probability p, Success Probability p, Success Probability

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

There are many examples of using a prior belief in ML:

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

There are many examples of using a prior belief in ML:

A prior over the parameters of a linear model

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

There are many examples of using a prior belief in ML:

A prior over the parameters of a linear model (L1 regularization)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

There are many examples of using a prior belief in ML:

A prior over the parameters of a linear model (L1 regularization)

A prior over the parameters of a neural network

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

There are many examples of using a prior belief in ML:

A prior over the parameters of a linear model (L1 regularization)

A prior over the parameters of a neural network (Bayesian Neural

Network)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

There are many examples of using a prior belief in ML:

A prior over the parameters of a linear model (L1 regularization)

A prior over the parameters of a neural network (Bayesian Neural

Network)

A prior over the latent space of an Auto-Encoder

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

There are many examples of using a prior belief in ML:

A prior over the parameters of a linear model (L1 regularization)

A prior over the parameters of a neural network (Bayesian Neural

Network)

A prior over the latent space of an Auto-Encoder (VAE)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

There are many examples of using a prior belief in ML:

A prior over the parameters of a linear model (L1 regularization)

A prior over the parameters of a neural network (Bayesian Neural

Network)

A prior over the latent space of an Auto-Encoder (VAE)

A prior over the possible functions

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

There are many examples of using a prior belief in ML:

A prior over the parameters of a linear model (L1 regularization)

A prior over the parameters of a neural network (Bayesian Neural

Network)

A prior over the latent space of an Auto-Encoder (VAE)

A prior over the possible functions (Gaussian Processes)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

Frequentist Bayesian

Fixed Parameters Random Parameters

Random Data Fixed Data

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

The first step in Bayesian ML is constructing a probabilistic model.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

The first step in Bayesian ML is constructing a probabilistic model.

How to represent a probabilistic model?

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Bayesian Machine Learning

The first step in Bayesian ML is constructing a probabilistic model.

How to represent a probabilistic model?

Probabilistic Graphical Models

Probabilistic Programming

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Graphical Models

Probabilistic Graphical Models

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Graphical Models

Probabilistic Graphical Models

Simplifying representation using Conditional Independence

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Graphical Models

Probabilistic Graphical Models

Simplifying representation using Conditional Independence

Directed Graphical Models : Bayesian Networks

Undirected Graphical Models : Markov Random Fields

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Graphical Models

Directed Graphical Models

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Graphical Models

Directed Graphical Models

B C D

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Graphical Models

Directed Graphical Models

B C D

p(A, B, C, D, E) =

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Graphical Models

Directed Graphical Models

B C D

p(A, B, C, D, E) = p(E|B, C)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Graphical Models

Directed Graphical Models

B C D

p(A, B, C, D, E) = p(E|B, C)p(B|A)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Graphical Models

Directed Graphical Models

B C D

p(A, B, C, D, E) = p(E|B, C)p(B|A)p(C|A)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Graphical Models

Directed Graphical Models

B C D

p(A, B, C, D, E) = p(E|B, C)p(B|A)p(C|A)p(D|A)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Graphical Models

Directed Graphical Models

B C D

p(A, B, C, D, E) = p(E|B, C)p(B|A)p(C|A)p(D|A)p(A)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Graphical Models

Undirected Graphical Models

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Graphical Models

Undirected Graphical Models

x1 x2 x3

x4 x5 x6

x7 x8 x9

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Graphical Models

Undirected Graphical Models

x1 x2 x3

x4 x5 x6

x7 x8 x9

1 ∏
p(x) = ψc (xc )
Z
c∈C

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Programming

There is a more general representation

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Programming

There is a more general representation

Writing computer programs to represent probabilistic models!!!

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Programming

There is a more general representation

Writing computer programs to represent probabilistic models!!!

It’s called Probabilistic Programming

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Programming

There are several tools for Probabilistic Programming:

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Programming

There are several tools for Probabilistic Programming:

Stan (C++, R, Python)

PyMC3 (Python, Backend: Theano)

Pyro (Python, Backend: PyTorch)

Edward (Python, Backend: TensorFlow)

Turing (Julia)

TensorFlow Probability

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Programming

We will use PyMC3!

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Programming

Consider the number of attendees in the book review

sessions:

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Programming

Consider the number of attendees in the book review

sessions:
attendees = [Link]([33, 30, 25, 32, 34, 33, 35, 33, 36])

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Programming

Consider the number of attendees in the book review

sessions:
attendees = [Link]([33, 30, 25, 32, 34, 33, 35, 33, 36])

Now we write a probabilistic program that assumes the data

follows a Poisson distribution. furthermore, we impose a prior on
the λ parameter of the Poisson distribution.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Programming

Consider the number of attendees in the book review

sessions:
attendees = [Link]([33, 30, 25, 32, 34, 33, 35, 33, 36])

Now we write a probabilistic program that assumes the data

follows a Poisson distribution. furthermore, we impose a prior on
the λ parameter of the Poisson distribution.
with [Link]() as model:
mu = [Link](’mu’, 0.5)
number_of_attendees = [Link](’count’, mu=mu)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Coding Time

Coding Time!
Constructing Probabilistic Models in PyMC3

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Programming

Some of the available distributions in PyMC3:

[Link]()
[Link]()
[Link]()
[Link]()
[Link]()
[Link]()
[Link]()
[Link]()
[Link]()
[Link]()

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Probabilistic Programming

And some of the discrete densities:

[Link]()
[Link]()
[Link]()
[Link]()
[Link]()
[Link]()

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Inference Problem

As I mentioned before, the representation power of probabilistic

programming is impressive. Control flows, recursion and etc make
the representation power richer. You can even use nonlinear
function like neural networks in model definition!

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Inference Problem

As I mentioned before, the representation power of probabilistic

programming is impressive. Control flows, recursion and etc make
the representation power richer. You can even use nonlinear
function like neural networks in model definition!

The drawback of this flexible approach the difficulty of inference.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Why inference is difficult?

p(x|z)p(z)
p(z|x) =
p(x)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Why inference is difficult?

p(x|z)p(z)
p(z|x) =
p(x)
To obtain p(x) we have to marginalize all possible
hypotheses:

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Why inference is difficult?

p(x|z)p(z)
p(z|x) =
p(x)
To obtain p(x) we have to marginalize all possible
hypotheses: ∫
p(x) = p(x, z)dz

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Why inference is difficult?

p(x|z)p(z)
p(z|x) =
p(x)
To obtain p(x) we have to marginalize all possible
hypotheses: ∫
p(x) = p(x, z)dz

Now imagine what does p(x) look like if we have used something
like Neural Networks inside the model!

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Why inference is difficult?

p(x|z)p(z)
p(z|x) =
p(x)
To obtain p(x) we have to marginalize all possible
hypotheses: ∫
p(x) = p(x, z)dz

Now imagine what does p(x) look like if we have used something
like Neural Networks inside the model!

This integral is usually intractable

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Why inference is difficult?

p(x|z)p(z)
p(z|x) =
p(x)
To obtain p(x) we have to marginalize all possible
hypotheses: ∫
p(x) = p(x, z)dz

Now imagine what does p(x) look like if we have used something
like Neural Networks inside the model!

This integral is usually intractable

Accordingly the posterior is intractable

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
However, By carefully choosing likelihood and prior, Exact
Inference is possible.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
However, By carefully choosing likelihood and prior, Exact
Inference is possible.

Likelihood Conjugate Prior

Bernoulli Beta
Binomial Beta
Poisson Gamma
Categorical Dirichlet
Uniform Pareto

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Coding Time

Coding Time!
MAP Inference

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Approximate Inference Methods

Nevertheless, Exact Inference is not possible for most of the

models.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Approximate Inference Methods

Nevertheless, Exact Inference is not possible for most of the

models.

Fortunately there are some methods for approximate inference

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Approximate Inference Methods

Nevertheless, Exact Inference is not possible for most of the

models.

Fortunately there are some methods for approximate inference

Markov Chain Monte Carlo

Variational Inference

Expectation Propagation

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Markov Chain Monte Carlo

As I said before, computing the posterior p(z|x) is intractable.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Markov Chain Monte Carlo

As I said before, computing the posterior p(z|x) is intractable.

If we were able to generate samples from p(z|x) then we could

approximate the posterior.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Markov Chain Monte Carlo

Consider a particle in Rn with an initial position (State) X0 . When

the particle is in a position Xt it will move to a position Xt+1 with
probability p(Xt+1 |Xt ) so we have a sequence of random variables

X0 , X1 , X2 , ...

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Markov Chain Monte Carlo

Consider a particle in Rn with an initial position (State) X0 . When

the particle is in a position Xt it will move to a position Xt+1 with
probability p(Xt+1 |Xt ) so we have a sequence of random variables

X0 , X1 , X2 , ...
Such a stochastic process is called Markov Chain

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Markov Chain Monte Carlo

Consider a particle in Rn with an initial position (State) X0 . When

the particle is in a position Xt it will move to a position Xt+1 with
probability p(Xt+1 |Xt ) so we have a sequence of random variables

X0 , X1 , X2 , ...
Such a stochastic process is called Markov Chain
Under some conditions after a time τ the Markov Chain will forget
it’s initial State and becomes stationary

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Markov Chain Monte Carlo

Consider a particle in Rn with an initial position (State) X0 . When

the particle is in a position Xt it will move to a position Xt+1 with
probability p(Xt+1 |Xt ) so we have a sequence of random variables

X0 , X1 , X2 , ...
Such a stochastic process is called Markov Chain
Under some conditions after a time τ the Markov Chain will forget
it’s initial State and becomes stationary
In other words the terms in the sequence

Xτ +1 , Xτ +2 , Xτ +3 , ...

will be random samples from the stationary distribution of Markov

Chain
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Markov Chain Monte Carlo

Is it possible to construct a Markov chain that converges to a

specific distribution?

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Markov Chain Monte Carlo

Is it possible to construct a Markov chain that converges to a

specific distribution?
We need a way to guaranty the existence of stationary distribution.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Markov Chain Monte Carlo

Definition

A Markov chain is called reversible if it satisfies the detailed

balance equations. It means the probability of being in a State xi
then transitioning to a State xj is equal to the probability of being
in xj and then transitioning to xi . formally:

π(xi )P (xj |xi ) = π(xj )P (xi |xj )

Where π(x) is the stationary distribution

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Markov Chain Monte Carlo

Definition

A Markov chain is called reversible if it satisfies the detailed

balance equations. It means the probability of being in a State xi
then transitioning to a State xj is equal to the probability of being
in xj and then transitioning to xi . formally:

π(xi )P (xj |xi ) = π(xj )P (xi |xj )

Where π(x) is the stationary distribution

The idea is to define the transition probability P (x′ |x) such that it
satisfies the detailed balance equations for the target distribution
π(x)
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Metropolis-Hastings Algorithm

So we use the detailed balance equation:

π(x)P (x′ |x) = π(x′ )P (x|x′ )

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Metropolis-Hastings Algorithm

So we use the detailed balance equation:

π(x)P (x′ |x) = π(x′ )P (x|x′ )

P (x′ |x) π(x′ )

=
P (x|x′ ) π(x)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Metropolis-Hastings Algorithm

So we use the detailed balance equation:

π(x)P (x′ |x) = π(x′ )P (x|x′ )

P (x′ |x) π(x′ )

=
P (x|x′ ) π(x)
Now let’s break the transition is two steps. First, we propose a new
State with probability g(x′ |x).

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Metropolis-Hastings Algorithm

So we use the detailed balance equation:

π(x)P (x′ |x) = π(x′ )P (x|x′ )

P (x′ |x) π(x′ )

=
P (x|x′ ) π(x)
Now let’s break the transition is two steps. First, we propose a new
State with probability g(x′ |x).Second, Deciding whether we move
to the proposed State x′ according to an acceptance distribution
A(x′ , x)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Metropolis-Hastings Algorithm

So we use the detailed balance equation:

π(x)P (x′ |x) = π(x′ )P (x|x′ )

P (x′ |x) π(x′ )

=
P (x|x′ ) π(x)
Now let’s break the transition is two steps. First, we propose a new
State with probability g(x′ |x).Second, Deciding whether we move
to the proposed State x′ according to an acceptance distribution
A(x′ , x)
P (x′ |x) = g(x′ |x)A(x′ , x)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Metropolis-Hastings Algorithm

Now we rewrite the equation

A(x′ , x) π(x′ ) g(x|x′ )

=
A(x, x′ ) π(x) g(x′ |x)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Metropolis-Hastings Algorithm

Now we rewrite the equation

A(x′ , x) π(x′ ) g(x|x′ )

=
A(x, x′ ) π(x) g(x′ |x)

Metropolis-Hastings algorithm defines an acceptance ratio that

satisfies the above condition
π(x′ ) g(x|x′ )
A(x′ , x) = min(1, )
π(x) g(x′ |x)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Metropolis-Hastings Algorithm

′ ′
The funny fact is that for computing π(x ) g(x|x )
π(x) g(x′ |x) we don’t need to
know the normalization constant of π(x)!

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Metropolis-Hastings Algorithm

Algorithm Metropolis-Hastings
1: x0 is the initial State
2: Tmax is the maximum number of iterations
3: t ← 0
4: while t < Tmax do
5: x′ ← sample a new candidate from g(x′ |xt )
′ ) g(x|x′ )
6: α ← min(1, π(x π(x) g(x′ |x) )
7: u ← sample from a uniform distribution on [0, 1]
8: if u < α then
9: xt+1 ← x′
10: else
11: xt+1 ← xt
12: t←t+1

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Gibbs Sampling

In high dimensional spaces, the behavior of each dimension can be

different so proposing for all of them in one step is not efficient
and decreases the chance of acceptance.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Gibbs Sampling

In high dimensional spaces, the behavior of each dimension can be

different so proposing for all of them in one step is not efficient
and decreases the chance of acceptance.

The idea is to sample one dimension at each step

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Gibbs Sampling

In high dimensional spaces, the behavior of each dimension can be

different so proposing for all of them in one step is not efficient
and decreases the chance of acceptance.

The idea is to sample one dimension at each step

Or at least,
one group of dimensions at each step

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Gibbs Sampling

In Gibbs Sampling, When we want to sample the (i + 1)th sample

X i+1 we sample its components one by one, by conditioning on
the rest of components. However we use the most recent value for
each component. consider a 3-dimensional example:

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Gibbs Sampling

In Gibbs Sampling, When we want to sample the (i + 1)th sample

X i+1 we sample its components one by one, by conditioning on
the rest of components. However we use the most recent value for
each component. consider a 3-dimensional example:

xi+1
1 ∼ p(xi+1
1 |x2 , x3 )
i i

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Gibbs Sampling

In Gibbs Sampling, When we want to sample the (i + 1)th sample

X i+1 we sample its components one by one, by conditioning on
the rest of components. However we use the most recent value for
each component. consider a 3-dimensional example:

xi+1
1 ∼ p(xi+1
1 |x2 , x3 )
i i

xi+1
2 ∼ p(xi+1
2 |x1 , x3 )
i+1 i

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Gibbs Sampling

In Gibbs Sampling, When we want to sample the (i + 1)th sample

X i+1 we sample its components one by one, by conditioning on
the rest of components. However we use the most recent value for
each component. consider a 3-dimensional example:

xi+1
1 ∼ p(xi+1
1 |x2 , x3 )
i i

xi+1
2 ∼ p(xi+1
2 |x1 , x3 )
i+1 i

xi+1
3 ∼ p(xi+1
3 |x1 , x2 )
i+1 i+1

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Gibbs Sampling

Gibbs sampling works much better than simple Metropolis

[Link] a popular program for sampling from Bayesian
models, uses Gibbs sampling. However it has some disadvantages:

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Gibbs Sampling

Gibbs sampling works much better than simple Metropolis

[Link] a popular program for sampling from Bayesian
models, uses Gibbs sampling. However it has some disadvantages:

We have to obtain the conditional distribution for each

component

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Gibbs Sampling

Gibbs sampling works much better than simple Metropolis

[Link] a popular program for sampling from Bayesian
models, uses Gibbs sampling. However it has some disadvantages:

We have to obtain the conditional distribution for each

component

Sampling the components sequentially makes the procedure

slow specially for a large number of components

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Advances in MCMC methods

More advanced MCMC methods:

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Advances in MCMC methods

More advanced MCMC methods:

Langevin Monte Carlo

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Advances in MCMC methods

More advanced MCMC methods:

Langevin Monte Carlo

Hamiltonian Monte Carlo

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Advances in MCMC methods

More advanced MCMC methods:

Langevin Monte Carlo

Hamiltonian Monte Carlo, NUTS

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Advances in MCMC methods

More advanced MCMC methods:

Langevin Monte Carlo

Hamiltonian Monte Carlo, NUTS

Inference Compilation

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Coding Time

Coding Time!
Inference using MCMC in PyMC3

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Problems with MCMC methods

MCMC methods raise a few challenges:

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Problems with MCMC methods

MCMC methods raise a few challenges:

It takes time for the Markov chain to converge, So we should

discard the samples until convergence.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Problems with MCMC methods

MCMC methods raise a few challenges:

It takes time for the Markov chain to converge, So we should

discard the samples until convergence.

The samples are not independent

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Problems with MCMC methods

MCMC methods raise a few challenges:

It takes time for the Markov chain to converge, So we should

discard the samples until convergence.

The samples are not independent

In high dimensional spaces, It will be hard for proposal

distribution to propose suitable sates so the average
acceptance rate decreases as the number of dimensions
[Link], It can take a really long time for collecting the
samples.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Problems with MCMC methods

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Variational Inference

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Variational Inference

Another approach to approximate inference is trying to find a

tractable distribution as close as possible to the posterior.

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Variational Inference

Another approach to approximate inference is trying to find a

tractable distribution as close as possible to the posterior.

It’s called Variational Inference

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Variational Inference

Another approach to approximate inference is trying to find a

tractable distribution as close as possible to the posterior.

It’s called Variational Inference

We define a parameterized family of distribution q(z; λ) called

Variational Distribution then try to minimize the distance with
true posterior p(z|x).

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Variational Inference

Another approach to approximate inference is trying to find a

tractable distribution as close as possible to the posterior.

It’s called Variational Inference

We define a parameterized family of distribution q(z; λ) called

Variational Distribution then try to minimize the distance with
true posterior p(z|x).

But How is it possible to perform optimization ?

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Variational Inference

Another approach to approximate inference is trying to find a

tractable distribution as close as possible to the posterior.

It’s called Variational Inference

We define a parameterized family of distribution q(z; λ) called

Variational Distribution then try to minimize the distance with
true posterior p(z|x).

But How is it possible to perform optimization ?

Remember that we can not compute p(z|x) !!!

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Variational Inference

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Variational Inference

∫
log p(x) = log p(x, z)dz
∫
p(x, z)q(z; λ)
= log dz
q(z; λ)
p(x, z)
= log Eq(z;λ)
q(z; λ)
p(x, z)
≥ Eq(z;λ) log
q(z; λ)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Variational Inference

∫
log p(x) = log p(x, z)dz
∫
p(x, z)q(z; λ)
= log dz
q(z; λ)
p(x, z)
= log Eq(z;λ)
q(z; λ)
p(x, z)
≥ Eq(z;λ) log
q(z; λ)

The last term above is called Evidence Lower Bound:

p(x, z)
L (λ) = Eq(z;λ) log
q(z; λ)
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Variational Inference

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Variational Inference

surprisingly we have

log p(x) = L (λ) + DKL (q||p)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Variational Inference

surprisingly we have

log p(x) = L (λ) + DKL (q||p)

Where DKL is the KullbackLeibler divergence

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Variational Inference

surprisingly we have

log p(x) = L (λ) + DKL (q||p)

Where DKL is the KullbackLeibler divergence

Maximizing L (λ) is equivalent to minimizing DKL (q||p)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Variational Inference

surprisingly we have

log p(x) = L (λ) + DKL (q||p)

Where DKL is the KullbackLeibler divergence

Maximizing L (λ) is equivalent to minimizing DKL (q||p)

Wow!!!

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Coding Time

Coding Time!
Variational Inference in PyMC3

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Advances in Variational Inference

Basic Variational Inference

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Advances in Variational Inference

Basic Variational Inference

Stochastic Variational Inference

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Advances in Variational Inference

Basic Variational Inference

Stochastic Variational Inference

Black-Box Variational Inference

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Advances in Variational Inference

Basic Variational Inference

Stochastic Variational Inference

Black-Box Variational Inference

Stein Methods

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Summary

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Practical Examples

Practical Examples
A review on a few practical examples

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Advances in Bayesian ML

Research trends in Bayesian ML:

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Advances in Bayesian ML

Research trends in Bayesian ML:

Advanced inference methods (Inference Compilation, Stein

methods)

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Advances in Bayesian ML

Research trends in Bayesian ML:

Advanced inference methods (Inference Compilation, Stein

methods)

Bayesian Deep Learning

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Advances in Bayesian ML

Research trends in Bayesian ML:

Advanced inference methods (Inference Compilation, Stein

methods)

Bayesian Deep Learning

Deep Probabilistic Models

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Advances in Bayesian ML

Research trends in Bayesian ML:

Advanced inference methods (Inference Compilation, Stein

methods)

Bayesian Deep Learning

Deep Probabilistic Models

Normalizing Flows

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Advances in Bayesian ML

Research trends in Bayesian ML:

Advanced inference methods (Inference Compilation, Stein

methods)

Bayesian Deep Learning

Deep Probabilistic Models

Normalizing Flows

Gaussian Processes

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Advances in Bayesian ML

Research trends in Bayesian ML:

Advanced inference methods (Inference Compilation, Stein

methods)

Bayesian Deep Learning

Deep Probabilistic Models

Normalizing Flows

Gaussian Processes

Energy-based Models

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
Discussion

Discussion and Conclusion

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
References I

Betancourt, Michael (2018). A Conceptual Introduction to

Hamiltonian Monte Carlo. arXiv: 1701.02434 [[Link]].

Ghahramani, Zoubin (2015). “Probabilistic machine learning and

artificial intelligence”. In: Nature 521.7553, pp. 452–459. doi:
10.1038/nature14541. url:
[Link]

Hoffman, Matthew D. et al. (2013). “Stochastic Variational

Inference”. In: Journal of Machine Learning Research 14.4,
pp. 1303–1347. url:
[Link]

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
References II

Jansen, Stefan (2020). Machine Learning for Algorithmic Trading:

Predictive models to extract signals from market and Alternative
Data for systematic trading strategies with python. Packt.

Kingma, Diederik P and Max Welling (2014). Auto-Encoding

Variational Bayes. arXiv: 1312.6114 [[Link]].

Koller, Daphne and Nir Friedman (2009). Probabilistic Graphical

Models: principles and techniques. MIT press.

Le, Tuan Anh, Atilim Gunes Baydin, and Frank Wood (2017).
“Inference compilation and universal probabilistic programming”.
In: Artificial Intelligence and Statistics. PMLR, pp. 1338–1348.

Liu, Qiang and Dilin Wang (2016). “Stein variational gradient

descent: A general purpose bayesian inference algorithm”. In:
Advances in neural information processing systems 29.
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
References III

Murphy, Kevin P (2012). Machine learning: a probabilistic

perspective. MIT press.

Rezende, Danilo Jimenez and Shakir Mohamed (2016). Variational

Inference with Normalizing Flows. arXiv: 1505.05770
[[Link]].

Salvatier, John, Thomas V Wiecki, and Christopher Fonnesbeck

(2016). “Probabilistic programming in Python using PyMC3”.
In: PeerJ Computer Science 2, e55.

Zhang, Cheng et al. (2018). Advances in Variational Inference.

arXiv: 1711.05597 [[Link]].

. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .