Scribd
Upload
3 views1 page

Metropolis Hastings Sampling Explained

The document discusses the challenges of sampling from complex probability distributions, particularly in high dimensions, where traditional methods like inversion and rejection sampling are often ineffective. It introduces the Metropolis Hastings algorithm as a more efficient alternative for generating samples from such distributions. The document emphasizes the importance of obtaining better samples for effective integration and evaluation of average properties in statistical analysis.

Uploaded by

Syed Hashim Shah Hashmi
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
3 views1 page

Metropolis Hastings Sampling Explained

The document discusses the challenges of sampling from complex probability distributions, particularly in high dimensions, where traditional methods like inversion and rejection sampling are often ineffective. It introduces the Metropolis Hastings algorithm as a more efficient alternative for generating samples from such distributions. The document emphasizes the importance of obtaining better samples for effective integration and evaluation of average properties in statistical analysis.

Uploaded by

Syed Hashim Shah Hashmi
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Statistical_Sampling

Last modified: 7:08 PM - July 10, 2024

Related Notes

Metropolis Hastings algorithm


To Sample a particular distribution p(x), whose functional form have to a multiplicative constant.
It can be multivariate like weights of NN, or conformation of proteins etc.
Then it is hard to generate samples from this pdf because of two things.

If we use inversion sampling then we have to integrate the function and we cannot usually do it for other than simple functions. We need
the CDF of a function, and then uniform samples in inverse CDF can give us desired distribution.
CDF requires integration of PDF which is hard numerically for larger dimensions. And impossible analytically usually
Inverting a CDF is another problem, but numerically can be done if we have good enough numerical integration.
Secondly, Rejection sampling doesn't work, because in higher dimensions most volume of is located in a thin shell. Curse of dimensionality

[Link]
If we use rejection sampling to sample a region of interest, for example, assume the red region is an energy basin and we want to sample
states. If we randomly generate states, we will end up rejecting 26 out of 27 samples in 3 dimensions.
So we need a better way to sample from these distributions.
Also if we can generate better samples from a pdf we can integrate it well as well which can help us evaluate the average properties of a
sample quite well.
There are many ways to generate more relevant samples from a probability distribution than a random sampling of the phase space.
One is metropolis hasting sampling.

[Link]

[Link]

You might also like