Bayesian Learning:
EM (Expectation-Maximization) Algorithm
Unit 2
Dr. Sushil Kumar
Associate Professor
Expectation Maximization (EM)
• It is a statistical algorithm used for finding the maximum likelihood
estimates of parameters in probabilistic models, especially when the
model depends on latent (hidden) variables.
• It is an iterative process that alternates between two main steps:
1. Expectation step (E-step)
2. Maximization step M-step).
Steps of the EM Algorithm
• Initialization:
• Start with initial guesses for the parameters of the model.
• E-step (Expectation):
• Using the current parameter estimates, calculate the expected value of the latent
variables given the observed data. This involves calculating the posterior probabilities
of the latent variables.
• M-step (Maximization):
• Update the parameters by maximizing the expected log-likelihood found in the E-step
with respect to the parameters. Essentially, find the parameters that maximize the
likelihood of the observed data given the latent variables.
• Repeat:
• Alternate between the E-step and M-step until convergence is reached (i.e., the
parameters stop changing significantly between iterations).
EM Process
Example:
Example Cont…
Example Cont…
Example:
Example:
Advantages
Disadvantages