Deep Generative Models
Variational Autoencoders
Generative Nets
• Generative models that represent probability
distributions over multiple variables in some
way.
• Directed Generative Nets
– Differentiable Generator Nets
Differentiable Generator Nets
• Many generative models are based on the idea of using a
differentiable generator network.
• The model transforms samples of latent variables z to
samples x or to distributions over samples x using a
differentiable function 𝑔 𝑧; 𝜃 𝑔 , typically represented
using a NN
1. Variational autoencoders - which pair the generator net with
an inference net
2. Generative adversarial networks - which pair the generator
network with a discriminator network
3. Techniques that train generator networks in isolation.
Generator Networks
• Generator networks are essentially just
parameterized computational procedures for
generating samples
– the architecture provides the family of possible distributions to sample
from
– the parameters select a distribution from within that family.
• Example, the standard procedure for drawing samples from a
normal distribution with mean µ and covariance Σ is to feed
samples z from a normal distribution with zero mean and
identity covariance into a very simple generator network.
– This generator network contains just one affine layer
𝑥=𝑔𝑧 =𝜇+𝐿𝑧
L is the Cholesky decomposition of Σ
Generator networks
• To generate samples from more complicated distributions, we may
use a feedforward network to represent a parametric family of
nonlinear functions 𝑔, and use training data to infer the
parameters selecting the desired function.
• We can think of gas providing a nonlinear change of variables that
transforms the distribution over 𝑧 into the desired distribution over
𝑥.
• we often use indirect means of learning 𝑔
• In some cases, rather than using g to provide a sample of x
directly, we use g to define a conditional distribution over x. For
example, we could use a generator net whose final layer consists
of sigmoid outputs to provide the mean parameters of Bernoulli
distributions
𝑝 𝑥𝑖 = 1 𝑧 = 𝑔(𝑧)𝑖
• In this case, when we use g to define p(x | z), we
impose a distribution over x by marginalizingz:
𝑝 𝑥 = 𝐸𝑧𝑝𝑥 𝑧
• The two different approaches to
formulating generator nets
– emitting the parameters of a conditional distribution
versus
– directly emitting samples
have complementary strengths and weaknesses
1. emitting the parameters of a conditional distribution
2. directly emitting the samples
• When the generator net defines a conditional
distribution over x, it is capable of generating discrete
data as well as continuous data.
• When the generator net provides samples directly, it is
capable of generating only continuous data.
• The advantage to direct sampling is that we are no longer
forced to use conditional distributions whose form can
be easily written down and algebraically manipulated by
a human designer
• Generative modeling seems to be more difficult than
classification or regression because the learning process
requires optimizing intractable criteria.
• In differentiable generator nets, the criteria are intractable
because the data does not specify both the inputs z and the
outputs x
• The learning procedure needs to determine how to arrange z
space in a useful way and additionally how to map from z to x
• Several approaches to training differentiable generator nets
given only training samples of x
Variational Autoencoder
• Graphical models + Neural networks
• A directed model that uses learned approximate
inference and can be trained purely with gradient-based
methods
• Lets us design complex generative models of data, and fit
them to large datasets.
• They can be used to learn a low dimensional representation Z
of high dimensional data X such as images (of e.g. faces).
• X and Z are random variables. It’s therefore possible to sample
X from the distribution P(X|Z), thus creating e.g. images of
faces, MNIST Digits, or speech.
Variational AutoEncoders
Variational Autoencoders (VAEs) are generative models that learn a
smooth, probabilistic latent space, allowing them not only to compress
and reconstruct data but also to generate entirely new, realistic
samples. VAEs capture the underlying structure of a dataset and
produce outputs that closely resemble the original data.
Learns a continuous latent representation
Enables controlled and meaningful data generation
Widely used in image synthesis, anomaly detection, and representation
learning
1. Encoder (Understanding the Input)
The encoder takes input data like images or text and learns its key
features. Instead of outputting one fixed value, it produces two vectors
for each feature:
Mean (μ): A central value representing the data.
Standard Deviation (σ): It is a measure of how much the values can
vary.
These two values define a range of possibilities instead of a single
number.
2. Latent Space (Adding Some
Randomness)
Instead of encoding the input as one fixed point it pick a random point
within the range given by the mean and standard deviation. This
randomness lets the model create slightly different versions of data
which is useful for generating new, realistic samples.
3. Decoder (Reconstructing or Creating New Data)
The decoder takes the random sample from the latent space and tries to
reconstruct the original input. Since the encoder gives a range, the decoder
can produce new data that is similar but not identical to what it has seen.
Mathematics behind Variational
Autoencoder
Mathematics behind Variational
Autoencoder
Manifold Hypothesis
Variational auto encoders
(idea of low dim
manifold)
Generative Adversarial Network (GAN)
Generative Adversarial Networks (GAN) help machines to create new, realistic
data by learning from existing examples. It is introduced by Ian Goodfellow and
his team in 2014 and they have transformed how computers generate images,
videos, music and more.
Unlike traditional models that only recognize or classify data, they take a creative
way by generating entirely new content that closely resembles real-world data.
This ability helped various fields such as art, gaming, healthcare and data
science. In this article, we will see more about GANs and its core concepts.
Architecture
1. Generator Model
2. Discriminator Model