M.

Sc (IT) Sem IV | Deep Learning | Module 4: Introduction to Generative Models

DEEP LEARNING
Module 4 — Introduction to Generative Models
[Link] (IT) | Year II / Semester IV | SVKM's UPG College | 2024-25
Duration: 15 Lectures · Exam Weightage: ~25%

Part A — VAEs: Overview of Generative Modelling, Probabilistic Interpretation, Latent Variables,

VAE Architecture, Encoder-Decoder, Variational Inference, ELBO, Latent Space Interpolation
Part B — GANs: Adversarial Training Framework, GAN Training, DCGAN, WGAN, Applications —
Image Generation, Style Transfer, Super-Resolution, Data Augmentation & Synthesis

Page 1 | SVKM's UPG College | Deep Learning Notes 2024-25

[Link] (IT) Sem IV | Deep Learning | Module 4: Introduction to Generative Models

SECTION A — OVERVIEW OF GENERATIVE

MODELLING

1. Overview of Generative Modelling

📌 Generative Model: A model that learns the underlying probability distribution p(x) of the training
data and can generate NEW samples that look as if they came from that distribution.
Generative models are one of the most powerful and active areas in deep learning. They power
ChatGPT's text generation, Stable Diffusion's image synthesis, AlphaFold's protein structure
prediction, and drug discovery pipelines.

1.1 Generative vs Discriminative Models

Aspect Discriminative Model Generative Model

Goal Learn P(y|x) — map input to Learn P(x) or P(x,y) —
label understand data distribution
What is learned Decision boundary between How the data was generated
classes
Can generate? No — only classify Yes — sample new x from
P(x)
Examples CNN classifiers, SVM, BERT VAE, GAN, Diffusion Models,
(fine-tuned) GPT
Training signal Labelled data required Unlabelled data sufficient
(unsupervised)
Typical use Classification, regression Image synthesis, data
augmentation, anomaly
detection

1.2 Types of Generative Models

Explicit Density Models: Explicitly define and optimise P(x).
• Tractable: PixelCNN, PixelRNN — model P(x) exactly as autoregressive product. Slow
generation.
• Approximate: VAE — optimise lower bound on log P(x) via variational inference.
Implicit Density Models: Learn to sample from P(x) without explicitly computing P(x). GAN is the
prime example.
Flow-Based Models: Learn invertible transformations. Exact likelihood. Change of variables.
Glow, RealNVP.
Diffusion Models: Add noise step by step, learn to denoise. DALL-E 2, Stable Diffusion,
Midjourney. Currently SOTA for image generation.

Page 2 | SVKM's UPG College | Deep Learning Notes 2024-25

[Link] (IT) Sem IV | Deep Learning | Module 4: Introduction to Generative Models

Model Density Sampling Quality Key Idea

Speed
VAE Approximate Fast Moderate Variational inference +
ELBO
GAN Implicit Fast Very High Adversarial game:
Generator vs
Discriminator
Normalising Flow Exact Fast High Invertible bijective
transformations
Diffusion Model Score-based Slow Highest Learn to reverse
(SOTA) Gaussian noise
process
Autoregressive Exact Moderate High (text) P(x)=ΠP(xᵢ|x<ᵢ), token
(GPT) by token

💡 EXAM TIP: Generative models overview: discriminative vs generative (P(y|x) vs P(x)), 5

types (VAE/GAN/Flow/Diffusion/Autoregressive). Know which model fits which scenario —
VAE for smooth latent space, GAN for high-quality images, Diffusion for SOTA quality, GPT for
text.

2. Probabilistic Interpretation and Latent Variables

2.1 The Core Probabilistic Goal
Given a dataset of observations X = {x⁽¹⁾, ..., x ⁽ᴺ ⁾}, we want to learn a model that assigns high
probability to likely data points and low probability to unlikely ones.
Objective: Maximise log-likelihood log P_θ(X) = Σᵢ log P_θ(x⁽ⁱ⁾)
• P_θ(x): Model probability assigned to data point x. θ: model parameters.
• Challenge for complex data (images): P_θ(x) must be computed over an exponential space
— 256³ × 224 × 224 possible 224×224 RGB images. Intractable to compute directly.
• Solution: Introduce latent variables to simplify the structure of P_θ(x).

2.2 Latent Variables

📌 Latent Variable (z): An unobserved, hidden variable that explains the observed data x. The
model assumes data is generated by first sampling z, then generating x from z.
Generative process (how we assume data is created):
Step 1: Sample latent code z ~ p(z) = N(0, I) [prior distribution]
Step 2: Generate observation x ~ p_θ(x|z) [likelihood/decoder]
Marginal likelihood: P_θ(x) = ∫ p_θ(x|z) p(z) dz
• z ∈ ℝᵈ: Low-dimensional latent code capturing the 'essence' of x. E.g., for face images: z
might encode age, gender, expression, lighting.
• p(z) = N(0, I): Standard normal prior — a simple, tractable distribution. Chosen so we can
easily sample z.
• p_θ(x|z): Decoder network — maps latent code z to a distribution over observations x.
• The integral ∫ p_θ(x|z) p(z) dz: Sums over all possible latent codes that could explain x.
INTRACTABLE for complex decoders — cannot compute exactly.

Page 3 | SVKM's UPG College | Deep Learning Notes 2024-25

[Link] (IT) Sem IV | Deep Learning | Module 4: Introduction to Generative Models

✎ Example: Face generation: z = [0.8, -0.3, 1.2, ...] (32-dim) might encode 'young female, smiling,
indoor lighting'. Decoder p_θ(x|z) generates a realistic face image matching these attributes.

2.3 Posterior Inference Problem

We also need the posterior p(z|x) — given observed data x, what latent code z explains it? This is
needed for:
• Encoding: Given image x, what is its latent code z?
• Learning: Training requires computing posterior expectations.
p(z|x) = p_θ(x|z) p(z) / p_θ(x) = p_θ(x|z) p(z) / ∫p_θ(x|z)p(z)dz
• The denominator p_θ(x) is the same intractable integral! So the posterior p(z|x) is also
intractable.
• Solution — Variational Inference: Approximate the true posterior p(z|x) with a tractable
distribution q_φ(z|x) (encoder network).
💡 EXAM TIP: Latent variables: know the generative process (z~p(z) then x~p(x|z)), why P(x) is
intractable (integral over all z), and why posterior p(z|x) is also intractable. This motivates
VAE's variational inference.

Page 4 | SVKM's UPG College | Deep Learning Notes 2024-25

[Link] (IT) Sem IV | Deep Learning | Module 4: Introduction to Generative Models

SECTION B — VARIATIONAL AUTOENCODERS (VAEs)

3. Variational Autoencoders — Architecture

📌 VAE: A deep generative model (Kingma & Welling, 2013) that learns a continuous, structured
latent space by: (1) encoding x to a distribution q_φ(z|x) = N(μ(x), σ²(x)) and (2) decoding sampled
z back to x. Trained to maximise the ELBO.
VAE combines ideas from: variational Bayesian inference (approximate posteriors), deep learning
(neural network encoder/decoder), and probabilistic graphical models (latent variable models).

3.1 Encoder — Approximate Posterior Network

q_φ(z|x) = N(z; μ_φ(x), diag(σ²_φ(x)))
μ_φ(x) = Neural_Network_μ(x; φ) [mean vector, d-dimensional]
log σ²_φ(x) = Neural_Network_σ(x; φ) [log-variance, d-dimensional]
• Encoder network: Takes input x, outputs TWO vectors — mean μ and log-variance log σ².
• Why log σ²? σ² must be positive. Predict log σ² (unbounded), then σ² = exp(log σ²).
• q_φ(z|x) is a DIAGONAL Gaussian — each latent dimension is independent given x.
• φ: Encoder network parameters (weights and biases).
• This is the approximate posterior — our best estimate of p(z|x) using a neural network.

3.2 Reparameterisation Trick — Making Sampling Differentiable

📌 Reparameterisation Trick: Rewrite the sampling operation z ~ N(μ, σ²) as z = μ + σ ⊙ ε where
ε ~ N(0, I). This separates the stochastic (ε) from deterministic (μ, σ) parts, allowing gradients to
flow through z.
WRONG: z ~ N(μ_φ(x), σ²_φ(x)) [sampling not differentiable —
stops gradients]
RIGHT: ε ~ N(0, I), z = μ_φ(x) + σ_φ(x) ⊙ ε [differentiable w.r.t.
φ]
• Without reparameterisation: z is sampled stochastically → cannot backpropagate gradients
through z to encoder.
• With reparameterisation: z = μ + σ⊙ε. Gradients ∂L/∂μ and ∂L/∂σ flow normally. ε is just
noise — no parameters.
• This is the KEY technical innovation of VAE that makes end-to-end training possible.
✎ Example: μ=[0.5, -0.2], σ=[1.0, 0.8], ε=[-0.3, 1.1] → z = [0.5+1.0×(-0.3), -0.2+0.8×1.1] = [0.2,
0.68]. Gradients flow back to μ and σ networks.

3.3 Decoder — Generative Network

p_θ(x|z) = N(x; μ_θ(z), I) [for continuous data like images:
Gaussian decoder]
p_θ(x|z) = Bernoulli(x; μ_θ(z)) [for binary data: sigmoid output]
μ_θ(z) = Decoder_Network(z; θ) [reconstructed mean image]
• Decoder takes z and produces the parameters of a distribution over x.
• For images: Decoder outputs mean image μ_θ(z). Reconstruction loss ≈ MSE between x
and μ_θ(z).
• θ: Decoder network parameters.

Page 5 | SVKM's UPG College | Deep Learning Notes 2024-25

[Link] (IT) Sem IV | Deep Learning | Module 4: Introduction to Generative Models

• p(z) = N(0, I): The prior. Decoder must generate realistic x for any z sampled from this prior.

4. Variational Inference and the ELBO

We cannot maximise log P_θ(x) directly (intractable). Instead, we derive and maximise the
Evidence Lower BOund (ELBO) — a tractable lower bound on log P_θ(x).

4.1 Derivation of the ELBO

Starting from log P_θ(x), introduce approximate posterior q_φ(z|x):
log P_θ(x) = E_{q_φ(z|x)} [log P_θ(x)]
= E_{q} [log (P_θ(x,z) / P_θ(z|x))]
= E_{q} [log (P_θ(x|z) p(z) / P_θ(z|x))]
= E_{q} [log P_θ(x|z)] + E_{q} [log p(z)/q_φ(z|x)] + E_{q}
[log q_φ(z|x)/P_θ(z|x)]
= E_{q} [log P_θ(x|z)] - KL(q_φ(z|x)||p(z)) + KL(q_φ(z|x)||
P_θ(z|x))
Since KL divergence ≥ 0, we get the ELBO (Evidence Lower BOund):
log P_θ(x) ≥ ELBO = E_{q_φ(z|x)} [log P_θ(x|z)] - KL(q_φ(z|x) || p(z))
= Reconstruction Term - KL Regularisation Term
• ELBO ≤ log P_θ(x). Maximising ELBO maximises a lower bound on the true log-likelihood.
• The gap between log P_θ(x) and ELBO = KL(q_φ(z|x) || P_θ(z|x)) — how well q
approximates true posterior.

4.2 The Two ELBO Terms — Intuition

Term 1 — Reconstruction Term: E_{q_φ(z|x)}[log P_θ(x|z)]
• 'How well does the decoder reconstruct x from the sampled z?'
• For Gaussian decoder: = -||x - μ_θ(z)||² / (2σ²) — proportional to negative MSE.
• For Bernoulli decoder: = Σ [xᵢ log μᵢ + (1-xᵢ) log(1-μᵢ)] — binary cross-entropy.
• Maximising this term pushes encoder to encode enough information to reconstruct x.
Term 2 — KL Regularisation: -KL(q_φ(z|x) || p(z))
• 'How close is the approximate posterior q_φ(z|x) to the prior p(z) = N(0,I)?'
• For diagonal Gaussians, the KL has a closed form:
KL(N(μ,σ²) || N(0,I)) = -½ Σⱼ (1 + log σⱼ² - μⱼ² - σⱼ²)
• Minimising KL pushes encoder to output distributions close to N(0,I). Regularises the latent
space.
• Without KL term: Encoder could map each x to a very narrow distribution around a specific
z — latent space has 'holes'. Cannot sample new data.
• With KL term: Encoder distributions overlap and cover N(0,I). Latent space is smooth. Can
sample z~N(0,I) and decode to get new data.

4.3 VAE Training Loss

L_VAE(θ,φ; x) = -ELBO = E_ε [||x - Decoder(μ+σ⊙ε)||²] + KL(q_φ(z|x)||
N(0,I))
= Reconstruction Loss + KL Divergence
• Note: We minimise this loss (negative ELBO). Training: SGD on L_VAE.

Page 6 | SVKM's UPG College | Deep Learning Notes 2024-25

[Link] (IT) Sem IV | Deep Learning | Module 4: Introduction to Generative Models

• β-VAE: Scale KL term by β > 1. Stronger regularisation → more disentangled latent space
(different dimensions encode different independent factors).
L_β-VAE = Reconstruction Loss + β × KL Divergence (β > 1)
• β-VAE trade-off: Higher β → more disentangled BUT worse reconstruction quality.

4.4 VAE Training Algorithm

• 1. Sample mini-batch {x⁽¹⁾, ..., x⁽ᴮ⁾}
• 2. Encoder forward pass: compute μ_φ(x), σ_φ(x) for each x
• 3. Sample: ε ~ N(0,I), z = μ + σ⊙ε [reparameterisation]
• 4. Decoder forward pass: x̂ = Decoder_θ(z)
• 5. Compute loss: L = ||x - x̂ ||² + KL(N(μ,σ²) || N(0,I))
• 6. Backpropagate gradients through decoder (θ) and encoder (φ)
• 7. Update θ and φ with Adam/AdamW
• 8. Repeat until convergence
💡 EXAM TIP: VAE exam: Know the ELBO formula (reconstruction + KL). Reparameterisation
trick (z=μ+σ⊙ε, not direct sampling). Closed-form KL for diagonal Gaussians: -½Σ(1+logσ²-μ²-
σ²). Explain each term's role: reconstruction=fidelity, KL=smooth latent space. β-VAE for
disentanglement.

5. Encoder-Decoder Architecture in VAEs

The VAE encoder and decoder are typically deep neural networks. The architecture depends on
the data type:

5.1 VAE for Images — Convolutional Architecture

Encoder Architecture (x → μ, log σ²):
• Input: Image x ∈ ℝ^(H×W×C) (e.g., 64×64×3 RGB image)
• Conv Block 1: Conv(32, 4×4, S=2) → BatchNorm → ReLU [32×32×32]
• Conv Block 2: Conv(64, 4×4, S=2) → BatchNorm → ReLU [16×16×64]
• Conv Block 3: Conv(128, 4×4, S=2) → BatchNorm → ReLU [8×8×128]
• Conv Block 4: Conv(256, 4×4, S=2) → BatchNorm → ReLU [4×4×256]
• Flatten: [4×4×256 = 4096-dim vector]
• FC → μ ∈ ℝᵈ and log σ² ∈ ℝᵈ [two separate heads, d = latent dim e.g. 128]
Decoder Architecture (z → x̂ ):
• Input: z ∈ ℝᵈ
• FC → Reshape to [4×4×256]
• TransposedConv 1: ConvT(128, 4×4, S=2) → BatchNorm → ReLU [8×8×128]
• TransposedConv 2: ConvT(64, 4×4, S=2) → BatchNorm → ReLU [16×16×64]
• TransposedConv 3: ConvT(32, 4×4, S=2) → BatchNorm → ReLU [32×32×32]
• TransposedConv 4: ConvT(3, 4×4, S=2) → Sigmoid [64×64×3]
• Decoder is the MIRROR of the encoder. Conv layers replaced by transposed convolutions
(upsampling).

5.2 Transposed Convolution (Deconvolution)

📌 Transposed Convolution: Performs learnable upsampling. The TRANSPOSE of a convolution
operation — increases spatial dimensions.

Page 7 | SVKM's UPG College | Deep Learning Notes 2024-25

[Link] (IT) Sem IV | Deep Learning | Module 4: Introduction to Generative Models

Output size: H_out = (H_in - 1)×S - 2P + F

• Unlike pooling (fixed upsampling), transposed conv learns the upsampling weights.
• Alternative: Bilinear upsampling followed by standard conv. Avoids 'checkerboard artifacts'
common in transposed conv.
✎ Example: Input: 4×4, TranspConv(F=4, S=2, P=1) → H_out = (4-1)×2 - 2×1 + 4 = 8. Doubles
spatial size.

6. Latent Space Interpolation

📌 Latent Space Interpolation: Moving between two points in the latent space z and decoding
intermediate points to observe smooth transitions in the output space.
The most compelling demonstration of a good generative model is smooth interpolation — gradual,
semantically meaningful transitions between two data points when moving linearly through latent
space.

6.1 Linear Interpolation

z_interp(t) = (1-t) × z₁ + t × z₂, t ∈ [0,1]
x̂ (t) = Decoder_θ(z_interp(t))
• t=0: Decode z₁ → reconstruct image 1. t=1: Decode z₂ → reconstruct image 2.
• t=0.5: Midpoint in latent space → image halfway between.
• With good VAE: Intermediate images look realistic and gradually morph between the two.
✎ Example: Interpolate between z_smiling_woman and z_frowning_man in a face VAE:
intermediate z values decode to faces that gradually change from smiling to neutral to frowning while
transitioning from female to male features.

6.2 Spherical Interpolation (SLERP)

SLERP(z₁,z₂,t) = sin((1-t)Ω)/sin(Ω) × z₁ + sin(tΩ)/sin(Ω) × z₂
where Ω = arccos(z₁·z₂ / (||z₁||·||z₂||))
• Follows the great circle on the high-dimensional sphere rather than straight line.
• Better for latent spaces with spherical structure (e.g., normalised embeddings,
hyperspherical VAE).
• Used in GAN latent space traversal where latent is typically on a sphere.

6.3 Latent Space Properties of a Well-Trained VAE

• Smoothness: Nearby z points produce similar outputs. No 'holes' or discontinuities.
• Completeness: Every point in the prior N(0,I) decodes to a valid (realistic) output.
• Disentanglement (β-VAE): Individual latent dimensions correspond to independent data
factors (age, smile, hair colour). Change one dim → change one attribute.
• Clustering: Similar data points cluster together in latent space. Different classes in different
regions.
✎ Example: MNIST VAE with 2D latent space: Different digit classes form separate clusters.
Interpolating from z_3 to z_8 shows digits gradually morphing 3→5→8 with smooth transitions.

💡 EXAM TIP: Latent space interpolation: Linear formula z(t)=(1-t)z₁+tz₂. Why VAE
interpolation is smooth (KL term forces continuous latent space). Contrast with standard AE

Page 8 | SVKM's UPG College | Deep Learning Notes 2024-25

[Link] (IT) Sem IV | Deep Learning | Module 4: Introduction to Generative Models

(no KL → holes in latent space → interpolation produces garbage). β-VAE for

disentanglement. Draw diagram of 2D latent space showing clusters.

Page 9 | SVKM's UPG College | Deep Learning Notes 2024-25

[Link] (IT) Sem IV | Deep Learning | Module 4: Introduction to Generative Models

SECTION C — GENERATIVE ADVERSARIAL

NETWORKS (GANs)

7. Adversarial Training Framework

📌 GAN: Generative Adversarial Network (Goodfellow et al., 2014). Two neural networks —
Generator G and Discriminator D — trained simultaneously in a minimax game. G tries to fool D; D
tries to detect G's fakes.

7.1 The Two Networks

Generator G(z; θ_G):
• Input: Random noise vector z ~ p_z(z) = N(0,I) or Uniform(-1,1). Latent dimension: 100
typical.
• Output: Fake data sample G(z) — same shape as real data (e.g., 64×64×3 image).
• Goal: Generate fake samples so realistic that D cannot distinguish them from real data.
• Architecture: Upsampling network (transposed convolutions for images).
Discriminator D(x; θ_D):
• Input: A data sample x — either real (from training set) or fake (from G).
• Output: Scalar probability D(x) ∈ (0,1). D(x) ≈ 1: 'real'. D(x) ≈ 0: 'fake'.
• Goal: Correctly classify real vs fake samples. Binary classifier.
• Architecture: Downsampling network (standard convolutions for images).

7.2 The Minimax Objective

min_G max_D V(G,D) = E_{x~p_data}[log D(x)] + E_{z~p_z}[log(1-D(G(z)))]
• D maximises V: Wants D(x)→1 for real data and D(G(z))→0 for fake. Both log terms
increase.
• G minimises V: Wants D(G(z))→1 so log(1-D(G(z)))→-∞. Minimising V means maximising
D(G(z)).
• Nash Equilibrium: Optimal solution when G perfectly models p_data and D(x)=0.5
everywhere (cannot distinguish real from fake).

7.3 GAN Training Procedure

Step 1 — Train Discriminator (k steps, typically k=1):
• Sample real batch: {x⁽¹⁾,...,x⁽ᴮ⁾} ~ p_data
• Sample noise: {z⁽¹⁾,...,z⁽ᴮ⁾} ~ p_z. Generate fakes: {G(z⁽¹⁾),...,G(z ⁽ᴮ ⁾)}
L_D = -[1/B Σ log D(x⁽ⁱ⁾) + 1/B Σ log(1 - D(G(z⁽ⁱ⁾)))] [maximise →
negate to minimise]
• Update θ_D: θ_D ← θ_D - α ∇_{θ_D} L_D
Step 2 — Train Generator (1 step):
• Sample new noise: {z⁽¹⁾,...,z⁽ᴮ⁾} ~ p_z
L_G = -1/B Σ log D(G(z⁽ⁱ⁾)) [non-saturating loss — maximise log
D(G(z)) instead of minimise log(1-D(G(z)))]
• Update θ_G: θ_G ← θ_G - α ∇_{θ_G} L_G

Page 10 | SVKM's UPG College | Deep Learning Notes 2024-25

[Link] (IT) Sem IV | Deep Learning | Module 4: Introduction to Generative Models

• Non-saturating loss: Original log(1-D(G(z))) saturates (gradient ≈ 0) when D is confident G

is fake (early training). Replacing with -log D(G(z)) gives stronger gradients even when D is
confident.
• Repeat: Alternate D and G updates. Typically 1:1 ratio.

7.4 GAN Training Challenges

1. Mode Collapse:
• G generates only a few types of outputs (one 'mode' of the data distribution) that fool D,
ignoring most of p_data.
• E.g., GAN for MNIST might only generate '0' digits — they fool D, so G stops trying others.
• Fix: Mini-batch discrimination (D sees statistics of batch, not single sample), unrolled GAN,
Wasserstein distance.
2. Training Instability:
• D learns too fast → G gradient vanishes (D output 0 for all fakes → -log(1-0)→∞ but
gradient of G→0).
• G learns too fast → D becomes random → no training signal.
• Fix: Careful learning rate balance, feature matching, gradient penalty (WGAN-GP).
3. Vanishing Gradients:
• With optimal D: JS divergence between p_data and p_G has zero gradient almost
everywhere.
• Fix: Wasserstein distance (WGAN) — has smooth gradients everywhere.
4. Evaluation Difficulty:
• No clear loss metric like cross-entropy. Loss values don't indicate quality.
• Metrics: FID (Fréchet Inception Distance) — lower is better. IS (Inception Score) — higher
is better.
💡 EXAM TIP: GAN framework: Minimax formula (know it exactly). Discriminator loss (classify
real/fake). Generator loss (non-saturating -log D(G(z)) preferred over log(1-D(G(z)))). Training
loop (alternate D then G). 3 training challenges: mode collapse, instability, vanishing gradients
with solutions.

8. GAN Architectures
8.1 DCGAN (Deep Convolutional GAN, Radford et al. 2015)
📌 DCGAN: First stable and scalable GAN for high-quality image generation. Uses convolutional
networks for both G and D with specific architectural guidelines.
DCGAN Architecture Guidelines:
• Generator: Use transposed convolutions (not upsample+conv). BatchNorm after each layer
except output. ReLU activation in hidden layers. Tanh activation in output layer (output
range -1 to 1).
• Discriminator: Use strided convolutions (not pooling). BatchNorm after each layer except
first and last. LeakyReLU activation (slope=0.2) in all hidden layers. Sigmoid output.
• No fully connected layers in G or D. All-convolutional architecture.
• Latent dim: 100. Learning rates: α_G=0.0002, α_D=0.0002. β₁=0.5 for Adam (not 0.9 —
less momentum for stability).
DCGAN Generator (100→64×64×3):
• z (100) → FC → Reshape [4×4×512]
• ConvT(256, 4×4, S=2, P=1) → BN → ReLU [8×8×256]

Page 11 | SVKM's UPG College | Deep Learning Notes 2024-25

[Link] (IT) Sem IV | Deep Learning | Module 4: Introduction to Generative Models

• ConvT(128, 4×4, S=2, P=1) → BN → ReLU [16×16×128]

• ConvT(64, 4×4, S=2, P=1) → BN → ReLU [32×32×64]
• ConvT(3, 4×4, S=2, P=1) → Tanh [64×64×3]
DCGAN Discriminator (64×64×3→scalar):
• Conv(64, 4×4, S=2, P=1) → LeakyReLU(0.2) [32×32×64]
• Conv(128, 4×4, S=2, P=1) → BN → LeakyReLU [16×16×128]
• Conv(256, 4×4, S=2, P=1) → BN → LeakyReLU [8×8×256]
• Conv(512, 4×4, S=2, P=1) → BN → LeakyReLU [4×4×512]
• Flatten → FC → Sigmoid [scalar output]
• Significance: DCGAN established the 'recipe' for stable GAN training and demonstrated
GAN's ability to learn disentangled representations in latent space (vector arithmetic:
z_king - z_man + z_woman ≈ z_queen).

8.2 Conditional GAN (cGAN, Mirza & Osindero 2014)

📌 Conditional GAN: Both G and D receive an additional condition y (class label, text, image). G
generates samples of class y. D judges if x is a real sample of class y.
min_G max_D V(G,D) = E_{x,y}[log D(x|y)] + E_{z,y}[log(1-D(G(z|y)|y))]
• y can be: Class label (generate specific digit), Text embedding (text-to-image),
Segmentation map (image-to-image), Keypoints (pose-guided generation).
• Generator: Concatenate z with embedded y → upsampling network.
• Discriminator: Concatenate x with embedded y → downsampling → real/fake.
• Applications: Pix2Pix (paired image-to-image translation), text-to-image (early DALL-E
concept).
✎ Example: cGAN for MNIST: Condition y=[0...9]. Generate specific digit on demand. G(z,y=7) →
image of 7. Discriminator also receives y to verify generated digit matches condition.

8.3 Wasserstein GAN (WGAN, Arjovsky et al. 2017)

📌 WGAN: Replaces JS divergence (used in original GAN) with Wasserstein-1 distance (Earth
Mover's Distance). Provides meaningful gradients everywhere and correlates with visual quality.
Stable training.
Wasserstein-1 Distance:
W(p_data, p_G) = sup_{||f||_L ≤ 1} E_{x~p_data}[f(x)] - E_{z~p_z}
[f(G(z))]
• Intuitively: Minimum cost of transporting mass from distribution p_data to p_G.
• f: 1-Lipschitz function. Critic (WGAN's D) must be Lipschitz continuous.
WGAN Changes from Standard GAN:
• No sigmoid in Discriminator (now called Critic). Outputs real value, not probability.
L_Critic = E_{z}[D(G(z))] - E_{x}[D(x)] [maximise: real high, fake
low]
L_Generator = -E_{z}[D(G(z))] [minimise: make critic score
fake high]
•
Lipschitz constraint enforcement: Weight clipping (original WGAN — clip weights to [-c,c]).
Works but can cause training issues (all weights at ±c).
WGAN-GP (Gradient Penalty, Gulrajani et al. 2017) — Improved:

Page 12 | SVKM's UPG College | Deep Learning Notes 2024-25

[Link] (IT) Sem IV | Deep Learning | Module 4: Introduction to Generative Models

L_GP = λ E_{x̂ } [(||∇_{x̂ } D(x̂ )||₂ - 1)²] [penalty for ||gradient|| ≠

1]
x̂ = εx + (1-ε)G(z), ε~Uniform(0,1) [interpolate between real and
fake]
L_Critic_total = L_Critic + λ L_GP
• GP penalises gradient norm deviating from 1 at interpolated samples. Stronger and more
stable than weight clipping.
• λ=10 typically. No BatchNorm in Critic (conflicts with GP). Use LayerNorm or InstanceNorm
instead.
• Advantages of WGAN/WGAN-GP: Meaningful loss metric (Wasserstein distance decreases
as quality improves). Stable training. No mode collapse. Works with many architectures.

8.4 Progressive Growing GAN (ProGAN, Karras et al. 2018)

• Start with low-resolution (4×4) generator and discriminator. Progressively add new layers to
increase resolution: 4×4 → 8×8 → 16×16 → ... → 1024×1024.
• Each new resolution is faded in gradually using a linear blend with the previous resolution.
• Benefit: Training stability — start simple, gradually increase complexity. Also faster.
• Achievement: First GAN to generate 1024×1024 photorealistic face images.
• Foundation for StyleGAN.

8.5 StyleGAN (Karras et al. 2019) — State of the Art

📌 StyleGAN: Controls image synthesis at different scales using style injection at each generator
layer via AdaIN (Adaptive Instance Normalisation). Enables independent control of coarse (pose,
shape) and fine (colour, texture) attributes.
• Mapping network: z → w (intermediate latent space W via 8-layer MLP). W is more
disentangled than Z.
• Style injection: w controls each generator layer via AdaIN: normalise feature maps, scale
and shift with affine transformation of w.
• Stochastic variation: Per-pixel Gaussian noise added at each layer — controls fine details
(hair placement, skin pores).
• StyleGAN2: Fixes PPL (Perceptual Path Length) artifacts. Removes progressive growing.
Better quality.
• StyleGAN3: Equivariant architecture — consistent identity under translation/rotation.
• Applications: [Link] — generates photorealistic non-existent faces.

8.6 Other Important GAN Architectures

Architecture Key Innovation Application

Pix2Pix (2017) cGAN for paired image-to-image Sketch→photo, day→night,
translation. U-Net G + PatchGAN map→satellite
D
CycleGAN (2017) Unpaired image translation via Horse↔zebra, summer↔winter,
cycle-consistency loss. 2 GANs photo↔painting
BigGAN (2018) Large-scale cGAN. Class- High-quality ImageNet class-
conditional, truncation trick. Batch conditional generation
size 2048
SRGAN (2016) Perceptual loss + GAN for 4× Photo-realistic image super-
super-resolution. ResNet G resolution

Page 13 | SVKM's UPG College | Deep Learning Notes 2024-25

[Link] (IT) Sem IV | Deep Learning | Module 4: Introduction to Generative Models

Architecture Key Innovation Application

StyleGAN2 (2020) Weight demodulation, Photorealistic face synthesis,
progressive growing removed. 1024×1024
Best faces
GauGAN (NVIDIA) Spatially adaptive normalisation Photorealistic landscape from
(SPADE). Semantic map→image segmentation map

💡 EXAM TIP: GAN architectures: DCGAN (architectural guidelines: ConvT for G, stride conv
for D, BN, LeakyReLU, Tanh output), WGAN (Wasserstein distance, no sigmoid, weight
clipping replaced by GP penalty), CycleGAN (unpaired, cycle consistency loss). StyleGAN and
Pix2Pix for applications context.

Page 14 | SVKM's UPG College | Deep Learning Notes 2024-25

[Link] (IT) Sem IV | Deep Learning | Module 4: Introduction to Generative Models

SECTION D — APPLICATIONS OF GANs

9. Image Generation
The most direct GAN application — unconditionally or conditionally generating new, realistic
images that did not exist in the training set.

9.1 Unconditional Image Generation

• Face synthesis: StyleGAN2 generates photorealistic human faces at 1024×1024 resolution.
[Link] — every refresh generates a new, non-existent person.
• Scene generation: BigGAN generates diverse high-quality scenes from ImageNet classes
(dogs, landscapes, vehicles).
• Fashion: GAN generates novel clothing designs never worn by real people. Fashion brands
use GANs for digital prototyping.
• Art generation: GAN trained on paintings generates new artworks in learned style.

9.2 Class-Conditional Image Generation

• BigGAN: Condition on ImageNet class label. Sample z and class c → generate
photorealistic image of that class. Truncation trick: sample z from truncated Normal — trade
diversity for quality.
• DALL-E (first version): Transformer-based conditional generation. Text description → token
sequence → image tokens decoded by GAN/diffusion.
• Evaluation metrics:
– FID (Fréchet Inception Distance): Compare distribution of real vs generated images in
InceptionV3 feature space. FID = Fréchet distance between two Gaussians. Lower =
more realistic. FID=0: generated = real.
– IS (Inception Score): Generated images should be sharp (high P(y|x)) and diverse
(uniform P(y)). Higher = better.
– Precision and Recall: Precision = fraction of generated images in real manifold. Recall =
fraction of real distribution covered.
FID = ||μ_r - μ_g||² + Tr(Σ_r + Σ_g - 2√(Σ_r Σ_g))
• FID benchmarks: StyleGAN2 on FFHQ (faces): FID ≈ 3. BigGAN on ImageNet 256: FID ≈
7.
💡 EXAM TIP: Image generation metrics: FID formula (lower = better, 0 = perfect), IS (higher =
better), and what each measures. FID compares feature distributions using InceptionV3. Know
approximate FID values for state-of-the-art models.

10. Style Transfer

📌 Style Transfer: The task of rendering a content image in the visual style of another (style)
image — preserving the content but adopting the texture, colour palette, and brushstroke of the
style image.

10.1 Neural Style Transfer (Gatys et al. 2015) — Pre-GAN

Classic approach using pre-trained VGG features, no GAN needed:

Page 15 | SVKM's UPG College | Deep Learning Notes 2024-25

[Link] (IT) Sem IV | Deep Learning | Module 4: Introduction to Generative Models

• Content representation: Feature maps of content image at deep VGG layer (conv4_2).
• Style representation: Gram matrices Gˡ = FˡᵀFˡ / (H×W×C)² of VGG features at multiple
layers. Gram matrix captures texture/style by measuring feature correlations.
L_content = ||F_content - F_generated||²_F
L_style = Σₗ wₗ ||G_style^l - G_generated^l||²_F
L_total = α L_content + β L_style [optimise generated image pixels]
• Iterative optimisation: Start from random noise or content image, update pixel values via
gradient descent to minimise L_total.
• Limitation: Slow (1-2 min per image — requires SGD on image). Not real-time.

10.2 Fast Neural Style Transfer (Johnson et al. 2016)

• Train a feed-forward network (style network) to directly output stylised image in one forward
pass.
• Train on large image dataset with perceptual loss using VGG features.
• Inference: 1000× faster than iterative Gatys method. Real-time on GPU.
• Limitation: One network per style. To apply a different style, need a different network.

10.3 Arbitrary Style Transfer

• AdaIN (Adaptive Instance Normalisation, Huang & Belongie 2017): Normalise content
features, then scale/shift using statistics of style features.
AdaIN(x, y) = σ(y) × (x - μ(x))/σ(x) + μ(y)
• Any style at test time — single network. Fast real-time inference.
• Used in: Prisma app, TikTok filters, Instagram style filters.

10.4 CycleGAN for Unpaired Style Transfer

• Two generators: G_AB (domain A→B), G_BA (domain B→A). Two discriminators: D_A,
D_B.
• Cycle-consistency loss: G_BA(G_AB(x)) ≈ x. Translate A→B→A should recover original.
L_cycle = ||G_BA(G_AB(x)) - x||₁ + ||G_AB(G_BA(y)) - y||₁
L_total = L_adv_AB + L_adv_BA + λ L_cycle
• Applications: Photo→Monet painting, horse→zebra, summer→winter landscape, medical
domain adaptation (MRI→CT).
• Advantage over Pix2Pix: No paired training data needed. Horse images and zebra images
— no paired horse-zebra images.
💡 EXAM TIP: Style Transfer exam: Gatys (Gram matrix for style, VGG features, iterative
optimisation), AdaIN (arbitrary fast style transfer), CycleGAN (unpaired, cycle consistency loss
formula). Gram matrix = FᵀF captures texture statistics.

11. Super-Resolution
📌 Super-Resolution: Reconstructing a high-resolution image from a low-resolution input. GAN-
based SR achieves photorealistic results far beyond simple upsampling methods.

11.1 SRGAN (Ledig et al. 2016) — First GAN-based SR

• Generator: Deep ResNet that upscales LR image by 4× (e.g., 64→256 pixels).
• Discriminator: Standard CNN that classifies real HR vs generated SR image.

Page 16 | SVKM's UPG College | Deep Learning Notes 2024-25

[Link] (IT) Sem IV | Deep Learning | Module 4: Introduction to Generative Models

Perceptual Loss:
L_perceptual = ||φ(G(LR)) - φ(HR)||²_F
• φ: VGG feature maps. Compares deep features, not pixel values.
• Why perceptual loss: Pixel-wise MSE produces blurry results (average of plausible high-
frequencies). Perceptual loss encourages texture and sharpness.
L_SRGAN = L_perceptual + λ L_adversarial + η L_MSE
• Result: Much sharper, more realistic textures vs. bicubic interpolation. State of art in 2016.

11.2 ESRGAN (Enhanced SRGAN, Wang et al. 2018)

• Residual-in-Residual Dense Blocks (RRDB): Denser skip connections than ResNet. No
BatchNorm (causes artefacts in SR).
• Relativistic Discriminator: D predicts whether real image is MORE realistic than fake, not
just real/fake.
• Improved perceptual loss: Use pre-activation VGG features.
• FID much better than SRGAN. Won PIRM-SR Challenge 2018.
• Real-ESRGAN (2021): Handle real-world degradations (noise, compression, blur). Blind
SR.

11.3 Other SR Approaches

• SwinIR: Transformer-based SR. Self-attention captures long-range dependencies. SOTA
on many SR benchmarks.
• Diffusion SR (SR3, StableSR): Use diffusion models for photorealistic SR. Even better
perceptual quality than GANs.
• LIIF (Local Implicit Image Function): Continuous representation — upscale to ANY
resolution.
• Applications: Satellite imagery enhancement, medical image upsampling, video streaming
(bandwidth-efficient send LR, upscale on device), forensic image enhancement.
💡 EXAM TIP: Super-resolution exam: SRGAN architecture (ResNet G, standard CNN D),
perceptual loss (VGG features, not pixel MSE), why perceptual loss (avoids blurriness).
Mention ESRGAN (RRDB, relativistic D) as improvement. Real-ESRGAN for real-world
degradations.

12. GANs for Data Augmentation and Synthesis

One of the most practically valuable applications of GANs — generating synthetic training data to
augment datasets, especially when labeled real data is scarce or expensive to collect.

12.1 Medical Image Synthesis

• Problem: Medical datasets are small (hundreds of patients), expensive to annotate
(requires expert radiologists), and privacy-sensitive (HIPAA compliance).
• GAN solution: Train GAN on available real scans, generate synthetic annotated data.
• CT→MRI translation (CycleGAN): Hospitals have more CT scans than MRI. Synthesise
MRI from CT.
• Tumour augmentation: Generate synthetic tumours in various sizes/locations for training
segmentation models.
• Retinal fundus synthesis: Synthesise images with specific disease stages (diabetic
retinopathy grades 0-4) for training diagnostic models.

Page 17 | SVKM's UPG College | Deep Learning Notes 2024-25

[Link] (IT) Sem IV | Deep Learning | Module 4: Introduction to Generative Models

• Study (2018): Adding GAN-generated skin lesion images improved melanoma classifier
accuracy from 76.6% to 85.3%.
✎ Example: Chest X-ray augmentation: Train DCGAN on 10,000 CXR images. Generate 10,000
synthetic CXR. Train pneumonia detector on original + synthetic — 7% accuracy improvement.

12.2 Autonomous Driving Data Synthesis

• Rare scenario synthesis: Training self-driving cars requires millions of examples including
rare dangerous scenarios (pedestrian suddenly crossing, car spinning out). Real data
collection is expensive and dangerous.
• Domain adaptation: Simulate-to-real transfer. Train in simulation (cheap), adapt to real-
world with CycleGAN (simulated road → real road appearance).
• Weather augmentation: Generate rainy, foggy, nighttime versions of training images with
CycleGAN.
• CARLA + GAN: Generate photorealistic training data from simulator + GANs.

12.3 Class Balancing and Long-Tail Learning

• Problem: Imbalanced datasets — 90% normal, 10% defective. Classifier biased toward
majority.
• GAN oversampling: Train GAN on minority class, generate synthetic minority examples
until balanced.
• Long-tail recognition: Many rare classes with few examples (e.g., 1000 ImageNet classes
— common dog breeds have 1000 examples, rare fish species have 50). GAN fills the
gaps.
• Comparison with SMOTE: SMOTE creates convex combinations of feature vectors (only for
tabular data). GAN generates realistic new samples in the original data space (works for
images).

12.4 Privacy-Preserving Data Synthesis

• Problem: Cannot share real patient data (HIPAA), real financial transactions (regulations),
real biometric data (GDPR).
• Solution: Train GAN on real private data, share only the trained GAN or its synthetic output.
• Differential Privacy GAN (DP-GAN): Add formal privacy guarantees to GAN training via
differential privacy noise.
• Synthetic data for ML benchmarks: Share synthetic data that has same statistical properties
as real data but no individual records.
• Synthea + GAN: Synthetic electronic health records for medical ML research.

12.5 Text-Conditioned Image Synthesis

• AttnGAN (2018): Attention mechanism selects relevant words from text description for each
image region.
• DALL-E v1 (OpenAI, 2021): Transformer-based text-to-image. 12B parameters. First
commercial-quality text-to-image.
• CLIP (Contrastive Language-Image Pre-training): Learn joint text-image embeddings. Used
to guide GAN generation.
• CLIP-guided GAN: Optimise GAN latent code z to maximise CLIP similarity with text
prompt.
• Modern successors: DALL-E 2, Midjourney, Stable Diffusion — use diffusion models but
trained on CLIP embeddings.
• Key application: Generate product images from descriptions, create training data for visual
classifiers from text descriptions.

Page 18 | SVKM's UPG College | Deep Learning Notes 2024-25

[Link] (IT) Sem IV | Deep Learning | Module 4: Introduction to Generative Models

12.6 Face Generation and Manipulation

• Face aging: GAN trained to age/de-age faces. Forensic age progression.
• Face attribute editing: AttGAN, StarGAN — edit specific attributes (add glasses, change
hair colour) without changing others.
• Face swapping: DeepFake — transfer face of person A to video of person B. Concerns:
misuse for disinformation.
• Face anonymisation: Replace real faces in datasets with GAN-generated faces. Privacy-
preserving surveillance.

Application GAN Type Key Benefit

Balanced medical datasets cGAN / ACGAN Minority class oversampling,
improved classifier accuracy
CT to MRI synthesis CycleGAN (unpaired) No paired data needed, cross-
modal translation
Autonomous driving Domain adaptation GAN Rare scenario synthesis, sim-to-real
scenarios transfer
Fashion / Product images DCGAN / StyleGAN Generate product variations without
photography
Text-to-image AttnGAN / CLIP-GAN Generate images from natural
language descriptions
Privacy-preserving data DP-GAN Share synthetic data without
exposing individuals
Face manipulation StarGAN / AttGAN Attribute editing (age, expression,
hair colour)
Super-resolution SR SRGAN / ESRGAN Enhance satellite/medical image
resolution

💡 EXAM TIP: GAN applications: At least 4 areas with specific model and benefit. Medical
augmentation (accuracy improvement numbers if remembered), autonomous driving (sim-to-
real), class imbalance (minority oversampling), privacy (DP-GAN). DALL-E/Stable Diffusion
context for text-to-image.

Page 19 | SVKM's UPG College | Deep Learning Notes 2024-25

[Link] (IT) Sem IV | Deep Learning | Module 4: Introduction to Generative Models

13. VAE vs GAN — Comprehensive Comparison

Dimension VAE GAN

Core principle Maximise ELBO (lower bound Minimax adversarial game
on log P(x))
Training signal Reconstruction + KL Discriminator real/fake signal
divergence loss
Generated image quality Moderate — often blurry (MSE High — sharp, photorealistic
loss)
Latent space Smooth, continuous, well- Unstructured (unless
organised interpolated carefully)
Interpolation Excellent — smooth semantic Possible but less natural
transitions
Training stability Stable — straightforward VAE Unstable — mode collapse,
loss vanishing gradients
Generation speed Fast (single decoder forward Fast (single generator forward
pass) pass)
Evaluation Reconstruction loss, FID FID, IS, Precision/Recall
Likelihood estimate Yes — ELBO provides No — implicit model
estimate
Best for Representation learning, Photorealistic generation, style
anomaly detection, smooth transfer, SR
interpolation
Major models VAE, β-VAE, CVAE, VQ-VAE DCGAN, WGAN, StyleGAN,
CycleGAN, BigGAN
Paper Kingma & Welling (2013) Goodfellow et al. (2014)

💡 EXAM TIP: VAE vs GAN comparison is a guaranteed 5-10 mark question. Know all
dimensions: quality (GAN better), latent space (VAE smoother), stability (VAE more stable),
likelihood (VAE yes/GAN no), use cases. Connect to module: VAE=representation learning,
GAN=high-quality generation.

MODULE 4 — COMPLETE QUICK REVISION SUMMARY

GENERATIVE MODELLING OVERVIEW:
• Generative model learns P(x). 5 types: VAE (ELBO), GAN (adversarial), Flow (invertible),
Diffusion (denoise), Autoregressive (GPT).
• Discriminative: P(y|x). Generative: P(x) — unlabelled data, can generate new samples.
LATENT VARIABLES:
• z~p(z)=N(0,I) → x~p_θ(x|z). P_θ(x) = ∫p_θ(x|z)p(z)dz — intractable integral.
• True posterior p(z|x) also intractable → approximate with q_φ(z|x) = N(μ_φ(x), σ²_φ(x)).

Page 20 | SVKM's UPG College | Deep Learning Notes 2024-25

[Link] (IT) Sem IV | Deep Learning | Module 4: Introduction to Generative Models

VAE KEY FORMULAS:

• ELBO: log P_θ(x) ≥ E[log P_θ(x|z)] - KL(q_φ(z|x)||p(z))
• KL closed form: -½ Σ(1 + log σ² - μ² - σ²)
• Reparameterisation: z = μ + σ⊙ε, ε~N(0,I) [differentiable sampling]
• Loss = Reconstruction MSE + KL divergence. β-VAE: β×KL for disentanglement.
• Training: x→Encoder→(μ,σ)→z=μ+σε→Decoder→x̂ . Backprop through both.
GAN KEY FORMULAS:
• Minimax: min_G max_D [E log D(x) + E log(1-D(G(z)))]
• D loss: -(E log D(x) + E log(1-D(G(z)))) [maximise → negate]
• G loss: -E log D(G(z)) [non-saturating. Maximise log D(G(z))]
• WGAN: W(p,q) = sup||f||≤1 [E f(x) - E f(G(z))]. No sigmoid. Lipschitz constraint.
• WGAN-GP: L_GP = λ E[(||∇D(x̂ )||₂ - 1)²] where x̂ = εx+(1-ε)G(z).
GAN ARCHITECTURES:
• DCGAN: ConvT in G, stride conv in D, BN, LeakyReLU, Tanh output in G. LR=0.0002,
β₁=0.5.
• cGAN: Both G and D conditioned on y. min max E[log D(x|y)] + E[log(1-D(G(z|y)|y))].
• CycleGAN: G_AB + G_BA + cycle loss ||G_BA(G_AB(x))-x||₁. Unpaired translation.
• StyleGAN: Mapping z→w, AdaIN style injection, stochastic noise per layer.
APPLICATIONS:
• Image generation: FID (lower=better), IS (higher=better). StyleGAN FID≈3 on faces.
• Style transfer: Gram matrix (FᵀF) for style. CycleGAN for unpaired. AdaIN =
σ(y)×(x-μ)/σ+μ(y).
• Super-resolution: SRGAN perceptual loss (VGG features not pixels). ESRGAN
improvement.
• Data augmentation: Medical datasets (7-8% accuracy boost), class balancing, privacy (DP-
GAN).

Topic Likely Exam Question Key Answer Points

Generative models 5 marks — Types of generative 5 types with density/speed/quality
models comparison table
Latent variables 5 marks — Probabilistic Generative process z→x, P(x)=∫,
interpretation of generative intractable posterior, need for
models variational inference
VAE architecture 10 marks — Explain VAE Encoder(μ,σ), reparameterisation
completely trick, decoder, ELBO derivation, KL
closed form
ELBO 5 marks — Derive/explain ELBO = Reconstruction - KL. Each
ELBO term's role. β-VAE extension
Latent space 5 marks — Latent space Linear formula z(t)=(1-t)z₁+tz₂, why
interpolation smooth (KL term), disentanglement
GAN framework 10 marks — Explain GAN with Minimax formula, D loss, G loss
training (non-saturating), training loop, 3
challenges+solutions
DCGAN 5 marks — DCGAN ConvT, stride conv, BN,
architecture guidelines LeakyReLU, Tanh, LR=0.0002,
β₁=0.5
WGAN 5 marks — WGAN vs standard Wasserstein distance, no sigmoid,

Page 21 | SVKM's UPG College | Deep Learning Notes 2024-25

[Link] (IT) Sem IV | Deep Learning | Module 4: Introduction to Generative Models

Topic Likely Exam Question Key Answer Points

GAN weight clipping → WGAN-GP
formula
CycleGAN 5 marks — CycleGAN and style Unpaired translation, cycle
transfer consistency loss formula,
horse↔zebra
GAN applications 5 marks — List 4 applications Image gen (FID), style transfer
(Gram matrix), SR (perceptual loss),
data augmentation
VAE vs GAN 5 marks — Compare VAE and 12-dimension comparison table:
GAN quality/stability/latent
space/likelihood

— END OF DEEP LEARNING MODULE 4 NOTES — ALL 4 MODULES

COMPLETE —
15 Lectures · 13 Major Topics · Sections: Generative Overview | Latent Variables | VAEs | GANs |
Applications
Good luck with your Deep Learning Semester IV Examination! 🎓

Page 22 | SVKM's UPG College | Deep Learning Notes 2024-25