Generative AI

This document serves as a rigorous, self-contained reference for university students and faculty
exploring the full landscape of Generative Artificial Intelligence. From foundational probability
theory to state-of-the-art diffusion models, transformer architectures, alignment techniques, and
deployment considerations, every major topic is treated with academic precision and practical
intuition. Content is structured to progress naturally from first principles to advanced concepts,
ensuring both conceptual clarity and mathematical grounding throughout.

Foundations Architectures
Probability, latent spaces, likelihood VAEs, GANs, Diffusion Models,
estimation, generative vs. discriminative Transformers, Encoder-Decoder
models frameworks

Training & Tuning Evaluation & Ethics

Pretraining, fine-tuning, RLHF, LoRA, Perplexity, BLEU, FID, bias, hallucination,
adapters, prompt engineering responsible AI practices
Generative AI refers to a category of machine learning models designed to generate new data that
resembles the data they were trained on. Instead of simply classifying or predicting outcomes like
traditional discriminative models, generative models attempt to learn the underlying probability
distribution of a dataset. By understanding this distribution, the model can create new examples
that are statistically similar to the original data. Because these models must capture the full
structure of the data, they are generally more complex to train but also far more flexible in their
applications.
Definition & Intuition of Generative AI
Generative AI refers to a class of machine learning models whose primary objective is to learn the
underlying probability distribution of observed data and then draw new samples from that
distribution. Formally, given a dataset X = {x₁, x₂, ..., xₙ} drawn from an unknown true distribution
p_data(x), a generative model attempts to approximate p_data(x) with a parameterized model
distribution p_θ(x). Once this approximation is sufficiently accurate, the model can synthesize
entirely new, previously unseen data points that are statistically consistent with the training corpus.

The intuition is straightforward: imagine you have studied ten thousand hand-written digits. A
generative model does not merely learn to classify which digit is which — it learns the texture,
stroke patterns, curvature tendencies, and spatial distributions so intimately that it can
independently produce a new digit that looks authentically hand-written, even though it was never
in the training set. This is the core distinction between generation and recognition.

Generative Model Discriminative Model

Models the full joint distribution p(x, y). Can Models only the conditional p(y | x). Learns
synthesize new data. Examples: VAE, GAN, decision boundaries. Examples: Logistic
Diffusion, GPT. Regression, SVM, BERT (for classification).

A helpful numerical analogy: suppose you observe the dataset {2, 4, 6, 8, 10}. A discriminative
model learns "even numbers go to class A." A generative model learns the underlying pattern well
enough to produce 12 or 3.7 as plausible continuations. The difference is between boundary
learning and distribution learning. This distinction has profound implications for architecture
design, loss functions, and evaluation strategies.

Key Insight 1 Key Insight 2 Key Insight 3

Generative models must The quality of a Every generative model is
handle the full complexity generative model is implicitly or explicitly
of the data manifold, not judged not just by what it solving a density
just class separability. produces, but by the estimation problem,
This makes them diversity and fidelity of whether through
inherently harder to train that production — two likelihood, score
but far more versatile in objectives that are often matching, or adversarial
output. in tension. feedback.
Evolution of Generative Models &
Types
The history of generative modeling in AI spans several decades, each era introducing
fundamentally new ideas about how to parameterize and optimize a learned data distribution. The
earliest approaches were explicitly probabilistic: Boltzmann Machines (1985) used energy-based
representations to model binary data distributions. Restricted Boltzmann Machines (RBMs)
simplified this to a bipartite structure, enabling tractable contrastive-divergence training and
forming the backbone of early deep belief networks.

The decade of the 2010s witnessed three landmark architectural innovations. In 2013, Variational
Autoencoders (VAEs) introduced the reparameterization trick, enabling gradient-based
optimization of a lower bound on the data likelihood while simultaneously learning a structured
latent space. In 2014, Generative Adversarial Networks (GANs) reframed generation as a minimax
game between a generator and a discriminator — a radical departure from explicit likelihood
maximization. By the late 2010s, Autoregressive models (PixelCNN, WaveNet, GPT) demonstrated
that simply predicting the next token in a sequence — given all previous tokens — could produce
remarkably coherent text and images when scaled sufficiently.
 1985 1
Boltzmann Machines — energy-based
generative models

2 2013
Variational Autoencoders (VAE) — latent
variable models with ELBO
2014 3
Generative Adversarial Networks (GAN)
— adversarial minimax training
4 2017
Transformers ("Attention is All You
Need") — attention-based sequence
2020–23 5 modeling
Diffusion Models + LLMs — DALL-E,
Stable Diffusion, GPT-4, Gemini

The most recent paradigm shift came with Denoising Diffusion Probabilistic Models (DDPMs), which
model generation as the reversal of a Markov chain of incremental Gaussian noise additions. Score-
based variants (Song & Ermon, 2020) unified the theoretical framework via stochastic differential
equations. Concurrently, transformer-based language models scaled to billions of parameters,
demonstrating emergent capabilities in code generation, reasoning, and cross-modal synthesis.
Autoregressive Models VAE
Factorize joint distribution as a product Learn a compressed latent
of conditionals: p(x) = ∏ p(xᵢ | x₁,...,xᵢ₋₁). representation z and reconstruct data
Exact likelihood, sequential generation. via decoder. Optimize the Evidence
Examples: GPT, PixelCNN. Lower BOund (ELBO). Enables smooth
interpolation in latent space.

GAN Diffusion Models

Two-network adversarial game. Forward process adds Gaussian noise
Generator G creates fakes; over T steps; reverse process learns to
Discriminator D distinguishes real from denoise. State-of-the-art image and
fake. Nash equilibrium = perfect audio synthesis quality.
generation.
Probability Foundations of Generative
Models
Every generative model is, at its core, a solution to a statistical problem: how do we represent, estimate, and
sample from a high-dimensional probability distribution? The mathematical machinery required spans joint
probability modeling, likelihood estimation, and latent variable theory. Students who deeply internalize
these foundations will find that every architecture — VAE, GAN, Diffusion — is simply a different
computational strategy for the same underlying objective.

Joint Probability and Likelihood

The joint probability p(x₁, x₂, ..., xₙ) of a dataset describes the probability that all observed variables take
their specific values simultaneously. For a model with parameters θ, the likelihood is defined as L(θ) = p_θ(X)
= ∏ᵢ p_θ(xᵢ) under the i.i.d. assumption. Maximizing this quantity — known as Maximum Likelihood
Estimation (MLE) — is the canonical training objective for explicit generative models.

In practice, we maximize the log-likelihood for numerical stability: ℓ(θ) = Σᵢ log p_θ(xᵢ). Consider a simple
Gaussian model where p_θ(x) = N(x; μ, σ²). Given five data points {1.0, 2.0, 3.0, 4.0, 5.0}, the MLE estimates
= 3.0 and σ̂ ² = 2.0. This means the learned model assigns highest probability density around x = 3, and
are μ̂
samples drawn from N(3, 2) will resemble the training distribution.

Gaussian Density p(x)

0.25

0.2

0.15

0.1

0.05

0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6
Data Value (x)

The chart above shows the MLE-fitted Gaussian density over the data range. Maximum density at x = 3.0
corresponds precisely to the estimated mean μ̂ = 3.0, confirming that MLE correctly identifies the center of
the data distribution. Generating new samples means drawing random values from this bell curve.
Latent Variable Modeling
Many real-world distributions are too complex to model directly. Latent variable models introduce
an unobserved variable z such that p_θ(x) = ∫ p_θ(x | z) p(z) dz. The key insight is that conditioning
on z makes the generative process tractable: for a given z, generating x is simple, even though the
marginal p(x) is complex. The prior p(z) is typically chosen as a standard Gaussian N(0, I) for
mathematical convenience and sampling simplicity.
Variational Autoencoders (VAE)
The Variational Autoencoder, introduced by Kingma and Welling in 2013, is one of the most
theoretically elegant generative architectures in deep learning. It addresses the intractability of
computing p_θ(x) = ∫ p_θ(x|z)p(z)dz by introducing a variational approximation: an encoder network
q_φ(z|x) that approximates the true posterior p_θ(z|x). The encoder and decoder are jointly trained
by maximizing the Evidence Lower BOund (ELBO).

The ELBO Objective

The ELBO is derived from Jensen's inequality applied to the log-likelihood:

L(θ, ϕ; x) = Eqϕ (z∣x) [log pθ (x∣z)] − DKL (qϕ (z∣x)∥p(z))

​
​ ​ ​ ​

The first term is the reconstruction loss — how well the decoder reproduces the input from the
latent code. The second term is the KL divergence regularizer — it penalizes the encoder for
producing a posterior that deviates from the standard Gaussian prior. Together, these two terms
create a productive tension: the model must compress data meaningfully while keeping the latent
space well-structured.

Reparameterization Trick
Sampling from z ~ q_φ(z|x) = N(μ_φ(x), σ²_φ(x)) is non-differentiable. The reparameterization trick
rewrites this as z = μ_φ(x) + σ_φ(x) · ε where ε ~ N(0, I). This moves randomness outside the
computational graph, enabling clean backpropagation through the encoder. It is one of the most
impactful "tricks" in modern deep learning.

Step-by-Step VAE Forward Pass Numerical Micro-Example

1. Input x → Encoder → outputs μ and log σ² Let encoder output μ = 0.5, σ = 0.8. Sample ε =

2. Sample ε ~ N(0,I) 1.2.

3. Compute z = μ + σ·ε (reparameterization) z = 0.5 + 0.8 × 1.2 = 0.5 + 0.96 = 1.46
4. z → Decoder → reconstructed x̂
Decoder maps z = 1.46 → reconstructed
5. Compute ELBO loss; backpropagate output.
through μ, σ
KL term: -0.5 × (1 + log(0.64) - 0.25 - 0.64) =
0.068

This small KL value means the encoder

posterior is close to the prior — desirable for a
well-regularized latent space.
Strengths Limitations
Principled probabilistic framework with Blurry reconstructions due to Gaussian
tractable ELBO decoder assumption
Smooth, interpolatable latent space Posterior collapse when KL term
Enables controlled generation via latent dominates
traversal Inferior sample quality vs. GANs and
Diffusion models
Generative Adversarial Networks (GAN)
Introduced by Ian Goodfellow et al. in 2014, Generative Adversarial Networks represent one of the
most creative reframings in machine learning history. Rather than maximizing likelihood directly, a
GAN trains two competing networks through a minimax game: the Generator G learns to produce
realistic synthetic data, while the Discriminator D learns to distinguish real samples from generated
fakes. The generator's only source of training signal is the discriminator's judgment — it never sees
the actual training data directly.

Adversarial Objective
min max V (D, G) = Ex∼pdata [log D(x)] + Ez∼pz [log(1 − D(G(z)))]
​ ​

​
​

​
​

G D

The discriminator D maximizes this expression: it wants D(x) → 1 for real data and D(G(z)) → 0 for
fakes. The generator G minimizes it: it wants D(G(z)) → 1, i.e., to fool the discriminator completely.
At the theoretical Nash equilibrium, p_G = p_data and D(x) = 0.5 everywhere — the discriminator is
maximally confused.

Sample Real Data

Draw minibatch from
dataset
Generate Fake Data
G maps noise z to samples

GAN Training Loop

Adversarially train
generator and
discriminator

Discriminate
D scores real versus fake Backpropagate
Update G and D with
gradients

The diagram above captures the adversarial feedback cycle that drives GAN training. Each iteration
tightens the discriminator's judgment and sharpens the generator's output, in an escalating
competition that converges (ideally) to photorealistic synthesis.
Generator Network Discriminator Network

Input: random noise vector z ~ N(0, I), typically Input: either real image x or fake G(z).
100-dimensional. Architecture: series of Architecture: standard convolutional
transposed convolutions (deconvolutions) that classifier. Output: scalar probability D(x) ∈
upsample from dense latent code to full image [0,1] — probability input is real. Uses
resolution. Output: synthetic image G(z). Uses LeakyReLU activations and no batch
Tanh activation at final layer to match pixel normalization in early layers to maintain
value range [-1, 1]. gradient flow.

1 2 3

Training Instabilities Variants Applications

Mode collapse: Generator DCGAN (deep Face synthesis, data
produces limited variety. convolutional), CGAN augmentation, super-
Vanishing gradients: (conditional), StyleGAN resolution, drug molecule
When D is too strong, G (style-based high-res), design, and anomaly
receives no useful signal. CycleGAN (image-to- detection through
Addressed by Wasserstein image translation without discriminator repurposing.
GAN (WGAN) using Earth paired data).
Mover's distance.
Diffusion Models
Denoising Diffusion Probabilistic Models (DDPMs), formalized by Ho et al. (2020), have emerged
as the dominant paradigm for high-fidelity image synthesis, surpassing GANs on several
benchmarks. The core idea is remarkably intuitive: if you can learn to reverse the process of
gradually adding noise to data, you can generate new data by starting from pure noise and
iteratively denoising.

Forward Process
The forward process is a fixed Markov chain that adds Gaussian noise at each of T timesteps:

q(xt ∣xt−1 ) = N (xt ;

​ ​ ​ 1 − βt xt−1 , βt I)
​ ​ ​ ​

where β_t ∈ (0,1) is a noise schedule. By t = T, the data x_T is approximately standard Gaussian —
all structure has been destroyed. A key mathematical identity allows sampling at any arbitrary
timestep directly: x_t = √ᾱ_t · x_0 + √(1-ᾱ_t) · ε, where ᾱ_t = ∏ₛ(1-βₛ) and ε ~ N(0,I).

Reverse Process & Training

The reverse process p_θ(x_{t-1} | x_t) is a learned Gaussian whose mean and variance are
predicted by a neural network (typically a U-Net). Training minimizes the simplified loss:

Lsimple = Et,x0 ,ϵ [∥ϵ − ϵθ (xt , t)∥2 ]

​

​
​ ​ ​

The model learns to predict the noise ε that was added at step t. This is equivalent to score
matching — estimating the gradient of the log data density. During inference, generation proceeds
by sampling x_T ~ N(0,I) and iteratively applying the learned denoiser for T steps.
 Noise 1ᾱ_t) Signal (ᾱ_t)

0.8

0.6

0.4

0.2

0
0 50 100 150 200 250 300 350 400 450 500 550 600 650 700 750 800 850 900 950 1k
Timestep t

The chart illustrates the signal-to-noise ratio throughout the forward diffusion process. At t = 0, the
data is clean (signal = 1.0). By t = 1000, the signal is fully destroyed and only noise remains. The
reverse process must reconstruct this journey in the opposite direction — recovering structure
from randomness — which is what the neural network learns.
Transformer-Based Generative Models
The Transformer architecture (Vaswani et al., 2017) is the foundational backbone for virtually all
modern large-scale generative AI systems. Its key innovation — scaled dot-product self-attention
— allows every token in a sequence to attend directly to every other token, capturing long-range
dependencies that recurrent architectures (LSTMs, GRUs) struggled with. The attention mechanism
is defined as:

QK T
Attention(Q, K, V ) = softmax ( )V
dk
Here, Q (queries), K (keys), and V (values) are linear projections of the input embeddings. The
scaling by √d_k prevents vanishing gradients in the softmax when dimensionality is large. Multi-
head attention runs this operation in parallel with h different projection matrices, allowing the
model to attend to multiple "aspects" of context simultaneously.

Autoregressive Generation (GPT)

Decoder-only architecture. Each token is predicted from all preceding tokens via causal
(masked) self-attention. Training objective: L = -Σ log p_θ(xᵢ | x₁,...,xᵢ₋₁). Used in GPT-2,
GPT-3, GPT-4, LLaMA, Gemini.

Encoder-Decoder (T5, BART)

Encoder processes entire input bidirectionally; decoder generates output
autoregressively conditioned on encoder representations. Ideal for conditional
generation tasks: translation, summarization, question answering.
Token Probability Modeling
At each generation step, the transformer outputs a logit vector of dimension equal to the
vocabulary size (e.g., 50,257 for GPT-2). A softmax converts these to probabilities. For a
vocabulary V = {a, b, c} with logits {2.0, 1.0, 0.5}, the softmax probabilities are:

Token Logit exp(logit) Probability

a 2.0 7.389 0.659

b 1.0 2.718 0.242

c 0.5 1.649 0.147

Token a is sampled 65.9% of the time under greedy/temperature-1 sampling. Changing the
temperature parameter T rescales logits as logit/T: T → 0 produces greedy deterministic output; T
> 1 increases diversity and randomness. This simple parameter dramatically influences generation
style.
Training Techniques: Pretraining, Fine-
Tuning & Instruction Tuning
Modern large generative models are trained in a carefully orchestrated multi-stage pipeline. Each
stage serves a distinct objective, and understanding the interplay between stages is essential for
practitioners who wish to build domain-specific or task-aligned AI systems.

Stage 1: Pretraining
Pretraining is the large-scale unsupervised (or self-supervised) phase where the model is exposed
to enormous text corpora — often trillions of tokens scraped from web pages, books, code
repositories, and scientific papers. The objective is simply next-token prediction. During this phase,
the model does not receive any task-specific guidance; it learns the statistical regularities of
language, factual associations, grammatical structures, and implicit reasoning patterns purely from
the distributional statistics of text. This phase requires immense computational resources: GPT-3
required approximately 3.14 × 10²³ FLOPS to train on 300 billion tokens.

Stage 2: Fine-Tuning
Fine-tuning adapts the pretrained model to a specific domain or task using a smaller, curated
labeled dataset. Standard fine-tuning updates all model parameters. A fine-tuned model will
converge faster than training from scratch and typically requires orders of magnitude less data,
because the pretrained weights encode rich general representations that need only minor
adjustment.

Stage 3: Instruction Tuning

Instruction tuning (Wei et al., 2022) is a form of supervised fine-tuning where the dataset consists
of (instruction, response) pairs — natural language descriptions of tasks paired with high-quality
demonstrations. This phase teaches the model to follow instructions in zero-shot settings,
dramatically improving generalization. FLAN-T5, GPT-3.5-turbo, and LLaMA-2-Chat are prominent
instruction-tuned variants.
 1 2

retraining Fine-Tuning
Massive corpus, self-supervised, next-token Curated labeled data, task-specific
prediction. Builds world knowledge. adaptation. Transfers pretrained
representations.

3 4

Instruction Tuning RLHF / Alignment

(Instruction, Response) pairs. Teaches zero- Human preference data trains a reward
shot instruction following. model. PPO/DPO aligns output with human
values.
Prompt Engineering & Chain-of-
Thought Prompting
Prompt engineering is the discipline of designing input text sequences (prompts) that elicit optimal
behavior from a pretrained language model without modifying its weights. Since large language
models are sensitive to phrasing, context, and framing, carefully designed prompts can
dramatically improve output quality, factual accuracy, and task alignment — often outperforming
naive approaches by large margins.

Core Prompt Design Strategies

Zero-Shot Few-Shot
Directly state the task. Example: "Classify Provide 2–8 input-output examples before
the sentiment of the following review: the target query. Leverages in-context
[review]". Relies entirely on pretrained learning. Highly effective for structured
knowledge. No examples provided. tasks like classification, extraction,
translation.

Chain-of-Thought System Prompts

Include explicit intermediate reasoning Define the model's persona, constraints,
steps in examples. "Let's think step by and behavioral rules before user input.
step…" activates the model's latent Critical for production deployments.
reasoning capabilities, dramatically Example: "You are a concise academic
improving math and logic performance. assistant. Never speculate."

Chain-of-Thought: Numerical Demonstration

Standard prompt: "Roger has 5 tennis balls. He buys 2 more cans of 3 balls each. How many does
he have?" → Naive model might output: "11." Chain-of-thought prompt adds: "Let's think step by
step." → Model outputs: "Roger starts with 5 balls. He buys 2 cans × 3 balls = 6 balls. 5 + 6 = 11
balls." The intermediate steps force the model to decompose the problem, reducing arithmetic
errors from ~40% to under 10% on GSM8K benchmark (Wei et al., 2022). This is not instruction
tuning — it is a pure inference-time technique requiring no gradient updates.
Prompting Method

Zero-Shot (GPT-3)

Few-Shot (8 examples)

Chain-of-Thought

CoT + Self-Consistency

0 10 20 30 40 50 60 70 80
Accuracy (%) on GSM8K

The accuracy gains from chain-of-thought prompting are substantial — a 3× improvement over
zero-shot, with self-consistency (sampling multiple CoT paths and taking the majority answer)
pushing accuracy to 74% without any weight updates. This chart demonstrates that how you
prompt can matter as much as which model you use.
Retrieval-Augmented Generation
(RAG)
Retrieval-Augmented Generation (RAG), introduced by Lewis et al. (2020), addresses one of the
most critical limitations of standalone language models: their knowledge is static, bounded by the
training cutoff date, and prone to hallucination on factual queries. RAG hybridizes a parametric
language model (which stores knowledge in weights) with a non-parametric external knowledge
store (which stores documents in a searchable format), enabling the model to ground its responses
in retrieved evidence.

RAG Architecture Pipeline

User Query

Encode Query

Retrieve Chunks

Concatenate Context

The five-stage pipeline above shows how RAG elegantly separates the retrieval problem from the
generation problem. The retrieval component is responsible for finding relevant evidence; the
generation component is responsible for synthesizing a coherent, grounded answer from that
evidence. This separation enables independent updates — the knowledge base can be refreshed
without retraining the language model.

Embeddings & Vector Databases

Sentence embeddings are dense vector representations (typically 768–1536 dimensions) that
encode semantic meaning such that semantically similar sentences have small cosine distance.
Models like OpenAI Ada-002, SentenceBERT, and E5 produce these representations. A vector
database (e.g., Pinecone, Weaviate, FAISS) indexes millions of such vectors for approximate
nearest-neighbor search in sub-millisecond latency.
Cosine Similarity Chunking Strategy Context Window
sim(A,B) = (A·B) / (|A||B|). Documents are split into Retrieved chunks are
Values close to 1.0 indicate overlapping chunks of prepended to the query
high semantic similarity. ~256–512 tokens before before generation. With
The retriever returns top-k embedding. Overlap modern 128k-token context
documents with highest ensures boundary windows (GPT-4 Turbo),
cosine similarity to the information is preserved. retrieving 10–20 chunks is
query embedding. Chunk size is a critical feasible, substantially
hyperparameter affecting increasing coverage.
retrieval precision.
Multimodal Generative AI
Multimodal generative AI extends generative modeling beyond single-modality outputs (text only,
image only) to systems capable of understanding and generating content across multiple
modalities simultaneously: text, images, audio, video, and structured data. This capability requires
learned representations that can bridge the semantic gap between modalities — aligning, for
example, the textual concept "a golden retriever playing in snow" with its visual manifestation in
pixel space.

Text-to-Image
CLIP embeddings align text and image spaces. Diffusion models conditioned on text
embeddings synthesize photorealistic images. Examples: DALL-E 3, Stable Diffusion
XL, Midjourney v6.

Text-to-Video
Extends image diffusion with temporal attention layers for coherent motion.
Challenges: maintaining cross-frame consistency, physics plausibility. Examples:
Sora, Gen-2, Lumiere.

Image-to-Text
Vision encoders (ViT, CLIP) extract visual features; language decoder generates
descriptions. Enables image captioning, VQA, OCR. Examples: GPT-4V, LLaVA,
Flamingo.

Audio Generation
Autoregressive (AudioLM) or diffusion-based (AudioLDM) models generate speech,
music, and sound effects from text descriptions or conditioning signals. Examples:
Bark, MusicGen, ElevenLabs.

Cross-Modal Alignment: CLIP

The Contrastive Language-Image Pretraining (CLIP) model (Radford et al., 2021) trains a text
encoder and image encoder jointly on 400 million image-caption pairs using a contrastive loss that
maximizes cosine similarity between matched text-image pairs and minimizes it for mismatched
pairs. CLIP embeddings serve as the conditioning backbone for most text-to-image systems — the
text prompt is embedded by CLIP, and the diffusion model is guided to produce images whose CLIP
embedding closely matches the prompt embedding.
Evaluation of Generative Models
Evaluating generative models is fundamentally harder than evaluating discriminative models, where
a clean test-set accuracy metric suffices. For generation, quality is multidimensional: outputs must
be fluent, factually accurate, diverse, and faithful to the input. No single metric captures all
dimensions, which is why practitioners use a portfolio of complementary evaluation metrics.

0 1 ~0 ~0
Perplexity BLEU Score (0–1) ROUGE (0–1) FID Score
Ideal value; lower is Bilingual Evaluation Recall-Oriented Fréchet Inception
better. Measures how Understudy. Measures Understudy for Gisting Distance for image
well the model n-gram overlap Evaluation. Recall- generation. Compares
predicts a test corpus. between generated focused n-gram feature-space
PP(W) = and reference text. overlap. ROUGE-L distributions. Lower is
p(w₁,...,wₙ)^(-1/n). GPT- Widely used in uses longest common better; state-of-the-
2 achieves ~35 on translation; criticized subsequence. art diffusion models
Penn Treebank; GPT-3 for penalizing valid Standard for achieve FID < 5 on
~20. paraphrases. summarization CIFAR-10.
evaluation.

Perplexity: Step-by-Step Numerical Example

Given a 4-token test sentence with assigned probabilities p(w₁)=0.3, p(w₂)=0.5, p(w₃)=0.2, p(w₄)=0.4:

Token p(wᵢ) log₂ p(wᵢ) Contribution to Entropy

w₁ 0.30 -1.737 -1.737 / 4 = -0.434

w₂ 0.50 -1.000 -1.000 / 4 = -0.250

w₃ 0.20 -2.322 -2.322 / 4 = -0.580

w₄ 0.40 -1.322 -1.322 / 4 = -0.330

Average log-probability = -(0.434 + 0.250 + 0.580 + 0.330) = -1.594. Perplexity = 2^(1.594) = 3.02.
Interpretation: the model is, on average, as uncertain as choosing uniformly among 3 options at
each step. A better model would reduce this to closer to 1.0 (perfect certainty) or at least below 2.0
for well-structured text.
RLHF, Alignment & Bias/Hallucination
Reinforcement Learning from Human Feedback (RLHF) is the training paradigm that transformed
instruction-following base models into aligned assistants like ChatGPT and Claude. The fundamental
challenge is that the objective of "be helpful, harmless, and honest" is difficult to encode as a simple
mathematical loss function. RLHF solves this by training a reward model on human preference data,
then using that reward model as an optimization signal through reinforcement learning.

Reward Model
Training
Learn human
preference
comparisons (A vs
B)

PPO
Supervised Reinforcement
Fine-Tuning Learning
Train on (prompt, Update LLM to
response) maximize reward
demonstrations scores

Preference Optimization Methods

PPO (Proximal Policy Optimization) DPO (Direct Preference Optimization)

Classic RL algorithm. LLM is the policy; reward Eliminates the separate RL phase by directly
model scores responses. A KL-divergence optimizing the preference data with a supervised
penalty prevents the policy from drifting too far loss derived from the Bradley-Terry preference
from the SFT model. Computationally expensive model. Simpler, more stable, increasingly
— requires 4 models simultaneously: actor, preferred over PPO. Used in LLaMA-3 and Mistral
critic, reward, reference. alignment.

Hallucination in LLMs
Hallucination refers to the generation of text that is fluent and confident but factually incorrect or
fabricated. It arises because LLMs are trained to predict statistically likely next tokens, not to verify
factual accuracy. A model trained on biased internet text may reproduce plausible-sounding
falsehoods. Mitigation strategies include RAG (grounding generation in retrieved evidence), chain-of-
thought verification, calibration training, and Constitutional AI principles (Anthropic). Critically,
perplexity-based metrics do not detect hallucination — a hallucinated sentence may have low perplexity
if the fabricated fact sounds linguistically plausible.
Data Bias Hallucination Reward Hacking
Training corpora over- Model generates confident In RLHF, models learn to
represent certain but false statements. exploit weaknesses in the
demographics, languages, Exacerbated by insufficient reward model — producing
and viewpoints. Models grounding, long-tail outputs that score highly by
inherit these biases, knowledge gaps, and reward the reward model but are not
producing systematically hacking. Mitigation: RAG, genuinely helpful. Requires
skewed outputs. Mitigation: fact-checking pipelines, reward model regularization
dataset curation, debiasing uncertainty quantification. and red-teaming.
embeddings, counterfactual
data augmentation.
Parameter-Efficient Fine-Tuning: LoRA,
Adapters & Prefix Tuning
Full fine-tuning of large language models — updating all billions of parameters — is prohibitively
expensive for most research groups and companies. Parameter-Efficient Fine-Tuning (PEFT)
methods address this by freezing the pretrained model's weights and introducing a small number
of trainable parameters that adapt the model's behavior. This enables high-quality task adaptation
at a fraction of the computational cost.

Low-Rank Adaptation (LoRA)

LoRA (Hu et al., 2021) is the most widely adopted PEFT method. Its core insight: the weight updates
ΔW during fine-tuning are low-rank — most of the important adaptation can be captured in a much
smaller subspace than the full weight matrix. For a weight matrix W ∈ ℝ^{d×k}, LoRA decomposes
the update as:

W ′ = W + ΔW = W + BA

where B ∈ ℝ^{d×r} and A ∈ ℝ^{r×k} with rank r ≪ min(d, k). Only A and B are trained; W is frozen.
For a typical d = k = 4096 matrix, full fine-tuning trains 16.7 million parameters. With r = 8, LoRA
trains only 2 × 4096 × 8 = 65,536 parameters — a 256× reduction.

Trainable Parameters (M) GPU Memory (GB)

6k

4k

2k

0
Full Fine-Tuning Adapter Layers Prefix Tuning LoRA (r=16) LoRA (r=4)
Fine-Tuning Method
Adapter Layers Prefix Tuning QLoRA
Small bottleneck modules Prepends k learned Combines LoRA with 4-bit
inserted between "virtual" token quantization of the frozen
transformer layers. Each embeddings to every base model. Enables fine-
adapter has a down- layer's key and value tuning of 70B-parameter
projection (d → r), non- matrices. These soft models on a single 48GB
linearity, and up- prompts modulate GPU. The dominant
projection (r → d). Frozen attention without approach for academic
pretrained layers pass modifying any pretrained and resource-constrained
activations through; only weights. Effective for practitioners (Dettmers et
adapter weights are table-to-text and al., 2023).
trained. 0.5–3% of total summarization tasks.
parameters.
Computational Considerations &
Deployment
Understanding the computational demands of generative models is essential for responsible
research and engineering practice. The economics of large-scale generative AI are dominated by
three factors: training cost, memory usage, and inference latency. Each presents unique
optimization challenges and has driven a rich ecosystem of specialized techniques.

$100M ~1T 6x
GPT-4 Training Cost Parameters in GPT-4 FLOPs per Parameter
Estimated compute cost for a Estimated parameter count Training each parameter
single training run of GPT-4, (unconfirmed). At BF16 requires approximately 6× its
not including R&D, failed runs, precision, storing weights count in FLOPs. A 1B parameter
or infrastructure. Highlights requires ~2TB of VRAM — far model trained on 1T tokens
why only a few organizations exceeding any single GPU's requires ~6×10²¹ FLOPs
can train frontier models. capacity. (Chinchilla scaling law).

Inference Optimization
While training cost is a one-time expenditure, inference cost accumulates across every query
served in production. Key optimization techniques include:

KV-Cache Quantization & Speculative Decoding

During autoregressive generation, Post-training quantization (INT8, INT4)

intermediate key and value matrices from the reduces memory footprint and increases
attention mechanism are cached and reused arithmetic throughput. Speculative decoding
across decoding steps, eliminating redundant uses a small draft model to propose multiple
recomputation. This reduces time-per-token tokens at once, verified by the main model in
by 5–10× at no quality cost. parallel — providing up to 3× speedup for long
generation.
Distributed Training Strategies

Data Parallelism Tensor Parallelism Pipeline Parallelism

Replicate model across Split individual weight Different transformer
devices; split data. matrices across devices. layers assigned to different
Gradients averaged via all- Each device holds a shard; devices. Micro-batching
reduce. Standard for results combined via overlaps computation and
models that fit on a single communication. Used in communication. Used for
GPU. Megatron-LM. training models with
hundreds of layers.
Strengths, Limitations & Ethical
Considerations
✅ Strengths of Generative AI ⚠️ Limitations
Generalization across tasks: A single Hallucination: Confident generation of
pretrained model can perform translation, factually incorrect content without
summarization, coding, and reasoning uncertainty signaling.
without task-specific architectures. Computational cost: Training and serving
Data augmentation: Generating synthetic frontier models is accessible only to well-
training data alleviates data scarcity in low- resourced organizations.
resource domains (medical imaging, rare Data bias propagation: Models encode
languages). and amplify societal biases present in
Creative productivity: Accelerates content training corpora.
creation, code generation, drug discovery, Lack of grounding: No inherent
and materials design at unprecedented mechanism for verifying factual claims
scale. against the real world.
Zero/few-shot learning: Can perform new Context window limitations: Finite context
tasks from brief descriptions, reducing the window constrains document-level
need for expensive labeled datasets. coherence.
Multimodal synthesis: Bridges modalities Interpretability gap: Internal
— text, image, audio, video — enabling representations remain opaque;
entirely new interaction paradigms. mechanistic understanding is nascent.
Ethical Considerations & Responsible AI

Consent and Attribution Misinformation and Deepfakes

Training data sourced from the web High-fidelity text, image, and video
contains content created by human generation capabilities lower the barrier
authors without explicit consent for AI to creating convincing disinformation.
training. Copyright and intellectual Watermarking techniques (SynthID, C2PA
property law is actively evolving to standards) and media provenance
address generated content ownership. frameworks are critical countermeasures
Responsible practitioners audit data but remain imperfect.
provenance and support opt-out
mechanisms.

Environmental Impact Access Equity

Training GPT-3 emitted approximately Concentration of frontier model
552 metric tons of CO₂ equivalent. development in a handful of large
Responsible AI practice includes corporations creates structural inequity
transparent reporting of compute costs, in AI access. Open-source alternatives
preferring efficient architectures, and (LLaMA, Mistral, Falcon) and model
utilizing renewable energy sources for sharing platforms play a crucial
training infrastructure. democratizing role.
Real-World Applications of Generative
AI
Generative AI has transitioned from a research curiosity to a transformative production technology
across virtually every industry sector. The following cases illustrate how the theoretical
architectures and training techniques discussed throughout this document manifest in deployed
systems that create measurable value.

Drug Discovery
Variational autoencoders and graph neural networks generate novel molecular
structures with desired pharmacological properties. Models like AlphaFold2 and
RoseTTAFold predict protein 3D structures with experimental accuracy, compressing
decades of experimental biology into hours of computation. Insilico Medicine used
generative AI to identify a novel fibrosis drug candidate, advancing it to Phase II
clinical trials in approximately 18 months — a fraction of the typical 5–10 year timeline.

Code Generation
GitHub Copilot, powered by OpenAI Codex (a fine-tuned GPT model), completes 40–
55% of newly written code in participating developers, according to GitHub's own
telemetry. Autoregressive code models are trained on billions of lines of source code
from public repositories, learning not just syntax but idiomatic patterns, API usage,
and algorithmic structures across dozens of programming languages.

Content & Advertising

Generative models produce advertising copy, product descriptions, social media
content, and entire articles at scale. Multimodal systems generate complete brand
campaigns — coordinated text, imagery, and video — from brief textual briefs. The
advertising agency WPP reported using AI to reduce campaign asset creation time by
70% for select clients.

Healthcare Reporting
Radiology report generation systems fine-tune language models on electronic health
records to produce structured diagnostic summaries from medical imaging. RAG-
enhanced clinical decision support tools retrieve relevant literature and drug
interaction databases in real time, grounding LLM responses in current medical
evidence.
Padding in Neural Networks: Real-Life
Problem & Step-by-Step Solution
Padding is an essential preprocessing and architectural operation in convolutional neural networks
(CNNs) and transformer models. In CNNs, padding adds extra values (typically zeros) around the
border of an input feature map before applying a convolutional filter, controlling the spatial
dimensions of the output. Without padding, repeated convolution operations progressively shrink
the feature map, discarding boundary information and limiting network depth. Understanding
padding numerically is critical for correctly computing output dimensions, receptive fields, and
feature map shapes throughout a deep network.

The Problem: Preserving Feature Map Dimensions

Consider a real-life image classification task: a 5×5 grayscale image patch from a medical X-ray is
being processed by a convolutional layer with a 3×3 filter and stride = 1. Without padding, the
output would be (5-3)/1 + 1 = 3×3 — the spatial information from border pixels is under-represented.
With same padding (padding = 1), the output remains 5×5, preserving all spatial information. Let's
work this through completely.

Dataset: Input Feature Map

Row \ Col C1 C2 C3 C4 C5

R1 2 4 1 3 2

R2 5 6 7 2 1

R3 3 8 5 4 6

R4 1 2 9 3 7

R5 4 5 6 1 2
Step 1: Apply Zero Padding (p=1)
We add a border of zeros around the entire 5×5 input, creating a 7×7 padded matrix:

Row\Col 0 1 2 3 4 5 0

0 0 0 0 0 0 0 0

1 0 2 4 1 3 2 0

2 0 5 6 7 2 1 0

3 0 3 8 5 4 6 0

4 0 1 2 9 3 7 0

5 0 4 5 6 1 2 0

0 0 0 0 0 0 0 0
Padding: Convolution Computation &
Output Analysis
Step 2: Define the 3×3 Convolution Filter
1 0 -1

1 0 -1

1 0 -1

This is a standard vertical edge detection filter (Sobel-style). It responds strongly to regions where pixel
intensity transitions from high to low horizontally — exactly the type of boundary a radiologist or
segmentation network needs to detect in medical imaging.

Step 3: Compute Output at Position (1,1) — Top-Left

Corner
We center the filter over position (1,1) of the padded matrix. The 3×3 neighborhood from the padded matrix
is:

0 0 0

0 2 4

0 5 6

Element-wise multiplication with filter, then sum:

(0×1)+(0×0)+(0×-1) + (0×1)+(2×0)+(4×-1) + (0×1)+(5×0)+(6×-1)

= 0 + 0 + 0 + 0 + 0 + (-4) + 0 + 0 + (-6) = -10

Step 4: Compute Output at Position (1,3) — Center

Top
Padded neighborhood centered at (1,3):

0 0 0

4 1 3

6 7 2

(
(0×1)+(0×0)+(0×-1)+(4×1)+(1×0)+(3×-1)+(6×1)+(7×0)+(2×-1) = 0+0+0+4+0-3+6+0-2 = 5

Output Summary Table (Selected Positions)

Output Position Input Region Center Computation Result Interpretation

(1,1) Top-left (with -10 Strong negative

padding) edge — left
boundary

(1,3) Top-center +5 Weak positive edge

response

(3,3) Image center 0 Uniform region — no

vertical edge

(5,5) Bottom-right (with -1 Weak edge at

padding) bottom-right
boundary

Convolution Output Value

-5

-10
(1,1) (1,2) (1,3) (2,1) (2,3) (3,3) (4,4) (5,5)
Output Position (row, col)

The bar chart confirms that padding allows the convolution to produce a full 5×5 output with
meaningful responses at boundary positions — the corner value of -10 indicates a strong left-
boundary edge, while central positions near zero indicate flat regions. Without padding, positions (1,1)
and (5,5) would not exist in the output at all, causing the network to lose critical boundary context
entirely. This demonstrates why padding is not merely an implementation detail — it is a principled
design choice that directly affects the network's ability to detect features at all spatial locations,
including the boundaries most relevant for object detection and medical segmentation tasks.
When TO Use Padding When NOT TO Use Padding
Use padding when preserving spatial Avoid padding when intentional spatial
resolution through deep networks, when downsampling is desired (use stride instead),
boundary features matter (edge detection, when the network architecture is designed
segmentation, medical imaging), when for classification only and boundary detail is
building fully-convolutional architectures, or irrelevant, or when computational budget is
when output must match input dimensions extremely constrained and the additional
exactly (image-to-image translation). padded operations cannot be afforded.