MODULE 2

HOW TO USE AND ADAPT LARGE LANGUAGE MODELS

(Reference: Kamath Ch.6–8, Atkinson Ch.4–5)

1. USING LLMs WITHOUT CHANGING PARAMETERS

(Prompt-Based Learning)

1.1 Prompting
Definition (Kamath Ch.6)
Prompting is a method of adapting pre-trained LLMs by designing input text (prompts) that guide the
model to perform a specific task without updating model parameters.
LLMs are trained as next-token predictors. Prompting reformulates downstream tasks as next-token
prediction problems.

1.2 Reformulating Tasks as Language Modeling

Instead of:
Input: "I love this movie"
Task: Sentiment Classification
We convert it into:
"I love this movie. The sentiment is ____."
Now it becomes a masked prediction problem.

1.3 Types of Prompting (Atkinson Ch.4)

Type Description Example
Zero-shot Instruction only "Classify sentiment: I love this movie."
One-shot 1 example Provide 1 labeled example
Few-shot 2–10 examples Provide small demonstration set
Cloze-style Fill in blank "This movie is [MASK]."
2. FEW-SHOT LEARNING

2.1 Definition
Few-shot learning is the ability of LLMs to perform tasks using a small number of examples provided in the
prompt, without gradient updates.
Large models (GPT-3 scale and above) show emergent few-shot capabilities.

2.2 Example: Natural Language Inference

Prompt:
Premise: The dog is running.
Hypothesis: An animal is moving.
Answer: Entailment

Premise: The boy is sleeping.

Hypothesis: The boy is awake.
Answer:
 Model predicts: Contradiction

2.3 Why Few-Shot Works (Kamath Ch.6)

 Transformers learn general linguistic patterns.
 Attention mechanism extracts relationships from examples.
 Larger models show better in-context generalization.

2.4 Advantages
 No retraining cost
 Fast deployment
 Works well for large models
2.5 Limitations
 Prompt sensitive
 Order sensitive
 No persistent learning
 Not reliable for small models

3. PROMPTING AS PARAMETER-EFFICIENT FINE-TUNING (PEFT)

Instead of updating all model weights, update only a small subset of parameters.

3.1 Full Fine-Tuning vs PEFT

Full Fine-Tuning
 Updates all parameters
 High GPU cost
 Risk of overfitting
PEFT
 Freeze original model
 Train small additional components
 Efficient and scalable
 PEFT Architecture

Attribute Full Fine-Tuning PEFT (Parameter-Efficient Fine-Tuning)

Updates only a small subset or

Updates every parameter of
Parameters Updated adds small modules;
the model (billions of weights).
base model stays frozen.

Needs very high compute Can run on a single GPU or

Compute Requirement
(multi-GPU / TPU). modest hardware.

Stores a full model for each

Stores only small adapter weights;
Storage Requirement task; heavy storage usage.
base model reused.

Strong results but expensive Almost same performance but much

Performance
And less scalable. cheaper and scalable.

Practical for edge devices,

Difficult in low-resource
Practicality startups,
setups; fits large labs.
universities, research groups.

Applications of PEFT
Some key applications of PEFT include:
1. Edge Deployment: Fine-tuning models to run on devices with limited memory or computing power such
as mobile phones, IoT devices or embedded systems.
2. Multi-Task Learning: Using one shared base model with separate lightweight adapters for different tasks.
This avoids storing a full copy of the model for each task.
3. Few-Shot and Zero-Shot Learning: Applying methods like prompt or prefix tuning to get strong results
with little or no labeled data.
4. Personalized AI Models: Creating task-specific or user-specific adapters so organizations and individuals
can have custom models without retraining everything.
5. Domain Adaptation: Specializing a general-purpose model for specific areas such as healthcare, law,
finance or scientific research without the cost of full fine-tuning.
 Challenges with PEFT
While Parameter-Efficient Fine-Tuning solves many problems of traditional fine-tuning, it also comes with
its own set of challenges:
1. Performance Gaps: In some cases, full fine-tuning still delivers slightly higher accuracy, especially for very
complex tasks that require deep adaptation.
2. Picking the Right Method: It includes techniques like LoRA, Adapters, Prefix-Tuning and BitFit. Each has its
strengths but also limitations. Choosing the right one for a specific task is not always straightforward.
3. Generalization Issues: Some PEFT models may perform well on the training dataset but struggle to
generalize to different domains or unseen data.
4. Managing Multiple Adapters: When we use one model for many tasks, we often end up with many small
adapters. Keeping track of them and integrating them efficiently can become complex.

4. PREFIX-TUNING

4.1 Definition
Prefix-Tuning adds trainable continuous vectors (prefix tokens) to each transformer layer while freezing
original parameters.

4.2 Concept

Instead of modifying model weights:

We add:
[PREFIX TOKENS] + [INPUT TOKENS]
At every transformer layer, prefix vectors influence attention.

4.3 Architecture

1. Overview of Prefix Tuning

 Prefix Tuning is a Parameter-Efficient Fine-Tuning (PEFT) method where:
 The pretrained Transformer model is frozen (its weights are not updated).
 Only a small set of learnable prefix vectors are trained.
 These prefix vectors are added to each transformer layer to guide task-specific behavior.
This significantly reduces the number of trainable parameters.

2. Components in the Architecture

Input Tokens
On the left side of the diagram:
[CLS] X1 X2 ... [SEP]
 These are the original input tokens.
 They are converted into embeddings.
 These embeddings are fed into the transformer layers.
Important: The original model parameters remain frozen.

Prefix Parameters (Core Idea)

In the center:
 A set of trainable prefix parameters is introduced.
 These are converted into Prefix Vectors.
 Often passed through a small MLP (Multi-Layer Perceptron) to generate better representations.
These prefix vectors act as virtual tokens.
They are:
 Not actual words
 Learnable embeddings
 Added before the input sequence

Prefix Embeddings
At the bottom:
P1 P2 ... Pp
These are the learned prefix embeddings.
They are:
 Prepended to the input
 Injected into each transformer layer
 Used as additional Key (K) and Value (V) vectors in attention

Transformer Layers (Right Side)

The transformer stack:
Transformer Layer
Transformer Layer
Transformer Layer
...
Each layer receives:
 Original token embeddings
 Prefix embeddings
In attention:
Instead of just:
Attention(Q, K, V)
It becomes:
Attention(Q, [Prefix_K ; K], [Prefix_V ; V])
 So the model attends to both:
 Original tokens
 Learned prefix vectors

3. How Prefix Tuning Works (Step-by-Step)

1. Input tokens are embedded.
2. Prefix parameters generate prefix vectors.
3. Prefix vectors are added to every transformer layer.
4. During training:
o Only prefix parameters are updated.
o Transformer weights stay frozen.
5. During inference:
o Prefix guides model behavior for the task.

4. Why Prefix Tuning is Powerful

Advantages
 Very few trainable parameters (≈ 0.1–3%)
 Saves memory
 Faster training
 Good for large LLMs (GPT, BERT, etc.)
 Works well in low-data scenarios
Limitations
 Slightly less flexible than full fine-tuning
 May require careful prefix length selection
 Performance depends on task complexity
 A transformer block modified for prefix tuning

4.4 Advantages
 Memory efficient
 Suitable for large LLMs
 Multi-task adaptable
4.5 Disadvantages
 Performance slightly lower than full fine-tuning
 Requires tuning prefix length

5. FACTUAL PROBING IS [MASK]

1. Definition
Factual Probing is an evaluation method used to test whether a Large Language Model (LLM) stores and
retrieves factual knowledge from its parameters.
 It examines whether LLMs:
 Truly store factual knowledge internally
 Or simply memorize patterns from training data

2. Basic Idea
Factual probing uses masked or cloze-style prompts to test knowledge.
Example:
The capital of France is [MASK].
Model prediction:
Paris
If the model correctly predicts Paris, it suggests that the factual relation:
France → Capital → Paris
is stored inside the model.

3. How Factual Probing Works

Step 1: Create a factual template
Example:
 "The CEO of Apple is [MASK]."
 "The chemical symbol of gold is [MASK]."
Step 2: Feed to model
 Masked Language Model (like BERT) predicts missing token.
 Autoregressive model (like GPT) completes the sentence.
Step 3: Compare prediction with true answer
 Correct → Knowledge is likely encoded.
 Incorrect → Model lacks knowledge or hallucinates.

4. Types of Knowledge Tested

Factual probing evaluates:
1. World knowledge
2. Historical facts
3. Scientific facts
4. Entity relationships
5. Knowledge graph relations
Example relations:
 (Country → Capital)
 (Person → Birthplace)
 (Company → Founder)

5. Why Factual Probing is Important

1. Knowledge Evaluation
 Measures how much factual information LLMs contain.
 Helps compare different models.
2. Hallucination Analysis
 Detects when models generate incorrect facts confidently.
 Important for medical, legal, and research applications.
3. Understanding Model Memory
 Helps analyze whether knowledge is:
o Stored in parameters
o Retrieved from context
o Or pattern-matched
 4. Model Editing & Knowledge Updating
 Used in research on:
o Updating incorrect facts
o Injecting new knowledge

6. Factual Probing vs Memorization

Aspect Factual Knowledge Memorization
Generalization
Can answer rephrased questions
Fails if phrasing changes
RobustnessWorks across contexts Sensitive to prompt
Meaning Encodes relationships Stores surface patterns

6. P-TUNING v2

6.1 Definition
P-Tuning v2 improves prefix tuning by:
 Adding deep prompt encoders
 Making performance comparable to full fine-tuning across model sizes
 6.2 Key Insight
Earlier prompt tuning worked only for large models.
P-Tuning v2 works well even for smaller models.

6.3 Advantage
 Universally effective
 Scalable across tasks

7. LoRA (Low-Rank Adaptation)

7.1 Definition
LoRA freezes pre-trained weights and injects small low-rank matrices into attention layers.

7.2 Mathematical Idea

Instead of updating weight matrix W:
We decompose update as:
ΔW = A × B
Where:
 A and B are low-rank matrices
 Rank r << original dimension

7.3 Architecture
 7.4 Why Low-Rank?
Research shows:
Weight updates lie in low-dimensional subspace.
So full matrix update is unnecessary.

7.5 Advantages
 Extremely memory efficient
 Faster training
 Widely used in industry
 Supports multi-task adapters
7.6 Disadvantages
 Rank selection affects performance
 Slight overhead during inference

8. UNIFIED VIEW OF PARAMETER-EFFICIENT TRANSFER LEARNING

All PEFT methods modify model behavior in one of three ways:

Method What is Modified?
Prompting Input tokens
Prefix / P-Tuning Hidden states
LoRA Weight updates
Adapters Small additional layers

Unified perspective:
Task-Specific Adaptation =
Modify Input OR
Modify Intermediate Representations OR
Modify Weight Subspace

9. COMPARISON TABLE
 Method Train Parameters Memory Performance Use Case
Full FT 100% High Best Small models
Prompting 0% Very Low Medium Quick tasks
Prefix <1% Low Good Large LLMs
P-Tuning v2 <1% Low Very Good All scales
LoRA <1% Very Low Very Good Industry standard

10. KEY CONCEPTUAL UNDERSTANDING

Why PEFT is Important?
 LLMs have billions of parameters
 Storing multiple fine-tuned copies is expensive
 PEFT enables:
o Multi-task adaptation
o Faster experimentation
o Lower computational cost

11. Diagram: Overall Adaptation Landscape

INDUSTRIAL APPLICATIONS OF LLMs