NLP Final Term

Complete Study Guide

Topics Covered:
• Fine-Tuning

• Prompt Engineering

• Transformers: Architecture Overview

• Data Collection & Annotation

• BERT and Pre-trained Language Models

• Current Trends & Research

Includes: Practice Questions for All Topics

Answer to Practice Question

Scenario: An NLP model fine-tuned for sentiment analysis achieves 98% accuracy on training data but
performs poorly on unseen test data.

Q4. Identify the core problem in this case.

The core problem is overfitting. The model has memorized the training data rather than learning
generalizable patterns. It fits the training examples too closely, including their noise and peculiarities, so it
fails to generalize to new, unseen data.

Q5. Which fine-tuning strategies could reduce this issue?

Regularization techniques such as dropout (randomly disabling neurons during training) and weight decay
(L2 regularization) prevent the model from relying too heavily on specific features.

Early stopping means monitoring validation loss and stopping training when it starts increasing, even if
training loss continues decreasing.

Reducing model complexity by using a smaller model or freezing some layers of the pre-trained model so
fewer parameters are updated.

Data augmentation expands the training set through techniques like synonym replacement,
back-translation, or random insertion/deletion.

Learning rate adjustments using a smaller learning rate or learning rate schedulers prevents drastic
weight changes that lead to overfitting.

Q6. How can validation techniques help during fine-tuning?

Holdout validation reserves a portion of data (typically 10-20%) as a validation set to monitor performance
on unseen data during training.

K-fold cross-validation splits data into k parts, trains on k-1 folds, and validates on the remaining fold,
rotating through all combinations. This gives a more reliable estimate of generalization.

Monitoring validation metrics allows you to detect when the gap between training and validation
performance starts widening—a clear sign of overfitting.

Hyperparameter tuning uses validation performance to select optimal settings for learning rate, batch size,
and regularization strength.
1. FINE-TUNING
What is Fine-Tuning?
Fine-tuning is the process of taking a pre-trained model (trained on a large general corpus) and further training
it on a smaller, task-specific dataset. The pre-trained model has already learned general language
representations, and fine-tuning adapts these to your specific task.

Why Fine-Tune?
Pre-trained models like BERT learn universal language features from massive datasets. Training from scratch
would require enormous data and compute. Fine-tuning leverages this existing knowledge—you're essentially
customizing a general-purpose tool for a specific job.

The Fine-Tuning Process

Step 1: Select a pre-trained model appropriate for your task (BERT for understanding, GPT for generation).

Step 2: Prepare task-specific data with proper formatting and labels.

Step 3: Add task-specific layers on top of the pre-trained model (e.g., a classification head for sentiment
analysis).

Step 4: Train with a small learning rate (typically 2e-5 to 5e-5) to avoid destroying pre-trained knowledge.

Step 5: Evaluate and iterate using validation data.

Key Concepts
Catastrophic forgetting occurs when fine-tuning causes the model to lose previously learned knowledge.
Mitigate this with small learning rates and freezing lower layers.

Layer freezing means keeping some layers fixed (not updating their weights) during fine-tuning. Lower layers
capture general features; higher layers capture task-specific features. Common approach: freeze early layers,
fine-tune later layers.

Discriminative learning rates apply different learning rates to different layers—smaller rates for lower layers,
larger for upper layers.

Types of Fine-Tuning
Full fine-tuning updates all model parameters. Most flexible but requires more data and risks overfitting.

Feature extraction freezes the entire pre-trained model and only trains the new task-specific layers.

Adapter-based fine-tuning inserts small trainable modules between frozen layers, updating only these
adapters.

LoRA (Low-Rank Adaptation) adds small trainable matrices to existing weights, dramatically reducing
trainable parameters.
2. PROMPT ENGINEERING
What is Prompt Engineering?
Prompt engineering is the art of crafting input text (prompts) to guide language models toward desired outputs
without changing model weights. It's about communicating effectively with the model.

Why Does It Matter?

Large language models are sensitive to how you phrase requests. The same underlying question can yield
vastly different results depending on prompt structure. Good prompts unlock capabilities; poor prompts waste
potential.

Key Prompting Techniques

Zero-shot prompting asks the model to perform a task without any examples.

Example: "Classify the sentiment of this review as positive or negative: 'The food was amazing!'"

Few-shot prompting provides examples before the actual task.

Example: "Review: 'Great movie!' → Positive | Review: 'Terrible service.' → Negative | Review: 'The book was
fascinating.' → ?"

Chain-of-Thought (CoT) prompting encourages step-by-step reasoning by adding phrases like "Let's think
step by step" or showing worked examples. This dramatically improves performance on reasoning tasks.

Role prompting assigns a persona to the model: "You are an expert data scientist..." This frames the response
style and expertise level.

Instruction prompting gives explicit directions about format, length, and style: "Explain in simple terms
suitable for a 10-year-old."

Prompt Components
A well-structured prompt often includes: Context/background information, Clear instructions about what to do,
Input data to process, Output format specification, and Examples (for few-shot).

Best Practices
Be specific and explicit—ambiguity leads to inconsistent results. Use delimiters (quotes, brackets, XML tags)
to clearly separate different parts of the prompt. Iterate and refine based on outputs. Specify constraints like
word limits or forbidden content.

Prompt Engineering vs Fine-Tuning

Aspect Prompt Engineering Fine-Tuning

Model weights Unchanged Updated

Data needed None to few examples Labeled dataset

Compute cost Low High

Flexibility High (easy to modify) Lower (requires retraining)

Best for General tasks, rapid prototyping Specialized tasks, max performance
3. TRANSFORMERS: ARCHITECTURE OVERVIEW
The Revolution
Introduced in the 2017 paper "Attention Is All You Need," Transformers replaced recurrent architectures (RNNs,
LSTMs) and became the foundation for modern NLP. The key innovation: self-attention allows processing all
positions simultaneously rather than sequentially.

Core Components
Self-Attention Mechanism
Self-attention computes relationships between all words in a sequence. For each word, it asks: "How much
should I focus on every other word?"

Three vectors are computed for each token:

Query (Q): What am I looking for?

Key (K): What do I contain?

Value (V): What information do I provide?

Formula: Attention scores = softmax(QK^T / √d_k) × V

The scaling factor √d_k prevents extremely small gradients when dimensions are large.

Multi-Head Attention
Instead of single attention, multiple attention "heads" run in parallel. Each head can learn different relationship
types—one might capture syntactic relationships, another semantic ones. Outputs are concatenated and
linearly transformed.

Positional Encoding
Since Transformers process all positions simultaneously, they have no inherent sense of word order. Positional
encodings (sinusoidal functions or learned embeddings) are added to input embeddings to inject position
information.

Feed-Forward Networks
After attention, each position passes through identical feed-forward networks (two linear transformations with
ReLU activation). This adds non-linearity and processes attention outputs.

Layer Normalization and Residual Connections

Each sub-layer has a residual connection (adding input to output) followed by layer normalization. This
stabilizes training and allows gradients to flow through deep networks.

Encoder-Decoder Structure
Encoder: Processes input sequence, creates contextualized representations. Uses self-attention where each
position attends to all positions.
Decoder: Generates output sequence. Uses masked self-attention (each position can only attend to previous
positions) plus cross-attention to encoder outputs.

Architecture Variants
Encoder-only (BERT): Good for understanding tasks (classification, NER, question answering)

Decoder-only (GPT): Good for generation tasks (text completion, creative writing)

Encoder-Decoder (T5, BART): Good for sequence-to-sequence tasks (translation, summarization)

Why Transformers Excel

Parallelization: Unlike RNNs, all positions process simultaneously, enabling efficient GPU utilization.

Long-range dependencies: Self-attention directly connects distant words without information degradation.

Scalability: Architecture scales well to billions of parameters.

4. DATA COLLECTION & ANNOTATION
The Foundation of NLP
Data quality determines model quality. "Garbage in, garbage out" applies strongly to NLP—even the best
architecture fails with poor data.

Data Collection Methods

Web scraping gathers text from websites, forums, and social media. Considerations include [Link]
compliance, rate limiting, and handling HTML/noise.

APIs from platforms like Twitter, Reddit, or news services provide structured access to text data.

Existing corpora such as Wikipedia dumps, Common Crawl, and academic datasets offer pre-collected
resources.

Crowdsourcing through platforms like Amazon Mechanical Turk collects human-written text or labels.

Domain-specific sources like medical records (with privacy considerations), legal documents, or scientific
papers provide specialized text.

Data Preprocessing
Cleaning removes HTML tags, special characters, duplicates, and irrelevant content.

Normalization involves lowercasing, handling contractions, and standardizing formats.

Tokenization splits text into units (words, subwords, or characters). Subword tokenization (BPE, WordPiece,
SentencePiece) handles unknown words elegantly.

Handling imbalanced data through oversampling minority classes, undersampling majority classes, or using
weighted loss functions.

Annotation
What is annotation? Adding labels or metadata to raw text—sentiment labels, named entity tags,
part-of-speech tags, etc.

Annotation types: Classification labels (positive/negative/neutral), Sequence labels (B-PER, I-PER, O for
named entities), Span annotations (question answer spans), Relations between entities.

Ensuring Quality Annotations

Clear guidelines: Detailed instructions with examples and edge cases reduce annotator confusion.

Inter-annotator agreement: Multiple annotators label the same examples. Metrics like Cohen's Kappa or
Fleiss' Kappa measure consistency. Low agreement indicates ambiguous guidelines or subjective tasks.

Adjudication: Disagreements are resolved through discussion or expert review.

Iterative refinement: Guidelines evolve based on edge cases encountered.

Challenges
Subjectivity: Tasks like sentiment analysis involve genuine ambiguity.

Cost and scale: Human annotation is expensive and slow.

Bias: Annotator demographics and perspectives influence labels.

Domain expertise: Medical or legal annotation requires specialized knowledge.

Modern Approaches
Active learning selects the most informative examples for annotation, reducing total annotation needed.

Weak supervision uses heuristic rules, knowledge bases, or other models to generate noisy labels at scale.

Data augmentation artificially expands datasets through synonym replacement, back-translation,

paraphrasing, or generative models.
5. BERT AND PRE-TRAINED LANGUAGE MODELS
The Pre-training Revolution
Before BERT, NLP models trained from scratch on each task. Pre-trained language models changed this by
learning general language understanding from massive unlabeled corpora, then transferring this knowledge to
downstream tasks.

BERT (Bidirectional Encoder Representations from Transformers)

Released by Google in 2018, BERT achieved state-of-the-art results across 11 NLP tasks.

Key Innovation: Bidirectionality

Previous models like GPT were unidirectional (left-to-right). BERT is bidirectional—each word can attend to
words on both sides. This captures fuller context.

Pre-training Objectives
Masked Language Modeling (MLM): 15% of tokens are masked, and the model predicts them. This forces
understanding of context from both directions.

Example: Input: "The cat [MASK] on the mat" → Target: "sat"

Of the 15% selected tokens: 80% are replaced with [MASK], 10% with random tokens, 10% kept unchanged.
This variation prevents the model from learning that [MASK] always means "predict here."

Next Sentence Prediction (NSP): Given two sentences, predict whether the second actually follows the first.
This teaches document-level understanding.

BERT Architecture
BERT-Base: 12 layers, 768 hidden size, 12 attention heads, 110M parameters

BERT-Large: 24 layers, 1024 hidden size, 16 attention heads, 340M parameters

Special Tokens: [CLS] at the start (its embedding used for classification), [SEP] between/after sentences,
[MASK] for masked positions.

Other Important Pre-trained Models

GPT: Decoder-only, unidirectional, excels at text generation.

RoBERTa: BERT optimized—removed NSP, trained longer with more data, dynamic masking.

ALBERT: Lighter BERT with parameter sharing and factorized embeddings.

DistilBERT: Distilled (compressed) BERT, 60% smaller, 97% performance.

T5: Frames all tasks as text-to-text (e.g., "translate English to German: ...").

ELECTRA: Uses a discriminator to detect replaced tokens—more sample efficient than MLM.

Using BERT for Downstream Tasks

Text Classification: Use [CLS] token embedding with a classification head.
Named Entity Recognition: Use each token's embedding with a token-level classifier.

Question Answering: Predict start and end positions of answer span.

Semantic Similarity: Compare [CLS] embeddings or use sentence-transformers.

Transfer Learning Paradigm

Pre-training on large unlabeled corpus learns general language understanding. Fine-tuning on labeled
task-specific data adapts this knowledge. This requires far less task-specific data than training from scratch.
6. CURRENT TRENDS & RESEARCH
Large Language Models (LLMs)
Models have scaled dramatically: GPT-3 (175B parameters), PaLM (540B), GPT-4 (rumored trillions). Scale
brings emergent abilities—capabilities that appear suddenly at certain sizes, like in-context learning and
chain-of-thought reasoning.

Efficient Methods
Parameter-Efficient Fine-Tuning (PEFT): Methods like LoRA, adapters, and prefix-tuning update only a small
fraction of parameters, making fine-tuning accessible with limited compute.

Quantization reduces precision (32-bit to 8-bit or 4-bit), shrinking model size with minimal performance loss.

Knowledge distillation trains smaller "student" models to mimic larger "teacher" models.

Retrieval-Augmented Generation (RAG)

Combines generation with retrieval: the model retrieves relevant documents from a knowledge base before
generating responses. This grounds outputs in facts, reduces hallucination, and allows updating knowledge
without retraining.

Instruction Tuning and Alignment

Instruction tuning fine-tunes models on diverse tasks framed as instructions, improving zero-shot
generalization.

RLHF (Reinforcement Learning from Human Feedback) aligns model outputs with human preferences.
Process: collect human rankings of outputs, train a reward model, use reinforcement learning to optimize the
language model.

Multimodal Models
Models like GPT-4V, Gemini, and LLaVA process multiple modalities—text, images, audio. Vision-language
models combine visual understanding with language capabilities.

Challenges and Active Research

Hallucination: Models generate plausible-sounding but factually incorrect content.

Reasoning: Despite improvements, complex multi-step reasoning remains challenging.

Bias and fairness: Models inherit and can amplify biases from training data.

Interpretability: Understanding why models make specific predictions is crucial.

Context length: Extending effective context windows while maintaining efficiency.

Emerging Directions
Mixture of Experts (MoE): Only activates relevant "expert" subnetworks for each input, enabling larger models
with less compute per forward pass.

State Space Models (Mamba): Alternative to attention with linear scaling in sequence length.

Constitutional AI: Training models to follow principles rather than just examples.

Agentic systems: LLMs that can use tools, plan multi-step actions, and interact with environments.
PRACTICE QUESTIONS
Fine-Tuning Questions
Q1. A company fine-tuned BERT on a small dataset of 500 customer reviews for sentiment analysis. The
model achieves 95% training accuracy but only 68% test accuracy.
a) What problem does this indicate?

b) Suggest three specific solutions.

c) How would freezing layers help in this scenario?

Q2. Compare full fine-tuning versus LoRA in terms of:

a) Number of trainable parameters

b) Computational requirements

c) Risk of catastrophic forgetting

d) Use cases where each is preferred

Q3. Explain why a learning rate of 2e-5 is typically used for fine-tuning rather than 1e-3 (common in
training from scratch).

Prompt Engineering Questions

Q4. A student wants to use GPT to solve math word problems but gets incorrect answers with direct
questions. Describe how chain-of-thought prompting could help, and write an example prompt.

Q5. For the task of named entity recognition, design:

a) A zero-shot prompt

b) A few-shot prompt with 2 examples

c) Explain which would likely perform better and why

Q6. What is the key difference between prompt engineering and fine-tuning? In what scenarios would
you choose one over the other?

Transformers Questions
Q7. Explain the self-attention mechanism:
a) What are Query, Key, and Value vectors?

b) Why is scaling by √d_k necessary?

c) How does multi-head attention differ from single-head?

Q8. Why do Transformers need positional encoding? How would the model behave without it?

Q9. Compare encoder-only, decoder-only, and encoder-decoder architectures. Give one example model
and one suitable task for each.

Q10. What problem do residual connections solve in deep Transformer networks?

Data Collection & Annotation Questions
Q11. You're building a sentiment classifier for medical patient reviews.
a) What data collection challenges might you face?

b) Why is domain expertise important for annotation?

c) How would you handle privacy concerns?

Q12. Two annotators labeled 100 tweets for sentiment. They agreed on 72 labels.
a) What does this agreement level suggest?

b) What steps could improve inter-annotator agreement?

c) How would you handle the 28 disagreements?

Q13. Explain how active learning reduces annotation costs. Give an example scenario.

BERT and Pre-trained Models Questions

Q14. Describe BERT's two pre-training objectives:
a) Explain Masked Language Modeling with an example

b) Explain Next Sentence Prediction with an example

c) What does each objective teach the model?

Q15. How is BERT used for:

a) Text classification tasks

b) Named entity recognition tasks

c) What architectural additions are needed for each?

Q16. Compare BERT and GPT:

a) Directionality

b) Training objective

c) Best suited tasks

d) Architecture type

Current Trends Questions

Q17. Explain Retrieval-Augmented Generation (RAG):
a) How does it work?

b) What problem does it solve?

c) Give an example application.

Q18. What is RLHF? Explain its three main steps and why it's important for aligning language models
with human preferences.

Q19. A language model confidently states a false fact. This is called hallucination.
a) Why does hallucination occur?
b) Suggest two approaches to reduce it.

Q20. What is parameter-efficient fine-tuning? Why has it become important with the rise of very large
language models?

Mixed/Applied Questions
Q21. You're building a customer support chatbot.
a) Would you use fine-tuning or prompt engineering? Justify.

b) What data would you need?

c) How would you evaluate performance?

Q22. Trace the journey of text through a Transformer encoder:

a) Input embedding + positional encoding

b) Self-attention computation

c) Feed-forward network

d) Output

Q23. A model trained on English news articles performs poorly on social media text.
a) Identify this problem (domain shift)

b) Suggest solutions using concepts from the topics covered

Good luck with your exam!