SUB:Generative AI

Unit II : Generative models for Text

Language Model Basics (Generative AI)

A language model in Generative AI is a type of machine learning model that

is used to generate human-like text and predict the next word in a sequence
based on the given input.

Language models are the core components of generative AI systems. They

are trained on large amounts of text data and learn the patterns, grammar, and
structure of language. Using this knowledge, they can generate meaningful
and coherent sentences.

The main goal of a language model is to estimate the probability of a

sequence of words and use this probability to generate new text. This allows
machines to create responses that are contextually relevant.

for example:
Input: “Artificial Intelligence is”
Output: “transforming the world”

This shows how the model generates text based on learned patterns.

Language models are widely used in generative AI applications such as:

 Chatbots (like ChatGPT)

 Text generation
 Content writing
 Code generation
 Question answering systems

Mathematically, a language model represents:

P (w1, w2,... wn)

Where each word depends on the previous words in the sequence.

Modern generative AI systems use advanced language models such as
transformer-based models, which can understand long context and generate
high-quality text.

In conclusion, language models are fundamental to generative AI, as they

enable machines to generate, understand, and interact using human language
effectively

Building Blocks of Language Model

Language models are built using several fundamental components called
building blocks. These components work together to help the model
understand, process, and generate human language effectively.
Understanding these building blocks is important for designing and analyzing
modern NLP systems.
Tokenization
Tokenization is the first step in building a language model. It is the process of
breaking text into smaller units called tokens.
These tokens can be:
 Words
 Sub words
 Characters
For example:
“Language models are powerful” → [Language, models, are, powerful]
Tokenization helps the model process text in a structured format.
Vocabulary
Vocabulary refers to the set of all unique tokens used by the language model.
(Vocabulary is the set of all unique tokens in the dataset)
A small vocabulary reduces complexity but may miss important words.
A large vocabulary improves coverage but increases computational cost.
Special tokens are also included, such as:
 <UNK> (unknown word)
 <PAD> (padding)
 <SOS> (start of sentence)
 <EOS> (end of sentence)
 Word Embeddings
Word embeddings are used to convert words into numerical vector
representations.
Machines cannot understand text directly, so each word is represented as a
vector of numbers.
Embeddings capture semantic relationships between words.
For example:
king – man + woman ≈ queen
Popular embedding techniques include:
 Word2Vec
 GloVe
Context Representation
Language models rely on context, which means understanding the
relationship between words in a sentence.
Example:
“The bank is near the river” vs “I deposited money in the bank”
Here, the word “bank” has different meanings depending on context.
Models use context to predict the correct meaning and next word.
Model Architecture
The architecture defines how the model processes input data. Common
architectures include:
 Feed forward Neural Networks
 Recurrent Neural Networks (RNNs)
 Long Short-Term Memory (LSTM)
 Transformers
Modern language models mainly use transformer architecture, which handles
long-range dependencies efficiently.
Training Data
Language models require large amounts of text data for training.
Data can come from books, websites, articles, etc.
More data improves model accuracy
Poor-quality data may introduce bias
Objective Function
The objective of a language model is to predict the next word in a sequence.
During training, the model minimizes a loss function (usually cross-entropy
loss).
This helps the model improve its predictions over time.
Optimization Algorithm
Optimization algorithms are used to update model parameters.
While training a language model, the model makes predictions for the next
word. These predictions may not always be correct. The difference between
the actual output and the predicted output is called error (loss).
Common optimizers include:
 Gradient Descent
 Adam Optimizer
They help in minimizing the error during training.
Evaluation Metrics
Evaluation metrics are used to measure model performance.
The most common metric is:
Perplexity
 Measures how well the model predicts text
 Lower perplexity indicates better performance
Regularization Techniques
These techniques are used to prevent over fitting. Regularization Techniques
are methods used to prevent over fitting in a language model and to improve
its performance on new, unseen data.
In simple words, regularization helps the model not to memorize the training
data too much, but to learn general patterns.
Examples:
 Dropout
 Early stopping
What is Overfitting?
Overfitting occurs when a model:
 Performs very well on training data
 But performs poorly on new data
This happens because the model memorizes data instead of understanding it.
Inference (Text Generation)
Inference is the stage where the trained model is used to generate text or
make predictions.
The model predicts the next word step by step to form a complete sentence.
Transformer Architecture:
The Transformer architecture is a deep learning model introduced for
processing sequential data such as text. It is widely used in Generative AI and
Natural Language Processing (NLP) tasks like machine translation, text
generation, and question answering.

The Transformer model was introduced in the paper “Attention is All You
Need” (2017). It removed the need for recurrent networks (RNNs) and
instead uses a mechanism called self-attention to understand relationships
between words in a sentence.

Before Transformers, models like RNN and LSTM were used for sequence
processing. However, they had limitations such as:

 Difficulty in handling long sentences

 Slow training due to sequential processing
 Poor performance on long-range dependencies

To overcome these issues, Transformer architecture was introduced, which

allows parallel processing of words and better context understanding.

The Transformer model consists of two main parts:

1. Encoder
2. Decoder
The encoder reads and understands the input sentence.
The decoder generates the output sentence.
For example, in translation:
Input: English sentence
Output: Telugu sentence
Main Components of Transformer
Input Embedding

Words are first converted into numerical vectors called embeddings. These
embeddings represent the meaning of words in vector form

Positional Encoding

Since Transformers do not process words sequentially, positional encoding is

added to give information about the position of words in a sentence.

For example:
“The cat sat” and “sat cat the” have different meanings, so position is
important.

Self-Attention
Self-Attention is the most important component of a Transformer model,
based on the Attention Mechanism.
It allows each word in a sentence to focus on other words and determine their
importance while understanding context.
Each word is converted into three vectors:
o Query (Q)
o Key (K)
o Value (V)
The Query of one word is compared with Keys of all words.
Scores are calculated to find importance.
These scores are applied to Values to get the final output.
Example:
Sentence:
“The dog chased the cat because it was fast.”
 The word “it” can refer to either dog or cat.
 Self-attention helps the model identify that “it” refers to “dog” by
giving more importance to relevant words.
Multi-Head Attention
Multi-Head Attention is an extension of the Attention Mechanism used in
Transformer models.
It allows the model to focus on different parts of a sentence simultaneously
by using multiple attention mechanisms in parallel.
Example
Sentence:
“The boy is playing cricket in the ground.”
Different attention heads may focus on:
 Head 1 → grammatical structure
 Head 2 → subject-action relation (boy → playing)
 Head 3 → object relation (playing → cricket)

Feed Forward Neural Network (FFN)

The Feed Forward Neural Network (FFN) is an important component of the
Transformer model used after the Attention Mechanism.
It is a fully connected neural network applied independently to each word in
the sequence.
Working of Feed Forward Neural Network (FFN)
 The FFN takes the output from the attention layer.
 It passes this output through two fully connected (dense) layers:
o The first layer increases (expands) the dimension of the input.
o An activation function like ReLU is applied to introduce non-
linearity.
o The second layer reduces the dimension back to the original size.
 This process is applied independently to each word (token) in the
sequence.

The FFN refines the output of the attention layer by transforming each
word’s representation, making the model more powerful and accurate.

Add & Norm:

Add & Norm adds the input of a layer to its output and then normalizes the
result to ensure stable and efficient training.
Working
 The output from a sub-layer (like attention or FFN) is taken.
  The original input to that sub-layer is added to the output (residual
connection).
 The result is then passed through Layer Normalization.
This can be represented as:
Output=LayerNorm(x+Sublayer(x))
Encoder
An encoder is a part of a neural network that:
Takes input text (or data)
Converts it into a meaningful numerical representation (context vector /
embeddings)
In simple terms:
Encoder = “Reader” → understands the input
Key Functions:
Processes the entire input sentence
Captures context & meaning
Handles relationships between words using attention

Decoder
A decoder:
Takes the encoded representation
Generates output text step-by-step
In simple terms:
Decoder = “Writer” → produces output
Key Functions:
Generates one word at a time
Uses previous outputs + encoder context
Predicts next token

Encoder–Decoder Architecture

Used in tasks like:

Machine Translation
Text Summarization
Chat bots
Workflow:

1. Input sentence → Encoder

2. Encoder creates context vector
3. Decoder uses this vector
4. Output sentence is generated

Encoder in Transformers

1. Input Embedding
Computers don’t understand words directly, so each word is converted
into numbers (vectors)
This process is called embedding
Example:
“cat” → [0.2, 0.8, 0.5, …]
Key Point:
Words with similar meaning have similar vectors
Helps model understand semantic meaning
2. Positional Encoding
Transformer doesn’t read words in order like RNNs
So we add position information to each word
Example:
“I love AI” ≠ “AI love I”

How it works:
Adds a position value to embeddings
Uses sine & cosine functions (mathematical patterns)
3. Self-Attention Mechanism

This is the most important part

Each word looks at all other words in the sentence

Decides which words are important
Example:
Sentence: “The dog chased the cat because it was fast”
“it” refers to → “dog”
Self-attention helps identify this relation
How it works:

Uses 3 vectors:

Query (Q)
Key (K)
Value (V)
Formula:
Attention = similarity (Q, K) → applied to V

4. Feed Forward Network

It is a small neural network
Applied to each word separately
Same network is used for all words
Simple meaning:
“Each word is processed individually to improve its meaning”

Decoder in Transformers

Masked Self-Attention
Masked self-attention allows each word in the output to look only at the
words that come before it, not the future words. This restriction is applied
using a mask so that the model generates the sentence step by step from left
to right. It prevents the model from “cheating” by seeing future words during
training.

Example:
If the output sentence is “I love AI”, while predicting the word “love”, the
model can only see “I” and not “AI”. This ensures proper sequence
generation.

Encoder–Decoder Attention
Encoder–decoder attention helps the decoder use the information from the
encoder. While generating each word, the decoder looks at the encoded input
sentence to understand its meaning and produce the correct output. This
creates a connection between the input and output sequences.
Example:
If the input sentence is “Hello” and the output is “Namaste”, while generating
“Namaste”, the decoder refers to the encoder’s representation of “Hello” to
understand its meaning and translate it correctly.

Feed Forward Network (FFN)

The feed forward network is a simple neural network applied to each word
independently after the attention layers. It further processes and improves the
representation of each word, making the output more accurate and
meaningful. Even though it works on each word separately, it plays an
important role in refining the final result.

Example:
After attention understands the sentence “I love AI”, the FFN processes each
word like “I”, “love”, and “AI” individually to improve their internal
representation and make the prediction stronger.

Attention Mechanism

The attention mechanism is a key concept in Generative AI that helps models

focus on the most important parts of the input while producing output. Earlier
sequence models like RNNs tried to compress the entire sentence into a
single vector, which often led to loss of important information, especially in
long sentences. Attention solves this problem by allowing the model to
dynamically decide which words are more important at each step. In simple
terms, it works like human attention—when we read or answer a question, we
focus more on relevant words and ignore less important ones.

Basic Idea of Attention

The main idea behind attention is to assign different importance (weights) to
different words in a sentence. Not all words contribute equally to the meaning
of a sentence, so the model learns to focus more on important words. For
example, in the sentence “The dog chased the cat because it was fast,” the
word “it” may refer to “dog,” and attention helps the model understand this
relationship by focusing more on relevant words. This improves the model’s
ability to understand context and meaning.

Working of Attention (Query, Key, Value)

The attention mechanism works using three vectors: Query (Q), Key (K), and
Value (V). Each word in the sentence is converted into these vectors. The
Query represents the word we are currently focusing on, the Key represents
all words in the sentence, and the Value contains the actual information. The
model compares Query with all Keys to find similarity and assigns weights
accordingly. These weights are then applied to the Values to produce the final
output. In simple terms, the model checks which words are important and
give them more weight.

Self-Attention Mechanism
Self-attention is a type of attention where each word in a sentence attends to
all other words in the same sentence. This helps the model understand
relationships between words regardless of their position. For example, in the
sentence “I love AI,” the word “love” can attend to both “I” and “AI” to
understand the full meaning. Self-attention is widely used in transformer
models because it captures context very effectively and works well for long-
range dependencies.

Masked Self-Attention
Masked self-attention is used in the decoder part of transformer models. In
this mechanism, a word is allowed to attend only to previous words and not
future words. This is done using a mask to prevent the model from seeing
future information during training. For example, while generating the
sentence “I love AI,” when predicting the word “love,” the model can only
see “I” and not “AI.” This ensures proper step-by-step generation of text.

Encoder–Decoder Attention

Encoder–decoder attention connects the encoder and decoder in transformer

models. In this case, the decoder uses the encoded input information to
generate output. While producing each word, the decoder refers to the
encoder’s output to understand the input sentence better. For example, in
translation, if the input is “Hello,” the decoder uses this encoded meaning to
generate the correct output like “Namaste.” This helps in maintaining correct
alignment between input and output.

Importance in Generative AI

Attention mechanism is the backbone of modern Generative AI models like

GPT and BERT. It helps these models understand context, handle long
sentences, and generate accurate outputs. Because of attention, transformers
can process data in parallel, making them faster and more efficient compared
to older models. It is widely used in applications like translation,
summarization, chatbots, and text generation.

Advantages and Limitations

The attention mechanism improves context understanding, handles long-

range dependencies, and increases accuracy in language tasks. It also allows
parallel processing, which makes training faster. However, it can be
computationally expensive when dealing with very long sequences, as it
requires comparing every word with every other word.
Generation of text
Learning from Data:
Text generation works by training AI models on large amounts of text data.
These models analyze thousands or even millions of documents to understand
language patterns, sentence structures, and relationships between words.
Similar to how a student learns a language by reading and practicing, AI
models learn from data to produce meaningful text.

Pattern Recognition:
During training, the model identifies common patterns in language such as
grammar, context usage, and word order. This helps the model understand
how words are typically used together in sentences.

Next Word Prediction:

When a user provides a prompt, the model generates text by predicting the
most probable next word or phrase based on learned patterns. This prediction
happens step-by-step, one word at a time.

Use of Context:
The model considers previous words (context) while generating the next
word, ensuring that the output is coherent and logically connected.

Advanced Generation:
This process is similar to predictive text on smart phones, but much more
advanced. By continuously predicting and refining each word, AI models can
generate complete sentences, paragraphs, or even full articles with proper
flow and meaning.

Different Approaches to Text Generation

Autoregressive Models:
Autoregressive models generate text by predicting one word at a time based
on previously generated words. Each new word depends on the earlier
sequence, ensuring logical flow and coherence. Models like GPT-4 follow
this approach, similar to how humans think of the next word while writing.

Sequence-to-Sequence (Seq2Seq) Models:

Seq2Seq models work by converting an input sequence into an output
sequence. They are widely used in applications like machine translation,
where a sentence in one language is transformed into another language. This
approach is useful when structured input needs to be mapped to structured
output, maintaining meaning and context.

Fine-Tuned Models:
Fine-tuned models are pre-trained models that are further trained on domain-
specific data. This customization helps them generate more accurate and
relevant text for specific fields such as medical, legal, or financial domains.
Fine-tuning improves the model’s performance for specialized ta

BERT
BERT (Bidirectional Encoder Representations from Transformers) is a
powerful language model developed to improve natural language
understanding. It is based on the transformer architecture and has achieved
state-of-the-art performance in many NLP tasks. The main objective of
BERT is to understand the meaning and context of words in a sentence more
accurately, especially in applications like search engines and question
answering systems.
Working of BERT
BERT works by reading text in both directions, that is, from left to right and
right to left simultaneously. This bidirectional approach helps the model
understand the full context of a word based on its surrounding words. It is
built using only the encoder part of the transformer architecture, which
focuses on understanding input text rather than generating new text. During
processing, the input sentence is converted into tokens, and special tokens
like [CLS] (classification token) and [SEP] (separator token) are added to
help in different tasks.
Training Techniques
BERT is pre-trained using two important techniques. The first is Masked
Language Modeling, where some words in a sentence are hidden and the
model learns to predict them based on context. The second is Next Sentence
Prediction, where the model learns to determine whether one sentence
logically follows another. These training methods help BERT develop a deep
understanding of language structure and relationships.
Importance of BERT
BERT plays a crucial role in improving the way machines understand human
language. It allows systems to interpret the meaning of words based on
context, which reduces ambiguity and improves accuracy. It has significantly
enhanced search engines like Google by enabling better understanding of
user queries. Additionally, BERT performs exceptionally well in tasks such
as question answering, sentiment analysis, and text classification, making it a
widely used model in NLP applications.

Applications of BERT
BERT is used in a variety of real-world applications including search
engines, chat analysis, text classification, sentiment analysis, and named
entity recognition. It is especially useful in systems that require deep
understanding of language rather than text generation.
GPT
Generative Pre-trained Transformer (GPT) is a powerful language model
used in Generative AI that can both understand and generate human-like text.
It learns patterns, relationships, and structures of language from large
amounts of data and is capable of performing multiple language-related tasks
using a single model. GPT can read and interpret text, generate meaningful
and natural-sounding sentences, and produce contextually relevant responses.
Due to its training on vast datasets, it can perform tasks such as writing
content, summarizing information, answering questions, and even assisting in
coding. Its ability to understand how words relate to each other allows it to
maintain context and coherence while generating responses.
Architecture
1. Input Embedding
Input: The text is first broken into tokens (words or subwords).
Embedding: Each token is converted into a dense vector representation.
2. Positional Information
Since the model does not understand order by default, positional information
is added to the embeddings to preserve sequence order.

3. Decoder Blocks
Layer Norm: Each block starts with layer normalization.
Masked Multi-Head Self-Attention: The model attends only to previous
tokens using multiple attention heads.
Add & Norm: The attention output is added to the input (residual connection)
and normalized.
Feed Forward: A feed-forward network (Linear ->GeLU -> Linear) processes
the data further.
Add & Norm: Again, the output is added back and normalized.
4. Layer Stack

Multiple decoder blocks are stacked (e.g., 12 blocks) to capture deeper

patterns and relationships in the text.

5. Final Layers
Layer Norm: A final normalization is applied.
Linear: The output is mapped to the vocabulary size.
SoftMax: Probabilities are generated for predicting the next token or task
output.

GPT Evolution
GPT models have evolved rapidly, improving in scale, reasoning and real
world performance. Earlier versions focused on basic language patterns,
while newer models handle complex tasks more effectively.
1. Early Development
 GPT (2018): Introduced the idea of pre-training + fine-tuning for
language tasks.
 GPT-2 (2019): Improved text generation and coherence with larger
scale.
2. Scaling Phase
 GPT-3 (2020): Major leap in size and capability, enabling few-
shot and zero-shot learning
3. Advanced Capabilities
 GPT-4 (2023): Stronger reasoning, better context handling and
multimodal abilities.
4. Recent Improvements
 GPT-4.1 / 4.5 (2025): Better coding, longer context understanding and
improved instruction following.
5. Current Generation:
 GPT-5 (2026): A more advanced and efficient system that balances
speed and deep reasoning. It shows improvements in accuracy, reduced
errors, better understanding of complex tasks and stronger performance
across coding, reasoning and real world applications.
Applications
The versatility of GPT models allows for a wide range of applications,
including but not limited to:
 Content Creation: GPT can generate articles, stories and poetry,
assisting writers with creative tasks.
 Customer Support: Automated chat bots and virtual assistants
powered by GPT provide efficient and human-like customer service
interactions.
 Education: GPT models can create personalized tutoring systems,
generate educational content and assist with language learning.
 Programming: GPT's ability to generate code from natural language
descriptions aids developers in software development and debugging.
 Healthcare: Applications include generating medical reports, assisting
in research by summarizing scientific literature and providing
conversational agents for patient support.
Exploring ChatGPT

ChatGPT is an advanced AI-powered chatbot developed by OpenAI that is

capable of understanding and generating human-like text. It is based on the
Generative Pre-trained Transformer (GPT) model and is widely used for
conversational interactions, content generation, and problem-solving tasks.
ChatGPT can respond to user queries in a natural and interactive way,
making it a powerful tool in Generative AI.

Working of ChatGPT

ChatGPT works by taking user input in the form of text prompts and
generating responses based on learned patterns from large datasets. It uses a
transformer-based architecture and predicts the next word in a sequence to
produce meaningful and coherent responses. The model is trained using pre-
training and fine-tuning techniques, which help it understand context and
provide relevant answers.

Features of ChatGPT

ChatGPT can perform a wide range of tasks such as answering questions,

writing essays, generating code, summarizing text, and translating languages.
It supports conversational interaction, meaning it can remember context
within a conversation and provide continuous responses. It is also capable of
adapting its tone and style based on user input.

Applications of ChatGPT

ChatGPT is used in many real-world applications including education,

customer support, content creation, virtual assistants, and software
development. It helps students in learning, assists professionals in writing and
coding, and improves user interaction in digital platforms.

Prompt Engineering in Generative AI

Definition

Prompt engineering is the process of designing clear, specific, and effective

prompts that guide AI models like GPT-4 to generate accurate and relevant
responses. It mainly focuses on structuring input in a way that helps the
model understand the task and produce the desired output, especially in
Natural Language Processing tasks.

Importance of Prompt Engineering

Prompt engineering plays a crucial role in improving the quality of outputs

generated by AI models. When prompts are specific and well-structured, they
help the model better understand the requirements of a task, leading to more
accurate and meaningful responses. Detailed prompts reduce ambiguity and
ensure that the output closely matches the user’s expectations. Additionally,
effective prompts improve performance in NLP tasks and contribute to more
efficient interaction between the user and the model.

What are Prompts?

Prompts are short pieces of text given as input to guide AI models in

generating responses. They provide context, instructions, and direction to the
model. Well-designed prompts help in producing outputs that are relevant
and aligned with the intended purpose. They improve accuracy by clearly
defining user intent, avoiding overly vague instructions, and maintaining a
balance between too little and too much information. Proper prompts ensure
that the model focuses only on the relevant context while generating
responses.

How Prompt Engineering Works

Prompt engineering works by carefully designing instructions for the AI
model, similar to guiding an assistant to perform a task. The process begins
with crafting a prompt that clearly defines the requirement, including context
and expected output. Different techniques such as zero-shot prompting,
where no examples are provided, and few-shot prompting, where a few
examples are included, can be used depending on the task. The process often
involves refining the prompt through trial and error, adjusting wording and
structure until the desired output is achieved.

Applications of Prompt Engineering

Prompt engineering is widely used in various NLP applications. In language

translation, well-designed prompts help in accurately converting text from
one language to another by specifying context and style. In question-
answering systems and chatbots, prompts play a key role in generating
relevant and precise responses based on user queries. In text generation tasks,
prompts help control the tone, structure, and purpose of the generated
content, making them essential for applications like content writing,
summarization, and communication systems.

Reinforcement Learning from Human Feedback (RLHF)

Reinforcement Learning from Human Feedback (RLHF) is a training

approach used to align machine learning models, especially large language
models like GPT-4, with human preferences and values. Instead of relying
only on labeled data, RLHF uses human feedback such as rankings and
evaluations to guide the learning process. This helps the model generate
responses that are not only accurate but also helpful, safe, and aligned with
human intent.

Working of RLHF

RLHF works through a structured process where human feedback is used to

improve model behavior. It focuses on making AI outputs more aligned with
what humans expect by continuously refining responses using reward-based
learning.

Stages of RLHF

1. Supervised Fine-Tuning (Initial Learning Phase)

In this stage, a pre-trained language model is fine-tuned using high-quality
examples created by humans. These examples consist of prompt-response
pairs that teach the model how to respond correctly. This step helps the
model understand instructions better, reduces randomness, and builds a
strong foundation for further improvement.

2. Reward Model Training (Human Feedback Integration)

In this stage, human evaluators compare and rank different outputs generated
by the model. These rankings are used to train a reward model, which assigns
a score to each response based on its quality, helpfulness, safety, and
accuracy. The reward model acts as a guide for the main model during further
training.

3. Policy Optimization (Reinforcement Learning Phase)

In the final stage, reinforcement learning techniques are applied to improve
the model’s responses. The model generates outputs, and the reward model
provides feedback in the form of scores. Algorithms like Proximal Policy
Optimization (PPO) are used to update the model in a stable manner,
ensuring gradual improvement. This process helps the model produce better,
more consistent and human-aligned responses.

Applications of RLHF

1. Chatbots and Conversational AI: RLHF helps fine tune language

models like ChatGPT to generate more helpful, polite and context
aware responses based on human preferences.

2. Content Moderation: It enables AI systems to learn judgments from

human reviewers improving the detection and handling of harmful or
inappropriate content.
 3. Recommendation Systems: By integrating user feedback RLHF
refines recommendations to better reflect individual preferences and
evolving user behavior.

4. Autonomous Vehicles: It can be used to train self driving cars to make

safer and more human like driving decisions based on expert fee

What is Retrieval-Augmented Generation (RAG)?

Retrieval-Augmented Generation (RAG) is a way to make AI answers more

reliable by combining searching for relevant information and then generating
a response. Instead of guessing based only on old training data, it first finds
useful data from external sources (like documents or databases) and then uses
it to give a better answer.

 Fetches up-to-date data and reduces incorrect or made-up answers

 Works well with specialized data like medical or legal content
 No need to retrain the model every time new data comes in
 Can use user-specific data to give more relevant responses

For example, a platform like GeeksforGeeks has its own large collection of
coding articles and tutorials. If a user asks a question, instead of giving a
general answer, a RAG-based system can:

 Search relevant articles from their dataset

 Pick the most useful content
 Generate an answer based on that specific information
This way, the response is more accurate, aligned with the platform’s content
and actually helpful for the user.

Components of RAG

The main components of RAG are:

1. External Knowledge Source: Stores domain specific or general

information like documents, APIs or databases.

2. Text Chunking and Preprocessing: Breaks large text into smaller,

manageable chunks and cleans it for consistency.

3. Embedding Model: Converts text into numerical vectors that capture

semantic meaning.

4. Vector Database: Stores embeddings and enables similarity search for

fast information retrieval.

5. Query Encoder: Transforms the user’s query into a vector for

comparison with stored embeddings.

6. Retriever: Finds and returns the most relevant chunks from the
database based on query similarity.

7. Prompt Augmentation Layer: Combines retrieved chunks with the

user’s query to provide context to the LLM.

8. LLM (Generator): Generates a grounded response using both the

query and retrieved knowledge.

9. Updater (Optional): Regularly refreshes and re-embeds data to keep

the knowledge base up to date.

Working of RAG
The system first searches external sources for relevant information based on
the user’s query instead of relying only on existing training data.

1. Creating External Data: External data from APIs, databases or

documents is chunked, converted into embeddings and stored in a
vector database to build a knowledge library.

2. Retrieving Relevant Information: User queries are converted into

vectors and matched against stored embeddings to fetch the most
relevant data ensuring accurate responses.

3. Augmenting the LLM Prompt: Retrieved content is added to the user’s

query giving the LLM extra context to work with.

4. Answer Generation: LLM uses both the query and retrieved data to
generate a factually accurate, context aware response.

5. Keeping Data Updated: External data and embeddings are refreshed

regularly in real time or scheduled so the system always retrieves latest
information.
What Problems does RAG solve?

Some the problems that RAG solves are:

1. Hallucinations: Traditional generative models can produce incorrect

information. RAG reduces this risk by retrieving verified, external data
to ground responses in factual knowledge.

2. Outdated Information: Static models rely on training data that may

become outdated. It dynamically retrieves latest information ensuring
relevance and accuracy in real time.

3. Contextual Relevance: Generative models often struggle with

maintaining context in complex or multi turn conversations. RAG
retrieves relevant documents to enrich the context improving coherence
and relevance.

4. Domain Specific Knowledge: Generic models may lack expertise in

specialized fields. It integrates domain specific external knowledge for
tailored and precise responses.

5. Cost and Efficiency: Fine tuning large models for specific tasks is
expensive. It eliminates the need for retraining by dynamically
retrieving relevant data reducing costs and computational load.

6. Scalability across Domains: It is adaptable to diverse industries from

healthcare to finance without extensive retraining making it highly
scalable.

RAG Applications

Here are some examples to illustrate the applications of RAG we discussed

earlier:

1. Question-Answering Systems: It enables chatbots or virtual assistants

to pull information from a knowledge base or documents and generate
accurate, context aware answers.
 2. Content Creation and Summarization: It can gather information
from multiple sources and generate concise, simplified summaries or
articles.

3. Conversational Agents and Chatbots: It enhances chatbots by

grounding their responses in reliable data making interactions more
informative and personalized.

4. Information Retrieval: Goes beyond traditional search by retrieving

documents and generating meaningful summaries of their content.

5. Educational Tools and Resources: Provides students with

explanations, diagrams or multimedia references tailored to their
queries.

Multimodal Large Language Models

Multimodal large language models (LLMs) integrate and process various

types of data such as text, images, audio and video to enhance understanding
and generate responses. For example, an MLLM can interpret a text
description, analyze a corresponding image and generate a response that
encompasses both forms of input. This capability allows them to perform
tasks that require understanding of various types of data, making them more
versatile and powerful.

Key Components of Multimodal Large Language Models

 Data Integration: MLLMs use algorithms to combine data from

multiple sources, ensuring that the information from each modality is
accurately represented and integrated.

 Feature Extraction: The model extracts relevant features from each

type of input. For example, it might identify objects and their
relationships in an image while understanding the context and meaning
of accompanying text.
  Joint Representation: By creating a joint representation of the
multimodal data, the model can make inferences and generate outputs
that consider all available information.

 Cross-Modal Attention: Techniques like cross-modal attention help

the model focus on relevant parts of the data from different modalities,
improving its ability to generate coherent and contextually appropriate
responses.

Popular Multimodal Large Language Models (MLLMs)

GPT-4o
Works with text, images, audio, and video. Used in chatbots, content
creation, and accessibility. It gives real-time and natural responses but needs
high resources.

Gemini 2.5 Pro

Handles large data like text, images, and video. Used for coding, long
conversations, and data analysis. Very powerful but mainly for
advanced/business use.

Qwen 2.5 VL
Supports text and images with strong multilingual ability. Used in customer
support and education. Good language support but less popular globally.

Llama 4
Open-source model for text, images, and video tasks. Used in research and
customization. Flexible but requires strong hardware.

Claude 3.5 Sonnet

Focuses on safe and accurate responses with text and images. Used in reports
and research. Safe but limited in some features.

Issues of Large Language Models (LLMs)

 Hallucination:
Models like GPT-4 may generate incorrect or fake information that
appears true.
 Bias in Output:
If training data has bias, the model may produce unfair or biased
responses.
 Lack of True Understanding:
LLMs do not truly understand language; they only predict based on
patterns.
 High Computational Cost:
Training and running LLMs require large memory, power, and
expensive hardware.
 Privacy & Security Issues:
Sensitive or personal data may be exposed or misused.
 Harmful or Unsafe Content:
Models may generate inappropriate or misleading content if not
controlled.
 Inconsistent Responses:
Same question may produce different answers at different times.
 Dependence on Training Data:

Performance depends on quality and size of training data.