INT428

ARTIFICIAL INTELLIGENCE
ESSENTIALS
Introduction To Deep Neural Networks
& Natural Language Processing
Unit Four
The Essence of Neural Networks
The Inspiration: Our brain has billions of neurons connected by synapses.

The Mimic: Artificial Neural Networks (ANN) mimic this structure.

The Goal: Pattern recognition (not just calculation).

Think of a Neural Network like a child learning to identify a dog.

Input: The child sees furry ears and a tail.

Processing: The brain says, 'Furry + Tail = Dog?'

Feedback: If the parent says, 'No, that's a cat,' the brain adjusts the
connection. Next time, it looks for a barking sound, too. That 'adjustment'
is exactly what an artificial neural network does during training."
 Deep Neural Networks

A Deep Neural Network is an Artificial Neural Network (ANN) with multiple hidden layers (usually more than two)
between the input and output.

➢ Shallow Network: Can learn simple, linear relationships (e.g., "If x > 5, buy stock").
➢ Deep Network: Can model complex, non-linear patterns (e.g., "This stock pattern looks like the crash of 2008").
 How DNNs Actually "See"
Key Concept: The magic of Deep Learning is Hierarchy. The network breaks complex data down into simple blocks
automatically.

➢ Low-Level Features: The first layers detect simple things (Lines, Edges, Colors).

➢ Mid-Level Features: The middle layers combine those lines into shapes (Curves, Corners, Textures).

➢ High-Level Features: The deep layers combine shapes into objects (Faces, Cars, Words).
DNN Cont.
 The Perceptron
A perceptron is a type of artificial neuron or the simplest form of a neural network. It is a model of a single neuron that
can be used for binary classification problems, which means it can decide whether an input represented by a vector of
numbers belongs to one class or another.

The Components:
➢ Inputs (x): The raw data (e.g., "Is it raining?", "Is it cold?").
➢ Weights (w): The importance of each input (e.g., Rain matters a lot, Cold matters a little).
➢ Bias (b): The threshold. (How easy is it to get a "Yes"?).

The Maths:
Output = Activation( ∑(weight × Input ) + Bias)
How Perceptron "Thinks": Linear Separability
The Logic:
➢ A Perceptron is a Linear Classifier.
➢ It tries to draw a single straight line to separate answers (Yes vs. No).

The Rule:
➢ Points Above the line = Class A (1)
➢ Points Below the line = Class B (0)

The Limitation
If the data is complex (like a curve or a
circle) or we have more than two
classes, a single Perceptron fails. This
is why we need Multi-Layer
Perceptrons (MLP).
 Multi-Layer Perceptron (MLP)
A Neural Network that stacks perceptrons together, adding at least one "Hidden Layer" between the input and output.
Why do we need it?
➢ A single Perceptron can only draw a straight line.
➢ MLPs can solve Non-Linear Problems (like the XOR checkerboard).

Key takeaway:
➢ It’s a "Universal Function Approximator" (it can theoretically learn almost any pattern given enough neurons).
Inside the "Black Box" What Hidden Layers Do
The Intuition:
➢ The Hidden Layer doesn't just pass data along; it transforms it.
➢ It acts as a Feature Extractor, finding clues and patterns that aren't obvious in the raw input.

The Result:
➢ The hidden layer twists and warps the data space so that the final output layer can easily draw a straight
line through it.
 Convolutional Neural Networks (CNN)
A specialized type of neural network designed to process data that has a grid-like topology, such as images (2D grid of pixels).

Why are they special?

➢ Standard neural networks (like MLPs) lose spatial information when you flatten an image into a long list of numbers.

➢ CNNs preserve the spatial relationship

between pixels (e.g., knowing that a pixel
representing an eye is next to a pixel
representing a nose).
➢ Key Application: Image recognition, object
detection, facial recognition, and powering
self-driving cars' vision systems.
 Working of CNN:
1. Input image:

Each number represents pixel intensity.

2. Convolution Layer (Feature Detection)
A small filter (kernel) slides over the image to detect
patterns like:
•edges
•textures
•shapes
•Corners

What happens:
[Link] moves across the image
[Link] values
[Link] them together
[Link] a new feature map
3. Activation Function (ReLU)
CNN applies an activation function.
Most common:
ReLU (Rectified Linear Unit)
Meaning:
•Negative values → 0
•Positive values → kept

Example:
Input:
[-3, 5, -2, 8]
Output after ReLU:
[0, 5, 0, 8]

Why?
✔ Removes unnecessary signals
✔ Makes learning faster
4. Pooling Layer (Dimension Reduction)
Pooling reduces the size of the feature map.
Common method: Max Pooling
Example:
Original feature map
4826
3951
7263
4815
After 2×2 Max Pooling
96
86
Benefits:
✔ Reduces computation
✔ Keeps important features
✔ Makes model faster
5. Flattening

After several convolution + pooling layers:

The feature maps are converted into a single long vector.
Example:
[2 5
7 9]
Flatten →
[2,5,7,9]
 Core Operation - Convolution (The "Filter")
The "Filter" (or Kernel):
➢ A small matrix of numbers (weights) that acts as a feature detector.
➢ The network learns the best numbers for these filters during training.

The Process:
➢ Place the filter over a patch of the input image.
➢ Perform element-wise multiplication and sum the results into a single number.
➢ Slide the filter over by one step (stride) and repeat.

The Result:
➢ A Feature Map. This shows where a specific feature (like a vertical edge) was found in the original image.
 Downsampling with Pooling (CNN Cont.)
The Problem: The Solution: The Pooling Layer
➢ After convolution, feature maps can get ➢ It reduces the spatial dimensions (width and height) of the
very large. data.
➢ They contain too much precise detail (e.g., ➢ Max Pooling (Most Common): As shown in the image, it
the exact pixel location of an eyelash). We looks at a small window (like a 2x2 colored zone) and keeps
need the "big picture." only the largest number.
Applications of CNN
 Recurrent Neural Network (RNN)
The Limitation of Standard Networks (Feedforward):
➢ They have "amnesia." They process one input, produce an output, and forget everything immediately.
➢ They cannot handle Sequential Data, where the order matters (e.g., Text, Speech, Stock Prices).
➢ Example: Trying to understand the word "bank" without knowing if the previous sentence mentioned a "river" or
"money."

The RNN Solution:

✓ RNNs have a "Recurrent

Connection"—a loop.
✓ This allows information to persist. The
output of step 1 becomes part of the
input for step 2.
The Key Concept:
RNN (Cont.)
➢ The Hidden State .This is the network's memory. It's a vector (a list of numbers) holding the context of everything seen
so far.

The Process at Every Time Step (t):

➢ The network receives two inputs simultaneously:
1. The New Data (X_t) (e.g., the current word being read).
2. The Old Memory (ht-1) (The context from the previous step).
➢ It combines them to create a New Memory (h_t) and an Output (Y_t).

The Technical Formula (Simplified):

New Memory = Activation(New Input + Previous Memory)
RNN (Cont.)
 INTRODUCTION TO TRANSFORMERS
The Origin: Introduced by Google Brain in 2017.
The Problem with RNNs: They were slow (sequential processing) and forgot long-term context (vanishing gradient).
The Transformer Solution:
➢ Parallelization: It reads the entire sentence at once, not word by word.
➢ Context: It sees the relationship between all words simultaneously.
Transformer’s Architecture
Transformer’s Architecture
Transformer’s Architecture
1. Input Embedding: First, the input sentence is converted into numerical vectors.
Example sentence:
I love AI
The model converts words into vectors:
I → vector
love → vector
AI → vector
This allows the computer to process language mathematically.

2. Positional Encoding: Transformers process all words at the same time, so they must know the
order of words.
Example:
Dog bites man
Man bites dog
Both have the same words but different meanings. So positional encoding adds information about
word position.
3. Encoder: The encoder extracts meaning from the sentence. Each encoder layer has two parts:
A. Self-Attention
B. Feed Forward Network

A. Self-attention helps each word look at other words in the sentence.

Example sentence:
The animal didn't cross the street because it was tired
The word “it” will pay more attention to animal rather than the street.
So the model understands:
it → animal

B. Feed Forward Network: After attention, the data passes through a small neural network to refine the
information.
So each encoder layer does:
Self Attention
↓
Feed Forward Network
This process is repeated multiple times (layers).
4. Decoder: The decoder generates the output sequence.
Example: language translation
The decoder uses:

1️⃣ Masked Self-Attention

2️⃣ Encoder–Decoder Attention:
3️⃣ Feed Forward Network
 The Decoder – Generating Output

➢ Masked Attention: The decoder can only see

words it has already generated, not future words
(no cheating!).

➢ Cross-Attention (Encoder-Decoder): The

specific layer where the Decoder looks back at
the Encoder's work to generate the next word.

➢ Role: Takes the Encoder's summary and

generates the final response (e.g., the
translation or the answer).
 Self Attention – The Heart of Transformer
The method by which the model decides which parts of the input are important for understanding the current
word is known as Self Attention.

How it works :(Q, K, V):

➢ Query (Q): What I am looking for?
➢ Key (K): What can I offer?
➢ Value (V): The actual content.
The Encoder – Understanding Context

➢ Positional Encoding: Since the model reads everything at once, it

needs a math trick to know that "Dog bites Man" is different from
"Man bites Dog.“
➢ Multi-Head Attention: Running several "attention" mechanisms in
parallel. One "head" focuses on grammar, another on vocabulary,
another on context.
➢ Role: The Encoder maps the input sequence into a rich, numerical
representation (vectors).
 Transformer’s Components
1. Positional Encoding (The "Timestamp"):
➢ Problem: Since the model reads all words at once, it doesn't know if "Man" came before "Dog".
➢ Solution: We add a unique mathematical pattern (Sine/Cosine waves) to each word's vector. This acts like a timestamp or
a page number.
2. Residual Connections (The"Shortcut"):
Output = Input + Function(Input).
➢ It creates a "superhighway" for data to flow through the network, preventing information from getting lost in deep
models.
3. Layer Normalization:
➢ Keeps the numbers stable and balanced at every step, ensuring the math doesn't explode during training.
Transformers (Cont.)
 NATURAL LANGUAGE PROCESSING (NLP)
➢ What is it? NLP is the branch of AI concerned with giving computers the ability to understand, interpret, and generate
human language in a valuable way.

➢ The Core Problem: Humans communicate in messy, context-dependent ways. Computers only understand rigid
mathematics. NLP is the translator between these two worlds.
 Ambiguities in NLP
➢ Syntactic Ambiguity: When the grammatical structure of a sentence allows for multiple interpretations (e.g., "I saw
the man with the telescope" – Who has the telescope?).

➢ Semantic Ambiguity: When a single word has multiple meanings based on context (e.g., The word "Bank" can mean
a financial institution or the side of a river).

The Takeaway: Rules aren't enough. Computers need to understand context to figure out meaning.
NLP (Cont.)
TOKENIZATION
 Embedding
An embedding is a dense vector representation of data in a continuous, high-dimensional space.
 ATTENTION
The Challenge (The Bottleneck):

➢ Traditional Sequence-to-Sequence (Seq2Seq) models (like RNNs or LSTMs) process an input sentence and compress all
information into a single, fixed-length Context Vector.

➢ Limitation: This vector struggles to retain details for long sentences, leading to "forgetting" early parts of the input.

The Solution (Attention):

➢ Instead of relying on one static vector, Attention allows the model to "search" and focus on the most relevant parts of
the input sequence dynamically at every step of generation.

➢ Analogy: When translating a sentence, a human translator doesn't memorize the whole sentence instantly; they look
back at specific words (e.g., "La pomme") when writing the translation ("The apple").
 The Foundation (Language Models)
➢A probability distribution over sequences of words. In simple terms, it assigns a
probability to a sequence of words to determine how "natural" or "likely" it is.
 Evolution of Language Models
The Evolution:
• N-Grams (Old Era): Simple statistical counting (e.g., "how often does
'apple' follow 'green'?"). Limited context.
• Neural LMs (New Era): Uses embeddings (dense vectors) to
understand semantic similarity (e.g., "King" - "Man" + "Woman" ≈
"Queen").
The Key Intuition:
• If you can predict the next word (or a missing word) accurately, you
must "understand" the language—syntax, grammar, and world
knowledge.
 How do we read text?
➢Option A (Auto-Regressive): Reading left-to-right. You don't know what comes
next. (Used by GPT).Eg: Chatbot, story writing, text generation.
➢Option B (Auto-Encoding): Looking at the whole sentence at once. You see the
past and future simultaneously. (Used by BERT). Eg: text classification, question
answering
 GPT: Generative Pre-trained Transformer
• GPT uses masked multi-head attention instead of a standard attention
mechanism.
• Masking hides future tokens, forcing predictions using only previous
words.
• This autoregressive design makes GPT excel at generating sequential text.
• GPT uses embeddings, attention layers, normalisation, and feedforward
networks.
• GPT performs prediction and classification, but excels at coherent human-
like text generation
 GPT
The Mechanism: Autoregressive (Unidirectional).

➢ It predicts the next token based only on previous tokens. It cannot "cheat" by looking ahead.
➢ Why it matters: This mirrors how humans speak or write text—one word at a time.
 The Evolution of GPT (Zero-Shot Learning)
Scale is Everything:

• GPT-1 to GPT-4 wasn't just about changing architecture; it was about massive
scaling of data and parameters.

Emergent Abilities
Zero-Shot Learning: The ability to do tasks it wasn't explicitly trained for.

• Example: You don't train it to "translate French." You just type "Translate this to
French:" and it figures it out because it has seen so much text.

Best Use Cases: Text Generation, Summarisation, Code Writing, Creative Dialogue.
 BERT
The "Bi-directional" Superpower:
➢ Most models read Left → Right.
➢ BERT reads Left ↔ Right simultaneously.
➢ Why it matters: To understand the word "Bank" in "Bank of the river" vs. "Bank of America," you need to
see the words after "Bank."
 BERT: Bidirectional Encoder Representations from Transformers

• BERT starts with text and positional embeddings.

• Embeddings pass through multiple transformer layers.
• Each layer has a multi-head attention mechanism.
• Attention attends to tokens before and after.
• Data flows through Add, Norm, Feed Forward.
• Classifier uses bidirectional context for tasks.
 How BERT Learns (The "Cloze" Test)
Training Objective 1: Masked Language Modeling (MLM):

➢ We hide 15% of words (replace with [MASK]) and force BERT to guess them.

➢ Example: "The [MASK] sat on the mat." -> BERT predicts "cat."

➢ Analogy: It’s like a "Fill in the Blanks" test in school.

Training Objective 2: Next Sentence Prediction (NSP):

➢ Does Sentence B logically follow Sentence A?

➢ Helps BERT understand paragraphs and logical flow.

➢ Best Use Cases: Classification, Sentiment Analysis, Question Answering (Extracting answers from text).
BERT Vs GPT
Summary (BERT / GPT)
 Introduction to Chatbots and Digital Assistants
Chatbots are software programs that simulate human conversation through text or voice, while digital assistants (like Siri
or Alexa) are advanced versions that handle tasks like setting reminders or controlling devices. They use AI to interact
naturally.
Difference between Chatbot and Digital Assistant

Feature Chatbot Digital Assistant

Purpose Simple Q&A, limited tasks Task completion + Q&A
Intelligence Often rule-based AI-powered, context-aware
Memory Usually short-term (session only) Can remember preferences and context
Integrates with apps, smart devices, calendars,
Integration Limited to a website or app
emails
Input Mostly text Text + voice
 Types of Chatbots
➢ Rule-based chatbots: These follow predefined rules and scripts. If a user asks a specific question, it matches keywords
and gives a fixed response. Simple to build but limited—can't handle unexpected questions. Example: A menu-driven bot
in a restaurant app where you select options.
➢ AI-based chatbots: Use machine learning to understand context and learn from data. They handle complex queries,
improve over time, and feel more human-like. Powered by NLP and neural networks. Example: ChatGPT, which generates
creative response
 Key Components of a Chatbot

A chatbot has several interconnected parts working like a team to process user input and respond.
 Natural Language Understanding (NLU)
✓What is NLU? Part of NLP that helps chatbots "understand" human language by
breaking down sentences into intent (what the user wants) and entities (key details
like names or dates).
✓Process: Tokenization (splitting words), part-of-speech tagging (noun/verb
identification), and intent classification using models like BERT. Example: User says
"Book flight to Delhi on Friday"—intent: book flight; entities: Delhi, Friday.
✓Challenges: Handles slang, typos, or ambiguity (e.g., "apple" as fruit or company).
✓Importance in chatbots: Without good NLU, responses are wrong—it's the "ears" of
the bot.
✓Key takeaway: NLU turns messy human talk into structured data for the bot to act on.
Working of NLU
 Natural Language Generation (NLG)
✓ What is NLG? The process of creating human-like responses from structured data. It's the "voice" of the chatbot.

✓ How it works: Templates for simple responses (e.g., "Your balance is [amount]") or advanced AI models like GPT for

creative text. Steps: Content planning, sentence structuring, and realization.

✓ Example: Input data: Weather=Sunny, Temp=25C → Output: "It's a sunny day with 25 degrees—perfect for a walk!"

✓ Importance: Makes bots engaging and personalized, not robotic.

✓ Key takeaway: NLG turns data into stories that users relate to.
HOW SENTIMENT ANALYSIS WORKS
 THE DEPLOYMENT PROCESS
❖Steps to deploy: Train the model, test with real users, host on cloud (e.g., AWS), and
monitor performance.

❖Integration: Connect to platforms like WhatsApp, websites, or apps via APIs. Example:
Embed in an e-commerce site for instant support.

❖Best practices: Ensure security (data privacy), scalability (handle many users), and
updates based on feedback. Tools: Heroku or Kubernetes.

❖Challenges: Compatibility with devices and handling high traffic.

❖Key takeaway: Deployment turns your bot from code to a real helper in the world.
THE DEPLOYMENT (AWS Example)
TRANSLATION
TRANSLATION (The BlackBox)
Inside The BlackBox
SUMMARIZATION