Deep Learning — Full Lecture Notes

■ Deep Learning
Full Lecture Explained Simply
Neural Networks · Architectures · Applications · Limitations

■ What is Deep Learning?

Deep Learning is a type of AI that uses Neural Networks — multi-layered computational structures
inspired by the human brain. The term "deep" refers to the many layers stacked on top of each other,
enabling the system to learn complex patterns from large amounts of data.

■ Simple analogy: Just like your brain has many layers of neurons that help you think, Deep
Learning has many layers of artificial neurons that help computers "think."

■■ Example: Face Recognition

Imagine showing a computer thousands of face photos. Here is how the layers process information:

Layer What it Does Real Example

Input Layer Receives raw photo Sees pixels, colors, edges

Hidden Layer 1 Finds basic features Detects eyes, nose, lips

Hidden Layer 2 Puts features together Recognizes it as a face

Output Layer Gives final answer "This is Ali's face!"

■ More hidden layers = ability to solve harder problems!

■ What are Neural Networks?

A Neural Network (ANN) is a computer system inspired by the human brain. Think of it like a group of
workers in a factory:

→ Each worker (neuron) receives information, processes it, and passes it forward
→ The strength of each connection is called a weight — like how much you trust each worker's opinion
→ The activation function decides whether a neuron should "fire" or stay quiet

Page 1
 Deep Learning — Full Lecture Notes

■ Example: Like a game of telephone — each person processes and passes the message,
and the weights decide how loud or quiet each voice is.

■■ Key Components: Weights, Bias & Sigmoid

Weights & Bias

The weighted sum formula is: z = (w1x1 + w2x2 + ...) + b

Weights (w) How important each input is

Bias (b) A correction value to fine-tune the result

Combined score = (marks × importance) + bonus

Sigmoid Function
Formula: y = 1 / (1 + e^(-z))
Converts any number into a value between 0 and 1 — like converting a raw score into a percentage (0%
to 100%).

Sigmoid Curve Behaviour:

Very negative z (e.g. −10) Output → 0 (near zero)

z near 0 Output ≈ 0.5 (uncertain)

Very positive z (e.g. +10) Output → 1 (near certain)

■ Worked Example

Given Values

Weights (w) [0.2, 0.3, 0.9]

Bias (b) 0.5

Inputs (x) [0.5, 0.6, 0.1]

Step 1 — Calculate z:
(0.5×0.2) + (0.6×0.3) + (0.1×0.9) + 0.5 = 0.10 + 0.18 + 0.09 + 0.5 =
0.87

Step 2 — Apply Sigmoid:

y = 1 ÷ (1 + e^(-0.87)) = 0.70

■ The neuron outputs 0.70 — meaning it is 70% confident!

Page 2
 Deep Learning — Full Lecture Notes

■ Types of Deep Learning Networks

Network Type Short Name

Feed Forward Neural Network FFNN

Recurrent Neural Network RNN

Convolutional Neural Network CNN

Deconvolutional Neural Network DNN

Restricted Boltzmann Machine RBM

Generative Adversarial Network GAN

Transformer Neural Network TNN

➡■ 1. Feed Forward Neural Network (FFNN)

Information flows in ONE direction only — forward, never backward.

■ Analogy: Like an assembly line — each worker does their job and passes it to the next,
never going back.

Uses: Data compression, pattern recognition, speech recognition, handwriting recognition, computer
vision.

■ 2. Recurrent Neural Network (RNN)

RNN has memory — it looks at the current input AND remembers previous inputs.

■ Analogy: Like reading a book — to understand the current sentence, you remember what
happened in previous sentences.

Uses: Machine translation, speech recognition, music composition, time series prediction, robot control.

■■ 3. Convolutional Neural Network (CNN)

CNNs are specially designed for images. They use three key layers to scan, compress, and classify
images.

Layer Job Simple Meaning

Like a magnifying glass sliding over a

Convolutional Scans image with a filter
photo

Page 3
 Deep Learning — Full Lecture Notes

Pooling Shrinks the image Keeps important info, removes extra detail

Connects everything and gives the

Fully Connected Makes final decision
answer

■ Analogy: Like a magnifying glass sliding over a photo, keeping important info and removing
extra detail.

Uses: Face recognition, tumour detection, video analysis, drug discovery.

■ 4. Deconvolutional Neural Network (DNN)

This is a CNN working in reverse — instead of shrinking images, it rebuilds them! Unpooling doubles
the image size back up step by step.

■ Analogy: CNN is like compressing a ZIP file. DNN is like unzipping it back to the original.

Uses: Image creation, art generation, image analysis.

■ 5. Restricted Boltzmann Machine (RBM)

RBM has only 2 layers — visible (input) and hidden. It is "restricted" because neurons only connect
across layers, not within the same layer.

Phase 1 (Forward): Input activates hidden layer. | Phase 2 (Backward): Hidden layer
reconstructs the input.

■ Analogy: Like a student (input) learning from a teacher (hidden layer), then the teacher
testing the student to see if they really learned it.

Uses: Computer vision, speech recognition, risk detection, business analysis.

■ 6. Generative Adversarial Network (GAN)

GAN creates brand new fake data that looks real! It has two competing parts.

Part Role Analogy

Generator Creates fake images A counterfeiter making fake money

Discriminator Judges real vs fake A detective catching fakes

■ They keep competing until the generator gets so good, even the detective cannot tell fake
from real!

Uses: Deepfakes, text-to-image generation, image style transfer, video prediction.

Page 4
 Deep Learning — Full Lecture Notes

■ 7. Transformer Neural Network (TNN)

Transformers are used in language tasks (like ChatGPT!). They process sentences in two stages:
Encoder (reads & understands context) and Decoder (generates output words one by one). A final
Softmax layer picks the word with the highest probability.

■ Analogy: Like a translator — they read the full original sentence (encoder), understand the
meaning, then write the translation word by word (decoder).

Uses: Language translation, chatbots, text generation, question answering, summarisation.

■ Deep Learning Applications

Application Example

■ Self-Driving Cars Tesla reads road signs, detects pedestrians

■■ Voice Assistants Siri understands your voice commands

■■ Image Captioning Auto-generates captions for photos

■ Translation Google Translate

■ Chatbots Customer service bots

■ Music Composition AI composing songs

■ NLP Understanding human language

■ Fraud Detection Banks catching suspicious activity

■■ Limitations of Deep Learning

Limitation Simple Meaning

Learns from observation only Can't reason like humans

Bias issues If data is biased, results are biased

Needs lots of data Needs millions of examples to learn

Expensive to train Requires powerful (costly) computers

No strong theory We don't fully understand why it works

Overfitting Memorizes training data but fails on new data

Data quality dependence Garbage in = Garbage out

Page 5
 Deep Learning — Full Lecture Notes

Difficult to understand "Black box" — hard to explain decisions

■■ Complete Summary Map

Deep Learning
■■■ Uses Neural Networks (brain-inspired)
■■■ Key Components: Weights, Bias, Sigmoid
■■■ Types of Networks:
■ ■■■ 1. FFNN → One-way flow (assembly line)
■ ■■■ 2. RNN → Has memory (reads books)
■ ■■■ 3. CNN → Images (magnifying glass)
■ ■■■ 4. DNN → Reverse CNN (unzipping)
■ ■■■ 5. RBM → 2-layer, forward + backward
■ ■■■ 6. GAN → Creates fake data (counterfeiter)
■ ■■■ 7. TNN → Language tasks (translation)
■■■ Applications: Cars, Voice, Translation, Chatbots ...
■■■ Limitations: Needs data, expensive, biased, black box

■ Bottom Line: Deep Learning is like teaching a computer to learn from experience, just like a
child learns by seeing, hearing, and practicing — but it needs a LOT of examples and powerful
computers to do it!

Page 6