Chapter 1: Introducing Deep Learning: Biological and Machine Vision
[Link] Vision
Five hundred fifty million years ago, in the prehistoric Cambrian period, the number
of species on the planet began to surge. There was a time called the Cambrian period.
Back then, life on Earth was mostly in the oceans with tiny, simple creatures.
Suddenly, during this period, a huge number of new species appeared.
Scientists call this sudden increase the Cambrian explosion.
One of those ancient sea animals was the trilobite, a small, crab-like creature.
It developed light detectors, the first simple eyes.
Before eyes, animals could only sense nearby things using touch or smell.
But with vision, trilobites could now:
• See food from far away,
• Spot enemies before being attacked,
• Notice mates to reproduce.
Vision worked faster and more accurately than smell or touch almost like a superpower in
that ancient world.
Once trilobites could see, their predators and prey had to adapt to survive.
• The prey had to hide better or move faster.
• The predators had to hunt more cleverly.
This back-and-forth competition made species evolve rapidly, leading to the explosion of
new life forms.
Over hundreds of millions of years, eyes became more and more advanced.
In modern mammals, including humans, a large part of the brain’s outer layer (the cerebral
cortex) is used just for seeing and understanding what we see.
In the late 1950s, two scientists David Hubel and Torsten Wiesel were working at Johns
Hopkins University.
They were physiologists, their goal was to find out how the brain processes visual
information.
They focused on a part of the brain called the primary visual cortex ,this is the first area in
the brain that receives signals from the eyes.
Think of it as the brain’s “first screen” where raw images from the eyes are shown and
processed.
To study this, Hubel and Wiesel used cats because cat brains have a similar visual system to
humans.
Here’s what they did:
1. They showed images (like slides) on a screen in front of cats.
2. The cats were anesthetized (put to sleep so they wouldn’t move).
3. The scientists recorded signals from single neurons (brain cells) in the cat’s visual
cortex while the images were shown.
At first, they showed simple shapes, like dots, on the screen.
But to their surprise, nothing happened, the neurons in the visual cortex didn’t respond at all.
They were confused and frustrated because these brain cells were supposed to be the main
entry point for visual information, yet they seemed silent even when the cat’s eyes saw
something. Hubel and Wiesel discovered that the brain “builds” what we see step by step from
detecting simple lines to recognizing full shapes. Their work showed how our brain turns light
entering the eyes into meaningful images the foundation for modern neuroscience and even
computer vision today.
[Link] VISION
The figure (Figure 1.8) shows two different paths for machines trying to “see”:
It starts with trilobites, the first animals that developed simple eyes.
Then jumps to Hubel and Wiesel’s discovery (1959) ,showing how the brain processes
images layer by layer.
That idea of hierarchical vision became the foundation for artificial models of vision.
Deep Learning Path:
It represents modern machine vision using deep learning systems that automatically learn
visual features (edges, shapes, patterns) layer by layer, just like the brain.
These models don’t need humans to tell them what to look for; they learn directly from
images.
Traditional Machine Learning Path
This shows the older approach to computer vision:
• Engineers had to manually tell the computer what features to detect (like edges,
corners, colours).
• The computer could only use those limited instructions to recognize objects.
It worked, but it wasn’t flexible or powerful.
[Link] Neocognitron
The Neocognitron is an early artificial neural network created in the late 1970s by a
Japanese engineer named Kunihiko Fukushima.
It was the first computer model inspired by the human visual system , especially the
discoveries of Hubel and Wiesel, who showed how the brain processes what we see step by
step.
Biological Inspiration
Hubel and Wiesel had found that:
• Simple cells in the visual cortex detect edges and lines.
• Complex cells combine those edges into shapes and patterns.
Fukushima thought:
If the brain uses layers of cells to recognize what it sees, maybe a computer can do the same.
The Neocognitron processes images in layers, just like the brain.
Input Layer
Receives the raw visual image (like the eye sending signals to the brain).
Simple Cell Layer (S-Cells)
Detects basic features- straight lines, edges, or angles in different directions.
Complex Cell Layer (C-Cells)
Combines several simple features to detect bigger shapes or patterns, such as corners or
curves.
Higher Layers
Each higher layer forms more complex and abstract representations, eventually recognizing
full objects (like a face, number, or animal).
So, information flows step by step —
from simple → to complex → to complete recognition.
Fukushima’s model learned automatically it didn’t need humans to program specific features.
It could adjust its own connections through training, just like the brain strengthens or weakens
neuron connections.
The Neocognitron was the foundation for modern Deep Learning, especially:
• Convolutional Neural Networks (CNNs) — used today in computer vision systems.
• Applications like face detection, self-driving cars, and virtual try-on tools
[Link]-5
LeNet-5 is one of the earliest and most famous Convolutional Neural Networks (CNNs),
created by Yann LeCun and his team in 1998.
It was designed to recognize handwritten digits, such as those in the MNIST dataset
LeNet-5 was a breakthrough because it showed that a computer could learn to see without
being told what features (edges, curves, etc.) to look for.
LeNet-5 has seven layers (not counting the input), and each layer extracts more complex
features from the image.
Input Layer
• The model receives a grayscale image of a handwritten digit (e.g., “2”).
• Size: 32×32 pixels (slightly larger than the 28×28 MNIST digits to allow convolution
edges).
Convolution Layer (C1)
• Applies 6 filters of size 5×5.
• Detects simple patterns like lines, edges, and curves.
• Output: 6 feature maps (28×28).
Subsampling Layer (S2)
• Performs average pooling with a 2×2 filter.
• Reduces size to 14×14, keeping essential information.
• Helps make the model less sensitive to position changes.
Convolution Layer (C3)
• Applies 16 filters of size 5×5.
• Detects combinations of edges — more complex shapes (like loops or corners).
• Output: 16 feature maps (10×10).
Subsampling Layer (S4)
• Another average pooling layer.
• Reduces to 5×5 feature maps.
• Keeps the most important features while reducing data size.
Fully Connected Layer (C5)
• Flattens the previous layer and connects all neurons.
• Learns to combine features into meaningful patterns (e.g., “this looks like a 2”).
• Has 120 neurons.
Fully Connected Layer (F6)
• Further refines the representation.
• Has 84 neurons, similar to the number of neurons in biological visual systems (like
Hubel and Wiesel discovered).
Output Layer
• Has 10 neurons, each representing one digit (0–9).
• The neuron with the highest activation corresponds to the predicted number.
[Link] Traditional Machine Learning Approach
Even though LeNet-5 worked well for handwritten digit recognition, researchers thought it
wouldn’t work for more complex images (like faces, animals, or cars).The main criticism: deep
learning models automatically learned features, but that process seemed too slow and
limited at the time. So researchers shifted back to traditional machine learning, where
humans designed the features manually — a process called feature engineering.
Feature engineering means:
Humans decide which parts of the image are important, and then convert those into measurable
features that a machine learning model can use.
For example:
• Instead of feeding an entire photo of a face into the model, you first extract features
like:
o The contrast between eyes and cheeks
o The edge of the nose
o The distance between mouth and eyes
These engineered features are then given to simpler algorithms like:
• Support Vector Machines (SVM)
• Decision Trees
• Random Forests
Example: Viola–Jones Face Detection (2001)
Paul Viola and Michael Jones created a famous face detection algorithm that used engineered
features.
• They used rectangular filters (black-and-white bars) that move across an image.
• Each filter detects simple visual patterns like:
o Vertical edges (e.g., sides of the nose)
o Horizontal edges (e.g., eyebrows or mouth)
• These filter responses become features.
• Then a traditional ML model uses these features to decide if the image contains a face.
These black-and-white rectangles are shown in Figure 1.13 — they are called Haar-like
features.
Limitations of Traditional ML
Even though feature engineering was successful, it had some problems:
• Required many years of human research for each new object type (e.g., faces, cars,
animals).
• Needed domain experts to design new filters and algorithms.
• Could not easily generalize — a model trained to detect faces couldn’t detect houses
without redesigning all features again.
[Link] and the ILSVRC
ImageNet is a massive collection of labeled images created in 2009 by Fei-Fei Li and her
team at Princeton University. It was designed to give machine learning models the kind of
large, diverse, and well-organized data that humans naturally experience.
Each image in ImageNet is labeled with the object it contains — for example:
• “Cat”
• “Airplane”
• “Orange”
• “Electric guitar”
Before ImageNet:
• Datasets were small and narrow — like MNIST, which had only 70,000 handwritten
digits.
• Neural networks like LeNet-5 couldn’t be trained on large-scale, real-world data
because such data didn’t exist publicly.
ImageNet changed this.
It provided massive, labeled data for the first time, allowing computers to learn general
visual understanding — not just digits or faces.
Starting in 2010, Fei-Fei Li launched the ImageNet Large Scale Visual Recognition
Challenge (ILSVRC) — an annual competition for testing computer vision models.
• Researchers used a subset of ImageNet:
➢ 1.4 million images
➢ 1,000 categories
• These included many similar objects (like different dog breeds), to test how precisely
a model could distinguish between closely related images.
The goal:
→ Train a computer to correctly identify what is shown in an image — for example, “This is
a golden retriever, not a Labrador.”
AlexNet is a deep convolutional neural network (CNN) that classifies images into 1,000
categories from the ImageNet dataset.
It has 8 layers that have learnable parameters — 5 convolutional layers and 3 fully
connected layers.
Input Layer
• Input: 227 × 227 × 3 image (height × width × RGB channels)
• Purpose: Feed the image pixels into the network.
• The image is preprocessed (resized, normalized, and augmented).
Convolutional Layer 1
• Filters: 96 filters of size 11 × 11 × 3
• Stride: 4
• Output: 55 × 55 × 96
• Activation: ReLU
• Followed by: Max Pooling (3 × 3 window, stride 2)
• Purpose: Detects basic features like edges, corners, and colors.
Convolutional Layer 2
• Filters: 256 filters of size 5 × 5 × 48
• Output: 27 × 27 × 256
• Activation: ReLU
• Followed by: Max Pooling (3 × 3 window, stride 2)
• Purpose: Learns more complex features like textures and shapes.
Convolutional Layer 3
• Filters: 384 filters of size 3 × 3 × 256
• Output: 13 × 13 × 384
• Activation: ReLU
• Purpose: Recognizes object parts and patterns.
Convolutional Layer 4
• Filters: 384 filters of size 3 × 3 × 192
• Output: 13 × 13 × 384
• Activation: ReLU
• Purpose: Learns fine-grained details and complex shapes.
Convolutional Layer 5
• Filters: 256 filters of size 3 × 3 × 192
• Output: 13 × 13 × 256
• Activation: ReLU
• Followed by: Max Pooling (3 × 3 window, stride 2)
• Purpose: Combines all learned features into a final feature map.
Flattening
• Converts the 3D feature maps into a 1D vector for feeding into fully connected
layers.
Fully Connected Layer 1
• Neurons: 4096
• Activation: ReLU
• Uses Dropout (p = 0.5) to prevent overfitting.
• Purpose: Learns combinations of features for classification.
Fully Connected Layer 2
• Neurons: 4096
• Activation: ReLU
• Dropout applied again.
• Purpose: Further refines feature relationships.
Fully Connected Layer 3 (Output Layer)
• Neurons: 1000 (for 1000 ImageNet classes)
• Activation: Softmax
• Purpose: Produces final class probabilities.