■ DEEP LEARNING
The Coolest Guide for Kids (and Curious Humans!)
Based on CISC 867 — Lectures 1 & 2 • Queen's University
What You'll Learn Today
■ Image Classification • K-Nearest Neighbors • Linear Classifiers • Datasets • Hyperparameters • The
History of AI • Why GPUs are awesome • And why your face should NOT be used to judge if you
deserve a job!
Who Is This For?
■ This guide is for ANYONE who wants to understand how computers learn to recognize cats, dogs, cars,
and basically everything you see on Instagram. No PhD required. No brain cells harmed. Pinky promise!
■
'Without data, we have nothing.' — Prof. Hazem Abbas ■
Page 1 • Deep Learning for Kids ■
CHAPTER 1: What Even IS AI? ■
Imagine you have a robot friend. You want it to recognize your pet cat. Back in the old days, you'd have to
write a HUGE list of rules: "if the thing has pointy ears AND whiskers AND meows, it's a cat!" This is called
an Expert System — basically, you hire a smart person, ask them everything they know, then write it all as
if-else statements. Super boring. Super hard. Super bad at edge cases. ■
DEEP LEARNING says: "Forget the rules! Just show me 1 million cat photos and I'll figure it out myself."
That's the magic. The computer LEARNS from DATA, not from a rulebook.
The AI Family Tree ■
■ Year ■ Event ■ Why It Matters
1943 McCulloch & Pitts invent artificial neurons First ever brain cell copy!
1958 Rosenblatt invents the Perceptron In the New York Times! A psychologist!
1969 Minsky & Papert say 'perceptrons are dumb' AI Winter #1 ■ Everyone gave up
1980s Fukushima invents New Cognitron The real father of CNN (often ignored ■)
Mid-80s Backpropagation invented! THE revolution. Everything exploded!
Early 90s Yann LeCun creates CNNs properly Images can now be understood!
2012 AlexNet crushes image competitions Deep Learning goes mainstream!
2015 ResNet beats human accuracy (3.57% vs 5.1%) Computers > Humans! ■
2018 Hinton, LeCun, Bengio win Turing Award The Nobel Prize of Computing ■
2024 Hinton wins actual Nobel Prize! AI literally wins Nobel! Wild!
■ Fun Fact: Fukushima invented the CNN idea in 1980, but didn't get credit. The people who popularized it later kinda... forgot to
mention him. Don't be that person in group projects! ■
Page 2 • Deep Learning for Kids ■
CHAPTER 2: The Magical World of Datasets ■
Before a computer can learn ANYTHING, it needs data. TONS of it. Think of it like studying for an exam —
you need practice problems! These collections of images+labels are called datasets.
■■ Dataset ■ Classes ■■ Train Images ■ Test Images ■ Size
MNIST 10 28×28
50,000 10,000
(Handwritten digits) (0-9) grayscale
10 32×32
CIFAR-10 50,000 10,000
(cats, dogs...) colour
100 50,000 10,000 32×32
CIFAR-100
(20 superclasses) (500/class) (100/class) colour
Variable
ImageNet 1,000 ~1.3 Million 100,000
(224×224 used)
MIT Places 365 scenes ~8 Million 328,500 256×256
1,623 chars
Omniglot 20/category — Various
50 alphabets
CIFAR-10 is your new best friend!
■ In this course (and in YOUR homework), you'll use CIFAR-10 the most. It has 10 classes: airplane,
automobile, bird, cat, deer, dog, frog, horse, ship, truck. Each image is tiny — only 32x32 pixels — but
there are 50,000 training images! That's like staring at 50,000 tiny photos. Your eyes would hurt. Good
■ The Machine Learning Recipe (3 Simple Steps!):
1■■ COLLECT DATA — Get images AND their labels (what the image IS)
2■■ TRAIN A CLASSIFIER — Feed the data into a machine learning algorithm
3■■ EVALUATE ON NEW IMAGES — Test if it learned properly on stuff it's NEVER seen before!
The Exam Analogy (THIS IS CRUCIAL!)
■■ Imagine I give you 10 solved math problems to study. If I then TEST you using those SAME 10
problems — you'd just memorize them! That's called OVERFITTING. In machine learning, the test data
must ALWAYS be new/unseen data. Never train and test on the same stuff. Never. Ever. I'm serious. ■
Page 3 • Deep Learning for Kids ■
CHAPTER 3: K-Nearest Neighbors (KNN) ■■
KNN is the laziest classifier in history — and I mean that as a compliment! ■ Here's the idea in one
sentence: "To classify a new image, just find the most similar training images and copy their label!"
■ Cat samples
■ Dog samples
■ Mystery point → KNN votes!
K-Nearest Neighbor: 'Who are my closest buddies?'
■ The yellow dot is a mystery image. KNN looks at its 'K' closest neighbours and takes a vote!
How KNN Actually Works ■
Training: Do... absolutely nothing. Just save all the images and their labels in memory. That's it. KNN is
sometimes called a 'lazy learner' because it does zero work during training. All the hard work happens at
test time!
Testing: For each new mystery image, calculate the distance to EVERY training image, find the K closest
ones, let them vote, and pick the winner. So if K=3 and 2 cats + 1 dog are closest → it's a cat! ■
The Two Distance Metrics You Must Know ■
Metric Formula Shape of 'equal distance' Nickname
Manhattan Distance
L1 Distance |pixel1 - pixel2| summed ■ Diamond / Rhombus
(like NYC streets!)
Euclidean Distance
L2 Distance sqrt((pixel1-pixel2)² summed) ■ Circle / Sphere
(straight line!)
NEVER use KNN with raw pixels in real life!
■ Here's a dirty secret: if you shift an image by 1 pixel, flip the color slightly, or cover part of it — the pixel
distance changes MASSIVELY, even though it's clearly the same object! Pixel distance is NOT how
humans think. Modern deep learning uses feature distances instead. KNN is for learning the concept,
Page 4 • Deep Learning for Kids ■
CHAPTER 4: The K Problem — Hyperparameters! ■■
So... what value of K should you use? 1? 3? 7? 100? And which distance metric? These are called
hyperparameters — settings you choose BEFORE training that affect how the model behaves. They're not
learned from data; you set them yourself.
Parameter vs Hyperparameter — What's the Difference?
■ PARAMETER = part of the model itself (like the weights W in linear regression y=Wx+b). The model
learns these from data automatically. HYPERPARAMETER = stuff YOU set externally, like K in KNN, or
the learning rate. KNN has ZERO parameters but TWO hyperparameters: K and the distance metric!
The Big Question: How to Choose K? 3 Wrong Ways + 1 Right Way ■
Approach How? Good or Bad? Why?
Pick K that works best on K=1 always wins (memorizes everything)
Idea 1 ■ ■ TERRIBLE
training data This is overfitting!
Pick K that works best on You're 'peeking' at the test!
Idea 2 ■ ■ ALSO BAD
test data No idea how it'll do on truly new data
Split: Train + Validation + Test Validation acts as a 'fake test'.
Idea 3 ■ ■ GOOD!
Choose K using Validation only Real test used ONCE at the very end!
Cross-Validation (k-fold) ■ BEST Average of 5 runs = more reliable!
Idea 4 ■■
Try each fold as validation (small datasets) But too slow for deep learning
■ THE GOLDEN RULE: Your test set is sacred. Touch it only ONCE, at the very, very end. Think of it as the final boss — you only
get one shot! ■
What Happens When K Changes? ■
■ K=1 (small K): Very jagged decision boundaries. Easily confused by noise and mislabeled data. Can
create 'islands' — an isolated wrong-class region in the middle of another class. That weird island =
overfitting! ■■
■ K=3 or K=5 (medium K): Smoother boundaries. The mislabeled island disappears because 2 out of 3
neighbours vote for the correct class. But some uncertain 'white zones' appear.
■ Very large K: Even smoother, but too many uncertain zones. Also, K should be ODD to avoid ties in
voting! (Even K can lead to 50-50 stalemates — awkward! ■)
Page 5 • Deep Learning for Kids ■
CHAPTER 5: The Curse of Dimensionality ■
Here's a nightmare scenario. Imagine you want your KNN to work perfectly on 32×32 color images. That
means each image has 3,072 dimensions (32 × 32 × 3 colors). To have enough training examples to cover
all that space... you'd need more images than there are atoms in the universe. Literally. ■
Points needed for
■ Dimensions Example
uniform coverage
1D (a line) 4 points 4 numbers on a ruler
2D (a square) 4² = 16 points 4×4 grid
3D (a cube) 4³ = 64 points 4×4×4 grid
3,072D (32×32 RGB image!) 4^3072 points More than atoms in universe! ■
32×32 Binary Images = 10^308 possibilities
■ The number of possible 32x32 BINARY (black/white) images is 2^(32×32) = 10^308. For reference, the
number of elementary particles in the ENTIRE visible universe is only 10^97. So collecting enough
training data to 'fill the space' is literally impossible. This is why we need deep learning — it finds clever
Page 6 • Deep Learning for Kids ■
CHAPTER 6: The Linear Classifier ■
Time to level up! Instead of memorising ALL training images (KNN style), the Linear Classifier learns a
mathematical formula to separate classes. It's the foundation of ALL modern deep learning!
The Magic Formula: f(x, W) = Wx + b
Let's break this down like a pizza recipe ■:
■ x = your input image, flattened into a giant vector. A 32×32×3 image becomes 3,072 numbers in a
column.
■ W = the weight matrix. Shape: (10 × 3072) for CIFAR-10. This is what the model LEARNS. 10 rows
because 10 classes!
■ b = the bias vector. Shape: (10 × 1). It's like an offset — allows the model to shift its predictions up or
down.
■ f(x,W) = the output: a vector of 10 scores, one per class. Highest score = predicted class!
# Python example: Linear Classifier forward pass
import numpy as np
# image: 32x32x3 → flatten to 3072 numbers
x = [Link]() # shape: (3072,)
# W: weight matrix, b: bias
W = [Link](10, 3072) # 10 classes x 3072 features
b = [Link](10) # 10 biases
# Forward pass
scores = W @ x + b # shape: (10,)
predicted_class = [Link](scores) # highest score wins!
Three Ways to Think About It ■
■■ Viewpoint ■ What it means ■ Key idea
Algebraic f(x,W) = Wx + b Each row of W is a template.
Viewpoint Matrix multiplication Dot product = similarity score!
Visual Each row of W, reshaped to 32×32, The horse template actually looks
Viewpoint becomes a 'template image' kinda horsey after training! ■
Page 7 • Deep Learning for Kids ■
■■ Viewpoint ■ What it means ■ Key idea
Geometric Each class = one hyperplane One side = class A,
Viewpoint cutting through image space other side = class B. Simple!
The Bias Trick — Neat Little Shortcut!
■■ Instead of writing Wx + b separately, you can append the bias INTO the weight matrix and add a 1 to
the end of your input vector x. So x becomes (3072+1) = 3073 numbers, and W becomes (10 × 3073).
One multiplication instead of add+multiply. Computers love this! ■
Page 8 • Deep Learning for Kids ■
CHAPTER 7: Limitations & What Comes Next ■
Why Linear Classifiers Sometimes Fail Spectacularly
■
The linear classifier draws a single straight line (or hyperplane in many dimensions) to separate classes.
But some problems are NOT linearly separable! Classic example: the XOR problem (which is also the
XNOR problem that Minsky pointed out in 1969).
Problem Why it fails Visual hint
Class 1 in quadrants 1&3, No single line can separate them!
Like + split diagonally
Class 2 in quadrants 2&4 You'd need a cross-shape.
One class wraps around the other.
Class 1: ring around Class 2 Like a bullseye ■
A line can't split a donut!
Three separate regions for one class.
Class 1 has 3 separate blobs Like islands ■■■■■■
One line can only separate TWO sides.
The Deep Learning Performance Chart ■
Error Rate (%) — lower is better ■
28%
16%
11%
7%
5.1%
3.6%
2010 2012 2013 2014 2015 Human
(No DL) AlexNet ZFNet VGG ResNet
■ By 2015, ResNet (a deep CNN) achieved 3.57% error on ImageNet — beating the human benchmark of 5.1%! And we're now 11
years past that! ■
The Road Ahead: What This Course Covers ■■
Pixels Layer Layer 2 Scores
Input Hidden Hidden Output
Page 9 • Deep Learning for Kids ■
■ A neural network: signals flow from pixels (left) through hidden layers to final scores (right)
■ Topic ■ What You'll Learn
Loss Functions How does the model know it made a mistake? How do we measure 'wrongness'?
Optimization Gradient descent — how the model fixes its mistakes step by step
Backpropagation The magic algorithm that trains any neural network. THE most important thing!
Multi-Layer
Stack layers together to solve non-linear problems!
Perceptrons (MLP)
Convolutional Neural
The kings of image recognition. Inspired by your own eyeballs! ■
Networks (CNN)
Recurrent Networks
For sequences: text, speech, time series data
(RNN/LSTM)
Transformers &
The tech behind ChatGPT and basically everything in 2024+
Attention
Generative AI &
How AI creates images from text descriptions (Midjourney etc.)
Diffusion Models
Page 10 • Deep Learning for Kids ■
CHAPTER 8: Why NOW? — GPUs, Data & Frameworks ■
Deep learning algorithms have existed since the 1980s. So why did everything explode around 2012?
Three ingredients came together at the right time:
■ Ingredient ■ What it is ■ Fun Analogy
Like having 10,000 students
■■ GPUs Graphics Processing Units — originally made for doing
video math
games!simultaneously
They process thousands of matrix operations in PA
vs 1 really smart professor
Like having an infinite
■ Big Data The internet gave us billions of images, texts, videos.
library
ImageNet
of textbooks
alone:
to 1.3 MILLION labeled images!
learn from ■
Like LEGO for AI —
■ Frameworks PyTorch & TensorFlow let you build neural networks
just in
snap
justpieces
a few lines
together
of Python code.
and go! ■
GPUs Were Made for Games — Seriously!
■ GPUs were originally designed so that video games could smoothly transition between levels without
that annoying flicker. Turns out, the same parallel math that renders game graphics is PERFECT for
neural network calculations (which are all matrix multiplications). The cost of one 'FLOP' went from
■ Quick Framework Cheatsheet
Framework Company Good for? Still used?
PyTorch ■ Meta (Facebook) Research & most courses YES — #1 in research!
TensorFlow Google Production & mobile YES — industry fav
Keras Open Source Beginner-friendly wrapper Yes (on top of TF)
Caffe Berkeley Early CNN experiments Mostly deprecated ■
Page 11 • Deep Learning for Kids ■
CHAPTER 9: With Great Power Comes Great Responsibility ■■
Before you go build the next Skynet, we need to talk about the dark side of AI. This stuff is real, it's
happening NOW, and your professor WILL ask about it.
The Gorilla Problem — Real Racism in AI
■ A Google photo app once classified Black people as gorillas. This is NOT just a 'oops' moment — it's a
symptom of biased training data and a lack of diverse teams building these systems. If your training
data doesn't represent all people equally, your model will be unfair. Full stop.
Face Scanning for Job Applications — Also Real
■■ Companies have used facial recognition to decide if you deserve a job interview. Not your skills. Not
your grades. YOUR FACE. This has been shown to discriminate against women and people of colour.
When you build AI, you MUST ask: who could this harm? ■
The 3 Questions Every AI Builder Must Ask ■
■ Why are these tasks challenging? — What makes classification hard? Lighting, occlusion, viewpoint
changes, scale, background clutter...
■ What other real-world problems can deep learning solve? — Medical diagnosis, autonomous cars,
climate science, accessibility tools...
■ What ethics apply? — Privacy, bias, accountability, explainability, consent, job displacement,
surveillance...
■ BONUS: Quick Cheatsheet — Problem Type →
Solution
■ Problem ■ Use This ■ Key Formula/Code
Classify an image dist = sum(|img1 - img2|) → L1
K-Nearest Neighbors
(basic, first approach) dist = sqrt(sum((img1-img2)²)) → L2
Classify an image scores = W @ x + b
Linear Classifier
(parametric) predict = argmax(scores)
Split: 70% train, 10% val, 20% test
Choose best K value Cross-validation
Pick K with best val accuracy
Image has 3 channels
Flatten to 1D vector x = [Link]() # 32×32×3 → 3072
(colour)
Page 12 • Deep Learning for Kids ■
■ Problem ■ Use This ■ Key Formula/Code
Model memorizes
You have overfitting! Fix: more data, regularization, bigger K
training but fails test
■ That's Lectures 1 & 2 — You Now Know More Than Most Adults About AI! ■
Next up: Loss Functions & Optimization — how the model actually learns to fix its mistakes!
'Starting next class, we begin half an hour late — 4:30pm. Everyone needs time to digest lunch... and this guide!' — Prof. Hazem
Abbas ■
Page 13 • Deep Learning for Kids ■