TOPIC 1 – AI vs Machine Leaning vs Deep Learning patterns as they are exposed to more data

over time.
AI Breakthroughs - Ex. It labels emails as spam or not, these
programs learn from repeatedly seeing data.
- Image classification (At 2015, computers can
- Features (attributes of the data) and Target
be trained to perform better on this task than
(Column to be predicted)
human ‘dog and cat classification’
- Machine translation (As of 2016, we have Deep Learning Definition
achieved near-human performance using the
latest AI techniques ‘translate language’ - Machine learning that involves using very
complicated models called deep neural
Human Intelligence networks
- Models determine best representation of
- Use available information to take decision
original data, in classic machine learning,
- Communicate with people
human must do this
- Identify patterns in data
- Remember what people have said Difference of Machine Learning vs Deep Learning
- Adapt to new situations
- Ang machine learning kita mag extract sa
Artificial Intelligence feature, while deep learning ang mag feature
extraction is di kita
• AI develops computer systems that can
accomplish tasks that require human Machine Learning
intelligence
• Interacts with humans using their natural - The system is able to make predictions or
language take decisions based on past data
• Provides more accurate results
• Learn from their mistakes and adapt to new 6 Machine Learning Description
environments, Learns from the data and o Better decisions and predictions
automates repetitive learning Example. Alexa o Quick and accurate outcomes
o More powerful processing capability
Difference of the three o Analyzing complex big data
o Managing large amounts of data
➢ Artificial Intelligence – a program that can o Inexpensive
sense, reason, act and adapt
o Intelligent machines that think and 6 Deep Learning Descriptions
act like human
➢ Machine Learning – algorithms whose o Performance improves with more
performance improve as they are exposed to data
more data over time o Better scalability
o Systems learn things without being o Problems solved in an end-to-end
programmed to do so method
➢ Deep Learning – subset of machine learning o Best features are selected by the
in which multilayered neural networks learn system
from vast amounts of data o A subset of machine learning
o Machine thinks like human brains o Lesser testing time
using artificial neural network
Concepts of AI
Artificial Intelligence Definition
- NLP
- a branch of computer science dealing with the - Machine Learning (an approach to create
simulation of intelligent behavior in artificial intelligence)
computers. - Knowledge representation
- a program that can sense, reason, act and - Automated reasoning
adapt - Computer vision (used in achieving deep
- applied when a machine mimics cognitive learning)
functions that humans associate with other - Robotics
human minds

Machine Learning Definition

Story of AI
- the study and construction of programs that
are not explicitly programmed, but learn
“AI Winter” late 1960s-1970s (Early algorithms & 2. Limited Memory
Expert Systems) 3. Theory of Mind
4. Self-awareness
“AI Winter” late 1980s-1990s (Neural Networks,
Machine Learning, then Deep Learning Types of Machine Learning

- 1950s: Early AI 1. Supervised Learning

o 1950 – Alan Turing developed the 2. Unsupervised Learning
Turing Test to test a machine ability 3. Reinforcement Learning
to exhibit intelligent behavior
o 1956 - Artificial Intelligence was
accepted as a field of Dartmouth
Conference
o 1967 – Frank Rosenblatt invented
the perceptron algorithm. This was
the precursor to modern neural
networks.
o 1959 – Arthur Samuel published an
algorithm for a checker programs
using machine learning
- The First “AI Winter” (1966-1973)
- 1980’s AI Boom
o Expert Systems – systems with
programmed rules designed to
mimic human experts
o Backpropagation algorithm is able to
train multi-layer perceptrons leading
to a new success
- Another AI Winter (late 1980’s – early 1990s)
- Late 1990’s to early 2000’s: Classical
Machine Learning
o Advancement in SVM led to it
becoming the machine learning
method
o Deep Blue chess system beat Garry
Kasparov
- 2006: Rise of Deep Learning
o ImageNet (2019)
o Neural Networks are rebranded to
deep learning
- Deep Learning Breakthroughs (2012-
present)
o Deep learning beats previous
benchmark on ImageNet
competition
o Similar breakthrough appeared in
language translations (2014),
computer vision algo can describe
photos
o Deep learning platform TensorFlow
is developed (2015)
o DeepMind AlphaGo developed by
Aja Huang beats master Lee Se-Dol

Cutting Edge results in a variety of field

➢ Bigger datasets
➢ Faster computers
➢ Neural nets

Types of Artificial Intelligence

1. Reactive Machines
TOPIC 2 – Computer Vision Applications - More granular

Image Segmentation -> Object Detection -> Image Two types of segmentation
Classification & Object Localization -> Object
Recognition 1. Instance segmentation – identifying the
boundaries of object and label their pixel with
Object Recognition colors
2. Semantic segmentation – labeling each pixel
- Technique of identifying the object present in in image with different color based on
image and videos category or class label
- Teach machines to understand the content of
an image Application of Semantic

Image Classification ➢ Self-driving cards

➢ MRI scan segmentation
- Takes an image as an input and outputs ➢ Traffic Analytics
classification label ➢ RBC Cell Segmentation
- Convolutional Neural Network (convolution -> ➢ Product Classification
pooling -> fully-connected) ➢ Detecting and quantifying infected cells with
malaria
Applications of Image Classification
➢ Analyzing cancer biomarker
➢ Image tagging (google, fb, amazon)
➢ Visual Search
➢ Recommender engines
➢ Covid diagnosis
➢ Plant disease classification

Image Localization

- Identify the location of a single object in the

given image

Object Detection

- For multiple object present

- Predict location along with the class
- A combination of image classification and
object localization
- Produce one or more bounding boxes
- Capable enough to deal with multi-class
classification and localization

Challenges of Object Detection

- Bounding boxes are always rectangular

- Cannot accurately estimate some
measurement such as area and perimeter

Application of Object Detection

➢ Yield estimation
➢ Smile detection
➢ Mask detection
➢ Facial emotion analysis
➢ Parking Management system

Image Segmentation

- Understand what image consist of

- Divide or partition the image into segments
- Image – a collection or set of pixel
- Group pixel that have similar attributes
- Extension of object detection which we mark
the presence of object through pixel-wise
mask
TOPIC 3 – Artificial Neural Networks ➢ Forward propagation – data is propagated
through a network
➢ Soma – lignin na line ➢ Output layer – the neuron with the highest
➢ Dendrites – sanga sa soma value fires the output (probability)
➢ Axon – murag tube ➢ Back propagation – transferred back to the
➢ Synapses – ga connect sa axon ug lignin sa network
last

Definition
Remember, during training, predicted output is
- Neurons transfer the signal, synapse to compared with actual output. The error gives indication
another neuron and magnitude of change to reduce the error.
- Dendrites receive signal from another neuron
- Axon transmits the signal to next neuron - Weight are adjusted
- Chain stops at the last node which is brain - Cycle is iteratively performed
- Process continues until weight are assigned
Part sa ANN such that network can predict
- Input Layer
- Hidden Layer
- Output Layer

➢ Input nodes – information/input

➢ Hidden Layers – info being process, greater
number the deeper the network (perform
calculation)
➢ Output node – firing of the signal (predicts
final output)

McCulloh-Pitts model, 1949

- Firing and the strength of the exiting signal are

controlled by activation function (AF)
- Cell body (limiting or threshold function)
- Inputs to the signal are donated by Xn
- Each of inputs are multiplied by a connection
weight, Wn
- Sum = W1X1+W2X2+W3X3+…
- These products are summed, fed through
activation function to generate result

ANN (Process)

• Classify shapes
• Made up of layers of neurons
• Core processing units of networks
• Input layer received input, output predicts final
and hidden performs calculation
• 28x28 dimension of circle
• Each pixel is fed to neuron in input layer
• Neurons from 1st layer are connected to other
neurons
• Each of these channels is assigned value i.e
weight
• The inputs are multiplied to weights
• Sum served as input to neurons in hidden
layers

Each neurons is associated of numerical values i.e.

bias. Value is added to a threshold function called
activation function. The result determines if the
neurons is activated or not.
TOPIC 4 – Convolutional Neural Network (CNN) c. Flattening – process of converting all
the resultant 2 dimensional arrays
How image recognition works? from pooling feature map into a
single long continuos linear vector
Deep Learning recognized the object in an image using d. Flattened matrix is fed as input to
Convolutional Neural Network (CNN) fully connected layer
- Pixels of image fed as input to the input layer 4. Fully Connected Layer
-> hidden layers –> output layer a. The flattened matrix from pooling
layer is fed as input to the fully
connected layer to classify

➢ Input Layer accepts the pixels of the image as Every image is considered as matrix of pixel values.
input in the form of array Sliding the filter matrix over image and computing the
➢ Hidden layers carry out feature extraction by dot product to detect patterns
performing certain calculation and
manipulation PART 2: IMAGE CLASSIFICATION USING CNN
o Uses MATRIX FILTER and performs ➢ Input layer (1, 8, 28)
convolution operation to detect ➢ 1st layer: Conv2D consist of 32 layers and
patterns on image ‘relu’ af with kernel size (3,3)
o Layers of hidden layer ➢ 2nd layer: Conv2D consist of 64 layers,
▪ Convolution layer kernel size (3,3)
▪ ReLU layer ➢ 3rd layer: MaxPooling (2,2)
▪ Pooling Layer ➢ 5th layer: Flatten is used to flatten its
➢ Convolution – matrix filter inputs into single dimension
➢ ReLU activation function is applied to ➢ 6th layer: Dense has 128 neurons
convolution layer to get a rectified feature map ➢ Dropout has 0.5 value
➢ Pooling layer uses multiple filter to detect
edges, corners, etc. MNIST
➢ Fully connected layer identifies object in
image ✓ Modified National Institute of Standards and
Technology
Yann LeCun ✓ Built by Yann Le Cun et al.
✓ Composed of images that are handwritten
- Pioneer of CNN digits (0-9), split into:
- Director of FB AI research ✓ A training set of 50,000 images
- Built the first CNN called LeNet in 1988 ✓ A test set of 10,000 images
CNN is a feed forward neural network to analyze ✓ Each image is 28 x 28 pixels in width and
image. Also called “ConvNet” height.

- Convolution operations forms the basis of any Image Classification Using CNN: CIFAR-10
CNN ❖ Canadian Institute for Advanced Research
- Every image is represented in the form of ❖ 60,000 32 x 32 color images in 10 classes,
arrays of pixel values (0s and 1s) with 6,000 images per class
LAYERS IN CNN ❖ 50,000 training images and 10,000 test
images
1. Convolution Layer – has a number of filters ❖ Images are colored in CIFAR-10, as
that performs convolution operation compared to the black-and-white texture of
a. Sliding the filter matrix over the MNIST.
image and computing the dot ❖ Each image is 32 x 32 pixels.
product to detect paterns ❖ Images are taken in varying lighting
2. ReLU Layer conditions and at different angles, resulting in
a. performs element wise operation many variations in the color of similar objects.
b. sets all negative pixels to 0
c. Introduces non-linearity to the Model Improvements for CIFAR-10
network
• Increase the number of Conv2D layers to
d. Output is a rectified feature map build a deeper model.
3. Pooling Layer
• Increase the number of filters to learn more
a. Down sampling operation that
features.
reduces the dimensionality of the
• Add Dropout for regularization.
feature map
b. The rectified feature map now goes • Add more Dense layers.
through a pooling layer.
Additional CIFAR-10 Information ➢ Allowed for deeper and finer feature
extraction.
• CIFAR-100 is available in Keras. VGGNet
• Contains 100 classes.
❖ Proposed by Karen Simonyan and Andrew
Image Classification Using CNN: ImageNet Zisserman from the University of Oxford in
2014.
• Main database behind the ImageNet Large ❖ Input: Takes a 224 x 224 pixel RGB image.
Scale Recognition Challenge (ILSVRC). ❖ Convolutional layers:
• Known as the "Olympics of Computer Vision." o Very small receptive field (3x3).
• The competition that made CNNs popular o 1x1 convolution filters acting as
initially; every year, the best research teams linear transformations of the input.
across industries and academia compete with o ReLU activation unit.
their best algorithms on computer vision o Fixed stride of 1 pixel to preserve
tasks. spatial resolution after convolution.
❖ Fully connected layers: Three fully-
ImageNet Dataset connected layers with the first two having
4096 channels each and the third having
• Has more than 14 million images, hand-
1000 channels (one for each class).
labeled across 20,000 categories.
• Each image is 224 x 224 pixels.
• Processing a dataset of this size requires a GoogleNet (Inception)
significant amount of computing power in
terms of CPU, GPU, and RAM. Introduced by Szegedy et al. in "Going
Deeper with Convolutions" in 2014.
One of the major breakthroughs in neural
networks.
PART 3: Image Classification Using Transfer
Contains three versions of Inception
Learning
Networks (Inception Version 1, 2, and 3).
Evolution of Image Classification Developed by a team at Google.
Uses filters of multiple sizes that operate on
LeNet the same level, making the network wider
rather than deeper.
✓ Developed by Yann Le Cun in 1998 with 5 22 layers deep, with 27 pooling layers
layers. included.
✓ An input image is fed to the network.
✓ Filters of a given size scan the image and
perform convolutions. Inceptionv3
✓ The obtained features go through an
activation function. ▪ Introduced by Szegedy et al. in "Rethinking
✓ The output undergoes a succession of the Inception Architecture for Computer
pooling and other convolution operations. Vision" in 2015.
✓ Features are reduced in dimension as the ▪ Focuses on reducing computational power
network progresses. by modifying previous Inception
✓ At the end, high-level features are flattened architectures.
and fed to fully connected layers, which yield ▪ Suggested several techniques for optimizing
class probabilities through a softmax layer. the network to ease model adaptation,
✓ During training, the network learns to including factorized convolutions,
recognize the features that classify a sample regularization, dimension reduction, and
into a given class through backpropagation. parallelized computations.
AlexNet

➢ Developed by Alex Krizhevsky in 2012 with 8 SqueezeNet

layers.
➢ Introduced by Iandola et al. in "SqueezeNet:
➢ Introduced better non-linearity in the network
AlexNet-level Accuracy with 50x Fewer
with the ReLU activation function, which has
Parameters and <0.5MB Model Size."
a derivative of 0 for negative features and 1
➢ Proposed by researchers at DeepScale, the
for positive values.
University of California, Berkeley, and
➢ Introduced dropout as a regularization
Stanford University.
technique.
➢ A smaller network designed as a more
➢ Implemented data augmentation (random
compact replacement for AlexNet.
translation, rotation, crop) to make the
➢ Has almost 50x fewer parameters than
network more aware of image attributes.
AlexNet while performing 3x faster.
 ➢ Strategies include: • A machine learning technique where a model
o Using 1 x 1 filters instead of 3 x 3 filters. trained on one task is repurposed for a
o Decreasing the number of input second related task.
channels to 3 x 3 filters. • Transfer learning and domain adaptation
o Downsampling later in the network for refer to the situation where knowledge
larger activation maps. gained in one setting is exploited to improve
generalization in another.
• It improves learning in a new task by
ResNet transferring knowledge from a related task
❖ Introduced by He et al. in "Deep Residual that has already been learned:
Learning for Image Recognition" in 2016. o First, train a base network on a
❖ Introduced a residual learning framework to base dataset and task, then
improve training of networks that are repurpose the learned features to a
substantially deeper second target network for training
❖ To solve the problem vanishing/exploding on a target dataset and task.
gradients (degradation problem) by adding a o Tends to work best when the
skip/shortcut connection that adds the input features are general, suitable for
xxx to the output after a few weight layers. both base and target tasks, rather
than specific to the base task.

DenseNet
Steps in Transfer Learning
✓ Introduced by Huang et al. in "Densely
Connected Convolutional Networks" in 2017. 1. Select Source Model: Choose a pre-trained
✓ Jointly invented by Cornell University, model from available options. Many research
Tsinghua University, and Facebook AI institutions release models on large,
Research (FAIR). challenging datasets.
✓ Each layer receives additional inputs from all 2. Reuse Model: Use the pre-trained model as
preceding layers and passes its feature the starting point for the second task of
maps to all subsequent layers, creating a interest, using all or parts of the model as
"collective knowledge" effect. necessary.
• Advantages include:
3. Tune Model: Optionally adapt or refine the
o Strong gradient flow. model on the input-output pair data available
o Parameter and computational efficiency. for the task of interest.
o More diversified features.
o Maintenance of low complexity features.
MobileNet

Introduced by Howard et al. in "MobileNets:

Efficient Convolutional Neural Networks for
Mobile Vision Applications" by Google in
2017.
Features a very small, low-latency, and
efficient architecture for mobile and
embedded vision applications.
o Uses depthwise separable
convolutions along with two
hyperparameters (width multiplier
and resolution multiplier).
Depthwise separable convolution consists of
depthwise convolution and pointwise
convolution.
Demonstrates how to build smaller and
faster MobileNets by trading off a reasonable
amount of accuracy to reduce size and
latency.

Transfer Learning
TOPIC 5 – Object Detection

Object Detection

• computer vision technique in which a

software system can detect, locate, and
trace the object from a given image or video
• identifies the class of object (person, table,
chair, etc.) and their location-specific
coordinates in the given image
• location is pointed out by drawing a bounding
box around the object
• bounding box may or may not accurately
locate the position of the object
• the ability to locate the object inside an
image defines the performance of the
algorithm used for detection

TensorFlow Object Detection API

➢ open source framework built on top of

TensorFlow that makes it easy to construct,
train and deploy object detection models
➢ There are already pre-trained models in their
framework which are referred to as Model
Zoo
➢ includes a collection of pre-trained models
trained on various datasets such as the
o COCO (Common Objects in
Context) dataset
o the KITTI dataset
o and the Open Images Dataset

TensorFlow Object Detection API

❖ Data Collection
❖ Data Augmentation
❖ Image Annotation
❖ Training
❖ Testing
❖ Inference

Testing

Intersection over Union (IoU)

• evaluation metric used to measure the

accuracy of an object detector on a particular
dataset

Testing

Intersection over Union (IoU)

ground-truth bounding boxes - the hand

labeled bounding boxes from the testing set
that specify where in the image our object is
predicted bounding boxes - from the model
An Intersection over Union score > 0.5 is
normally considered a “good” prediction
Topic 6: Object Detection Network Architecture
How does object detection work
What is object detection? ➢ Single-stage object detectors – does
- Predict target class not include any intermediate
- Predict object location processing. It takes input image and
directly outputs the class and bounding
Image classification vs object detection box.
Image Classification o These are object detection that
- Object classification perform localization and
classification is a single step.
Object Detection o Does not require a separate
- Object Classification region. Predict bounding boxes
- Object Localization and class probabilities from he
- Single object (single class), multiple input image, making them
object (single class), multiple object significantly faster and suitable
(multiple class) for real time app.
o YOLO
Application o SDD
➢ Crown counting ➢ Two-stage object detectors – performs
➢ Self-driving cars two separate tasks. First involves a
➢ Video surveillance region proposal network that gives us
➢ Face detection the region of interest (ROI) where an
➢ Anomaly detection object might be present. Second is uses
➢ Detect Inspection ROI to produce bounding boxes.
o RCNN
Disadvantage o Fast RCNN
- Object that are elongated – use instance o Faster RCNN
segmentation o RFCN
- Object that have no physical presence – o Mask RCNN
use classification
- Object that have no clear boundaries at Yolo
different angles – use semantic - Single-shot detector, Using a deep
segmentation neural network (DNN), adapting the
- Objects that are often occluded – use output spaces generating score
instance segmentation if possible - Has a high accuracy score, is easy to
train. developed2015
Types and modes of object detection - Resizes input image into 448x448 in
- Before deep learning in 2003, almost all convolutional network
object detection was done through - 1x1 convolution is first applied to
classical ml techniques reduce the number of channels, then
o Viola-jones object detection 2x2. Activation function is ReLU, except
technique, scale-invariant on final, which uses linear activation
features transforms (SIFT), and function
histogram of oriented gradients - Residual blocks
- Detect a number of common features o Diving original image into grids,
across the image, and classify their each cell is responsible for
clusters using logistic regression, color localizing and predicting class
histogram and random forests
 along with the probability or
confidence value Faster RCNN
- Bounding box regression - Region-based cnn, uses cnn features,
o Determine bounding boxes innovative ways to use deep learning
which corresponds to rectangle - Started at 2013, developed by Microsoft
highlighting all the object - Deep convolutional network used for
o Have as many bounding box as object detection, that appears to the
there are object user as a single, end-to-end, unified
- Intersection Over Unions or IOU network. Can accurately and quickly
o The user defines its IOU predict the locations
selection threshold - RPN generates region proposals (takes
o Computes IOU which is the an image as input, and output a set of
intersection are divided by the rectangular object)
union are - A fixed-length feature vector is
o Ignore prediction of the grid extracted using ROI pooling layer
cells having an IOU <=threshold - Class scores are returned
and consider those with an IOU
> threshold Comparison
- Non-Max Suppression or NMS SSD
o Setting a threshold for the IOU ➢ Single shot, real-time, good balance,
is not enough coz an object can low complexity, efficient real-time
have multiple boxes performance
o We can use NMS to keep Faster R-CNN
highest probability score ➢ Two-stage, moderate speed, high
accuracy, high complexity, accurate but
Double-stage networks computationally intensive
1. Feature extraction YOLO
o Scanning the image to recognize ➢ Single-shot, real-time, moderate
basic shapes accuracy, low complexity, fast but
2. Generating several region proposals slightly lower accuracy
o Act like bunch of mini spotlight RetinaNet
o Generates thousands of small ➢ Single shot, real-time, high accuracy,
boxes (called ‘proposals’) moderate complexity, focal loss for class
around potential object imbalance
o Model then takes these EfficientNet
proposed regions and zooms in ➢ Single shot, high efficiency, high
on them to more precise accuracy, balanced, scalable
3. Classification architecture
o Model assign label to each Cascade R-CNN
region ➢ Multi-stage, moderate speed, very high
o Decides what each object accuraqcy, high complexity, multi-stage
refinement
Faster R-CNN
2 modules R-CNN
➢ Deep fully convolutional network that - Input image
proposes regions - Extract region(~2k)
➢ Fast R-CNN detector that uses the - Compute CNN features
proposed regions - Classify regions
Uses the CNN network on each proposed region

Fast R-CNN
- Uses CNN network once on the image
and classifies the region projections in
the feature map
- The region preproposal is an external
algo that was not tuned for object
detection
- Still quite slow for most of practical app
- Replaces the region proposal algorithm
- Feature map anchor points, it has
predefined box
- 512 channels, 3x3 kernel

Which anchor bounding box to use for a

specific object during training?

- Computing IoU between anchor box

and target object, by calculating are of
the intersection by diving the union
- Rule no. 1: IoU > 0.7 is positive
- Rule no. 2: IoU < 0.3 is negative
- Rule no 3: 0.5 < IoU <= 0.7 + no IoU >
0.7 is highest one positive
- Rule no. 4: 0.3 <= IoU <= 0.5 is not
positive not negative

YOLO

YOLO – You Only Look Once

- Is one of the most powerful real time object
detector algorithm
- It only needs an image to pass one time
through its network
- Uses a neural network using characteristics
of the entire image to predict multiple object
- Simultaneously
- Image is divided by S x S region, the region
in question is responsible for detecting object
- Each of the cells in this grid is responsible for
predicting B boxes
- If there are no object the score is zero, if
there is an object the score is qeual to the
intersection over union (IoU) between the
predicted box and the ground truth of the
image