Chapter -5 –Advanced ML and Deep Learning
Intoroduction
Ensemble Learning ML
Ensemble techniques in machine learning involve combining
multiple individual models
(often referred to as base models or weak learners) to improve overall
predictive performance.
These techniques leverage the concept that combining diverse models
can often produce more accurate and robust predictions compared to
using a single model.
2
Ensemble Learning ML
Ensemble methods are widely used across various machine learning
tasks, including classification, regression, and anomaly detection.
3
Bagging
Bagging, short for Bootstrap Aggregating, is an ensemble learning
technique used to improve the stability and accuracy of machine
learning algorithms.
It works by combining the predictions of multiple base models, each
trained on a subset of the original dataset.
4
How Bagging Works
Bagging starts by creating multiple bootstrap samples from
the original dataset.
Bootstrap sampling involves randomly selecting samples from the
original dataset with replacement.
This means that some instances may be selected multiple times, while
others may not be selected at all
Base Model Training: Once the bootstrap samples are created, a base
model (e.g., decision tree, random forest, etc.) is trained on each
bootstrap sample.
5
Bagging
Bagging, short for Bootstrap Aggregating, is an ensemble learning
technique used to improve the stability and accuracy of machine
learning algorithms.
It works by combining the predictions of multiple base models, each
trained on a subset of the original dataset.
Voting or Averaging: After training all the base models, bagging
combines their predictions through a voting mechanism (for
classification tasks) or averaging (for regression tasks).
6
Bagging ML
Bagging helps to reduce overfitting by reducing the variance of the
model.
7
Boosting
Boosting is another ensemble learning technique used to improve the performance
of machine learning models. Unlike bagging, which trains each base model
independently,
boosting trains base models sequentially, with each subsequent model focusing
more on the instances that were misclassified by the previous models.
The succeeding models are dependent on the previous model.
8
How boosting works
A. Base Model Training:
Boosting starts by training a base model (often a weak learner) on the original dataset.
B. Instance Weighting:
After the first model is trained, the misclassified instances are given higher weights, while the
correctly classified instances are given lower weights. This allows subsequent models to focus more on
the difficult instances.
C. Sequential Training:
The subsequent models are trained sequentially, with each model focusing more on the instances that
were misclassified by the previous models. The weights of the instances are adjusted after each model
is trained.
9
Boosting cont..
D. Combining Predictions:
Finally, the predictions of all the models are combined through a weighted sum or voting
to produce the final prediction.
Example : AdaBoost (Adaptive Boosting) and Gradient Boosting,
10
Bagging - Example
11
AdaBoost - Example
Adaptive boosting or AdaBoost: is one of the simplest boosting algorithms. Usually,
decision trees are used for modelling.
12
Unsupervised Learning
➔Unsupervised learning aims to find the underlying structure or the distribution
➔ of data. We want to explore the data to find some intrinsic structures in them.
➔ models itself find the hidden patterns and insights from the given data.
➔ Unsupervised learning cannot be directly applied to a regression or classification problem
➔ Unsupervised learning is much similar as a human learns to think by their own experiences,
which makes it closer to the real AI.
➔ Unsupervised learning works on unlabeled and uncategorized data which make
unsupervised learning more important.
13
Basic Steps In ML
Clustering: Clustering is a method of grouping the objects into clusters such that objects with most
similarities remains into a group.
Association: An association rule is an unsupervised learning method which is used for finding the
relationships between variables in the large database. It determines the set of items that occurs together
in the dataset.
Dimensionality Reduction: Reducing the number of features (variables) in a dataset while
preserving important information
14
Unsupervised Learning algorithms
Below is the list of some popular unsupervised learning algorithms:
➔ K-means clustering
->Principal component analysis
15
K-means clustering
16
K-means Algorithm
Definition: K-Means is a partitioning clustering algorithm that separates a dataset
into K distinct, non-overlapping subsets (clusters).
Working Principle
Initialization: Randomly select K data points as initial cluster centers.
Assignment: Assign each data point to the cluster whose center is nearest.
Update Centers: Recalculate the cluster centers as the mean, variance, Euclidian
distance of the data points in each cluster.
Repeat: Iterate steps 2 and 3 until convergence (when cluster assignments stabilize).
17
Common Terms
Given k, the k-means algorithm is implemented in 5 steps:
[Link]
18
Python Implementation
EX: Problem Statement:
A retail store wants to get insights about its customers. And then build a system
that can cluster customers into different groups.
19
Introduction to Deep Learning
Deep Learning
20
What is Deep Learning
Deep learning is a subset of machine learning, which itself is a subset of
artificial intelligence (AI).
Deep learning models are inspired by the structure and function of the
human brain, specifically neural networks, and they are designed to
automatically learn features and patterns from large amounts of data.
The output from each preceding layer is taken as input by each one of the
successive layers
21
Deep Learning Cont..
Deep learning models are capable enough to focus on the accurate
features themselves by requiring a little guidance from the programmer
and are very helpful in solving out the problem of dimensionality.
Deep learning algorithms are used, especially when we have a huge no
of inputs and outputs.
Deep learning is a collection of statistical techniques of machine learning
for learning feature hierarchies that are actually based on artificial neural
networks
22
Deep Learning Implementation
Deep learning is implemented by the help of deep networks, which are
nothing but neural networks with multiple hidden layers
23
Artificial Neural Network
Neural Networks processes information in a similar way the human
brain does, and these networks actually learn from examples, you cannot
program them to perform a specific task.
They will learn only from past experiences as well as examples
Artificial Neural Network is biologically inspired by the neural network,
which constitutes after the human brain
24
Artificial Neural Network
25
Input Layer
This layer consists of neurons that directly accept the input features of
the data.
It does not perform any transformations or computations. Instead, it acts as
a channel to pass the raw data into the network.
The primary role of the input layer is to serve as the entry point for the
input data into the neural network.
For example, if the input is an image of 28x28 pixels, the input layer will
have 784 neurons (one for each pixel).
26
Hidden Layer
Hidden layers are where the actual computation and learning take
place.
These layers consist of neurons that apply weights and biases to the input,
followed by an activation function to introduce non-linearity
There can be one or multiple hidden layers, and each layer transforms
the input data into a more abstract and useful representation
27
Output - Layer
The output layer is the final layer of the neural network that produces the
output prediction
The number of neurons in this layer corresponds to the number of
output classes or the number of values to predict in the case of
regression tasks.
In a classification task, it typically uses a softmax activation function to
provide class probabilities.
28 In regression tasks, it may use a linear activation function
Perceptron's
The Neural Network's basic unit is called a Perceptron.
So, a perceptron can be defined as a neural network with a single layer
that classifies the linear data
It further constitutes four major components
It was introduced by Frank Rosenblatt in 1957 and is used for binary
classification tasks.
29
Perceptron's parts
A. Input Features (x): These are the inputs to the perceptron.
Each input is associated with a feature of the data.
For example, in an image recognition task, each pixel value can be an
input feature.
B. Weights (w): Each input feature is multiplied by a corresponding
weight. Weights are parameters that the perceptron learns during training.
They determine the importance of each input feature in making the
30 prediction
Perceptron's parts
C. Bias (b):
The bias term is added to the weighted sum of inputs. It allows the activation
function to be shifted left or right, enabling the perceptron to make more flexible
decisions.
Bias helps in the adjustment of the curve of activation function so as to accomplish a
precise output
Activation Function: The weighted sum of the inputs plus the bias is passed through
an activation function. The activation function determines the output of the perceptron
based on this sum.
31
Perceptron's parts
For a basic perceptron, the activation function is usually a step function (Heaviside step
function), which outputs 1 if the weighted sum is greater than a certain threshold and 0
otherwise.
introducing non-linearity, allowing them to learn complex patterns and relationships in
data.
32
Perceptron's parts
33
Types of Activation function
34
Types of Activation function
35
Types of Activation function
36
Types of Activation function
37
When to use which Activation function
Sigmoid: Use for binary classification tasks where the output needs to be interpreted as a
probability between 0 and 1.
Hyperbolic Tangent (Tanh): Use in hidden layers of neural networks for zero-centered
activations and when outputs need to be in the range (-1, 1).
Rectified Linear Unit (ReLU): Use in hidden layers of deep neural networks for faster
convergence and to overcome the vanishing gradient problem.
Leaky ReLU: Use to address the "dying ReLU" problem in deep neural networks where some
neurons become inactive during training.
Softmax: Use in the output layer of neural networks for multi-class classification tasks where
outputs need to be interpreted as probabilities and the sum of probabilities equals 1.
38
When to use which Activation function
39
Perceptron's Summary
40
Simple implementation
41
Thank you Had a Great Time
42