Virtual Environment

Hope_Artificial_Intelligence:HAI 1
Purpose

Hope_Artificial_Intelligence:HAI 2
Ramisha Rani K, AI Trainer

Key Terms
Mrs. Ramisha Rani K, AI Trainer.

[Link] Hope Artificial Intelligence:HAI @hope_artificial_intelligence

Ramisha Rani K, AI Trainer

KEY TERMS
Key Dataset
Term
01 The information about an event
Key Variable
Term
02 Input Features | Attributes

Population
Key
Term Entire Values of a Variable
03
Sample
Key
Term Sample of data taken from population dataset which should
04 represent the properties of population

Key Statistic
Term Finding the value for sample dataset.(E.g., Mean)
05

[Link] Hope Artificial Intelligence:HAI @hope_artificial_intelligence

Ramisha Rani K, AI Trainer
KEY TERMS

Key Parameter
Term
06 Statistics which gives value for population
Key Data
Term
07 Values of variable

[Link] Hope Artificial Intelligence:HAI @hope_artificial_intelligence

Ramisha Rani K, AI Trainer
Key Terms
Dataset The information about an event

[Link] Hope Artificial Intelligence:HAI @hope_artificial_intelligence

Ramisha Rani K, AI Trainer
Key Terms
Variable Input Features

[Link] Hope Artificial Intelligence:HAI @hope_artificial_intelligence

Ramisha Rani K, AI Trainer
Key Terms

Data Value of the dataset

[Link] Hope Artificial Intelligence:HAI @hope_artificial_intelligence

Ramisha Rani K, AI Trainer
Key Terms
Entire values of a Variable or Entire
Population Dataset

[Link] Hope Artificial Intelligence:HAI @hope_artificial_intelligence

#learnaiwithramisha

Do you think Artificial Intelligence is only

for Computer Science Department?

l EEE Wire
l
M echanica Netw ess
ork
Ph
CE ysi
E cs

Ch
vil

em
Ci

ist
ry
tics

His
ma

tor
the

y.
Ma

[Link] @hope_artificial_intelligence
#learnaiwithramisha
Artificial Intelligence

Past data Algorithm Output

Heart SVM
Disease KNN Final
Forest Fire … Model
…

@hope_artificial_intelligence
[Link] Hope_Artificial_Intelligence:HAI 11
#learnaiwithramisha

Types of Problem Statement in in Artificial Intelligence

@hope_artificial_intelligence
[Link] Hope_Artificial_Intelligence:HAI 12
#learnaiwithramisha

Supervised
Learning
Past Data/input Data

Input/Variables/Feature

Output/label

@hope_artificial_intelligence
[Link] Hope_Artificial_Intelligence:HAI 13
#learnaiwithramisha

Take way for Supervised Learning

Requirement should be clear.

Input and Output are well defined.

@hope_artificial_intelligence
[Link] Hope_Artificial_Intelligence:HAI 14
 SCREENING-1
#learnaiwithramisha

Types of Problem Statement in Artificial Intelligence

@hope_artificial_intelligence
[Link] Hope_Artificial_Intelligence:HAI 15
#learnaiwithramisha
Unsupervised
Learning
Past Data/input Data

Input/Variables/Feature

@hope_artificial_intelligence
[Link] Machine Deep Learning(TM)
Hope_Artificial_Intelligence:HAI 16
#learnaiwithramisha
Unsupervised
Learning

@hope_artificial_intelligence
[Link] Hope_Artificial_Intelligence:HAI 17
#learnaiwithramisha

Take way for Unsupervised Learning

We don’t know what we need.

Only Input data, So we can do

Clustering.

@hope_artificial_intelligence
[Link] Hope_Artificial_Intelligence:HAI 18
#learnaiwithramisha

Types of Problem Statement in Artificial Intelligence

@hope_artificial_intelligence
[Link] Hope_Artificial_Intelligence:HAI 19
#learnaiwithramisha

Semi Supervised

Past Data/input Data

Input/Variables/Feature

Output/label

@hope_artificial_intelligence
[Link] Hope_Artificial_Intelligence:HAI 20
#learnaiwithramisha

Take way for Semi Supervised Learning

We know requirements

But Half of the output are not

labelled.

@hope_artificial_intelligence
[Link] Machine Deep Learning(TM)
Hope_Artificial_Intelligence:HAI 21
#learnaiwithramisha

SCREENING - 2

Artificial Intelligence
Problem Identification

@hope_artificial_intelligence
[Link] Hope_Artificial_Intelligence:HAI 22
#learnaiwithramisha
Problem Identification on Supervised Learning
Classification
Classifying the output based on the input parameter.

Yes/No
Dog/ Cat
Categorical Value House/not house

@hope_artificial_intelligence
[Link] Hope_Artificial_Intelligence:HAI 23
#learnaiwithramisha
Problem Identification on Supervised Learning
Regression

Numerical Value

Hope_Artificial_Intelligence:HAI 24

@hope_artificial_intelligence
[Link]
#learnaiwithramisha

See the Picture Tell a Story

@hope_artificial_intelligence
[Link] Hope_Artificial_Intelligence:HAI 25
#learnaiwithramisha

Linear Graph

@hope_artificial_intelligence
[Link] Hope_Artificial_Intelligence:HAI 26
#learnaiwithramisha

Multiple Linear

@hope_artificial_intelligence
[Link] Hope_Artificial_Intelligence:HAI 27
#learnaiwithramisha

Artificial Intelligence

Machine Learning
Deep Learning

Supervised Learning Unsupervised Learning Semi Supervised Learning

Regression Classification Clustering

 Simple Linear Regression  Logistic Classification  K-Means Clustering
 Multiple Linear Regression  K-Nearest Neighbor(K-NN)  Hierarchical Clustering
 Logistic Regression  Support Vector Machine (SVM)
 Polynomial Regression  Kernel SVM
 Support Vector Machine Regression  Naive Bayes
 Decision Tree Regression  Decision Tree Classification
 Random Forest Regression  Random Forest Classification

@hope_artificial_intelligence
Hope_Artificial_Intelligence:HAI 28
[Link]
#learnaiwithramisha
Algorithms

Algorithm for both Regression Algorithm Classification Algorithm

Regression and Classification

Support Vector Machine Linear Algorithm

Logistic Algorithm

Multiple Linear Navies Bayes

Decision Tree Regression KNN

Random Forest Polynomial Regression

@hope_artificial_intelligence
[Link] Hope_Artificial_Intelligence:HAI 29
+91 95979 40880

Hope_Artificial_Intelligence:HAI 30
#learnaiwithramisha
Polynomial Graph

[Link] Hope_Artificial_Intelligence:HAI 31
Image Source: [Link] @hope_artificial_intelligence
#learnaiwithramisha

 Take Away Concepts In Simple Linear Regression

 Why it is Simple Linear Regression?

 How this regression helps for future prediction?

 Validating Parameter:
 Sum of Square Error(SSE) or Residual Sum of Square(RSS)
 Sum of Square Regression(SSR) or Explained Sum of Square(ESS)
 Sum of Square Total(SST)
 R Squared(R2)
 Adjusted R Squared

 When to use simple linear regression?

[Link] Hope_Artificial_Intelligence:HAI 32
@hope_artificial_intelligence
#learnaiwithramisha

 Why it is Simple Linear Regression?

It has one input(X/Independent) and one output (y/Dependent Variable/Response variable).

It uses the straight line equation y= mX+c

Where,
y= output(which forms straight line by adding all data points)

m= slope= dy/dx= weight (which says about constant distance between two data points)

X= Input( if input changes, then output will also change accordingly)

c= bias= intercept= starting of the straight line= initial value= minimum value.

[Link] Hope_Artificial_Intelligence:HAI 33
@hope_artificial_intelligence
#learnaiwithramisha How this regression helps for future prediction?
SIMPLE LINEAR REGRESSION

Dataset

Predicted Value X Y
10
Y=2X+0 1 2
Y Dependant Variable 8 2 4
6 3 6
4 8
4
w 5 10

2 Initial Value(b)/Minimum value/ Origin

0 3 5
1 2 4
Y =wX +b X Independent Variable
y)( n (∑ 𝑥𝑦 ¿−(∑ x ) ( ∑ 𝑦)
W= Slope= ___________________ B= Bias=Initial Value=Minimum Value= ___________________
( (

[Link] Hope_Artificial_Intelligence:HAI 34
@hope_artificial_intelligence
#learnaiwithramisha
Types of scattered data with Linear Regression line

D.V D.V

Actual Value I.V

I.V
Predicted Value

D.V
D.V

I.V
I.V

[Link] Hope_Artificial_Intelligence:HAI
@hope_artificial_intelligence
#learnaiwithramisha Validating parameter: 1 .Sum of Square Error(SSE) or Residual Sum of Square(RSS)

Formula:
yi

10 Error
−
−Y Error= Actual Value(yi) – Predicted Value(yi)
Y Dependant Variable

8
Where, i = Observation point
6
n= number of Observation point
_
4 = Actual Value

2 =Predicted Value

0
1 2 3 4 5
X Independent Variable
[Link] Hope_Artificial_Intelligence:HAI 36
@hope_artificial_intelligence
#learnaiwithramisha

Error
Input Actual Output Predicted Output Error=(Actual -Predicted)2
1 3.8 3.5 0.09
3 4.5 4.7 0.02
4 5.6 5.3 0.09
5 4.6 1.4 10.24
6 2.3 3.4 1.44
9 7.6 7.1 0.25
10 3.4 2.3 0.87

[Link] Hope_Artificial_Intelligence:HAI 37
@hope_artificial_intelligence
#learnaiwithramisha Validating parameter: [Link] of Square Error(SSE) or Residual Sum of Square(RSS)

Formula:
𝑛
𝑆𝑢𝑚 𝑜𝑓 𝑆𝑞𝑢𝑎𝑟𝑒 𝐸𝑟𝑟𝑜𝑟 (𝑆𝑆𝐸)=∑ ( 𝑦𝑖−−
𝑦𝑖 ) 2
r 𝑖=0
r ro Where, i = Observation point
reE yi

10 u a
f Sq −Y n= number of Observation point
o
m _
Su
Y Dependant Variable

8 = Actual Value

6 =Predicted Value

Take away:
2 If,
Higher the SSE, then predicted value is poor
0 Smaller the SSE, then predicted value is good
1 2 3 4 5
X Independent Variable

[Link] Hope_Artificial_Intelligence:HAI 38
@hope_artificial_intelligence
#learnaiwithramisha Validating parameter: [Link] of Square Regression(SSR) or Explained Sum of Square(ESS)

Formula:
𝑛
𝑆𝑢𝑚 𝑜𝑓 𝑆𝑞𝑢𝑎𝑟𝑒 𝑅𝑒𝑔𝑟𝑒𝑠𝑠𝑖𝑜𝑛(𝑆𝑆𝑅)=∑−
( 𝑦 𝑖− 𝑦𝑚𝑒𝑎𝑛 ) 2
𝑖=0

10 Where, i = Observation point

−Y
Y Dependant Variable

8 n= number of Observation point

SSR _
6 =Predicted Value
ymean
= Mean of Dependant
4 Variable(Response variable)

2
Take away:
 Higher the SSR(or)ESS, better the model performance
0
1 2 3 4 5

X Independent Variable
[Link] Hope_Artificial_Intelligence:HAI 39
@hope_artificial_intelligence
#learnaiwithramisha Validating parameter: [Link] of Square Total(SST)

Formula: 𝑛
𝑆𝑢𝑚 𝑆𝑞𝑢𝑎𝑟𝑒 𝑇𝑜𝑡𝑎𝑙(𝑆𝑆𝑇 )=∑ ( 𝑦 𝑖 − 𝑦𝑚𝑒𝑎𝑛 ) 2
𝑖=0
SST= SSR+SSE
Where, i = Observation point
Y
10
n= number of Observation point
Y Dependant Variable

8 SST = Actual Value

6 = Mean of Dependant
ymean
Variable(Response variable)
4

2
Take away:
0
1 2 3 4 5 If,
Smaller the SST, better the model
X Independent Variable
[Link] Hope_Artificial_Intelligence:HAI
@hope_artificial_intelligence
#learnaiwithramisha Validating parameter: 4. R Squared(R2)

𝑛
_
SSR
∑_________________
( 𝑦 𝑖− 𝑦𝑚𝑒𝑎𝑛 ) 2 Purpose of R2 :
To know , how well the model is fitted.
______ = 𝑖= 0
R 2 =
SST 𝑛

∑ ( 𝑦 𝑖− 𝑦𝑚𝑒𝑎𝑛 ) 2 How R2 differs from other parameters like SSE,SSR and SST ?
𝑖= 0
SSE, SSR and SST range varies with dataset to dataset.
But,
Where, i = Observation point R2 exists between 0 and 1
If,
n= number of Observation point R2 = nearly to 1, then built model has better performance.
_ R2 = nearly to 0, then built model has poor performance.
= Actual Value
The only drawback of R2 is that if new predictors (X) are added
=Predicted Value to our model, R2 only increases or remains constant but it never
decreases. We can not judge that by increasing complexity of
= Mean of Dependant our model, are we making it more accurate
Variable(Response variable)
[Link] Hope_Artificial_Intelligence:HAI
@hope_artificial_intelligence
#learnaiwithramisha

Graphs with Equation

@hope_artificial_intelligence
[Link] Hope_Artificial_Intelligence:HAI 42
#learnaiwithramisha
Validating parameter: [Link] R2

n 1
R2  1 (1  R2 );
nk
n  number of observations,
k  number of independent variables.

R2 Adjusted R2

 R2 shows, how the model is fitted with  Adjusted R2 helps to find most
the actual data points. significant independent variable.

 R2 gets increase when new independent  Adjusted R2 only gets increase when
variable is added to the existing model most significant independent variable
irrespective of poor significant or most is added to the model otherwise
significant independent variable. stays constant.

 It has only positive value  It may have negative value.

between o to 1.

[Link] Hope_Artificial_Intelligence:HAI 43
@hope_artificial_intelligence
#learnaiwithramisha
The Purpose of Training and Test Set

Original Data Training

Test Set: 20
set : 100 Dataset: 80

Hope_Artificial_Intelligence:HAI 44
[Link]
#learnaiwithramisha

Training Dataset: 80

Weight
,Bais
Y= 0.3x1+0.4x2+0
Using Training
Data Set

Model

[Link] Hope_Artificial_Intelligence:HAI 45
@hope_artificial_intelligence
#learnaiwithramisha

Test Set: 20

x1 x2 Y Y= 0.3x1+0.4x2+0 Found using Training Set

+0 =27.95
35.5

If(y>30):
print(“Unfit”)
Else
print(“fit”)

[Link] Hope_Artificial_Intelligence:HAI 46
@hope_artificial_intelligence
Types of fitting

Hope_Artificial_Intelligence:HAI 47
 Simple Linear Regression- Take Away points
#learnaiwithramisha Model = 0.3X + 5 (for understanding purpose 0.3 = weight or slope, 5= Intercept or Initial value )

Concept Name Formula Inference

Simple Linear Y=wX+b One Dependent variable and One Independent
Variable
y)( Spread of data should be in linear
W= ___________________
(
B= (∑ 𝑥𝑦 ¿−(∑ x ) ( ∑ 𝑦)
n ___________________
(
Sum of Square Error(SSE) SSE= SSE maximum, poor the model
(or) (or) SSE minimum, better the model
Residual Sum of Square (RSS) SSE= summation(Actual value – Predicted Value) 2
(or) (Note: SSE value varies dataset to dataset)
Unexplained Sum of Square
Sum of Square Regression(SSR) SSR= SSR maximum, better the model
(or) (or) SSR minimum, poor the model
Explained Sum of Square(ESS) SSR=summation(Predicted Value – Dependent Value
mean)2 (Note: SSR value varies dataset to dataset)

Sum of Square Total(SST) SST= SST maximum, poor the Model

(or) SST minimum, better the model
SST = summation(Actual value -Predicted Value) 2
(Note: SST value varies dataset to dataset)

R-Squared (R2) R2 = SSR/SST If, R2 = nearly to 1, then built model has

better performance.
R2 = nearly to 0, then built model has poor
performance.

(Note: R2 exists between 0 and 1)

[Link] Hope_Artificial_Intelligence:HAI 48
@hope_artificial_intelligence
#learnaiwithramisha

Assumption for Linear Regression

BEFORE THE MODEL

The Quantitative Data Condition Should be numbers

The Straight Enough Condition (or “linearity”)  should be in Linear Pattern

The Outlier Condition Should not have outlier

AFTER THE MODEL

Normality of Error

Homoscedasticity  Variance should be equal in overall spread

Log Transform if slightly curve can use Log Transform to convert perfect linear.

[Link] Hope_Artificial_Intelligence:HAI 49
@hope_artificial_intelligence
 Steps
Problem Identification

Reg/classification

Check Pattern

Split our dataset into training set and Test set

Model(Training Set)

Validating parameter(Test set)

Assumption

Before the model and After the model(Linear problem)

Hope_Artificial_Intelligence:HAI 50
#learnaiwithramisha

Multiple Linear Regression

Multiple Linear
Regression

[Link] Hope_Artificial_Intelligence:HAI 51
@hope_artificial_intelligence
#learnaiwithramisha

Multiple Linear Regression

Multiple Linear
Regression

Simple Linear
Regression

[Link] Hope_Artificial_Intelligence:HAI 52
@hope_artificial_intelligence
#learnaiwithramisha

Assumption for Multiple Linear Regression

BEFORE THE MODEL

The Quantitative Data Condition Should be numbers

The Straight Enough Condition (or “linearity”)  should be in Linear Pattern

The Outlier Condition Should not have outlier

AFTER THE MODEL

Homoscedasticity  Variance should be equal in overall spread

No Multicollinearity  Should not have Multicollinearity

[Link] Hope_Artificial_Intelligence:HAI 53
@hope_artificial_intelligence
#learnaiwithramisha

Algorithm

NON-LINEAR ALGORITHM

Polynomial Algorithm
LINEAR ALGORITHM
Support Vector Machine
Simple Linear Algorithm
Decision Tree
Multiple Linear Algorithm
Random Forest

KNN

Naive's Bayes

[Link] Hope_Artificial_Intelligence:HAI 54
@hope_artificial_intelligence
#learnaiwithramisha

Problem Statement of Non-Linear Algorithm

Finding Truth

[Link] Hope_Artificial_Intelligence:HAI 55
@hope_artificial_intelligence
#learnaiwithramisha
Types of Fitting| Over fitting, Under fitting, well fitting

[Link] Hope_Artificial_Intelligence:HAI 56
@hope_artificial_intelligence
#learnaiwithramisha

Polynomial Regression

Polynomial
Regression

[Link] Hope_Artificial_Intelligence:HAI 57
@hope_artificial_intelligence
#learnaiwithramisha

Ccomparison

Multiple
Linear
Regression

Simple
Linear
Regression

Polyno
mial
Regressi
on

[Link] Hope_Artificial_Intelligence:HAI 58
@hope_artificial_intelligence
#learnaiwithramisha

Assumption for Polynomial Regression

The Quantitative Data

The Outlier

Data Spread should be in Curve

[Link] Hope_Artificial_Intelligence:HAI 59
@hope_artificial_intelligence
#learnaiwithramisha

Support Vector Machine

Image source: [Link]

[Link] Hope_Artificial_Intelligence:HAI 60
@hope_artificial_intelligence
#learnaiwithramisha

Support Vector Machine

[Link] Hope_Artificial_Intelligence:HAI 61
Image source: [Link] @hope_artificial_intelligence
#learnaiwithramisha

Support Vector Machine

Image source: [Link]

[Link] Hope_Artificial_Intelligence:HAI 62
@hope_artificial_intelligence
#learnaiwithramisha

Support Vector Machine

What if closer data point Exists?

Image source: [Link]

[Link] Hope_Artificial_Intelligence:HAI 63
@hope_artificial_intelligence
#learnaiwithramisha

Support Vector Machine

What if closer data point Exists?

[Link] Hope_Artificial_Intelligence:HAI 64
Image source: [Link] @hope_artificial_intelligence
#learnaiwithramisha

Support Vector Machine

Non- Separable Dataset

[Link] Hope_Artificial_Intelligence:HAI 65
Image source: [Link] @hope_artificial_intelligence
#learnaiwithramisha

Support Vector Machine

Non- Separable Dataset- Three Dimensional

[Link] Hope_Artificial_Intelligence:HAI 66
Image source: [Link] @hope_artificial_intelligence
#learnaiwithramisha

Support Vector Machine

Non- Separable Dataset- Three Dimensional- One Plane

[Link] Hope_Artificial_Intelligence:HAI 67
Image source: [Link] @hope_artificial_intelligence
#learnaiwithramisha

Support Vector Machine

Non- Separable Dataset- Three Dimensional -3 Plane

Image source: [Link]

[Link] Hope_Artificial_Intelligence:HAI 68
@hope_artificial_intelligence
#learnaiwithramisha

Support Vector Machine

Non- Separable Dataset- Three Dimensional-3 Plane

Image source: [Link]

[Link] Hope_Artificial_Intelligence:HAI 69
@hope_artificial_intelligence
#learnaiwithramisha

Assumptions for Support Vector Machine

The data spread should be Non-Linear

Pattern

No Outliers

[Link] Hope_Artificial_Intelligence:HAI 70
@hope_artificial_intelligence
#learnaiwithramisha

Decision Tree

Image Source: [Link]

[Link] Hope_Artificial_Intelligence:HAI 71
@hope_artificial_intelligence
#learnaiwithramisha

[Link] Hope_Artificial_Intelligence:HAI 72
@hope_artificial_intelligence
#learnaiwithramisha

Decision Tree

Image Source: [Link]

[Link] Hope_Artificial_Intelligence:HAI 73
@hope_artificial_intelligence
#learnaiwithramisha

Important Terminology related to Decision Trees

[Link] Node: It represents the entire population or sample and this further gets divided into two or more
homogeneous sets.
[Link]: It is a process of dividing a node into two or more sub-nodes.
[Link] Node: When a sub-node splits into further sub-nodes, then it is called the decision node.
[Link] / Terminal Node: Nodes do not split is called Leaf or Terminal node.
[Link]: When we remove sub-nodes of a decision node, this process is called pruning. You can say
the opposite process of splitting.
[Link] / Sub-Tree: A subsection of the entire tree is called branch or sub-tree.
[Link] and Child Node: A node, which is divided into sub-nodes is called a parent node of sub-nodes
whereas sub-nodes are the child of a parent node.

[Link] Hope_Artificial_Intelligence:HAI 74
@hope_artificial_intelligence
#learnaiwithramisha

How to select the best variable from the dataset for Root Node

Entropy, Information Gini index, Gain Ratio,

gain,

Reduction in
Variance

[Link] Hope_Artificial_Intelligence:HAI 75
@hope_artificial_intelligence
How to select the best variable from the dataset for Root Node

Entropy

If ,
Entropy is larger Randomness is high  perfectly will not able to
predict and Vice versa

Hope_Artificial_Intelligence:HAI 76
#learnaiwithramisha
How to select the best variable from the dataset for Root Node

Entropy

ID3 follows the rule — A branch with an entropy of zero is a

leaf node and A branch with entropy more than zero needs
further splitting.
[Link] Hope_Artificial_Intelligence:HAI 77
@hope_artificial_intelligence
#learnaiwithramisha
How to select the best variable from the dataset for Root Node

Information
gain

Constructing a decision tree is all about finding an attribute that returns the highest
information gain and the smallest entropy.

[Link] Hope_Artificial_Intelligence:HAI 78
@hope_artificial_intelligence
#learnaiwithramisha
How to select the best variable from the dataset for Root Node

Information
gain

Where “before” is the dataset before the split, K is the number of subsets generated by the split, and
(j, after) is subset j after the split.

[Link] Hope_Artificial_Intelligence:HAI 79
@hope_artificial_intelligence
#learnaiwithramisha
How to select the best variable from the dataset for Root Node

Gini index,

You can understand the Gini index as a cost function used to evaluate splits in the dataset.
It is calculated by subtracting the sum of the squared probabilities of each class from one.
It favors larger partitions and easy to implement whereas information gain favors smaller partitions with
distinct values.

[Link] Hope_Artificial_Intelligence:HAI 80
@hope_artificial_intelligence
#learnaiwithramisha
How to select the best variable from the dataset for Root Node

Gain Ratio,

Gain ratio overcomes the problem with information gain by taking into account the number
of branches that would result before making the split.
It corrects information gain by taking the intrinsic information of a split into account.

[Link] Hope_Artificial_Intelligence:HAI 81
@hope_artificial_intelligence
#learnaiwithramisha
How to select the best variable from the dataset for Root Node

Reduction in
Variance

This algorithm uses the standard formula of variance to choose the best split.
The split with lower variance is selected as the criteria to split the population:

[Link] Hope_Artificial_Intelligence:HAI 82
@hope_artificial_intelligence
#learnaiwithramisha
How to avoid/counter Overfitting in Decision Trees?

Pruning Decision Tree

[Link] Hope_Artificial_Intelligence:HAI 83
@hope_artificial_intelligence
#learnaiwithramisha
How to avoid/counter Overfitting in Decision Trees?

Pruning Decision Tree

[Link] Hope_Artificial_Intelligence:HAI 84
@hope_artificial_intelligence
#learnaiwithramisha

Decision Tree

Image Source: [Link]

[Link] Hope_Artificial_Intelligence:HAI 85
@hope_artificial_intelligence
#learnaiwithramisha

Important points for Decision Tree

There is Possible of overfitting, because of huge

decision split.
If we change the data in training, there is huge
difference in model

[Link] Hope_Artificial_Intelligence:HAI 86
@hope_artificial_intelligence
Random Forest

Hope_Artificial_Intelligence:HAI 87
#learnaiwithramisha

Random Forest

Ensemble Learning

Bagging or Bootstrap Aggregation

Random feature selection

[Link] Hope_Artificial_Intelligence:HAI 88
@hope_artificial_intelligence
#learnaiwithramisha Random Forest
Ensemble Learning

Image Source: [Link]

[Link] Hope_Artificial_Intelligence:HAI 89
@hope_artificial_intelligence
#learnaiwithramisha Random Forest
Ensemble Learning

Image Source: [Link]

[Link] 90
Hope_Artificial_Intelligence:HAI @hope_artificial_intelligence
#learnaiwithramisha
Random Forest
Bagging or Bootstrap Aggregation

[Link] Hope_Artificial_Intelligence:HAI 91
@hope_artificial_intelligence
#learnaiwithramisha
Random Forest
Ensemble Learning

[Link] Hope_Artificial_Intelligence:HAI 92
@hope_artificial_intelligence
#learnaiwithramisha
Random Forest

Image Source: [Link]

[Link] Hope_Artificial_Intelligence:HAI 93
@hope_artificial_intelligence
#learnaiwithramisha

Pure Classification Algorithm

K-Nearest Neighbor

Navies' Bayes

[Link] Hope_Artificial_Intelligence:HAI 94
@hope_artificial_intelligence
#learnaiwithramisha

K- Nearest Neighbour

[Link] Hope_Artificial_Intelligence:HAI 95
@hope_artificial_intelligence
#learnaiwithramisha

K- Nearest Neighbour

[Link] Hope_Artificial_Intelligence:HAI 96
@hope_artificial_intelligence
#learnaiwithramisha

K- Nearest Neighbour

[Link] Hope_Artificial_Intelligence:HAI 97
@hope_artificial_intelligence
#learnaiwithramisha

Low Bias, Low Variance- Good Model

Low Bias, High Variance- Over Fitting

Model

High Bias, Low Variance- Under Fitting

Model

High Bias, High Variance- Poor Model

[Link] Hope_Artificial_Intelligence:HAI 98
Image Source: [Link] @hope_artificial_intelligence
#learnaiwithramisha

Navies' Bayes

Naïve Bayes is a probabilistic machine learning

algorithm based on the Bayes Theorem.

Conditional probability is a measure of the

probability of an event occurring given that another
event has (by assumption, presumption, assertion, or
evidence) occurred.
[Link] Hope_Artificial_Intelligence:HAI 99
@hope_artificial_intelligence
#learnaiwithramisha

Navies' Bayes

In simpler terms, Bayes’ Theorem is a way of finding a

probability when we know certain other probabilities.

[Link] Hope_Artificial_Intelligence:HAI 100

@hope_artificial_intelligence
#learnaiwithramisha

Navies' Bayes

[Link] Hope_Artificial_Intelligence:HAI 101

@hope_artificial_intelligence
#learnaiwithramisha

Navies' Bayes
Assumptions

Independent  Each variable

should not have any connection

Equal  All the variables are

equally important

[Link] Hope_Artificial_Intelligence:HAI 102

@hope_artificial_intelligence
#learnaiwithramisha

Navies' Bayes

The variable y is the class variable(stolen?), which represents if the car is stolen or not given the conditions.
Variable X represents the parameters/features.

[Link] Hope_Artificial_Intelligence:HAI 103

@hope_artificial_intelligence
#learnaiwithramisha
Navies' Bayes

[Link] Hope_Artificial_Intelligence:HAI 104

@hope_artificial_intelligence
#learnaiwithramisha

Navies' Bayes

Since 0.144 > 0.048, Which means given the features RED
SUV and Domestic, our example gets classified as ’NO’ the car
is not stolen.
[Link] Hope_Artificial_Intelligence:HAI 105
@hope_artificial_intelligence
#learnaiwithramisha

Navies' Bayes
The zero-frequency problem

[Link] Hope_Artificial_Intelligence:HAI 106

@hope_artificial_intelligence
#learnaiwithramisha

Types of Algorithm based on spread on data

Linear Algorithm
• Linear
• Multiple Linear

Non-Linear Algorithm(One Problem

Statement-4 models)
• Polynomial
• Support Vector Machine
• Decision Tree
• Random Forest

[Link] Hope_Artificial_Intelligence:HAI 107

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Regression Minimum Error

[Link] Hope_Artificial_Intelligence:HAI 108

@hope_artificial_intelligence
#learnaiwithramisha

[Link] Hope_Artificial_Intelligence:HAI 109

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class

N= Positive Negative

Positive True Positive False Negative

(TP) (FN)
Actual Class

Negative False Positive True Negative

(FP) (TN)

[Link] Hope_Artificial_Intelligence:HAI 110

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class Positive (P) : Observation is positive (for example: is an apple).

N= Positive Negative

Positive True Positive False Negative

Actual Class

(TP) (FN)

Negative False Positive True Negative

(FP) (TN)

[Link] Hope_Artificial_Intelligence:HAI 111

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class Positive (P) : Observation is positive (for example: is an apple).

N= Positive Negative Negative (N) : Observation is not positive (for example: is

not an apple).
Positive True Positive False Negative
Actual Class

(TP) (FN)

Negative False Positive True Negative

(FP) (TN)

[Link] Hope_Artificial_Intelligence:HAI 112

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class Positive (P) : Observation is positive (for example: is an apple).

N= Positive Negative Negative (N) : Observation is not positive (for example: is

not an apple).
Positive True Positive False Negative
True Positive (TP) : Observation is positive, and is predicted
Actual Class

(TP) (FN)
to be positive.

Negative False Positive True Negative

(FP) (TN)

[Link] Hope_Artificial_Intelligence:HAI 113

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class Positive (P) : Observation is positive (for example: is an apple).

N= Positive Negative Negative (N) : Observation is not positive (for example: is

not an apple).
Positive True Positive False Negative
True Positive (TP) : Observation is positive, and is predicted
Actual Class

(TP) (FN)
to be positive.

Negative False Positive True Negative False Negative (FN) : Observation is positive, but is predicted
(FP) (TN) negative.

[Link] Hope_Artificial_Intelligence:HAI 114

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class Positive (P) : Observation is positive (for example: is an apple).

N= Positive Negative Negative (N) : Observation is not positive (for example: is

not an apple).
Positive True Positive False Negative
True Positive (TP) : Observation is positive, and is predicted
Actual Class

(TP) (FN)
to be positive.

Negative False Positive True Negative False Negative (FN) : Observation is positive, but is predicted
(FP) (TN) negative.
False Positive (FP) : Observation is negative, but is predicted
positive.

[Link] Hope_Artificial_Intelligence:HAI 115

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class Positive (P) : Observation is positive (for example: is an apple).

N= Positive Negative Negative (N) : Observation is not positive (for example: is

not an apple).
Positive True Positive False Negative
True Positive (TP) : Observation is positive, and is predicted
Actual Class

(TP) (FN)
to be positive.

Negative False Positive True Negative False Negative (FN) : Observation is positive, but is predicted
(FP) (TN) negative.
False Positive (FP) : Observation is negative, but is predicted
positive.
True Negative (TN) : Observation is negative, and is predicted
to be negative.

[Link] Hope_Artificial_Intelligence:HAI 116

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class Positive (P) : Observation is positive (for example: is an apple).

N= Positive Negative Negative (N) : Observation is not positive (for example: is

not an apple).
Positive True Positive False Negative
True Positive (TP) : Observation is positive, and is predicted
Actual Class

(TP) (FN)
Type I Error to be positive.

Negative False Positive True Negative False Negative (FN) : Observation is positive, but is predicted
(FP) (TN) negative.
True Negative (TN) : Observation is negative, and is predicted
to be negative.

False Positive (FP) : Observation is negative, but is predicted

positive.

[Link] Hope_Artificial_Intelligence:HAI 117

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class Positive (P) : Observation is positive (for example: is an apple).

N= Positive Negative Negative (N) : Observation is not positive (for example: is

not an apple).
Positive True Positive False Negative
True Positive (TP) : Observation is positive, and is predicted
Actual Class

(TP) (FN)
Type I Error to be positive.

Negative False Positive True Negative False Negative (FN) : Observation is positive, but is predicted
(FP) (TN) negative.
Type II Error
True Negative (TN) : Observation is negative, and is predicted
to be negative.

False Positive (FP) : Observation is negative, but is predicted

positive.

[Link] Hope_Artificial_Intelligence:HAI 118

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class Positive (P) : Observation is positive (for example: is an apple).

N= Positive Negative Negative (N) : Observation is not positive (for example: is

not an apple).
Positive True Positive False Negative
True Positive (TP) : Observation is positive, and is predicted
Actual Class

(TP) (FN)
Type I Error to be positive.

Negative False Positive True Negative False Negative (FN) : Observation is positive, but is predicted
(FP) (TN) negative.
Type II Error
True Negative (TN) : Observation is negative, and is predicted
to be negative.

False Positive (FP) : Observation is negative, but is predicted

positive.

[Link] Hope_Artificial_Intelligence:HAI 119

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class
N= Positive Negative

Positive True Positive False Negative

Actual Class

(TP) (FN)
Error I
Negative False Positive True Negative
(FP) (TN)
Error II

[Link] Hope_Artificial_Intelligence:HAI 120

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Recall (Sensitivity) - Recall is the ratio of

correctly predicted positive observations to
the all observations in actual class - yes.
Predicted Class
N= Positive Negative

Positive True Positive False Negative

Actual Class

(TP) (FN)
Error I
Negative False Positive True Negative
(FP) (TN)
Error II

[Link] Hope_Artificial_Intelligence:HAI 121

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Precision(Specificity) - Precision is the

ratio of correctly predicted positive
observations to the total predicted positive
Predicted Class observations.

N= Positive Negative

Positive True Positive False Negative Recall (Sensitivity) - Recall is the ratio of
Actual Class

(TP) (FN) correctly predicted positive observations to

Error I the all observations in actual class - yes.

Negative False Positive True Negative

(FP) (TN)
Error II

[Link] Hope_Artificial_Intelligence:HAI 122

@hope_artificial_intelligence
#learnaiwithramisha

F1 score - F1 Score is the weighted average of Precision and Recall.

Therefore, this score takes both false positives and false negatives into account.
Intuitively it is not as easy to understand as accuracy, but F1 is usually more useful
than accuracy, especially if you have an uneven class distribution.
Accuracy works best if false positives and false negatives have similar cost. If the cost
of false positives and false negatives are very different, it’s better to look at both
Precision and Recall. In our case, F1 score is 0.701.

F1 Score = 2*(Recall * Precision) / (Recall + Precision)

[Link] Hope_Artificial_Intelligence:HAI 123

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class
N= Positive Negative

Positive True Positive False Negative

Actual Class

(TP) (FN)
Error I
Negative False Positive True Negative
(FP) (TN)
Error II

[Link] Hope_Artificial_Intelligence:HAI 124

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class

N= Positive Negative

Positive True Positive False Negative

Actual Class

(TP) (FN)
Error I
Negative False Positive True Negative
(FP) (TN)
Error II

[Link] Hope_Artificial_Intelligence:HAI 125

@hope_artificial_intelligence
#learnaiwithramisha
Algorithms

Algorithm for both Regression Algorithm Classification Algorithm

Regression and Classification

Support Vector Machine Linear Algorithm

Logistic Algorithm

Multiple Linear Navies Bayes

Decision Tree Regression KNN

Random Forest Polynomial Regression

@hope_artificial_intelligence
[Link] Hope_Artificial_Intelligence:HAI 126
#learnaiwithramisha

Logistic Algorithm

[Link] Hope_Artificial_Intelligence:HAI 127

@hope_artificial_intelligence
#learnaiwithramisha

[Link] Hope_Artificial_Intelligence:HAI 128

@hope_artificial_intelligence
#learnaiwithramisha

Unsupervised : Clustering Algorithm

K-Means

Hierarchical

[Link] Hope_Artificial_Intelligence:HAI 129

@hope_artificial_intelligence
#learnaiwithramisha

K means

[Link] Hope_Artificial_Intelligence:HAI 130

@hope_artificial_intelligence
#learnaiwithramisha

K means

[Link] Hope_Artificial_Intelligence:HAI 131

@hope_artificial_intelligence
#learnaiwithramisha

Hierarchical

Agglomerative
• Compute the proximity matrix
• Let each data point be a cluster
• Repeat: Merge the two closest clusters
and update the proximity matrix
• Until only a single cluster remains

Divisive
• Opposite of Agglomerative

[Link] Hope_Artificial_Intelligence:HAI 132

@hope_artificial_intelligence
#learnaiwithramisha
Hierarchical
Agglomerative

[Link] Hope_Artificial_Intelligence:HAI 133

@hope_artificial_intelligence
 sad

[Link]
n-trees-28a4e3c7be14

[Link]
[Link]

[Link]
ted/regression_diagnostics.html

Hope_Artificial_Intelligence:HAI 134