Big Data and Data Analytics
Unit 3.2
Advanced analytics theory and methods
Machine Learning
What is Artificial Intelligence
and Machine Learning?
• Artificial intelligence (AI) refers to the simulation of human intelligence in machines
that are programmed to think and learn like humans.
• Machine learning is a subset of artificial intelligence that involves the use of algorithms
and statistical models to enable computer systems to learn from data, identify
patterns, and make decisions without being explicitly programmed.
• It involves training a computer system with large amounts of data and allowing it to
learn from that data, so it can make predictions or decisions based on new input
Types Of Machine Learning
Types Of Machine Learning
1. Supervised Learning:
Supervised learning is a type of machine learning where the algorithm learns from labeled
data.
In supervised learning, the input data is labeled with the correct output, and the
algorithm learns to predict the output based on the input data.
The goal of supervised learning is to build a model that can accurately predict the output
for new, unseen input data.
Examples of supervised learning include image classification, speech recognition, and
spam filtering.
Types Of Machine Learning
1. Unsupervised Learning:
Unsupervised learning is a type of machine learning where the algorithm is trained on a
dataset without any labeled output.
The goal of unsupervised learning is to identify patterns and relationships within the data,
without being given any specific target variable to predict.
This type of learning is often used for clustering, dimensionality reduction, and anomaly
detection. In unsupervised learning, the algorithm must find its own structure in the data,
which can be more challenging than supervised learning where the algorithm is given a
clear objective to optimize.
Types Of Machine Learning
Naïve Bayes Algorithm
It is a classification technique based on Bayes’ Theorem with an independence
assumption among predictors. In simple terms, a Naive Bayes classifier assumes that the
presence of a particular feature in a class is unrelated to the presence of any other
feature.
For example, a fruit may be considered to be an apple if it is red, round, and about 3
inches in diameter. Even if these features depend on each other or upon the existence of
the other features, all of these properties independently contribute to the probability
that this fruit is an apple and that is why it is known as ‘Naive’.
Naïve Bayes Algorithm
Bayes theorem provides a way of computing posterior probability P(c|x) from P(c), P(x)
and P(x|c). Look at the equation below:
• P(c|x) is the posterior probability of class (c, target) given predictor (x, attributes).
• P(c) is the prior probability of class.
• P(x|c) is the likelihood which is the probability of the predictor given class.
• P(x) is the prior probability of the predictor.
Naïve Bayes Algorithm Example
Consider the following dataset of weather,Humidity and corresponding target variable ‘Play’ (suggesting possibilities of playing).
Now, we need to classify whether players will play or not based on weather condition. Let’s follow the below steps to perform it.
Problem: Players will play if the weather is sunny and humidity is low. Is this statement correct?
Weather Humidity Play
Sunny High No
Overcast Low Yes
Rainy Low No
Sunny High Yes
Sunny Low Yes
Overcast High Yes
Rainy High No
Rainy High No
Sunny Low Yes
Sunny Low Yes
Naïve Bayes Algorithm Example
Step 1: Calculate probability of each outcome in Play
Count the number of times Play is Yes and also Play is No.
Prior Probability
P(Yes) and P(No)
Count Probability
Yes 7 7/10
No 3 3/10
Naïve Bayes Algorithm Example
Step 2: Calculate probability of weather being sunny (Count number of times it is Sunny
(5) and divide by the total number of weather (14))
And also calculate probability of humidity being Low.
Prior Probability
P(Sunny) and P(Low)
Count Probability
Sunny 5 5/14
Low 5 5/14
Naïve Bayes Algorithm Example
Step 3: Convert the data set into a frequency table
Count the number of times weather is sunny and Play is No (1) and have count of No as
denominator, also count number of times weather is sunny and play is Yes (3) and have
count of Yes as denominator.
Count the number of times Humidity is Low and Play is No (1) and have count of No as
denominator, also count number of times Humidity is Low and play is Yes (4) and have
count of Yes as denominator.
Frequency Table (weather) Frequency Table (Humidity)
P(Suuny|No) and P(Sunny|Yes) P(Low|No) and P(Low|Yes)
No Yes No Yes
Sunny 1/3 3/7 Low 1/3 4/7
Naïve Bayes Algorithm Example
Step 4: Now lets calculate probabilities whether players will play (Yes) or not (No)
P(Yes | weather=sunny, humidity=low) = [ P(sunny|yes)*P(Yes)/P(sunny) ] * [ P(low|yes)*P(Yes)/P(low) ]
= [ (3/7) * (7/10)/ (5/14) ] * [ (4/7) * (7/10) / (5/14) ]
= [ (21/70) /(5/14) ] * [ (28/70)/(5/14)]
= 0.84 * 1.12
= 0.94
P(No | weather=sunny, humidity=low) = [ P(sunny|No)*P(No)/P(low) ] * [ P(low|No)*P(No)/P(low) ]
= [ (1/3) * (3/10)/ (5/14) ] * [ (1/3) * (3/10) / (5/14) ]
= [ (1/10) /(5/14) ] * [ (1/10)/(5/14)]
= 0.28 * 0.28
= 0.078
Since Yes (0.94) is having higher value than No (0.078) , it means the palyers are going to Play
Apriori Algorithm
Apriori algorithm is used for frequent itemset mining and association rule learning over
relational databases.
It proceeds by identifying the frequent individual items in the database and extending
them to larger and larger item sets as long as those item sets appear sufficiently often in
the database.
Components of Apriori algorithm
The given three components comprise the apriori algorithm.
1. Support
2. Confidence
3. Lift
Apriori Algorithm
Support
In Apriori, support refers to the frequency of occurrence of a particular itemset in a given
dataset.
It is used to identify the most frequent itemsets in a dataset and is an important
parameter in association rule mining.
The support value is calculated as the ratio of the number of transactions containing a
particular itemset to the total number of transactions in the dataset.
A high support value indicates that the itemset is frequently occurring in the dataset and
can be considered for further analysis.
Apriori Algorithm Example
Consider the following dataset, apply the Apriori algorithm to the dataset of transactions
and identify all frequent k-itemsets.
Let minimum support count is 2/9
Table 1
Apriori Algorithm Example
Step 1 a : Create a table containing support count of each item present in dataset – Called C1(candidate set).
To have support just count the number of times an item appears.
For an example l1 has support of 6 since its in Transaction (T1,T4,T5,T7,T8 and T9)
Apriori Algorithm Example
Step 1 b: Compare candidate set item’s support count with minimum support count(here min_support=2/9 if support_count of
candidate set items is less than min_support then remove those items).
For example support count for l1 = 6 , compare if 6 with 2 , if 6 is greater than 2 then put item l1 in itemset.
Apriori Algorithm Example
Step 2 a: Create candidate set using only items in the previous itemset. Now you need to have all pairs of the items in the itemset.
For example l1,l2 means count number of transactions where both l1 and l2 appear in Table 1. L1 and l2 appear in transaction T1,T4,T8
and T9 hence a support count of 4 (we always count items in Table 1)
Apriori Algorithm Example
Step 2 b: Create itemset from candidate set in step 2a. Compare support count in candidate set with support of item , if item support is
greater than minimum support move the item to the itemset.
L1,l4 and l3,l4 and l3,l5 and l4,l5 are removed from itemset since they have support less than 2.
Apriori Algorithm Example
Step 3 a: Create candidate set using only items in the previous itemset. Now you need to have sets of threes of the items in the
itemset.
For example l1,l2,l3 means count number of transactions where l1,l2 and l3 appear in Table 1. L1, l2 and l3 appear in transaction T8
and T9 (we always count items in Table 1) , hence a support count of 2.
Apriori Algorithm Example
Step 3 b: Create itemset from candidate set in step 3a. Compare support count in candidate set with support of item , if item support is
greater than minimum support move the item to the itemset.
K-Means Algorithm
K-Means Clustering is an unsupervised learning algorithm that is used to solve the clustering problems in machine learning or data
science.
In this topic, we will learn what is K-means clustering algorithm, how the algorithm works, along with the Python implementation of
k-means clustering.
K-Means Clustering is an Unsupervised Learning algorithm, which groups the unlabeled dataset into different clusters.
Here K defines the number of pre-defined clusters that need to be created in the process, as if K=2, there will be two clusters, and for
K=3, there will be three clusters, and so on.
The below diagram explains the working of the K-means Clustering Algorithm:
K-Means Algorithm
How does the K-Means Algorithm Work?
The working of the K-Means algorithm is explained in the below steps:
Step-1: Select the number K to decide the number of clusters.
Step-2: Select random K points or centroids. (It can be other from the input dataset).
Step-3: Assign each data point to their closest centroid, which will form the predefined K clusters.
Step-4: Calculate the variance and place a new centroid of each cluster.
Step-5: Repeat the third steps, which means reassign each datapoint to the new closest centroid of each cluster.
Step-6: If any reassignment occurs, then go to step-4 else go to FINISH.
Step-7: The model is ready.
K-Means Algorithm Example
Consider the following data points. Form two clusters with initial centroids being x1 and x2
x1 = (5,2) ; x2 = (5,3) ; x3 = (0,0) and x4 = (1,2)
Step 1: Select the number K to decide the number of clusters.
K = 2 (K means the number of clusters you are intending to have)
Step 2: Select random K points or centroids. (It can be other from the input dataset).
x1 = (5,2)
x2 = (4,3)
K-Means Algorithm Example
Step 3: Assign each data point to their closest centroid, which will form the predefined K clusters.
We use manhattan distance |a1 –a2| + |b1-b2| to calculate distance between two points . For example x3 to cluster 1
(5,2) is calculated as . X3= (0,0) and cluster1 = (5,2)
X3 can be represented as (a1,b1) similarly cluster1 can be represented as (a2,b2) hence
a1 = 0 , b1= 0 , a2 = 5 and b2 = 2
Using our formulae we get | (0- 5)| + |(0-2)|
= |-5| + |-2|
=5+2
=7
Cluster 1 (5,2) Cluster 2 (5,3)
x1 (5,2) 0 1
X2 (5,3) 1 0
X3 (0,0) 7 8
X4 (1,2) 4 5
K-Means Algorithm Example
Step 3 Continued: Assign each data point to their closest centroid, which will form the predefined K clusters.
Assign each data point to a cluster that has the smallest value . For example x1 will belong to cluster1 since 0 is smaller than 1 ,
similarly x3 will belong to cluster 1 as 7 is smaller than 8.
Cluster 1 (5,2) Cluster 2 (5,3) Cluster
x1 (5,2) 0 1 cluster1
X2 (5,3) 1 0 cluster2
X3 (0,0) 7 8 cluster1
X4 (1,2) 4 5 cluster1
K-Means Algorithm Example
Step 4: Calculate the variance and place a new centroid of each cluster.
Cluster1 contains x1 (5,2) , x3(0,0) and x4(1,2)
And cluster2 contains x2(5,3)
New centroid for cluster1 is the average of the data points
= ((5+0+1)/ 3 , (2+0+2)/3)
= (6/3 , 4/3)
= (2, 1.3)
New centroid for cluster2 is the average of the data points
= (5,3)
K-Means Algorithm Example
Step 5: Repeat step 3 with new centroids of cluster1 and cluster2
Cluster 1 (2,1.3) Cluster 2 (5,3)
x1 (5,2) 3.7 1
X2 (5,3) 4.7 0
X3 (0,0) 3.3 8
X4 (1,2) 1.7 5
Cluster 1 (5,2) Cluster 2 (5,3) Cluster
x1 (5,2) 3.7 1 cluster2
X2 (5,3) 4.7 0 cluster2
X3 (0,0) 3.3 8 cluster1
X4 (1,2) 1.7 5 cluster1
K-Means Algorithm Example
Step 6: Repeat step 4
Calculate the variance and place a new centroid of each cluster.
Cluster1 contains x3(0,0) and x4(1,2)
And cluster2 contains x1 (5,2) and x2(5,3)
New centroid for cluster1 is the average of the data points
= ((0+1)/ 2 , (0+2)/2)
= (1/2 , 2/2)
= (0.5, 1)
New centroid for cluster2 is the average of the data points
= ((5+5)/ 2 , (2+3)/3)
= (5 , 5/3)
= (5,1.7)
K-Means Algorithm Example
Step 7: Repeat step 3 with new centroids of cluster1 and cluster2
Cluster 1 (0.5,1) Cluster 2 (5,1.7)
x1 (5,2) 5.5 0.3
X2 (5,3) 6.5 1.3
X3 (0,0) 1.5 6.7
X4 (1,2) 1.5 5.3
Cluster 1 (5,2) Cluster 2 (5,3) Cluster
x1 (5,2) 5.5 0.3 cluster2
X2 (5,3) 6.5 1.3 cluster2
X3 (0,0) 1.5 6.7 cluster1
X4 (1,2) 1.5 5.3 cluster1
Since all the data points did not change the clusters they belong to in the previous step we stop our algorithm