Difference between Supervised and

Unsupervised Learning
Supervised and Unsupervised learning are the two techniques of machine
learning. But both the techniques are used in different scenarios and with
different datasets. Below the explanation of both learning methods along
with their difference table is given.

Supervised Machine Learning:

Supervised learning is a machine learning method in which models are
trained using labeled data. In supervised learning, models need to find the
mapping function to map the input variable (X) with the output variable
(Y).

Supervised learning needs supervision to train the model, which is similar

to as a student learns things in the presence of a teacher. Supervised
learning can be used for two types of
problems: Classification and Regression.

Learn more Supervised Machine Learning

Example: Suppose we have an image of different types of fruits. The task

of our supervised learning model is to identify the fruits and classify them
accordingly. So to identify the image in supervised learning, we will give
the input data as well as output for that, which means we will train the
model by the shape, size, color, and taste of each fruit. Once the training
is completed, we will test the model by giving the new set of fruit. The
model will identify the fruit and predict the output using a suitable
algorithm.

Unsupervised Machine Learning:

Unsupervised learning is another machine learning method in which
patterns inferred from the unlabeled input data. The goal of unsupervised
learning is to find the structure and patterns from the input data.
Unsupervised learning does not need any supervision. Instead, it finds
patterns from the data by its own.

Learn more Unsupervised Machine Learning

Unsupervised learning can be used for two types of

problems: Clustering and Association.

Example: To understand the unsupervised learning, we will use the

example given above. So unlike supervised learning, here we will not
provide any supervision to the model. We will just provide the input
dataset to the model and allow the model to find the patterns from the
data. With the help of a suitable algorithm, the model will train itself and
divide the fruits into different groups according to the most similar
features between them.

The main differences between Supervised and Unsupervised learning are

given below:
Supervised Learning Unsupervised Learning

Supervised learning Unsupervised learning algorithms are trained using

algorithms are unlabeled data.
trained using labeled
data.

Supervised learning Unsupervised learning model does not take any

model takes direct feedback.
feedback to check if
it is predicting
correct output or not.

Supervised learning Unsupervised learning model finds the hidden

model predicts the patterns in data.
output.

In supervised In unsupervised learning, only input data is

learning, input data provided to the model.
is provided to the
model along with the
output.

The goal of The goal of unsupervised learning is to find the

supervised learning hidden patterns and useful insights from the
is to train the model unknown dataset.
so that it can predict
the output when it is
given new data.

Supervised learning Unsupervised learning does not need any

needs supervision to supervision to train the model.
train the model.

Supervised learning Unsupervised Learning can be classified

can be categorized in Clustering and Associations problems.
in Classification an
d Regression proble
ms.

Supervised learning Unsupervised learning can be used for those cases

can be used for where we have only input data and no
those cases where corresponding output data.
we know the input as
well as
corresponding
outputs.

Supervised learning Unsupervised learning model may give less

model produces an accurate result as compared to supervised
accurate result. learning.

Supervised learning Unsupervised learning is more close to the true

is not close to true Artificial Intelligence as it learns similarly as a child
Artificial intelligence learns daily routine things by his experiences.
as in this, we first
train the model for
each data, and then
only it can predict
the correct output.
Note: The supervised and unsupervised learning both are the machine learning
methods, and selection of any of these learning depends on the factors related to the
structure and volume of your dataset and the use cases of the problem.

Regression Analysis in Machine learning

Regression analysis is a statistical method to model the relationship
between a dependent (target) and independent (predictor) variables with
one or more independent variables. More specifically, Regression analysis
helps us to understand how the value of the dependent variable is
changing corresponding to an independent variable when other
independent variables are held fixed. It predicts continuous/real values
such as temperature, age, salary, price, etc.

We can understand the concept of regression analysis using the below

example:

Example: Suppose there is a marketing company A, who does various

advertisement every year and get sales on that. The below list shows the
advertisement made by the company in the last 5 years and the
corresponding sales:

Now, the company wants to do the advertisement of $200 in the year

2019 and wants to know the prediction about the sales for this
year. So to solve such type of prediction problems in machine learning,
we need regression analysis.

Regression is a supervised learning technique which helps in finding the

correlation between variables and enables us to predict the continuous
output variable based on the one or more predictor variables. It is mainly
used for prediction, forecasting, time series modeling, and
determining the causal-effect relationship between variables.

In Regression, we plot a graph between the variables which best fits the
given datapoints, using this plot, the machine learning model can make
predictions about the data. In simple words, "Regression shows a line
or curve that passes through all the datapoints on target-
predictor graph in such a way that the vertical distance between
the datapoints and the regression line is minimum." The distance
between datapoints and line tells whether a model has captured a strong
relationship or not.

Some examples of regression can be as:

o Prediction of rain using temperature and other factors

o Determining Market trends
o Prediction of road accidents due to rash driving.

Terminologies Related to the Regression Analysis:

o Dependent Variable: The main factor in Regression analysis which we
want to predict or understand is called the dependent variable. It is also
called target variable.
o Independent Variable: The factors which affect the dependent variables
or which are used to predict the values of the dependent variables are
called independent variable, also called as a predictor.
o Outliers: Outlier is an observation which contains either very low value or
very high value in comparison to other observed values. An outlier may
hamper the result, so it should be avoided.
o Multicollinearity: If the independent variables are highly correlated with
each other than other variables, then such condition is called
Multicollinearity. It should not be present in the dataset, because it creates
problem while ranking the most affecting variable.
o Underfitting and Overfitting: If our algorithm works well with the
training dataset but not well with test dataset, then such problem is
called Overfitting. And if our algorithm does not perform well even with
training dataset, then such problem is called underfitting.

Why do we use Regression Analysis?

As mentioned above, Regression analysis helps in the prediction of a
continuous variable. There are various scenarios in the real world where
we need some future predictions such as weather condition, sales
prediction, marketing trends, etc., for such case we need some
technology which can make predictions more accurately. So for such case
we need Regression analysis which is a statistical method and used in
machine learning and data science. Below are some other reasons for
using Regression analysis:

o Regression estimates the relationship between the target and the

independent variable.
o It is used to find the trends in data.
o It helps to predict real/continuous values.
o By performing the regression, we can confidently determine the most
important factor, the least important factor, and how each factor
is affecting the other factors.

Types of Regression
There are various types of regressions which are used in data science and
machine learning. Each type has its own importance on different
scenarios, but at the core, all the regression methods analyze the effect of
the independent variable on dependent variables. Here we are discussing
some important types of regression which are given below:

o Linear Regression
o Logistic Regression
o Polynomial Regression
o Support Vector Regression
o Decision Tree Regression
o Random Forest Regression
o Ridge Regression
o Lasso Regression:
Linear Regression:
o Linear regression is a statistical regression method which is used for
predictive analysis.
o It is one of the very simple and easy algorithms which works on regression
and shows the relationship between the continuous variables.
o It is used for solving the regression problem in machine learning.
o Linear regression shows the linear relationship between the independent
variable (X-axis) and the dependent variable (Y-axis), hence called linear
regression.
o If there is only one input variable (x), then such linear regression is
called simple linear regression. And if there is more than one input
variable, then such linear regression is called multiple linear
regression.
o The relationship between variables in the linear regression model can be
explained using the below image. Here we are predicting the salary of an
employee on the basis of the year of experience.
 o Below is the mathematical equation for Linear regression:

1. Y= aX+b

Here, Y = dependent variables (target variables),

X= Independent variables (predictor variables),
a and b are the linear coefficients

Some popular applications of linear regression are:

o Analyzing trends and sales estimates

o Salary forecasting
o Real estate prediction
o Arriving at ETAs in traffic.

Cluster analysis in Machine

learning
C lustering is a statistical classification approach for the

supervised learning. Cluster analysis or clustering is the

task of grouping a set of objects in such a way that objects
in the same group (called a cluster) are more similar to
each other than to those in other groups (clusters).

It is a main task of exploratory data mining, and a common

technique for statistical data analysis, used in many fields
including machine learning, pattern recognition, image
analysis and data compression.

Clustering can be achieved by various algorithms that

differ significantly in their understanding of what
constitutes a cluster and how to efficiently find them.
Popular notions of clusters include groups with small
distances between cluster members, dense areas of the
data space, intervals or particular statistical distributions.
Applications of unsupervised machine learning
Some applications of unsupervised machine learning
techniques are:

 Clustering automatically split the dataset into groups

base on their similarities

 Anomaly detection can discover unusual data points in

your dataset. It is useful for finding fraudulent
transactions

 Association mining identifies sets of items which often

occur together in your dataset

 Latent variable models are widely used for data

preprocessing. Like reducing the number of features in
a dataset or decomposing the dataset into multiple
components

Disadvantages of Unsupervised Learning

 You cannot get precise information regarding data
sorting, and the output as data used in unsupervised
learning is labeled and not known

 Less accuracy of the results is because the input data is

not known and not labeled by people in advance. This
means that the machine requires to do this itself.

 The spectral classes do not always correspond to

informational classes.
 The user needs to spend time interpreting and label the
classes which follow that classification.

 Spectral properties of classes can also change over time

so you can’t have the same class information while
moving from one image to another.

The notion of a cluster, as found by different algorithms,

varies significantly in its properties. Understanding these
“cluster models” is key to understanding the differences
between the various algorithms. Typical cluster models
include:

 Connectivity models( Hierarchy clustering): clustering

builds models based on distance connectivity.

 Centroid models: k-means algorithm represents each

cluster by a single mean vector.

 Distribution models: clusters are modeled using

statistical distributions, such as multivariate normal
distributions.

 Density models: DBSCAN and OPTICS defines clusters

as connected dense regions in the data space.

A “clustering” is essentially a set of such clusters, usually

containing all objects in the data set. Additionally, it may
specify the relationship of the clusters to each other, for
example, a hierarchy of clusters embedded in each other.
Clustering can be roughly distinguished as:

 Hard clustering: each object belongs to a cluster or not

 Soft clustering (also: fuzzy clustering): each object

belongs to each cluster to a certain degree (for
example, a likelihood of belonging to the cluster).

1. Connectivity-based clustering (hierarchical

clustering)

C onnectivity-based clustering, also known as

hierarchical clustering, is based on the core idea of objects

being more related to nearby objects than to objects
farther away. These algorithms connect “objects” to form
“clusters” based on their distance. A cluster can be
described largely by the maximum distance needed to
connect parts of the cluster. At different distances,
different clusters will form, which can be represented
using a dendogram, which explains where the common
name “hierarchical clustering” comes from: these
algorithms do not provide a single partitioning of the data
set, but instead provide an extensive hierarchy of clusters
that merge with each other at certain distances.

These methods will not produce a unique partitioning of

the data set, but a hierarchy from which the user still
needs to choose appropriate clusters. They are not very
robust towards outliers, which will either show up as
additional clusters or even cause other clusters to merge.

Hierarchical clustering relies upon using these clustering

techniques to find a hierarchy of clusters, where this
hierarchy resembles a tree structure, called a
dendrogram. “Hierarchical clustering is the hierarchical
decomposition of the data based on group similarities”

Hierarchical clustering based on continents

2. Centroid models (k-means):

In centroid-based clustering, clusters are represented

by a central vector, which may not necessarily be a

member of the data set. When the number of clusters is
fixed to k, k-means clustering gives a formal definition as
an optimization problem: find the k cluster centers and
assign the objects to the nearest cluster center, such that
the squared distances from the cluster are minimized.

Kmeans clustering
K means clustering with different steps

3. Distribution models

T he clustering model most closely related to statistics is

based on distribution models. Clusters can then easily be

defined as objects belonging most likely to the same
distribution. A convenient property of this approach is that
this closely resembles the way artificial data sets are
generated: by sampling random objects from a
distribution.
While the theoretical foundation of these methods is
excellent, they suffer from one key problem known as
overfitting, unless constraints are put on the model
complexity. A more complex model will usually be able to
explain the data better, which makes choosing the
appropriate model complexity inherently difficult.

Distribution-based clustering produces complex models for

clusters that can capture correlation and dependence
between attributes. However, these algorithms put an
extra burden on the user: for many real data sets, there
may be no concisely defined mathematical model (e.g.
assuming Gaussian distributions is a rather strong
assumption on the data).
Different types of Gaussian distribution mixture models — Source: [Link]

4. Density models

In density-based clustering, clusters are defined as

areas of higher density than the remainder of the data set.

Objects in these sparse areas — that are required to
separate clusters — are usually considered to be noise and
border points.

The most popular density based clustering method

is DBSCAN. However, it only connects points that satisfy a
density criterion, in the original variant defined as a
minimum number of other objects within this radius. A
cluster consists of all density-connected objects (which can
form a cluster of an arbitrary shape, in contrast to many
other methods) plus all objects that are within these
objects’ range. Another interesting property of DBSCAN is
that its complexity is fairly low — it requires a linear
number of range queries on the database — and that it will
discover essentially the same results OPTICS is a
generalization of DBSCAN that removes the need to
choose an appropriate value for the range parameter ,and
produces a hierarchical result related to that of linkage
clustering.

K-Means Clustering Algorithm

K-Means Clustering is an unsupervised learning algorithm that is used to
solve the clustering problems in machine learning or data science. In this
topic, we will learn what is K-means clustering algorithm, how the
algorithm works, along with the Python implementation of k-means
clustering.

What is K-Means Algorithm?

K-Means Clustering is an Unsupervised Learning algorithm, which groups
the unlabeled dataset into different clusters. Here K defines the number of
pre-defined clusters that need to be created in the process, as if K=2,
there will be two clusters, and for K=3, there will be three clusters, and so
on.

It is an iterative algorithm that divides the unlabeled dataset into k different clusters in such a
way that each dataset belongs only one group that has similar properties.

It allows us to cluster the data into different groups and a convenient way
to discover the categories of groups in the unlabeled dataset on its own
without the need for any training.
It is a centroid-based algorithm, where each cluster is associated with a
centroid. The main aim of this algorithm is to minimize the sum of
distances between the data point and their corresponding clusters.

The algorithm takes the unlabeled dataset as input, divides the dataset
into k-number of clusters, and repeats the process until it does not find
the best clusters. The value of k should be predetermined in this
algorithm.

The k-means clustering algorithm mainly performs two tasks:

o Determines the best value for K center points or centroids by an iterative

process.
o Assigns each data point to its closest k-center. Those data points which
are near to the particular k-center, create a cluster.

Hence each cluster has datapoints with some commonalities, and it is

away from other clusters.

The below diagram explains the working of the K-means Clustering

Algorithm:

How does the K-Means Algorithm Work?

The working of the K-Means algorithm is explained in the below steps:

Step-1: Select the number K to decide the number of clusters.

Step-2: Select random K points or centroids. (It can be other from the
input dataset).
Step-3: Assign each data point to their closest centroid, which will form
the predefined K clusters.

Step-4: Calculate the variance and place a new centroid of each cluster.

Step-5: Repeat the third steps, which means reassign each datapoint to
the new closest centroid of each cluster.

Step-6: If any reassignment occurs, then go to step-4 else go to FINISH.

Step-7: The model is ready.

Let's understand the above steps by considering the visual plots:

Suppose we have two variables M1 and M2. The x-y axis scatter plot of
these two variables is given below:

o Let's take number k of clusters, i.e., K=2, to identify the dataset and to put
them into different clusters. It means here we will try to group these
datasets into two different clusters.
o We need to choose some random k points or centroid to form the cluster.
These points can be either the points from the dataset or any other point.
So, here we are selecting the below two points as k points, which are not
 the part of our dataset. Consider the below image:

o Now we will assign each data point of the scatter plot to its closest K-point
or centroid. We will compute it by applying some mathematics that we
have studied to calculate the distance between two points. So, we will
draw a median between both the centroids. Consider the below image:
From the above image, it is clear that points left side of the line is near to
the K1 or blue centroid, and points to the right of the line are close to the
yellow centroid. Let's color them as blue and yellow for clear visualization.

o As we need to find the closest cluster, so we will repeat the process by

choosing a new centroid. To choose the new centroids, we will compute
the center of gravity of these centroids, and will find new centroids as
 below:

o Next, we will reassign each datapoint to the new centroid. For this, we will
repeat the same process of finding a median line. The median will be like
below image:
From the above image, we can see, one yellow point is on the left side of
the line, and two blue points are right to the line. So, these three points
will be assigned to new centroids.

As reassignment has taken place, so we will again go to the step-4, which

is finding new centroids or K-points.
o We will repeat the process by finding the center of gravity of centroids, so
the new centroids will be as shown in the below image:

o As we got the new centroids so again will draw the median line and
reassign the data points. So, the image will be:
 o We can see in the above image; there are no dissimilar data points on
either side of the line, which means our model is formed. Consider the
below image:

As our model is ready, so we can now remove the assumed centroids, and
the two final clusters will be as shown in the below image:
EM Algorithm in Machine Learning
Expectation-Maximization (EM)

In most real-life applications of machine learning, it is found that several

relevant learning features are available, but very few of them are
observable, and the rest are unobservable. If the variables are observable,
then it can predict the value using instances. On the other hand, the
variables which are latent or directly not observable, for such variables

What is an EM algorithm?
The Expectation-Maximization (EM) algorithm is defined as the
combination of various unsupervised machine learning algorithms, which
is used to determine the local maximum likelihood estimates
(MLE) or maximum a posteriori estimates (MAP) for unobservable
variables in statistical models. Further, it is a technique to find maximum
likelihood estimation when the latent variables are present. It is also
referred to as the latent variable model.

A latent variable model consists of both observable and unobservable

variables where observable can be predicted while unobserved are
inferred from the observed variable. These unobservable variables are
known as latent variables.

Key Points:
 o It is known as the latent variable model to determine MLE and MAP
parameters for latent variables.
o It is used to predict values of parameters in instances where data is
missing or unobservable for learning, and this is done until convergence of
the values occurs.

EM Algorithm
The EM algorithm is the combination of various unsupervised ML
algorithms, such as the k-means clustering algorithm. Being an
iterative approach, it consists of two modes. In the first mode, we
estimate the missing or latent variables. Hence it is referred to as
the Expectation/estimation step (E-step). Further, the other mode is
used to optimize the parameters of the models so that it can explain the
data more clearly. The second mode is known as the maximization-step
or M-step.

o Expectation step (E - step): It involves the estimation (guess) of all

missing values in the dataset so that after completing this step, there
should not be any missing value.
o Maximization step (M - step): This step involves the use of estimated
data in the E-step and updating the parameters.
o Repeat E-step and M-step until the convergence of the values occurs.

The primary goal of the EM algorithm is to use the available observed data
of the dataset to estimate the missing data of the latent variables and
then use that data to update the values of the parameters in the M-step.
What is Convergence in the EM algorithm?
Convergence is defined as the specific situation in probability
based on intuition, e.g., if there are two random variables that have
very less difference in their probability, then they are known as
converged. In other words, whenever the values of given variables are
matched with each other, it is called convergence.

Steps in EM Algorithm
The EM algorithm is completed mainly in 4 steps, which include
Initialization Step, Expectation Step, Maximization Step, and
convergence Step. These steps are explained as follows:

o 1st Step: The very first step is to initialize the parameter values. Further,
the system is provided with incomplete observed data with the
assumption that data is obtained from a specific model.

o 2nd Step: This step is known as Expectation or E-Step, which is used to

estimate or guess the values of the missing or incomplete data using the
observed data. Further, E-step primarily updates the variables.
o 3rd Step: This step is known as Maximization or M-step, where we use
complete data obtained from the 2 nd step to update the parameter values.
Further, M-step primarily updates the hypothesis.
 o 4th step: The last step is to check if the values of latent variables are
converging or not. If it gets "yes", then stop the process; else, repeat the
process from step 2 until the convergence occurs.

Advantages of EM algorithm
o It is very easy to implement the first two basic steps of the EM algorithm in
various machine learning problems, which are E-step and M- step.
o It is mostly guaranteed that likelihood will enhance after each iteration.
o It often generates a solution for the M-step in the closed form.

Disadvantages of EM algorithm
o The convergence of the EM algorithm is very slow.
o It can make convergence for the local optima only.
o It takes both forward and backward probability into consideration. It is
opposite to that of numerical optimization, which takes only forward
probabilities.

What is association rule mining?

Grokking the Behavioral Interview

Many candidates are rejected or down-leveled in technical interviews due to poor performance in
behavioral or cultural fit interviews. Ace your interviews with this free course, where you will
practice confidently tackling behavioral interview questions.
Get Free Course

Association rule mining is a data mining technique that aims to discover interesting
relationships, patterns, and correlations within large datasets. It focuses on identifying strong
associations between different items or variables in the data. It presents these associations in the
form of if-then rules, commonly known as association rules.

An association rule consists of an antecedent (if part) and a consequent (then part). The
dataset contains an antecedent, and we derive a consequent by using the antecedent.

Metrics for evaluating association rules

Association rules are carefully derived from the dataset. Several metrics are commonly used to
evaluate the performance of association rule mining algorithms. Let us consider the following
transaction table.

Transaction ID Items purchased

1 Item1, Item2
 2 Item1, Item3 ,Item4, Item5
3 Item2, Item3, Item4, Item6
4 Item1, Item2, Item3, Item4
5 Item1, Item2, Item3, Item6

Support

Support is the proportion of transactions in the dataset that contain a specific itemset. It indicates
the frequency with which the itemset appears in the data. Higher support values indicate that the
rule is more common or significant in the dataset. Rules with low support are considered less
relevant.

In the above table, we have:

�������{����4}=35Support{Item4}=53

There are five transactions; three of those have ����4Item4 appearing in them.

�������{����2,����6}=25Support{Item2,Item6}=52

Out of five transactions,{����2, ����6}{Item2, Item6} appear together in two

transactions.

Confidence

It is the ratio of the number of transactions containing both the antecedent and the consequent to
the number of transactions containing only the antecedent.

����������{�→�}=�������{�→�}�������{�}Con
fidence{X→Y}=support{X}support{X→Y}

����������{����2→����6}=�������{����2→�
���6}�������{����2}Confidence{Item2→Item6}=support{Item
2}support{Item2→Item6}

����������{����2→����6}=2545=0.5Confidence{Item2→It
em6}=5452=0.5

It is not symmetric, meaning the confidence for {�→�}{X→Y} is not the same
as {�→�}{Y→X}.

Lift

Lift quantifies how likely the consequent is to occur when the antecedent is present compared to
when the two events are independent.

����{�→�}=�������{�→�}�������{�}×�������{
�}=����������{�→�}�������{�}Lift{X→Y}=support{X}×s
upport{Y}support{X→Y}=support{Y}confidence{X→Y}
����{����2→����6}=����������{����2→����6
}�������{����6}=1225=1.25Lift{Item2→Item6}=support{Item6}con
fidence{Item2→Item6}=5221=1.25

Applications

Different fields use association rules for analysis and calculations. Some examples are:

 Market basket analysis: In market basket analysis, association rule mining finds items
that customers frequently buy together to boost sales and meet some business
objectives.
 Healthcare: Researchers use association rules to analyze patient data, identify co-
occurring medical conditions, and discover potential risk factors. This information aids in
disease diagnosis, treatment planning, and medical research.

Conclusion

In conclusion, association rule mining is a technique that discovers interesting relationships and
patterns within large datasets. It involves identifying frequent itemsets, which are combinations of
items that appear together frequently in the data, and generating association rules from these
frequent itemsets.

Association Rule Learning

Association rule learning is a type of unsupervised learning technique that
checks for the dependency of one data item on another data item and
maps accordingly so that it can be more profitable. It tries to find some
interesting relations or associations among the variables of dataset. It is
based on different rules to discover the interesting relations between
variables in the database.

The association rule learning is one of the very important concepts

of machine learning, and it is employed in Market Basket analysis,
Web usage mining, continuous production, etc. Here market basket
analysis is a technique used by the various big retailer to discover the
associations between items. We can understand it by taking an example
of a supermarket, as in a supermarket, all products that are purchased
together are put together.

For example, if a customer buys bread, he most likely can also buy butter,
eggs, or milk, so these products are stored within a shelf or mostly
nearby. Consider the below diagram:
Association rule learning can be divided into three types of algorithms:

1. Apriori
2. Eclat
3. F-P Growth Algorithm

We will understand these algorithms in later chapters.

How does Association Rule Learning work?

Association rule learning works on the concept of If and Else Statement,
such as if A then B.

Here the If element is called antecedent, and then statement is called

as Consequent. These types of relationships where we can find out some
association or relation between two items is known as single cardinality. It
is all about creating rules, and if the number of items increases, then
cardinality also increases accordingly. So, to measure the associations
between thousands of data items, there are several metrics. These
metrics are given below:
 o Support
o Confidence
o Lift

Let's understand each of them:

Support
Support is the frequency of A or how frequently an item appears in the
dataset. It is defined as the fraction of the transaction T that contains the
itemset X. If there are X datasets, then for transactions T, it can be written
as:

Confidence
Confidence indicates how often the rule has been found to be true. Or how
often the items X and Y occur together in the dataset when the
occurrence of X is already given. It is the ratio of the transaction that
contains X and Y to the number of records that contain X.

Types of Association Rule Lerning

Association rule learning can be divided into three algorithms:

Apriori Algorithm
This algorithm uses frequent datasets to generate association rules. It is
designed to work on the databases that contain transactions. This
algorithm uses a breadth-first search and Hash Tree to calculate the
itemset efficiently.

It is mainly used for market basket analysis and helps to understand the
products that can be bought together. It can also be used in the
healthcare field to find drug reactions for patients.
Eclat Algorithm
Eclat algorithm stands for Equivalence Class Transformation. This
algorithm uses a depth-first search technique to find frequent itemsets in
a transaction database. It performs faster execution than Apriori
Algorithm.

F-P Growth Algorithm

The F-P growth algorithm stands for Frequent Pattern, and it is the
improved version of the Apriori Algorithm. It represents the database in
the form of a tree structure that is known as a frequent pattern or tree.
The purpose of this frequent tree is to extract the most frequent patterns.

Applications of Association Rule Learning

It has various applications in machine learning and data mining. Below are
some popular applications of association rule learning:

o Market Basket Analysis: It is one of the popular examples and

applications of association rule mining. This technique is commonly used
by big retailers to determine the association between items.
o Medical Diagnosis: With the help of association rules, patients can be
cured easily, as it helps in identifying the probability of illness for a
particular disease.
o Protein Sequence: The association rules help in determining the
synthesis of artificial Proteins.
o It is also used for the Catalog Design and Loss-leader Analysis and
many more other applications.

Apriori Algorithm in Machine Learning

The Apriori algorithm uses frequent itemsets to generate association
rules, and it is designed to work on the databases that contain
transactions. With the help of these association rule, it determines how
strongly or how weakly two objects are connected. This algorithm uses
a breadth-first search and Hash Tree to calculate the itemset
associations efficiently. It is the iterative process for finding the frequent
itemsets from the large dataset.

This algorithm was given by the R. Agrawal and Srikant in the

year 1994. It is mainly used for market basket analysis and helps to find
those products that can be bought together. It can also be used in the
healthcare field to find drug reactions for patients.

What is Frequent Itemset?

Frequent itemsets are those items whose support is greater than the
threshold value or user-specified minimum support. It means if A & B are
the frequent itemsets together, then individually A and B should also be
the frequent itemset.

Suppose there are the two transactions: A= {1,2,3,4,5}, and B= {2,3,7},

in these two transactions, 2 and 3 are the frequent itemsets.

Note: To better understand the apriori algorithm, and related term such as support and
confidence, it is recommended to understand the association rule learning.

Steps for Apriori Algorithm

Below are the steps for the apriori algorithm:

Step-1: Determine the support of itemsets in the transactional database,

and select the minimum support and confidence.

Step-2: Take all supports in the transaction with higher support value
than the minimum or selected support value.

Step-3: Find all the rules of these subsets that have higher confidence
value than the threshold or minimum confidence.

Step-4: Sort the rules as the decreasing order of lift.

Apriori Algorithm Working

We will understand the apriori algorithm using an example and
mathematical calculation:

Example: Suppose we have the following dataset that has various

transactions, and from this dataset, we need to find the frequent itemsets
and generate the association rules using the Apriori algorithm:
Solution:
Step-1: Calculating C1 and L1:
o In the first step, we will create a table that contains support count (The
frequency of each itemset individually in the dataset) of each itemset in
the given dataset. This table is called the Candidate set or C1.

o Now, we will take out all the itemsets that have the greater support count
that the Minimum Support (2). It will give us the table for the frequent
itemset L1.
Since all the itemsets have greater or equal support count than the
minimum support, except the E, so E itemset will be removed.
Step-2: Candidate Generation C2, and L2:
o In this step, we will generate C2 with the help of L1. In C2, we will create
the pair of the itemsets of L1 in the form of subsets.
o After creating the subsets, we will again find the support count from the
main transaction table of datasets, i.e., how many times these pairs have
occurred together in the given dataset. So, we will get the below table for
C2:

o Again, we need to compare the C2 Support count with the minimum

support count, and after comparing, the itemset with less support count
will be eliminated from the table C2. It will give us the below table for L2

Step-3: Candidate generation C3, and L3:

o For C3, we will repeat the same two processes, but now we will form the
C3 table with subsets of three itemsets together, and will calculate the
support count from the dataset. It will give the below table:

o Now we will create the L3 table. As we can see from the above C3 table,
there is only one combination of itemset that has support count equal to
 the minimum support count. So, the L3 will have only one combination,
i.e., {A, B, C}.

Step-4: Finding the association rules for the subsets:

To generate the association rules, first, we will create a new table with the
possible rules from the occurred combination {A, B.C}. For all the rules,
we will calculate the Confidence using formula sup( A ^B)/A. After
calculating the confidence value for all rules, we will exclude the rules that
have less confidence than the minimum threshold(50%).

Consider the below table:

Rules Support Confidence

A ^B → C 2 Sup{(A ^B) ^C}/sup(A ^B)= 2/4=0.5=50%

B^C → A 2 Sup{(B^C) ^A}/sup(B ^C)= 2/4=0.5=50%

A^C → B 2 Sup{(A ^C) ^B}/sup(A ^C)= 2/4=0.5=50%

C→ A ^B 2 Sup{(C^( A ^B)}/sup(C)= 2/5=0.4=40%

A→ B^C 2 Sup{(A^( B ^C)}/sup(A)= 2/6=0.33=33.33%

B→ B^C 2 Sup{(B^( B ^C)}/sup(B)= 2/7=0.28=28%

As the given threshold or minimum confidence is 50%, so the first three

rules A ^B → C, B^C → A, and A^C → B can be considered as the strong
association rules for the given problem.

Advantages of Apriori Algorithm

o This is easy to understand algorithm
o The join and prune steps of the algorithm can be easily implemented on
large datasets.

Disadvantages of Apriori Algorithm

o The apriori algorithm works slow compared to other algorithms.
 o The overall performance can be reduced as it scans the database for
multiple times.
o The time complexity and space complexity of the apriori algorithm is
O(2D), which is very high. Here D represents the horizontal width present
in the database.

Note: We can choose the number of clusters equal to the given data points. If we choose
the number of clusters equal to the data points, then the value of WCSS becomes zero,
and that will be the endpoint of the plot.

Non linear Regression examples – ML

 Read
 Discuss
 Courses
 Practice




In this article, we will see some examples of non-linear regression
which are generally used in regression analysis the reason being
that most of the real-world data follow highly complex and non-
linear relationships between the dependent and the independent
variables.

What is a Non-Linear Regression?

Non-Linear regression is a type of polynomial regression. It is a
method to model a non-linear relationship between the dependent
and independent variables. It is used in place when the data shows a
curvy trend and linear regression would not produce very accurate
results when compared to non-linear regression. This is because in
linear regression it is pre-assumed that the data is linear.
There are many different regressions that exist and can be used to
fit whatever the dataset looks like such as quadratic, cubic
regression, and so on to infinite degrees according to our
requirement.

How does a Non-Linear Regression work?

If we observe closely then we will realize that to evolve from linear
regression to non-linear regression. We are just supposed to add the
higher-order terms of the dependent features in the feature space.
This is sometimes also known as feature engineering but not
exactly.
The addition of non-linear terms is what allows us to fit a curvilinear
model to the data at hand. Even though the non-linear regression is
similar to the linear one but the different types of challenges are
faced by the Machine Learning practitioner while training such a
model. And hence several established methods, such as Levenberg-
Marquardt and Gauss-Newton, are used to develop nonlinear
models.