Revolutionising B.

Tech
Programme: [Link] - CSE (Cyber Security) Date: 18-11-2023

Introduction to
Artificial Intelligence & Machine Learning
(22CSDE21)

Module-3
 Machine Learning Models
Feature Selection: What
You have some data, and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
Machine learning models can be broadly categorized into three main types based on the nature of
learning and the type of task they perform:
Feature Selection: What
Supervised Learning Models: In supervised learning, models are trained on labeled data, where the
input data is paired with the correct output. These models learn from the labeled examples to
make predictions or decisions. Types of supervised learning models include:
You have some data, and you want to use it to build a
Classification Models: These models predict categorical labels or classes. Examples include
classifier,
logistic regression, so random
decision trees, that forests,
you etc.,can predict something
Regression Models:(e.g. These models
likelihood of predict
cancer)continuous numerical values. Examples include
linear regression, multiple regression etc.,
Unsupervised Learning Models: In unsupervised learning, models are given input data without
The data
explicit output labels. hasis10,000
The goal fields (features)
to find patterns, structure, or relationships within the data. Types
of unsupervised learning models include:
you need
Clustering Models: These modelsto group
cut itsimilar
down data
to 1,000
pointsfields before
together based on some measure
of similarity. you try machine learning. Which 1,000?
Dimensionality Reduction Models: These models aim to reduce the number of features while
preserving as much
The information
process as possible.
of choosing the 1,000 fields to use is called
Reinforcement Learning Models: In reinforcement learning, models learn by interacting with an
Feature Selection
environment to achieve a specific goal. The model receives feedback in the form of rewards or
penalties based on its actions.
Feature Selection: What
You have some data, and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
 Learning from Unclassified Data (Unlabelled Data)
Feature Selection: What
• Learning from unclassified (or) unlabeled data typically falls under the domain of
unsupervised learning in machine learning.
• In this scenario,You
the data
haveprovided
somedoesn't
data, have
and explicit
you wantlabelstoor use
categories
it to assigned
build ato it.
• The goal is to find patterns, structures, or relationships within the data without any
classifier, so that you can predict something
predefined classifications.
(e.g. likelihood
• Unsupervised learning techniquesof
aimcancer)
to explore the inherent structure or hidden patterns
within the data.
• The target variable
The data‘Y’ is
has 10,000 fields
completely absent (features)
from the dataset in unsupervised learning.

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
Feature Selection: What
You have some data, and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
 Clustering
• Feature
Clustering in machineSelection: What
learning is an unsupervised learning technique used to group similar data
points or objects into clusters.
You have some data, and you want to use it to build a
• Unlike supervised learning, where the algorithm is trained on labeled data, clustering deals with
classifier, so that you can predict something
unlabeled data and aimslikelihood
(e.g. to find patterns within the dataset.
of cancer)
• The primary objective of clustering in machine learning is to group similar data points together
into clusters, The
baseddata has 10,000
on certain criteria fields (features)
or similarity measures, without any predefined labels or
categories
you need to cut it down to 1,000 fields before
• Example for clustering
youalgorithms are: learning. Which 1,000?
try machine
K-Means clustering, hierarchical clustering, Gaussian Mixture Models (GMM), and more.
The process of choosing the 1,000 fields to use is called
• Clustering algorithms use a distance or similarity metric (e.g., Euclidean distance, cosine similarity)
Feature Selection
to measure the similarity between data points.
 Under-fitting
Feature Selection: What
You have some data, and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
 Applications of Clustering
Here's a list of applications across various domains where clustering techniques find utility:
Feature Selection: What
• Marketing and Customer Segmentation
• Healthcare
You have some data, and you want to use it to build a
• Cybersecurity: Clustering is used to group similar patterns of network traffic or system behavior, which can
classifier,
help in detecting and so that you can predict something
preventing cyberattacks
• Image Processing(e.g. likelihood
and Computer Vision of cancer)
• Anomaly Detection
• Crime analysisThe data has 10,000 fields (features)
• Recommendation Systems
• Document Clustering
you need to cut it down to 1,000 fields before
•
you try machine learning. Which 1,000?
Social Network Analysis
• Time Series Analysis
The process of choosing the 1,000 fields to use is called
• Fraud Detection
Feature Selection
• Climate Science
• Traffic Analysis etc.,
 Hard vs. Soft clustering
Feature Selection: What
Hard and soft clustering are two distinct approaches in clustering algorithms that handle
the assignment ofYou
datahave
pointssome
to clusters
data,differently.
and you want to use it to build a
Hard Clustering:classifier, so that you can predict something
• Each data sample (or)
(e.g. object (or)of
likelihood document
cancer)belongs to exactly one cluster.
• More common and easier to do.
• Examples ofThe data
Hard has 10,000
Clustering fields (features)
Algorithms: K-means, K-medoids (PAM) etc.
Soft Clustering:
• In soft clustering, you
also need
knowntoascut it down
fuzzy to 1,000
clustering, datafields
pointsbefore
can belong to multiple
clusters. you try machine learning. Which 1,000?
• A document can belong to more than one cluster.
The process of choosing the 1,000 fields to use is called
• Examples Feature
of Soft Selection
Clustering Algorithms: Fuzzy C-means (FCM), Gaussian Mixture
Models (GMM) etc.
 Comparison between Classification and Clustering
Feature Selection: What
You have some data, and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
Feature Selection: What
You have some data, and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
Feature Selection: What
You have some data, and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
 Distance Measures
Feature Selection: What
Distance measure will determine how the similarity of two elements is calculated and it will
influence the shape of the clusters.
You have some data, and you want to use it to build a
Consider a distance function d(x, y) that can compute the distance between any two points x, y
classifier, so that you can predict something
and must satisfy the following
(e.g. properties
likelihood for any choice of elements.
of cancer)
•d(x, y) ≥0 (non−negative)
The data has 10,000 fields (features)
•d(x, y) = 0,⟺ x = y (identity)
you need to cut it down to 1,000 fields before
•d(x, y) = d(y, x) (symmetry)
you try machine learning. Which 1,000?

•d(x, z) ≤ d(x, The

y) +process
d(y, z) of
(triangle
choosinginequality)
the 1,000 fields to use is called
Feature Selection
The first bullet claims that the distance between any two things can never be negative (hence called “non-negativity”) and

that the distance between two things can only be zero if those two things are actually the same thing.
 Distance
Feature Selection: What Measures
Euclidean Distance:
• This is the mostYou havedistance
common some measure
data, and
usedyou want to use it to build a
in clustering.
• If p = (𝑝1 , 𝑝classifier,
2 ,…, 𝑝𝑛 ) and so
q = that, 𝑞 ,…,
(𝑞1 2
you𝑞 ) can
𝑛 are twopredict
points , something
then the distance (d)
(e.g. likelihood of cancer)
from p to q, or from q to p is given by
The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection

We can calculate the distance between points and then define the similarity between them.
Feature Selection: What
You have some data, and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
Therefore, the Euclidean distance between points A(3, 4) and B(6, 8) is 5 units.
The Euclidean distance formula can be extended to higher dimensions as well.
Feature Selection: What
You have some data, and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
 Feature Selection: What
You have some data, and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Example:
Feature Selection
Feature Selection: What
You have some data, and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
 Feature Selection: What
• K-Means clustering is an unsupervised machine learning technique.
You have some data, and you want to use it to build a
• It is used for grouping data points into K distinct clusters based on their attributes or features.
classifier, so that you can predict something
• It's a fundamental and widely used algorithm for clustering tasks because of its simplicity and
efficiency.
(e.g. likelihood of cancer)
• It partitions the given data set into k predefined distinct clusters.
The data has 10,000 fields (features)
• A cluster is defined as a collection of data points exhibiting certain similarities.
It partitions the data set
yousuchneed
that-to cut it down to 1,000 fields before
• Each data point belongs
you totrya cluster
machinewith the nearest mean.
learning. Which 1,000?
• Data points belonging to one cluster have high degree of similarity.
• Data pointsThe processto of
belonging choosing
different the have
clusters 1,000 fields
high to use
degree is called
of dissimilarity.
Feature Selection
 Feature Selection: What
The following are the steps used in the k-means clustering algorithm .
[Link]:
Choose the number of clusters (k) you want to identify within the dataset and Randomly select k data
points from theYoudatasethave some
as initial clusterdata, and you want to use it to build a
centroids.
[Link]: classifier, so that you can predict something
Calculate the distance (often Euclidean
(e.g. likelihood distance) between each data point and the centroids and
of cancer)
Assign each data point to the nearest centroid, making it part of that cluster.
[Link] Centroids:
Recalculate The data has
the centroid 10,000
of each fields
cluster (features)
by taking the mean of all the data points assigned to that
cluster. This recalculates the centroid's position.
[Link]-assignment: you need to cut it down to 1,000 fields before
Reassign each data point to the cluster based on the updated centroid positions.
[Link]: you try machine learning. Which 1,000?
Repeat steps 3 and 4 until convergence. Convergence occurs when either the centroids no longer
The process
change significantly of achoosing
or after set number theof1,000 fields to use is called
iterations.
[Link] Result: Feature Selection
When the algorithm converges, you'll have k clusters, and each data point will belong to one of these
clusters.
 K-Means algorithms for clustering
Feature Selection: What
You have some data, and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
Feature Selection: What
You have some data, and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
K-Means clustering example
Feature Selection: What
You have some data, and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
Feature Selection: What
You have some data, and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

Assume,
you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
Solution:
Feature Selection: What
You have some data, and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

Since,
The process of choosing the 1,000 fields to use is called
Feature
C1-centroid= Mean of Selection
cluster C1
= (2+2+4/3, 4+6+7/3)=(2.66,5.66)
C2-centroid= Mean of cluster C2= (5+4/2, 2+4/2)=(4.5,3)
C3-centroid= Mean of cluster C3= (5+8+6+5+6/5, 6+3+6+7+3/5)=(6,5)
Feature Selection: What Iteration-2 Iteration-1

You have some data, and you want to use it to build a

classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
Feature Selection: What Iteration-3

You have some data, and you want to use it to build a

classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
Feature Selection: What Iteration-4

You have some data, and you want to use it to build a

classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
 Feature Selection: What
You have some data, and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Hence, the algorithm converges ( the clusters of iteration-4 and iteration-5 are same)
Feature Selection
which indicates no movement of data points and hence these are the final clusters.
Feature Selection: What
You have some data, and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
Feature Selection: What
You have some data, and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
 Advantages and Disadvantages of K-Means Clustering
Feature Selection: What
You have some data, and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
 Hierarchical Clustering in Machine Learning
Feature Selection: What
• Hierarchical clustering is a connectivity-based clustering model that groups the data points
together that are close to each other based on the measure of similarity or distance.
• The assumptionYou is thathave
data points
somethatdata,
are close
andto you
each other
wantaretomore
usesimilar
it toor build
related a
than data
points that are farther apart.
classifier, so that you can predict something
• A tree-like figure produced by hierarchical clustering, depicts the hierarchical relationships
between groups.(e.g. likelihood of cancer)
• Individual data points are located at the bottom of the dendrogram, while the largest clusters,
which include Theall the
datadata points,
has are located
10,000 fieldsat(features)
the top.
• Types of Hierarchical Clustering: Basically, there are two types of hierarchical Clustering:
1) Agglomerative Clustering
you need to cut it down to 1,000 fields before
2) Divisive clustering
you try machine
• Hierarchical Agglomerative Clustering:learning. Whichas 1,000?
It is also known the bottom-up approach or
hierarchical agglomerative clustering (HAC). Bottom-up algorithms treat each data as a singleton
The
cluster at the process
outset of choosing
and then the agglomerate
successively 1,000 fieldspairs
to use
of is calleduntil all clusters have
clusters
been merged into a single
Feature cluster that contains all data.
Selection
Hierarchical Agglomerative Clustering
Feature Selection: What
Steps:
•Consider each alphabet as a single cluster and
You have some data, and youcalculate wantthetodistance
use ofitonetocluster
buildfrom aall the other
classifier, so that you clusters. can predict something
•In the second step, comparable clusters are merged
(e.g. likelihood of cancer) together to form a single cluster. Let’s say cluster (B)
and cluster (C) are very similar to each other therefore
we merge them in the second step similarly to cluster
The data has 10,000 fields (features) (D) and (E) and at last, we get the clusters [(A), (BC),
(DE), (F)]
•We recalculate the proximity according to the
you need to cut it down to 1,000 algorithmfields before
and merge the two nearest clusters([(DE),
you try machine learning. Which 1,000?
(F)]) together to form new clusters as [(A), (BC),
(DEF)]
•Repeating the same process; The clusters DEF and
The process of choosing the 1,000 fieldsBCto are
usecomparable
is calledand merged together to form a
Feature Selection new cluster. We’re now left with clusters [(A),
(BCDEF)].
•At last, the two remaining clusters are merged
together to form a single cluster [(ABCDEF)].
Feature Selection: What
You have some data, and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
 Linkage Criteria
Feature Selection: What
The linkage creation step in Agglomerative clustering is where the distance between clusters is
calculated. So basically, a linkage is a measure of dissimilarity between the clusters. There are several
methods of linkage creation. Some of them are:
•Single Linkage
You have some data, and you want to use it to build a
•Complete Linkageclassifier, so that you can predict something
•Average Linkage (e.g. likelihood of cancer)
•Centroid Linkage
Thethedata
In Single Linkage, has 10,000
distance between fields
the two(features)
clusters is the minimum distance between clusters’
data points.
you need to cut it down to 1,000 fields before
you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
In Complete Linkage, the distance between two clusters is the maximum distance between
Feature Selection: What
clusters’ data points.

You have some data, and you want to use it to build a

classifier, so that you can predict something
(e.g. likelihood of cancer)

The data
In Average Linkage, the has 10,000
distance fields
between (features)
clusters is the average distance between each data
point in one cluster to every data point in the other cluster.
In Centroid Linkage, theyoudistance
need to cut it two
between down to 1,000
clusters fields before
is the distance between their centroids.
you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
Feature Selection: What
You have some data, and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
 Hierarchical Agglomerative Clustering (HAC) algorithm
Hierarchical Agglomerative Clustering (HAC) is a bottom-up hierarchical clustering technique that creates a hierarchy of
Feature Selection: What
clusters. The following are Steps of Hierarchical Agglomerative Clustering:
[Link]:
Begin with each data point as a single cluster. Treat each point as its own cluster to start.
You have some data, and you want to use it to build a
[Link] Pairwise Distances:
Compute the distance (e.g., Euclidean distance) between each pair of clusters or data points. This forms a distance
matrix.
classifier, so that you can predict something
[Link] Closest Clusters:(e.g. likelihood of cancer)
Find the two closest clusters based on the distance matrix and merge them into a single cluster. This reduces the total
number of clusters by one.
The data has 10,000 fields (features)
[Link] Distance Matrix:
Recalculate the distances between the new cluster and the existing clusters or data points.
[Link]:
you need to cut it down to 1,000 fields before
Iterate steps 3 and 4, merging the closest clusters and updating the distance matrix, until only a single cluster
you
(containing all data points) try machine learning. Which 1,000?
remains.

The process of choosing the 1,000 fields to

In the distance, d(𝑝use is called
𝑖 , 𝑝𝑗 ) represent the distance values between
Feature Selection 𝑝𝑖 and 𝑝𝑗 .
let's create a simple numerical example for Hierarchical Agglomerative Clustering using single
Feature Selection: What
linkage. Consider the following dataset with three data points:
Points: A(2, 3), B(5, 4), C(9, 6)
Computer the Euclidean
You distance betweendata,
have some these points.
and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields toMerge

use isthecalled
closest clusters again:
Feature Selection The minimum distance is between clusters {A, B}
and C (distance = 7.81).
Final cluster: {{A, B}, C}
 Feature Selection: What
Dendrogram
In this dendrogram:
The cluster {A, B}You
is merged
havefirst at a certain
some data, height.
and you want to use it to build a
Then, C is merged with the {A, B} cluster at a greater height, indicating a larger distance
classifier,
than the previous merge
so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
 Advantages
Feature and Disadvantages
Selection: What of Hierarchical clustering
Advantages of Hierarchical clustering:
You have
•It is simple to implement and some data,
gives the and you
best output want
in some to
cases. use it to build a
classifier, so that you can predict something
•It is easy and results in a hierarchy, a structure that contains more information.
(e.g. likelihood of cancer)
•It does not need us to pre-specify the number of clusters.
The
Disadvantages data has 10,000
of hierarchical fields (features)
clustering:
•It breaks the large clusters.
you need to cut it down to 1,000 fields before
•It is Difficult to handle different sized clusters and convex shapes.
you try machine learning. Which 1,000?
•It is sensitive to noise and outliers.
The process of choosing the 1,000 fields to use is called
•The algorithm can never be changed or deleted once it was done previously.
Feature Selection
 Hierarchical Agglomerative Clustering (HAC) Applications
Feature Selection: What
1. Biology and Genetics: Used to cluster genes or proteins based on their similarities, aiding in
understanding biological relationships.
You have some data, and you want to use it to build a
[Link] Segmentation: Divides an image into regions or segments based on similarities in pixel values or
classifier, so that you can predict something
features.
(e.g. likelihood of cancer)
[Link] Mining: Clustering documents by topic or similarity of content to aid in information retrieval and
The
text summarization. data has 10,000 fields (features)
[Link] Segmentation: Helps in segmenting customers based on purchasing behavior, demographics, or
you need
preferences for targeted marketing.
to cut it down to 1,000 fields before
you try machine learning. Which 1,000?
[Link] Network Analysis: Identifies communities or groups within social networks based on connections
between users. The process of choosing the 1,000 fields to use is called
FeatureDetecting
[Link] Detection: Selectionoutliers or unusual patterns in data by identifying clusters that deviate
from the norm.
 Expectation Maximization (EM) for Soft Clustering
Feature Selection: What
Expectation Maximization (EM) is a powerful algorithm used for soft clustering, particularly in scenarios where data
points might belong toYou have
multiple some
clusters or whendata, and youabout
there's uncertainty want
whichto usea point
cluster it to build
should belongato. One
classifier,
common application of sois thethat
EM in soft clustering Gaussian you can(GMM).
Mixture Model predict something
(e.g.
Gaussian Mixture Model likelihood
(GMM): of cancer)
•Representation: GMM assumes that the data is generated from a mixture of several Gaussian distributions, each
The
representing a cluster. data has 10,000 fields (features)
•Soft Assignment: Instead of hard assignment of points to clusters (as in K-means), GMM provides a probability
you
distribution over the clusters need
for each datato cutallowing
point, it down toor1,000
for soft fields
probabilistic before
assignment.
•EM Algorithm: GMM usesyou try
the EM machine
algorithm learning.
to iteratively estimateWhich 1,000?
the parameters of the Gaussian distributions and
update the probabilities of data points belonging to each cluster.
The process of choosing the 1,000 fields to use is called
Feature Selection
EM Algorithm Flowchart
Feature Selection: What
You have some data, and you want to use it to build a
classifier, so that you can predict something
(e.g. likelihood of cancer)

The data has 10,000 fields (features)

you need to cut it down to 1,000 fields before

you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
EM Algorithm
• Iterate between two steps

• Expectation step: (probabilistically) assign points to clusters

• Maximation step: estimate model parameters that maximize

the likelihood for the given assignment of points
 Semi-supervised learning
Semi-supervised learning is a paradigm in machine learning where both labeled and unlabeled data
Feature Selection: What
are used for training. In many real-world scenarios, labeled data (data points with associated target
You orhave
labels) might be scarce some
expensive data, while
to obtain, and unlabeled
you want datato use it to
is abundant. build a
Semi-supervised
classifier,
learning aims to leverage so that
this combination youand unlabeled
of labeled can predict something
data to improve the learning
(e.g. likelihood of cancer)
process and the model's performance.
Key Aspects of Semi-Supervised Learning:
The data has 10,000 fields (features)
[Link] Labeled Data: Semi-supervised learning assumes that labeled data is limited or costly
to acquire. you need to cut it down to 1,000 fields before
you
[Link] Unlabeled tryThere's
Data: machine learning.
a large Whichdata
pool of unlabeled 1,000?
available, which may not
have associatedThe
target [Link] choosing the 1,000 fields to use is called
process
Feature
[Link] of Selection
Unlabeled Data: Unlabeled data is used to provide additional information about
the underlying structure of the data distribution.
 Feature Selection: What
Advantages of Semi-Supervised Learning:
[Link] Learning with Limited Labels: Allows for more effective learning than
You have some data, and you want to use it to build a
using only labeled data, especially
classifier, so inthat
cases ofyou
limited can
labeled predict
data. something
(e.g.
[Link] of likelihood
Unlabeled Data:ofTakes
cancer)
advantage of the vast amount of unlabeled
data, which can help in capturing underlying data distributions.
The data has 10,000 fields (features)
[Link]-Efficient: Reduces the cost associated with manual labeling of data by
you need
leveraging readily available to cut it
unlabeled down
to 1,000 fields before
data.
you try machine learning. Which 1,000?

The process of choosing the 1,000 fields to use is called

Feature Selection
Assessment

Question :

The Euclidean distance between the data points (1,2)

and (4,8) is--------------

The Euclidean distance between 3-dimensional data

points p and q is---------------

Where p=(2,4,8)
q=(3,5,6)
•