Scribd
Upload
4 views10 pages

Understanding Support Vector Machines

The document explains the fundamentals of Support Vector Machines (SVM), focusing on key concepts such as hyperplanes, support vectors, and margins. It describes the process of transforming data into higher dimensions to achieve separability and outlines the differences between hard and soft margins. The goal of SVM is to find the optimal hyperplane that maximizes the margin between different classes of data.

Uploaded by

Abhinav Praneeth
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
4 views10 pages

Understanding Support Vector Machines

The document explains the fundamentals of Support Vector Machines (SVM), focusing on key concepts such as hyperplanes, support vectors, and margins. It describes the process of transforming data into higher dimensions to achieve separability and outlines the differences between hard and soft margins. The goal of SVM is to find the optimal hyperplane that maximizes the margin between different classes of data.

Uploaded by

Abhinav Praneeth
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Support Vector

Machine (SVM)
Key words
• Hyperplane: A decision boundary separating different
classes in feature space and is represented by the
equation wx + b = 0 in linear classification.

• Support Vectors: The closest data points to the


hyperplane, crucial for determining the hyperplane and
margin in SVM are support vectos.

• Margin: The distance between the hyperplane and the


support vectors. SVM aims to maximize this margin for
• Kernel: A function that maps data to a higher-
dimensional space enabling SVM to handle non-linearly
separable data.

• Hard Margin: A maximum-margin hyperplane that perfectly


separates the data without misclassifications.

• Soft Margin: Allows some misclassifications by


introducing slack variables, balancing margin
maximization and misclassification penalties when data
is not perfectly separable.
• The basic idea of SVM is to increase
dimensionality of the samples so that they can
be separable.
• The idea is to find a hyper-plane to place
samples from the same class inside it .
• Assume the original samples are in the x space
and they are transformed (dimensionality
enlargement) to the y space, where y = φ(x).
suppose we have some hyperplane which separates
the positive from the negative examples (a
“separating hyperplane”) provided by
WTx +b=0
where x is an input vector (sample set) and w is
the weight vector (to be adjusted) and b is a
bias value. The distance between the closest
data point to the hyperplane is called Margin of
Separation and it is denoted by ρ.
• The goal is to maximize this margin.
• The hyperplane that is doing this separation is
the optimal hyperplane.
• Assuming that di is the desired respond (output
value) the +1 or -1 values show the linearly
separable classes.
WTxi +b ≥ 0 for di = 1
WTxi +b < 0 for di = -1
where xi is the pattern for the ith example.
There fore the optimal discriminant hyperplane will be
g(x) = wTo x+ b o
where the optimal hyperplane will be
wTo x+ bo =0
also
x = xp +r *wo / ||wo||
where xp is the normal projection on to x onto the
optimal hyperplane,
and r is signed distance of data point x from the
optimal hyperplane.
r will be positive if x is placed on the positive side
of the hyperplane and negative once it is placed on the
negative side of the hyperplane.
wo / ||wo|| Is a unit vector in the direction of the
weight vector, representing the normal direction from
the hyperplane.
The value for r can be calculated as follows
r = g(x) / ||wo||
Thank you

You might also like