Scribd
Upload
2 views26 pages

Understanding Support Vector Machines

Support Vector Machine (SVM) is a supervised machine learning algorithm used for classification and regression, aiming to find the optimal hyperplane that separates classes with the maximum margin. It utilizes support vectors, which are critical data points that determine the hyperplane's position and orientation, and can employ soft margin techniques to handle noisy or overlapping data. When data is not linearly separable, SVM applies the kernel trick to project data into a higher-dimensional space for effective classification.

Uploaded by

rkr201759
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
2 views26 pages

Understanding Support Vector Machines

Support Vector Machine (SVM) is a supervised machine learning algorithm used for classification and regression, aiming to find the optimal hyperplane that separates classes with the maximum margin. It utilizes support vectors, which are critical data points that determine the hyperplane's position and orientation, and can employ soft margin techniques to handle noisy or overlapping data. When data is not linearly separable, SVM applies the kernel trick to project data into a higher-dimensional space for effective classification.

Uploaded by

rkr201759
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd

Support Vector Machine

Dr. Rajashree Nayak


What is SVM

• Support Vector Machine (SVM) is a supervised machine learning


algorithm used for both classification and regression tasks.

• Hyperplane: SVM finds the best decision boundary (hyperplane) that


separates data points of different classes in the feature space with the
maximum margin — i.e., it tries to find the widest possible gap between
classes.

• Each data point is represented as a vector in n-dimensional space.

• SVM tries to find a hyperplane that divides the classes such that:

 Points closest to the hyperplane (called support vectors) determine its


position and orientation.

 The margin (distance between the hyperplane and support vectors) is


Mathematical Form: y= wx + b
maximized.
The Problem and Idea
The Concept
The Concept
The Math behind LSVM
 The central red line is the decision boundary — this is the hyperplane that separates
the two classes.
 The two yellow dashed lines are margin boundaries. These lines define the region of
separation between the two classes.

• All points of Class = +1 should ideally be on or outside the line


• All points of Class = –1 should ideally be on or outside the line

Support Vectors
• The data points that lie exactly on these dashed lines are called support vectors.
• They are the most critical points in the dataset because they determine the position
and orientation of the separating hyperplane.
• Removing other points won’t affect the boundary much — but removing a support
vector will.
The Math behind LSVM
The Math behind LSVM
The Math behind LSVM
Errors in LSVM

• Works well only for clean, linearly separable data.

• But if data is noisy or overlapping, it becomes impossible to find such a


perfect separation.
Errors in LSVM
In real-world datasets, some points:
• Are mislabeled or have noise,
• Fall on the wrong side of the margin or even on the wrong side of the
hyperplane.
• If we force perfect classification (Hard Margin), the model:
• Tries too hard to fit every point,
• Overfits, performing poorly on unseen data.

Soft Margin SVM — The Solution

To fix this, SVM introduces Slack Variables ξi​(xi), which allow some flexibility in
classification.
Soft Margin in LSVM
Soft Margin Optimization
Hard Margin vs. Soft Margin
Soft Margin
Drawback of LSVM

• This works only if the data is linearly separable — i.e., you can draw a straight line (in
2D) or plane (in higher dimensions) to divide the classes.

• When data is not linearly separable, SVM uses a kernel trick to project data into a
higher-dimensional feature space, where it becomes linearly separable.
The Concept of Non-linear SVM
Math for Non-linear SVM
Kernel Trick
Some Kernel Functions
Multi-class Problem
Example

Classify the data points with hard-margin linear SVM and get the weight vector
W, bias (b), support vectors, and the margin.

Solution:
Some Useful Resources

You might also like