Scribd
Upload
4 views16 pages

Support Vector Machine Updated

Support Vector Machine (SVM) is a supervised machine learning algorithm used for classification and regression, focusing on finding the optimal hyperplane that separates different classes. Key concepts include hyperplanes, support vectors, margins, and kernel functions, which allow SVM to handle non-linearly separable data by mapping it to higher-dimensional spaces. The document also outlines the dual optimization formulation and steps for classifying test points using SVM.

Uploaded by

Mubarrat Hossain Samin
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
4 views16 pages

Support Vector Machine Updated

Support Vector Machine (SVM) is a supervised machine learning algorithm used for classification and regression, focusing on finding the optimal hyperplane that separates different classes. Key concepts include hyperplanes, support vectors, margins, and kernel functions, which allow SVM to handle non-linearly separable data by mapping it to higher-dimensional spaces. The document also outlines the dual optimization formulation and steps for classifying test points using SVM.

Uploaded by

Mubarrat Hossain Samin
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Support Vector Machine

Sadia Islam
Assistant Professor
Department of Computer Science and engineering
United International University
Support Vector Machine
Support Vector Machine (SVM) is a supervised machine learning algorithm used
for classification and regression tasks. It tries to find the best boundary known as
hyperplane that separates different classes in the data.
Key Concepts of Support Vector Machine
● Hyperplane: A decision boundary separating different classes in feature space and is
represented by the equation wx + b = 0 in linear classification.
● Support Vectors: The closest data points to the hyperplane, crucial for determining the
hyperplane and margin in SVM.
● Decision boundary: A hyperplane that maximizes the margin between the classes is the
decision boundary.
● Margin: The distance between the hyperplane and the support vectors. SVM aims to
maximize this margin for better classification performance.
● Kernel: A function that maps data to a higher-dimensional space enabling SVM to handle
non-linearly separable data.
● Hard Margin: A maximum-margin hyperplane that perfectly separates the data without
misclassifications.
● Soft Margin: Allows some misclassifications by introducing slack variables, balancing
margin maximization and misclassification penalties when data is not perfectly separable.
● C: A regularization term balancing margin maximization and misclassification penalties. A
higher C value forces stricter penalty for misclassifications.
What to do if data are not linearly separable?
When data is not linearly separable i.e it can't be divided by a straight line, SVM
uses a technique called kernels to map the data into a higher-dimensional space
where it becomes separable. This transformation helps SVM find a decision
boundary even for non-linear data.

A kernel is a function that maps data points into a higher-dimensional space


without explicitly computing the coordinates in that space. This allows SVM to
work efficiently with non-linear data by implicitly performing the mapping.
Kernel Functions
● Linear Kernel: For linear separability.
● Polynomial Kernel: Maps data into a polynomial space.
● Radial Basis Function (RBF) Kernel: Transforms data into a space based on
distances between data points.
Example
● Training data:

x1=1 y1=+1,
x2=3 y2=−1

Kernel function:

Polynomial kernel of degree d=1:

K(x,y)=(x⋅y+1)
Regularization parameter:
C=5

Test point:
z=2
Step 1: Dual Optimization Formulation
Step 2: Kernel Matrix
Step 3: Constraint Simplification

Constraint:
Step 4: Simplify Objective
Step 5: Maximize
Step 6: Discriminant Function
Step 7: Solve for Bias, b
Using support vector condition,

Thus, we can write,


Step 8: classification of Test Point, z = 2
Thank You!

You might also like