Support Vector Machine:
Support Vector Machine (SVM) is supervised learning algorithm primarily designed for binary
classification problems. The core idea of SVM is to find a maximum marginal hyperplane that
best divides the dataset into classes.
The data points closest to the hyperplane are called support vectors, and they are critical in
defining the decision boundary.
• Support Vectors – The data points closest to hyperplane. These points will define
the separating line better by calculating margins.
• Hyperplane – A hyperplane is a decision plane which separates objects of two different classes.
• Margin – Margin is a gap between two lines on closest data points of two different classes.
• General equation of a hyperplane: WTX+ b=0
• equation of a positive hyperplane: WTX+ b= +1
• equation of a negative hyperplane: WTX+ b= -1
• The margin of a decision boundary as the distance to its support vectors = 2/|| w||
In order to maximize the margin, we need to minimize ||w||. With the condition that
there are no datapoints between two marginal hyperplanes:
xi•w+b +1 when yi =+1
xi•w+b -1 when yi =-1
Constraint: Can be combined into yi(xi•w +b) 1
Types of SVM:
• Linear SVM: Linear SVM is used for linearly separable data, which means if a dataset can be
classified into two classes by using a single straight line, then such data is termed as linearly separable
data, and classifier is used called as Linear SVM classifier.
Compiled by Ms. Minakshi
• Non-linear SVM: Non-Linear SVM is used for non-linearly separable data, which means if a dataset
cannot be classified by using a straight line, then such data is termed as non-linear data and classifier
used is called as Non-linear SVM classifier. It uses kernel trick.
How to choose the Hyperplane: SVM chooses the hyperplane with maximum margin
• Hard Margin SVM
• Soft Margin SVM
When the data is linearly separable, and we don’t want to have any misclassifications, we use SVM with
a Hard margin.
When a linear boundary is not feasible, or we want to allow some misclassifications in the hope of
achieving better generality, we can opt for a soft margin for our classifier.
Hard SVM Soft SVM
Compiled by Ms. Minakshi
Non- Linearly Separable Data Points:
Kernel
• Some problems can’t be solved using linear hyperplane. In such situation, SVM uses a kernel trick to
transform the input space to a higher dimensional space
• Kernel is a mathematical function used to transform data from lower dimension to higher dimension
and to manipulate data using dot product at higher dimensions. After transformation classification is
performed as shown in the figure.
Different Kernal Functions are:
• Linear Kernel
• Polynomial
• Gaussian Kernel
• Radial Basis Function (RBF)
• Sigmoid
Compiled by Ms. Minakshi
Linear Kernel
When data is linearly separable, then Linear Kernel is used. For e.g., we have 2 vectors x1
and y1, then linear Kernel K (x1,y1) is:
Polynomial Kernel (Homogeneous kernel)
• Where d = degree of polynomial
• Very popular in Image Processing.
Polynomial Kernel (Inhomogeneous kernel)
• Where d = degree of polynomial and c constant
• Very popular in Image Processing.
Gaussian Kernel
Radial Basis Function (RBF)
Sigmoid Kernel
Compiled by Ms. Minakshi
Applications of SVM
• Text and Document Classification
• Image Classification
• Speech and Audio Recognition
• Medical Diagnosis etc.
Compiled by Ms. Minakshi