SUPPORT VECTOR MACHINE
Support Vectors
Maximum Margin Classifier
Key Terms
1. Support vectors: These are data points in a dataset that are closest to the
hyperplane and play a crucial role in determining the optimal hyperplane in SVM.
2. Hyperplane: In SVM, a hyperplane is a decision boundary that separates the data
points into different classes. The goal of SVM is to find the optimal hyperplane that
maximizes the margin between the two classes.
3. Margin: The margin in SVM refers to the distance between the hyperplane and the
closest data points from each class. The goal of SVM is to maximize this margin to
increase the classifier's ability to generalize to new data.
Drawback of Maximum Margin Classifier
Key Terms
● Soft margin is a technique used in Support Vector Machine (SVM) classification
that allows for some misclassification of data points in order to achieve a wider
margin and a more flexible decision boundary.
● In traditional SVM classification with a hard margin, the goal is to find a hyperplane
that completely separates the two classes of data with no misclassification.
However, in real-world datasets, this is often not possible due to noise, outliers, or
other factors that make the data non-separable.
● The objective is to find the optimal hyperplane that maximizes the margin while
also minimizing the amount of misclassification.
● Cross Validation is used to determine the best soft margin.
Support Vector Classifier
Limitations of SVC
The limitation of SVC is compensated by SVM non-linearly. And that’s the difference between SVM
and SVC. If the hyperplane classifies the dataset linearly then the algorithm we call it as SVC and the
algorithm that separates the dataset by non-linear approach then we call it as SVM.
Kernel Trick and Kernel Function
1. Kernel function: In SVM, a kernel function is used to transform the input data into a higher
dimensional space where it can be more easily separated. Popular kernel functions include
linear, polynomial, and radial basis function (RBF) kernels.
2. Kernel trick is a technique used in machine learning, specifically in Support Vector
Machine (SVM) algorithms, to transform a low-dimensional input space to a higher
dimensional feature space without actually computing the coordinates of the data in the
higher dimensional space.
3. The kernel trick is based on the observation that the computation of the dot product
between two vectors in a high-dimensional space can be done implicitly by defining a
kernel function that measures the similarity between two data points in the original input
space.
4. By using the kernel function to compute the dot product, the data points are effectively
mapped to a higher dimensional space without explicitly computing their coordinates.
5. The kernel trick is a powerful technique that enables SVM to handle complex, non-linear
classification problems, while avoiding the computational complexity and memory
requirements of explicitly computing the coordinates in a high-dimensional feature space.
Advantages of Support Vector Machine (SVM)
1. Regularization capabilities: SVM has L2 Regularization feature. So, it has good generalization capabilities which prevent it from over-fitting.
2. Handles non-linear data efficiently: SVM can efficiently handle non-linear data using Kernel trick.
3. Solves both Classification and Regression problems: SVM can be used to solve both classification and regression problems. SVM is
used for classification problems while SVR (Support Vector Regression) is used for regression problems.
4. Stability: A small change to the data does not greatly affect the hyperplane and hence the SVM. So the SVM model is stable.
Disadvantages of Support Vector Machine (SVM)
1. Choosing an appropriate Kernel function is difficult: Choosing an appropriate Kernel function (to handle the non-linear data) is not an
easy task. It could be tricky and complex. In case of using a high dimension Kernel, you might generate too many support vectors which reduce
the training speed drastically.
2. Extensive memory requirement: Algorithmic complexity and memory requirements of SVM are very high. You need a lot of memory since
you have to store all the support vectors in the memory and this number grows abruptly with the training dataset size.
3. Requires Feature Scaling: One must do feature scaling of variables before applying SVM.
4. Long training time: SVM takes a long training time on large datasets.
5. Difficult to interpret: SVM model is difficult to understand and interpret by human beings unlike Decision Trees.