Support Vector Machine (SVM)
A Support Vector Machine (SVM) is a supervised machine learning algorithm primarily used for
classification but also applicable to regression and outlier detection tasks. SVMs work by finding
the optimal hyperplane that best separates data points of different classes in an N-dimensional
space, maximizing the margin between the closest points of each class-these closest points are
called support vectors [1] [2] [3] [4] .
Hyperplane in SVM
A hyperplane is a decision boundary that separates different classes in the feature space. In
two dimensions, this is simply a line; in three dimensions, it is a plane; and in higher dimensions, it
is called a hyperplane. The SVM algorithm seeks the hyperplane that maximizes the margin-the
distance between the hyperplane and the nearest data points from each class (the support
vectors). Maximizing this margin helps the model generalize better to unseen data [1] [2] [3] [5] [4] .
Types of Support Vector Kernels
When data is not linearly separable, SVMs use kernel functions to transform the data into a
higher-dimensional space where a linear separation is possible. This approach is known as the
“kernel trick.” The choice of kernel function is crucial, as it determines how the data is mapped
and separated [1] [3] [6] [7] [8] .
Common types of SVM kernels include:
Linear Kernel: Used when data is linearly separable. It computes the dot product between
feature vectors.
$$ K(x_1, x_2) = x_1 \cdot x_2 \ ) [6] [7] [8]
Polynomial Kernel: Suitable for non-linear data, it computes the similarity as a polynomial
function of the original features.
$$ K(x_1, x_2) = (x_1 \cdot x_2 + c)^d \ ) [6] [7] [8]
Radial Basis Function (RBF) or Gaussian Kernel: Effective for non-linear data, maps data
into infinite-dimensional space.
$$ K(x_i, x_j) = \exp(-\gamma |x_i - x_j|^2) \ ) [6] [7] [8]
Sigmoid Kernel: Based on the hyperbolic tangent function, similar to activation functions in
neural networks.
$$ K(x_1, x_2) = \tanh(\alpha x_1 \cdot x_2 + c) \ ) [6] [7] [8]
Other kernels include exponential, Laplacian, quadratic, and more, but the above are the most
commonly used in practice [6] [7] [8] .
Summary Table: SVM Kernels
Kernel Type Equation/Function Use Case
Linear $ x_1 \cdot x_2 $ Linearly separable data
Polynomial $ (x_1 \cdot x_2 + c)^d $ Non-linear, polynomial patterns
RBF (Gaussian) $ \exp(-\gamma |x_i - x_j|^2) $ Non-linear, complex boundaries
Sigmoid $ \tanh(\alpha x_1 \cdot x_2 + c) $ Non-linear, neural network-like
In summary:
SVMs are powerful classifiers that find the optimal hyperplane to separate classes. When data is
not linearly separable, kernel functions (linear, polynomial, RBF, sigmoid, etc.) are used to map
data into higher dimensions, enabling effective classification [1] [3] [6] [7] . The hyperplane is
central to SVMs, acting as the decision boundary with the largest possible margin between
classes [1] [2] [5] [4] .
⁂
1. [Link]
2. [Link]
3. [Link]
4. [Link]
5. [Link]
6. [Link]
7. [Link]
8. [Link]