SUPPORT VECTOR MACHINE
🧠 What is SVM?
SVM (Support Vector Machine) is a supervised learning algorithm used for classification and
regression — but it's mostly famous for binary classification.
🧲 Goal: Find the best decision boundary (hyperplane) that separates classes with the maximum
margin.
✨ Core Idea:
Imagine you’re drawing a line between two groups of points (let's say cats 🐱 vs dogs 🐶). There could
be many lines...
But SVM finds the one that is:
✅ Far away from both classes
✅ Has the maximum margin between the classes
This line (or hyperplane in higher dimensions) is the best decision boundary.
📐 Key Concepts:
Term Meaning
Hyperplane The decision boundary (a line in 2D, plane in 3D, etc.)
Support Vectors Data points closest to the hyperplane — they “support” or define the boundary
Margin Distance between the hyperplane and the nearest data points (support vectors)
🚀 How It Works:
For Linearly Separable Data:
1. Finds the optimal hyperplane that separates the two classes.
2. Uses support vectors to calculate the margin.
3. Maximizes margin for better generalization.
For Non-Linearly Separable Data:
SVM uses a Kernel Trick! 🎩✨
It transforms the data into higher dimensions where a linear separator can be found.
Example: A circular pattern in 2D → SVM maps it into 3D, where a plane can separate them.
Common Kernels:
Linear
Polynomial
RBF (Gaussian)
Sigmoid
Regularization parameter. A smaller C gives a wider margin with more tolerance for
'C'
misclassification. A larger C tries to classify all points correctly (may overfit).
Defines how far the influence of a single training example reaches. Small gamma = far
'gamma'
influence (smoother). High gamma = close influence (tighter fit, may overfit).
The type of kernel function. Here you're using 'rbf' (Radial Basis Function), which is great
'kernel'
for non-linear classification.
SVC() The SVM model you're tuning.
param_grid The dictionary of hyperparameters to try.
Once the best parameters are found, it automatically refits the model on the entire
refit=True
training data.
verbose=3 Shows progress output on the console (level 3 = very detailed).
So this line creates a grid object that will test all combinations of the C and gamma values using
cross-validation.
[Link](X_train, y_train)
This line starts the grid search:
Tries each combination of C and gamma
Uses cross-validation (default is 5-fold)
Tracks performance
Finds the best combination
Re-trains the final model with the best parameters
✅ Precision
Precision = TP / (TP + FP)
Measures how many selected items were relevant
"Of all predictions labeled as class A, how many were actually class A?"
High precision = low false positives
Example:
If the model predicts 10 "spam" emails and only 7 are actually spam, precision = 0.7
🔁 Recall (Sensitivity / True Positive Rate)
Recall = TP / (TP + FN)
Measures how many relevant items were selected
"Of all actual class A samples, how many did we catch?"
High recall = low false negatives
Example:
If there are 20 spam emails and your model caught 15 → recall = 0.75
🎯 F1-Score
F1 = 2 × (Precision × Recall) / (Precision + Recall)
Harmonic mean of precision and recall
Good when you want a balance between precision & recall
Example:
If precision = 0.8, recall = 0.6 →
F1 = 2 × (0.8 × 0.6) / (0.8 + 0.6) = 0.685
🧮 Support
Number of actual samples of that class in the dataset
Just counts how many examples belong to that class in y_true
📚 Averages at the Bottom:
🟠 macro avg
Average of precision/recall/F1 across all classes
Unweighted → each class contributes equally
Good for balanced view, even when class distribution is uneven
⚖️weighted avg
Average of precision/recall/F1, weighted by support
Takes into account how common each class is
Good for overall performance, especially in imbalanced datasets