Crafting Decision Boundaries: The SVM Approach
Section 1: Learn
What is SVM (Support Vector Machine)?
Support Vector Machine (SVM) is a supervised learning algorithm used for
classification and regression.
It finds the optimal decision boundary (hyperplane) that maximizes the
margin between different classes.
Why is SVM Important?
● Effective for Complex Datasets – Works well even when classes are not
linearly separable.
● Handles High-Dimensional Data – Performs well when the number of
features is high.
● Robust Against Overfitting – Uses regularization (C parameter) to
prevent overfitting.
How Does SVM Work?
1. Finds the Best Hyperplane – Maximizes the margin between different
classes.
2. Uses Support Vectors – The closest points to the hyperplane that
define the boundary.
3. Handles Non-Linearity with Kernel Trick – Uses polynomial, RBF
(Radial Basis Function), and sigmoid kernels.
Real-Life Applications of SVM
● Face Recognition – Classifies facial features in image processing.
● Spam Email Filtering – Differentiates spam from non-spam emails.
● Medical Diagnosis – Used in cancer detection based on cell
classification.
Anecdote: How SVM Helped Detect Cancer
SVM has been widely used in breast cancer detection, improving accuracy in
early diagnosis.
Section 2: Practice
1. Implementing SVM for Classification in Python
import pandas as pd
from sklearn.model_selection import train_test_split
from [Link] import SVC
from [Link] import accuracy_score
# Sample dataset: Loan Approval Prediction
data = {
"Salary": [25000, 30000, 40000, 50000, 60000, 20000, 75000, 85000,
95000, 100000],
"Credit_Score": [600, 650, 700, 750, 800, 500, 820, 900, 950, 990],
"Loan_Approved": [0, 0, 1, 1, 1, 0, 1, 1, 1, 1]
}
df = [Link](data)
# Splitting data
X = df[["Salary", "Credit_Score"]]
y = df["Loan_Approved"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42)
# Training SVM Model
svm_model = SVC(kernel="linear")
svm_model.fit(X_train, y_train)
# Predictions
y_pred = svm_model.predict(X_test)
# Model Evaluation
accuracy = accuracy_score(y_test, y_pred)
print(f"SVM Accuracy: {accuracy:.2f}")
2. Choosing the Right Kernel for SVM
# Using different kernels
svm_rbf = SVC(kernel="rbf") # RBF Kernel (used for non-linear data)
svm_poly = SVC(kernel="poly", degree=3) # Polynomial Kernel
# Training models
svm_rbf.fit(X_train, y_train)
svm_poly.fit(X_train, y_train)
# Predictions
y_pred_rbf = svm_rbf.predict(X_test)
y_pred_poly = svm_poly.predict(X_test)
# Evaluating performance
accuracy_rbf = accuracy_score(y_test, y_pred_rbf)
accuracy_poly = accuracy_score(y_test, y_pred_poly)
print(f"SVM RBF Accuracy: {accuracy_rbf:.2f}")
print(f"SVM Polynomial Accuracy: {accuracy_poly:.2f}")
Section 3: Know More
Frequently Asked Questions (FAQs)
1. What is the role of Support Vectors?
● Support vectors are the data points closest to the decision boundary.
● They determine the position and orientation of the hyperplane.
2. When should I use SVM?
● When working with small to medium-sized datasets.
● When dealing with high-dimensional data.
3. What is the Kernel Trick in SVM?
● A technique that transforms non-linear data into a higher-dimensional
space where it becomes linearly separable.
4. What are the different types of Kernels in SVM?
● Linear Kernel – Used for linearly separable data.
● Polynomial Kernel – Maps features into higher dimensions.
● RBF Kernel – Works well for non-linear data.
5. How do I choose the best SVM parameters?
● Adjust C (Regularization Parameter) → Controls margin width.
● Choose the right kernel → Experiment with linear, polynomial, and
RBF kernels.
Conclusion:
SVM is a powerful classification algorithm that creates optimal decision
boundaries.
With kernel tricks and margin maximization, it performs well on complex
datasets.