Scikit-learn (sklearn)

The Complete Beginner's Guide

Theory + Hands-on Code — Random Forest, SVM, and the Full Machine Learning Workflow

Written so that ANY student can follow along, step by step

Table of Contents

1. What is Scikit-learn?
2. Installing Scikit-learn
3. The Golden Rule of Sklearn: fit / predict / transform
4. The Standard Machine Learning Workflow
5. Step 1 — Loading and Understanding Data
6. Step 2 — Splitting Data (train_test_split)
7. Step 3 — Preprocessing (Scaling & Encoding)
8. Step 4 — Model Evaluation Metrics
9. Random Forest — Theory
10. Random Forest — Full Code Walkthrough
11. Support Vector Machine (SVM) — Theory
12. Support Vector Machine (SVM) — Full Code Walkthrough
13. Random Forest vs SVM — Comparison
14. Cross-Validation & Hyperparameter Tuning
15. Common Beginner Mistakes
16. Final Cheat Sheet
 1. What is Scikit-learn?

Scikit-learn (imported in Python as sklearn ) is the most popular open-source machine learning library for Python. Think of it as a giant toolbox full of ready-
made machine learning algorithms — you don't need to write the maths yourself. You just plug in your data, and sklearn does the heavy lifting.

Simple analogy: If machine learning is cooking, sklearn is a kitchen full of pre-built appliances (mixer, oven, blender). You still choose the ingredients (your
data) and the recipe (your algorithm), but you don't have to build the oven from scratch.

Scikit-learn is built on top of three other famous libraries:

Library Role

NumPy Fast numerical arrays — sklearn uses this to store and compute data

SciPy Scientific computing functions (optimization, linear algebra)

Matplotlib Used alongside sklearn to visualize results

What can sklearn do? It covers almost the entire classical machine learning pipeline:

Classification — predicting a category (e.g. spam / not spam)

Regression — predicting a number (e.g. house price)
Clustering — grouping similar data together (e.g. customer segments)
Dimensionality Reduction — simplifying data (e.g. PCA)
Model Selection — comparing and tuning models
Preprocessing — cleaning and preparing raw data

In this guide we will focus heavily on classification, and build two of the most famous and powerful models: Random Forest and Support Vector Machine
(SVM).

2. Installing Scikit-learn

Before writing any code, install the library using pip (Python's package manager). Open your terminal / command prompt and run:

pip install scikit-learn numpy pandas matplotlib seaborn

We are also installing numpy (arrays), pandas (data tables), matplotlib and seaborn (plotting) because you will almost always use these together with
sklearn.

To check that it installed correctly, open Python and run:

import sklearn
print(sklearn.__version__)

If a version number prints out (e.g. 1.5.0 ) with no errors, you are ready to go.
 3. The Golden Rule of Sklearn: fit / predict / transform

Every single algorithm in sklearn — whether it's Random Forest, SVM, Linear Regression, or K-Means — follows the exact same pattern. Once you learn this
pattern, you can use any model in sklearn with almost no extra learning. This is the single most important idea in this entire guide.

The Universal Pattern:

1. Import the model class

2. Create an instance of the model (this is called an "estimator")
3. Fit the model to your training data — [Link](X_train, y_train)
4. Predict on new data — [Link](X_test)
5. Evaluate how good the predictions were

Here is that pattern in the most minimal code possible:

from [Link] import RandomForestClassifier

model = RandomForestClassifier() # create the model

[Link](X_train, y_train) # teach it using training data
predictions = [Link](X_test) # ask it to predict on new data

That's it. Notice X (capital) is used for the input features (a table of numbers), and y (lowercase) is the target/label you want to predict. This naming convention
is used everywhere in sklearn and in almost every ML tutorial online.

Two extra methods you will also see:

Method What it does

predict_proba(X) Returns the probability of each class instead of a hard label

transform(X) Used by preprocessing tools (like scalers) to modify data

fit_transform(X) Shortcut that fits and transforms in one line

score(X, y) Quickly returns accuracy on given data

4. The Standard Machine Learning Workflow

Almost every ML project in sklearn follows these six steps, in this exact order:

1 Load the data

2 Explore & clean the data
3 Split into train and test sets
4 Preprocess (scale / encode) the data
5 Train the model (fit)
6 Evaluate the model (predict + metrics)

We will now go through each of these steps one by one, then apply all of them to build a Random Forest model and an SVM model.
 5. Step 1 — Loading and Understanding Data

Sklearn comes with several small "toy" datasets built in, perfect for practice. The most famous one is the Iris dataset, where we predict the species of a flower
from 4 measurements.

from [Link] import load_iris

import pandas as pd

data = load_iris()
X = [Link] # the features (measurements)
y = [Link] # the labels (species: 0, 1, or 2)

# Turn it into a readable table

df = [Link](X, columns=data.feature_names)
df['species'] = y
print([Link]())
print([Link]())

Other beginner-friendly built-in datasets:

Dataset Function Task

Iris flowers load_iris() Classification (3 classes)

Breast cancer load_breast_cancer() Classification (2 classes)

Wine load_wine() Classification (3 classes)

Diabetes load_diabetes() Regression

In real projects you'll usually load a CSV file instead:

df = pd.read_csv("my_data.csv")
X = [Link]("target_column", axis=1)
y = df["target_column"]

6. Step 2 — Splitting Data (train_test_split)

We never train and test a model on the exact same data — that would be like giving a student the exam questions before the exam and then being impressed
that they scored 100%. It tells you nothing about how well they'd do on new questions.

Sklearn gives us train_test_split to randomly divide data into a training portion (used to teach the model) and a testing portion (used to honestly check it).

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(

X, y,
test_size=0.2, # 20% of data reserved for testing
random_state=42 # fixes randomness so results are reproducible
)

print(X_train.shape, X_test.shape)

Tip: random_state=42 is not magic — it's just a seed number. Using the same seed means you (and your teacher checking your work) will get the exact
same split every time you run the code.

Common mistake: Never fit a scaler or a model on your test set. The test set must stay completely "unseen" until the very final evaluation, or your results
will be misleadingly good.
 7. Step 3 — Preprocessing (Scaling & Encoding)

7.1 Feature Scaling

Many algorithms (especially SVM, KNN, and anything based on distance) are sensitive to the scale of numbers. If one feature ranges from 0–1 and another ranges
from 0–100,000, the second feature will unfairly dominate. StandardScaler fixes this by transforming every feature to have a mean of 0 and a standard
deviation of 1.

from [Link] import StandardScaler

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train) # fit AND transform on train
X_test_scaled = [Link](X_test) # only transform on test

Important rule: Always call fit_transform() on the training data, but only transform() (no fit) on the test data. This prevents "data leakage" — letting
information from the test set sneak into training.

Random Forest does not require scaling (it splits data on thresholds, not distances), but SVM absolutely does. Scaling never hurts, so it's a good habit to always
do it.

7.2 Encoding Categorical Data

Machine learning models only understand numbers. If your data has text categories like "Red", "Blue", "Green", you must convert them to numbers first.

from [Link] import LabelEncoder, OneHotEncoder

# LabelEncoder: turns categories into 0, 1, 2, ...

le = LabelEncoder()
y_encoded = le.fit_transform(y) # e.g. ['cat','dog'] -> [0, 1]

# OneHotEncoder: turns categories into separate 0/1 columns (better for input features)
ohe = OneHotEncoder(sparse_output=False)
X_encoded = ohe.fit_transform(X[['color']])

Rule of thumb: Use LabelEncoder for the target/label column (y). Use OneHotEncoder (or pd.get_dummies) for categorical input features (X), because
label encoding input features can trick the model into thinking there's a numeric order (e.g. that "Green"=2 is bigger than "Red"=0), which is meaningless.

8. Step 4 — Model Evaluation Metrics

Once a model makes predictions, we need to measure how good it actually is. Sklearn provides all standard metrics in [Link] .

from [Link] import (

accuracy_score, precision_score, recall_score,
f1_score, confusion_matrix, classification_report
)

y_pred = [Link](X_test)

print("Accuracy:", accuracy_score(y_test, y_pred))

print("Precision:", precision_score(y_test, y_pred, average='macro'))
print("Recall:", recall_score(y_test, y_pred, average='macro'))
print("F1-score:", f1_score(y_test, y_pred, average='macro'))
print(confusion_matrix(y_test, y_pred))
print(classification_report(y_test, y_pred))

Metric Meaning

Accuracy % of predictions that were correct overall

Precision Of everything predicted "positive", how much was actually positive?

Recall Of all real positives, how many did the model catch?

F1-score Balance between precision and recall (harmonic mean)

Confusion Matrix A table showing correct vs incorrect predictions per class

When to worry about accuracy alone: If 95% of your data is one class (e.g. "not fraud"), a lazy model that always predicts "not fraud" gets 95% accuracy
while being useless. In such imbalanced cases, always check precision, recall, and F1-score too.
 9. Random Forest — Theory

9.1 Start with a Decision Tree

To understand a Random Forest, you must first understand a single Decision Tree. A decision tree asks a series of yes/no questions about the data to split it into
smaller and smaller groups, until it can confidently predict a class. For example, to classify an animal, it might ask: "Does it have fur?" → "Does it bark?" → "Dog".

Internally, at each split, the tree picks the question (feature + threshold) that best separates the classes, measured using metrics like Gini impurity or Entropy
(both measure how "mixed up" a group of classes is — lower is purer).

The problem with a single tree: A single decision tree easily "memorizes" the training data (overfitting) — it performs great on data it has seen, but
poorly on new data.

9.2 The Random Forest Idea

A Random Forest fixes this by building many decision trees (e.g. 100) instead of just one, and combining their answers. This is called an ensemble method. Two
sources of randomness make each tree different from the others:

1. Bagging (Bootstrap Aggregating): Each tree is trained on a random sample of the training data (drawn with replacement), so no two trees see exactly the
same data.
2. Random feature selection: At each split, the tree only considers a random subset of features, not all of them, forcing the trees to be different from each
other.

Once all trees are trained, the forest makes a final prediction by majority vote (for classification) or averaging (for regression):

Why it works: Individually, each tree might make mistakes and overfit in different, random ways. But when you average many independent "opinions", the
random errors cancel each other out, leaving a much more accurate, stable, and generalizable prediction — the "wisdom of the crowd" effect.

9.3 Key Hyperparameters

Parameter Meaning

n_estimators Number of trees in the forest (more = usually better, but slower). Default 100.

max_depth Maximum depth of each tree. Limits overfitting.

min_samples_split Minimum samples required to split a node further.

max_features Number of features considered at each split.

criterion How splits are measured — 'gini' or 'entropy' .

random_state Seed for reproducibility.

 10. Random Forest — Full Code Walkthrough

Let's build a complete, working Random Forest classifier from start to finish, using the built-in breast cancer dataset (predicting whether a tumor is malignant or
benign).

# --- 1. Import everything we need ---

import pandas as pd
from [Link] import load_breast_cancer
from sklearn.model_selection import train_test_split
from [Link] import StandardScaler
from [Link] import RandomForestClassifier
from [Link] import accuracy_score, classification_report, confusion_matrix

# --- 2. Load the data ---

data = load_breast_cancer()
X = [Link]
y = [Link]
print("Features:", data.feature_names[:5], "...")
print("Classes:", data.target_names) # ['malignant' 'benign']

# --- 3. Split into train and test sets ---

X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y
)

# --- 4. Scale the data (optional for RF, but good habit) ---
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = [Link](X_test)

# --- 5. Create and train the Random Forest model ---

rf_model = RandomForestClassifier(
n_estimators=200, # 200 trees
max_depth=6, # limit tree depth to avoid overfitting
random_state=42
)
rf_model.fit(X_train_scaled, y_train)

# --- 6. Make predictions ---

y_pred = rf_model.predict(X_test_scaled)

# --- 7. Evaluate ---

print("Accuracy:", accuracy_score(y_test, y_pred))
print("\nClassification Report:\n", classification_report(y_test, y_pred))
print("Confusion Matrix:\n", confusion_matrix(y_test, y_pred))

# --- 8. Bonus: Which features mattered most? ---

importances = [Link](rf_model.feature_importances_, index=data.feature_names)
print(importances.sort_values(ascending=False).head(10))

Notice stratify=y in the split: This ensures both the train and test sets keep the same proportion of each class as the original data — very important for
imbalanced datasets like disease detection.

A huge benefit of Random Forest is the feature_importances_ attribute — it tells you which columns of data actually mattered most for the prediction, which is
great for explaining your model to others.

10.1 Random Forest for Regression

Predicting a number instead of a category? Use RandomForestRegressor the exact same way:

from [Link] import RandomForestRegressor

from [Link] import mean_squared_error, r2_score

reg = RandomForestRegressor(n_estimators=200, random_state=42)

[Link](X_train, y_train)
preds = [Link](X_test)
print("R2 score:", r2_score(y_test, preds))
print("MSE:", mean_squared_error(y_test, preds))
 11. Support Vector Machine (SVM) — Theory

11.1 The Core Idea

Imagine plotting your data points on a graph, where each point belongs to one of two classes (e.g. red dots and blue dots). A Support Vector Machine tries to draw
the best possible line (or plane, in higher dimensions) that separates the two classes.

But there could be many lines that separate the classes — which one is "best"? SVM chooses the line that has the maximum margin: the largest possible
distance between the line and the closest points from each class. Those closest points are called the support vectors — they are the only points that actually
matter for defining the boundary (hence the name).

Why maximum margin matters: A wider margin means the decision boundary is more confident and generalizes better to new, unseen data, rather than
barely squeezing between the points.

11.2 What if data isn't a straight line apart?

Real data is rarely perfectly separable by a straight line. SVM handles this in two ways:

1. Soft margin (the C parameter): allows the model to accept a few misclassified points in exchange for a wider, more generalizable margin. A small C =
more tolerant of errors (wider margin, simpler boundary). A large C = tries hard to classify every point correctly (narrower margin, risk of overfitting).
2. The Kernel Trick: SVM can project data into a higher dimension where it becomes separable by a straight line/plane, using clever maths called "kernels" —
without actually having to compute those extra dimensions directly (which would be very expensive). This lets SVM draw curved, complex boundaries in the
original space.

11.3 Common Kernels

Kernel Use case

'linear' Data is roughly separable by a straight line

'rbf' (default) Data has complex, curved boundaries — most common choice

'poly' Polynomial-shaped boundaries

'sigmoid' Similar to a neural network activation shape

11.4 Key Hyperparameters

Parameter Meaning

C Controls the trade-off between margin width and misclassification. Default 1.0.

kernel Which kernel function to use to transform the data.

gamma For 'rbf'/'poly'/'sigmoid' kernels: how far the influence of one point reaches. High gamma = tighter, more local boundary (risk of overfitting).

Critical: SVM is extremely sensitive to feature scale. You must always apply StandardScaler before training an SVM, or your results will likely be poor.
 12. Support Vector Machine (SVM) — Full Code Walkthrough

# --- 1. Import everything we need ---

from [Link] import load_breast_cancer
from sklearn.model_selection import train_test_split
from [Link] import StandardScaler
from [Link] import SVC
from [Link] import accuracy_score, classification_report, confusion_matrix

# --- 2. Load the data ---

data = load_breast_cancer()
X = [Link]
y = [Link]

# --- 3. Split into train and test sets ---

X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y
)

# --- 4. Scale the data (REQUIRED for SVM) ---

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = [Link](X_test)

# --- 5. Create and train the SVM model ---

svm_model = SVC(
kernel='rbf', # curved decision boundary
C=1.0, # moderate regularization
gamma='scale', # sklearn auto-computes a sensible gamma
random_state=42
)
svm_model.fit(X_train_scaled, y_train)

# --- 6. Make predictions ---

y_pred = svm_model.predict(X_test_scaled)

# --- 7. Evaluate ---

print("Accuracy:", accuracy_score(y_test, y_pred))
print("\nClassification Report:\n", classification_report(y_test, y_pred))
print("Confusion Matrix:\n", confusion_matrix(y_test, y_pred))

12.1 SVM for Regression

SVM also has a regression version, SVR , used the same way:

from [Link] import SVR

reg = SVR(kernel='rbf', C=1.0)

[Link](X_train_scaled, y_train)
preds = [Link](X_test_scaled)

12.2 Visualizing a simple 2D SVM boundary

To truly build intuition, it helps to see the boundary drawn. Here's minimal code using just 2 features so it can be plotted:

import [Link] as plt

import numpy as np

X2 = X_train_scaled[:, :2] # just first 2 features for visualization

svm2 = SVC(kernel='rbf', C=1.0).fit(X2, y_train)

x_min, x_max = X2[:, 0].min()-1, X2[:, 0].max()+1

y_min, y_max = X2[:, 1].min()-1, X2[:, 1].max()+1
xx, yy = [Link]([Link](x_min, x_max, 200),
[Link](y_min, y_max, 200))
Z = [Link](np.c_[[Link](), [Link]()]).reshape([Link])

[Link](xx, yy, Z, alpha=0.3)

[Link](X2[:, 0], X2[:, 1], c=y_train, edgecolors='k')
[Link]("SVM Decision Boundary")
[Link]()
 13. Random Forest vs SVM — Comparison

Aspect Random Forest SVM

Type Ensemble of decision trees Margin-based boundary

Needs feature scaling? No Yes, essential

Handles large datasets Yes, scales well Can be slow on very large data

Handles high dimensions Good Very good (esp. with kernel trick)

Interpretability Medium (feature importances available) Low (especially with non-linear kernels)

Overfitting risk Lower (averaging reduces variance) Can overfit with high C / gamma

Training speed Fast, parallelizable Slower on large datasets

Good default choice when... You want a strong, low-maintenance baseline You have a smaller, cleanly scaled dataset with a clear margin

Practical advice: In most real-world beginner projects, start with Random Forest as your baseline — it's forgiving, needs little preprocessing, and often
performs very well out-of-the-box. Try SVM as a strong alternative, especially on smaller, well-scaled datasets.

14. Cross-Validation & Hyperparameter Tuning

A single train/test split can be a bit lucky or unlucky. Cross-validation gives a more reliable score by splitting the data into several "folds", training/testing
multiple times, and averaging the results.

from sklearn.model_selection import cross_val_score

scores = cross_val_score(rf_model, X, y, cv=5) # 5-fold cross-validation

print("Average accuracy:", [Link]())

To automatically find the best hyperparameters, use GridSearchCV , which tries every combination you specify and keeps the best one:

from sklearn.model_selection import GridSearchCV

param_grid = {
'n_estimators': [100, 200, 300],
'max_depth': [4, 6, 8, None]
}

grid = GridSearchCV(RandomForestClassifier(random_state=42), param_grid, cv=5)

[Link](X_train, y_train)

print("Best parameters:", grid.best_params_)

print("Best score:", grid.best_score_)
best_model = grid.best_estimator_

For SVM, a typical grid search looks like:

param_grid = {
'C': [0.1, 1, 10, 100],
'gamma': ['scale', 0.01, 0.1, 1],
'kernel': ['rbf', 'linear']
}
grid = GridSearchCV(SVC(), param_grid, cv=5)
[Link](X_train_scaled, y_train)
print(grid.best_params_)
 15. Common Beginner Mistakes

1. Fitting the scaler on the whole dataset before splitting. Always split first, then fit the scaler only on training data.
2. Judging a model by accuracy alone on imbalanced data. Check precision/recall/F1 too.
3. Forgetting to scale data before SVM, KNN, or logistic regression. These algorithms are distance/margin based and need scaled input.
4. Not setting random_state , making results impossible to reproduce or debug consistently.
5. Using default hyperparameters and assuming that's the best the model can do. Always try cross-validation and a grid search before concluding a
model "doesn't work well".
6. Testing on the training data and being surprised when real-world performance is much worse.
7. Ignoring class imbalance. If one class is rare, consider class_weight='balanced' , available in both RandomForestClassifier and SVC .

16. Final Cheat Sheet

# THE FULL SKLEARN WORKFLOW IN ONE BLOCK

from sklearn.model_selection import train_test_split, cross_val_score
from [Link] import StandardScaler
from [Link] import RandomForestClassifier
from [Link] import SVC
from [Link] import accuracy_score, classification_report

# 1. Split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# 2. Scale
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = [Link](X_test)

# 3. Choose & train a model

model = RandomForestClassifier(n_estimators=200, random_state=42) # or SVC(kernel='rbf')
[Link](X_train, y_train)

# 4. Predict & evaluate

y_pred = [Link](X_test)
print(accuracy_score(y_test, y_pred))
print(classification_report(y_test, y_pred))

Task Sklearn Tool

Split data train_test_split

Scale features StandardScaler

Encode categories LabelEncoder / OneHotEncoder

Classification (trees) RandomForestClassifier

Classification (margin) SVC

Regression (trees) RandomForestRegressor

Regression (margin) SVR

Evaluate classification accuracy_score , classification_report , confusion_matrix

Evaluate regression r2_score , mean_squared_error

Cross-validate cross_val_score

Tune hyperparameters GridSearchCV

Final advice for a beginner: You don't need to memorize every parameter. Start with defaults, get a working pipeline end-to-end (load → split → scale →
fit → predict → evaluate), and only then start tuning. Once you've built this pipeline once, every future sklearn model — whether it's Logistic Regression,
KNN, Gradient Boosting, or Neural Networks — follows the exact same shape.