■ Python · Data Science

Machine Learning & Deep

Learning
Complete Mastery Guide
From Zero to Production — Beginner to Expert

Covers: Python Fundamentals · NumPy · Pandas · Matplotlib · Statistics

Scikit-Learn · ML Algorithms · Feature Engineering · Model Evaluation
Neural Networks · CNNs · RNNs · Transformers · PyTorch · Keras

Real-World Examples & Projects Throughout Every Chapter

Table of Contents

PART 1 — Python Foundations

• Chapter 1 – Python Basics: Variables, Types & Operators
• Chapter 2 – Control Flow & Functions
• Chapter 3 – Data Structures: Lists, Tuples, Dicts, Sets
• Chapter 4 – Object-Oriented Programming (OOP)
• Chapter 5 – File I/O, Error Handling & Modules
• Chapter 6 – Functional Programming & Comprehensions

PART 2 — Math & Statistics Prerequisites

• Chapter 7 – Linear Algebra for ML
• Chapter 8 – Probability & Statistics
• Chapter 9 – Calculus Intuition for Gradient Descent

PART 3 — Data Science Toolkit

• Chapter 10 – NumPy: Numerical Computing
• Chapter 11 – Pandas: Data Manipulation
• Chapter 12 – Matplotlib & Seaborn: Visualization
• Chapter 13 – Exploratory Data Analysis (EDA)
• Chapter 14 – Feature Engineering & Data Preprocessing

PART 4 — Machine Learning

• Chapter 15 – ML Foundations & the Scikit-Learn API
• Chapter 16 – Linear & Logistic Regression
• Chapter 17 – Decision Trees & Random Forests
• Chapter 18 – Support Vector Machines
• Chapter 19 – K-Nearest Neighbors & Naive Bayes
• Chapter 20 – Unsupervised Learning: K-Means & PCA
• Chapter 21 – Model Evaluation, Bias-Variance & Tuning
• Chapter 22 – End-to-End ML Project

PART 5 — Deep Learning

• Chapter 23 – Neural Networks from Scratch
• Chapter 24 – Keras & PyTorch Fundamentals
• Chapter 25 – Convolutional Neural Networks (CNNs)
• Chapter 26 – Recurrent Networks & LSTMs
• Chapter 27 – Transfer Learning
• Chapter 28 – Transformers & Attention Mechanism
• Chapter 29 – Deploying ML/DL Models
 PART 1

Python Foundations
Variables · Control Flow · OOP · File I/O · Functional Programming
 Chapter 1 — Python Basics: Variables, Types & Operators

Python is a high-level, interpreted, dynamically typed language beloved in data science, machine learning,
web development, and automation. Its clean syntax means you spend less time wrestling with the language
and more time solving real problems.

Why Python for ML & Data Science?

Python dominates data science because of its rich ecosystem: NumPy for fast arrays, Pandas for data
manipulation, scikit-learn for ML, and PyTorch/TensorFlow for deep learning. Every major AI lab ships
Python APIs.

Variables & Data Types

Python is dynamically typed: you don't declare a type, Python infers it at runtime. The four primitive types
you'll use constantly:

Type Example Use in ML

int age = 25 Epochs, batch size

float lr = 0.001 Learning rate, loss

str name = 'Alice' Labels, file paths

bool is_trained = True Flags, conditions

NoneType result = None Missing/default values

Type Checking & Conversion

x = 42
print(type(x)) # <class 'int'>

# Type conversion (casting)

pi_str = '3.14'

pi_float = float(pi_str) # 3.14

pi_int = int(pi_float) # 3 (truncates)

# f-strings (Python 3.6+) — used everywhere in data science output

name, score = 'Model', 0.9234

print(f'{name} accuracy: {score:.2%}') # Model accuracy: 92.34%

■ Real-World Application

In a fraud detection pipeline, you receive transaction data as strings from a CSV. Casting str → float (amount)
and str → int (user_id) is always the first data cleaning step. Type errors here cause silent bugs — always
validate types early.

Operators
Category Operators Example

Arithmetic + - * / // % ** loss ** 2, n // batch_size

Comparison == != < > <= >= if accuracy >= 0.95

Logical and or not if trained and not overfitting

Assignment += -= *= /= epoch += 1

Membership in not in if label in class_names

Identity is is not if result is None

 Chapter 2 — Control Flow & Functions

Control Flow
Programs make decisions with if/elif/else, repeat with for/while loops, and escape early with
break/continue/return.

# Training loop — the heartbeat of every ML model

for epoch in range(1, 101):

train_loss = train_one_epoch(model, data)

val_loss = validate(model, val_data)

if val_loss < best_loss:

best_loss = val_loss

save_model(model, '[Link]')

elif val_loss > best_loss * 1.05:

print(f'Epoch {epoch}: possible overfitting')

break # early stopping

print(f'Epoch {epoch:3d} | Train: {train_loss:.4f} | Val: {val_loss:.4f}')

Functions
Functions are the building blocks of clean ML code. Python supports default arguments, keyword arguments,
*args, **kwargs, and type hints.

def train_model(X, y, lr=0.01, epochs=100, verbose=True):

'''

Train a simple model.

Args:

X : feature matrix

y : labels

lr : learning rate (default 0.01)

epochs : number of passes (default 100)

Returns:

weights : trained parameter vector

'''

weights = initialize_weights([Link][1])

for e in range(epochs):

preds = predict(X, weights)

loss = mse_loss(preds, y)

weights = update_weights(weights, X, preds, y, lr)

if verbose and e % 10 == 0:

print(f'Epoch {e}: Loss = {loss:.4f}')

return weights
Lambda Functions & Higher-Order Functions
Lambda functions are one-liner anonymous functions heavily used with map/filter/sorted in data pipelines.

# Sort models by validation accuracy (descending)

models = [('RandomForest', 0.91), ('SVM', 0.88), ('XGBoost', 0.94)]

sorted_models = sorted(models, key=lambda m: m[1], reverse=True)

# [('XGBoost', 0.94), ('RandomForest', 0.91), ('SVM', 0.88)]

# map: apply a function to every element

losses = [0.8, 0.5, 0.3, 0.1]

squared = list(map(lambda x: x**2, losses))

# filter: keep elements that pass a test

good_models = list(filter(lambda m: m[1] > 0.90, models))

 Chapter 3 — Data Structures: Lists, Tuples, Dicts, Sets

Lists — Ordered, Mutable Sequences

Lists are the most versatile Python structure. In ML you use them for storing batch samples, history of losses,
class names, and feature lists.

class_names = ['cat', 'dog', 'bird', 'fish']

loss_history = []

# Append, extend, insert

loss_history.append(0.85)

loss_history.extend([0.73, 0.61, 0.52])

# Slicing — Python's superpower

last_5_losses = loss_history[-5:]

every_other = loss_history[::2]

# List comprehension — elegant one-liners

squared_losses = [l**2 for l in loss_history]

above_half = [l for l in loss_history if l > 0.5]

# Enumerate — get index + value simultaneously

for i, cls in enumerate(class_names):

print(f' Class {i}: {cls}')

Dictionaries — Key-Value Stores

Dictionaries power configuration objects, hyperparameter grids, label mappings, and result tracking in every
real ML project.

# Hyperparameter configuration (common pattern in ML)

config = {

'model' : 'resnet50',

'learning_rate': 0.001,

'batch_size' : 32,

'epochs' : 50,

'optimizer' : 'adam',

# Access and update

print(config['learning_rate']) # 0.001

config['epochs'] = 100

[Link]({'dropout': 0.3})

# defaultdict — useful for grouping

from collections import defaultdict

label_counts = defaultdict(int)
 for label in y_train:

label_counts[label] += 1

Tuples & Sets

# Tuples — immutable, used for fixed data like image dimensions

img_shape = (224, 224, 3) # height, width, channels

train_split = (X_train, y_train)

H, W, C = img_shape # unpacking

# Sets — unique elements, used for vocabulary and de-duplication

vocab = set(all_words) # unique words

overlap = set(train_ids) & set(test_ids) # intersection

assert len(overlap) == 0, 'Data leakage!'

 Chapter 4 — Object-Oriented Programming (OOP)

Object-Oriented Programming (OOP) organizes code into classes and objects. Every neural network in
PyTorch is a class. Understanding OOP is mandatory for deep learning.

Core Concepts
Concept Meaning ML Example

Class Blueprint for objects NeuralNetwork class

Object/Instance One concrete example of a class model = NeuralNetwork()

Attribute Data stored in an object [Link], [Link]

Method Function belonging to a class [Link](x), [Link]()

Inheritance Child class extends parent class CNN([Link])

Encapsulation Hide internal state private _weights attribute

Polymorphism Same interface, different behavior forward() for CNN vs RNN

class LinearRegression:

'''A simple linear regression model built from scratch.'''

def __init__(self, learning_rate=0.01, n_iterations=1000):

[Link] = learning_rate

self.n_iterations = n_iterations

[Link] = None

[Link] = None

def fit(self, X, y):

n_samples, n_features = [Link]

[Link] = [Link](n_features)

[Link] = 0

for _ in range(self.n_iterations):

y_pred = self._predict(X)

dw = (1/n_samples) * X.T @ (y_pred - y)

db = (1/n_samples) * [Link](y_pred - y)

[Link] -= [Link] * dw

[Link] -= [Link] * db

def _predict(self, X):

return X @ [Link] + [Link]

def predict(self, X):

return self._predict(X)

# Usage

model = LinearRegression(learning_rate=0.001, n_iterations=500)

[Link](X_train, y_train)

predictions = [Link](X_test)
 PART 2

Math & Statistics Prerequisites

Linear Algebra · Probability · Statistics · Calculus Intuition
 Chapter 7 — Linear Algebra for ML

Every machine learning operation is secretly a linear algebra operation. Understanding vectors, matrices,
and their operations is non-negotiable.

Vectors & Matrices

A vector is a 1D array of numbers — think of a single data point with multiple features (e.g. a patient's age,
blood pressure, cholesterol). A matrix is a 2D array — think of your entire dataset (rows = samples, columns
= features).

import numpy as np

# A patient's features: [age, bp, cholesterol, glucose]

patient = [Link]([45, 130, 220, 95])

# Dataset: 4 patients × 4 features

X = [Link]([

[45, 130, 220, 95],

[52, 145, 260, 110],

[38, 120, 180, 88],

[61, 155, 300, 125],

])

# Key shapes — you'll check these constantly in ML

print([Link]) # (4, 4)

print([Link]) # 2

# Matrix multiplication — the CORE of neural networks

W = [Link](4, 2) # weight matrix: 4 inputs → 2 outputs

output = X @ W # (4,4) @ (4,2) = (4,2)

# Transpose — swap rows and columns

print([Link]) # (4, 4) here, but (n_features, n_samples) in general

Key Linear Algebra Concepts in ML

Concept Definition Used In

Dot Product Sum of element-wise products Similarity, linear layer

Transpose Flip rows and columns Gradient computation

Inverse A such that A·A■¹ = I Solving linear systems

Eigenvalues Scaling factors of eigenvectors PCA dimensionality reduction

Norm (L2) Euclidean distance from origin Regularization, distances

SVD Singular Value Decomposition PCA, recommender systems

 Chapter 8 — Probability & Statistics

Statistics is how we understand data before feeding it to models. Probability is how models express
uncertainty in predictions.

Descriptive Statistics
import numpy as np

from scipy import stats

# House prices dataset (in $1000s)

prices = [Link]([150, 200, 250, 180, 300, 210, 175, 420, 195, 225])

print(f'Mean: ${[Link](prices):.1f}k') # average price

print(f'Median: ${[Link](prices):.1f}k') # middle value (robust to outliers)

print(f'Std Dev: ${[Link](prices):.1f}k') # spread of prices

print(f'Variance: ${[Link](prices):.1f}k²') # std²

print(f'Min: ${[Link](prices):.1f}k')

print(f'Max: ${[Link](prices):.1f}k')

print(f'Skewness: {[Link](prices):.3f}') # >0 = right tail (outliers high)

# Percentiles — used in feature normalization

q1, q3 = [Link](prices, [25, 75])

iqr = q3 - q1 # Interquartile Range

print(f'IQR: ${iqr:.1f}k → Outlier threshold: > ${q3 + 1.5*iqr:.1f}k')

Probability Distributions
Understanding distributions shapes how you preprocess features and how you interpret model outputs. The
four most important:

Distribution Shape ML Usage

Normal (Gaussian) Bell-shaped, symmetric Weight init, errors, feature scaling

Bernoulli Binary: 0 or 1 Binary classification output

Categorical K discrete outcomes Multi-class softmax output

Uniform Equal probability over range Random init, data augmentation

Poisson Count data NLP word counts, event prediction

Correlation & Hypothesis Testing

# Correlation: how strongly two features move together (-1 to +1)

import pandas as pd

df = [Link]({'age': [25,30,35,40,45], 'income': [30,40,55,65,80]})

print([Link]()) # 0.998 → very strong positive correlation

# Pearson correlation coefficient

r, p_value = [Link](df['age'], df['income'])

print(f'r = {r:.3f}, p = {p_value:.4f}') # p < 0.05 → statistically significant

# Why this matters in ML:

# Highly correlated features = multicollinearity = unstable weights
# Low p-value = the feature likely has real predictive signal
 Chapter 9 — Calculus Intuition for Gradient Descent

You don't need to hand-compute derivatives for ML — frameworks like PyTorch do it automatically
(autograd). But you DO need the intuition of what a derivative represents.

■ The Core Intuition

Imagine you're blindfolded on a hilly landscape and want to reach the lowest valley (minimum loss). The
derivative tells you the slope at your current position. You take a small step downhill (opposite to the gradient).
Repeat until you reach the bottom. This is gradient descent.

# Gradient descent in its purest form

def gradient_descent(loss_fn, gradient_fn, theta_init, lr=0.01, n_steps=1000):

theta = theta_init

history = []

for step in range(n_steps):

grad = gradient_fn(theta) # direction of steepest ascent

theta = theta - lr * grad # step OPPOSITE to gradient

[Link](loss_fn(theta))

return theta, history

# Real example: minimise f(x) = (x - 3)²

# f'(x) = 2(x - 3)

loss_fn = lambda x: (x - 3)**2

gradient_fn = lambda x: 2 * (x - 3)

optimal_x, losses = gradient_descent(loss_fn, gradient_fn, theta_init=10.0)

print(f'Optimal x ≈ {optimal_x:.4f}') # Should be very close to 3.0

Variant Data per Step Pros / Cons

Batch GD Full dataset + Stable, – Slow on large data

Stochastic GD 1 sample + Fast update, – Very noisy

Mini-Batch GD 32/64/128 samples + Best of both worlds (industry standard)

 PART 3

Data Science Toolkit

NumPy · Pandas · Matplotlib · EDA · Feature Engineering
 Chapter 10 — NumPy: Numerical Computing

NumPy (Numerical Python) is the foundation of every data science and ML library. It provides ndarray — a
blazing-fast multi-dimensional array that runs C-speed operations. When you use Pandas, scikit-learn, or
PyTorch, NumPy is running underneath.

Creating Arrays
import numpy as np

# From lists

a = [Link]([1, 2, 3, 4, 5])

m = [Link]([[1,2,3],[4,5,6],[7,8,9]])

# Special creators — used constantly

zeros = [Link]((3, 4)) # all zeros — common for weight init

ones = [Link]((2, 5))

eye = [Link](4) # identity matrix

rand = [Link](100, 10) # Gaussian noise — weight init!

arange = [Link](0, 1, 0.01) # [0.00, 0.01, ..., 0.99]

linsp = [Link](0, 1, 50) # 50 evenly spaced points

Broadcasting — NumPy's Secret Power

Broadcasting lets you perform operations between arrays of different shapes without explicit loops. This
makes feature normalization, bias addition, and batch operations elegant and fast.

# Normalise all 100 samples (each with 10 features) by feature mean and std

X = [Link](100, 10) # (100, 10) dataset

mu = [Link](axis=0) # (10,) — mean of each feature

std = [Link](axis=0) # (10,) — std of each feature

X_norm = (X - mu) / std # broadcasting: (100,10) - (10,) → (100,10)

# Add bias to each sample in a batch

logits = [Link](32, 5) # 32 samples, 5 class scores

bias = [Link]([0.1, -0.2, 0.3, 0.0, -0.1])

logits_biased = logits + bias # broadcasts bias across all 32 samples

 Chapter 11 — Pandas: Data Manipulation

Pandas is Python's answer to Excel — but infinitely more powerful. It introduces the DataFrame (table) and
Series (column). Every data scientist spends 60-70% of their time in Pandas.

Loading & Exploring Data

import pandas as pd

# Loading data — the starting point of every project

df = pd.read_csv('[Link]')

# First steps: always do these

print([Link]) # (891, 12) — 891 passengers, 12 columns

print([Link]()) # first 5 rows

print([Link]()) # column names, types, null counts

print([Link]()) # mean, std, percentiles of numeric columns

print([Link]().sum()) # count missing values per column

# Selecting data

ages = df['Age'] # single column → Series

subset = df[['Name', 'Age', 'Survived']] # multiple columns → DataFrame

adults = df[df['Age'] >= 18] # filter rows

women = df[(df['Sex']=='female') & (df['Pclass']==1)] # compound filter

Handling Missing Data

# Check missing data

missing = [Link]().sum()
missing_pct = (missing / len(df)) * 100

# Strategy 1: Drop columns with >50% missing

df = [Link](columns=['Cabin']) # 77% missing — not salvageable

# Strategy 2: Fill with median (robust to outliers)

df['Age'].fillna(df['Age'].median(), inplace=True)

# Strategy 3: Fill categorical with mode

df['Embarked'].fillna(df['Embarked'].mode()[0], inplace=True)

# Strategy 4: Interpolate time series

df['sensor_reading'].interpolate(method='linear', inplace=True)

GroupBy & Aggregation

# Survival rate by class and sex — classic EDA

survival = [Link](['Pclass','Sex'])['Survived'].agg(['mean','count'])

[Link] = ['survival_rate', 'count']

survival['survival_pct'] = (survival['survival_rate'] * 100).round(1)

print(survival)

# Output:
# survival_rate count survival_pct
# Pclass Sex
# 1 female 0.968 94 96.8
# 1 male 0.369 122 36.9
# 3 female 0.500 144 50.0
# 3 male 0.135 347 13.5
 Chapter 13 — Exploratory Data Analysis (EDA)

EDA is the practice of investigating data before modelling. You look for patterns, anomalies, distributions,
and relationships. Skipping EDA causes bad models.

■ The EDA Checklist

1. Dataset shape and types 2. Missing values 3. Class balance (classification) 4. Distribution of each feature 5.
Outlier detection 6. Correlation matrix 7. Feature vs target plots 8. Time trends (if time series)

import pandas as pd

import numpy as np

import [Link] as plt

import seaborn as sns

# --- STEP 1: Load and overview ---

df = pd.read_csv('house_prices.csv')

print([Link]())

# --- STEP 2: Visualize target distribution ---

[Link](figsize=(10, 4))

[Link](1,2,1)

df['SalePrice'].hist(bins=50, color='steelblue', edgecolor='white')

[Link]('SalePrice Distribution (raw — right skewed)')

[Link](1,2,2)

np.log1p(df['SalePrice']).hist(bins=50, color='green', edgecolor='white')

[Link]('log(SalePrice+1) — more normal, better for regression')

plt.tight_layout()

# --- STEP 3: Correlation heatmap ---

numeric_cols = df.select_dtypes(include=[[Link]]).columns

corr_matrix = df[numeric_cols].corr()

[Link](figsize=(12, 10))

[Link](corr_matrix, annot=False, cmap='coolwarm', center=0)

[Link]('Feature Correlation Matrix')

# --- STEP 4: Check class balance ---

print(df['target'].value_counts(normalize=True))

# If 95% class 0, 5% class 1 → severe imbalance → use SMOTE or class weights

 Chapter 14 — Feature Engineering & Data Preprocessing

Feature engineering transforms raw data into a form that learning algorithms can exploit. It is often the single
most impactful step in improving model performance.

Scaling & Normalization

Most ML algorithms assume features are on comparable scales. Neural networks and SVMs are especially
sensitive to this.

from [Link] import StandardScaler, MinMaxScaler, RobustScaler

# StandardScaler: mean=0, std=1 (best for Gaussian-ish data)

scaler = StandardScaler()

X_scaled = scaler.fit_transform(X_train) # fit ONLY on train!

X_test_scaled = [Link](X_test) # transform test with same params

# MinMaxScaler: range [0, 1] (best for neural networks, images)

mm_scaler = MinMaxScaler()

X_mm = mm_scaler.fit_transform(X_train)

# RobustScaler: uses median+IQR (best when data has outliers)

rob = RobustScaler()

X_rob = rob.fit_transform(X_train)

# ■■ GOLDEN RULE: NEVER fit scaler on test set!

# Fitting on test leaks future information → inflated performance metrics

Encoding Categorical Features

from [Link] import LabelEncoder, OneHotEncoder
import pandas as pd

df = [Link]({'color': ['red','blue','green','red','green']})

# One-Hot Encoding — for nominal categories (no order)

# red=[1,0,0] blue=[0,1,0] green=[0,0,1]

ohe = pd.get_dummies(df['color'], prefix='color')

# Label Encoding — for ordinal categories (has order)

# small=0, medium=1, large=2

size_map = {'small': 0, 'medium': 1, 'large': 2}

df['size_encoded'] = df['size'].map(size_map)

# Target Encoding — replace category with mean target value (advanced)

# Powerful but can cause overfitting — use with cross-validation

target_mean = [Link]('city')['price'].mean()

df['city_encoded'] = df['city'].map(target_mean)
 PART 4

Machine Learning
Supervised · Unsupervised · Model Evaluation · Scikit-Learn
 Chapter 15 — ML Foundations & the Scikit-Learn API

Machine learning is about building systems that learn patterns from data to make predictions or decisions
without being explicitly programmed for each case.

The Three Types of ML

Type Has Labels? Goal Examples

Supervised Yes (X, y) Learn X→y mapping Spam filter, house price, cancer detection

Unsupervised No (X only) Find hidden structure Customer segments, anomaly detection

Reinforcement Reward signal Maximize cumulative reward Chess AI, robot navigation, trading

The Universal Scikit-Learn API

Every scikit-learn model follows the same three-method API. Learn it once, apply it everywhere.

from [Link] import RandomForestClassifier

from sklearn.model_selection import train_test_split

from [Link] import accuracy_score, classification_report

# 1. Split data

X_train, X_test, y_train, y_test = train_test_split(

X, y, test_size=0.2, random_state=42, stratify=y

# 2. Instantiate (set hyperparameters)

model = RandomForestClassifier(n_estimators=100, max_depth=5, random_state=42)

# 3. Fit (train)
[Link](X_train, y_train)

# 4. Predict

y_pred = [Link](X_test)

y_proba = model.predict_proba(X_test) # probability scores

# 5. Evaluate

print(f'Accuracy: {accuracy_score(y_test, y_pred):.2%}')

print(classification_report(y_test, y_pred, target_names=class_names))

 Chapter 16 — Linear & Logistic Regression

Linear Regression
Linear Regression models the relationship between features and a continuous target by fitting a straight line
(or hyperplane) that minimizes the sum of squared errors.

■ The Equation

■ = w■x■ + w■x■ + ... + w■x■ + b where w = weights (learned), b = bias Loss (MSE) = (1/n) Σ (y■ - ■■)² Goal:
find w and b that minimize MSE.

from sklearn.linear_model import LinearRegression, Ridge, Lasso

from [Link] import mean_squared_error, r2_score

import numpy as np

# Real-world example: predict house price from size & bedrooms

model = LinearRegression()

[Link](X_train, y_train)

print('Coefficients:', model.coef_)

# e.g. [150.3, 20000.5] → each sq ft adds $150, each bedroom adds $20k

print('Intercept: ', model.intercept_)

y_pred = [Link](X_test)

rmse = [Link](mean_squared_error(y_test, y_pred))

r2 = r2_score(y_test, y_pred)

print(f'RMSE: ${rmse:,.0f} R²: {r2:.3f}')

# Ridge (L2 regularization) — prevents overfitting

ridge = Ridge(alpha=1.0) # alpha = regularization strength

[Link](X_train, y_train)

# Lasso (L1 regularization) — can zero out irrelevant features

lasso = Lasso(alpha=0.1) # drives unimportant weights to exactly 0

[Link](X_train, y_train)

Logistic Regression — Binary Classification

Despite the name, Logistic Regression is a classification algorithm. It predicts the probability that a sample
belongs to a class using the sigmoid function: σ(z) = 1/(1+e^-z). Real-world uses: spam detection, disease
diagnosis, credit default.

from sklearn.linear_model import LogisticRegression

from [Link] import confusion_matrix, roc_auc_score

# Example: predict diabetes (0=No, 1=Yes)

clf = LogisticRegression(C=1.0, max_iter=1000, random_state=42)

# C = inverse regularization strength (small C → strong regularization)

[Link](X_train, y_train)

y_pred = [Link](X_test)

y_proba = clf.predict_proba(X_test)[:, 1] # probability of class=1

# Confusion Matrix

cm = confusion_matrix(y_test, y_pred)

# [[TN, FP],
# [FN, TP]]

# AUC-ROC — gold standard for binary classification

auc = roc_auc_score(y_test, y_proba)

print(f'AUC-ROC: {auc:.3f}') # 1.0 = perfect, 0.5 = random

 Chapter 17 — Decision Trees & Random Forests

Decision Trees split data based on feature thresholds to create a flowchart of decisions. Random Forests
combine hundreds of trees (ensemble learning) to dramatically reduce overfitting and improve accuracy.

How Decision Trees Work

Term Meaning

Root Node First split — feature with most information gain

Internal Node Subsequent splits on sub-features

Leaf Node Final prediction (class or regression value)

Depth Number of splits from root to leaf

Gini Impurity Measure of class mixing at a node (lower = purer)

Information Gain Reduction in entropy after a split

from [Link] import DecisionTreeClassifier, export_text

from [Link] import RandomForestClassifier, GradientBoostingClassifier

# --- Decision Tree ---

dt = DecisionTreeClassifier(max_depth=5, min_samples_leaf=10, random_state=42)

[Link](X_train, y_train)

print(export_text(dt, feature_names=feature_names)) # human-readable rules!

# --- Random Forest ---

# Each tree sees a random subset of features and data (bootstrap sampling)
# Final prediction = majority vote across all trees

rf = RandomForestClassifier(
n_estimators=200, # 200 trees

max_features='sqrt', # each tree sees sqrt(n_features) features

max_depth=10,

min_samples_leaf=5,

n_jobs=-1, # use all CPU cores

random_state=42

[Link](X_train, y_train)

# Feature importance — understand which features drive predictions

importances = [Link](rf.feature_importances_, index=feature_names)

print(importances.sort_values(ascending=False).head(10))

■ Real-World: Credit Risk Scoring

Banks use Random Forests to score loan applications. A model trained on 50,000 historical loans (features:
income, debt ratio, credit history, employment) achieves AUC-ROC of 0.89. Feature importance reveals
'credit_utilization' and 'months_since_delinquency' are the top 2 predictors. Regulators require model
explainability — decision rules from the tree make this possible.
 Chapter 21 — Model Evaluation, Bias-Variance & Tuning

Building a model is easy. Building one that generalizes to unseen data is hard. This chapter covers the
diagnostics and tuning techniques every practitioner must master.

The Bias-Variance Tradeoff

Problem Symptom Cause Fix

High Bias
Bad train AND test Model too simple More features, deeper model
(Underfitting)

High Variance
Good train, bad test Model too complex Regularize, more data, dropout
(Overfitting)

Ideal Balance Good train AND test Right complexity Cross-validate, tune carefully

Cross-Validation & Hyperparameter Tuning

from sklearn.model_selection import cross_val_score, GridSearchCV, RandomizedSearchCV

from [Link] import Pipeline

from [Link] import StandardScaler

# 5-Fold Cross Validation — get reliable performance estimate

scores = cross_val_score(model, X, y, cv=5, scoring='roc_auc', n_jobs=-1)

print(f'CV AUC: {[Link]():.3f} ± {[Link]():.3f}')

# Pipeline — prevents data leakage (scale only sees train fold)

pipe = Pipeline([

('scaler', StandardScaler()),

('classifier', RandomForestClassifier(random_state=42))
])

# Grid Search — exhaustive search over parameter grid

param_grid = {

'classifier__n_estimators': [100, 200],

'classifier__max_depth' : [5, 10, None],

'classifier__min_samples_leaf': [1, 5, 10],

gs = GridSearchCV(pipe, param_grid, cv=5, scoring='roc_auc', n_jobs=-1)

[Link](X_train, y_train)

print(f'Best params: {gs.best_params_}')

print(f'Best AUC: {gs.best_score_:.3f}')

 PART 5

Deep Learning
Neural Networks · CNNs · RNNs · Transformers · PyTorch · Keras
 Chapter 23 — Neural Networks from Scratch

A neural network is a stack of transformations. Each layer learns to extract increasingly abstract
representations. The first layer might detect edges; deeper layers detect faces, objects, or concepts.

Anatomy of a Neuron
A single neuron computes: output = activation(Σ w■x■ + b). It takes weighted inputs, sums them with a
bias, then applies a nonlinear activation function. Without nonlinearity, stacking layers is pointless — you'd
just get a linear model.

Activation Formula Range Used In

ReLU max(0, x) [0, ∞) Hidden layers (default)

Sigmoid 1/(1+e^-x) (0, 1) Binary output layer

Tanh (e^x - e^-x)/(e^x + e^-x) (-1, 1) RNN hidden states

Softmax e^x■ / Σe^x■ (0,1), Σ=1 Multi-class output

Leaky ReLU max(0.01x, x) (-∞, ∞) When ReLU neurons die

GELU x·Φ(x) (-∞, ∞) Transformers (BERT, GPT)

Backpropagation — How Neural Networks Learn

Backpropagation computes gradients of the loss with respect to every weight using the chain rule of calculus.
The framework (PyTorch/TensorFlow) does this automatically.

Neural Network in Pure NumPy

import numpy as np

class TwoLayerNet:

def __init__(self, input_size, hidden_size, output_size):

# He initialization (good for ReLU)

self.W1 = [Link](input_size, hidden_size) * [Link](2/input_size)

self.b1 = [Link](hidden_size)

self.W2 = [Link](hidden_size, output_size) * [Link](2/hidden_size)

self.b2 = [Link](output_size)

def relu(self, z): return [Link](0, z)

def sigmoid(self, z): return 1 / (1 + [Link](-z))

def forward(self, X):

self.z1 = X @ self.W1 + self.b1

self.a1 = [Link](self.z1)

self.z2 = self.a1 @ self.W2 + self.b2

self.a2 = [Link](self.z2) # output probability

return self.a2
def backward(self, X, y, lr=0.01):

m = [Link][0]

# Output layer gradient

dz2 = self.a2 - [Link](-1, 1)

dW2 = self.a1.T @ dz2 / m

db2 = [Link](axis=0)

# Hidden layer gradient

da1 = dz2 @ self.W2.T

dz1 = da1 * (self.z1 > 0) # ReLU gradient

dW1 = X.T @ dz1 / m

db1 = [Link](axis=0)

# Update weights

self.W2 -= lr * dW2; self.b2 -= lr * db2

self.W1 -= lr * dW1; self.b1 -= lr * db1

 Chapter 24 — Keras & PyTorch Fundamentals

Keras — High-Level API

Keras (now part of TensorFlow) makes building neural networks intuitive with a LEGO-like API. Best for rapid
prototyping and beginners.

import tensorflow as tf

from tensorflow import keras

from [Link] import layers

# Build a model for tabular classification

model = [Link]([

[Link](shape=(n_features,)),

[Link](256, activation='relu'),

[Link](),

[Link](0.3), # randomly zero 30% of neurons during training

[Link](128, activation='relu'),

[Link](0.2),

[Link](n_classes, activation='softmax')

])

[Link](

optimizer=[Link](learning_rate=0.001),

loss='sparse_categorical_crossentropy',

metrics=['accuracy']

# Callbacks: early stopping + learning rate reduction

callbacks = [

[Link](patience=10, restore_best_weights=True),

[Link](factor=0.5, patience=5),

[Link]('best_model.keras', save_best_only=True),

history = [Link](

X_train, y_train,

epochs=100, batch_size=64,

validation_split=0.15,

callbacks=callbacks,

verbose=1

PyTorch — Research-Grade Framework

PyTorch gives you full control with dynamic computation graphs. The standard choice for research, custom
architectures, and cutting-edge models.

import torch

import [Link] as nn

import [Link] as optim

from [Link] import DataLoader, TensorDataset

# Define model as a class (standard PyTorch pattern)

class MLP([Link]):

def __init__(self, input_dim, hidden_dim, output_dim, dropout=0.3):

super().__init__()

[Link] = [Link](

[Link](input_dim, hidden_dim),

nn.BatchNorm1d(hidden_dim),

[Link](),

[Link](dropout),

[Link](hidden_dim, hidden_dim // 2),

[Link](),

[Link](hidden_dim // 2, output_dim)

def forward(self, x):

return [Link](x)

# Training loop

model = MLP(input_dim=20, hidden_dim=256, output_dim=2).to(device)

optimizer = [Link]([Link](), lr=1e-3, weight_decay=1e-4)

criterion = [Link]()
scheduler = optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=50)

for epoch in range(100):

[Link]()

for X_batch, y_batch in train_loader:

X_batch, y_batch = X_batch.to(device), y_batch.to(device)

optimizer.zero_grad()

loss = criterion(model(X_batch), y_batch)

[Link]() # compute gradients

[Link]() # update weights

[Link]()
 Chapter 25 — Convolutional Neural Networks (CNNs)

CNNs are the backbone of computer vision. They use convolutional filters to detect local patterns (edges,
textures, shapes) while being translation-invariant — a cat is a cat whether in the top-left or bottom-right of an
image.

CNN Building Blocks

Layer Operation Purpose

Conv2D Slide filter over image, compute dot products Detect local features (edges, textures)

ReLU max(0, x) applied elementwise Add nonlinearity

MaxPooling2D Take max in each 2×2 window Reduce spatial size, keep dominant features

BatchNorm Normalize across batch Stabilize training, allow higher lr

Dropout Randomly zero neurons during training Prevent overfitting

Flatten Reshape 3D feature map to 1D vector Connect to fully connected layers

Dense Standard fully-connected layer Final classification/regression

import [Link] as nn

class CNN([Link]):

'''Image classifier for 32×32 RGB images (e.g. CIFAR-10).'''

def __init__(self, num_classes=10):

super().__init__()

[Link] = [Link](

nn.Conv2d(3, 32, kernel_size=3, padding=1), # 3 channels → 32 filters

nn.BatchNorm2d(32),
[Link](inplace=True),

nn.MaxPool2d(2, 2), # 32×32 → 16×16

nn.Conv2d(32, 64, kernel_size=3, padding=1), # 32 → 64 filters

nn.BatchNorm2d(64),

[Link](inplace=True),

nn.MaxPool2d(2, 2), # 16×16 → 8×8

nn.Conv2d(64, 128, kernel_size=3, padding=1),

nn.BatchNorm2d(128),

[Link](inplace=True),

nn.MaxPool2d(2, 2), # 8×8 → 4×4

[Link] = [Link](

[Link](),

[Link](128 * 4 * 4, 512),

[Link](inplace=True),
 [Link](0.5),

[Link](512, num_classes)

def forward(self, x):

return [Link]([Link](x))

■ Real-World: Medical Imaging

CNNs power chest X-ray diagnosis (pneumonia, COVID), retinal disease screening, skin cancer classification,
and pathology slide analysis. A ResNet-50 fine-tuned on 50,000 dermoscopy images achieves
dermatologist-level accuracy (AUC 0.91) for melanoma detection.
 Chapter 27 — Transfer Learning

Transfer learning is arguably the single most practical deep learning technique. Instead of training from
scratch (needs millions of images + days of GPU time), you start from a powerful pre-trained model and
fine-tune it on your small dataset.

When to Use Transfer Learning

✓ You have < 10,000 labeled images (almost always)
✓ Your domain is similar to ImageNet (photos, medical images, satellite images)
✓ You need fast results — fine-tuning takes hours, not weeks
✓ You want to avoid overfitting on small datasets

import [Link] as models

import [Link] as nn

# Load pretrained ResNet-50 (trained on 1.2M ImageNet images)

backbone = models.resnet50(weights='IMAGENET1K_V2')

# Strategy 1: Freeze ALL layers — only train the final classifier

# Use when: very small dataset, domain close to ImageNet

for param in [Link]():

param.requires_grad = False

# Replace final layer for your task (e.g. 5-class flower classifier)

[Link] = [Link]([Link].in_features, 5)

# Only fc layer parameters will be updated

# Strategy 2: Fine-tune entire model with low LR

# Use when: larger dataset, domain slightly different
for param in [Link]():

param.requires_grad = True

optimizer = [Link]([

{'params': [Link](), 'lr': 1e-4},

{'params': [Link](), 'lr': 1e-3}, # higher LR for new layer

], weight_decay=1e-4)
 Chapter 28 — Transformers & Attention Mechanism

Transformers (Vaswani et al., 2017 — 'Attention is All You Need') revolutionized NLP and have since
conquered vision, audio, biology, and almost every other domain. GPT-4, BERT, Stable Diffusion, AlphaFold
— all Transformers.

Self-Attention — The Core Innovation

Self-attention lets every token in a sequence look at every other token and decide how much to attend to
each. This captures long-range dependencies that RNNs struggle with (the meaning of 'it' in a sentence
might depend on a word 50 positions earlier).

■ The QKV Mechanism

For each token, we compute three vectors: Q (Query): What am I looking for? K (Key): What do I offer to others?
V (Value): What information do I carry? Attention(Q, K, V) = softmax(QK^T / √d_k) · V The dot product QK^T
measures compatibility between tokens. Dividing by √d_k prevents gradient vanishing for large dimensions.
Softmax normalizes to a probability distribution over tokens.

import torch

import [Link] as nn

import [Link] as F

class SelfAttention([Link]):

'''Single-head self-attention.'''

def __init__(self, d_model, d_k):

super().__init__()

self.d_k = d_k

self.W_q = [Link](d_model, d_k, bias=False)

self.W_k = [Link](d_model, d_k, bias=False)

self.W_v = [Link](d_model, d_k, bias=False)

def forward(self, x):

# x: (batch, seq_len, d_model)

Q = self.W_q(x) # (batch, seq, d_k)

K = self.W_k(x) # (batch, seq, d_k)

V = self.W_v(x) # (batch, seq, d_k)

scores = Q @ [Link](-2, -1) / self.d_k**0.5

weights = [Link](scores, dim=-1) # attention weights

return weights @ V # weighted sum of values

# In practice: use [Link] (multiple parallel attention heads)

mha = [Link](embed_dim=512, num_heads=8, batch_first=True)

Using Pre-Trained Transformers with HuggingFace

from transformers import AutoTokenizer, AutoModelForSequenceClassification

import torch

# Sentiment analysis with BERT — 5 lines of code!

model_name = 'distilbert-base-uncased-finetuned-sst-2-english'

tokenizer = AutoTokenizer.from_pretrained(model_name)

model = AutoModelForSequenceClassification.from_pretrained(model_name)

texts = [

'This product is absolutely amazing!',

'Terrible experience, would not recommend.',

inputs = tokenizer(texts, padding=True, truncation=True, return_tensors='pt')

with torch.no_grad():

logits = model(**inputs).logits

predictions = [Link](dim=-1)

labels = ['Negative', 'Positive']

for text, pred in zip(texts, predictions):

print(f'{labels[pred]:8s}: {text[:50]}')
 Chapter 29 — Deploying ML/DL Models

A model that lives only in a Jupyter notebook helps no one. This chapter covers the essential patterns for
serving models in production.

Saving & Loading Models

# Scikit-learn models

import joblib

[Link](model, '[Link]') # save

loaded_model = [Link]('[Link]') # load

# PyTorch — save only weights (recommended)

[Link](model.state_dict(), '[Link]')

model.load_state_dict([Link]('[Link]'))

[Link]() # critical: disable dropout/batchnorm inference mode

# Keras

[Link]('[Link]') # save architecture + weights

loaded = [Link].load_model('[Link]')

REST API with FastAPI

The industry standard for serving ML models is a REST API. FastAPI is the modern Python choice — fast,
async, automatic documentation.

from fastapi import FastAPI

from pydantic import BaseModel

import joblib, numpy as np

app = FastAPI(title='House Price Predictor', version='1.0')

model = [Link]('house_price_model.joblib')

scaler = [Link]('[Link]')

class HouseFeatures(BaseModel):

square_feet: float

bedrooms: int

bathrooms: float

age_years: int

zip_code: str

@[Link]('/predict', response_model=dict)

async def predict_price(features: HouseFeatures):

X = [Link]([[

features.square_feet, [Link],

[Link], features.age_years

]])
 X_scaled = [Link](X)

price = [Link](X_scaled)[0]

return {'predicted_price': round(float(price), 2), 'currency': 'USD'}

# Run: uvicorn app:app --host [Link] --port 8000

# Docs: [Link] (auto-generated Swagger UI)
 Quick Reference — Metrics, Algorithms & Hyperparameters

Choosing the Right Algorithm

Problem Data Size Start With If More Perf Needed

Binary classification Any Logistic Regression XGBoost / Neural Net

Multi-class classif. Any Random Forest Gradient Boosting / CNN

Regression Any Ridge Regression Gradient Boosting / MLP

Image classification Large ResNet (transfer) EfficientNet / ViT

Text classification Medium+ TF-IDF + LR BERT / DistilBERT

Anomaly detection Any Isolation Forest Autoencoder

Clustering Any K-Means DBSCAN / GMM

Recommendation Large Matrix Factorization Neural Collaborative Filter

Time series forecast Any ARIMA / Prophet LSTM / Temporal Fusion

Evaluation Metrics Reference

Metric Task Formula When to Use

Accuracy Classification Correct / Total Balanced classes

Precision Classification TP / (TP+FP) False positives costly

Recall Classification TP / (TP+FN) False negatives costly

F1-Score Classification 2·P·R / (P+R) Imbalanced classes

AUC-ROC Binary classif. Area under ROC curve Ranking quality

MSE Regression Mean(y-■)² Penalizes large errors

RMSE Regression √MSE Same units as target

MAE Regression Mean|y-■| Robust to outliers

R² Score Regression 1 - SS_res/SS_tot Explained variance (0-1)

Log Loss Probability -Σ y■·log(■■) Probabilistic outputs

IoU Object Detection Intersection/Union Bounding box overlap

BLEU NLP/Translation n-gram precision Text generation quality

 Learning Roadmap & Recommended Resources

Your 6-Month Learning Path

Month Focus Milestone Project

Month 1 Python + NumPy + Pandas basics Titanic survival analysis with EDA

Month 2 Matplotlib, Seaborn, Statistics, Feature Eng. Housing price EDA + Kaggle submission

Month 3 Scikit-learn: LR, Trees, RF, SVM, evaluation End-to-end classification pipeline

Month 4 Keras: Fully-connected nets, CNNs MNIST / CIFAR-10 classifier, 95%+ acc

Month 5 PyTorch, Transfer Learning, RNNs Sentiment analysis, image classifier

Month 6 Transformers, Deployment, Capstone Fine-tune BERT, deploy as REST API

Curated Resources by Level

• Beginner: [Link] official tutorial · Kaggle Learn (free courses) · [Link] Practical DL Part 1

• Intermediate: Hands-On ML with Scikit-Learn, Keras & TensorFlow (Géron) · Deep Learning (Goodfellow
et al.)

• Practice: Kaggle competitions · UCI ML Repository · HuggingFace Datasets

• Papers: [Link] · Papers With Code (code + benchmarks)

• Courses: [Link] specializations · Stanford CS229 (YouTube) · [Link]

The best time to start was yesterday. The second best time is now.
Open a Jupyter notebook, type your first line of Python, and begin.