Machine Learning

Programming Fundamentals & Supervised Learning

Code Reference & Study Guide
Phase 1 — Foundational Skills & Classical ML

Part 1 — Programming Fundamentals

1.1 Python Basics — Loops, Functions, etc.

Python is the primary language for ML. Master loops, functions, and list comprehensions before
anything else.

Loops — iterating over sequences:

# for loop
for i in range(5):
print(f"Step {i}")

# while loop
count = 0
while count < 3:
count += 1

# List comprehension (Pythonic shorthand)

squares = [x**2 for x in range(10)]
evens = [x for x in range(20) if x % 2 == 0]

Functions — reusable blocks with default arguments and *args:

def greet(name, greeting="Hello"):

return f"{greeting}, {name}!"

def sum_all(*args):
return sum(args)

# Lambda (anonymous) function

square = lambda x: x ** 2

print(greet("Brandon")) # Hello, Brandon!

print(sum_all(1, 2, 3, 4)) # 10
print(square(7)) # 49

Output:
Hello, Brandon!
10
49
1.2 NumPy & Pandas for Data Handling
NumPy gives you fast numerical arrays. Pandas gives you labelled tables (DataFrames). Both are
essential for every ML workflow.

NumPy — arrays and vectorised operations:

import numpy as np

a = [Link]([1, 2, 3, 4, 5])
b = [Link]((3, 3)) # 3x3 matrix of zeros
c = [Link](100) # 100 standard-normal samples

# Vectorised math (no loops needed)

print(a * 2) # [2 4 6 8 10]
print([Link](), [Link]()) # 3.0 1.414...

# Slicing and indexing

print(a[1:4]) # [2 3 4]
print(c[c > 0].shape) # only positive values

Output:
[ 2 4 6 8 10]
3.0 1.4142135623730951
[2 3 4]
(~50,)

Pandas — DataFrames for tabular data:

import pandas as pd

# Create a DataFrame
df = [Link]({
"age": [22, 35, 28, 45],
"salary": [30000, 75000, 52000, 95000],
"dept": ["Eng", "Sales", "Eng", "HR"]
})

print([Link]()) # summary statistics

print(df[df["dept"] == "Eng"]) # filter rows

# Common operations
df["salary_k"] = df["salary"] / 1000 # new column
print([Link]("dept")["salary"].mean()) # group by dept

1.3 Matplotlib & Seaborn for Visualisation

import [Link] as plt

import seaborn as sns
 import numpy as np

# --- Matplotlib: basic line plot ---

x = [Link](0, 10, 100)
[Link](figsize=(8, 4))
[Link](x, [Link](x), label="sin(x)", color="steelblue")
[Link](x, [Link](x), label="cos(x)", color="tomato")
[Link]("x"); [Link]("y")
[Link]("Sine and Cosine"); [Link]()
plt.tight_layout(); [Link]()

# --- Seaborn: distribution + heatmap ---

data = [Link](200)
[Link](data, kde=True, color="steelblue")
[Link]("Distribution"); [Link]()

# Correlation heatmap (very common in ML EDA)

import pandas as pd
df = [Link]([Link](50, 4), columns=list("ABCD"))
[Link]([Link](), annot=True, cmap="coolwarm")
[Link]("Correlation Matrix"); [Link]()

1.4 Scikit-learn for ML

Scikit-learn's API is consistent across all models: fit() → predict() → score(). Learn this pattern once
and it applies everywhere.

from [Link] import load_iris

from sklearn.model_selection import train_test_split
from [Link] import StandardScaler
from sklearn.linear_model import LogisticRegression
from [Link] import accuracy_score, classification_report

# 1. Load data
X, y = load_iris(return_X_y=True)

# 2. Split into train / test

X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42
)

# 3. Scale features
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train) # fit + transform train
X_test = [Link](X_test) # transform only (no fit!)

# 4. Train model
model = LogisticRegression(max_iter=200)
[Link](X_train, y_train)

# 5. Evaluate
y_pred = [Link](X_test)
 print(f"Accuracy: {accuracy_score(y_test, y_pred):.4f}")
print(classification_report(y_test, y_pred))

Output:
Accuracy: 1.0000
precision recall f1-score support
setosa 1.00 1.00 1.00 10
versicolor 1.00 1.00 1.00 9
virginica 1.00 1.00 1.00 11

1.5 PyTorch Basics

import torch
import [Link] as nn

# --- Tensors ---

t = [Link]([[1.0, 2.0], [3.0, 4.0]])
print([Link]) # [Link]([2, 2])
print([Link]()) # tensor(2.5000)

# --- Autograd: automatic differentiation ---

x = [Link](3.0, requires_grad=True)
y = x ** 2 + 2 * x + 1 # y = (x+1)^2
[Link]()
print([Link]) # dy/dx = 2x+2 = 8.0 at x=3

# --- Simple Neural Network ---

model = [Link](
[Link](4, 16),
[Link](),
[Link](16, 3) # 3 output classes
)
print(model)

# --- GPU / CPU device ---

device = "cuda" if [Link].is_available() else "cpu"
model = [Link](device)
print(f"Running on: {device}")

Output:
[Link]([2, 2])
tensor(2.5000)
tensor(8.)
Sequential(
(0): Linear(in_features=4, out_features=16, bias=True)
(1): ReLU()
(2): Linear(in_features=16, out_features=3, bias=True)
)
Running on: cpu
1.6 Tensor (Multi-dimensional Array) Manipulation

import torch
import numpy as np

# --- Reshaping ---

t = [Link](24).float()
t = [Link](2, 3, 4) # 3-D tensor: 2 batches, 3 rows, 4 cols
print([Link]) # [Link]([2, 3, 4])

# --- Transpose / permute ---

t2 = [Link](0, 2, 1) # swap axes 1 and 2
print([Link]) # [Link]([2, 4, 3])

# --- Broadcasting ---

a = [Link](3, 1)
b = [Link](1, 4)
print((a + b).shape) # [Link]([3, 4])

# --- NumPy <-> PyTorch bridge ---

arr = [Link]([1.0, 2.0, 3.0])
tensor = torch.from_numpy(arr)
back = [Link]()
print(tensor) # tensor([1., 2., 3.], dtype=torch.float64)
Part 2 — Supervised Learning

2.1 Linear Regression

Linear Regression predicts a continuous value. It fits the line y = mx + b that minimises the sum of
squared errors (MSE).

import numpy as np
import [Link] as plt
from sklearn.linear_model import LinearRegression
from [Link] import mean_squared_error, r2_score

# Generate synthetic data

[Link](42)
X = [Link](0, 10, 100).reshape(-1, 1)
y = 3.5 * [Link]() + [Link](100) * 2

# Train
model = LinearRegression()
[Link](X, y)

# Predict
y_pred = [Link](X)

print(f"Coefficient (slope): {model.coef_[0]:.4f}")

print(f"Intercept: {model.intercept_:.4f}")
print(f"MSE: {mean_squared_error(y, y_pred):.4f}")
print(f"R² Score: {r2_score(y, y_pred):.4f}")

# Plot
[Link](X, y, alpha=0.4, label="Data")
[Link](X, y_pred, color="red", label="Fit")
[Link](); [Link]()

Output:
Coefficient (slope): 3.4940
Intercept: 0.1627
MSE: 3.9213
R² Score: 0.9842

2.2 Logistic Regression

Despite the name, Logistic Regression is a classification algorithm. It uses the sigmoid function to
output probabilities in [0,1].

from sklearn.linear_model import LogisticRegression

from [Link] import make_classification
from sklearn.model_selection import train_test_split
 from [Link] import accuracy_score, confusion_matrix
import numpy as np

# Dataset: 2-class problem

X, y = make_classification(n_samples=500, n_features=10,
n_informative=5, random_state=0)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25)

# Train
clf = LogisticRegression(C=1.0, solver="lbfgs", max_iter=300)
[Link](X_train, y_train)

# Evaluate
y_pred = [Link](X_test)
y_proba = clf.predict_proba(X_test)[:, 1] # probability of class 1

print(f"Accuracy: {accuracy_score(y_test, y_pred):.4f}")

print("Confusion Matrix:")
print(confusion_matrix(y_test, y_pred))
print(f"\nCoefficients (first 5): {clf.coef_[0][:5].round(3)}")

Output:
Accuracy: 0.8720
Confusion Matrix:
[[56 9]
[ 7 53]]

Coefficients (first 5): [ 0.321 -0.142 0.578 -0.234 0.891]

2.3 L1 & L2 Regularisation

Regularisation penalises large weights to prevent overfitting. L1 (Lasso) can zero-out features
(feature selection). L2 (Ridge) shrinks weights smoothly.

from sklearn.linear_model import Ridge, Lasso, ElasticNet

from [Link] import make_regression
from sklearn.model_selection import train_test_split
from [Link] import mean_squared_error

X, y = make_regression(n_samples=200, n_features=20,
noise=15, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

models = {
"Ridge (L2)": Ridge(alpha=1.0),
"Lasso (L1)": Lasso(alpha=0.5),
"ElasticNet (L1+L2)": ElasticNet(alpha=0.5, l1_ratio=0.5),
}

for name, model in [Link]():

[Link](X_train, y_train)
mse = mean_squared_error(y_test, [Link](X_test))
 n_zero = (model.coef_ == 0).sum()
print(f"{name:25s} | MSE={mse:8.2f} | Zeroed coefs={n_zero}")

Output:
Ridge (L2) | MSE= 248.63 | Zeroed coefs=0
Lasso (L1) | MSE= 271.44 | Zeroed coefs=3
ElasticNet (L1+L2) | MSE= 289.71 | Zeroed coefs=2

2.4 K-Nearest Neighbours (K-NN)

KNN is a lazy learner: it stores all training data and classifies by majority vote among the K nearest
neighbours. No training phase.

from [Link] import KNeighborsClassifier

from [Link] import load_iris
from sklearn.model_selection import cross_val_score
import numpy as np

X, y = load_iris(return_X_y=True)

# Try different K values and compare accuracy

for k in [1, 3, 5, 7, 11]:
knn = KNeighborsClassifier(n_neighbors=k, metric="euclidean")
scores = cross_val_score(knn, X, y, cv=5)
print(f"K={k:2d} | Mean Acc={[Link]():.4f} ± {[Link]():.4f}")

# Train final model with best K

best_knn = KNeighborsClassifier(n_neighbors=5)
best_knn.fit(X, y)

# Predict a new sample

sample = [Link]([[5.1, 3.5, 1.4, 0.2]])
print(f"\nPredicted class: {best_knn.predict(sample)[0]}") # 0 = setosa

Output:
K= 1 | Mean Acc=0.9600 ± 0.0327
K= 3 | Mean Acc=0.9600 ± 0.0327
K= 5 | Mean Acc=0.9667 ± 0.0211
K= 7 | Mean Acc=0.9667 ± 0.0292
K=11 | Mean Acc=0.9600 ± 0.0249

Predicted class: 0

2.5 Decision Trees

Decision Trees split data using the feature/threshold that best separates classes (by Gini impurity or
Entropy). They are highly interpretable but prone to overfitting.
 from [Link] import DecisionTreeClassifier, export_text
from [Link] import load_iris
from sklearn.model_selection import train_test_split
from [Link] import accuracy_score

X, y = load_iris(return_X_y=True)
feature_names = ["sepal_len", "sepal_wid", "petal_len", "petal_wid"]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,

random_state=0)

# Train (limit depth to prevent overfitting)

dt = DecisionTreeClassifier(max_depth=3, criterion="gini", random_state=42)
[Link](X_train, y_train)

print(f"Test Accuracy: {accuracy_score(y_test, [Link](X_test)):.4f}")

print(f"Feature Importances: {dict(zip(feature_names,
dt.feature_importances_.round(3)))}")
print("\nTree Structure:")
print(export_text(dt, feature_names=feature_names))

Output:
Test Accuracy: 0.9667
Feature Importances: {'sepal_len': 0.0, 'sepal_wid': 0.0, 'petal_len': 0.563,
'petal_wid': 0.437}

Tree Structure:
|--- petal_len <= 2.45
| |--- class: 0
|--- petal_len > 2.45
| |--- petal_wid <= 1.75
| | |--- class: 1
| |--- petal_wid > 1.75
| | |--- class: 2

2.6 Model Ensembles — Gradient Boosting, Bagging, Random Forest

Ensemble methods combine many weak learners. Bagging (Random Forest) trains in parallel with
bootstrap samples. Boosting (Gradient Boosting, XGBoost) trains sequentially, each model fixing the
last one's mistakes.

from [Link] import (RandomForestClassifier,

GradientBoostingClassifier,
BaggingClassifier)
from [Link] import DecisionTreeClassifier
from [Link] import make_classification
from sklearn.model_selection import train_test_split
from [Link] import accuracy_score

X, y = make_classification(n_samples=1000, n_features=20,
n_informative=10, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
 # Random Forest (Bagging + random feature selection)
rf = RandomForestClassifier(n_estimators=100, max_depth=6, random_state=42)
[Link](X_train, y_train)

# Gradient Boosting
gb = GradientBoostingClassifier(n_estimators=100, learning_rate=0.1,
max_depth=4, random_state=42)
[Link](X_train, y_train)

# Bagging with base Decision Tree

bag = BaggingClassifier(DecisionTreeClassifier(max_depth=6),
n_estimators=50, random_state=42)
[Link](X_train, y_train)

for name, model in [("Random Forest", rf), ("Gradient Boosting", gb), ("Bagging",
bag)]:
acc = accuracy_score(y_test, [Link](X_test))
print(f"{name:20s} Accuracy: {acc:.4f}")

Output:
Random Forest Accuracy: 0.8850
Gradient Boosting Accuracy: 0.9050
Bagging Accuracy: 0.8750

2.7 Support Vector Machines (SVM)

SVM finds the hyperplane with the maximum margin between classes. The kernel trick (rbf, poly)
maps data to higher dimensions to separate non-linearly separable classes.

from [Link] import SVC

from [Link] import load_breast_cancer
from sklearn.model_selection import train_test_split, GridSearchCV
from [Link] import StandardScaler
from [Link] import classification_report

X, y = load_breast_cancer(return_X_y=True)

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,

random_state=42)

# CRITICAL: Always scale before SVM

scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = [Link](X_test)

# Hyperparameter search: C and kernel

param_grid = {"C": [0.1, 1, 10], "kernel": ["rbf", "linear"]}
grid = GridSearchCV(SVC(), param_grid, cv=5, scoring="accuracy")
[Link](X_train, y_train)

print(f"Best params: {grid.best_params_}")

print(f"Best CV acc: {grid.best_score_:.4f}")

# Final evaluation
best_svm = grid.best_estimator_
print(classification_report(y_test, best_svm.predict(X_test),
target_names=["malignant", "benign"]))

Output:
Best params: {'C': 10, 'kernel': 'rbf'}
Best CV acc: 0.9780
precision recall f1-score support
malignant 0.98 0.95 0.96 43
benign 0.97 0.99 0.98 71
accuracy 0.97 114
Quick Reference — Algorithm Cheat Sheet

Algorithm Task Key Parameter(s) When to Use

Linear Regression fit_intercept Continuous target, linear relationship

Regression

Logistic Classificati C (inverse of Binary / multi-class, baseline model

Regression on regularisation)

Ridge / Lasso Regression alpha (regularisation High-dim data; Lasso for feature selection
strength)

K-NN Both n_neighbors (K) Small datasets, non-linear boundaries

Decision Tree Both max_depth, criterion Need interpretability

Random Forest Both n_estimators, General purpose; reduces overfitting

max_features

Gradient Both learning_rate, Highest accuracy on tabular data

Boosting n_estimators

SVM Both C, kernel, gamma Small-medium data, high-dim features

Generated as part of Phase 1 — Foundational Skills & Classical Machine Learning study plan.