ML Lab Code Explained | ESC S203 | Non-AI/ML Student Guide

ML Lab Code — Plain English Guide

ESC S203 | For Non-AI/ML Students

This document explains every experiment from your ML Lab File in simple, everyday language
— no prior knowledge of AI or machine learning needed. Each section breaks down what the
code does, why it does it, and uses real-life analogies to make concepts stick.

Exp 01 K-Nearest Neighbors Find similar items by measuring distance — like asking
(KNN) your nearest neighbours for advice

Exp 02 Linear Regression Draw the best straight line through data to predict
future values

Exp 03 Decision Tree A flowchart of yes/no questions that leads to an

Classification answer

Exp 04 Regression (Advanced) Predict numbers using curved lines and tree-based
splitting

Exp 05 Reinforcement Learning Train an agent to navigate a grid by rewarding good

moves

Page 1 of 13
ML Lab Code Explained | ESC S203 | Non-AI/ML Student Guide

Understanding the Import Lines

Every Python program starts by importing 'libraries' — these are pre-written toolkits made by experts
that you can use for free. Think of them like apps on your phone: you don't build WhatsApp from
scratch, you just install and use it.

Code / Term What it means (plain English)

import numpy as np NumPy — a maths toolkit. Lets you work with large lists
of numbers very fast. 'as np' is just a short nickname.
import [Link] Matplotlib — a chart/graph drawing toolkit. Like Excel
charts but from code.
from sklearn import ... Scikit-learn — THE most popular ML toolkit. Has ready-
made algorithms (KNN, Trees, etc.) built in.
import seaborn as sns Seaborn — makes prettier statistical charts on top of
Matplotlib.
import gym OpenAI Gym — provides ready-made game
environments to train AI agents in.

Experiment 01 — K-Nearest Neighbors (KNN)

The Big Idea — What Is KNN?

💡 Real-Life Analogy: Imagine you move to a new city and want to know if your new
neighbourhood is 'safe'. You ask the 5 nearest houses. If 4 out of 5 say 'yes, it is safe',
you conclude it is safe too. KNN works exactly like this — it looks at the K nearest data
points and takes a vote.

KNN is used to classify (categorise) new data by looking at its K closest neighbours in the training data
and using majority voting. There is no real 'training' — it just memorises all the data and compares at
prediction time.

Step-by-Step Code Explanation

Step 1 — Loading the Dataset
iris = load_iris()
X, y = [Link], [Link]

📝 Note: The Iris dataset contains measurements of 150 flowers (petal length, petal width,
sepal length, sepal width) from 3 species. X = the measurements (features). y = the
species label (what we want to predict).

Page 2 of 13
ML Lab Code Explained | ESC S203 | Non-AI/ML Student Guide

Step 2 — Splitting into Train and Test

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3,
random_state=42)

Code / Term What it means (plain English)

train_test_split Shuffles and divides the data. Like splitting a deck of
cards into two piles.
test_size=0.3 30% of data goes to testing, 70% goes to training. The
model never sees test data during learning.
random_state=42 A fixed 'seed' so the shuffle is the same every time you
run it. Just a reproducibility trick.

Step 3 — Feature Scaling

scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = [Link](X_test)

💡 Real-Life Analogy: Suppose one feature is 'height in cm' (160–180) and another is
'weight in kg' (60–80). KNN uses distance — a 1 cm height difference would unfairly
dominate a 1 kg weight difference. Scaling brings everything to the same scale (like
converting all to percentages), so all features are treated fairly.

Step 4 — KNN Written From Scratch

class KNNScratch:
def fit(self, X, y): # 'fit' just stores the training data
self.X_train = X
self.y_train = y

def euclidean(self, a, b): # Measures straight-line distance between 2

points
return [Link]([Link]((a - b) ** 2))

def _pred_one(self, x): # For one test point:

dists = [...] # Calculate distance to every training
point
k_idx = [Link](dists)[:self.k] # Pick the K closest ones
k_labels = self.y_train[k_idx] # Get their labels
return [Link](k_labels).argmax() # Return the majority label

Code / Term What it means (plain English)

euclidean distance The straight-line distance between two points, like
measuring with a ruler on a graph.
[Link](dists)[:k] Sort distances smallest first, then take only the first K
indices (the K nearest neighbours).
[Link]().argmax() Count how many votes each class got, return the class
with the most votes.

Page 3 of 13
ML Lab Code Explained | ESC S203 | Non-AI/ML Student Guide

Step 5 — Finding the Best K

for k in range(1, 21):
model = KNeighborsClassifier(n_neighbors=k)
[Link](X_train, y_train)
acc = accuracy_score(y_test, [Link](X_test))

📝 Note: We try every value of K from 1 to 20 and measure accuracy each time. Too small
a K (like K=1) memorises noise. Too large a K ignores local patterns. We pick the K with
the highest test accuracy — in this case K=5.

✅ Key Takeaway: KNN is one of the simplest ML algorithms. It has no equations to learn
— it just memorises data and compares distances. Best accuracy on Iris dataset:
~97.78% at K=5.

Page 4 of 13
ML Lab Code Explained | ESC S203 | Non-AI/ML Student Guide

Experiment 02 — Linear Regression

The Big Idea — What Is Linear Regression?

💡 Real-Life Analogy: You notice that as you study more hours, your exam score
improves. If you plot Hours vs Score on a graph, you can draw a straight line through the
points. Linear Regression finds the BEST such straight line so you can predict scores for
any number of study hours — even ones you haven't tried yet.

Linear Regression is used to predict a number (not a category). The formula is: y = b0 + b1×x, where
b0 is where the line crosses the y-axis (intercept) and b1 is the slope (how steep the line is).

Step-by-Step Code Explanation

Step 1 — Creating Synthetic (Fake) Data
X_simple = 2 * [Link](100, 1)
y_simple = 4 + 3 * X_simple + [Link](100, 1)*2

Code / Term What it means (plain English)

[Link](100,1) Generate 100 random numbers between 0 and 1. These
are our 'x' values.
4 + 3 * X_simple The true relationship: y = 4 + 3x. We're pretending we
know this, then trying to rediscover it.
[Link]()*2 Add random 'noise' — because real-world data is never
perfectly on a line.

Step 2 — Linear Regression From Scratch (Maths)

X_b = np.c_[[Link]((len(X), 1)), X] # Add a column of 1s (for intercept
b0)
[Link] = [Link](X_b.T @ X_b) @ X_b.T @ y # Closed-form solution

📝 Note: This scary-looking formula is called the 'Normal Equation'. It mathematically

calculates the exact best-fit line in one shot — no guessing needed. [Link] finds the
matrix inverse, @ is matrix multiplication. You don't need to memorise this — just know it
gives us b0 and b1 directly.

Step 3 — Scikit-learn Does It in 2 Lines

lr = LinearRegression()
[Link](X_train_s, y_train)
y_pred = [Link](X_test_s)

💡 Real-Life Analogy: The scratch version above was like building a calculator from

Page 5 of 13
ML Lab Code Explained | ESC S203 | Non-AI/ML Student Guide

transistors. Scikit-learn is like just buying a calculator. Same result, much less effort — we
use scratch only to understand what is happening inside.

Step 4 — Evaluating the Model

Code / Term What it means (plain English)
MSE (Mean Squared Error) Average of (predicted - actual)² for all test points.
Penalises big mistakes heavily. Lower is better.
RMSE Square root of MSE — gives error in the same units as
your target variable. Easier to interpret.
MAE (Mean Absolute Error) Average of |predicted - actual|. Treats all errors equally.
Lower is better.
R² (R-squared) How much of the variation in y is explained by your
model. R²=1 is perfect, R²=0 means the model is useless.
Higher is better.

✅ Key Takeaway: Linear Regression is the foundation of all predictive modelling. Our
scratch model found intercept≈4.2 and slope≈2.98 — very close to the true values of 4
and 3. R²=0.90 means the model explains 90% of the variation in the data.

Page 6 of 13
ML Lab Code Explained | ESC S203 | Non-AI/ML Student Guide

Experiment 03 — Decision Tree Classification

The Big Idea — What Is a Decision Tree?

💡 Real-Life Analogy: Think of the 20-questions game. 'Is it an animal? → Does it have 4
legs? → Does it bark?' Each question splits the possibilities in half until you reach an
answer. A Decision Tree does exactly this — it learns which questions to ask (and in what
order) automatically from data.

A Decision Tree is a series of if-else questions arranged in a tree shape. Each internal node = a
question about a feature. Each branch = a possible answer. Each leaf = a final prediction (class label).

Step-by-Step Code Explanation

Step 1 — The Dataset (Breast Cancer)
data = load_breast_cancer()
X, y = [Link], [Link] # 569 patients, 30 features each

📝 Note: The Breast Cancer dataset has measurements from tumour scans (radius,
texture, perimeter, etc.) for 569 patients. y = 0 means malignant (cancerous), y = 1 means
benign (not cancerous). The model learns to classify new tumours.

Step 2 — The Gini Impurity Concept

dt = DecisionTreeClassifier(max_depth=d, criterion='gini')

💡 Real-Life Analogy: Imagine a bag of marbles. If ALL marbles are red, the bag is 'pure'
(Gini = 0). If it's 50% red and 50% blue, it's maximally 'impure' (Gini = 0.5). The tree picks
questions that most reduce impurity — like sorting mixed marbles into purer bags at each
step.
Code / Term What it means (plain English)
criterion='gini' Use Gini Impurity to decide which feature to split on at
each node.
max_depth=d How many levels of questions the tree can ask. Deep tree
= more specific but may over-memorise.

Step 3 — Overfitting vs Underfitting

for d in range(1, 16):
dt = DecisionTreeClassifier(max_depth=d)
train_acc.append(...) # accuracy on data it saw
test_acc.append(...) # accuracy on new, unseen data

Page 7 of 13
ML Lab Code Explained | ESC S203 | Non-AI/ML Student Guide

📝 Note: When max_depth is very large, the tree memorises the training data perfectly
(train accuracy = 100%) but fails on new data — this is called OVERFITTING (like
memorising answers without understanding). Too shallow = UNDERFITTING (too simple).
We pick the depth where test accuracy is highest — in this case depth=5.

Step 4 — Cross Validation

cv_scores = cross_val_score(dt_best, X, y, cv=5)

💡 Real-Life Analogy: Instead of testing on just one fixed test set, we split the data into 5
parts. We train on 4 parts and test on the 5th, repeating 5 times using a different test part
each time. The average of 5 results is much more reliable than one single test. Like taking
5 mock exams instead of one to estimate your real exam performance.

Step 5 — Feature Importance

feat_imp = dt_best.feature_importances_

📝 Note: After training, the tree tells us which features (questions) were most useful for
making decisions. 'worst concave points' scored 0.58 — it was used in 58% of the
decision-making. Features with importance near 0 are basically useless for prediction.

✅ Key Takeaway: Decision Trees are powerful and easy to explain to humans. Our tree
hit 94.41% accuracy with just 5 levels of questions on cancer data. The confusion matrix
showed only 1-2 misclassifications per class.

Page 8 of 13
ML Lab Code Explained | ESC S203 | Non-AI/ML Student Guide

Experiment 04 — Regression (DT Regressor & Polynomial)

The Big Idea — Predicting Numbers, Not Categories

Experiments 1 and 3 classified data into groups (species, malignant/benign). Regression predicts
actual numbers. This experiment has two parts: (1) Polynomial Regression — fit a curved line to data.
(2) Decision Tree Regressor — split data into regions and predict the average value in each region.

Part A — Polynomial Regression

💡 Real-Life Analogy: Linear Regression draws a straight line. But what if the true
relationship is curved — like a ball thrown in the air (height follows a curve, not a line)?
Polynomial Regression bends the line by adding x², x³, etc. as extra features. Degree 1 =
straight line. Degree 2 = parabola. Degree 3 = S-curve.

Code: Creating a Polynomial Pipeline

pipe = Pipeline([
('poly', PolynomialFeatures(degree=deg)), # Create x, x², x³...
('scaler', StandardScaler()), # Scale the new features
('lr', LinearRegression()), # Fit a line through them
])

Code / Term What it means (plain English)

Pipeline Chains multiple steps together so they run in sequence
automatically.
PolynomialFeatures(d=3) Takes one column x and creates x, x², x³. Now the model
can fit curves.
degree How bendy the curve is. Degree 1=line, 2=parabola,
3=cubic curve. Degree 6+ usually overfits.

📝 Note: Our synthetic data was generated with a cubic formula (x³). So degree=3 gave
the best R²=0.92. Degree=6 actually did WORSE (R²=0.90) because it started fitting the
noise — this is overfitting.

Part B — Decision Tree Regressor

💡 Real-Life Analogy: Imagine dividing all houses in a city into regions by

neighbourhood. You predict the price of any house as the AVERAGE price of all houses in
its region. The Decision Tree Regressor does this automatically — it finds the regions
(splits) that minimise prediction error.

Page 9 of 13
ML Lab Code Explained | ESC S203 | Non-AI/ML Student Guide

Code: DT Regressor on Housing Data

dtr = DecisionTreeRegressor(max_depth=d, random_state=42)
[Link](X_train, y_train)
mse = mean_squared_error(y_test, [Link](X_test))

📝 Note: Same concept as the classification tree in Exp 03, but instead of voting for a
class, the leaf predicts the MEAN of all training values in that region. Best depth=7
achieved RMSE=0.69 and R²=0.63 on the California Housing dataset.

✅ Key Takeaway: For complex curved relationships, use Polynomial Regression. For
non-linear data with many features, Decision Tree Regressor often works better. Both
demonstrate the universal bias-variance trade-off: more complexity helps until it starts
hurting.

Page 10 of 13
ML Lab Code Explained | ESC S203 | Non-AI/ML Student Guide

Experiment 05 — Reinforcement Learning (Q-Learning)

The Big Idea — Learning by Trying

💡 Real-Life Analogy: Think of teaching a dog new tricks. You don't explain the trick in
words — you just give it a treat (reward) when it does the right thing and ignore it (or say
'no') when it doesn't. Over time, the dog learns which actions lead to treats. Q-Learning
trains an AI agent the same way — by giving +1 for reaching the goal and 0 for falling in a
hole.

Reinforcement Learning is completely different from the previous experiments. There is no labelled
dataset. Instead, an Agent interacts with an Environment, receives Rewards, and learns a Policy (a
strategy) that maximises total reward over time.

The FrozenLake Environment

env = [Link]('FrozenLake-v1', is_slippery=False)
# 4x4 grid: S=Start F=Frozen(safe) H=Hole(game over) G=Goal
# S F F F
# F H F H
# F F F H
# H F F G

📝 Note: The agent starts at S (top-left) and must reach G (bottom-right) without falling
into H (holes). Actions: 0=Left, 1=Down, 2=Right, 3=Up. Reward: +1 if it reaches G, 0
otherwise. is_slippery=False means actions always work as intended (deterministic).

The Q-Table — The Agent's Brain

Q = [Link]((n_states, n_actions)) # 16 states x 4 actions = 64 values

💡 Real-Life Analogy: The Q-table is like a cheat sheet. It has one row per location on
the grid (16 states) and one column per possible action (4 actions). Each cell Q[state,
action] stores the 'quality score' — how good it is to take that action from that state.
Initially all zeros (the agent knows nothing). It fills in as the agent explores.

Step-by-Step Code Explanation

The Bellman Update (The Learning Formula)
Q[state, action] += ALPHA * (
reward + GAMMA * [Link](Q[next_state]) - Q[state, action]
)

Code / Term What it means (plain English)

Page 11 of 13
ML Lab Code Explained | ESC S203 | Non-AI/ML Student Guide

ALPHA (0.8) Learning rate — how fast to update. 0=learn nothing,

1=forget everything old. 0.8 = learn aggressively.
GAMMA (0.95) Discount factor — how much to value FUTURE rewards
vs immediate ones. 0.95 means future rewards are
almost as valuable as now.
reward Immediate reward received after this action (+1 if goal
reached, 0 otherwise).
[Link](Q[next_state]) Best possible future reward from the next state —
encourages the agent to plan ahead.
Q[s,a] += ALPHA*(...) Blend the old estimate with the new experience — a
weighted moving average.

Epsilon-Greedy Exploration
if [Link]() < EPSILON:
action = env.action_space.sample() # Explore: try a RANDOM action
else:
action = [Link](Q[state]) # Exploit: use the BEST known action

EPSILON = max(EPSILON_MIN, EPSILON * EPSILON_DECAY) # Reduce randomness over

time

💡 Real-Life Analogy: Early on, the agent explores randomly (like a toddler wandering
everywhere). As it learns more, it exploits its knowledge more (like an adult who knows
the best route to work). EPSILON starts at 1.0 (100% random) and decays to 0.01 (99%
uses best known action) by the end of training.

The Learned Policy

# After 2000 episodes, the agent learned:
[['>' 'v' '>' 'v']
['>' '>' 'v' 'v']
['>' '>' 'v' 'H']
['H' '>' '>' 'G']]
# < = go left, > = go right, v = go down, ^ = go up

📝 Note: The arrows show the best action in each grid cell. Reading the path from S (top-
left): go right → go down → go down → go right → go right → GOAL! The agent learned
to navigate safely around all holes without ever being told the rules — purely from trial and
reward.

✅ Key Takeaway: After 2000 training episodes, the agent achieves a 98% success rate
in the last 100 episodes. Reinforcement Learning is used in real applications like teaching
robots to walk, playing video games (AlphaGo/AlphaZero), and autonomous driving.

Page 12 of 13
ML Lab Code Explained | ESC S203 | Non-AI/ML Student Guide

Quick Glossary — ML Terms in Plain English

Code / Term What it means (plain English)

Dataset A table of data used for training/testing. Rows = samples,
Columns = features.
Feature (X) An input variable / measurement used to make a
prediction (e.g., height, weight).
Label / Target (y) The output variable you want to predict (e.g., species,
price, class).
Training Showing the model labelled examples so it can learn
patterns.
Testing Evaluating the model on data it has NEVER seen, to
check real-world performance.
Accuracy % of predictions that were correct. 0.97 = 97% correct.
Overfitting Model memorises training data too well, fails on new
data. Like mugging answers without understanding.
Underfitting Model is too simple to capture the pattern. Like guessing
everything as the majority class.
Epoch / Episode One complete pass through training data (supervised) or
one full game attempt (RL).
Hyperparameter Settings YOU choose before training, like K in KNN or
max_depth in trees.
random_state=42 A fixed seed for the random number generator — makes
results reproducible. 42 is just a tradition.
fit() The training step — the model learns from your training
data.
predict() The inference step — the trained model makes
predictions on new data.
Pipeline A chain of preprocessing + model steps that execute in
sequence.
Agent (RL) The AI learner that takes actions in an environment.
Reward (RL) A numerical score the agent receives after each action
(+1 = good, 0/-1 = bad).

Page 13 of 13