MODULE - 4

1. Introduction to AI Model Building

When we build an AI (machine learning) model, we teach a program to find patterns
in past data and then use those patterns to make predictions on new, unseen data.
Analogy – Teaching a student:
• Give the student many example problems with correct answers (training data).
• The student learns the underlying rules (the model).
• Later, test with completely new questions (testing data) to see if real learning has
occurred.
In code, the “student” is a mathematical function (linear regression, decision tree, neural
network, etc.). “Learning” means adjusting internal parameters to minimise the difference
between predictions and true answers.

2. Training Data & Testing Data – The Two Pillars

2.1 Training Data – The Classroom Where Learning Happens
Training data is the labelled dataset that the model uses to discover patterns. It consists of
two parts:
• Features (input variables): The signals the model can observe. In a house-price
prediction task, these could be square_meters, number_of_bedrooms, age_of_house,
and distance_to_city_centre.
• Labels / Targets (output variables): The correct answer for each example. Here, that’s
the actual sale_price.
The model sees both features and labels during training. It tries to find a mathematical
function f such that f(features) ≈ label for as many training examples as possible. This is done
by minimising a loss function – a number that measures how far the model’s predictions are
from the true labels.
Concrete example
Imagine a tiny training set of five houses:

Square meters Bedrooms Age (years) Sale Price (€)

50 1 20 150,000

80 2 5 250,000

120 3 1 380,000

60 1 30 130,000

100 2 10 300,000
A simple linear regression model might learn that 𝒑𝒓𝒊𝒄𝒆 ≈ 𝟐𝟎𝟎𝟎 · 𝒔𝒒𝒎 + 𝟑𝟎𝟎𝟎𝟎 ·
𝒃𝒆𝒅𝒓𝒐𝒐𝒎𝒔 − 𝟏𝟓𝟎𝟎 · 𝒂𝒈𝒆 + 𝟓𝟎𝟎𝟎𝟎. During training, the algorithm adjusts those numbers
(the model’s parameters) so that on the training data the predictions are as close as possible to
the actual sale prices.
Analogy – The model is a student in a classroom. The training data is a workbook that
contains both the questions and the final answers. The student (model) studies the workbook,
tries to find the rules that connect the questions to the answers, and only when the rules start
working well on those solved problems is the lesson considered finished.

2.2 Testing Data

• A completely separate dataset that the model has never seen during training.
• Also has features and labels, but the labels are used only to evaluate the model’s
performance – not to train.
• Simulates how the model will behave in the real world on new data.
• If the model performs well on testing data, it has truly generalised and not just
memorised.

2.3 Why Not Test on Training Data?

If you evaluate your model on the same data it was trained on, you are giving the student
the exam with exactly the same questions and answers it has already memorised. It would score
100%, but the moment a slightly different question appears in the real world, it fails completely.
This phenomenon is called overfitting. Overfitting means the model learned the noise and
peculiarities of the training set instead of the underlying true relationship. It becomes a “savant”
on the exact training examples but useless on anything new.
2.4 Difference Between Training And Testing Data:-

3. Data Splitting Techniques

Splitting data is not a one-size-fits-all operation. The right method depends on the size
of your dataset, the nature of your problem (classification vs. regression, balanced vs.
imbalanced), and whether time is a factor. Below we explore the five essential techniques,
building from the simplest to the most specialised.
3.1 Simple Hold-Out (Train-Test Split)
This is the most straightforward method: you randomly partition the dataset into two
disjoint subsets, typically an 80% training set and a 20% test set (though 70/30 is also common).
How it works
1. Shuffle the entire dataset randomly.
2. Pick the first 80% of rows as training, the remaining 20% as test.
3. Train the model on the training set; evaluate on the test set.

3.2 Train-Validation-Test Split

When you need to tune hyperparameters (settings that are not learned directly from
the data, like the depth of a decision tree or the learning rate of a gradient descent), you cannot
use the test set for that tuning – doing so would leak information and give an over-optimistic
estimate. You need a third, intermediate set: the validation set.
Three roles
• Training set – used to learn the model’s parameters (weights, coefficients).
• Validation set – used to compare different models or hyperparameters and choose the
best one.
• Test set – used once, at the very end, to report the final, unbiased performance of the
chosen model.
Typical ratios: 60% train, 20% validation, 20% test (or 70/15/15, depending on data size).
3.3 K-Fold Cross-Validation
K-Fold Cross-Validation (CV) solves the instability problem of a single train-test split,
especially when data is limited. Instead of one fixed partition, you rotate the test set across
multiple folds.
How it works (k = 5 example)
1. Shuffle the dataset.
2. Split it into 5 equal-sized “folds” (groups).
3. Perform 5 iterations:
• Iteration 1: Fold 1 = test set, Folds 2-5 = training set.
• Iteration 2: Fold 2 = test set, Folds 1+3+4+5 = training set.
• …
• Iteration 5: Fold 5 = test set, Folds 1-4 = training set.
4. For each iteration, train a fresh model and record the evaluation metric (e.g., accuracy,
RMSE).
5. Report the average of the 5 metrics.

Pros
• Every data point gets to be in the test set exactly once, and in the training set k-1 times.
No information is wasted.
• The variance of the performance estimate drops significantly.
• Helps detect overfitting: if the metric varies wildly across folds, your model is sensitive
to the specific data it sees.
Cons
• Requires training k models, so it’s computationally k times more expensive.
• Not straightforward for time-series data (where order matters).
Common choices: k=5 or k=10. For very small datasets, one can even use Leave-One-Out
Cross-Validation (k = number of samples), but that is extremely expensive.
3.4 Stratified Splitting
When dealing with classification problems where the classes are imbalanced, a random
split can accidentally leave one class severely under-represented (or even absent) in the train
or test set. Stratified splitting preserves the class proportions.
The problem
Imagine a medical dataset with 1,000 patients, only 50 of whom have a rare disease
(5%). A random 80/20 split might, by bad luck, put only 2 of those 50 in the test set. Then your
test set has just 2 positive cases, making the evaluation of recall completely unreliable.
How stratified splitting works
1. Separate the dataset by class label.
2. For each class, randomly split its examples into train and test using the desired ratio
(e.g., 80/20).
3. Combine the per-class train portions to form the final training set; combine the test
portions to form the final test set.
3.5 Time-Series Split
When observations have a natural temporal order (stock prices, weather, sales over
time), you cannot randomly shuffle the dataset. Doing so would allow the model to train on
future data and test on the past – a blatant form of data leakage that would give absurdly
optimistic performance.
The rule: Training data must always come from before the test data.
How time-series splitting works:
A common technique is the expanding window (or rolling window) approach:
1. Sort the data by time (oldest first).
2. Pick a starting training size. For example, train on the first 12 months, test on the next
1 month.
3. Slide the window forward: train on first 13 months, test on month 14, and so on.
4. At each step, you train a model and evaluate on the subsequent time step(s). Average
the errors over all steps.
Key point: No random shuffling. The temporal order is sacred.
For a Java classroom, the most practical techniques to implement are hold-out
train-test split and stratified split.

4. Train-Test Split in Java – Full Implementation

Implementing data splitting from scratch in Java is one of the best ways to truly
understand how train-test separation works under the hood. In this section we build complete,
reusable implementations for simple hold-out, stratified, and k-fold splits using only
standard Java libraries. Every step is explained, and we provide visual diagrams to show the
flow of data.
We assume the data is stored in arrays:
• Feature matrix: double[][] X with shape N × D (N samples, D features).
• Labels: double[] y for regression, or int[] y for classification.
• The split functions will copy rows, ensuring the original arrays remain untouched.
4.1 Creating a Sample Dataset
To make the examples concrete, we define a tiny dataset of 10 students. The single
feature is hours studied, and the target is exam score.
package sampledata;
public class SampleData {
public static void main(String[] args) {
double[][] X = {
{1.0}, {2.0}, {3.0}, {4.0}, {5.0},
{6.0}, {7.0}, {8.0}, {9.0}, {10.0} };
double[] y = {50, 55, 65, 70, 72, 78, 80, 85, 88, 92};
[Link]("Dataset loaded successfully");
for (int i = 0; i < [Link]; i++) {
[Link]("Hours Studied: " + X[i][0]
+ " Score: " + y[i]);
} }}
Output:

4.2 Simple Random Train-Test Split

The simplest and most common method. The goal is to randomly assign each sample
to either the training set or the test set, guaranteeing no overlap and keeping the desired
proportion.
Algorithm Walk-through
1. Create an index array [0, 1, ..., N-1] representing the positions of all samples.
2. Shuffle the indices using a reproducible random permutation (Fisher–Yates shuffle).
3. Determine the split point:
𝑡𝑒𝑠𝑡𝑆𝑖𝑧𝑒 = (𝑖𝑛𝑡)(𝑁 ∗ 𝑡𝑒𝑠𝑡𝑅𝑎𝑡𝑖𝑜)
𝑡𝑟𝑎𝑖𝑛𝑆𝑖𝑧𝑒 = 𝑁 − 𝑡𝑒𝑠𝑡𝑆𝑖𝑧𝑒
4. Assign the first trainSize indices to the training set, the rest to the test set.
5. Build the actual X_train, y_train, X_test, y_test by copying the rows corresponding to
those indices.
This process guarantees a random partition that exactly respects the requested ratio
(modulo rounding).
 We introduce a helper class SplitResult to hold the four arrays. The split method
encapsulates the entire logic.

[Link]
import [Link];
import [Link];
import [Link];

public class ReadCSV {

public static void main(String[] args) {

String filePath =
"C:\\Users\\Lenovo\\OneDrive\\Documents\\NetBeansProjects\\TrainTestSplitDemo\\src\\dat
aset\\student_performance_dataset.csv";
// or full path like C:\\Users\\User\\Desktop\\[Link]

String line;
String csvSplitBy = ",";

try (BufferedReader br = new BufferedReader(new FileReader(filePath))) {

// Read header
String header = [Link]();
 [Link]("CSV HEADER: " + header);
[Link]("--------------------------------------");

// Read data rows

while ((line = [Link]()) != null) {

String[] data = [Link](csvSplitBy);

int studentId = [Link](data[0]);

int age = [Link](data[1]);
double studyHours = [Link](data[2]);
double attendance = [Link](data[3]);
double previousScore = [Link](data[4]);
int assignments = [Link](data[5]);
double finalMarks = [Link](data[6]);

[Link](
studentId + " | " +
age + " | " +
studyHours + " | " +
attendance + " | " +
previousScore + " | " +
assignments + " | " +
finalMarks
);
}

} catch (IOException e) {
[Link]();
}
}
}

[Link]
import [Link].*;
import [Link].*;

public class TrainTestSplitDemo {

public static class SplitResult {

public final double[][] X_train;
public final double[][] X_test;
public final double[] y_train;
 public final double[] y_test;

public SplitResult(double[][] X_train, double[][] X_test,

double[] y_train, double[] y_test) {
this.X_train = X_train;
this.X_test = X_test;
this.y_train = y_train;
this.y_test = y_test;
}
}

// ================= SPLIT FUNCTION =================

public static SplitResult split(double[][] X, double[] y,
double testRatio, long seed) {

int N = [Link];
Random rand = new Random(seed);

Integer[] indices = new Integer[N];

for (int i = 0; i < N; i++) indices[i] = i;

for (int i = N - 1; i > 0; i--) {

int j = [Link](i + 1);
int temp = indices[i];
indices[i] = indices[j];
indices[j] = temp;
}

int testSize = (int) (N * testRatio);

int trainSize = N - testSize;

double[][] X_train = new double[trainSize][];

double[] y_train = new double[trainSize];

for (int i = 0; i < trainSize; i++) {

int idx = indices[i];
X_train[i] = X[idx];
y_train[i] = y[idx];
}

double[][] X_test = new double[testSize][];

double[] y_test = new double[testSize];

for (int i = 0; i < testSize; i++) {

 int idx = indices[trainSize + i];
X_test[i] = X[idx];
y_test[i] = y[idx];
}

return new SplitResult(X_train, X_test, y_train, y_test);

}

// ================= CSV LOADER =================

public static void main(String[] args) {

String filePath =
"C:\\Users\\Lenovo\\OneDrive\\Documents\\NetBeansProjects\\TrainTestSplitDemo\\src\\dat
aset\\student_performance_dataset.csv";
List<double[]> XList = new ArrayList<>();
List<Double> yList = new ArrayList<>();

try (BufferedReader br = new BufferedReader(new FileReader(filePath))) {

String line = [Link]();

[Link]("HEADER: " + line);

int count = 0;

while ((line = [Link]()) != null) {

String[] data = [Link](",");

double[] features = new double[5];

features[0] = [Link](data[1]);
features[1] = [Link](data[2]);
features[2] = [Link](data[3]);
features[3] = [Link](data[4]);
features[4] = [Link](data[5]);

double target = [Link](data[6]);

[Link](features);
[Link](target);

count++;
}

[Link]("Total rows loaded: " + count);

 } catch (Exception e) {
[Link]();
}

// Convert List → Array

double[][] X = [Link](new double[0][]);
double[] y = [Link]().mapToDouble(Double::doubleValue).toArray();

// ================= TRAIN-TEST SPLIT =================

SplitResult result = split(X, y, 0.3, 42);

// Print result
[Link]("Train size: " + result.X_train.length);
[Link]("Test size: " + result.X_test.length);

[Link]("\nSample training data:");

for (int i = 0; i < [Link](5, result.X_train.length); i++) {
[Link]([Link](result.X_train[i]) + " -> " + result.y_train[i]);
}
}
}

Output

Notice that the testSize is (int)(10 * 0.3) = 3. Exactly 70% of the data went to training. The
shuffled indices determined which samples were assigned to which set. Changing the seed will
yield a different but equally random partition.
Key design choices
• Copy by reference: X_train[i] = X[idx] copies the reference to the row array, not the
array contents. This is safe for this use-case because we do not modify the feature rows
later. If modifications are needed, deep-copy the row.
• Fisher-Yates shuffle: O(N) and ensures every permutation is equally likely.
 • Reproducibility: The seed locks the random sequence, so the same split can be
recreated later (vital for debugging).

4.3 Stratified Train-Test Split (for Classification)

When you have imbalanced classes, a plain random split can accidentally put too many
or too few of a minority class into the test set, making evaluation misleading. Stratified splitting
preserves the class distribution in both subsets.
Algorithm Walk-through
1. Group indices by class label: Build a Map<Integer, List<Integer>> where each key is
a class and the value is a list of row indices belonging to that class.
2. For each class:
• Shuffle the list of its indices.
• Calculate testCount = (int)(classSize * testRatio).
Edge case handling: If testCount == 0 and testRatio > 0, force at least 1
sample into the test set. If testCount == classSize and testRatio < 1.0, keep at
least 1 sample for training.
• Take the first testCount indices for the test set, the rest for training.
3. Combine all the training indices from every class to form the overall training set;
similarly for the test set.
4. Build the final arrays by copying the rows at the accumulated indices.
This guarantees that every class appears in the test set with a proportion very close to its
global proportion.
We reuse the SplitResult class from SimpleTrainTestSplit. The method accepts integer
labels.

[Link]
package stratifiedtraintestsplit;
public class stratifiedtraintest {
public static class SplitResult {
public double[][] X_train;
public double[][] X_test;
public double[] y_train;
public double[] y_test;
public SplitResult(double[][] X_train, double[][] X_test,
double[] y_train, double[] y_test) {
this.X_train = X_train;
this.X_test = X_test;
this.y_train = y_train;
this.y_test = y_test;
}
}}
package stratifiedtraintestsplit;
import [Link].*;
public class StratifiedTrainTestSplit {
/**
* Stratified train-test split for classification.
*
* @param X feature matrix (N x D)
* @param y integer class labels (starting from 0)
* @param testRatio fraction for test set (0.0 to 1.0)
* @param seed random seed
* @return SplitResult (y_train and y_test cast to double)
*/
public static [Link] stratifiedSplit(
double[][] X, int[] y, double testRatio, long seed) {
int N = [Link];
Random rand = new Random(seed);
// 1. Group indices by class label
Map<Integer, List<Integer>> classToIndices = new HashMap<>();
for (int i = 0; i < N; i++) {
int label = y[i];
[Link](label, new ArrayList<>());
[Link](label).add(i);
}
List<Integer> trainIndices = new ArrayList<>();
List<Integer> testIndices = new ArrayList<>();
// 2. For each class, allocate test samples proportionally
for ([Link]<Integer, List<Integer>> entry : [Link]()) {
int classLabel = [Link]();
List<Integer> indices = [Link]();
// Shuffle the indices of this class
[Link](indices, new Random(seed + classLabel));
int classN = [Link]();
int testCount = (int)(classN * testRatio);
// Edge case: if testCount == 0 and we want a test set,
// ensure at least one sample goes to test
if (testCount == 0 && testRatio > 0 && classN > 0) {
testCount = 1;
}
// Edge case: if testCount == classN and we want training data,
// keep at least one for training
if (testCount == classN && testRatio < 1.0) {
testCount = classN - 1;
}
[Link]([Link](0, testCount));
 [Link]([Link](testCount, classN));
}
// 3. Build training arrays from the combined indices
double[][] X_train = new double[[Link]()][];
double[] y_train = new double[[Link]()];
for (int i = 0; i < [Link](); i++) {
int idx = [Link](i);
X_train[i] = X[idx];
y_train[i] = y[idx];
}
// 4. Build test arrays
double[][] X_test = new double[[Link]()][];
double[] y_test = new double[[Link]()];
for (int i = 0; i < [Link](); i++) {
int idx = [Link](i);
X_test[i] = X[idx];
y_test[i] = y[idx];
}
return new [Link](X_train, X_test, y_train, y_test);
}
// Demonstration
public static void main(String[] args) {
double[][] X = {{1},{2},{3},{4},{5},{6},{7},{8},{9},{10}};
int[] y = {0, 0, 0, 0, 0, 0, 0, 0, 1, 1}; // imbalanced: 80% class 0, 20% class 1
[Link] res = stratifiedSplit(X, y, 0.3, 42);
[Link]("Train labels: " + [Link](res.y_train));
[Link]("Test labels: " + [Link](res.y_test));
long testClass1 = [Link](res.y_test).filter(v -> v == 1.0).count();
[Link]("Class 1 count in test: " + testClass1 +
" (expected ~30% of 2 = ~1)");
}
}
Output:
Edge cases explained
• If a class has only 1 sample, and testRatio = 0.2, testCount would be 0. But then that
class would have no representative in the test set, which can be problematic for some
metrics (e.g., recall). We force testCount = 1 in that case.
• If testRatio = 1.0 (testing on everything, unusual), we avoid taking all samples from a
class by keeping at least one for training.
4.4 K-Fold Cross-Validation Skeleton
K-Fold cross-validation provides a more robust evaluation by rotating the test fold. Instead of
a single split, we perform k separate train-test splits, each using a different slice of the data as
the test set.
Algorithm Outline
1. Shuffle all indices (to avoid any ordering bias).
2. Divide the shuffled indices into k roughly equal-sized folds.
3. For each fold i (0 to k-1):
• Test set = fold i.
• Training set = all other folds combined.
• Build X_train, y_train, X_test, y_test and store the split.
Skeleton Code
The following code provides the basic structure; you can extend it to build the
actual SplitResult objects.
package kfoldskeleton;
import [Link].*;
public class Kfoldskeleton {
// SplitResult defined here — no external dependency needed
public static class SplitResult {
public double[][] X_train;
public double[][] X_test;
public double[] y_train;
public double[] y_test;

public SplitResult(double[][] X_train, double[][] X_test,

double[] y_train, double[] y_test) {
this.X_train = X_train;
this.X_test = X_test;
this.y_train = y_train;
this.y_test = y_test;
}
}
public static List<SplitResult> kFold(
double[][] X, double[] y, int k, long seed) {
int N = [Link];
List<Integer> indices = new ArrayList<>();
for (int i = 0; i < N; i++) [Link](i);
[Link](indices, new Random(seed));
 int foldSize = N / k;
List<SplitResult> folds = new ArrayList<>();

for (int fold = 0; fold < k; fold++) {

int testStart = fold * foldSize;
int testEnd = (fold == k - 1) ? N : (fold + 1) * foldSize;

List<Integer> testIdx = new ArrayList<>([Link](testStart, testEnd));

List<Integer> trainIdx = new ArrayList<>(indices);
[Link](testStart, testEnd).clear();

double[][] X_train = new double[[Link]()][];

double[] y_train = new double[[Link]()];
for (int i = 0; i < [Link](); i++) {
X_train[i] = X[[Link](i)];
y_train[i] = y[[Link](i)];
}

double[][] X_test = new double[[Link]()][];

double[] y_test = new double[[Link]()];
for (int i = 0; i < [Link](); i++) {
X_test[i] = X[[Link](i)];
y_test[i] = y[[Link](i)];
}

[Link](new SplitResult(X_train, X_test, y_train, y_test));

}
return folds;
}

public static void main(String[] args) {

double[][] X = {{1},{2},{3},{4},{5},{6},{7},{8},{9},{10}};
double[] y = {50, 55, 65, 70, 72, 78, 80, 85, 88, 92};

List<SplitResult> splits = kFold(X, y, 5, 42);

for (int i = 0; i < [Link](); i++) {
SplitResult res = [Link](i);
[Link]("Fold " + i + ": Train size = " +
res.X_train.length + ", Test size = " + res.X_test.length);
}
}
}
Output:

Important details
• The trainIdx list is created as a copy of all indices, then the test block is removed
using clear(). This works because subList is backed by the original list; clearing it
removes those elements.
• If N is not perfectly divisible by k, the last fold gets the extra samples. For
example, N=10, k=3 → folds: 3, 3, 4 samples. This is handled by the ternary
condition: (fold == k - 1) ? N : ....

5. Model Evaluation – Using the Splits

After you have split your data, trained the model only on the training set, and obtained
predictions on the unseen test set, the next step is to quantify the model’s performance. This
evaluation tells you how well the model will perform on new, real-world data. It’s the
moment of truth.
5.1 The Evaluation Workflow
The generic lifecycle, once the split is done, is:
1. Train: Fit your model using X_train and y_train.
2. Predict: Have the model generate outputs for X_test.
3. Compare: Use one or more evaluation metrics to measure the distance between the
predicted values and the true y_test.
4. Interpret: Decide if the model is good enough, or if you need to return to feature
engineering, model selection, or hyperparameter tuning — using only a validation set
for those decisions.

This workflow must never touch the test labels for anything other than the final scoring.
Golden rule: The test set is a locked vault. You open it only once, at the very end, to report
the final performance. Any decision that influences the model (feature selection, normalisation
constants, hyperparameters) must be made using a separate validation set or cross-validation
within the training data only.
5.2 Regression Metrics
Regression predicts a continuous number (price, temperature, etc.). The most common
metrics measure the difference between the predicted values 𝑦 𝑖 and the true values 𝑦𝑖 for 𝑛
test samples.
Mean Squared Error (MSE)
𝑛
1 2
𝑀𝑆𝐸 = ∑(𝑦𝑖 − 𝑦̂)
𝑖
𝑛
𝑖=1
• Interpretation: The average of the squared errors. Squaring heavily penalises large
mistakes. A lower MSE is better.
• Unit: The square of the target’s unit (e.g., if price is in euros, MSE is in €²), which
can be hard to interpret directly.
• Derivative metric: Root Mean Squared Error (RMSE), 𝑅𝑀𝑆𝐸 = 𝑀𝑆𝐸, brings the
error back to the original unit and is often preferred for reporting.
Mean Absolute Error (MAE)
𝑛
1
𝑀𝐴𝐸 = ∑|𝑥𝑖 − 𝑥|
𝑛
𝑖=1
Interpretation: The average absolute error. Less sensitive to outliers than MSE. In the same
unit as the target.
• A model with a low MAE is generally accurate on average without being dominated by
a few large errors.
R² Score (Coefficient of Determination)
𝟐
𝟐
∑𝒏𝒊=𝟏(𝒀𝒊 − 𝒀̂𝒊 )
𝑹 =𝟏− 𝒏
∑𝒊=𝟏(𝒀𝒊 − 𝒀̅𝒊 )𝟐
where 𝑦ˉ is the mean of the true values.
• Interpretation: How much of the variance in the target is explained by the
model. R2=1 means perfect prediction; R2=0 means the model is no better than always
predicting the mean; negative R2 means the model is worse than a simple average.
• It provides a scale-independent measure, useful for comparing across different datasets.
5.3 Classification Metrics
For classification, each prediction is a class label (e.g., spam/ham, disease/no‑disease).
The fundamental tool is the confusion matrix for a binary problem:
Actual Positive Actual Negative
Predicted Positive True Positive (TP) False Positive (FP)
Predicted Negative False Negative (FN) True Negative (TN)
5.4 Evaluation in Java – Example Code
Here’s how you might compute common metrics after obtaining predictions from your
model.
package metrics;
public class Metrics {
// ─── Regression
public static double mse(double[] yTrue, double[] yPred) {
double sum = 0;
 for (int i = 0; i < [Link]; i++) {
double diff = yTrue[i] - yPred[i];
sum += diff * diff;
}
return sum / [Link];
}

public static double rmse(double[] yTrue, double[] yPred) {

return [Link](mse(yTrue, yPred));
}

public static double r2(double[] yTrue, double[] yPred) {

double mean = 0;
for (double v : yTrue) mean += v;
mean /= [Link];

double ssRes = 0, ssTot = 0;

for (int i = 0; i < [Link]; i++) {
double diff = yTrue[i] - yPred[i];
double diffMean = yTrue[i] - mean;
ssRes += diff * diff;
ssTot += diffMean * diffMean;
}
return 1 - (ssRes / ssTot);
}

// ─── Classification
───────────────────────────────────────────
public static double accuracy(int[] yTrue, int[] yPred) {
int correct = 0;
for (int i = 0; i < [Link]; i++) {
if (yTrue[i] == yPred[i]) correct++;
}
return (double) correct / [Link];
}

public static double precision(int[] yTrue, int[] yPred, int positiveLabel) {

int tp = 0, fp = 0;
for (int i = 0; i < [Link]; i++) {
if (yPred[i] == positiveLabel) {
if (yTrue[i] == positiveLabel) tp++;
else fp++;
}
}
 return (tp + fp == 0) ? 0.0 : (double) tp / (tp + fp);
}

public static double recall(int[] yTrue, int[] yPred, int positiveLabel) {

int tp = 0, fn = 0;
for (int i = 0; i < [Link]; i++) {
if (yTrue[i] == positiveLabel) {
if (yPred[i] == positiveLabel) tp++;
else fn++;
}
}
return (tp + fn == 0) ? 0.0 : (double) tp / (tp + fn);
}

public static double f1(int[] yTrue, int[] yPred, int positiveLabel) {

double p = precision(yTrue, yPred, positiveLabel);
double r = recall (yTrue, yPred, positiveLabel);
return (p + r == 0) ? 0.0 : 2 * (p * r) / (p + r);
}

// ─── Main
─────────────────────────────────────────────────────
public static void main(String[] args) {

// ── Regression test data ──

double[] yTrue = {50, 55, 65, 70, 72, 78, 80, 85, 88, 92};
double[] yPred = {48, 57, 63, 72, 71, 79, 82, 84, 90, 91};

[Link]("=============================");
[Link](" REGRESSION METRICS ");
[Link]("=============================");
[Link]("MSE : %.4f%n", mse (yTrue, yPred));
[Link]("RMSE : %.4f%n", rmse(yTrue, yPred));
[Link]("R² : %.4f%n", r2 (yTrue, yPred));

// ── Classification test data ──

int[] classTrue = {0, 1, 1, 0, 1, 0, 1, 1, 0, 0};
int[] classPred = {0, 1, 0, 0, 1, 1, 1, 1, 0, 1};

[Link]("\n=============================");
[Link](" CLASSIFICATION METRICS ");
[Link]("=============================");
[Link]("Accuracy : %.4f%n", accuracy (classTrue, classPred));
[Link]("Precision : %.4f%n", precision(classTrue, classPred, 1));
 [Link]("Recall : %.4f%n", recall (classTrue, classPred, 1));
[Link]("F1 Score : %.4f%n", f1 (classTrue, classPred, 1));

// ── Confusion matrix breakdown ──

[Link]("\n=============================");
[Link](" CONFUSION MATRIX ");
[Link]("=============================");
int tp = 0, tn = 0, fp = 0, fn = 0;
for (int i = 0; i < [Link]; i++) {
if (classTrue[i] == 1 && classPred[i] == 1) tp++;
else if (classTrue[i] == 0 && classPred[i] == 0) tn++;
else if (classTrue[i] == 0 && classPred[i] == 1) fp++;
else if (classTrue[i] == 1 && classPred[i] == 0) fn++;
}
[Link](" Predicted");
[Link](" 0 1");
[Link]("Actual 0 | " + tn + " | " + fp + " |");
[Link](" 1 | " + fn + " | " + tp + " |");
}
}
OUTPUT
6. Introduction to Supervised Learning – A Deeper View
Supervised learning is the process of teaching a model a mapping f from inputs X to
outputs y using a set of labelled examples. But why does this work? The model assumes there
is an underlying, unknown relationship that generated the data, and it tries to approximate
that relationship. The better the approximation, the more accurate the predictions on new
data.
The training data is a sample from the true distribution. The model’s job is to recover
the signal, not the noise. That’s why evaluation on an unseen test set is so critical — it checks
whether the model captured the actual pattern or just memorised the sample.

Classification vs Regression – Visual Distinction

• Classification: The output space is discrete. The model draws decision boundaries
between classes.
• Regression: The output is a continuous value. The model fits a function (line, curve,
surface) through the data points.

7. Classification Models – Deep Intuition and Diagrams

7.1 Decision Tree – Building a Tree Step by Step
The decision tree is a flowchart that asks a sequence of questions about the features.
The “best” question is the one that maximises the reduction in class impurity — measured
by Information Gain.
How Entropy Drives the Splits
Entropy quantifies disorder. Suppose we have a binary classification with two classes
(Yes/No). The entropy curve looks like:

Information Gain compares the parent node’s entropy to the weighted average entropy
of its children. The feature that provides the largest gain is chosen.
Tree Construction Visualisation
 Imagine we are building a tree for the “Play Tennis” dataset. The first split is usually
on Outlook, because it gives the highest information gain. The diagram below illustrates the
splitting process:

The algorithm repeatedly finds the feature that makes the children as homogeneous as
possible. Pure nodes become leaves. This recursive partitioning gives the tree its
human-interpretable structure.
Decision Tree Prediction Path
For a new sample, the tree is traversed from root to leaf:

This simple rule-based prediction is why decision trees are often called “white-box”
models.
package decisiontree;
import [Link].*;

/**
* Decision Tree classifier using entropy and information gain.
* Supports numerical features and multi-class classification.
*/
public class DecisionTree {
private Node root;
private int minSamplesSplit = 2;
private int maxDepth = 10;

private static class Node {

int featureIndex;
double threshold;
Node left, right;
 Integer label; // if leaf
boolean isLeaf;

Node(Integer label) {
[Link] = label;
isLeaf = true;
}

Node(int featureIndex, double threshold, Node left, Node right) {

[Link] = featureIndex;
[Link] = threshold;
[Link] = left;
[Link] = right;
isLeaf = false;
}
}

public void fit(double[][] X, int[] y) {

Integer[] indices = new Integer[[Link]];
for (int i = 0; i < [Link]; i++) indices[i] = i;
root = buildTree(X, y, indices, 0);
}

private Node buildTree(double[][] X, int[] y, Integer[] indices, int depth) {

int n = [Link];
if (n == 0) return new Node(null);

// Extract labels for these indices

int[] labels = new int[n];
for (int i = 0; i < n; i++) labels[i] = y[indices[i]];

int majority = majorityClass(labels);

if (isPure(labels) || n < minSamplesSplit || depth >= maxDepth) {
return new Node(majority);
}

int bestFeature = -1;

double bestThreshold = 0;
double bestGain = -1.0;
int D = X[0].length;

for (int f = 0; f < D; f++) {

// Get unique sorted values
double[] values = new double[n];
 for (int i = 0; i < n; i++) values[i] = X[indices[i]][f];
[Link](values);
for (int i = 0; i < n - 1; i++) {
if (values[i] == values[i+1]) continue;
double threshold = (values[i] + values[i+1]) / 2.0;
List<Integer> leftList = new ArrayList<>();
List<Integer> rightList = new ArrayList<>();
for (int idx : indices) {
if (X[idx][f] <= threshold) [Link](idx);
else [Link](idx);
}
if ([Link]() || [Link]()) continue;

double gain = informationGain(y, indices, leftList, rightList);

if (gain > bestGain) {
bestGain = gain;
bestFeature = f;
bestThreshold = threshold;
}
}
}

if (bestFeature == -1) {
return new Node(majority);
}

List<Integer> leftList = new ArrayList<>();

List<Integer> rightList = new ArrayList<>();
for (int idx : indices) {
if (X[idx][bestFeature] <= bestThreshold) [Link](idx);
else [Link](idx);
}
Node leftChild = buildTree(X, y, [Link](new Integer[0]), depth + 1);
Node rightChild = buildTree(X, y, [Link](new Integer[0]), depth + 1);
return new Node(bestFeature, bestThreshold, leftChild, rightChild);
}

private double informationGain(int[] y, Integer[] parentIndices,

List<Integer> leftIndices, List<Integer> rightIndices) {
int n = [Link];
int[] parentLabels = extractLabels(y, parentIndices);
int[] leftLabels = extractLabels(y, leftIndices);
int[] rightLabels = extractLabels(y, rightIndices);
 double parentEntropy = entropy(parentLabels);
double leftEntropy = entropy(leftLabels);
double rightEntropy = entropy(rightLabels);
double childEntropy = ((double) [Link]() / n) * leftEntropy
+ ((double) [Link]() / n) * rightEntropy;
return parentEntropy - childEntropy;
}

private int[] extractLabels(int[] y, List<Integer> indices) {

int[] labels = new int[[Link]()];
for (int i = 0; i < [Link]; i++) labels[i] = y[[Link](i)];
return labels;
}

private int[] extractLabels(int[] y, Integer[] indices) {

int[] labels = new int[[Link]];
for (int i = 0; i < [Link]; i++) labels[i] = y[indices[i]];
return labels;
}

private double entropy(int[] labels) {

Map<Integer, Integer> counts = new HashMap<>();
for (int lbl : labels) [Link](lbl, [Link](lbl, 0) + 1);
double ent = 0.0;
int n = [Link];
for (int count : [Link]()) {
double p = (double) count / n;
ent -= p * ([Link](p) / [Link](2));
}
return ent;
}

private boolean isPure(int[] labels) {

int first = labels[0];
for (int lbl : labels) if (lbl != first) return false;
return true;
}
private int majorityClass(int[] labels) {
Map<Integer, Integer> counts = new HashMap<>();
for (int lbl : labels) [Link](lbl, [Link](lbl, 0) + 1);
int maxCount = 0, maxClass = labels[0];
for ([Link]<Integer, Integer> e : [Link]()) {
if ([Link]() > maxCount) {
maxCount = [Link]();
 maxClass = [Link]();
}
}
return maxClass;
}

public int predict(double[] sample) {

return predict(root, sample);
}
private int predict(Node node, double[] sample) {
if ([Link]) return [Link];
if (sample[[Link]] <= [Link])
return predict([Link], sample);
else
return predict([Link], sample);
}
// Example usage
public static void main(String[] args) {
// Simple dataset: XOR-like
double[][] X = {{0,0},{0,1},{1,0},{1,1}};
int[] y = {0,1,1,0};
DecisionTree dt = new DecisionTree();
[Link](X, y);
for (double[] sample : X) {
[Link]([Link](sample) + " -> " + [Link](sample));
}
}
}
OUTPUT

7.2 K-Nearest Neighbors – The Power of Proximity

KNN is a lazy learner: it stores the whole training set and, at prediction time, looks at
the 𝐾 closest training points.
Visualising KNN Decision Boundaries
For a 2-class problem in 2D, the decision surface changes dramatically with K:

Small K can model complex, noisy boundaries (overfitting risk), while

large K produces simpler, more stable regions (underfitting risk). The optimal K is usually
found via cross-validation.
Euclidean Distance and Feature Scaling
KNN relies on the distance metric. If one feature (e.g., salary in thousands) dominates
numerically over another (age in tens), distances become skewed:

Always normalise/standardise continuous features before using KNN or

gradient-based models.
package knearestneighbours;
import [Link].*;
/** * K-Nearest Neighbors classifier.
* Lazy learner: stores training data and computes distances at prediction time. */
public class KNearestNeighbours {
private double[][] X_train;
private int[] y_train;
private int k;
public KNearestNeighbours(int k) {
this.k = k;
}
public void fit(double[][] X, int[] y) {
this.X_train = X;
this.y_train = y;
}
public int predict(double[] sample) {
// Compute distances to all training points
int n = X_train.length;
double[] distances = new double[n];
for (int i = 0; i < n; i++) {
distances[i] = euclideanDistance(sample, X_train[i]);
}
// Find indices of k smallest distances (using a simple selection)
Integer[] indices = new Integer[n];
for (int i = 0; i < n; i++) indices[i] = i;
[Link](indices, [Link](i -> distances[i]));
// Majority vote among top k
Map<Integer, Integer> votes = new HashMap<>();
for (int i = 0; i < k; i++) {
int label = y_train[indices[i]];
[Link](label, [Link](label, 0) + 1);
}
int bestLabel = -1, bestCount = -1;
for ([Link]<Integer, Integer> e : [Link]()) {
if ([Link]() > bestCount) {
bestCount = [Link]();
bestLabel = [Link]();
}
}
return bestLabel;
}
private double euclideanDistance(double[] a, double[] b) {
double sum = 0.0;
for (int i = 0; i < [Link]; i++) {
double diff = a[i] - b[i];
sum += diff * diff;
}
return [Link](sum);
}
// Example usage
public static void main(String[] args) {
double[][] X = {{1,2},{2,3},{3,4},{5,6},{6,7}};
 int[] y = {0,0,0,1,1};
KNearestNeighbours knn = new KNearestNeighbours(3);
[Link](X, y);
// double[] test = {4,5};
double[] test = {5, 6};
[Link]("Prediction: " + [Link](test)); // expected 0 or 1
}
}
OUTPUT

8. Regression Models – From Line to Surface

8.1 Linear Regression – The Geometry of Fitting
Linear regression assumes a linear relationship between inputs and output. For a
single feature, it’s a line; for two features, a plane; for more, a hyperplane.

package linearregression;
import [Link];
/**
* Linear Regression using Gradient Descent.
* Includes a bias term (intercept).
*/
public class Linearregression {
private double[] weights; // includes bias at index 0
private double learningRate = 0.01;
private int maxIterations = 1000;
private double tolerance = 1e-6;

public Linearregression() {}

public Linearregression(double learningRate, int maxIterations) {

[Link] = learningRate;
[Link] = maxIterations;
}

public void fit(double[][] X, double[] y) {

 int n = [Link];
int d = X[0].length;
// Augment X with a column of ones for bias
double[][] X_aug = new double[n][d + 1];
for (int i = 0; i < n; i++) {
X_aug[i][0] = 1.0; // bias
for (int j = 0; j < d; j++) {
X_aug[i][j + 1] = X[i][j];
}
}
// Initialize weights to zero
weights = new double[d + 1];
// Gradient descent
for (int iter = 0; iter < maxIterations; iter++) {
double[] gradients = new double[d + 1];
for (int i = 0; i < n; i++) {
double prediction = predictInternal(X_aug[i]);
double error = prediction - y[i];
for (int j = 0; j < [Link]; j++) {
gradients[j] += error * X_aug[i][j];
}
}
// Average gradient
for (int j = 0; j < [Link]; j++) {
gradients[j] /= n;
}
// Update weights
boolean converged = true;
for (int j = 0; j < [Link]; j++) {
double step = learningRate * gradients[j];
weights[j] -= step;
if ([Link](step) > tolerance) converged = false;
}
if (converged) break;
}
}
private double predictInternal(double[] sampleAug) {
double sum = 0.0;
for (int i = 0; i < [Link]; i++) {
sum += weights[i] * sampleAug[i];
}
return sum;
}
public double predict(double[] sample) {
 // Augment sample with bias (1.0)
double[] aug = new double[[Link] + 1];
aug[0] = 1.0;
[Link](sample, 0, aug, 1, [Link]);
return predictInternal(aug);
}

public double[] predict(double[][] X) {

double[] preds = new double[[Link]];
for (int i = 0; i < [Link]; i++) {
preds[i] = predict(X[i]);
}
return preds;
}

// Mean Squared Error (for evaluation)

public double mse(double[] yTrue, double[] yPred) {
double sum = 0.0;
for (int i = 0; i < [Link]; i++) {
double diff = yTrue[i] - yPred[i];
sum += diff * diff;
}
return sum / [Link];
}
// Example usage
public static void main(String[] args) {
// Simple linear data: y = 2*x + 3 + noise
double[][] X = {{1},{2},{3},{4},{5}};
double[] y = {5.1, 7.2, 9.3, 11.0, 13.1}; // approx 2*x+3
Linearregression lr = new Linearregression(0.01, 1000);
[Link](X, y);
double[] test = {6};
double pred = [Link](test);
[Link]("Prediction for x=6: " + pred);
[Link]("Weights (bias, slope): " + [Link]([Link]));
}
}
OUTPUT:
The Cost Function Landscape
The Mean Squared Error (MSE) forms a convex “bowl” when plotted against the
weights. Gradient descent iteratively slides down this bowl to find the minimum:

The learning rate αα controls step size. Too small → slow convergence; too large →
divergence.
Gradient Descent Update Visualisation
For each weight wj:
2 𝑛
(𝑖) (𝑖) (𝑖)
𝑤𝑗 ≔ 𝑤𝑗 − 𝛼. ∑ (𝑦𝑝𝑟𝑒𝑑 − 𝑦𝑡𝑟𝑢𝑒 ). 𝑥𝑗
𝑛 𝑖=1
In Java, we accumulate the gradient across all training samples and then apply the
update. This is batch gradient descent.
Adding the Bias Term
We often augment the feature matrix with a column of ones so that the bias b becomes
just another weight w0. This allows the line/plane not to be forced through the origin.

9. Model Training Process in Java – A Unified Pipeline

Every model follows the same lifecycle. This modular design lets you swap models
without touching the rest of the code.
Example: Full KNN Pipeline with a Diagram
// Load data, split, fit, predict, evaluate
SplitResult split = [Link](X, y, 0.2, 42);
KNNClassifier knn = new KNNClassifier(3);
[Link](split.X_train, convertToInt(split.y_train));
int correct = 0;
for (int i = 0; i < split.X_test.length; i++) {
if ([Link](split.X_test[i]) == (int) split.y_test[i])
correct++;
}
double accuracy = 100.0 * correct / split.X_test.length;
The test set simulates future unseen data. The computed accuracy (e.g., 85%) is our
best estimate of real-world performance.
10. Evaluation Metrics – What They Really Measure
10.1 Accuracy – When It’s Enough and When It Lies
Accuracy is intuitive, but for imbalanced datasets it can be completely misleading. A
diagram makes it obvious:

In such cases, precision, recall, and F1-score (covered elsewhere) are mandatory. The
confusion matrix is the true diagnostic tool.
package evaluationaccuracy;
import [Link].*;
/**
* Comprehensive classification evaluation metrics.
* Computes confusion matrix, accuracy, precision, recall, and F1-score.
*/
public class Evaluationaccuracy {
/**
* Computes and prints the confusion matrix for binary or multi-class.
* @param yTrue true labels (as ints)
* @param yPred predicted labels (as ints)
* @return a 2D confusion matrix where [i][j] = count of true class i predicted as class j
*/
public static int[][] confusionMatrix(int[] yTrue, int[] yPred) {
// Determine number of classes
int maxClass = 0;
for (int v : yTrue) maxClass = [Link](maxClass, v);
for (int v : yPred) maxClass = [Link](maxClass, v);
int numClasses = maxClass + 1;
int[][] matrix = new int[numClasses][numClasses];
for (int i = 0; i < [Link]; i++) {
matrix[yTrue[i]][yPred[i]]++;
}
 return matrix;
}

public static void printConfusionMatrix(int[][] matrix) {

int n = [Link];
[Link]("Confusion Matrix:");
[Link](" ");
for (int j = 0; j < n; j++) [Link](" Pred%2d ", j);
[Link](" | Total");
for (int i = 0; i < n; i++) {
[Link]("True%2d ", i);
int rowSum = 0;
for (int j = 0; j < n; j++) {
[Link](" %5d ", matrix[i][j]);
rowSum += matrix[i][j];
}
[Link](" | %d", rowSum);
[Link]();
}
}

public static double accuracy(int[] yTrue, int[] yPred) {

int correct = 0;
for (int i = 0; i < [Link]; i++) {
if (yTrue[i] == yPred[i]) correct++;
}
return (double) correct / [Link];
}

// Precision, Recall, F1 for a specific positive class (binary or multi-class 'one-vs-rest')

public static double precision(int[] yTrue, int[] yPred, int positiveClass) {
int tp = 0, fp = 0;
for (int i = 0; i < [Link]; i++) {
if (yPred[i] == positiveClass) {
if (yTrue[i] == positiveClass) tp++;
else fp++;
}
}
return (tp + fp == 0) ? 0.0 : (double) tp / (tp + fp);
}

public static double recall(int[] yTrue, int[] yPred, int positiveClass) {

int tp = 0, fn = 0;
for (int i = 0; i < [Link]; i++) {
 if (yTrue[i] == positiveClass) {
if (yPred[i] == positiveClass) tp++;
else fn++;
}
}
return (tp + fn == 0) ? 0.0 : (double) tp / (tp + fn);
}

public static double f1Score(int[] yTrue, int[] yPred, int positiveClass) {

double p = precision(yTrue, yPred, positiveClass);
double r = recall(yTrue, yPred, positiveClass);
if (p + r == 0) return 0.0;
return 2 * (p * r) / (p + r);
}

// Demo
public static void main(String[] args) {
int[] yTrue = {0, 0, 0, 1, 1, 1, 2, 2, 2};
int[] yPred = {0, 0, 1, 1, 1, 2, 2, 2, 0}; // some misclassifications
int[][] cm = confusionMatrix(yTrue, yPred);
printConfusionMatrix(cm);
[Link]("Accuracy: %.2f%%\n", accuracy(yTrue, yPred) * 100);
[Link]("Precision (class 1): %.2f\n", precision(yTrue, yPred, 1));
[Link]("Recall (class 1): %.2f\n", recall(yTrue, yPred, 1));
[Link]("F1 (class 1): %.2f\n", f1Score(yTrue, yPred, 1));
}
}
OUTPUT:
10.2 MSE and RMSE – Measuring Typical Error Magnitude
• MSE is in squared units, making it hard to interpret.
• RMSE is in the same units as the target, giving a direct sense of average prediction
error.
For example, an RMSE of €5,000 on house prices tells you that typical predictions are off by
about €5,000.
package msermse;
/**
* Regression evaluation metrics.
* Includes MSE, RMSE, MAE, and R-squared.
*/
public class Msermse {
public static double mse(double[] yTrue, double[] yPred) {
double sum = 0.0;
for (int i = 0; i < [Link]; i++) {
double diff = yTrue[i] - yPred[i];
sum += diff * diff;
}
return sum / [Link];
}

public static double rmse(double[] yTrue, double[] yPred) {

return [Link](mse(yTrue, yPred));
}

public static double mae(double[] yTrue, double[] yPred) {

double sum = 0.0;
for (int i = 0; i < [Link]; i++) {
sum += [Link](yTrue[i] - yPred[i]);
}
return sum / [Link];
}

public static double r2Score(double[] yTrue, double[] yPred) {

double mean = 0.0;
for (double v : yTrue) mean += v;
mean /= [Link];

double ssRes = 0.0; // sum of squared residuals

double ssTot = 0.0; // total sum of squares
for (int i = 0; i < [Link]; i++) {
ssRes += [Link](yTrue[i] - yPred[i], 2);
ssTot += [Link](yTrue[i] - mean, 2);
}
 if (ssTot == 0) return 1.0; // perfect constant data
return 1 - (ssRes / ssTot);
}

// Demo
public static void main(String[] args) {
double[] yTrue = {100, 200, 300, 400, 500};
double[] yPred = {110, 190, 310, 390, 520}; // slight errors
[Link]("MSE: %.2f\n", mse(yTrue, yPred));
[Link]("RMSE: %.2f\n", rmse(yTrue, yPred));
[Link]("MAE: %.2f\n", mae(yTrue, yPred));
[Link]("R²: %.4f\n", r2Score(yTrue, yPred));
}
}
OUTPUT:

11. Common Pitfalls – Visualised and Prevented

Overfitting in Decision Trees
An unrestricted tree can grow until each leaf contains only one training sample. It will
have 100% training accuracy but terrible test performance.

Prevention: Limit maximum depth, require a minimum number of samples per leaf, or use
post-pruning.
package overfiitingdecisiontree;

import [Link].*;

/**
* Enhanced Decision Tree with explicit overfitting prevention parameters.
* Use maxDepth and minSamplesSplit to control tree growth.
*/
public class Overfiitingdecisiontree {

private Node root;

private int minSamplesSplit;
private int maxDepth;

// ─── Node
─────────────────────────────────────────────────────
private static class Node {
int featureIndex;
double threshold;
Node left, right;
Integer label;
boolean isLeaf;

Node(Integer label) {
[Link] = label;
[Link] = true;
}

Node(int featureIndex, double threshold, Node left, Node right) {

[Link] = featureIndex;
[Link] = threshold;
[Link] = left;
[Link] = right;
[Link] = false;
}
}

// ─── Constructor
──────────────────────────────────────────────
public Overfiitingdecisiontree(int maxDepth, int minSamplesSplit) {
[Link] = maxDepth;
[Link] = minSamplesSplit;
}
 // ─── Fit
─────────────────────────────────────────────────────
─
public void fit(double[][] X, int[] y) {
Integer[] indices = new Integer[[Link]];
for (int i = 0; i < [Link]; i++) indices[i] = i;
root = buildTree(X, y, indices, 0);
}

private Node buildTree(double[][] X, int[] y, Integer[] indices, int depth) {

int n = [Link];
if (n == 0) return new Node(null);

int[] labels = new int[n];

for (int i = 0; i < n; i++) labels[i] = y[indices[i]];
int majority = majorityClass(labels);

// Stop conditions — this is where overfitting is controlled

if (isPure(labels) || n < minSamplesSplit || depth >= maxDepth) {
return new Node(majority);
}

int bestFeature = -1;

double bestThreshold = 0;
double bestGain = -1.0;
int D = X[0].length;

for (int f = 0; f < D; f++) {

double[] values = new double[n];
for (int i = 0; i < n; i++) values[i] = X[indices[i]][f];
[Link](values);
for (int i = 0; i < n - 1; i++) {
if (values[i] == values[i+1]) continue;
double threshold = (values[i] + values[i+1]) / 2.0;
List<Integer> leftList = new ArrayList<>();
List<Integer> rightList = new ArrayList<>();
for (int idx : indices) {
if (X[idx][f] <= threshold) [Link](idx);
else [Link](idx);
}
if ([Link]() || [Link]()) continue;
double gain = informationGain(y, indices, leftList, rightList);
if (gain > bestGain) {
bestGain = gain;
 bestFeature = f;
bestThreshold = threshold;
}
}
}

if (bestFeature == -1) return new Node(majority);

List<Integer> leftList = new ArrayList<>();

List<Integer> rightList = new ArrayList<>();
for (int idx : indices) {
if (X[idx][bestFeature] <= bestThreshold) [Link](idx);
else [Link](idx);
}

Node leftChild = buildTree(X, y, leftList .toArray(new Integer[0]), depth + 1);

Node rightChild = buildTree(X, y, [Link](new Integer[0]), depth + 1);
return new Node(bestFeature, bestThreshold, leftChild, rightChild);
}

// ─── Predict
──────────────────────────────────────────────────
public int predict(double[] sample) {
return predict(root, sample);
}

private int predict(Node node, double[] sample) {

if ([Link]) return [Link];
if (sample[[Link]] <= [Link])
return predict([Link], sample);
else
return predict([Link], sample);
}

// ─── Helpers
──────────────────────────────────────────────────
private double informationGain(int[] y, Integer[] parentIndices,
List<Integer> leftIndices, List<Integer> rightIndices) {
int n = [Link];
double childEntropy = ((double) leftIndices .size() / n) * entropy(extractLabels(y,
leftIndices))
+ ((double) [Link]() / n) * entropy(extractLabels(y, rightIndices));
return entropy(extractLabels(y, parentIndices)) - childEntropy;
}
 private int[] extractLabels(int[] y, List<Integer> indices) {
int[] labels = new int[[Link]()];
for (int i = 0; i < [Link]; i++) labels[i] = y[[Link](i)];
return labels;
}

private int[] extractLabels(int[] y, Integer[] indices) {

int[] labels = new int[[Link]];
for (int i = 0; i < [Link]; i++) labels[i] = y[indices[i]];
return labels;
}

private double entropy(int[] labels) {

Map<Integer, Integer> counts = new HashMap<>();
for (int lbl : labels) [Link](lbl, [Link](lbl, 0) + 1);
double ent = 0.0;
for (int count : [Link]()) {
double p = (double) count / [Link];
ent -= p * ([Link](p) / [Link](2));
}
return ent;
}

private boolean isPure(int[] labels) {

for (int lbl : labels) if (lbl != labels[0]) return false;
return true;
}

private int majorityClass(int[] labels) {

Map<Integer, Integer> counts = new HashMap<>();
for (int lbl : labels) [Link](lbl, [Link](lbl, 0) + 1);
int maxCount = 0, maxClass = labels[0];
for ([Link]<Integer, Integer> e : [Link]()) {
if ([Link]() > maxCount) {
maxCount = [Link]();
maxClass = [Link]();
}
}
return maxClass;
}

// ─── Main
─────────────────────────────────────────────────────
 public static void main(String[] args) {
double[][] X = {{1,2},{2,3},{3,4},{4,5},{5,6},{6,7},{7,8},{8,9}};
int[] y = { 0, 0, 0, 0, 1, 1, 1, 1 };

// Overfit tree — no restrictions

Overfiitingdecisiontree overfitTree = new Overfiitingdecisiontree(10, 2);

// Safe tree — depth and sample limits prevent overfitting

Overfiitingdecisiontree safeTree = new Overfiitingdecisiontree(3, 4);

[Link](X, y);
safeTree .fit(X, y);

double[] test = {4.5, 5.5};

[Link]("======================================");
[Link](" OVERFITTING PREVENTION DEMO ");
[Link]("======================================");
[Link]("Test input: [4.5, 5.5]");
[Link]("Overfit tree (maxDepth=10, minSamples=2) -> Class: " +
[Link](test));
[Link]("Safe tree (maxDepth=3, minSamples=4) -> Class: " + safeTree
.predict(test));
[Link]("--------------------------------------");
[Link]("Overfitting prevention parameters:");
[Link](" maxDepth : limits how deep the tree grows");
[Link](" minSamplesSplit : minimum samples needed to split a node");
}
}

Output:
Feature Scaling Pitfall
Without scaling, gradient descent can zigzag because features with large ranges
dominate the error surface:

Prevention: Standardise (zero mean, unit variance) or normalise (e.g., MinMax) features
before training KNN and linear regression.

package featurescaling;
import [Link].*;
/**
* Feature scaling utilities to prevent distance/gradient domination.
*/
public class Featurescaling {

// ─── Min-Max Scaling

──────────────────────────────────────────
public static double[][] minMaxScale(double[][] X) {
if ([Link] == 0) return X;
int n = [Link];
int d = X[0].length;
double[][] scaled = new double[n][d];
double[] min = new double[d];
double[] max = new double[d];
for (int j = 0; j < d; j++) {
min[j] = Double.MAX_VALUE;
max[j] = -Double.MAX_VALUE;
}
for (double[] row : X) {
 for (int j = 0; j < d; j++) {
if (row[j] < min[j]) min[j] = row[j];
if (row[j] > max[j]) max[j] = row[j];
}
}
for (int i = 0; i < n; i++) {
for (int j = 0; j < d; j++) {
double range = max[j] - min[j];
scaled[i][j] = (range == 0) ? 0.0 : (X[i][j] - min[j]) / range;
}
}
return scaled;
}
// ─── Standardization (Z-score) ────────────────────────────────
public static double[][] standardize(double[][] X) {
if ([Link] == 0) return X;
int n = [Link];
int d = X[0].length;
double[][] scaled = new double[n][d];

double[] mean = new double[d];

double[] std = new double[d];

for (int j = 0; j < d; j++) {

double sum = 0.0;
for (double[] row : X) sum += row[j];
mean[j] = sum / n;
}
for (int j = 0; j < d; j++) {
double sumSq = 0.0;
for (double[] row : X) {
double diff = row[j] - mean[j];
sumSq += diff * diff;
}
std[j] = [Link](sumSq / n);
if (std[j] == 0) std[j] = 1.0;
}
for (int i = 0; i < n; i++) {
for (int j = 0; j < d; j++) {
scaled[i][j] = (X[i][j] - mean[j]) / std[j];
}
}
return scaled;
}
 // ─── Main
─────────────────────────────────────────────────────
public static void main(String[] args) {
double[][] data = {{1, 1000}, {2, 2000}, {3, 3000}};

[Link]("=============================");
[Link](" ORIGINAL ");
[Link]("=============================");
for (double[] row : data)
[Link]([Link](row));

double[][] minmax = minMaxScale(data);

[Link]("\n=============================");
[Link](" MIN-MAX SCALED [0,1] ");
[Link]("=============================");
for (double[] row : minmax)
[Link]([Link](row));
double[][] standardized = standardize(data);
[Link]("\n=============================");
[Link](" STANDARDIZED (Z-SCORE) ");
[Link]("=============================");
for (double[] row : standardized)
[Link]([Link](row));
}
}

OUTPUT

Not Shuffling Before Splitting (When Appropriate)

If data is sorted by class, the training set may contain only one class. Shuffling
ensures a representative distribution.
Prevention: Always shuffle for i.i.d. (independent and identically distributed) data; never
shuffle for time-series.

12. Introduction to Unsupervised Learning & Clustering

Supervised learning relies on labelled data – each example has a known correct answer.
In unsupervised learning, we work with raw, unlabelled data. The model must find structure
on its own: hidden groupings, patterns, or representations.
Clustering is the most common unsupervised task. It partitions a set of points into clusters
such that:
• Points inside the same cluster are highly similar.
• Points in different clusters are very different.
Real-world applications include:
• Customer segmentation: grouping shoppers by purchase behaviour.
• Image colour quantisation: reducing the number of colours by clustering pixel values.
• Document topic discovery: grouping articles that use similar words.
• Anomaly detection: isolating data points far from any cluster.
Unlike classification, there is no ground-truth label to validate against. Evaluation relies on
internal metrics or domain interpretation.
13. K-Means Clustering – The Core Intuition
13.1 How K-Means Thinks
Imagine a 2D scatterplot of points. K-Means places K “centroids” (cluster centres) and
repeatedly refines them:
• Assignment step: Each point is assigned to the nearest centroid.
• Update step: Each centroid is moved to the average of all points assigned to it.
These two steps repeat until the centroids stop moving – the clusters become stable.
This process is like finding natural centres of mass.
13.2 Visualising the Iterations

Example sequence for K=3 on a synthetic dataset:

The algorithm minimises the inertia (sum of squared distances from points to their
centroids), but it may get trapped in a local minimum. That’s why it’s common to run K-Means
several times with different initial centroids and keep the best result (lowest inertia).
package kmeansclustering;
import [Link].*;

/**
* K-Means clustering from scratch.
* Supports any number of features, random initialisation,
* empty cluster recovery, and inertia computation.
*/
public class Kmeansclustering {
private double[][] centroids;
private int[] assignments;
private int k;
private int maxIterations = 100;

public void fit(double[][] X, int k, long seed) {

this.k = k;
int n = [Link];
int d = X[0].length;
Random rand = new Random(seed);

// 1. Initialisation: Forgy method (pick k distinct random points)

centroids = new double[k][d];
List<Integer> chosen = new ArrayList<>();
for (int i = 0; i < k; i++) {
int idx;
do {
idx = [Link](n);
} while ([Link](idx));
[Link](idx);
centroids[i] = [Link](X[idx], d);
}

assignments = new int[n];

boolean changed = true;
int iter = 0;

while (changed && iter < maxIterations) {

changed = false;
iter++;

// 2. Assignment step
for (int i = 0; i < n; i++) {
int nearest = nearestCentroid(X[i]);
if (nearest != assignments[i]) {
 changed = true;
assignments[i] = nearest;
}
}

// 3. Update step
double[][] sum = new double[k][d];
int[] count = new int[k];
for (int i = 0; i < n; i++) {
int c = assignments[i];
count[c]++;
for (int j = 0; j < d; j++) {
sum[c][j] += X[i][j];
}
}

for (int c = 0; c < k; c++) {

if (count[c] == 0) {
// Empty cluster: reinitialise to a random point
int randomIdx = [Link](n);
centroids[c] = [Link](X[randomIdx], d);
} else {
for (int j = 0; j < d; j++) {
centroids[c][j] = sum[c][j] / count[c];
}
}
}
}
}

private int nearestCentroid(double[] point) {

int best = 0;
double bestDist = Double.POSITIVE_INFINITY;
for (int c = 0; c < [Link]; c++) {
double dist = squaredDistance(point, centroids[c]);
if (dist < bestDist) {
bestDist = dist;
best = c;
}
}
return best;
}

private double squaredDistance(double[] a, double[] b) {

 double sum = 0.0;
for (int i = 0; i < [Link]; i++) {
double diff = a[i] - b[i];
sum += diff * diff;
}
return sum;
}

public int predict(double[] point) {

return nearestCentroid(point);
}

public double[][] getCentroids() {

return centroids;
}

public int[] getAssignments() {

return assignments;
}

// 3.4 Inertia computation

public double inertia(double[][] X) {
double sum = 0.0;
for (int i = 0; i < [Link]; i++) {
int c = assignments[i];
sum += squaredDistance(X[i], centroids[c]);
}
return sum;
}

// Demo
public static void main(String[] args) {
// Simple 2D synthetic data (3 clusters)
double[][] X = {
{1, 1}, {1.5, 2}, {2, 1.5}, // cluster 0
{8, 8}, {8.5, 9}, {9, 8.5}, // cluster 1
{5, 10}, {5.5, 11}, {6, 10.5} // cluster 2
};
Kmeansclustering km = new Kmeansclustering();
[Link](X, 3, 42);
[Link]("Centroids:");
for (double[] c : [Link]()) {
[Link]([Link](c));
}
 [Link]("Assignments: " + [Link]([Link]()));
[Link]("Inertia: " + [Link](X));
}
}

OUTPUT

13.3 Choosing K – The Elbow Method

K is a hyperparameter. The elbow method helps:
1. Run K-Means for K=1,2,3,...
2. Record the inertia for each K.
3. Plot inertia vs. K – the curve usually drops sharply at first, then flattens.
4. The “elbow” – the point of diminishing returns – suggests a natural number of clusters.

If the data truly has well-separated groups, the elbow is clear. In practice, you also
rely on domain knowledge.

14. Java Implementation of K-Means Clustering

The provided KMeans class is a clean, from-scratch implementation. Let’s highlight
the crucial parts and add context.
14.1 Data Structures
• double[][] X – N points, each a D-dimensional array.
• double[][] centroids – K×D array holding current centroids.
• int[] assignments – for each point, the index of its assigned cluster (0 to K−1).
14.2 Key Implementation Details
Initialisation – Forgy method
Randomly pick K distinct data points as initial centroids. The seed ensures reproducibility.
centroids[i] = [Link](X[idx], D);
Assignment step
For each point, compute the squared Euclidean distance to all centroids (avoiding sqrt for
efficiency) and pick the nearest.
int nearest = nearestCentroid(X[n]);
if (nearest != assignments[n]) {
changed = true;
assignments[n] = nearest;
}
Update step
For each cluster, accumulate the sum of points and divide by count to get the new centroid.
for (int d = 0; d < D; d++) {
centroids[c][d] = sum[c][d] / count[c];
}
Empty cluster handling
If a cluster gets zero points (can happen with poor initialisation), reinitialise it to a random
data point to keep K clusters.
if (count[c] == 0) {
int randomIdx = [Link](N);
centroids[c] = [Link](X[randomIdx], D);
}
Convergence
The loop stops when no assignment changes or the maximum iterations are reached.
14.3 Prediction
A new point is simply assigned to the nearest centroid:
public int predict(double[] point) {
return nearestCentroid(point);
}
This makes K-Means a prototype-based method: the centroids are the model’s
learned parameters.
14.4 Inertia Computation
Inertia is the sum of squared distances from each point to its assigned centroid. It’s
used to measure cluster compactness.
public double inertia(double[][] X) {
double sum = 0.0;
for (int n = 0; n < [Link]; n++) {
 int c = assignments[n];
sum += squaredDistance(X[n], centroids[c]);
}
return sum;
}
14.5 Data Flow Diagram

15. Unsupervised Model Training Process

Clustering follows its own pipeline. Because there are no-
labels, preprocessing becomes even more critical.
15.1 The Full Pipeline

15.2 Why Feature Scaling is Mandatory

K-Means uses Euclidean distance. If one feature has a much larger numerical range
(e.g., “salary” in thousands) than another (“age” in tens), the distance will be dominated by
salary. The clusters will be formed almost exclusively along that dimension, ignoring age
entirely.
 In the provided ClusteringPipeline, the minMaxScale method transforms each feature
to the range [0,1], giving equal weight. Standardisation (zero mean, unit variance) is another
common choice.
15.3 Preprocessing Example Code Explanation
double[][] scaled = new double[N][D];
for (int i = 0; i < N; i++) {
for (int d = 0; d < D; d++) {
double range = max[d] - min[d];
if (range == 0) scaled[i][d] = 0;
else scaled[i][d] = (X[i][d] - min[d]) / range;
}
}
This ensures every feature is treated fairly, and distances are meaningful.
15.4 After Training – Using the Model
Once centroids are learned, any new data point can be assigned to a cluster
with [Link](newPoint). You can also analyse the centroids to understand typical
profiles for each cluster. For example, in customer segmentation, a centroid might represent
“high income, low spending” vs “medium income, high spending”.

16. Evaluation Metrics for Clustering

Without labels, we rely on internal metrics that measure cluster compactness and
separation.
16.1 Inertia (Within-Cluster Sum of Squares)
𝑵
𝟐
𝑰𝒏𝒆𝒓𝒕𝒊𝒂 = ∑‖𝒙𝒊 − 𝝁𝒄(𝒊) ‖
𝒊=𝟏
Measures how tight clusters are internally.
• Lower is better, but it always decreases with larger K.
• Used primarily for the elbow method, not as a standalone selection criterion.
The inertia() method in our KMeans class returns this value.
16.2 Silhouette Score – A Better Criterion (Conceptual)
The silhouette score considers both cohesion (how close a point is to its own cluster)
and separation (how close it is to the nearest other cluster). For each point i:
• a(i) = average distance to other points in the same cluster.
• b(i) = average distance to points in the nearest different cluster.
• s(i) = b(i)−a(i), s(i)=max{a(i), b(i)}b(i)−a(i).
The silhouette coefficient ranges from −1 to 1:
• Near +1: point well matched to its own cluster.
• Near 0: point on the boundary.
• Near −1: point likely in the wrong cluster.
The overall silhouette score is the average over all points. A higher average indicates
better-defined clusters.

Unlike inertia, silhouette doesn’t automatically improve with more clusters; it penalises
poor separation. It’s a robust guide for choosing K (pick K that maximises the silhouette score),
but it’s more complex to compute from scratch.
16.3 The Elbow Method in Practice
 The curve flattens after K=3; adding more clusters gives little extra compactness,
so K=3 is a sensible choice.
16.4 External Evaluation (When Labels Exist)
In rare cases where true labels are available (e.g., known customer tiers), you can use
supervised metrics like Adjusted Rand Index or normalised mutual information. These
are beyond our Java classroom scope but worth knowing.

I. Practice Questions on Train-Test Split in Java:-

1. Write a Java program to split a dataset containing 100 records into 70% training data and
30% testing data.
Input:
Total Records = 100
Output:
Training Records: 70
Testing Records: 30

2. Write a Java program that accepts the total number of records and training percentage from
the user and displays the number of training and testing records.
Input:
Enter Total Records: 200
Enter Training Percentage: 80
Output:
Training Records: 160
Testing Records: 40

3. Write a Java program to randomly shuffle the dataset before splitting it into 70% training
data and 30% testing data.
Input:
Dataset: [1,2,3,4,5,6,7,8,9,10]
Possible Output:
Shuffled Dataset: [5,2,9,1,8,6,3,10,4,7]
Training Data: [5,2,9,1,8,6,3]
Testing Data: [10,4,7]

4. Write a Java program to read records from a CSV file and split them into [Link] and
[Link] using an 80:20 ratio.
Input ([Link]):
ID,Name,Marks
1,Ram,80
2,Sam,75
3,Ravi,90
4,Kumar,85
5,John,70
Output:
[Link] created successfully.
[Link] created successfully.

Training Records: 4
Testing Records: 1

5. Write a Java program to compare the following train-test split ratios:

• 60:40
• 70:30
• 80:20
• 90:10
Input:
Total Records: 100
Output:
60:40 -> Train: 60, Test: 40
70:30 -> Train: 70, Test: 30
80:20 -> Train: 80, Test: 20
90:10 -> Train: 90, Test: 10

II. Practice Programs on Linear Regression and Decision Tree:-

1. Write a Java program to predict student marks based on study hours using the Linear
Regression equation:
Marks=5×Study Hours+20
Input:
Enter Study Hours: 8
Output:
Predicted Marks: 60.0

2. Write a Java program to calculate the slope (m) and intercept (c) for the given dataset.
Input:
X = [1, 2, 3, 4, 5]
Y = [25, 30, 35, 40, 45]
Output:
Slope (m): 5.0
Intercept (c): 20.0

3. Write a Java program to calculate the Mean Squared Error (MSE) between actual and
predicted values.
Input:
Actual Values: [60, 70, 80]
Predicted Values: [58, 72, 78]
Output:
Mean Squared Error: 4.0

4. Write a Java program to classify student results using the following rules:
• Marks ≥ 50 → Pass
• Marks < 50 → Fail
Input:
Enter Student Marks: 65
Output:
Classification: Pass

5. Write a Java program to predict whether a customer will purchase a product using the
following rules:
• Age ≥ 30 and Income ≥ 50000 → Purchase
• Otherwise → No Purchase
Input:
Enter Age: 35
Enter Income: 60000
Output:
Prediction: Customer Will Purchase

III. Practice Programs on K-Means Clustering:-

1. Write a Java program to group students into 2 clusters based on their marks using K-
Means clustering.
Input:
Student Marks: [35, 40, 45, 80, 85, 90]
Number of Clusters (K): 2
Output:
Cluster 1: [35, 40, 45]
Cluster 2: [80, 85, 90]

2. Write a Java program to group customers into 3 clusters based on their purchase amounts.
Input:
Purchase Amounts: [1000, 1200, 1500, 5000, 5500, 6000, 9000, 9500]
K=3
Output:
Cluster 1: [1000, 1200, 1500]
Cluster 2: [5000, 5500, 6000]
Cluster 3: [9000, 9500]
3. Write a Java program to divide employees into 3 salary groups using K-Means clustering.
Input:
Salaries: [20000, 25000, 30000, 60000, 65000, 70000, 100000]
K=3
Output:
Cluster 1: [20000, 25000, 30000]
Cluster 2: [60000, 65000, 70000]
Cluster 3: [100000]

4. Write a Java program to group products into 2 clusters based on monthly sales.
Input:
Sales: [100, 120, 150, 800, 850, 900]
K=2
Output:
Cluster 1: [100, 120, 150]
Cluster 2: [800, 850, 900]

5. Write a Java program to group exam scores into 2 clusters using K-Means clustering.
Input:
Scores: [30, 35, 40, 75, 80, 85]
K=2
Output:
Cluster 1: [30, 35, 40]
Cluster 2: [75, 80, 85]