MACHINE LEARNING

Assignment Report

Submitted By: Anees Ur Rehman

Student ID No.: B1/ 439
Course Title: Machine Learning
Submission Date: August 2, 2026
Task 1: Predictive Linear Regression Model Pipeline

2.1 Objective
The objective of this task is to construct a supervised regression pipeline capable of predicting a
continuous numerical value, namely a quantitative measure of disease progression, using patient
physiological and biochemical baseline measurements.

2.2 Dataset
The Diabetes Progression dataset (scikit-learn built-in, a standard benchmark regression dataset
comparable in structure to auto-mpg or California Housing) was used. It comprises 442 patient records
with 10 baseline physiological features (age, sex, BMI, average blood pressure, and six blood-serum
measurements) and a continuous target representing disease progression one year after baseline. To
faithfully demonstrate the required preprocessing step, 3% of BMI values were artificially set to missing
prior to imputation.

2.3 Methodology
A scikit-learn Pipeline was built consisting of the following stages, chained together to prevent data
leakage between training and test folds:

• Mean imputation of missing numeric values (SimpleImputer).

• Feature scaling via StandardScaler (zero mean, unit variance).
• Regularized linear regression using Ridge regression, with the regularization strength (α) tuned via
5-fold GridSearchCV over {0.01, 0.1, 1, 10, 50, 100}.
• An unregularized ordinary Linear Regression pipeline was also trained as a baseline for comparison.
The data was split 80/20 into training and test sets (random_state = 42 for reproducibility).

2.4 Key Code

pipeline = Pipeline(steps=[
("imputer", SimpleImputer(strategy="mean")),
("scaler", StandardScaler()),
("model", Ridge())
])

param_grid = {"model__alpha": [0.01, 0.1, 1.0, 10.0, 50.0, 100.0]}

grid = GridSearchCV(pipeline, param_grid, cv=5, scoring="r2", n_jobs=-1)
[Link](X_train, y_train)

preds = grid.best_estimator_.predict(X_test)
r2 = r2_score(y_test, preds)
rmse = [Link](mean_squared_error(y_test, preds))
2.5 Results
Best regularization strength selected by cross-validation: α = 50.0 (5-fold CV R² = 0.4564).

Model R² (test) RMSE (test) MAE (test)

Ridge Regression (α = 50) 0.4596 53.51 43.63

Linear Regression (baseline) 0.4532 53.82 43.27

Ridge regression achieved a marginally higher R² and lower RMSE than the unregularized baseline,
confirming that mild L2 regularization improves generalization on this feature set. The top three
predictors by absolute standardized coefficient were BMI (+23.65), the s5 serum measurement (+18.38),
and average blood pressure (+15.11) — all positively associated with faster disease progression,
consistent with established clinical understanding of diabetes risk factors.

Figure 2.1 — Actual vs. Predicted disease-progression values on the test set.
 Figure 2.2 — Standardized Ridge regression coefficients (feature importance).

Figure 2.3 — Residual plot; residuals scatter randomly around zero, indicating no strong systematic bias.

2.6 Discussion
An R² of approximately 0.46 indicates the model explains under half of the variance in disease
progression, which is typical for this benchmark dataset given the limited, purely physiological feature
set (no lifestyle, genetic, or longitudinal data are included). The residual plot shows a random scatter
around zero without an obvious funnel or curved pattern, suggesting the linear model assumptions are
reasonably well satisfied and that the errors are not strongly heteroscedastic. Further improvement
would likely require additional features or a non-linear model (e.g., Random Forest or Gradient Boosting
regressors).
Task 2: Multi-Class Image Classification with a Convolutional Neural Network
(CNN)

3.1 Objective
This task builds and trains a deep-learning computer-vision pipeline that classifies raw pixel image arrays into
one of ten handwritten-digit classes (0–9) using a custom convolutional neural network (CNN).

3.2 Dataset
The scikit-learn 'digits' dataset was used 1,797 handwritten digit images (8×8 grayscale pixel arrays),
functionally equivalent to a compact, offline version of MNIST. (Note: the full MNIST/CIFAR-10 archives
require an external download that is not reachable from this sandboxed execution environment; the
digits dataset is the standard offline substitute and preserves the same raw-pixel-array, 10-class
classification structure requested by the assignment. If you have direct MNIST/CIFAR-10 access on your
own machine, the identical script architecture below runs unchanged after swapping the data-loading
cell.) Images were normalized to [0,1] and reshaped to (8,8,1) tensors. The data was split into training
(1,221), validation (216), and test (360) sets using stratified sampling.

3.3 Model Architecture

A custom multi-layered CNN was implemented in TensorFlow/Keras:

• Conv2D(32, 3×3, ReLU) → Conv2D(32, 3×3, ReLU) → MaxPooling2D(2×2) → Dropout(0.25)

• Conv2D(64, 3×3, ReLU) → MaxPooling2D(2×2) → Dropout(0.25)
• Flatten → Dense(128, ReLU) → Dropout(0.4) → Dense(10, Softmax)
Total trainable parameters: 62,250. Optimizer: Adam; Loss: sparse categorical cross-entropy; trained for 30
epochs with batch size 32.

3.4 Data Augmentation

To reduce overfitting given the small image size, Keras' ImageDataGenerator applied random rotations (±10°),
width/height shifts (±8%), and zoom (±8%) to the training batches on the fly.

3.5 Key Code

model = [Link]([
[Link](shape=(8, 8, 1)),
layers.Conv2D(32, (3,3), padding="same", activation="relu"),
layers.Conv2D(32, (3,3), padding="same", activation="relu"),
layers.MaxPooling2D((2,2)),
[Link](0.25),
layers.Conv2D(64, (3,3), padding="same", activation="relu"),
layers.MaxPooling2D((2,2)),
[Link](0.25),
[Link](),
 [Link](128, activation="relu"),
[Link](0.4),
[Link](10, activation="softmax")
])
[Link](optimizer="adam",
loss="sparse_categorical_crossentropy",
metrics=["accuracy"])

history = [Link]([Link](X_train, y_train, batch_size=32),

validation_data=(X_val, y_val), epochs=30)

3.6 Results

Metric Value

Final Training Accuracy 95.2%

Final Validation Accuracy 98.6%

Test Accuracy 98.9%

Test Loss 0.0399

Figure 3.1 — Training vs. validation accuracy across 30 epochs.

 Figure 3.2 — Training vs. validation loss across 30 epochs.

Figure 3.3 — Confusion matrix on the held-out test set (10 digit classes).
 Figure 3.4 — Sample test images with true vs. predicted labels.

3.7 Discussion
The model converged smoothly, with validation accuracy tracking closely to (and briefly exceeding)
training accuracy — a direct effect of the Dropout layers and on-the-fly augmentation, which regularize
the network and prevent it from memorizing the small training set. The final test accuracy of 98.9%
confirms strong generalization. The confusion matrix shows misclassifications concentrated among
visually similar digit pairs (e.g., 3/8, 4/9), which is expected behavior for handwritten digit recognition
even with high-capacity models.
Task 3: Mini Project — High-Imbalance Clinical Disease Predictor Pipeline

4.1 Objective
The objective of this mini-project is to architect an end-to-end classification system that detects early
physiological abnormalities (malignant tumors) within a highly skewed diagnostic dataset, prioritizing
recall (sensitivity) so that disease-positive cases are not missed — a critical requirement in clinical
screening contexts where false negatives carry high risk.

4.2 Dataset & Imbalance Simulation

The Wisconsin Breast Cancer Diagnostic dataset (scikit-learn built-in, sourced from the UCI Machine
Learning Repository) was used, comprising 30 numeric features computed from digitized images of fine
needle aspirate (FNA) breast masses. To emulate a genuinely high-imbalance clinical screening scenario
(rather than the dataset's native ~1.7:1 ratio), the malignant (disease-positive) class was downsampled
to 60 records against 357 benign records, producing an imbalance ratio of approximately 5.95 : 1.

Split Benign (0) Malignant (1) Total

Full dataset (post-simulation) 357 60 417

Training set 267 45 312

Test set (held out) 90 15 105

4.3 Methodology
An imbalanced-learn Pipeline was constructed for each candidate model, chaining feature scaling,
SMOTE (Synthetic Minority Over-sampling Technique) applied only within each training fold (to avoid
test-set leakage), and the classifier itself. Three classifiers were tuned via 5-fold stratified cross-validated
GridSearchCV, optimizing directly for recall:

• Support Vector Machine (RBF kernel) — tuned over C, gamma, class_weight.

• Random Forest — tuned over n_estimators, max_depth, min_samples_leaf, class_weight.
• XGBoost — tuned over n_estimators, max_depth, learning_rate, scale_pos_weight.

4.4 Key Code

pipeline = ImbPipeline([
("scaler", StandardScaler()),
("smote", SMOTE(random_state=42)),
("clf", RandomForestClassifier(random_state=42))
])
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
grid = GridSearchCV(pipeline, param_grid, cv=cv,
scoring="recall", n_jobs=-1)
 [Link](X_train, y_train)

cv_res = cross_validate(grid.best_estimator_, X_train, y_train, cv=cv,

scoring=["recall","precision","f1","roc_auc","accuracy"])

4.5 Results
5-fold cross-validated performance (on training data, best hyperparameters):

Model CV Recall CV Precision CV F1 CV ROC-AUC

SVM (RBF) 0.978 ± 0.044 0.904 0.938 0.992

Random Forest 0.956 ± 0.089 0.935 0.944 0.986

XGBoost 0.978 ± 0.044 0.679 0.792 0.985

Held-out test-set performance (never seen during tuning):

Model Recall Precision F1-score Accuracy ROC-AUC

SVM (RBF) 0.867 0.867 0.867 0.962 0.990

Random Forest 0.867 1.000 0.929 0.981 0.996

XGBoost 0.933 0.667 0.778 0.924 0.980

Best model by recall (sensitivity) on the held-out test set: XGBoost, correctly identifying 14 of 15
malignant cases (93.3% recall), at the cost of a higher false-positive rate (precision 66.7%). Random
Forest offered the best balance overall — identical recall to SVM with perfect precision and the highest
ROC-AUC (0.996).

Figure 4.1 — Class distribution in the training set before and after SMOTE oversampling.
Figure 4.2 — Test-set recall (sensitivity) achieved by each tuned classifier.

Figure 4.3 — ROC curves for all three classifiers on the test set.
 Figure 4.4 — Confusion matrices for SVM, Random Forest, and XGBoost on the test set.

4.6 Discussion
The results illustrate the classic precision–recall trade-off inherent to imbalanced disease-screening
problems. XGBoost, driven by an aggressive scale_pos_weight of 3 combined with SMOTE, was tuned to
prioritize catching malignant cases and achieved the highest sensitivity, but at the expense of more false
positives an acceptable trade-off in a screening context where a missed malignant case (false negative)
is far costlier than an extra confirmatory test (false positive). Random Forest, in contrast, achieved both
high recall and perfect precision on this particular test split, making it the strongest overall model by
ROC-AUC. In a real clinical deployment, the final threshold and model choice would be selected in
consultation with domain experts, weighing the relative costs of false negatives versus false positives,
and validated on a larger, prospectively collected patient cohort rather than a single held-out split.

4.7 Conclusion
This mini-project demonstrates a complete, reproducible pipeline for high-imbalance clinical
classification: SMOTE-based resampling embedded correctly inside cross-validation folds (avoiding data
leakage), recall-focused hyperparameter tuning across three model families, and multi-metric evaluation
(recall, precision, F1, ROC-AUC) on a genuinely held-out test set. All three tuned models exceeded 86%
recall despite the ~6:1 class imbalance, confirming that the combination of synthetic oversampling and
class-weighting successfully mitigated the minority-class detection problem that a naive classifier (which
would default to always predicting the majority/benign class) would otherwise suffer from.