LAB 1:

Data Loading, Cleaning, Preprocessing & Visualization

AIM
To load, clean, preprocess, and visualize student performance and weather datasets using Pandas, NumPy,
and Matplotlib.
SOFTWARE / LIBRARIES REQUIRED
Python 3.x, Jupyter Notebook / Google Colab, NumPy, Pandas, Matplotlib, Seaborn (optional for
visualization).

THEORY (4 Paragraphs)
Machine learning workflows begin with collecting and preparing data, because raw datasets usually
contain missing values, inconsistent entries, noise, and irrelevant attributes. Proper data
preprocessing ensures that algorithms can learn patterns correctly and efficiently. Data loading
involves importing datasets from CSV, Excel, or databases and converting them into structured formats
like Pandas Data Frames for easier manipulation.
Data cleaning is the process of handling missing data, duplicates, outliers, and incorrect formats.
Techniques such as imputation, normalization, and type conversion improve dataset quality. Without
cleaning, the learning models may become inaccurate or unstable due to noise and inconsistencies.
Cleaning also includes understanding dataset structure using descriptive statistics.
Preprocessing includes feature scaling, encoding categorical variables, and splitting datasets into training
and test sets. Scaling standardizes numeric features so that no feature dominates due to a larger
magnitude. Label encoding or one-hot encoding converts categorical data into numeric form so that ML
algorithms can process them.
Data visualization is a crucial step for analyzing patterns, distributions, and relationships in the
dataset. Matplotlib and Seaborn help display graphs like histograms, scatter plots, correlation
heatmaps, and bar charts. Visualization enables interpreting trends such as factors affecting student
marks or weather parameters over time, making it easier to understand the dataset before model
building.
PROCEDURE WITH CODE
CONCLUSION
In this lab, we successfully loaded, cleaned, and preprocessed real-world datasets using Pandas and
NumPy. We removed missing values, duplicates, and encoded categorical attributes. Visualization using
Matplotlib and Seaborn revealed important patterns such as subject correlations and weather trends.
These steps form the foundation of any machine learning pipeline and greatly improve model
performance.
 LAB 2:
Linear, Multiple Linear & Logistic Regression Models
AIM
To develop and compare Linear Regression, Multiple Linear Regression, and Logistic Regression models
using housing, academic performance, and breast cancer datasets.
SOFTWARE / LIBRARIES REQUIRED
Python 3.x NumPy

Pandas
Matplotlib
Scikit-Learn (sklearn) Seaborn

THEORY
Regression analysis is a core supervised learning technique used to model relationships between inputs
and outputs. Linear regression predicts continuous values by fitting a straight line through data
points. It assumes a linear relationship between independent and dependent variables. Multiple linear
regression extends this concept by using several features to improve prediction accuracy.
Logistic regression, unlike linear regression, is used for classification problems. It predicts a
probability between 0 and 1 using the logistic (sigmoid) function. It is commonly used in medical
diagnosis, spam detection, and binary decision problems. Although called regression, it is
fundamentally a classification algorithm.
Datasets like housing price data help in building linear and multiple regression models. Features like
area, number of rooms, and location influence housing cost. Academic performance datasets allow
modeling student marks using various attributes like study time, attendance, and socio-economic
factors. These models help understand factors that significantly affect outcomes.
Classification datasets such as the breast cancer dataset allow logistic regression to classify tumors into
malignant or benign categories. The model evaluates feature importance, predicts risk, and generates
accuracy, precision, recall, and confusion matrices. Comparing regression and classification models helps
understand when to use each method effectively.
CONCLUSION
Linear regression predicts outcomes using a single feature, whereas multiple linear regression uses
multiple attributes to improve accuracy. Logistic regression helps classify data into categories such
as malignant or benign. The results show that multiple regression improves prediction performance
compared to simple linear regression, while logistic regression provides high accuracy for classification
tasks. These models are essential for predictive machine learning applications.
 LAB 3:
Bias–Variance Tradeoff using Polynomial Regression (UCI Energy Dataset)
AIM
To investigate the bias–variance tradeoff by fitting polynomial regression models of varying degrees to
the UCI Energy Efficiency dataset and analyzing model performance.
SOFTWARE/ LIBRARIES REQUIRED
Python 3.x

Jupyter Notebook / Google Colab Pandas

NumPy
Matplotlib
Scikit-Learn (sklearn)

THEORY (5–6 Paragraphs)

The bias–variance tradeoff is one of the most fundamental concepts in machine learning. It explains
how model complexity affects prediction performance. A model with high bias oversimplifies
relationships in the data, leading to underfitting. In contrast, a model with high variance becomes too
complex, capturing noise instead of true patterns, leading to overfitting. Balancing bias and variance
ensure good generalization on unseen data.
Polynomial regression extends linear regression by including polynomial terms of the features.
Increasing polynomial degree increases complexity, allowing the model to fit more intricate patterns.
Lower-degree polynomials may underfit, while very high-degree polynomials may overfit by conforming
too closely to training data. This makes polynomial regression ideal for studying bias and variance.
The UCI Energy Efficiency dataset contains building energy consumption values along with factors such
as surface area, glazing area, and roof area. Predicting heating load using polynomial regression
demonstrates how feature transformations affect performance. By training polynomial models of
degrees 1 to 10, we observe how training and testing errors evolve with complexity.
Training error typically decreases as model complexity increases because the model can better fit the
training data. However, test error decreases only up to a certain degree, after which it rises due to
overfitting. This gap between training and test error is evidence of variance dominating the model’s
behavior.
Visualization of errors using line plots helps in understanding the tradeoff clearly. The optimal model
is one that maintains a good balance—moderate complexity with
low test error. Polynomial regression thus provides an excellent experimental setup to explore bias and
variance in practice.
CONCLUSION
Polynomial regression on the Energy Efficiency dataset demonstrated the bias– variance tradeoff. Low-
degree models underfit while high-degree models overfit. The optimal polynomial degree minimized
test error, proving the importance of balancing complexity for good generalization.
 LAB 4:
Decision Tree Classifier on Iris Dataset + Confusion Matrix
AIM
To implement a Decision Tree classifier on the Iris dataset and evaluate its performance using a
confusion matrix.
SOFTWARE/ LIBRARIES REQUIRED
Python 3.x Jupyter / Colab Pandas NumPy Matplotlib /
Seaborn
Scikit-Learn

THEORY
Decision Trees are supervised machine learning algorithms used for both classification and regression.
They work by recursively splitting the dataset based on feature values to create a tree-like model of
decisions. Each internal node represents a test on a feature, each branch a decision outcome, and
each leaf node a final prediction.

The algorithm selects the best feature to split data using impurity measures such as Gini Index or

Entropy. A good split separates classes more cleanly, reducing uncertainty. The tree continues growing

until stopping conditions are met, such as maximum depth or minimal samples per leaf.

Decision Trees are easy to interpret because the model mimics human decision- making. They do not
require feature scaling and can handle both numerical and categorical data. However, they can overfit
by growing too deep, capturing noise in the training data.

The Iris dataset, containing setosa, versicolor, and virginica species, is ideal for evaluating classification

performance. Measuring accuracy alone is insufficient because misclassifications between classes should be

analyzed in detail. A confusion matrix provides a deeper evaluation by showing true vs. predicted class

counts.

This matrix reveals which classes are correctly identified and where mistakes occur. Decision Trees
generally perform well on simple datasets like Iris, providing high accuracy with interpretable
decision rules.
CONCLUSION
The Decision Tree classifier achieved high accuracy on the Iris dataset. The confusion matrix
demonstrated that most predictions were correct across all species. This shows that decision trees
perform reliably on structured datasets and are valuable for interpretable machine learning tasks.
 LAB 5: SVM Classifier with Linear, Polynomial & RBF Kernels (Iris Dataset)
AIM
To implement and compare SVM classifiers using different kernels (linear, polynomial, RBF) on the Iris
dataset and evaluate accuracy using confusion matrices.
SOFTWARE / LIBRAIES REQUIRED
Python 3.x
Jupyter / Colab NumPy

Pandas
Matplotlib / Seaborn Scikit-Learn

THEORY
Support Vector Machines (SVMs) are powerful supervised learning algorithms used mainly for
classification. The core idea is to find an optimal separating hyperplane that maximizes the margin
between classes. The points closest to this hyperplane are called support vectors and are crucial in
defining the model.
SVMs can handle linearly and non-linearly separable data using kernels. A kernel transforms input
features into a higher-dimensional space where classes become more separable. The linear kernel is
suitable for linearly separable data and trains faster than other kernels.

The polynomial kernel maps data into a higher-power polynomial feature space, enabling the model to

capture non-linear boundaries. The degree of the polynomial controls the complexity of the model. This

kernel is useful when relationships between variables are moderately complex.

The RBF (Radial Basis Function) kernel, also known as the Gaussian kernel, is the most powerful and widely

used. It maps data into infinite-dimensional space, allowing the classifier to form highly flexible decision

boundaries. RBF handles complex class distributions extremely well.

Evaluating SVM performance requires analyzing confusion matrices for each kernel. Accuracy alone does
not reveal class-specific misclassifications. Iris dataset, with three classes, helps compare kernel behavior
effectively and determine which kernel generalizes best.
CONCLUSION
SVM classifiers with different kernels were implemented and compared. The RBF kernel achieved the
highest accuracy due to its ability to model complex class boundaries. Linear and polynomial kernels
performed well but slightly below RBF. Confusion matrices demonstrated that RBF kernel misclassified
the fewest samples, making it the most effective kernel for Iris dataset classification.
 LAB 6:
Impact of Bagging, Boosting & Stacking on Decision Tree Performance (Pima Diabetes Dataset)
AIM
To analyze how Bagging, Boosting, and Stacking ensemble techniques affect the performance of a
Decision Tree classifier on the Pima Diabetes dataset.
SOFTWARE REQUIRED
Python 3.x Jupyter Notebook / Google Colab NumPy Pandas
Scikit-Learn Matplotlib / Seaborn

THEORY
Ensemble learning combines multiple machine learning models to improve predictive performance. A
single decision tree is simple and interpretable but may suffer from high variance or low accuracy.
Ensemble techniques like bagging, boosting, and stacking help overcome these limitations by
aggregating the predictions of several weak learners.
Bagging, short for Bootstrap Aggregating, reduces variance by training multiple decision trees on
randomly sampled subsets of the data. Each tree learns slightly different patterns, and their
combined prediction (usually through voting) becomes more stable. The most popular implementation
is the Random Forest algorithm, but standalone Bagging Classifier also works effectively for
performance improvement.
Boosting takes a different approach by sequentially training weak learners. Each new learner focuses on
correcting the mistakes of the previous one. Boosting reduces bias and variance, often resulting in
highly accurate models. Algorithms such as AdaBoost and Gradient Boosting are widely used and
particularly effective on noisy, moderate-sized datasets.
Stacking is an advanced ensemble method that combines predictions of multiple base models using
a meta-learner. Instead of simple majority voting, stacking trains a second-level model to learn how to
optimally combine base model outputs. This approach often leads to superior accuracy because it
learns relationships between model predictions.
The Pima Diabetes dataset, containing medical diagnostic measurements, is suitable for analyzing
ensemble methods because its classification task is challenging and often noisy. By applying bagging,
boosting, and stacking on the base decision tree classifier, we can observe improvements in accuracy
and robustness
CONCLUSION
Bagging, Boosting, and Stacking all enhanced the performance of the baseline Decision Tree
classifier. Bagging reduced variance, boosting improved accuracy by focusing on difficult samples, and
stacking achieved the best overall performance by combining multiple learners. Ensemble methods
significantly improve model robustness and generalization on the Pima Diabetes dataset.
 LAB 7:
Comparison of PCA and LDA for Dimensionality Reduction (UCI Wine Dataset)
AIM
To compare the performance of Principal Component Analysis (PCA) and Linear Discriminant Analysis
(LDA) for dimensionality reduction on the UCI Wine dataset.
SOFTWARE REQUIRED
Python 3.x NumPy Pandas Matplotlib Scikit-Learn

Seaborn
THEORY
Dimensionality reduction is essential in machine learning to simplify datasets, reduce noise, and
improve model performance. Many real-world datasets contain multiple correlated features, making
visualization and classification difficult. Techniques like
PCA and LDA project data into lower-dimensional spaces while retaining the most important
information.
Principal Component Analysis (PCA) is an unsupervised technique that transforms data into components
capturing maximum variance. It does not use class labels, meaning it simply compresses data based on
overall structure. PCA is widely used for preprocessing, noise reduction, and feature extraction in high-
dimensional datasets.
Linear Discriminant Analysis (LDA), unlike PCA, is a supervised method that uses class labels to maximize
class separability. LDA finds directions that best separate classes rather than just maximizing variance.
This makes LDA more effective for classification tasks.
The UCI Wine dataset contains 13 chemical attributes used to classify wines into three categories. With
multiple correlated attributes, it provides a good test case for dimensionality reduction. By applying PCA
and LDA to reduce dimensions, we can visualize the dataset and compare how separable the
transformed features are.
Comparing PCA and LDA shows the strengths of each method. PCA is useful for generalized structure
discovery, whereas LDA is better for classification and visualization of clusters. Evaluating classification
accuracy using transformed data helps determine which method performs better for the Wine dataset.
CONCLUSION
PCA and LDA were applied to the UCI Wine dataset for dimensionality reduction. PCA captured maximum
variance but did not fully separate classes due to its unsupervised nature. LDA, using class labels,
provided superior class separation and higher classification accuracy. Therefore, LDA is more effective
than PCA for supervised classification tas
23/