SWE 507: Machine Learning

Lab Practical File

Submitted towards the partial fulfilment of
the requirements of the award of the degree
of
Master of Technology
In
Software Engineering

Submitted by
Prince Verma
25/SWE/24
I Sem, I Year

Submitted to
Dr. Sanjay Patidar
Associate Professor
Department of Software Engineering

Delhi Technological University

(FORMERLY Delhi College of Engineering)
Bawana Road, New Delhi - 110042
November, 2025

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 1

 Index

Sr. No. Experiment Name Page No. Date Remarks

1. Using Python libraries perform data 4 13/08/25

cleaning, feature selection, finding null
values and analysis through plotting
different graphs between features on
Housing dataset(Housing)

2. Write a Python program to demonstrate 11 20/08/25

various Data Visualization Techniques on
(Housing and used_cars Datasets

3. Implement Simple and Multiple Linear 19 27/08/25

Regression Model on the dataset used_cars
and Housing.

4. Develop a Logistic Regression Model for a 26 03/09/25

Dataset (customer_churn_testing_master)
and analyze the performance Metrics.

5. Write a python program to implement KNN 32 10/09/2025

Algorithm on dataset emails and
mobile_price and analyse performance
metrics.

6. Write a Python Program to implement 38 17/09/2025

K-Means clustering on the given
Titanic-Dataset.

7. Write a Program to demonstrate the working 45 01/10/2025

of decision tree based ID3 algorithm. Use
dataset ds_salaries for building the decision
tree and classify the new sample.

8. Write a python program to implement 50 08/10/2025

Kmeans clustering algorithm on any of the
given datasets

9. Write a program to construct a Bayesian 55 15/10/2025

network considering medical data. Use this
model to demonstrate the diagnosis of
cancer patients using standard Cancer
Disease Dataset.

10. Build an Artificial Neural Network by 61 29/10/2025

implementing the Back-Propagation
algorithm and test using dataset
(earthquakes_2023_global and Test) and
analyse performance matrix

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 2

 APPENDIX I

S. No. Dataset

1. DATASET 1: CUSTOMER CHURN DATASET

2. DATASET 2: CHRONIC KIDNEY DISEASE DATASET

3. DATASET 3: SPAM CLASSIFICATION DATASET

4. DATASET 4: CAR PRICE PREDICTION DATASET

5. DATASET 5: BIG MART SALES DATASET

6. DATASET 6: TITANIC DATASET

7. DATASET 7: EARTHQUAKE 2023 DATASET

8. DATASET 8: MOBILE PRICE DATASET

9. DATASET 9: HOUSING PRICE DATASET

10. DATASET 10: 2023 DATA SCIENTIST SALARY DATASET

11. DATASET 11: BREAST CANCER DATASET

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 3

 Experiment 1

Aim:
Using Python libraries perform data cleaning, feature selection, finding null values and analysis
through plotting different graphs between features on Housing dataset(Housing).

Theory:

Data analysis is a systematic approach to examining datasets to extract useful insights and
patterns. In real-world datasets, raw data often contains inconsistencies, missing values,
irrelevant features, or noisy information that can affect the accuracy of any analytical or
predictive modeling. Data cleaning and feature selection are fundamental steps to ensure
high-quality analysis.

1.​ Data Cleaning:​

○​ The process of identifying and correcting errors or inconsistencies in the data.​

○​ Common steps include handling missing values, removing duplicates, fixing

incorrect data types, and standardizing formats.​

○​ Missing values can be handled by deletion, mean/median/mode imputation, or

using predictive models.​

2.​ Feature Selection:​

○​ The process of identifying the most important features (columns) in a dataset that
contribute significantly to the predictive or analytical task.​

○​ Reduces dimensionality, improves model performance, and makes analysis more

interpretable.​

○​ Techniques include correlation analysis, variance thresholding, and statistical

tests.​

3.​ Finding Null Values:​

○​ Missing values are often represented as NaN in Python.​

○​ Libraries like Pandas provide methods like .isnull().sum() to quickly identify

missing data in each column.​

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 4

 ○​ Visualization using heatmaps or bar charts can provide a clearer picture of
missing data patterns.​

4.​ Data Analysis and Visualization:​

○​ Visual representation of data helps in understanding relationships between

features.​

○​ Python libraries like Matplotlib, Seaborn, and Plotly provide tools to create
scatter plots, histograms, boxplots, bar charts, pair plots, etc.​

○​ Analysis can reveal trends, correlations, outliers, and patterns which can guide
further modeling or decision-making.​

5.​ Python Libraries Commonly Used:​

○​ Pandas: For data manipulation and cleaning.​

○​ NumPy: For numerical computations.​

○​ Matplotlib / Seaborn: For plotting and visualization.​

○​ Scikit-learn: For feature selection and preprocessing techniques.

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 5

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 6
Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 7
Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 8
Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 9
Conclusion

By performing data cleaning, null value detection, feature selection, and visualization in Python,
the dataset becomes more reliable, consistent, and interpretable.

●​ Data Cleaning ensures the removal of inconsistencies and missing data, which prevents
errors during analysis.​

●​ Feature Selection focuses on the most relevant variables, reducing noise and improving
analytical efficiency.​

●​ Visualization helps identify relationships, trends, and anomalies between features,

providing insights that may not be obvious from raw data alone.​

Overall, this approach improves the quality of data-driven decisions and forms the foundation for
further predictive modeling or machine learning tasks. Python’s libraries make this process
efficient and user-friendly, allowing analysts to gain meaningful insights quickly.

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 10

 Experiment 2
Aim : Write a Python program to demonstrate various Data Visualization Techniques on
Housing and used_cars.​
​
Theory:

Data visualization is a crucial part of data analysis that involves representing data in a visual
format to make patterns, trends, and relationships easier to understand. Python provides powerful
libraries to perform data visualization on datasets efficiently.

1. Importance of Data Visualization

●​ Helps in identifying patterns, trends, and correlations between variables.​

●​ Makes complex data easier to interpret and communicate.​

●​ Aids in detecting outliers, anomalies, and missing data.​

●​ Supports better decision-making based on insights extracted from visual analysis.​

2. Python Libraries for Data Visualization

●​ Matplotlib: A fundamental library to create line plots, bar charts, scatter plots,
histograms, and more.​

●​ Seaborn: Built on Matplotlib, it simplifies advanced visualization with aesthetically

pleasing plots like heatmaps, pair plots, and violin plots.​

●​ Plotly: Enables interactive and dynamic visualization for web-based applications.​

●​ Pandas Visualization: Quick plotting capabilities directly from DataFrames for

exploratory analysis.​

3. Common Data Visualization Techniques

1.​ Line Plot: Used to show trends over time or continuous variables.​

2.​ Bar Chart: Useful for comparing categorical data.​

3.​ Histogram: Shows the distribution of a single variable.​

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 11

 4.​ Scatter Plot: Displays relationships between two numerical variables.​

5.​ Box Plot: Visualizes the spread and outliers of a dataset.​

6.​ Heatmap: Shows correlations or missing values in a dataset.​

7.​ Pair Plot: Provides a matrix of scatter plots to observe pairwise relationships between
features.​

4. Application on Datasets

●​ By applying these visualization techniques on two given datasets, we can compare and
contrast patterns.​

●​ Visualization allows us to identify trends, correlations, and differences between the

datasets, providing actionable insights.​

●​ It also helps in detecting any inconsistencies, missing values, or outliers that may require
further data preprocessing.

IMPLEMENTATION:

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 12

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 13
Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 14
Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 15
Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 16
Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 17
Conclusion

Data visualization is an essential step in the data analysis workflow. By using Python libraries
like Matplotlib, Seaborn, and Plotly:

●​ We can easily explore and understand datasets, revealing hidden patterns and
relationships.​

●​ Different visualization techniques allow us to compare features, identify trends, and

spot anomalies.​

●​ Visualization enhances the interpretability of data and helps in communicating insights

effectively.​

●​ Applying multiple visualization methods on two datasets helps in drawing meaningful

comparisons, which is critical for data-driven decision-making.​

Overall, data visualization transforms raw data into a clear, understandable, and actionable
form, enabling analysts and stakeholders to make informed decisions.

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 18

 Experiment 3
AIM: Implement Simple and Multiple linear regression on the dataset used_cars and Housing.

Theory:

Linear Regression

Linear Regression is a fundamental statistical and machine learning technique used to model the
relationship between a dependent variable and one or more independent variables. The goal is to
find a linear equation that best predicts the dependent variable based on the values of the
independent variables.

Simple Linear Regression (SLR)

Simple Linear Regression models the relationship between a single independent variable
(feature) and a dependent variable (target) by fitting a linear equation to the observed data.

●​ Equation: y = β₀ + β₁*x + ε
○​ y: Dependent variable (Target)
○​ x: Independent variable (Feature)
○​ β₀: y-intercept (Bias term)
○​ β₁: Slope (Coefficient of the feature x)
○​ ε: Random error term
●​ Objective: To find the values of β₀ and β₁ that minimize the difference between the actual
observed values and the values predicted by the model.

Multiple Linear Regression (MLR)

Multiple Linear Regression extends SLR by modeling the relationship between two or more
independent variables and a single dependent variable.

●​ Equation: y = β₀ + β₁*x₁ + β₂*x₂ + ... + β *x + ε

○​ y: Dependent variable (Target)
○​ x₁, x₂, ..., x : Independent variables (Features)
○​ β₀: y-intercept
○​ β₁, β₂, ..., β : Coefficients for each feature
○​ ε: Error term
●​ Objective: Similar to SLR, but it finds the best-fit hyperplane in a multi-dimensional
space.

The Concept of "Best Fit"

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 19

The most common method to find the best-fit line is Ordinary Least Squares (OLS). OLS works
by minimizing the sum of the squares of the differences (residuals) between the observed values
and the values predicted by the model.

●​ Residual (eᵢ): eᵢ = y_actualᵢ - y_predictedᵢ

●​ Cost Function (to be minimized): Sum of Squared Residuals (SSR) = Σ(y_actualᵢ -
y_predictedᵢ)²

Evaluation Metrics

To assess the performance of a linear regression model, we use several metrics:

1.​ R-squared (R²): Represents the proportion of the variance in the dependent variable that
is predictable from the independent variables. A value closer to 1 indicates a better fit.
2.​ Mean Squared Error (MSE): The average of the squared differences between predicted
and actual values. Lower values are better.
3.​ Root Mean Squared Error (RMSE): The square root of MSE. It is in the same units as the
target variable, making it more interpretable.
4.​ Mean Absolute Error (MAE): The average of the absolute differences between predicted
and actual values. It is less sensitive to outliers than MSE/RMSE.

Methodology

The implementation of linear regression follows a structured machine learning pipeline.

Data Preprocessing

This is a critical first step to prepare the data for modeling.

1.​ Data Loading: Import the dataset (used_cars.csv and [Link]) into a Pandas
DataFrame.
2.​ Data Cleaning:
○​ Handle missing values using appropriate methods (e.g., mean/median imputation,
or dropping rows/columns).
○​ Identify and treat outliers if necessary, as they can significantly skew the
regression line.
3.​ Exploratory Data Analysis (EDA):
○​ Use [Link]() and [Link]() to understand the data structure.
○​ Create visualizations like scatter plots (for SLR) and correlation heatmaps (for
MLR) to understand relationships between variables.
4.​ Feature Encoding: Convert categorical variables (e.g., car brand, fuel type, location) into
numerical format using techniques like One-Hot Encoding or Label Encoding.

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 20

 5.​ Feature Scaling/Normalization: For Multiple Linear Regression, it is often beneficial to
scale features (e.g., using StandardScaler or MinMaxScaler) so that they contribute
equally to the model.
6.​ Train-Test Split: Split the dataset into a training set and a testing set (e.g., 80% train, 20%
test). This ensures the model is evaluated on unseen data to check for generalization.

Implementation:

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 21

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 22
Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 23
Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 24
Conclusion:

Both simple and multiple linear regression models were implemented on Used Cars and Housing
datasets. The models were evaluated using R² Score and Mean Squared Error. Results show that
multiple regression captures more relationships between variables, generally leading to better
performance.

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 25

 Experiment 4

Theory

Logistic Regression is a statistical and machine learning algorithm used for binary classification
problems. Unlike linear regression which predicts a continuous value, logistic regression predicts
the probability that a given instance belongs to a particular category.

The core of the model is the logistic function (also called the sigmoid function), which maps any
real-valued number into a value between 0 and 1. This output is interpreted as a probability.

The model works by first calculating a weighted sum of the input features (similar to linear
regression). This output is then passed through the sigmoid function. A threshold (typically 0.5)
is applied to this probability to make the final class prediction. For example, if the predicted
probability is >= 0.5, the instance is classified as the positive class (e.g., "Churn"), otherwise as
the negative class (e.g., "No Churn").

The model is trained by optimizing its coefficients (weights) to minimize a cost function,
typically log loss, which penalizes wrong predictions based on how confident and incorrect they
were.

Methodology

Data Preprocessing: The dataset will first be loaded and inspected for missing values and
inconsistencies. Missing values will be handled through imputation or removal. Categorical
variables (e.g., gender, internet service type) will be converted into a numerical format using
techniques like one-hot encoding.

Feature Scaling: Numerical features with different scales (e.g., tenure, monthly charges) will be
standardized or normalized. This ensures that no single feature dominates the model's learning
process due to its scale.

Feature-Target Split: The dataset will be divided into a matrix of features (independent variables)
and a vector for the target variable (dependent variable), which is 'Churn'.

Train-Test Split: The data will be split into a training set and a testing set. The training set is used
to train the logistic regression model, and the testing set is reserved to evaluate its performance
on unseen data.

Model Training: The Logistic Regression algorithm will be implemented on the training data.
This process involves finding the optimal coefficients that minimize the log loss function.

Prediction and Performance Analysis: The trained model will be used to make predictions on the
test set. Performance will be evaluated using several metrics:
Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 26
 ●​ Confusion Matrix: To visualize True Positives, True Negatives, False Positives, and False
Negatives.
●​ Accuracy: The overall proportion of correct predictions.
●​ Precision: The proportion of positive predictions that were actually correct.
●​ Recall (Sensitivity): The proportion of actual positives that were correctly identified.
●​ F1-Score: The harmonic mean of Precision and Recall, providing a single balanced
metric.
●​ ROC Curve and AUC: The Receiver Operating Characteristic curve and the Area Under
the Curve will be plotted to assess the model's ability to distinguish between classes
across different thresholds.

Implementation:

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 27

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 28
Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 29
Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 30
Conclusion

This project successfully developed a Logistic Regression model to predict customer churn. The
methodology involved a standard machine learning pipeline from data preprocessing to model
evaluation.

The performance metrics, particularly the confusion matrix and the ROC-AUC score, will
provide a comprehensive understanding of the model's strengths and weaknesses. The model's
interpretability is a key advantage, as the coefficients can reveal which features (e.g., contract
type, monthly charges) are most predictive of churn.

Based on the analysis, it will be concluded whether logistic regression is a suitable model for this
dataset. The results will offer actionable insights for the business to identify at-risk customers
and develop targeted retention strategies.

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 31

 Experiment 5

AIM: Write a python program to implement KNN Algorithm on dataset emails and mobile_price
and analyse performance metrics.

Theory

The k-Nearest Neighbors (KNN) algorithm is a simple, instance-based, and non-parametric

supervised machine learning algorithm used for both classification and regression tasks. Its core
principle is that similar data points exist in close proximity. In other words, it operates on the
assumption that things that are alike are near to each other.

For a given new, unlabeled data point, the KNN algorithm identifies the 'k' number of training
examples that are closest to it in the feature space. The distance is typically calculated using
measures like Euclidean or Manhattan distance.

●​ For Classification: The algorithm takes a majority vote among the 'k' nearest neighbors.
The class that appears most frequently among these neighbors is assigned to the new data
point.
●​ For Regression: The algorithm calculates the average (or weighted average) of the target
values of the 'k' nearest neighbors, and this value is assigned to the new data point.

The choice of 'k' is crucial. A small 'k' can make the model sensitive to noise (overfitting), while
a very large 'k' can make the model too general, potentially ignoring important local patterns
(underfitting).

Methodology

This section outlines the steps to implement the KNN algorithm on the provided datasets.

Data Preprocessing:

●​ Loading Data: The datasets ([Link] and mobile_price.csv) will be loaded into Pandas
DataFrames.
●​ Exploratory Data Analysis (EDA): Initial analysis will be performed to understand the
data structure, check for missing values, and examine the distribution of the target
variable.
●​ Feature Selection/Engineering: Relevant features will be selected. Irrelevant columns like
'ID' will be dropped.
●​ Handling Categorical Data: If any categorical features are present, they will be converted
into numerical format using appropriate techniques like Label Encoding or One-Hot
Encoding.
Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 32
 ●​ Feature Scaling: The KNN algorithm is distance-based, making it essential to scale the
features to a standard range (e.g., using StandardScaler or MinMaxScaler) to prevent
features with larger magnitudes from dominating the distance calculation.
●​ Train-Test Split: The cleaned dataset will be split into a training set (to fit the model) and
a testing set (to evaluate its performance), typically using an 80-20 or 70-30 split.

Model Implementation:

●​ The KNN classifier will be implemented using the KNeighborsClassifier class from the
[Link] library.
●​ The model will be trained (fitted) on the scaled training data.

Hyperparameter Tuning:

●​ The optimal value of 'k' (number of neighbors) will be determined using techniques like
GridSearchCV or by plotting the model's accuracy for a range of 'k' values and selecting
the one with the highest performance on the validation set.

Performance Analysis:

●​ Predictions will be made on the scaled test set.

●​ The model's performance will be evaluated using the following metrics:
○​ Accuracy: To measure the overall correctness of the model.
○​ Confusion Matrix: To visualize the performance and understand the types of
errors (True Positives, False Positives, True Negatives, False Negatives).
○​ Classification Report: To obtain key metrics such as Precision, Recall, and
F1-Score for each class, which are crucial for imbalanced datasets.
●​ This entire process will be repeated for both the emails (e.g., spam/ham classification)
and mobile_price (e.g., price range classification) datasets.

Implementation:

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 33

 ​

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 34

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 35
Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 36
Conclusion

The implementation of the KNN algorithm on the two datasets will provide a practical
understanding of its application in different classification contexts, such as email filtering and
product categorization.

The performance metrics (Accuracy, Precision, Recall, F1-Score) will be analyzed to draw
conclusions about the model's effectiveness for each specific task. The analysis will highlight the
importance of data preprocessing, especially feature scaling, and the critical role of selecting the
right 'k' value.

A comparison of the results between the two datasets will illustrate how the nature of the data
(e.g., feature types, class distribution) influences the performance of the KNN algorithm. Finally,
the conclusion will summarize the strengths of KNN, such as its simplicity and intuitiveness, and
its limitations, including computational cost for large datasets and sensitivity to irrelevant
features.

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 37

 Experiment 6
AIM: Write a Program to implement the Support Vector Machine for a training dataset
(earthquakes_2023_global) set stored as a .CSV file . Compute accuracy with few test datasets.

Theory

Support Vector Machine (SVM) is a powerful supervised machine learning algorithm primarily
used for classification tasks. The core idea behind SVM is to find the optimal hyperplane that
best separates data points of different classes in a high-dimensional feature space.

The "optimal" hyperplane is chosen as the one with the maximum margin, which is the greatest
possible distance between the hyperplane and the nearest data points from any class. These
closest data points are called "support vectors," as they are the critical elements that define the
position and orientation of the hyperplane.

SVM is particularly effective in scenarios where the data is not linearly separable. It employs a
technique called the "kernel trick," which implicitly maps the input features into a
higher-dimensional space where a linear separation becomes possible. Common kernel functions
include the linear, polynomial, and Radial Basis Function (RBF) kernel.

Methodology

The implementation of the Support Vector Machine for the earthquake dataset will follow a
structured machine learning pipeline.

Data Preprocessing: The first step involves loading the dataset from the CSV file. Key features
relevant to earthquake characterization (such as latitude, longitude, depth, and magnitude) will
be selected. The target variable will be defined, for instance, classifying earthquakes based on
magnitude into categories like "Minor" and "Major." The data will be cleaned by handling
missing values and normalized to ensure all features contribute equally to the model.

Data Splitting: The preprocessed dataset will be divided into two subsets: a training set and a
testing set. The training set is used to teach the SVM model the underlying patterns in the data,
while the testing set is held back to evaluate the model's performance on unseen data.

Model Training: An SVM classifier will be instantiated, typically starting with an RBF kernel
due to its effectiveness for non-linear problems. The model will be trained (or "fitted") using the
training data. During this phase, the algorithm learns to find the optimal hyperplane based on the
provided features and target labels.

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 38

Implementation:

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 39

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 40
Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 41
Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 42
Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 43
Conclusion

In this project, the Support Vector Machine algorithm is applied to classify global earthquake
data. The methodology outlines a clear path from data preparation to model evaluation. By
following this approach, we can build a predictive model capable of categorizing seismic events.
The computed accuracy on the test dataset will serve as the primary metric to assess the model's
generalization capability and its potential utility in seismological analysis. The success of the
model will depend on the quality of the data and the appropriate selection of features and SVM
parameters.

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 44

 EXPERIMENT 7
AIM: Write a Program to demonstrate the working of decision tree based ID3 algorithm. Use
dataset ds_salaries for building the decision tree and classify the new sample.

Theory

The ID3 (Iterative Dichotomiser 3) algorithm is a foundational algorithm used to generate a

decision tree from a dataset. It employs a top-down, greedy search approach to build the tree,
meaning it always selects the attribute that appears best at the current step to split the data.

The "best" attribute is chosen using a metric called Information Gain, which is based on the
concept of Entropy.

●​ Entropy: This measures the level of impurity or uncertainty in a group of samples. If all
samples belong to the same class, the entropy is 0 (pure). If the samples are evenly split
among classes, the entropy is 1 (impure). It is calculated as:

​
Information Gain: This measures the reduction in entropy achieved by splitting the
dataset S on a particular attribute A. ID3 calculates the Information Gain for every
attribute and selects the one with the highest value as the node for the split.

The algorithm recursively builds the tree until a stopping criterion is met, such as all
samples at a node belonging to the same class or no more attributes being available to
split on.

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 45

Methodology

The implementation follows these key steps:

Data Preprocessing

1.​ Load Data: Read the ds_salaries dataset.

2.​ Feature Selection: Select relevant categorical features for the tree, such as
experience_level, employment_type, company_size, and remote_ratio.
3.​ Discretization: The ID3 algorithm requires a categorical target variable. The continuous
salary_in_usd column must be discretized (binned) into a set of classes (e.g., 'Low',
'Medium', 'High') to serve as our target class.
4.​ Handle Data: Remove or simplify attributes with too many unique values (like job_title)
that could lead to overfitting.

ID3 Algorithm Implementation

1.​ Start: Begin with the entire preprocessed dataset as the root node.
2.​ Calculate Base Entropy: Calculate the entropy of the target class (the discretized salary
bins) for the current set of samples.
3.​ Calculate Information Gain: For every feature (e.g., experience_level), calculate the
weighted average entropy of the subsets created by splitting on that feature. Subtract this
from the base entropy to get the Information Gain.
4.​ Select Best Attribute: Choose the attribute with the highest Information Gain as the
splitting attribute (decision node) for the current node.
5.​ Build Tree: Create a new branch for each unique value of the selected attribute.
6.​ Recurse: Recursively apply the same process (steps 2-5) to the subset of data in each
branch.
7.​ Stop: The recursion for a branch stops when all samples in that branch belong to the
same target class (creating a leaf node) or when there are no more attributes to split on.

Classification

1.​ Define New Sample: Create a new, unseen data sample (e.g., experience_level='SE',
company_size='M', remote_ratio=100).
2.​ Traverse Tree: Pass this sample through the generated decision tree. Start at the root and
follow the branches that match the sample's attribute values.
3.​ Predict: The leaf node reached at the end of the path provides the predicted salary class
(e.g., 'High') for the new sample.

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 46

Implementation:​

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 47

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 48
Conclusion

This program successfully demonstrated the working of the ID3 decision tree algorithm. By
preprocessing the ds_salaries dataset and applying the core logic of Entropy and Information
Gain, a classification model was built.

The process illustrated how ID3 greedily selects the most informative attributes to create a
simple, interpretable, rule-based tree. The final step of classifying a new sample confirmed the
tree's ability to make predictions based on the patterns learned from the data. The experiment
highlights the importance of data preprocessing, especially discretization, when using traditional
algorithms like ID3.

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 49

 Experiment 8
AIM: Write a Python Program to implement K-Means clustering on the given TitanicDataset.

K-Means Clustering: The Theory

K-Means is an unsupervised machine learning algorithm used for clustering. Its primary
purpose is to partition a dataset into $k$ distinct, non-overlapping subgroups (clusters) where
each data point belongs to the cluster with the nearest mean (cluster centroid).

The algorithm works iteratively to minimize the within-cluster sum of squares (WCSS), which
is the total squared distance between each point and its assigned cluster's centroid.

The main steps are:

1.​ Initialization: $k$ initial "centroids" (cluster centers) are chosen randomly or
strategically.
2.​ Assignment: Each data point is assigned to the nearest centroid, usually based on
Euclidean distance.
3.​ Update: The centroids are recalculated as the mean (average) of all data points assigned
to that cluster.

Steps 2 and 3 are repeated until the cluster assignments no longer change, meaning the algorithm
has converged.

For the Titanic dataset, K-Means is not used to predict survival. Instead, it is used for pattern
discovery. The goal is to identify natural groupings or "profiles" of passengers based on their
shared characteristics (e.g., "wealthy families," "young solo male travelers").

Methodology for Titanic Dataset

Applying K-Means to the Titanic dataset requires several critical pre-processing steps before the
model can be trained.

Data Preprocessing
1.​ Feature Selection: First, we select relevant numerical and categorical features. Good
candidates include Pclass, Age, Fare, Sex, SibSp (Siblings/Spouses Aboard), and Parch
(Parents/Children Aboard). Features like Name or Ticket are usually dropped as they are
not useful for distance calculations.
2.​ Handling Missing Data: Missing values, particularly in the Age column, must be
handled. This is typically done by imputing the value using the mean, median, or a more
advanced method.
3.​ Encoding Categorical Data: K-Means only understands numerical data. Categorical
features like Sex (male/female) and Embarked (C/Q/S) must be converted into numerical
format, often using one-hot encoding or label encoding.
Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 50
 4.​ Feature Scaling: This is a critical step. K-Means is a distance-based algorithm, so
features with large scales (like Fare, ranging from 0 to 500+) will dominate features with
small scales (like Pclass, 1 to 3). All selected features must be scaled to a similar range
using a tool like StandardScaler or MinMaxScaler.

Model Application
●​ Finding Optimal $k$: The number of clusters ($k$) is a parameter we must choose. The
standard approach is the Elbow Method. This involves running K-Means for a range of
$k$ values (e.g., 1 to 10) and plotting the WCSS for each. The "elbow" of the resulting
curve—the point where the rate of decrease in WCSS sharply slows down—indicates a
good balance between the number of clusters and the variance within them.
●​ Clustering: Once an optimal $k$ is chosen (e.g., $k=4$), the K-Means algorithm is
trained on the fully preprocessed data.
●​ Analysis: Finally, each passenger in the dataset is assigned a cluster label.

Implementation:

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 51

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 52
Conclusion and Interpretation
The K-Means algorithm successfully partitions the Titanic passengers into $k$ distinct groups.
The conclusion is not the clusters themselves, but their interpretation.

By examining the centroid (average feature values) of each cluster, we can build a "persona" for
each group. For example, we might find:

●​ Cluster 0: High Fare, low Pclass (i.e., 1st), high Age (e.g., "Wealthy, Elderly
Passengers").
Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 53
 ●​ Cluster 1: Low Fare, high Pclass (i.e., 3rd), Sex=male, Age in 20s (e.g., "Young, Solo
Male Travelers").
●​ Cluster 2: High SibSp/Parch, mixed Pclass (e.g., "Large Families").

While the Survived column was not used during the clustering process (as it's unsupervised), we
can now analyze the survival rate within each cluster. This provides powerful insights. For
instance, we might find that the "Wealthy, Elderly Passengers" cluster had a 65% survival rate,
while the "Young, Solo Male Travelers" cluster had only a 15% survival rate. This demonstrates
how unsupervised segmentation can reveal hidden patterns related to a supervised outcome.

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 54

 EXPERIMENT 9
AIM: Write a program to construct a Bayesian network considering medical data. Use this model
to demonstrate the diagnosis of cancer patients using standard Cancer Disease Dataset.

Theory

A Bayesian Network (BN) is a probabilistic graphical model that represents a set of variables
and their conditional dependencies via a Directed Acyclic Graph (DAG). It combines principles
from graph theory and probability theory to efficiently model uncertainty in complex domains.

Key Components:

●​ Nodes: Represent random variables (e.g., Age, Smoking, Tumor Size). These can be
observable quantities, latent variables, or hypotheses.
●​ Edges: Represent direct probabilistic dependencies or causal influences between nodes.
An edge from node A to node B indicates that A has a direct influence on B.
●​ Conditional Probability Tables (CPTs): Each node has a CPT that quantifies the effect of
its parent nodes. For a node with no parents, this is simply its prior probability.

Theoretical Foundation: The Chain Rule

The fundamental rule that allows Bayesian networks to compactly represent a joint probability
distribution is the chain rule of probability. For a network with variables

, the full joint distribution is factored as:

This formula states that the joint probability of all variables is the product of the conditional
probability of each variable given its parents in the graph. This factorization drastically reduces
the number of parameters needed to define the model.

Inference in Bayesian Networks

The primary goal is to perform inference, which involves calculating the posterior probability of
a query variable given observed evidence. For a disease diagnosis, this is formulated using
Bayes' Theorem:

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 55

Where:

●​ P(Cancer∣Evidence)
●​ P(Cancer∣Evidence) is the posterior probability we want to compute—the probability of
cancer given the observed symptoms and test results.
●​ P(Evidence∣Cancer)
●​ P(Evidence∣Cancer) is the likelihood—the probability of observing the evidence if the
patient has cancer.
●​ P(Cancer)
●​ P(Cancer) is the prior probability—the general prevalence of cancer in the population.
●​ P(Evidence)
●​ P(Evidence) is the marginal likelihood or normalizing constant, often computed by
summing over all possible states of the hidden variables.

Methodology

This section outlines the steps to build and use a Bayesian network for cancer diagnosis.

Problem Definition and Variable Selection​

Define the primary objective: to diagnose the presence or absence of cancer. Identify the key
variables from the medical dataset. These typically include:

●​ Target Variable: Cancer (e.g., Malignant or Benign).

●​ Risk Factor Variables: Age, Smoking History, Genetic Markers, Family History.
●​ Symptom Variables: Lump Presence, Pain Level, Weight Loss.
●​ Diagnostic Test Variables: Tumor Size, Biopsy Result, MRI Scan Result.

Network Structure Learning​

Define the dependency relationships between the variables. This can be done in two ways:

●​ Expert Knowledge: A domain expert (oncologist) defines the links based on known
medical causality. For example, Smoking directly influences Cancer, and Cancer causes
changes in Tumor Size and Biopsy Result.
●​ Data-Driven Algorithms: Use algorithms like the K2 or PC algorithm to learn the
structure directly from the dataset by analyzing conditional independencies.

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 56

Implementation:

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 57

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 58
Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 59
Conclusion

Bayesian networks provide a powerful and intuitive framework for modeling medical diagnosis
under uncertainty. Their ability to incorporate both prior knowledge (from experts) and data
makes them highly suitable for complex domains like oncology. The model clearly represents the
causal relationships between risk factors, the disease itself, and its symptoms or test results.

The key advantage in a medical context is the capacity for explainable reasoning. Unlike some
"black-box" models, a BN can show how evidence from different sources combines to lead to a
specific diagnosis. Furthermore, it allows for flexible inference, enabling "what-if" scenarios. By
implementing this model on a standard cancer dataset, one can demonstrate an effective,
transparent, and statistically sound decision-support system to aid in the early and accurate
diagnosis of cancer.

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 60

 EXPERIMENT 10
AIM: Build an Artificial Neural Network by implementing the Back-Propagation algorithm and
test using dataset (earthquakes_2023_global and Bigmart Sales Dataset) and analyse
performance matrix.

Theory

An Artificial Neural Network (ANN) is a computational model inspired by the human

brain's network of biological neurons. It is designed to recognize underlying patterns and

relationships in a dataset. The fundamental building block is the artificial neuron, which

receives inputs, processes them with an activation function, and produces an output.

Multiple neurons are organized into layers: an input layer, one or more hidden layers, and an

output layer. Information flows from the input layer, through the hidden layers, and finally to

the output layer. The connections between these neurons have associated weights, which are

parameters that the network adjusts during learning.

The Back-Propagation algorithm is the core learning mechanism for training such

multi-layer networks. It is a supervised learning technique that works in two main

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 61
 phases: a forward pass and a backward pass. In the forward pass, an input is presented to

the network, and it propagates through the layers to generate an output. This output is

compared to the actual target value, and the difference is calculated as an error. In the

backward pass, this error is propagated back through the network from the output layer

to the input layer. As the error travels backward, the algorithm calculates the gradient of

the error with respect to each weight. These gradients are then used to update the

weights in a direction that minimizes the overall error of the network. This iterative

process of forward and backward passes over the training data allows the network to

learn and improve its predictive accuracy over time.

Weight Update Rule:

Methodology

Data Preparation: The first step involves loading the "earthquakes_2023_global"

dataset. This data will be preprocessed, which includes handling any missing values,

and selecting relevant features (like magnitude, depth, latitude, longitude) to be used as

input variables. The target variable for prediction will be defined, which could be, for

instance, the magnitude of an earthquake. The data will then be split into two subsets: a

training set used to teach the model, and a testing set (the "Test" dataset) used to

evaluate its final performance.

Model Development: An ANN architecture will be designed and implemented. This

involves defining the number of hidden layers, the number of neurons in each layer, and

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 62

 the activation functions for the neurons (such as ReLU or Sigmoid). The

Back-Propagation algorithm will be implemented to train this network. Key parameters

for the training process, including the learning rate and the number of training iterations

(epochs), will be chosen.

Training and Testing: The model will be trained on the training dataset. During this

phase, the Back-Propagation algorithm will iteratively adjust the network's weights to

minimize the prediction error. Once training is complete, the finalized model will be

used to make predictions on the unseen testing dataset.

Performance Analysis: The model's predictions on the test set will be compared against

the actual values. Performance metrics will be calculated to analyse the model's

effectiveness. For a regression task like earthquake magnitude prediction, key metrics

include Mean Absolute Error (MAE), which measures the average magnitude of errors;

Mean Squared Error (MSE), which gives more weight to larger errors; and R-Squared,

which indicates how well the model explains the variance in the target variable.

Implementation:

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 63

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 64
Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 65
Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 66
Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 67
 Conclusion

This project demonstrates the implementation of an Artificial Neural Network trained

with the Back-Propagation algorithm for analyzing earthquake data. The methodology

outlines a structured approach from data preparation to model evaluation. By training

the network on historical global earthquake data and testing it on a separate dataset, the

aim is to build a model capable of predicting seismic parameters. The final performance

matrix, comprising metrics like MAE, MSE, and R-Squared, will provide a

comprehensive analysis of the model's predictive accuracy and its ability to generalize

to new, unseen data. The success of the model will be determined by how low the error

metrics are and how well it captures the complex, non-linear relationships inherent in

seismic events.

Prince Verma (25/SWE/24) ​ ​ ​ ​ ​ ​ ​ ​ ​ 68