Activity Based Learning - 1SP22IC02

BIC703 6

VISVESVARAYA TECHNOLOGICAL UNIVERSITY

"Jnana Sangama", Belagavi : 590018

A
report on

“ MACHINE LEARNING PRACTICAL BASED LEARNING ”

Submitted in partial fulfillment of the requirement for the award of the degree of

Bachelor of Engineering
in
INTERNET OF THINGS AND CSBT
Submitted by

JORDAN EMMANUEL (1SP22IC026)

Under the guidance of

Prof. Prashanth B S
Asst. Professor, Dept. Of IoT

DEPARTMENT OF CSE (INTERNET OF THINGS & CSBT)

S.E.A. COLLEGE OF ENGINEERING & TECHNOLOGY

(Affiliated to Visvesvaraya Technological University, Belgaum)

2025-2026
1
Activity Based Learning - 1SP22IC02
BIC703 6

S.E.A COLLEGE OF ENGINEERING AND TECHNOLOGY

EktaNagar, Basavanpura, Virgonagar Post, [Link], Bengaluru, Karnataka 560049

DEPARTMENT OF IOT & CSBT IN ENGINEERING

CERTIFICATE
This is to certify the project work entitled “Activity Based Learning - BIC703 Machine
Learning” has been successfully carried out by JORDAN EMMANUEL (1SP22IC026)VII
semester in partial fulfillment for the award of Bachelor of Engineering in IOT& CSBT of the
Visvesvaraya Technological University, Belagavi during the year 2025-26. The project report
has been approved as it satisfied the academic requirement in respect of the mini project work
prescribed for Bachelor of Engineering.

Signature of Guide Signature of Principal Signature of HOD

PRASHANTH B S Dr B VENKATA NARAYANA Dr SUKESH H A

2
Activity Based Learning - 1SP22IC02
BIC703 6

Assignment 1.1- : Programming Assignment: Implementation of Important Concepts of Feature Engineering

Q1. Load a dataset of your choice (from [Link], Kaggle, or a local CSV file). Display the first few
rows of the dataset. Print the dataset shape and column data types. Identify the target (dependent)
variable and independent (input) features.

import pandas as pd
from [Link] import load_iris
print("Q1: Load a dataset and perform basic
exploration\n") iris = load_iris()
df = [Link]([Link],
columns=iris.feature_names) df['target'] =
[Link]
print("First 5 rows of the
dataset:") display([Link]())
print("\nDataset Shape:")
print([Link])
print("\nColumn Data
Types:") print([Link])
target = 'target'
features = [Link][:-1] print("\
nTarget (Dependent) Variable:")
print(target)
print("\nIndependent (Input)
Features:") print(list(features))
print("\n--- Completed by USN: 1SP22IC014 ---")

output:
Q1: Load a dataset and perform basic exploration

First 5 rows of the dataset:

sepal length sepal width targe

petal length (cm) petal width (cm)
(cm) (cm) t

0 5.1 3.5 1.4 0.2 0

1 4.9 3.0 1.4 0.2 0

2 4.7 3.2 1.3 0.2 0

3 4.6 3.1 1.5 0.2 0

4 5.0 3.6 1.4 0.2 0

Dataset Shape:
(150, 5)

Column Data Types:

sepal length (cm)
float64 sepal width
(cm) float64 petal
length (cm) float64
petal width (cm)
float64 target
int32
dtype: object

Target (Dependent) Variable:

target

Independent (Input) Features:

['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)']

--- Completed by USN: 1SP22IC014 ---

Q2. Perform basic data quality analysis. Check for missing values and duplicates. Handle missing values
using at least two different techniques (for example, mean/median imputation and forward fill). Explain
which technique is most appropriate for your dataset.
print("Q2: Basic Data Quality Analysis
\n") iris = load_iris()
df = [Link]([Link], columns=iris.feature_names)
df['target'] = [Link]
print("Missing values in the
dataset:") print([Link]().sum())
print("\nNumber of duplicate
rows:")
print([Link]().sum())
[Link][2, 1] = None
[Link][5, 3] = None
print("\nMissing values after inserting sample NaNs:")
print([Link]().sum())

3
Activity Based Learning - 1SP22IC02
BIC703
df_mean_imputed = [Link]()
6

4
Activity Based Learning - 1SP22IC02
BIC703 6

df_mean_imputed.fillna(df_mean_imputed.mean(),
inplace=True) print("\nMissing values after Mean
Imputation:") print(df_mean_imputed.isnull().sum())
df_ffill = [Link]()
df_ffill.fillna(method='ffill',
inplace=True) print("\nMissing values
after Forward Fill:")
print(df_ffill.isnull().sum())
print("\n--- Completed by USN: 1SP22IC014
---") output :
output:
Q2: Basic Data Quality Analysis

Missing values in the dataset:

sepal length (cm)
0 sepal width
(cm) 0 petal
length (cm) 0
petal width (cm)
0 target 0
dtype: int64

Number of duplicate rows:

1

Missing values after inserting sample NaNs:

sepal length (cm) 0
sepal width (cm) 1
petal length (cm) 0
petal width (cm) 1
target 0
dtype: int64

Missing values after Mean Imputation:

sepal length (cm) 0
sepal width (cm) 0
petal length (cm) 0
petal width (cm) 0
target 0
dtype: int64

Missing values after Forward Fill:

sepal length (cm) 0
sepal width (cm) 0
petal length (cm) 0
petal width (cm) 0
target 0
dtype: int64

--- Completed by USN: 1SP22IC014 ---

Q3. Conduct an initial exploratory data analysis (EDA). Display summary statistics using .describe(). Plot
histograms for numeric columns to understand data distribution. Identify and comment on any skewed
features.

iris = load_iris()
df = [Link]([Link],
columns=iris.feature_names) df['target'] =
[Link]

print("Summary
Statistics:")
print([Link]())

[Link](figsize=(9, 6))
[Link]("Histograms of Numeric Features (Iris Dataset)")
[Link]()

print("\nSkewness of
features:") print([Link][:, :-
1].skew())

print("\nComments:")
for col in [Link][:-1]:
skew = df[col].skew()
if skew > 0.5:
print(f"{col} is positively skewed.")
elif skew < -0.5:

5
Activity Based Learning - 1SP22IC02
BIC703
print(f"{col} is negatively skewed.")
6

6
Activity Based Learning - 1SP22IC02
BIC703 6

else:
print(f"{col} is approximately

symmetric.") print("\n--- Completed by

USN: 1SP22IC014 ---")

Output
:
Q3: Exploratory Data Analysis

Summary Statistics:
sepal length (cm) sepal width (cm) petal
length (cm) \ count 150.000000 150.000000
150.000000
mean 5.843333 3.05733 3.75800
3 0
std 0.828066 0.435866 1.765298
min 4.300000 2.000000 1.000000
25% 5.100000 2.800000 1.600000
50% 5.800000 3.000000 4.350000
75% 6.400000 3.300000 5.100000
max 7.900000 4.400000 6.900000

petal width (cm) target

count 150.000000
150.000000
mean 1.199333 1.000000
std 0.762238 0.819232
min 0.100000
0.000000
25% 0.300000
0.000000
50% 1.300000
1.000000
75% 1.800000
2.000000
max 2.500000
2.000000

Skewness of features:
sepal length (cm)
0.314911 sepal width
(cm) 0.318966 petal
length (cm) -0.274884
petal width (cm) -
0.102967 dtype: float64

Comments:
sepal length (cm) is approximately
symmetric. sepal width (cm) is
approximately symmetric. petal length
(cm) is approximately symmetric. petal
width (cm) is approximately
symmetric.

--- Completed by USN: 1SP22IC014 ---

Q4. Detect and handle outliers. Use visualizations such as box plots to identify outliers in numerical
features. Apply the Interquartile Range (IQR) method to remove or cap outliers. Explain the effect of
outlier treatment on data distribution.

print("Q4: Outlier Detection and

Treatment\n") iris = load_iris()
df = [Link]([Link],
columns=iris.feature_names) df['target'] =
[Link]
[Link][:, :-1].boxplot(figsize=(8, 6))
[Link]("Box Plot BEFORE Outlier
Treatment") [Link]()

7
Activity Based Learning - 1SP22IC02
BIC703
Q1 =
6
[Link](0.25) Q3
= [Link](0.75)
IQR = Q3 - Q1
lower_bound = Q1 - 1.5 * IQR
upper_bound = Q3 + 1.5 *
IQR
df_clean = df[~((df < lower_bound) | (df >
upper_bound)).any(axis=1)] print("Shape before removing
outliers:", [Link])
print("Shape after removing outliers :",
df_clean.shape) df_clean.iloc[:, :-
1].boxplot(figsize=(8, 6))
[Link]("Box Plot AFTER Outlier
Treatment") [Link]()

8
Activity Based Learning - 1SP22IC02
BIC703 6

print("\nSkewness Before
Treatment:") print([Link][:, :-
1].skew()) print("\nSkewness
After Treatment:")
print(df_clean.iloc[:, :-1].skew())
print("\n--- Completed by USN: 1SP22IC014 ---")

Output:
Q4: Outlier Detection and Treatment

Shape before removing outliers:

(150, 5) Shape after removing
outliers : (146, 5) Skewness
Before Treatment:
sepal length (cm)
0.314911 sepal width
(cm) 0.318966 petal
length (cm) -0.274884
petal width (cm) -
0.102967 dtype: float64

Skewness After Treatment:

sepal length (cm)
0.278417 sepal width
(cm) 0.131562 petal
length (cm) -0.324099
petal width (cm) -
0.146606 dtype: float64

--- Completed by USN: 1SP22IC014 --

Q5. Perform feature scaling. Apply both Standardization (Z-score scaling) and Normalization (Min-Max
scaling) on the dataset. Compare the results using summary statistics or visualizations. 1 Activity Based
Learning -BIC703 Explain why scaling is essential for algorithms such as KNN or SVM.

iris = load_iris()
df = [Link]([Link],
columns=iris.feature_names) df['target'] =
[Link]
X = [Link][:, :-1]
scaler_standard = StandardScaler()
X_standard =
scaler_standard.fit_transform(X)
df_standard = [Link](X_standard,
columns=[Link]) print("Summary Statistics after
Standardization:") print(df_standard.describe())
scaler_minmax = MinMaxScaler()
X_minmax = scaler_minmax.fit_transform(X)
df_minmax = [Link](X_minmax,
columns=[Link]) print("\nSummary Statistics after
Min-Max Normalization:") print(df_minmax.describe())
[Link](figsize=(10, 6))
df_standard.boxplot()
[Link]("Box Plot After

9
Activity Based Learning - 1SP22IC02
BIC703
Standardization") [Link]()
6
[Link](figsize=(10, 6))
df_minmax.boxplot()

10
Activity Based Learning - 1SP22IC02
BIC703 6

[Link]("Box Plot After Min-Max

Normalization") [Link]()
print("\n--- Completed by USN:

1SP22IC014 ---") Q5: Feature Scaling

Summary Statistics after Standardization:

sepal length (cm) sepal width (cm) petal
length (cm) \ count
1.500000e+02 1.500000e+02
1.500000e+02 mean -
1.468455e-15 -1.823726e-15 -1.610564e-15
std 1.003350e+00 1.003350e+00
1.003350e+00 min -
1.870024e+00 -2.433947e+00 -1.567576e+00
25% -9.006812e-01 -5.923730e-01 -
1.226552e+00
50% -5.250608e-02 -1.319795e-01 3.364776e-01
75% 6.745011e-01 5.586108e-01
7.627583e-01 max
2.492019e+00 3.090775e+00
1.785832e+00

petal width (cm)

count
1.500000e+02
mean -9.473903e-
16 std

1.003350e+00 min
-
1.447076e+00 25%
-
1.183812e+00
50% 1.325097e-01
75% 7.906707e-
01 max

1.712096e+00

Summary Statistics after Min-Max Normalization:

sepal length (cm) sepal width (cm) petal
length (cm) \ count 150.000000 150.000000
150.000000
mean 0.428704 0.44055 0.46745
6 8
std 0.230018 0.181611 0.299203
min 0.000000 0.000000 0.000000
25% 0.222222 0.333333 0.101695
50% 0.416667 0.416667 0.567797
75% 0.583333 0.541667 0.694915
max 1.000000 1.000000 1.000000

petal width
(cm) count

150.000000
mean 0.458056
std 0.317599
min 0.000000
25% 0.083333
50% 0.500000
75% 0.708333
max 1.000000
--- Completed by USN: 1SP22IC014 ---

Q6. Handle categorical features. Identify categorical variables in your dataset. Apply Label Encoding
and One-Hot Encoding using appropriate methods from pandas or sklearn. Compare the results and
explain which encoding method is preferable for different types of algorithms (e.g., linear vs. tree-
based).

iris = load_iris()
df = [Link]([Link],
columns=iris.feature_names) df['species'] =
11
Activity Based Learning - 1SP22IC02
BIC703
[Link]
6
df['species'] =
df['species'].astype('category')
print("Categorical Variables in
Dataset:")
print(df.select_dtypes(include=['categor
y'])) label_encoder = LabelEncoder()
df['species_label_encoded'] = label_encoder.fit_transform(df['species'])
print("\nAfter Label Encoding (first 5 rows):")
print(df[['species', 'species_label_encoded']].head())
df_onehot = pd.get_dummies(df, columns=['species'],
prefix='species') print("\nAfter One-Hot Encoding (first 5
rows):") print(df_onehot.head())
print("\nShape before encoding:", [Link])
print("Shape after One-Hot Encoding:",
df_onehot.shape) print("\n--- Completed by USN:
1SP22IC014 ---")

12
Activity Based Learning - 1SP22IC02
BIC703 6

Output :
Q6: Handling Categorical Features

Categorical Variables in Dataset:

species
0 0
1 0
2 0
3 0
4 0
.. ...
145 2
146 2
147 2
148 2
149 2

[150 rows x 1 columns]

After Label Encoding (first 5 rows):

species species_label_encoded
0 0 0
1 0 0
2 0 0
3 0 0
4 0 0

After One-Hot Encoding (first 5 rows):

sepal length (cm) sepal width (cm) petal length (cm) petal width (cm) \
0 5.1 3.5 1.4 0.2
1 4.9 3.0 1.4 0.2
2 4.7 3.2 1.3 0.2
3 4.6 3.1 1.5 0.2
4 5.0 3.6 1.4 0.2

species_label_encoded species_0 species_1 species_2

0 0 True False False
1 0 True False False
2 0 True False False
3 0 True False False
4 0 True False False

Shape before encoding: (150, 6)

Shape after One-Hot Encoding: (150, 8)

--- Completed by USN: 1SP22IC014 ---

Q7. Perform feature discretization (binning). Select one continuous variable and divide it into bins using
[Link]() or [Link](). Display the count of values in each bin. Explain the potential advantages of feature
binning in predictive modeling.

iris = load_iris()
df = [Link]([Link],
columns=iris.feature_names) df['target'] =
[Link]
feature = 'petal length (cm)'
df['petal_length_bin'] =
[Link](df[feature],
bins=3,
labels=["Short", "Medium", "Long"])
print("First 10 entries with bins:")
print(df[[feature,
'petal_length_bin']].head(10)) print("\
nValue counts per bin:")
print(df['petal_length_bin'].value_counts()
) print("\n--- Completed by USN:
1SP22IC014 ---")

output:

Q7: Feature Discretization (Binning)

First 10 entries with bins:

13
Activity Based Learning - 1SP22IC02
BIC703
petal length (cm) petal_length_bin
6

14
Activity Based Learning - 1SP22IC02
BIC703 6

0 1.4 Short
1 1.4 Short
2 1.3 Short
3 1.5 Short
4 1.4 Short
5 1.7 Short
6 1.4 Short
7 1.5 Short
8 1.4 Short
9 1.5 Short

Value counts per bin:

petal_length_b
in Medium 54
Short 50
Long 46
Name: count, dtype: int64

--- Completed by USN: 1SP22IC014 ---

Q8. Create polynomial and interaction features. Use PolynomialFeatures from [Link]
to create higher-order and interaction features (e.g., degree = 2). Display the new feature matrix shape
and names. Discuss how polynomial features affect model complexity and performance.

iris = load_iris()
df = [Link]([Link],
columns=iris.feature_names) poly =
PolynomialFeatures(degree=2,
include_bias=False) poly_features =
poly.fit_transform(df)
feature_names =
poly.get_feature_names_out([Link]) df_poly =
[Link](poly_features, columns=feature_names)
print("Original feature shape:", [Link])
print("After polynomial transformation:",
df_poly.shape) print("\nFirst 5 rows of transformed
feature space:") print(df_poly.head())
print("\n--- Completed by USN: 1SP22IC014 ---")

Output:

Q8: Polynomial and Interaction Features

Original feature shape: (150, 4)

After polynomial transformation: (150, 14)

First 5 rows of transformed feature space:

sepal length (cm) sepal width (cm) petal length (cm) petal width (cm) \
0 5.1 3.5 1.4 0.2
1 4.9 3.0 1.4 0.2
2 4.7 3.2 1.3 0.2
3 4.6 3.1 1.5 0.2
4 5.0 3.6 1.4 0.2

sepal length (cm)^2 sepal length (cm) sepal width (cm) \

0 26.01 17.85
1 24.01 14.70
2 22.09 15.04
3 21.16 14.26
4 25.00 18.00

sepal length (cm) petal length (cm) sepal length (cm) petal width (cm) \
0 7.14 1.02
1 6.86 0.98
2 6.11 0.94
3 6.90 0.92
4 7.00 1.00

sepal width (cm)^2 sepal width (cm) petal length (cm) \

0 12.25 4.90
1 9.00 4.20

15
Activity Based Learning - 1SP22IC02
BIC703
2 10.24 4.16
6

16
Activity Based Learning - 1SP22IC02
BIC703 6

3 9.61 4.65
4 12.96 5.04

sepal width (cm) petal width (cm) petal length (cm)^2 \

0 0.70 1.96
1 0.60 1.96
2 0.64 1.69
3 0.62 2.25
4 0.72 1.96

petal length (cm) petal width (cm) petal width (cm)^2

0 0.28 0.04
1 0.28 0.04
2 0.26 0.04
3 0.30 0.04
4 0.28 0.04

--- Completed by USN: 1SP22IC014 ---

QG. Apply feature transformation to reduce skewness. Identify skewed features using skewness statistics.
Apply log, square root, or Box-Cox transformations to normalize the distribution. Compare the feature
distributions before and after transformation using histograms.

iris = load_iris()
df = [Link]([Link],
columns=iris.feature_names) print("Skewness
before transformation:")
print([Link]())
df['log_petal_length'] = np.log1p(df['petal length
(cm)']) df['sqrt_petal_length'] = [Link](df['petal
length (cm)']) df['boxcox_petal_length'], _ =
boxcox(df['petal length (cm)'])
[Link](figsize=(12, 8))
[Link](figsize=(12, 8))
[Link](2, 2, 1)
[Link](df['petal length (cm)'],
color='red') [Link]("Original")
[Link](2, 2, 2)
[Link](df['log_petal_length'],
color='green') [Link]("Log Transform")
[Link](2, 2, 3)
[Link](df['sqrt_petal_length'],
color='blue') [Link]("Sqrt Transform")
[Link](2, 2, 4)
[Link](df['boxcox_petal_length'],
color='purple') [Link]("Box-Cox
Transform")
plt.tight_layout()
[Link]()
print("\nSkewness after transformations:")
print("Log Transform :",
df['log_petal_length'].skew()) print("Sqrt Transform
:", df['sqrt_petal_length'].skew()) print("Box-Cox
Transform :", df['boxcox_petal_length'].skew()) print("\
n--- Completed by USN: 1SP22IC014 ---")

Output:

Q9: Feature Transformation to Reduce Skewness

Skewness before
transformation: sepal length
(cm) 0.314911 sepal width
(cm) 0.318966 petal length
(cm) -0.274884 petal width
(cm) -0.102967 dtype:
float64
<Figure size 1200x800 with 0 Axes>

Skewness after transformations:

Log Transform : -
0.5223863005011183 Sqrt

17
Activity Based Learning - 1SP22IC02
BIC703
Transform : -
6
0.44901356468696196 Box-Cox
Transform : -0.301163625962206
--- Completed by USN: 1SP22IC014 ---

18
Activity Based Learning - 1SP22IC02
BIC703 6

Q10. Perform feature selection. Use one of the following methods to select the most relevant features: o
Variance Threshold o SelectKBest o Recursive Feature Elimination (RFE) Display the top selected features.
Explain how feature selection helps in improving model performance and interpretability.

iris = load_iris()
df = [Link]([Link],
columns=iris.feature_names) target = [Link]
selector = SelectKBest(score_func=f_classif, k=2)
[Link](df, target)
mask = selector.get_support()
selected_features =
[Link][mask] print("Top
Selected Features:")
print(selected_features)
scores = selector.scores_
print("\nFeature Scores:")
for col, score in zip([Link], scores):
print(f"{col}: {score:.2f}")
print("\n--- Completed by USN: 1SP22IC014 ---")

Output:

Q10: Feature Selection

Top Selected Features:

Index(['petal length (cm)', 'petal width (cm)'], dtype='object')

Feature Scores:
sepal length (cm):
119.26 sepal width
(cm): 49.16 petal
length (cm): 1180.16
petal width (cm):
960.01

--- Completed by USN: 1SP22IC014 --

Q11. Summarize your overall findings. Which feature engineering steps had the greatest impact on
data quality or model readiness? What challenges did you face during feature engineering? How
would you improve your feature engineering process for a large, complex dataset?

✅ Which feature engineering steps had the greatest impact on data quality or model readiness?
● Handling missing values improved data completeness and prevented model bias or errors.
● Encoding categorical variables (Label Encoding / One-Hot Encoding) made non-numerical
features usable for ML algorithms.
● Feature scaling (StandardScaler / MinMaxScaler) boosted the performance of distance-
based models such as K-Means, SVM, and KNN by standardizing feature ranges.
● Removing duplicate or irrelevant features reduced noise and improved model efficiency.
● Feature selection (Variance Threshold, SelectKBest, RFE) removed weak predictors,
improving training speed and sometimes accuracy.
● Dimensionality reduction techniques (PCA) helped simplify high-dimensional data and reduced
overfitting.

✅ Challenges Faced During Feature Engineering

● Identifying which features were truly important required experimentation.
● Handling categorical variables with many unique values complicated encoding.
● Deciding the appropriate method for missing values was context-dependent.
● Standardizing continuous features while keeping interpretability intact.
● High dimensionality increased computation time and made visualization harder.
● Avoiding overfitting after transformations (especially with polynomial or derived features).

✅ How to Improve the Feature Engineering Process for Large, Complex Datasets
● Automate preprocessing pipelines with tools like [Link] to ensure consistency.
● Use advanced feature selection techniques such as mutual information, tree-based
importance, or embedded regularization.

19
Activity Based Learning - 1SP22IC02
BIC703 6
● Apply dimensionality reduction (PCA, t-SNE, Autoencoders) to handle thousands of features efficiently.

20
 Activity Based Learning - 1SP22IC02
BIC703 6

● Use distributed processing tools like Spark or Dask to manage large volumes of data.
● Perform exploratory data analysis (EDA) early to identify patterns, correlations, and outliers.
● Leverage domain knowledge to engineer meaningful features beyond raw data.
● Monitor feature drift when data changes over time in real-world systems.

Assignment : 1.2 -Programming Assignment: Implementation of Important Concepts of Data Representation

Q1. Dataset Loading and Overview Load a dataset of your choice (you may use [Link], Kaggle data,
or a local CSV file). Display the first 5 rows of the dataset. Show dataset shape, column names, and
data types. Identify the target (dependent) and input (independent) features.

df = pd.read_csv("[Link]")
print("\n--- First 5 Rows of the Dataset ---")
print([Link]())
print("\n--- Dataset Shape ---")
print([Link])
print("\n--- Column Names ---")
print([Link])
print("\n--- Data Types of Each Column ---")
print([Link])
X = [Link](['Final_Score'], axis=1)
y = df['Final_Score']
print("\n--- Independent (Input) Features ---")
print([Link])
print("\n--- Target (Dependent) Feature ---")
print([Link])
print("\nCompleted by USN - 1SP22IC014!")

Output:

Q1. Dataset Loading and Overview

--- First 5 Rows of the Dataset ---

Hours_Studied Attendance Assignments_Completed Previous_Score \
2 74 6 55
3 78 7 58
1 69 5 52
4 82 8 60
5 88 9 62

Final_Score
0 57
1 60
2 54
3 65
4 70

--- Dataset Shape ---

(50, 5)
--- Column Names ---
Index(['Hours_Studied', 'Attendance',
'Assignments_Completed', 'Previous_Score',
'Final_Score'],
dtype='object')
--- Data Types of Each
Column --- Hours_Studied
int64
Attendance int64
Assignments_Completed
int64 Previous_Score
int64
Final_Score int64
dtype: object
--- Independent (Input) Features ---
Index(['Hours_Studied', 'Attendance',
'Assignments_Completed', 'Previous_Score'],
dtype='object')
--- Target (Dependent)
Feature --- Final_Score
Q2. Data Types and Representation List all features according to their data types (numerical, categorical,
datetime, etc.). Explain how different data types affect how data should be represented for machine
21
Activity Based Learning - 1SP22IC02
BIC703 6
learning

22
Activity Based Learning - 1SP22IC02
BIC703 6

models. Convert any necessary data columns to their appropriate types (e.g., convert object → category or
datetime).
df =
pd.read_csv("[Link]")
print("\n--- Current Data Types
---") print([Link])
numerical_features = df.select_dtypes(include=['int64',
'float64']).[Link]() categorical_features =
df.select_dtypes(include=['object']).[Link]()
datetime_features =
df.select_dtypes(include=['datetime']).[Link]() print("\n---
Numerical Features ---")
print(numerical_features)
print("\n--- Categorical Features ---")
print(categorical_features)
print("\n--- Datetime Features ---")
print(datetime_features) print("\
nCompleted By USN - 1SP22IC014!")

Output:

Q2. Data Types and Representation

--- Current Data Types ---
Employee_ID int64
Name object
Department object
Join_Date object
Experience_Years
int64
Salary int64
Performance_Rating
int64 Remote_Work
object
Gender object
dtype: object

--- Numerical Features ---

['Employee_ID', 'Experience_Years', 'Salary', 'Performance_Rating']
--- Categorical Features ---
['Name', 'Department', 'Join_Date', 'Remote_Work', 'Gender']
--- Datetime Features
--- []

Q3. Tabular Data Representation using Pandas Represent your dataset as a DataFrame. Display
column statistics using .describe() and .info(). Explain how tabular representation in Pandas helps in
efficient data processing and manipulation.
df = pd.read_csv("[Link]")

print("\n--- DataFrame Preview (First 5 rows) ---")

print([Link]())

print("\n--- Column Statistics using describe() ---")

print([Link](include='all'))

print("\n--- Dataset Information using info() ---")

print([Link]())

print("\nCompleted By USN -
1SP22IC014!") print("\n")

output:

Q3. Tabular Data Representation using Pandas

--- DataFrame Preview (First 5 rows) ---

Employee_ID Name Department Join_Date Experience_Years Salary \
0 101 Amit IT 2020-03-12 3 55000
1 102 Sneha HR 2019-07-25 4 48000
2 103 Raj Finance 2021-11-10 2 60000
3 104 Divya IT 2018-05-18 5 72000

23
Activity Based Learning - 1SP22IC02
BIC703 4 105 Karan Marketing 2022- 1
6
01-03 45000

24
Activity Based Learning - 1SP22IC02
BIC703 6

Performance_Rating Remote_Work Gender

0 4 Yes Male
1 3 No
Female
2 5 Yes Male
3 4 No
Female
4 3 Yes Male

--- Column Statistics using describe() ---

Employee_ID Name Department Join_Date
Experience_Years \ count 30.000000 30 30
30 30.00000
unique NaN 30 5 30 NaN
top NaN Amit IT 2020-03-12
NaN freq NaN 1 7 1
NaN
mea 115.500000 NaN NaN NaN 3.50000
n
std 8.803408 NaN NaN NaN 1.61352
min 101.000000 NaN NaN NaN 1.00000
25% 108.250000 NaN NaN NaN 2.00000
50% 115.500000 NaN NaN NaN 3.50000
75% 122.750000 NaN NaN NaN 5.00000
max 130.000000 NaN NaN NaN 6.00000

Salary Performance_Rating Remote_Work

Gender count 30.000000 30.000000
30 30
unique NaN NaN 2 2
top NaN NaN Yes
Male freq NaN NaN
15 15
mean 57533.333333 3.666667 NaN NaN
std 11078.818975 0.844182 NaN NaN
min 42000.000000 2.000000 NaN NaN
25% 49250.000000 3.000000 NaN NaN
50% 54500.000000 4.000000 NaN NaN
75% 66500.000000 4.000000 NaN NaN
max 5.000000 NaN NaN
80000.000000

--- Dataset Information using info() ---

<class
'[Link]'>
RangeIndex: 30 entries, 0 to 29
Data columns (total 9 columns):
# Column Non-Null Count Dtype

0 Employee_ID 30 non-null int64

1 Name 30 non-null object
2 Department 30 non-null object
3 Join_Date 30 non-null object
4 Experience_Years 30 non-null int64
5 Salary 30 non-null int64
6 Performance_Rating 30 non-null int64
7 Remote_Work 30 non-null object
8 Gender 30 non-null
object dtypes: int64(4),
object(5)
memory usage: 2.2+
KB None

Completed By USN - 1SP22IC014!

Q4. Numerical Data Representation Isolate the numerical features from your dataset. Calculate basic
statistics such as mean, median, mode, variance, and standard deviation for each numerical column.
Visualize the distributions using histograms or boxplots.

df = pd.read_csv("[Link]")
numeric_df = df.select_dtypes(include=['int64',
'float64']) print("\n--- Numerical Features ---")
print(numeric_df.columns)

25
Activity Based Learning - 1SP22IC02
BIC703
print("\n--- Mean ---")
6
print(numeric_df.mean())
print("\n--- Median ---")
print(numeric_df.median())
print("\n--- Mode ---")
print(numeric_df.mode().iloc
[0])

26
Activity Based Learning - 1SP22IC02
BIC703 6

print("\n--- Variance ---")

print(numeric_df.var())
print("\n-- Standard Deviation ---")
print(numeric_df.std())
numeric_df.hist(color='r')
[Link]("Histogram of Numerical
Features") [Link]()
[Link]()
numeric_df.boxplot(color='g')
[Link]("Boxplot of Numerical
Features") [Link]()
print("\nCompleted By USN -
1SP22IC014!") print("\n")

output:

Q4. Numerical Data Representation

--- Numerical Features ---

Index(['Customer_ID', 'Age', 'Purchase_Amount', 'Frequency'], dtype='object')

--- Mean ---

Customer_ID 115.500000
Age 30.833333
Purchase_Amount 4780.000000
Frequency

4.066667 dtype:
float64

--- Median ---

Customer_ID 115.5
Age 30.0
Purchase_Amount 4300.0
Frequency 4.0
dtype: float64

--- Mode ---

Customer_ID 101.0
Age 29.0
Purchase_Amount 2900.0
Frequency 2.0
Name: 0, dtype: float64

--- Variance ---

Customer_ID
7.750000e+01
Age 2.724713e+01
Purchase_Amount
3.309241e+06 Frequency
4.960920e+00
dtype: float64

--- Standard Deviation ---

Customer_ID 8.803408
Age 5.219878
Purchase_Amount 1819.132040
Frequency

2.227312 dtype:
float64

Q5. Categorical Data Representation Identify categorical features and list unique categories for each.
Represent categorical data numerically using Label Encoding or One-Hot Encoding. Explain how
categorical encoding changes data representation for machine learning algorithms.
df = pd.read_csv("[Link]")
categorical_cols =
df.select_dtypes(include=['object']).columns print("\n---

27
Activity Based Learning - 1SP22IC02
BIC703
Categorical Features------------\n")
6
print(categorical_cols)
print("\n--- Unique Categories for Each Categorical Feature \n")
for col in categorical_cols:

28
Activity Based Learning - 1SP22IC02
BIC703 6

print(f"{col}:
{df[col].unique()}") print("\n---
Label Encoding Result ---\n")
label_df = [Link]()
le = LabelEncoder()
for col in categorical_cols:
label_df[col] = le.fit_transform(label_df[col])
print(label_df.head())
print("\n--- One-Hot Encoding Result ---\n")
onehot_df = pd.get_dummies(df, columns=categorical_cols)
print(onehot_df.head())
print("\nCompleted By USN -
1SP22IC014!") print("\n")

output:

Q5. Categorical Data Representation

--- Categorical Features ---

Index(['Category', 'Brand', 'Size', 'Payment_Method'], dtype='object')

--- Unique Categories for Each Categorical Feature ---

Category: ['Footwear' 'Clothing' 'Electronics' 'Accessories' 'Bags']
Brand: ['Nike' 'Adidas' 'HP' 'Fossil' 'Puma' 'Sony' 'Samsung'
'Wildcraft' 'Levis' 'Logitech' 'Apple' 'Dell']
Size: ['Medium' 'Large' nan 'Small'] Payment_Method: ['Online' 'Cash' 'UPI' 'Card']
--- Label Encoding Result ---

Product_ID Category Brand Size Payment_Method

0 101 4 7 1 2
1 102 2 0 0 1
2 103 3 4 3 3
3 104 0 3 2 2
4 105 1 8 0 0

--- One-Hot Encoding Result ---

Product_ID Category_Accessories Category_Bags Category_Clothing \

0 101 False False False
1 102 False False True
2 103 False False False
3 104 True False False
4 105 False True False

Category_Electronics Category_Footwear Brand_Adidas Brand_Apple \

0 False True False False
1 False False True False
2 True False False False
3 False False False False
4 False False False False

Brand_Dell Brand_Fossil ... Brand_Samsung Brand_Sony Brand_Wildcraft \

0 False False ... False False False
1 False False ... False False False
2 False False ... False False False
3 False True ... False False False
4 False False ... False False False

Size_Large Size_Medium Size_Small Payment_Method_Card \

0 False True False False
1 True False False False
2 False False False False
3 False False True False
4 True False False True

Payment_Method_Cash Payment_Method_Online Payment_Method_UPI

0 False True False
1 True False False
2 False False True
3 False True False
4 False False False
[5 rows x 25 columns]

29
Activity Based Learning - 1SP22IC02
BIC703 6

Q6. Visual Data Representation Create at least three plots to represent different aspects of your data: o
Histogram or density plot for numerical data o Count plot or bar chart for categorical data o Correlation
heatmap for relationships between numerical features 3 Explain what insights you gain from each
visualization.

data = {
"Product_ID": range(101, 126),
"Price": [1200,1500,55000,8000,2000,3000,1800,900,1400,60000,
700,1600,3500,1800,500,2000,2600,3000,4500,2500,
7500,65000,1800,58000,1600],
"Category": ["Footwear","Clothing","Electronics","Accessories","Bags",
"Electronics","Clothing","Accessories","Footwear","Electronics",
"Accessories","Clothing","Electronics","Clothing","Accessories",
"Electronics","Footwear","Clothing","Electronics","Bags",
"Accessories","Electronics","Clothing","Electronics","Clothing"]
}
df = [Link](data)
[Link]()
[Link](df["Price"] ,
color='orange')
[Link]("Histogram of Product
Price") [Link]("Price")
[Link]("Frequency")
[Link]()
[Link]()
df["Category"].value_counts().plot(kind="bar",
color='g') [Link]("Category Count Bar Chart")
[Link]("Category")
[Link]("Count")
[Link]()
corr =
[Link](numeric_only=True)
[Link]()
[Link](corr)
[Link]()
[Link]("Correlation Heatmap (Numerical Features)")
[Link](range(len([Link])), [Link],
rotation=45) [Link](range(len([Link])),
[Link])
[Link]()
print("\n✅ Completed By USN - 1SP22IC014!")

Output:

Q6. Visual Data Representation

30
 Activity Based Learning - 1SP22IC02
BIC703 6

✅ Completed By USN - 1SP22IC014

Q7. Sparse vs. Dense Data Representation Create an example of a sparse matrix and a dense matrix
using NumPy or SciPy. Convert a dense matrix into a sparse representation. Explain where sparse
data representation is beneficial (e.g., text mining, recommender systems).

dense_matrix =
[Link]([ [5, 1, 3],
[2, 0, 4],
[0, 6, 7]
])
print("Dense Matrix:")
print(dense_matrix)
sparse_matrix =
[Link]([
[0, 0, 3, 0],
[4, 0, 0, 0],
[0, 0, 0, 5],
[0, 2, 0, 0]
])
print("\nSparse Matrix (normal ndarray):")
print(sparse_matrix)
sparse_csr = csr_matrix(sparse_matrix)
print("\nSparse Matrix (CSR
Representation):") print(sparse_csr)
print("\nData stored in CSR:")
print(sparse_csr.data)
print("\nRow indices:", sparse_csr.indices)
print("\n✅ Sparse representation stores only non-zero values
efficiently!") print("\n ---Completed By USN - 1SP22IC014")

Output:
Q7. Sparse vs Dense Data Representation

Dense Matrix:
[[5 1 3]
[2 0 4]
[0 6 7]]
Sparse Matrix (normal
ndarray): [[0 0 3
0]
[4 0 0 0]
[0 0 0 5]
[0 2 0 0]]
Sparse Matrix (CSR
Representation): (0,
2) 3
(1, 0) 4
(2, 3) 5
(3, 1) 2
Data stored in CSR:
[3 4 5 2]
31
 Activity Based Learning - 1SP22IC02
BIC703 6

Row indices: [2 0 3 1]

---Completed By USN - 1SP22IC014

Q8. Vector and Matrix Representation for ML Models Convert your dataset into a NumPy array or
matrix format. Display the shape and data type of this matrix. Explain how representing data as
vectors and matrices enables efficient mathematical computation in ML algorithms.

data = {
"Category": ["Footwear","Clothing","Electronics","Accessories","Bags",
"Electronics","Clothing","Accessories","Footwear","Electronics"],
"Brand": ["Nike","Adidas","HP","Fossil","Puma",
"Sony","Nike","Puma","Adidas","Apple"],
"Size":
["Medium","Large","None","Small","Large",
"Medium","Medium","Small","Large","None"],
"Payment_Method":
["Online","Cash","UPI","Online","Card",
"UPI","Cash","Card","Online","Card"]
}
df =
[Link](data)
print("Original
DataFrame:") print(df)
df_encoded = [Link](lambda col:
[Link](col)[0]) matrix =
df_encoded.to_numpy()
print("\nConverted NumPy
Matrix:") print(matrix)
print("\nShape of Matrix:",
[Link]) print("Data Type:",
[Link])
print("\n -- Completed By USN - 1SP22IC014")

Output:
Q8. Vector and Matrix Representation for ML

Models Original DataFrame:

Category Brand Size Payment_Method
0 Footwear Nike Medium Online
1 Clothing Adidas Large Cash
2 Electronics HP None UPI
3 Accessories Fossil Sma Online
4 Bags ll Large Card
5 Electronics Sony Pum UPI
a
Mediu
m
6 Clothing Nike Medium Cash
7 Accessories Puma Small Card
Converted NumPy Matrix:
[[0 0 0 0]
[1 1 1 1]
[2 2 2 2]
[3 3 3 0]
[4 4 1 3]
[2 5 0 2]
[1 0 0 1]
[3 4 3 3]
[0 1 1 0]
[2 6 2 3]]

Shape of Matrix:
(10, 4) Data Type:
int64

-- Completed By USN - 1SP22IC014

QG. Feature Scaling and Normalization for Representation Apply StandardScaler and MinMaxScaler on
numerical features. Compare how the values are represented before and after scaling. Explain how
scaling affects model convergence and distance-based algorithms.
32
Activity Based Learning - 1SP22IC02
BIC703
data = {
6
"Price": [1200, 55000, 8000, 2000, 3000, 1400, 60000, 3500, 500, 7500],

33
Activity Based Learning - 1SP22IC02
BIC703 6

"Discount": [5, 10, 7, 12, 3, 8, 15, 9, 4, 6],

"Rating": [4.2, 3.5, 4.5, 3.8, 4.0, 4.7, 3.9, 4.3, 4.1, 4.4] }
df = [Link](data)
print("Original Numerical
Data:") print(df)
std_scaler = StandardScaler()
standard_scaled =
std_scaler.fit_transform(df)
minmax_scaler = MinMaxScaler()
minmax_scaled = minmax_scaler.fit_transform(df)
df_std = [Link](standard_scaled,
columns=[Link]) df_minmax =
[Link](minmax_scaled, columns=[Link])
print("\n--- Standard Scaled Values (Z-Score) ---")
print(df_std)
print("\n--- MinMax Scaled Values (0 to 1
Range) ---") print(df_minmax)
print("\n- - -Completed By USN - 1SP22IC014")

output:

Q9. Feature Scaling and Normalization

Original Numerical Data:

Price Discount Rating
0 1200 5 4.2
1 55000 10
3.5
2 8000 7 4.5
3 2000 12 3.8
4 3000 3 4.0
5 1400 8 4.7
6 60000 15
3.9
7 3500 9 4.3
8 500 4 4.1
9 7500 6 4.4

--- Standard Scaled Values (Z-Score) ---

Price Discount Rating
0 -0.596632 -0.820572 0.177394
1 1.870607 0.594207 -1.892200
2 -0.284787 -0.254660 1.064362
3 -0.559944 1.160119 -1.005231
4 -0.514084 -1.386484 -0.413919
5 -0.587460 0.028296 1.655675
6 2.099904 2.008987 -0.709575
7 -0.491155 0.311252 0.473050
8 -0.628733 -1.103528 -0.118262
9 -0.307717 -0.537616 0.768706

--- MinMax Scaled Values (0 to 1 Range) ---

Price Discount Rating
0 0.011765 0.166667 0.583333
1 0.915966 0.583333 0.000000
2 0.126050 0.333333 0.833333
3 0.025210 0.750000 0.250000
4 0.042017 0.000000 0.416667
5 0.015126 0.416667 1.000000
6 1.000000 1.000000 0.333333
7 0.050420 0.500000 0.666667
8 0.000000 0.083333 0.500000
9 0.117647 0.250000 0.750000

---- Completed By USN - 1SP22IC014

Q10. Reflection and Discussion Answer the following questions in Markdown: 1. Why is proper data
representation crucial in building effective ML models? 2. How does data representation differ between
numerical, categorical, and text data? 3. What steps would you take to ensure your dataset is well-
represented for model training and interpretation?

[Link] is proper data representation crucial in building effective ML models?

34
 Activity Based Learning - 1SP22IC02
BIC703 6

Proper data representation is essential because machine learning models rely on patterns within the input data. If
data is poorly represented, the model may:
● Misinterpret relationships between features
● Produce biased or inaccurate predictions
● Perform poorly on unseen data
● Converge slower or not learn meaningful
patterns Good representation:
● Preserves useful information
● Reduces noise and redundancy
● Improves interpretability
● Enhances training efficiency and accuracy
In simple terms, better representation = better learning.

[Link] does data representation differ between numerical, categorical, and text data?
● Numerical Data
o Represents quantities and continuous values.
o Often requires scaling or normalization.
o Supports distance-based algorithms directly.
● Categorical Data
o Represents labels or groups (e.g., colors, types).
o Must be encoded (One-Hot, Label Encoding).
o Cannot be used directly in distance-based models.
● Text Data
o Represents unstructured language information.
o Requires tokenization, TF-IDF, embeddings, or vectorization.
o Has high dimensionality and sparsity.
Each type needs a different encoding strategy to be meaningful for model training.

[Link] steps would you take to ensure your dataset is well-represented for model training
and interpretation? To ensure high-quality representation:
● Handle missing values using strategies like mean, median, or modelling.
● Encode categorical variables using the appropriate technique.
● Scale numerical features for distance-based algorithms.
● Normalize distributions if skewed.
● Remove duplicates or irrelevant features to reduce noise.
● Perform feature selection to reduce dimensionality and improve generalization.
● Use text vectorization (TF-IDF, word embeddings) for documents.
● Detect and handle outliers to prevent bias.
● Conduct exploratory data analysis (EDA) to understand relationships.
● Apply dimensionality reduction (PCA) when dealing with high dimensions.

Assignment 1.3 : Programming Assignment: Implementation of Important Concepts of Regression Models

Q1. Dataset Loading and Overview Load a regression dataset (you may use [Link] such as California
Housing, Boston Housing, or any other dataset). Display the first 5 rows of the dataset. Print dataset
shape, column names, and data types. Identify the input (independent) features and target (dependent)
variable.

data = fetch_california_housing()
df = [Link]([Link], columns=data.feature_names)
df['Target'] = [Link]
print("\nFirst 5 rows of the dataset:")
print([Link]())
print("\nDataset Shape (rows, columns):")
print([Link])
print("\nColumn Names:")
print([Link]) print("\
nData Types:")
print([Link])
print("\nInput (Independent) Features:")
print(data.feature_names) print("\
nTarget (Dependent) Variable:")
print("Target (House Value)")

Output:

35
Activity Based Learning - 1SP22IC02
BIC703 6

Q1. DATASET LOADING AND OVERVIEW

First 5 rows of the dataset:

MedInc HouseAge AveRooms AveBedrms Population AveOccup Latitude \
0 8.3252 41.0 6.984127 1.023810 322.0 2.555556 37.88
1 8.3014 21.0 6.238137 0.971880 2401.0 2.109842 37.86
2 7.2574 52.0 8.288136 1.073446 496.0 2.802260 37.85
3 5.6431 52.0 5.817352 1.073059 558.0 2.547945 37.85
4 3.8462 52.0 6.281853 1.081081 565.0 2.181467 37.85

Longitude Target
0 -122.23 4.526
1 -122.22 3.585
2 -122.24 3.521
3 -122.25 3.413
4 -122.25 3.422

Dataset Shape (rows, columns):

(20640, 9)

Column Names:
Index(['MedInc', 'HouseAge', 'AveRooms', 'AveBedrms', 'Population',
'AveOccup', 'Latitude', 'Longitude', 'Target'],
dtype='object')

Data Types:
MedInc float64
HouseAge
float64
AveRooms
float64
AveBedrms
float64
Population
float64 AveOccup
float64 Latitude
float64
Longitude
float64 Target
float64
dtype: object
Input (Independent) Features:
['MedInc', 'HouseAge', 'AveRooms', 'AveBedrms', 'Population', 'AveOccup', 'Latitude', 'Longitude']

Target (Dependent) Variable:

Target (House Value)

Q2. Exploratory Data Analysis (EDA) Perform exploratory data analysis to understand the dataset. Check
for missing values and handle them if necessary. Show statistical summary using .describe(). 4 Plot
distributions of key features and the target variable. Create a correlation heatmap to identify relationships
between features.

data = fetch_california_housing()yy
df = [Link]([Link],
columns=data.feature_names) df['Target'] = [Link]
print("\nChecking missing
values:") print([Link]().sum())
if [Link]().sum().sum() > 0:
[Link]([Link](), inplace=True)
print("Missing values handled using mean
imputation.") else:
print("No missing values found.")
print("\nStatistical Summary:")
print([Link]())
print("\nPlotting distributions of key features...")
df[['MedInc', 'AveRooms', 'Population',
'Target']].hist(figsize=(10,8)) [Link]("Distribution of Key
Features", fontsize=14)
[Link]()
print("\nGenerating correlation heatmap...")
[Link](figsize=(10,6))
[Link]([Link](), annot=False,
cmap='coolwarm') [Link]("Correlation Heatmap")
36
Activity Based Learning - 1SP22IC02
BIC703
[Link]()
6
output:

37
Activity Based Learning - 1SP22IC02
BIC703 6

Q2. EXPLORATORY DATA ANALYSIS (EDA)

Checking missing values:

MedInc 0
HouseAge 0
AveRooms 0
AveBedrms 0
Population 0
AveOccup 0
Latitude 0
Longitude 0
Target 0
dtype:
int64
No missing values found.

Statistical Summary:
MedInc HouseAge AveRooms AveBedrms Population \
count 20640.000000 20640.000000 20640.000000 20640.000000 20640.000000
mean 3.870671 28.639486 5.429000 1.096675 1425.476744
std 1.899822 12.585558 2.474173 0.473911 1132.462122
min 0.499900 1.000000 0.846154 0.333333 3.000000
25% 2.563400 18.000000 4.440716 1.006079 787.000000
50% 3.534800 29.000000 5.229129 1.048780 1166.000000
75% 4.743250 37.000000 6.052381 1.099526 1725.000000
max 15.000100 52.000000 141.909091 34.066667 35682.000000

AveOccup Latitude Longitude Target

count 20640.000000 20640.000000 20640.000000 20640.000000
mean 3.070655 35.631861 -119.569704 2.068558
std 10.386050 2.135952 2.003532 1.153956
min 0.692308 32.540000 -124.350000 0.149990
25% 2.429741 33.930000 -121.800000 1.196000
50% 2.818116 34.260000 -118.490000 1.797000
75% 3.282261 37.710000 -118.010000 2.647250
max 1243.333333 41.950000 -114.310000 5.000010

Plotting distributions of key features... Generating correlation heatmap...

Q3. Data Preparation for Regression Split the dataset into training and testing sets (e.g., 80% train, 20%
test). Apply feature scaling using StandardScaler or MinMaxScaler. Print the shapes of training and
testing data. Explain why data splitting and scaling are important before training a regression model.

data = fetch_california_housing()
df = [Link]([Link],
columns=data.feature_names) df['Target'] = [Link]
X = [Link]('Target',
axis=1) y = df['Target']
X_train, X_test, y_train, y_test =
train_test_split( X, y, test_size=0.2,
random_state=42

38
Activity Based Learning - 1SP22IC02
BIC703 6

)
print("\nTraining data
shape:") print(X_train.shape,
y_train.shape) print("\
nTesting data shape:")
print(X_test.shape,
y_test.shape) scaler =
StandardScaler()
X_train_scaled =
scaler.fit_transform(X_train) X_test_scaled
= [Link](X_test) print("\nScaling
completed using StandardScaler.") print("\n✅
Q3 Completed by USN: 1SP22IC014")

output:
Q3. DATA PREPARATION FOR REGRESSION

Training data shape:

(16512, 8) (16512,)
Testing data shape:
(4128, 8) (4128,)
Scaling completed using StandardScaler.
✅ Q3 Completed by USN: 1SP22IC014

Q4. Simple Linear Regression Implement a Simple Linear Regression model using one independent
variable (e.g., MedInc → MedHouseValue). Train the model and display the regression coefficient and
intercept. Plot the regression line along with actual data points. Evaluate the model using R² score and
RMSE (Root Mean Squared Error).

data = fetch_california_housing()
df = [Link]([Link],
columns=data.feature_names) df['Target'] = [Link]
X=
df[['MedInc']] y
= df['Target']
X_train, X_test, y_train, y_test =
train_test_split( X, y, test_size=0.2,
random_state=42
)
model =
LinearRegression()
[Link](X_train, y_train)
y_pred =
[Link](X_test)
print("\nRegression Coefficient (Slope):",
model.coef_[0]) print("Regression Intercept:",
model.intercept_)
r2 = r2_score(y_test, y_pred)
rmse = [Link](mean_squared_error(y_test, y_pred))
print("\nR² Score:", r2)
print("RMSE:", rmse)
[Link](X_test, y_test)
[Link](X_test, y_pred, linewidth=2 ,color='g')
[Link]("Median Income (MedInc)")
[Link]("Median House Value (Target)")
[Link]("Simple Linear Regression: MedInc vs
Target") [Link]()
print("\n✅ Q4 Completed by USN: 1SP22IC014")

output:

Q4. SIMPLE LINEAR REGRESSION

Regression Coefficient (Slope) : 0.41933849393812

73
Regression Intercept : 0.44459729169078
72
R² Score : 0.45885918903846
656
RMSE : 0.84209012414144
54

39
Activity Based Learning - 1SP22IC02
BIC703
✅ Q4 Completed by USN: 1SP22IC014
6

Q5. Multiple Linear Regression Implement Multiple Linear Regression using all numerical features.
Display model coefficients and intercept. Evaluate performance using R² and RMSE on both training and
test sets.
Compare results with your Simple Linear Regression model and explain the difference.
data = fetch_california_housing()

40
Activity Based Learning - 1SP22IC02
BIC703 6

df = [Link]([Link],
columns=data.feature_names) df['Target'] = [Link]
X = [Link]('Target',
axis=1) y = df['Target']
X_train, X_test, y_train, y_test =
train_test_split( X, y, test_size=0.2,
random_state=42
)
model = LinearRegression()
[Link](X_train, y_train)
y_train_pred =
[Link](X_train)
y_test_pred =
[Link](X_test) print("\
nModel Coefficients:")
for feature, coef in zip([Link], model.coef_):
print(f"{feature}: {coef}")
print("\nModel Intercept:",
model.intercept_) train_r2 =
r2_score(y_train, y_train_pred)
train_rmse = [Link](mean_squared_error(y_train,
y_train_pred)) test_r2 = r2_score(y_test, y_test_pred)
test_rmse = [Link](mean_squared_error(y_test,
y_test_pred)) print("\nTraining R² Score:", train_r2)
print("Training RMSE:",
train_rmse) print("\nTesting R²
Score:", test_r2) print("Testing
RMSE:", test_rmse)
print("\n✅ Q5 Completed by USN: 1SP22IC014")

output:
Q5. MULTIPLE LINEAR REGRESSION

Model Coefficients:
MedInc : 0.4486749096657176
HouseAge :
0.009724257517905023
AveRooms : -0.12332334282795863
AveBedrms : 0.7831449067929727
Population :-
2.0296205801885327e-
06
AveOccup : -0.00352631848713407
Latitude : -0.41979248658835966
Longitude : -0.43370806496398745

Model Intercept : -37.02327770606405

Training R² Score :
0.6125511913966953 Training RMSE
:
0.7196757085831573

Testing R² Score :
0.5757877060324507 Testing RMSE
:
0.7455813830127765

✅ Q5 Completed by USN: 1SP22IC014

Q6. Polynomial Regression Use PolynomialFeatures (degree = 2 or 3) to create polynomial terms. Fit a
Polynomial Regression model using the transformed data. Compare the model’s R² and RMSE with Linear
Regression. Discuss the impact of polynomial degree on model performance (underfitting vs. overfitting).
data = fetch_california_housing()
df = [Link]([Link],
columns=data.feature_names) df['Target'] = [Link]
X=
df[['MedInc']] y
= df['Target']
X_train, X_test, y_train, y_test =
train_test_split( X, y, test_size=0.2,
random_state=42
41
Activity Based Learning - 1SP22IC02
BIC703
)
6
poly = PolynomialFeatures(degree=2)
X_train_poly =
poly.fit_transform(X_train) X_test_poly =
[Link](X_test) poly_model =
LinearRegression()
poly_model.fit(X_train_poly, y_train)
y_train_pred =
poly_model.predict(X_train_poly)
y_test_pred =
poly_model.predict(X_test_poly) train_r2
= r2_score(y_train, y_train_pred)
test_r2 = r2_score(y_test, y_test_pred)

42
Activity Based Learning - 1SP22IC02
BIC703 6

train_rmse = [Link](mean_squared_error(y_train,
y_train_pred)) test_rmse =
[Link](mean_squared_error(y_test, y_test_pred)) print("\
nPolynomial Regression Results:")
print("Training R²:", train_r2)
print("Testing R²:", test_r2)
print("Training RMSE:",
train_rmse) print("Testing
RMSE:", test_rmse)
simple_model =
LinearRegression()
simple_model.fit(X_train,
y_train)
y_test_simple =
simple_model.predict(X_test) simple_r2
= r2_score(y_test, y_test_simple)
simple_rmse = [Link](mean_squared_error(y_test,
y_test_simple)) print("\nSimple Linear Regression
Comparison:")
print("Simple Linear R²:", simple_r2)
print("Simple Linear RMSE:",
simple_rmse)
print("\n✅ Q6 Completed by USN: 1SP22IC014")

output:
Q6. POLYNOMIAL REGRESSION

Polynomial Regression Results:

Training R² :
0.48159777620556554 Testing R² :
0.46331772769346224
Training RMSE :
0.8324594871293055 Testing RMSE
:
0.8386138969115731

Simple Linear Regression Comparison:

Simple Linear R² :
0.45885918903846656 Simple Linear
RMSE : 0.8420901241414454

✅ Q6 Completed by USN: 1SP22IC014

Q7. Regularized Regression Models (Ridge and Lasso) Implement Ridge Regression and Lasso
Regression using scikit-learn. Train models using different regularization parameters (e.g., alpha = 0.1, 1,
10). Compare R² and RMSE for each model. Explain how regularization helps prevent overfitting.

data = fetch_california_housing()
df = [Link]([Link],
columns=data.feature_names) df['Target'] = [Link]
X = [Link]('Target',
axis=1) y = df['Target']
X_train, X_test, y_train, y_test =
train_test_split( X, y, test_size=0.2,
random_state=42
)
scaler = StandardScaler()
X_train =
scaler.fit_transform(X_train) X_test
= [Link](X_test) alphas
= [0.1, 1, 10]
print("\n----- Ridge Regression Results")
for alpha in alphas:
ridge =
Ridge(alpha=alpha)
[Link](X_train, y_train)
y_pred =
[Link](X_test) r2 =
r2_score(y_test, y_pred)
rmse = [Link](mean_squared_error(y_test,
y_pred)) print(f"\nAlpha = {alpha}")
print("R² Score:", r2)
print("RMSE:", rmse)
43
Activity Based Learning - 1SP22IC02
BIC703
print("\n----- Lasso Regression Results ")
6
for alpha in alphas:
lasso =
Lasso(alpha=alpha)
[Link](X_train, y_train)
y_pred =
[Link](X_test) r2 =
r2_score(y_test, y_pred)
rmse = [Link](mean_squared_error(y_test,
y_pred)) print(f"\nAlpha = {alpha}")
print("R² Score:", r2)
print("RMSE:", rmse)
print("\n✅ Q7 Completed by USN: 1SP22IC014")

44
Activity Based Learning - 1SP22IC02
BIC703 6

Output:

Q7. REGULARIZED REGRESSION MODELS (RIDGE C LASSO)

----- Ridge Regression Results -----

Alpha = 0.1
R² Score: 0.5757905180002312
RMSE: 0.7455789118982767

Alpha = 1
R² Score: 0.5758157428913682
RMSE: 0.745556744281478

Alpha = 10
R² Score: 0.576059903284837
RMSE: 0.7453421422218557

----- Lasso Regression Results -----

Alpha = 0.1
R² Score: 0.48136113250290735
RMSE: 0.8243961598848472

Alpha = 1
R² Score: -0.00021908714592466794
RMSE: 1.1448563543099792

Alpha = 10
R² Score: -0.00021908714592466794
RMSE: 1.1448563543099792

✅ Q7 Completed by USN: 1SP22IC014

Q8. Model Evaluation and Comparison Create a summary table comparing the following models: o Simple
Linear Regression o Multiple Linear Regression o Polynomial Regression 5 Activity Based Learning -BIC703
o Ridge Regression o Lasso Regression Include metrics such as RMSE and R² Score for each model.
Visualize comparison results using a bar chart.
data = load_diabetes()
df = [Link]([Link],
columns=data.feature_names) df["Target"] = [Link]
X = [Link]("Target",
axis=1) y = df["Target"]
X_train, X_test, y_train, y_test =
train_test_split( X, y, test_size=0.2,
random_state=42
)
scaler = StandardScaler()
X_train_scaled =
scaler.fit_transform(X_train)
X_test_scaled =
[Link](X_test) X_train_single
= X_train[["bmi"]] X_test_single =
X_test[["bmi"]]
simple_model = LinearRegression()
simple_model.fit(X_train_single, y_train)
y_pred_simple =
simple_model.predict(X_test_single)
multi_model = LinearRegression()
multi_model.fit(X_train_scaled, y_train)
y_pred_multi =
multi_model.predict(X_test_scaled) poly =
PolynomialFeatures(degree=2)
X_train_poly =
poly.fit_transform(X_train_single)
X_test_poly =
[Link](X_test_single) poly_model
= LinearRegression()
poly_model.fit(X_train_poly, y_train)
y_pred_poly =
poly_model.predict(X_test_poly) ridge =
Ridge(alpha=1)
45
Activity Based Learning - 1SP22IC02
BIC703
[Link](X_train_scaled, y_train)
6
y_pred_ridge =
[Link](X_test_scaled) lasso =
Lasso(alpha=1)
[Link](X_train_scaled, y_train)

46
Activity Based Learning - 1SP22IC02
BIC703 6

y_pred_lasso =
[Link](X_test_scaled) models = [
"Simple Linear
Regression", "Multiple
Linear Regression",
"Polynomial Regression",
"Ridge Regression",
"Lasso Regression"
]
r2_scores = [
r2_score(y_test,
y_pred_simple),
r2_score(y_test,
y_pred_multi),
r2_score(y_test,
y_pred_poly),
r2_score(y_test,
y_pred_ridge),
r2_score(y_test,
y_pred_lasso),
]
rmse_scores = [
[Link](mean_squared_error(y_test,
y_pred_simple)),
[Link](mean_squared_error(y_test,
y_pred_multi)),
[Link](mean_squared_error(y_test,
y_pred_poly)),
[Link](mean_squared_error(y_test,
y_pred_ridge)),
[Link](mean_squared_error(y_test,
y_pred_lasso)),
]
summary = [Link]({
"Model": models,
"R2 Score": r2_scores,
"RMSE": rmse_scores
})
print(summary)
[Link]()
[Link](models, r2_scores)
[Link](rotation=45)
[Link]("Model Comparison - R2
Score") [Link]("R2 Score")
[Link]()
[Link]()
[Link](models, rmse_scores)
[Link](rotation=45)
[Link]("Model Comparison -
RMSE") [Link]("RMSE")
[Link]()

Output:

================ Q8. Model Evaluation and Comparison ================

Model R2 Score RMSE

0 Simple Linear 0.233350
Regression 63.732456
1 Multiple Linear 0.452603 53.85344
Regression 6
2 Polynomial 0.228972
Regression 63.914204
3 Ridge Regression 0.454147
53.777454
4 Lasso Regression 0.466877
53.146666

✅ Q8 Completed by USN: 1SP22IC014

47
Activity Based Learning - 1SP22IC02
BIC703 6

QG. Residual Analysis Plot residuals (y_actual - y_predicted) for at least two models. Check if residuals
are randomly distributed. Explain what residual analysis tells you about model bias and variance.

48
Activity Based Learning - 1SP22IC02
BIC703 6

data = load_diabetes()
df = [Link]([Link],
columns=data.feature_names) df["Target"] = [Link]
X = [Link]("Target",
axis=1) y = df["Target"]
X_train, X_test, y_train, y_test =
train_test_split( X, y, test_size=0.2,
random_state=42
)
scaler = StandardScaler()
X_train_scaled =
scaler.fit_transform(X_train) X_test_scaled
= [Link](X_test) multi_model =
LinearRegression()
multi_model.fit(X_train_scaled, y_train)
y_pred_multi =
multi_model.predict(X_test_scaled) poly =
PolynomialFeatures(degree=2)
X_train_poly =
poly.fit_transform(X_train_scaled)
X_test_poly =
[Link](X_test_scaled)
poly_model = LinearRegression()
poly_model.fit(X_train_poly, y_train)
y_pred_poly =
poly_model.predict(X_test_poly)
residuals_multi = y_test - y_pred_multi
residuals_poly = y_test - y_pred_poly
[Link]()
[Link](y_pred_multi, residuals_multi , color='g')
[Link](0)
[Link]("Residuals - Multiple Linear
Regression") [Link]("Predicted
Values") [Link]("Residuals")
[Link]()
[Link]()
[Link](y_pred_poly, residuals_poly, color='g')
[Link](0)
[Link]("Residuals - Polynomial
Regression") [Link]("Predicted
Values") [Link]("Residuals")
[Link]()
print("\n✅ Q9 Completed by USN: 1SP22IC014")

Output:
================ Q9. Residual Analysis ================

✅ Q9 Completed by USN: 1SP22IC014

Q10. Reflection and Discussion Answer the following in Markdown: 1. Which regression model performed
the best on your dataset and why? 2. How did regularization affect model complexity and performance? 3.
What are the trade-offs between model interpretability and accuracy in regression models? 4. What steps
would you take next to further improve your model’s performance?

[Link] regression model performed the best on your dataset and why?
Among the models tested, the Ridge Regression / Polynomial Regression / Multiple Linear Regression
(choose based on your results) performed the best. This is because:

49
Activity Based Learning - 1SP22IC02
BIC703 6
● It captured the underlying relationship between features and the target more effectively.
● It handled multicollinearity better (especially Ridge).

50
 Activity Based Learning - 1SP22IC02
BIC703 6

● It generalized well on unseen data, indicated by lower test error.

Overall, the model achieved the best balance between bias and variance compared to simpler or more complex alternatives.

[Link] did regularization affect model complexity and

performance? Regularization (L2 in Ridge):
● Reduced model complexity by penalizing large coefficients.
● Prevented overfitting, especially when features were highly correlated.
● Improved generalization on the test set.
● Slightly reduced training accuracy, but increased reliability on unseen data.
In summary, regularization trades a small amount of training accuracy for improved stability and robustness.

[Link] are the trade-offs between model interpretability and accuracy in regression models?
● Simple models such as Linear Regression are highly interpretable. Each coefficient clearly explains a feature’s
contribution. However, these models may struggle with complex, non-linear relationships.
● More complex approaches like Polynomial Regression or regularized models can achieve higher accuracy but:
o Are harder to interpret
o Have coefficients that no longer map directly to simple feature
relationships So, more complexity often means less interpretability but potentially
higher predictive performance.

[Link] steps would you take next to further improve your model’s performance?
To improve performance further:
● Tune hyperparameters (degree of polynomial, regularization strength, etc.)
● Conduct more feature engineering such as interaction features or domain-specific transformations.
● Try advanced models like Gradient Boosting or Random Forest Regression.
● Perform cross-validation to ensure stable performance.
● Remove outliers, which can heavily influence regression results.
● Use feature selection to eliminate irrelevant predictors that
introduce noise. These steps would likely boost accuracy and make the
model more generalizable.

Program 1.4 : Programming Assignment: Implementation of Important Concepts of Nearest Neighbor Based Models

Q1. Dataset Loading and Overview Load a suitable dataset for a regression or classification task (you may
use [Link], Kaggle datasets, or a local CSV file). Display the first 5 rows of the dataset. Print
dataset shape, column names, and data types. Identify the target (dependent) variable and independent
features.

data = fetch_california_housing()
df = [Link]([Link], columns=data.feature_names)
df['PRICE'] = [Link]
print([Link]())
print([Link])
print([Link])
print([Link])
target = "PRICE"
features = [Link][:-1]
print(target)
print(list(features))

output:
MedInc HouseAge AveRooms AveBedrms Population AveOccup Latitude \
0 8.3252 41.0 6.984127 1.023810 322.0 2.555556 37.88
1 8.3014 21.0 6.238137 0.971880 2401.0 2.109842 37.86
2 7.2574 52.0 8.288136 1.073446 496.0 2.802260 37.85
3 5.6431 52.0 5.817352 1.073059 558.0 2.547945 37.85
4 3.8462 52.0 6.281853 1.081081 565.0 2.181467 37.85

Longitude PRICE
0 -122.23 4.526
1 -122.22 3.585
2 -122.24 3.521
3 -122.25 3.413
4 -122.25 3.422
(20640,
9)
Index(['MedInc', 'HouseAge', 'AveRooms', 'AveBedrms', 'Population', 'AveOccup', 'Latitude', 'Longitude',
'PRICE'], dtype='obj ect') MedInc float64

51
Activity Based Learning - 1SP22IC02
BIC703 HouseAge float64 6
AveRooms float64

52
Activity Based Learning - 1SP22IC02
BIC703 6

AveBedrms
float64
Population
float64 AveOccup
float64 Latitude
float64
Longitude
PRIC float64 PRICE
E float64
dtype: object
['MedInc', 'HouseAge', 'AveRooms', 'AveBedrms', 'Population', 'AveOccup', 'Latitude', 'Longitude']

Q2. Data Preprocessing Perform necessary preprocessing steps: Handle missing values if any. Encode
categorical variables using Label Encoding or One-Hot Encoding. Apply feature scaling (standardization or
normalization). Explain why feature scaling is critical for Nearest Neighbor algorithms.

data = fetch_california_housing()
df = [Link]([Link],
columns=data.feature_names) df["PRICE"] = [Link]
df =
[Link]([Link]())
scaler =
StandardScaler()
scaled = scaler.fit_transform([Link]("PRICE",
axis=1)) scaled_df = [Link](scaled,
columns=[Link][:-1]) scaled_df["PRICE"] =
df["PRICE"]
print(scaled_df.head())
X = scaled_df.drop("PRICE",
axis=1) y =
scaled_df["PRICE"]
x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

Output:

MedInc HouseAge AveRooms AveBedrms Population AveOccup Latitude \

0 2.344766 0.982143 0.628559 -0.153758 -0.974429 -0.049597 1.052548
1 2.332238 -0.607019 0.327041 -0.263336 0.861439 -0.092512 1.043185
2 1.782699 1.856182 1.155620 -0.049016 -0.820777 -0.025843 1.038503
3 0.932968 1.856182 0.156966 -0.049833 -0.766028 -0.050329 1.038503
4 -0.012881 1.856182 0.344711 -0.032906 -0.759847 -0.085616 1.038503

Longitude PRICE
0 -1.327835 4.526
1 -1.322844 3.585
2 -1.332827 3.521
3 -1.337818 3.413
4 -1.337818 3.422

Q3. Train-Test Split Split the dataset into training and testing sets (e.g., 80% train, 20% test). Display the
shapes of the resulting datasets. Explain the importance of splitting data for model evaluation.
data = fetch_california_housing()
df = [Link]([Link],
columns=data.feature_names) df["PRICE"] = [Link]
X = [Link]("PRICE",
axis=1) y = df["PRICE"]
x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42) print(x_train.shape)
print(x_test.shape)
print(y_train.shape)
print(y_test.shape)

Output:

(16512, 8)
(4128, 8)
(16512,)
(4128,)

53
Activity Based Learning - 1SP22IC02
BIC703 6
Q4. Implement K-Nearest Neighbors (KNN) Classifier or Regressor For classification tasks: use
KNeighborsClassifier. For regression tasks: use KNeighborsRegressor. Train the model with a specific

54
Activity Based Learning - 1SP22IC02
BIC703 6

value of k (number of neighbors). Display model parameters and basic performance metrics (e.g.,
accuracy for classification or R²/RMSE for regression).

data = fetch_california_housing()
df = [Link]([Link],
columns=data.feature_names) df["PRICE"] = [Link]
X = [Link]("PRICE",
axis=1) y = df["PRICE"]
x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42) model = KNeighborsRegressor(n_neighbors=5)
[Link](x_train, y_train)
y_pred =
[Link](x_test)
print(model)
print("R2 Score:", r2_score(y_test, y_pred))
print("RMSE:", [Link](mean_squared_error(y_test, y_pred)))

output :

KNeighborsRegressor()
R2 Score :
0.14631049965900345 RMSE
:
1.0576778270706204

Q5. Experiment with Different K Values Train the KNN model using different values of k (e.g., 1, 3, 5, 7, G, 11).
Evaluate and record the model’s performance for each k. Plot a graph showing model performance vs. k.
Identify the optimal k value and explain how it affects bias and variance.
data = load_iris()
X=
[Link] y
= [Link]
X_train, X_test, y_train, y_test =
train_test_split( X, y, test_size=0.2,
random_state=42)
k_values = [1, 3, 5, 7, 9, 11]
accuracies = []
for k in k_values:
knn = KNeighborsClassifier(n_neighbors=k)
[Link](X_train, y_train)
y_pred = [Link](X_test)
[Link](accuracy_score(y_test,
y_pred))
for k, acc in zip(k_values, accuracies):
print(f"k = {k}, Accuracy =
{acc:.3f}") [Link](figsize=(8, 5))
[Link](k_values, accuracies,
marker='o') [Link]("K value")
[Link]("Accuracy")
[Link]("KNN Accuracy vs K")
[Link](True)
[Link]()

output:

k = 1, Accuracy = 1.000
k = 3, Accuracy = 1.000
k = 5, Accuracy = 1.000
k = 7, Accuracy = 0.967
k = 9, Accuracy = 1.000
k = 11, Accuracy = 1.000

Q6. Distance Metrics Comparison Train KNN models using different distance metrics: o Euclidean o
Manhattan o Minkowski Compare performance results for each metric. Explain how the choice of
distance metric influences model performance.

data = fetch_california_housing()
df = [Link]([Link],
columns=data.feature_names) df["PRICE"] = [Link]
55
Activity Based Learning - 1SP22IC02
BIC703
X = [Link]("PRICE",
6
axis=1) y = df["PRICE"]
x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

56
Activity Based Learning - 1SP22IC02
BIC703 6

metrics = ["euclidean", "manhattan",

"minkowski"] results = {}
for m in metrics:
model = KNeighborsRegressor(n_neighbors=5,
metric=m) [Link](x_train, y_train)
y_pred = [Link](x_test)
results[m] = r2_score(y_test,
y_pred)
print(results)

output:

{'euclidean': 0.14631049965900345, 'manhattan': 0.24107833332250306, 'minkowski': 0.14631049965900345}

Q7. Effect of Feature Scaling (Practical Test) Train a KNN model without scaling and note its performance.
Then train the same model after scaling and compare results. Explain the observed difference in model
accuracy/performance.
data = fetch_california_housing()
df = [Link]([Link],
columns=data.feature_names) df["PRICE"] = [Link]
X = [Link]("PRICE",
axis=1) y = df["PRICE"]
x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42) model1 = KNeighborsRegressor(n_neighbors=5)
[Link](x_train,
y_train) pred1 =
[Link](x_test)
print("Without Scaling R2:", r2_score(y_test,
pred1)) scaler = StandardScaler()
x_train_scaled =
scaler.fit_transform(x_train)
x_test_scaled = [Link](x_test)
model2 =
KNeighborsRegressor(n_neighbors=5)
[Link](x_train_scaled, y_train)
pred2 = [Link](x_test_scaled)
print("With Scaling R2:", r2_score(y_test,
pred2))

output:

Without Scaling R2 :
0.14631049965900345 With Scaling
R2 :
0.6700101862970989

Q8. Model Evaluation and Visualization For classification tasks: o Display the confusion matrix and
classification report. o Plot the decision boundary (if dataset is 2D). For regression tasks: o Plot actual
vs. predicted values. o Visualize residuals. Interpret what these visualizations reveal about model
performance.
data = fetch_california_housing()
df = [Link]([Link],
columns=data.feature_names) df["PRICE"] = [Link]
X = [Link]("PRICE",
axis=1) y = df["PRICE"]
x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42) model = KNeighborsRegressor(n_neighbors=5)
[Link](x_train, y_train)
y_pred =
[Link](x_test)
[Link](y_test, y_pred,
color='r') [Link]("Actual
Values") [Link]("Predicted
Values") [Link]("Actual vs
Predicted") [Link]()
residuals = y_test - y_pred
[Link](y_pred, residuals ,
color='g') [Link]("Predicted
Values") [Link]("Residuals")
[Link]("Residual Plot")
[Link](0)

57
Activity Based Learning - 1SP22IC02
BIC703
[Link]()
6

output:

58
Activity Based Learning - 1SP22IC02
BIC703 6

QG. Hyperparameter Tuning using GridSearchCV Use GridSearchCV from scikit-learn to find the best
hyperparameters for the KNN model. Tune parameters such as: o n_neighbors o weights (uniform,
distance) o metric (euclidean, manhattan) 7 Activity Based Learning -BIC703 Display the best
parameters and their corresponding score. Discuss how hyperparameter tuning improved model
performance.

data = fetch_california_housing()
df = [Link]([Link],
columns=data.feature_names) df["PRICE"] = [Link]
X = [Link]("PRICE",
axis=1) y = df["PRICE"]
x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42) param_grid = {
'n_neighbors': [1, 3, 5, 7, 9, 11],
'weights': ['uniform', 'distance'],
'metric': ['euclidean',
'manhattan']
}
model = KNeighborsRegressor()
grid = GridSearchCV(model, param_grid, cv=5,
scoring='r2') [Link](x_train, y_train)
print(grid.best_params_)
print(grid.best_score_)

Output:

{'metric': 'manhattan', 'n_neighbors': 9, 'weights':

'distance'} 0.26048491781984023

Q10. Reflection and Discussion Answer the following in Markdown: 1. What are the main strengths and
weaknesses of K-Nearest Neighbor models? 2. How does the choice of k influence model bias and variance?
3. Why is KNN considered a lazy learning algorithm? 4. In what types of real-world applications would KNN
perform well or poorly? 5. What strategies can be used to improve the efficiency of KNN on large datasets?

[Link] are the main strengths and weaknesses of K-Nearest

Neighbor models? Strengths
● Simple and intuitive to understand.
● Works well with smaller, labeled datasets.
● No explicit training phase (fast to build).
● Can adapt to complex decision boundaries when data is well-
distributed. Weaknesses
● Computationally expensive during prediction (slow on large datasets).
● Sensitive to noise and irrelevant features.
● Requires feature scaling to avoid distance bias.
● Performance heavily depends on the choice of k and distance metric.

[Link] does the choice of k influence model bias and variance?

59
Activity Based Learning - 1SP22IC02
BIC703 6

● Small k (e.g., k = 1, 3)
o Low bias
o High variance
o Model becomes very sensitive to noise and overfits.
● Large k (e.g., k = 15, 20)
o High bias
o Low variance
o Model becomes too generalized and may
underfit. So, choosing k balances the bias-variance trade-off.

[Link] is KNN considered a lazy learning

algorithm? KNN is considered lazy because:
● It does not build an explicit training model.
● It simply stores the training data.
● Computation happens only at prediction time by calculating distances
to neighbors. This means training is fast, but prediction can be slow.

[Link] what types of real-world applications would KNN perform

well or poorly? Performs Well In
● Recommendation systems (similarity-based searching)
● Medical diagnosis (classifying similar patient records)
● Pattern recognition (handwritten digits, image similarity)
● Small to medium datasets with clean, well-scaled
features Performs Poorly In
● Large datasets (slow predictions)
● High-dimensional datasets (curse of dimensionality)
● Imbalanced datasets (dominant class bias)
● Features with different scales (distance dominated by large-scaled features)

[Link] strategies can be used to improve the efficiency of KNN on large datasets?
● Feature Scaling (StandardScaler / MinMaxScaler)
● Dimensionality Reduction (PCA, t-SNE) to reduce computation
● Feature Selection to remove noisy or irrelevant attributes
● Use Efficient Data Structures like KD-Trees or Ball Trees
● Approximate Nearest Neighbor (ANN) algorithms for faster lookups
● Sampling or clustering to reduce training set size
These approaches significantly reduce computational cost and memory usage.

Assignment 1.5: Programming Assignment: Implementation of Important Concepts of Decision Tree Models

Q1. Dataset Loading and Overview Load a suitable dataset (you may use [Link], a Kaggle dataset,
or a CSV file). Display the first few rows of the dataset. Show dataset shape, column names, and data
types. Identify the target (dependent) variable and input (independent) features. Explain whether your
problem is a classification or regression task.

from [Link] import load_breast_cancer

import pandas as pd
data = load_breast_cancer()
df = [Link]([Link],
columns=data.feature_names) df["target"] =
[Link]
print([Link]())
print([Link])
print([Link])
print([Link])
target = "target"
features = [Link][:-1]
print(target)
print(list(features))

Output:

mean radius mean texture mean perimeter mean area mean smoothness \

60
Activity Based Learning - 1SP22IC02
BIC703 6

0 17.99 10.38 122.80 1001.0 0.11840

1 20.57 17.77 132.90 1326.0 0.08474
2 19.69 21.25 130.00 1203.0 0.10960
3 11.42 20.38 77.58 386.1 0.14250
4 20.29 14.34 135.10 1297.0 0.10030

mean compactness mean concavity mean concave points mean symmetry \

0 0.27760 0.3001 0.14710 0.2419
1 0.07864 0.0869 0.07017 0.1812
2 0.15990 0.1974 0.12790 0.2069
3 0.28390 0.2414 0.10520 0.2597
4 0.13280 0.1980 0.10430 0.1809

mean fractal dimension ... worst texture worst perimeter worst area \
0 0.07871 ... 17.33 184.60

2019.0
1 0.05667 ... 23.41 158.80

1956.0
2 0.05999 ... 25.53 152.50

1709.0
3 0.09744 ... 26.50 98.87 567.7
4 0.05883 ... 16.67 152.20

1575.0

worst smoothness worst compactness worst concavity worst concave points \

0 0.1622 0.6656 0.7119 0.2654
1 0.1238 0.1866 0.2416 0.1860
2 0.1444 0.4245 0.4504 0.2430
3 0.2098 0.8663 0.6869 0.2575
4 0.1374 0.2050 0.4000 0.1625

worst symmetry worst fractal dimension target

0 0.4601 0.11890 0
1 0.2750 0.08902 0
2 0.3613 0.08758 0
3 0.6638 0.17300 0
4 0.2364 0.07678 0

[5 rows x 31 columns]
(569, 31)
Index(['mean radius', 'mean texture', 'mean perimeter', 'mean area', 'mean smoothness', 'mean
compactness', 'mean concavity','mean concave points', 'mean symmetry', 'mean fractal
dimension', 'radius error', 'texture error', 'perimeter error', 'area error', 'smoothness error',
'compactness error', 'concavity error','concave points error', 'symmetry error', 'fractal dimension
error','worst radius', 'worst texture', 'worst perimeter', 'worst area','worst smoothness', 'worst
compactness', 'worst concavity','worst concave points', 'worst symmetry', 'worst fractal
dimension','target'],dtype='object')

mean radius float64

mean texture float64
mean perimeter float64
mean area
float64
mean smoothness float64
mean compactness float64
mean concavity
float64
mean concave points
float64
mean symmetry
float64

61
Activity Based Learning - 1SP22IC02
BIC703 6
mean fractal dimension
float64 radius error
float64
texture error float64
perimeter error
float64 area
error float64
smoothness error
float64
compactness error float64
concavity error
float64
concave points error
float64 symmetry error
float64

62
Activity Based Learning - 1SP22IC02
BIC703 6

fractal dimension error

float64 worst radius

float64
worst texture float64
worst perimeter
float64 worst
area float64
worst smoothness
float64 worst
compactness float64 worst
concavity
float64 worst
concave points float64 worst
symmetry
float64 worst
fractal dimension float64
target int32
dtype:
object target
['mean radius', 'mean texture', 'mean perimeter', 'mean area', 'mean smoothness', 'mean
compactness', 'mean concavity', 'mean concave points', 'mean symmetry', 'mean fractal
dimension', 'radius error', 'texture error', 'perimeter error', 'area error', 'smoothness error',
'compactness error', 'concavity error', 'concave points error', 'symmetry error', 'fractal dimension
error', 'worst radius', 'worst texture', 'worst perimeter', 'worst area', 'worst smoothness', 'worst
compactness', 'worst concavity', 'worst concave points', 'worst symmetry', 'worst fractal
dimension']

Q2. Data Preprocessing Perform basic preprocessing steps: Handle missing values (if any). Encode
categorical features using Label Encoding or One-Hot Encoding. Apply feature scaling if necessary
(optional for tree-based models). Explain why Decision Trees may or may not require feature scaling.

data = load_breast_cancer()
df = [Link]([Link],
columns=data.feature_names) df["target"] =
[Link]
df =
[Link]([Link]())
scaler =
StandardScaler()
scaled = scaler.fit_transform([Link]("target",
axis=1)) scaled_df = [Link](scaled,
columns=[Link][:-1]) scaled_df["target"] =
df["target"]
print(scaled_df.head())

Output:

mean radius mean texture mean perimeter mean area mean smoothness \
0 1.097064 - 1.269934 1.568466
2.073335 0.984375
1 1.829821 - 1.685955 -0.826962
0.353632 1.908708
2 1.579888 0.456187 1.566503 1.558884 0.942210
3 -0.768909 0.253732 -0.592687 - 3.283553
0.764464
4 1.750297 - 1.776573 0.280372
1.151816 1.826229

mean compactness mean concavity mean concave points mean symmetry \

0 3.283515 2.652874 2.532475 2.217515
1 -0.487072 -0.023846 0.548144 0.001392
2 1.052926 1.363478 2.037231 0.939685
3 3.402909 1.915897 1.451707 2.867383
4 0.539340 1.371011 1.428493 -0.009560

mean fractal dimension ... worst texture worst perimeter worst area \
63
Activity Based Learning - 1SP22IC02
BIC703 6
0 2.255747 ... -1.359293 2.303601 2.001237
1 -0.868652 ... -0.369203 1.535126 1.890489
2 -0.398008 ... -0.023974 1.347475 1.456285
3 4.910919 ... 0.133984 -0.249939 -0.550021
4 -0.562450 ... -1.466770 1.338539 1.220724

worst smoothness worst compactness worst concavity worst concave points \

0 1.307686 2.616665 2.109526 2.296076
1 -0.375612 -0.430444 -0.146749 1.087084
2 0.527407 1.082932 0.854974 1.955000
3 3.394275 3.893397 1.989588 2.175786

64
Activity Based Learning - 1SP22IC02
BIC703 6

4 0.220556 -0.313395 0.613179 0.729259

worst symmetry worst fractal dimension target

0 2.750622 1.937015 0
1 -0.243890 0.281190 0
2 1.152255 0.201391 0
3 6.046041 4.935010 0
4 -0.868353 -0.397100 0

[5 rows x 31 columns]

Q3. Train-Test Split Split your dataset into training and testing sets (e.g., 80% train, 20% test). Print the
shapes of the resulting datasets. Explain the importance of separating training and testing data.

data = load_breast_cancer()
df = [Link]([Link],
columns=data.feature_names) df["target"] =
[Link]
X = [Link]("target",
axis=1) y = df["target"]
x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42) print(x_train.shape)
print(x_test.shape)
print(y_train.shape)
print(y_test.shape)

Output:

(455, 30)
(114, 30)
(455,)
(114,)

Q4. Build a Basic Decision Tree Model Import and train a DecisionTreeClassifier or
DecisionTreeRegressor from scikit learn. Display model parameters (e.g., max_depth, criterion,
min_samples_split). Evaluate the model’s performance on training and testing sets using appropriate
metrics: o For classification: Accuracy, Precision, Recall, F1-score. o For regression: R² score, RMSE.
Comment on whether the model shows signs of overfitting or underfitting.

data = load_breast_cancer()
df = [Link]([Link],
columns=data.feature_names) df["target"] =
[Link]
X = [Link]("target",
axis=1) y = df["target"]
x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42) model = DecisionTreeClassifier(max_depth=None,
criterion='gini', min_samples_split=2) [Link](x_train, y_train)
train_pred =
[Link](x_train)
test_pred =
[Link](x_test)
print("Train Accuracy:", accuracy_score(y_train,
train_pred)) print("Test Accuracy:",
accuracy_score(y_test, test_pred)) print("Precision:",
precision_score(y_test, test_pred)) print("Recall:",
recall_score(y_test, test_pred))
print("F1 Score:", f1_score(y_test, test_pred))

Output:

Train Accuracy : 1.0

Test Accuracy :
0.9298245614035088 Precision :
0.9436619718309859

65
Activity Based Learning - 1SP22IC02
BIC703 6
Recall : 0.9436619718309859
F1 Score : 0.9436619718309859

66
Activity Based Learning - 1SP22IC02
BIC703 6

Q5. Visualize the Decision Tree Plot the Decision Tree using plot_tree() or export_graphviz(). Label the
nodes with feature names and class labels. Interpret the structure: o Which features appear near the root
of the tree? o What does this imply about their importance?

data = load_breast_cancer()
df = [Link]([Link],
columns=list(data.feature_names)) df["target"] =
[Link]
X = [Link]("target",
axis=1) y = df["target"]
x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42) model = DecisionTreeClassifier()
[Link](x_train, y_train)
[Link](figsize=(15,8))
plot_tree(model, feature_names=list([Link]), class_names=["Benign","Malignant"],
filled=True) [Link]()

output:

Q6. Model Complexity and Pruning Experiment with different values of max_depth (e.g., 3, 5, 7,
10). Observe and record how accuracy or R² changes with depth. Plot model performance vs.
tree depth. Explain how pruning helps control overfitting in Decision Trees.

data = load_breast_cancer()
df = [Link]([Link],
columns=data.feature_names) df["target"] = [Link]
X = [Link]("target",
axis=1) y = df["target"]
x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42) depths = [3, 5, 7, 10]
scores = []
for d in depths:
model = DecisionTreeClassifier(max_depth=d)
[Link](x_train, y_train)
pred = [Link](x_test)
[Link](accuracy_score(y_test, pred))
[Link](depths, scores, marker='o',color='orange')
[Link]("Tree Depth")
[Link]("Accuracy")
[Link]("Decision Tree Performance vs
Depth") [Link]()

Output:

67
Activity Based Learning - 1SP22IC02
BIC703 6

Q7. Feature Importance Analysis Extract and display the feature importance scores from your trained model.
Visualize feature importance using a bar chart. Discuss which features contribute most to predictions
and why.
data = load_breast_cancer()
df = [Link]([Link],
columns=data.feature_names) df["target"] = [Link]
X = [Link]("target",
axis=1) y = df["target"]
x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42) model = DecisionTreeClassifier()
[Link](x_train, y_train)
importance =
model.feature_importances_
[Link](figsize=(10,6))
[Link]([Link], importance ,
color='r') [Link]("Feature
Importance Score")
[Link]("Features")
[Link]("Decision Tree Feature Importance")
[Link]()

output:

Q8. Compare Different Splitting Criteria Train multiple Decision Tree models using different criteria: o
For classification: gini vs. entropy. o For regression: squared_error vs. absolute_error. Compare
model performance for each criterion. Explain how the choice of criterion affects tree structure and
results.

data = load_breast_cancer()
df = [Link]([Link],
columns=data.feature_names) df["target"] = [Link]
X = [Link]("target",
axis=1) y = df["target"]
x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42) criteria = ["gini", "entropy"]
results = {}
for c in criteria:
model = DecisionTreeClassifier(criterion=c)
[Link](x_train, y_train)
pred = [Link](x_test)
results[c] = accuracy_score(y_test,
pred) print(results)

Output:

{'gini': 0.9298245614035088, 'entropy': 0.956140350877193}

68
Activity Based Learning - 1SP22IC02
BIC703 6

QG. Model Evaluation and Cross-Validation Perform k-fold cross-validation (e.g., k=5) to evaluate model
stability. Compare cross-validation accuracy with single train-test accuracy. Discuss what cross-
validation reveals about your model’s generalization capability.

data = load_breast_cancer()
df = [Link]([Link],
columns=data.feature_names) df["target"] = [Link]
X = [Link]("target",
axis=1) y = df["target"]
x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42) model = DecisionTreeClassifier()
[Link](x_train, y_train)
test_pred =
[Link](x_test)
print("Train-Test Accuracy:", accuracy_score(y_test,
test_pred)) cv_scores = cross_val_score(model, X, y,
cv=5)
print("Cross-Validation Accuracy:",
cv_scores) print("Mean CV Accuracy:",
cv_scores.mean())

Output:

Train-Test Accuracy : 0.9473684210526315

Cross-Validation Accuracy : [0.9122807 0.93859649 0.92982456 0.93859649 0.89380531]
Mean CV Accuracy : 0.9226207110697097

Q10. Reflection and Discussion Answer the following questions in Markdown:

1. Which Decision Tree configuration produced the best performance?
2. What are the main advantages and disadvantages of Decision Tree models?
3. How does pruning affect the bias–variance tradeoff?
4. Why are Decision Trees considered interpretable models?
5. How could you improve the performance of Decision Trees further (hint: ensembles like Random
Forests, Gradient Boosting)?

[Link] Decision Tree configuration produced the best performance?

The Decision Tree configuration that produced the best performance used:
● An optimal max_depth (preventing overfitting),
● A min_samples_split to control growth,
● And the Gini or Entropy criterion for splitting.
Tuning these hyperparameters helped the model generalize better on the test set, balancing
complexity and accuracy.

[Link] are the main advantages and disadvantages of Decision

Tree models? Advantages
● Easy to visualize and interpret.
● Requires little data preprocessing.
● Can handle both numerical and categorical features.
● Learns non-linear relationships naturally.
● Good for feature importance
analysis. Disadvantages
● Highly prone to overfitting, especially with deep trees.
● Small changes in data can drastically change the tree (high variance).
● Bias toward dominant features.
● Can become complex and lose interpretability.

[Link] does pruning affect the bias–variance tradeoff?

● Pruning increases bias slightly by simplifying the model.
● Pruning reduces variance by preventing the tree from
memorizing noise. This leads to:
✅ Better generalization
✅ Reduced overfitting
✅ More stable predictions
In other words, pruning sacrifices some training accuracy to improve test accuracy.

[Link] are Decision Trees considered interpretable models?

69
 Activity Based Learning - 1SP22IC02
BIC703 6

Decision Trees are interpretable because:

● Their structure resembles human decision-making.
● Each decision (split) can be followed step-by-step.
● Nodes clearly show which feature contributes to a prediction.
● They can be visualized as simple diagrams.
This makes them easy to explain to non-technical stakeholders (e.g., teachers, doctors, financial analysts).

[Link] could you improve the performance of Decision Trees further?

You can improve Decision Tree performance by using ensemble methods,
such as: Random Forest
● Reduces variance by averaging many trees.
● More robust and
stable. Gradient Boosting
● Learns sequentially, correcting previous mistakes.
● Very high accuracy and strong real-world
performance. XGBoost / LightGBM
● Efficient boosting frameworks for large
datasets. Other improvements:
● Tune hyperparameters (max_depth, min_samples_leaf, etc.)
● Apply feature engineering and selection
● Handle outliers appropriately
These techniques reduce overfitting and improve generalization accuracy.

Assignment 2 : Learning Supervised and Unsupervised Machine Learning Models

Data Understanding and Preprocessing

Q1. Load Dataset Load at least one dataset for regression and one for classification (e.g., California
Housing, Iris). Display first 5 rows, shape, column names, and data types. Identify target (dependent)
and input (independent) features.

import pandas as pd
from [Link] import load_wine,
fetch_openml from [Link] import
load_diabetes
reg = load_diabetes()
reg_df = [Link]([Link], columns=reg.feature_names)
reg_df["disease_progression"] = [Link]
reg_df = reg_df.head(1000)
print(reg_df.head())
print(reg_df.shape)
print(reg_df.columns)
print(reg_df.dtypes)
reg_target = "disease_progression"
reg_features = reg_df.columns[:-
1] print(reg_target)
print(list(reg_features))
cls = load_wine()
cls_df = [Link]([Link], columns=cls.feature_names)
cls_df["wine_class"] = [Link]
cls_df = [Link]([cls_df]*50).head(1000)
print(cls_df.head())
print(cls_df.shape)
print(cls_df.columns)
print(cls_df.dtypes)
cls_target =
"wine_class"
cls_features = cls_df.columns[:-1]
print(cls_target)
print(list(cls_features))

Output:

age sex bmi bp s1 s2 s3 \

0 0.038076 0.050680 0.061696 0.021872 -0.044223 -0.034821 -0.043401
1 -0.001882 -0.044642 -0.051474 -0.026328 -0.008449 -0.019163 0.074412
2 0.085299 0.050680 0.044451 -0.005670 -0.045599 -0.034194 -0.032356

70
Activity Based Learning - 1SP22IC02
BIC703 6

3 -0.089063 -0.044642 -0.011595 -0.036656 0.012191 0.024991 -0.036038

4 0.005383 -0.044642 -0.036385 0.021872 0.003935 0.015596 0.008142

s4 s5 s6 disease_progression
0 -0.002592 0.019907 - 151.0
0.017646
1 -0.039493 -0.068332 - 75.0
0.092204
2 -0.002592 0.002861 - 141.0
0.025930
3 0.034309 0.022688 - 206.0
0.009362
4 -0.002592 -0.031988 - 135.0
0.046641
(442, 11)
Index(['age', 'sex', 'bmi', 'bp', 's1', 's2', 's3', 's4',
's5', 's6', 'disease_progression'],
dtype='object'
) age

float64
sex float64
bmi float64
bp float64
s1 float64
s2 float64
s3 float64
s4 float64
s5 float64
s6 float64
disease_progression
float64 dtype: object
disease_progression
['age', 'sex', 'bmi', 'bp', 's1', 's2', 's3', 's4', 's5', 's6']

Q2. Explore Dataset Check for missing values and duplicates. Display statistical summaries (.describe())
and class distributions. Visualize numerical and categorical features.

import pandas as pd
from [Link] import load_diabetes,
load_wine import [Link] as plt
reg = load_diabetes()
reg_df = [Link]([Link],
columns=reg.feature_names) reg_df["target"] = [Link]
reg_df =
[Link]([reg_df]*20).head(1000) cls =
load_wine()
cls_df = [Link]([Link],
columns=cls.feature_names) cls_df["class"] = [Link]
cls_df =
[Link]([cls_df]*50).head(1000)
print(reg_df.isnull().sum())
print(reg_df.duplicated().sum())
print(cls_df.isnull().sum())
print(cls_df.duplicated().sum())
print(reg_df.describe())
print(cls_df.describe())
print(cls_df["class"].value_counts())
reg_df.hist(figsize=(10,8))
[Link]()
cls_df.hist(figsize=(10,8))
[Link]()
cls_df["class"].value_counts().plot(kind="
bar") [Link]()

Output:

age 0
sex 0
bmi 0
bp 0
s1 0
s2 0
s3 0
s4 0
71
Activity Based Learning - 1SP22IC02
BIC703 0
s5
6
s6 0
target 0
dtype: int64

72
Activity Based Learning - 1SP22IC02
BIC703 6

558
alcohol 0
malic_acid 0
ash 0
alcalinity_of_ash 0
magnesium 0
total_phenols 0
flavanoids 0
nonflavanoid_phenols 0
proanthocyanins 0
color_intensity 0
hue 0
od280/od315_of_diluted_wines0
proline 0
class 0
dtype: int64
822
age sex bmi bp s1 \
count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000
mean -0.001007 -0.000412 -0.000850 -0.001157 -0.001078
std 0.047884 0.047561 0.047073 0.047372 0.047173
min -0.107226 -0.044642 -0.090275 -0.112399 -0.126781
25% -0.038207 -0.044642 -0.035307 -0.036656 -0.034592
50% -0.044642 - - -
0.007284 0.005670 0.004321
0.005383
75% 0.050680 0.030440 0.03220 0.02732
1 6
0.038076
max 0.050680 0.17055 0.132044 0.15391
5 4
0.110727

s2 s3 s4 s5 s6 \
count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000
mean -0.001089 0.000897 -0.001317 -0.001195 -0.001169
std 0.047177 0.048045 0.047314 0.047396 0.047512
min -0.115613 -0.102307 -0.076395 -0.126097 -0.137767
25% -0.031376 -0.032356 -0.039493 -0.034522 -0.034215
50% -0.004602 -0.006584 -0.002592 -0.005142 -0.001078
75% 0.027496 0.030232 0.034309 0.031348 0.027917
max 0.198788 0.181179 0.185234 0.133597 0.135612

target
count 1000.000000
mean 150.504000
std 76.575988
min 25.000000
25% 85.750000
50% 139.000000
75% 206.000000
max 346.000000
alcohol malic_acid ash alcalinity_of_ash magnesium \
count 1000.000000 1000.000000 1000.00000 1000.000000 1000.000000
mean 13.012960 2.282030 2.36337 19.353800 99.925000
std 0.813528 1.088538 0.27538 3.334964 14.412741
min 11.030000 0.740000 1.36000 10.600000 70.000000
25% 12.360000 1.585000 17.000000
2.21000 88.000000
50% 13.050000 1.810000 19.000000
2.36000 98.000000
75% 13.695000 2.967500 21.500000
2.56000 108.000000
max 14.830000 5.800000 30.000000
3.23000 162.000000

total_phenols flavanoids nonflavanoid_phenols

proanthocyanins \ count 1000.000000 1000.000000
1000.000000
1000.000000
mean 2.322460 0.35718 1.609070
2.081900 0
std 0.622457 0.982021 0.123059 0.568308
min 0.980000 0.340000 0.130000 0.410000
25% 1.790000 1.272500 0.260000 1.250000

73
Activity Based Learning - 1SP22IC02
BIC703
50% 2.410000 2.190000 0.320000 1.560000
6
75% 2.800000 2.900000 0.430000 1.970000
max 3.880000 5.080000 0.660000 3.580000

color_intensity hue od280/od315_of_diluted_wines \

count 1000.000000 1000.0000
1000.000000 00
mean 4.990600 0.971196 2.649700
std 2.252854 0.223798 0.695719
min 1.280000 0.480000 1.270000

74
Activity Based Learning - 1SP22IC02
BIC703 6

25% 3.210000 0.810000 2.050000

50% 4.600000 0.995000 2.820000
75% 6.130000 1.120000 3.190000
max 13.000000 1.710000 4.000000

proline class
count 1000.000000 1000.000000
mean 758.618000 0.886000
std 319.300264 0.762618
min 278.000000 0.000000
25% 502.000000 0.000000
50% 680.000000 1.000000
75% 1020.000000 1.000000
max 1680.000000 2.000000
class
1 406
0 354
2 240
Name: count, dtype: int64

Q3. Handle Missing Values Impute missing values using appropriate techniques (mean, median, mode, or
forward fill). Explain why imputation method was chosen.

import pandas as pd
from [Link] import load_diabetes,
load_wine reg = load_diabetes()
reg_df = [Link]([Link],
columns=reg.feature_names) reg_df["target"] = [Link]
reg_df =
[Link]([reg_df]*20).head(1000) cls
= load_wine()
cls_df = [Link]([Link],
columns=cls.feature_names) cls_df["class"] = [Link]
cls_df =
[Link]([cls_df]*50).head(1000)
reg_df.iloc[0:10, 0] = None
cls_df.iloc[0:10, 0] = None
reg_df = reg_df.fillna(reg_df.mean())
cls_df =
cls_df.fillna(cls_df.mode().iloc[0])
print(reg_df.head())
print(cls_df.head())

Output:

age sex bmi bp s1 s2 s3 \

0 -0.00095 0.050680 0.061696 0.021872 -0.044223 -0.034821 -0.043401
1 -0.00095 -0.044642 -0.051474 -0.026328 -0.008449 -0.019163 0.074412
2 -0.00095 0.050680 0.044451 -0.005670 -0.045599 -0.034194 -0.032356
3 -0.00095 -0.044642 -0.011595 -0.036656 0.012191 0.024991 -0.036038
4 -0.00095 -0.044642 -0.036385 0.021872 0.003935 0.015596 0.008142

s4 s5 s6 target
0-0.002592 0.019907 -0.017646 151.0
1-0.039493 -0.068332 -0.092204 75.0
2-0.002592 0.002861 -0.025930 141.0
3 0.034309 0.022688 -0.009362 206.0
4-0.002592 -0.031988 -0.046641 135.0
alcohol malic_acid ash alcalinity_of_ash magnesium total_phenols \
0 12.37 1.71 2.43 15.6 127.0 2.80
1 12.37 1.78 2.14 11.2 100.0 2.65
2 12.37 2.36 2.67 18.6 101.0 2.80
3 12.37 1.95 2.50 16.8 113.0 3.85
4 12.37 2.59 2.87 21.0 118.0 2.80

flavanoids nonflavanoid_phenols proanthocyanins color_intensity hue \

0 3.06 0.28 2.29 5.64 1.04
1 2.76 0.26 1.28 4.38 1.05
2 3.24 0.30 2.81 5.68 1.03
3 3.49 0.24 2.18 7.80 0.86
4 2.69 0.39 1.82 4.32 1.04

od280/od315_of_diluted_wines proline
0 class
75
Activity Based Learning - 1SP22IC02
BIC703 3.92 1065.0 0 6

76
Activity Based Learning - 1SP22IC02
BIC703 6

1 3.40 1050.0 0
2 3.17 1185.0 0
3 3.45 1480.0 0
4 2.93 735.0 0

Q4. Encode Categorical Variables Encode categorical columns using Label Encoding or One-Hot Encoding.
Display the transformed dataset.

import pandas as pd
from [Link] import load_wine
from [Link] import
LabelEncoder cls = load_wine()
cls_df = [Link]([Link],
columns=cls.feature_names) cls_df["class"] = [Link]
cls_df =
[Link]([cls_df]*50).head(1000) label
= LabelEncoder()
cls_df["class"] = label.fit_transform(cls_df["class"])
print(cls_df.head())

Output:

alcohol malic_acid ash alcalinity_of_ash magnesium total_phenols \

0 14.23 1.71 2.43 15.6 127.0 2.80
1 13.20 1.78 2.14 11.2 100.0 2.65
2 13.16 2.36 2.67 18.6 101.0 2.80
3 14.37 1.95 2.50 16.8 113.0 3.85
4 13.24 2.59 2.87 21.0 118.0 2.80

flavanoids nonflavanoid_phenols proanthocyanins color_intensity hue \

0 3.06 0.28 2.29 5.64 1.04
1 2.76 0.26 1.28 4.38 1.05
2 3.24 0.30 2.81 5.68 1.03
3 3.49 0.24 2.18 7.80 0.86
4 2.69 0.39 1.82 4.32 1.04

od280/od315_of_diluted_wines proline
class
0 3.92 1065.0 0
1 3.40 1050.0 0
2 3.17 1185.0 0
3 3.45 1480.0 0
4 2.93 735.0 0

Q5. Feature Scaling and Normalization Apply StandardScaler or MinMaxScaler on numerical features.
Compare feature ranges before and after scaling. Explain why scaling is important for some algorithms.

import pandas as pd
from [Link] import load_wine
from [Link] import StandardScaler,
MinMaxScaler cls = load_wine()
df = [Link]([Link],
columns=cls.feature_names) df =
[Link]([df]*50).head(1000)
print("Before Scaling
(describe):")
print([Link]())
std_scaler = StandardScaler()
df_std = [Link](std_scaler.fit_transform(df),
columns=[Link]) print("\nAfter Standard Scaling (describe):")
print(df_std.describe())
mm_scaler =
MinMaxScaler()
df_mm = [Link](mm_scaler.fit_transform(df),
columns=[Link]) print("\nAfter MinMax Scaling (describe):")
print(df_mm.describe())

Output:

Before Scaling (describe):

77
Activity Based Learning - 1SP22IC02
BIC703
alcohol malic_acid ash alcalinity_of_ash magnesium \
6
count 1000.000000 1000.000000 1000.00000 1000.000000 1000.000000
mean 13.012960 2.282030 2.36337 19.353800 99.925000

78
Activity Based Learning - 1SP22IC02
BIC703 6

std 0.813528 1.088538 0.27538 3.334964 14.412741

min 11.030000 0.740000 1.36000 10.600000 70.000000
25% 12.360000 1.585000 17.000000
2.21000 88.000000
50% 13.050000 1.810000 19.000000
2.36000 98.000000
75% 13.695000 2.967500 21.500000
2.56000 108.000000
max 14.830000 5.800000 30.000000
3.23000 162.000000

total_phenols flavanoids nonflavanoid_phenols proanthocyanins \

count 1000.000000 1000.000000
1000.000000 1000.000000
mean 2.322460 2.081900 0.357180 1.609070
std 0.622457 0.982021 0.123059 0.568308
min 0.980000 0.340000 0.130000 0.410000
25% 1.790000 1.272500 0.260000 1.250000
50% 2.410000 2.190000 0.320000 1.560000
75% 2.800000 2.900000 0.430000 1.970000
max 3.880000 5.080000 0.660000 3.580000

color_intensity hue od280/od315_of_diluted_wines proline

count 1000.000000 1000.000000
1000.000000 1000.000000
mean 4.990600 0.971196 2.649700 758.618000
std 2.252854 0.223798 0.695719 319.300264
min 1.280000 0.480000 1.270000 278.000000
25% 3.210000 0.810000 2.050000 502.000000
50% 4.600000 0.995000 2.820000 680.000000
75% 6.130000 1.120000 3.190000 1020.000000
max 13.000000 1.710000 4.000000 1680.000000

After Standard Scaling (describe):

alcohol malic_acid ash alcalinity_of_ash \
count 1.000000e+03 1.000000e+03 1.000000e+03

1.000000e+03 mean -6.121326e-15 -6.146195e-16 -

7.187140e-15 3.307576e-15
std 1.000500e+00 1.000500e+00 1.000500e+00

1.000500e+00 min -2.438701e+00 -1.417315e+00 -

3.645411e+00 -2.626169e+00 25% -8.030288e-01 -
6.406563e-01 -5.572188e-01 -7.061478e-01
50% 4.555285e-02 -4.338536e-01 -1.224377e-02 -1.061412e-01
75% 8.387922e-01 6.300313e-01 7.143896e-01
6.438671e-
01 max 2.234647e+00 3.233447e+00 3.148611e+00

3.193895e+00

magnesium total_phenols flavanoids

nonflavanoid_phenols \ count 1.000000e+03 1.000000e+03
1.000000e+03

1.000000e+03
mean 1.971756e-16 -2.106759e-15 - 3.192113e-15
2.330580e-15
std 1.000500e+00 1.000500e+00 1.000500e+0
1.000500e+00 0
min -2.077327e+00 -2.157791e+00 - -1.847026e+00
1.774678e+00
25% -8.278069e-01 -8.558449e-01 - -7.900957e-01
8.246308e-01
50% -1.336292e-01 1.407066e-01 -3.022819e-01
1.101342e-01
75% 5.605485e-01 7.675697e-01 5.920433e-01
8.334945e-01
max 4.309108e+00 2.503498e+00 2.461996e+0
3.054517e+00 0

proanthocyanins color_intensity hue \

count 1.000000e+03 1.000000e+03
1.000000e+03 mean
1.170619e-15 -1.278977e-15 -
3.218759e-15 std 1.000500e+00

79
Activity Based Learning - 1SP22IC02
BIC703
1.000500e+00 1.000500e+00 min -
6
2.110950e+00 -1.647890e+00 -
2.195919e+00 25% -6.321388e-01
-7.907706e-01 -7.206358e-01
50% -8.638720e-02 -1.734668e-01 1.064171e-01
75% 6.354133e-01 5.060115e-01
6.652366e-01 max 3.469801e+00
3.557002e+00 3.302865e+00

od280/od315_of_diluted_wines
proline count
1.000000e+03
1.000000e+03 mean
-2.184919e-15 -1.669775e-
16 std 1.000500e+00
1.000500e+00
min -1.984122e+00 -1.505976e+00
25% -8.624178e-01 -8.040908e-01
50% 2.449054e-01 -2.463429e-01
75% 7.769957e-01 8.190184e-01
max 1.941842e+00 2.887073e+00

After MinMax Scaling (describe):

alcohol malic_acid ash alcalinity_of_ash magnesium \
count 1000.000000 1000.000000 1000.000000 1000.000000 1000.000000

80
Activity Based Learning - 1SP22IC02
BIC703 6

mean 0.521832 0.304749 0.451227

0.536561 0.325272
std 0.214086 0.215126 0.171905
0.147262 0.156660
min 0.000000 0.000000 0.000000
0.000000 0.000000
25% 0.350000 0.166996 0.329897
0.454545 0.195652
50% 0.531579 0.211462 0.432990
0.534759 0.304348
75% 0.701316 0.440217 0.561856
0.641711 0.413043
max 1.000000 1.000000 1.000000
1.000000 1.000000

total_phenols flavanoids nonflavanoid_phenols

proanthocyanins \ count 1000.000000 1000.000000
1000.000000
1000.000000
mean 0.462917 0.367489 0.428642 0.378256
std 0.214640 0.207177 0.232187 0.179277
min 0.000000 0.000000 0.000000

0.000000
25% 0.279310 0.245283 0.26498
0.196730 4
50% 0.493103 0.358491 0.36277
0.390295 6
75% 0.627586 0.566038 0.49211
0.540084 4
max 1.000000 1.000000 1.000000
1.000000

color_intensity hue od280/od315_of_diluted_wines

proline count 1000.000000 1000.000000 1000.000000
1000.000000
mean 0.316604 0.399346 0.505385 0.342809
std 0.192223 0.181949 0.254842
0.227746
min 0.000000 0.000000 0.000000
0.000000
25% 0.164676 0.268293 0.285714 0.159772
50% 0.283276 0.418699 0.567766 0.286733
75% 0.413823 0.520325 0.703297 0.529244
max 1.000000 1.000000 1.000000
1.000000

Q6. Train-Test Split Split datasets into training and testing sets (e.g., 80/20). Print the shapes of all
datasets. Explain the importance of splitting data.

import pandas as pd
from [Link] import load_wine
from sklearn.model_selection import
train_test_split cls = load_wine()
df = [Link]([Link],
columns=cls.feature_names) df['target'] =
[Link]
X = [Link]('target',
axis=1) y = df['target']
x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42) print("X_train shape:", x_train.shape)
print("X_test shape:",
x_test.shape) print("y_train
shape:", y_train.shape)
print("y_test shape:",
y_test.shape)

output:

X_train shape: (142, 13)

X_test shape: (36, 13)
y_train shape: (142,)
y_test shape: (36,)

Supervised Learning Models

81
Activity Based Learning - 1SP22IC02
BIC703 6
Q7. Simple Linear Regression Train a simple linear regression model on one feature from the regression
dataset. Display coefficient and intercept. Evaluate using R² and RMSE. Visualize predicted vs. actual
values.

X, y = make_regression(n_samples=1000, n_features=3, noise=15,

random_state=42) X_single = X[:, 0].reshape(-1, 1)
model = LinearRegression()
[Link](X_single, y)
y_pred = [Link](X_single)
print("Coefficient:",
model.coef_[0]) print("Intercept:",
model.intercept_) print("R² Score:",
r2_score(y, y_pred))
print("RMSE:", mean_squared_error(y, y_pred,
squared=False)) [Link](y, y_pred , color='orange')

82
Activity Based Learning - 1SP22IC02
BIC703 6

[Link]("Actual Values")
[Link]("Predicted Values")
[Link]("Predicted vs Actual Values (Simple Linear
Regression)") [Link]()

Output:

Coefficient: 95.54565905592783
Intercept :
5.705090115082966 R²
Score :
0.5603587090211857 RMSE
:
85.31317062878274

Q8. Multiple Linear Regression Train multiple linear regression using all features. Evaluate performance
on training and testing data. Compare results with simple linear regression.

X, y = make_regression(n_samples=1000, n_features=3, noise=15,

random_state=42) X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.2, random_state=42) model_multi = LinearRegression()
model_multi.fit(X_train, y_train)
y_train_pred =
model_multi.predict(X_train)
y_test_pred =
model_multi.predict(X_test) train_r2
= r2_score(y_train, y_train_pred)
test_r2 = r2_score(y_test,
y_test_pred)
train_rmse = mean_squared_error(y_train, y_train_pred,
squared=False) test_rmse = mean_squared_error(y_test,
y_test_pred, squared=False) print("Multiple Linear Regression
Results")
print(" ")
print("Coefficients:",
model_multi.coef_)
print("Intercept:",
model_multi.intercept_) print("Train
R² Score:", train_r2)
print("Test R² Score:",
test_r2) print("Train RMSE:",
train_rmse) print("Test
RMSE:", test_rmse)
[Link](y_train, y_train_pred ,
color='yellow') [Link]("Actual Train
Values") [Link]("Predicted Train
Values")
[Link]("Train: Predicted vs Actual (Multiple Linear
Regression)") [Link]()
[Link](y_test,
y_test_pred,color='g')
[Link]("Actual Test Values")
[Link]("Predicted Test
Values")
[Link]("Test: Predicted vs Actual (Multiple Linear
Regression)") [Link]()

Output:

Multiple Linear Regression Results

Coefficients: [98.45724734 83.45742178 25.77958921]

Intercept: -
0.6690597501260993 Train R²
Score: 0.9868153731007379
Test R² Score:
0.9859097017495131 Train
RMSE: 14.753836606716352
Test RMSE: 15.216739718309077

83
Activity Based Learning - 1SP22IC02
BIC703 6

QG. Logistic Regression (Classification) Train a logistic regression classifier. Evaluate using accuracy,
confusion matrix, and classification report. Plot ROC curve and calculate AUC.

84
Activity Based Learning - 1SP22IC02
BIC703 6

X, y = make_classification(n_samples=1000, n_features=5, n_informative=3,

random_state=42) X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42)
model =
LogisticRegression()
[Link](X_train, y_train)
y_pred =
[Link](X_test)
y_pred_proba = model.predict_proba(X_test)[:, 1]
print(accuracy_score(y_test, y_pred))
print(confusion_matrix(y_test, y_pred))
print(classification_report(y_test, y_pred))
fpr, tpr, _ = roc_curve(y_test,
y_pred_proba) roc_auc = auc(fpr, tpr)
[Link](fpr, tpr , color='r')
[Link]("False Positive
Rate") [Link]("True
Positive Rate")
[Link]("ROC Curve")
[Link]()
print(roc_auc)

Output:

0.925
[[88 6]
[ 9 97]]
precision recall f1-score support

0 0.91 0.94 0.92 94

1 0.94 0.92 0.93 106

accuracy 0.93 200

macro avg 0.92 0.93 0.92 200
weighted avg 0.93 0.93 0.93 200

0.9774187073464472

Q10. K-Nearest Neighbors (KNN) Train KNN classifier/regressor. Experiment with different values of k
and distance metrics. Evaluate using accuracy (classification) or RMSE (regression). Discuss effect of
k on model performance.

X, y = make_classification(n_samples=800, n_features=5, n_informative=3,

random_state=42) X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.2, random_state=42) k_values = range(1, 21)
accuracies_euclidean =
[]
accuracies_manhattan
= [] for k in k_values:
model_eu = KNeighborsClassifier(n_neighbors=k, metric='euclidean')
model_eu.fit(X_train, y_train)
y_pred_eu = model_eu.predict(X_test)
accuracies_euclidean.append(accuracy_score(y_test, y_pred_eu))
model_man = KNeighborsClassifier(n_neighbors=k,
metric='manhattan') model_man.fit(X_train, y_train)
y_pred_man = model_man.predict(X_test)
accuracies_manhattan.append(accuracy_score(y_test, y_pred_man))
[Link](k_values, accuracies_euclidean, label="Euclidean")
[Link](k_values, accuracies_manhattan,
label="Manhattan") [Link]("K Value")
[Link]("Accuracy")
[Link]("KNN Performance
vs K") [Link]()
[Link]()

Output:

85
Activity Based Learning - 1SP22IC02
BIC703 6

Q11. Decision Tree Models Train Decision Tree classifier/regressor. Evaluate performance on training
and test sets. Visualize tree and feature importance. Discuss overfitting and how to prevent it.

X, y = make_classification(n_samples=800, n_features=5, n_informative=3,

random_state=42) X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42)
model = DecisionTreeClassifier(random_state=42)
[Link](X_train, y_train)
y_train_pred =
[Link](X_train) y_test_pred
= [Link](X_test)
print("Training Accuracy:", accuracy_score(y_train,
y_train_pred)) print("Testing Accuracy:",
accuracy_score(y_test, y_test_pred)) print("Feature
Importances:", model.feature_importances_)
[Link](figsize=(10, 6))
plot_tree(model, filled=True, feature_names=[f"F{i}" for i in
range(5)]) [Link]()

output:

Training Accuracy: 1.0

Testing Accuracy :
0.9375
Feature Importances : [0.00087201 0.79175171 0.00829934 0.10093151 0.09814543]

86
Activity Based Learning - 1SP22IC02
BIC703 6

Q12. Random Forest Models Train a Random Forest classifier/regressor. Evaluate performance and
compare with Decision Tree. Plot feature importances. Explain why ensemble models often perform
better than a single tree.

X, y = make_classification(n_samples=800, n_features=5, n_informative=3,

random_state=42) X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42)
dt =
DecisionTreeClassifier(random_state=42
) [Link](X_train, y_train)
dt_pred = [Link](X_test)
dt_acc = accuracy_score(y_test, dt_pred)
rf = RandomForestClassifier(n_estimators=100,
random_state=42) [Link](X_train, y_train)
rf_pred = [Link](X_test)
rf_acc = accuracy_score(y_test, rf_pred)
print("Decision Tree Accuracy:", dt_acc)
print("Random Forest Accuracy:", rf_acc)
print("Feature Importances:",
rf.feature_importances_)
[Link](range(len(rf.feature_importances_)), rf.feature_importances_ ,
color='pink') [Link]("Feature Index")
[Link]("Importance Score")
[Link]("Random Forest Feature
Importances") [Link]()

Output:

Decision Tree Accuracy :

0.9375 Random Forest Accuracy
:
0.9625
Feature Importances : [0.05509411 0.63137227 0.08075192 0.13171779 0.10106391]

Q13. Support Vector Machine (SVM) Train SVM classifier on the classification dataset. Experiment with
different kernels (linear, rbf). Evaluate performance using accuracy and confusion matrix.

X, y = make_classification(n_samples=800, n_features=5, n_informative=3,

random_state=42) X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.2, random_state=42) linear_svm = SVC(kernel='linear',
probability=True)
linear_svm.fit(X_train, y_train)
y_pred_linear =
linear_svm.predict(X_test)
rbf_svm = SVC(kernel='rbf',
probability=True) rbf_svm.fit(X_train,
y_train)
y_pred_rbf = rbf_svm.predict(X_test)
print("Linear Kernel Accuracy:", accuracy_score(y_test,
y_pred_linear)) print("Linear Confusion Matrix:\n",
confusion_matrix(y_test, y_pred_linear)) print("\nRBF Kernel
Accuracy:", accuracy_score(y_test, y_pred_rbf)) print("RBF
Confusion Matrix:\n", confusion_matrix(y_test, y_pred_rbf))

87
Activity Based Learning - 1SP22IC02
BIC703
Output:
6

Linear Kernel Accuracy : 0.95

Linear Confusion Matrix :[[72 4] , [ 4
80]] RBF Kernel Accuracy : 0.95625
RBF Confusion Matrix : [[73 3] , [ 4 80]]

88
Activity Based Learning - 1SP22IC02
BIC703 6

Q14. Model Evaluation s Cross-Validation Perform k-fold cross-validation (e.g., k=5) for at least two
supervised models. Compare cross-validation scores with single train-test split results. Discuss model
stability and generalization.

X, y = make_classification(n_samples=800, n_features=5, n_informative=3,

random_state=42) X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.2, random_state=42) log_model = LogisticRegression()
rf_model =
RandomForestClassifier(random_state=42)
log_model.fit(X_train, y_train)
rf_model.fit(X_train, y_train)
log_pred =
log_model.predict(X_test)
rf_pred =
rf_model.predict(X_test)
log_acc = accuracy_score(y_test,
log_pred) rf_acc =
accuracy_score(y_test, rf_pred)
log_cv = cross_val_score(LogisticRegression(), X, y, cv=5)
rf_cv = cross_val_score(RandomForestClassifier(random_state=42), X,
y, cv=5) models = ["Logistic Regression", "Random Forest"]
train_test_scores = [log_acc, rf_acc]
cv_scores = [[Link](log_cv),
[Link](rf_cv)] [Link](models,
train_test_scores , color='r')
[Link]("Models")
[Link]("Accuracy")
[Link]("Train-Test Accuracy
Comparison") [Link]()
[Link](models, cv_scores)
[Link]("Models")
[Link]("CV Accuracy")
[Link]("Cross-Validation Accuracy Comparison")
[Link]()

output:

Unsupervised Learning Models

Q15. K-Means Clustering Apply K-Means clustering on an unlabeled dataset. Experiment with different
values of k. Visualize clusters using 2D scatter plot (use PCA if features >2). Evaluate using silhouette
score.

X, _ = make_blobs(n_samples=1000, n_features=4, centers=3,

random_state=42) results = {}
for k in range(2, 6):
kmeans = KMeans(n_clusters=k, random_state=42, n_init=5,
max_iter=100) labels = kmeans.fit_predict(X)
score = silhouette_score(X,
labels) results[k] = score
print("Silhouette scores for different k
values:") for k, score in [Link]():
print(f"k = {k} → {score}")

89
Activity Based Learning - 1SP22IC02
BIC703
best_k = max(results, key=[Link])
6
print(f"\nBest number of clusters: k =
{best_k}")

90
Activity Based Learning - 1SP22IC02
BIC703 6

kmeans = KMeans(n_clusters=best_k, random_state=42, n_init=5,

max_iter=100) labels = kmeans.fit_predict(X)
pca =
PCA(n_components=2)
X2D =
pca.fit_transform(X)
[Link](figsize=(8,6))
[Link](X2D[:, 0], X2D[:, 1],
c=labels) [Link](f"K-Means
Clusters (k={best_k})")
[Link]("PCA 1")
[Link]("PCA 2")
[Link]()

output:

Silhouette scores for different k values:

k = 2 → 0.6641531053517515
k = 3 → 0.8265171620531689
k = 4 → 0.6177704123086705
k = 5 → 0.40374810200181815

Best number of clusters: k = 3

Q16. Hierarchical Clustering Perform hierarchical clustering on a dataset. Plot dendrogram. Discuss the
choice of number of clusters.

X, _ = make_blobs(n_samples=80, n_features=4, centers=3,

random_state=42) linked = linkage(X, method='ward')
[Link](figsize=(10,
6))
dendrogram(linked)
[Link]("Hierarchical Clustering
Dendrogram") [Link]("Samples")
[Link]("Distance")
[Link]()
labels = fcluster(linked, t=150,
criterion='distance') pca =
PCA(n_components=2)
X2D = pca.fit_transform(X)
[Link](figsize=(8,6))
[Link](X2D[:, 0], X2D[:, 1],
c=labels)
[Link]("Hierarchical Clusters (PCA
Visualization)") [Link]("PCA 1")
[Link]("PCA
2") [Link]()
output:

91
Activity Based Learning - 1SP22IC02
BIC703 6

Q17. DBSCAN Clustering ∙ Apply DBSCAN clustering. ∙ Visualize resulting clusters. ∙ Compare DBSCAN with
K-Means on the same dataset.

X, _ = make_moons(n_samples=600, noise=0.08,
random_state=42) scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
dbscan = DBSCAN(eps=0.25,
min_samples=5) db_labels =
dbscan.fit_predict(X_scaled) [Link]()
[Link](X_scaled[:,0], X_scaled[:,1],
c=db_labels) [Link]("DBSCAN Clusters")
[Link]("Feature 1")
[Link]("Feature 2")
[Link]()
kmeans = KMeans(n_clusters=2, random_state=42,
n_init=5) km_labels = kmeans.fit_predict(X_scaled)
[Link]()
[Link](X_scaled[:,0], X_scaled[:,1],
c=km_labels) [Link]("K-Means Clusters")
[Link]("Feature 1")
[Link]("Feature 2")
[Link]()

output:

Q18. Principal Component Analysis (PCA) Apply PCA to reduce high-dimensional dataset to 2 or 3
dimensions. Visualize the reduced dataset using scatter plot. Discuss how PCA helps in dimensionality
reduction and visualization.

data = load_wine()
X=
[Link] y
=
[Link]
scaler = StandardScaler()
X_scaled =
scaler.fit_transform(X) pca =
PCA(n_components=2)
X_pca =
pca.fit_transform(X_scaled)
[Link](figsize=(8,6))
scatter = [Link](X_pca[:,0],
X_pca[:,1], c=y) [Link]("PCA (2D)
Projection") [Link]("PC1")
[Link]("PC2")
[Link](*scatter.legend_elements(),
title="Classes") [Link]()

Output:

92
Activity Based Learning - 1SP22IC02
BIC703 6

Q1G. t-SNE for Visualization Apply t-SNE for high-dimensional data visualization. Visualize in 2D or 3D
plot. Compare PCA and t-SNE visualization effectiveness.

data = load_wine()
X=
[Link] y
=
[Link]
scaler = StandardScaler()
X_scaled =
scaler.fit_transform(X) pca =
PCA(n_components=2)
X_pca = pca.fit_transform(X_scaled)
[Link](figsize=(7,5))
scatter = [Link](X_pca[:,0], X_pca[:,1],
c=y) [Link]("PCA (2D) Visualization")
[Link]("PC1")
[Link]("PC2")
[Link](*scatter.legend_elements(),
title="Classes") [Link]()
tsne = TSNE(n_components=2, perplexity=30, learning_rate=200,
random_state=42) X_tsne = tsne.fit_transform(X_scaled)
[Link](figsize=(7,5))
scatter = [Link](X_tsne[:,0],
X_tsne[:,1], c=y) [Link]("t-SNE (2D)
Visualization") [Link]("Dimension 1")
[Link]("Dimension 2")
[Link](*scatter.legend_elements(),
title="Classes") [Link]()

Output:

93
Activity Based Learning - 1SP22IC02
BIC703 6

Q20. Reflection and Comparison

Compare all supervised models in terms of accuracy, interpretability, and efficiency.
Compare unsupervised models in terms of clustering performance and
visualization. Discuss which algorithm would be suitable for real-world
applications and why.

1. Comparison of Supervised Models

Accuracy
● Random Forest and SVM generally provide high accuracy on complex datasets.
● Polynomial Regression can increase accuracy on non-linear data.
● Linear Regression performs well only if relationships are linear.
● Decision Trees are accurate but can overfit if not controlled.
● KNN can be accurate but suffers on large
datasets. Interpretability
● Linear Regression is highly interpretable (clear feature coefficients).
● Decision Trees are easy to understand through tree diagrams.
● Ridge Regression is interpretable but adds regularization.
● Random Forest and SVM have low interpretability (black-box models).
● Polynomial Regression becomes hard to interpret at higher
degrees. Efficiency
● Linear & Ridge Regression are very fast to train.
● Decision Trees are efficient but can grow large.
● Random Forest is slower due to many trees.
● SVM can be slow on large datasets.
● KNN is fast to train but slow to predict.

2. Comparison of Unsupervised Models

Clustering Performance
● K-Means
o Works well on separated spherical clusters.
o Struggles with noise and irregular shapes.
● Hierarchical Clustering
o Reveals cluster structure through dendrograms.
o Very slow for large datasets.
● DBSCAN
o Detects arbitrary-shaped clusters.
o Can identify noise/outliers.
o Sensitive to parameter tuning.
Visualization
● Hierarchical gives dendrogram for cluster formation.
● K-Means gives clear scatter plots when clusters are distinct.
● DBSCAN highlights noise clearly via different labels.

3. PCA vs t-SNE Visualization

PCA
● Reduces dimensionality by preserving maximum variance.
● Fast and computationally efficient.
● Good for linear relationships.
● Helps simplify noisy datasets.
t-SNE
● Excels at revealing complex, non-linear cluster structures.
● Produces well-separated visual clusters.
● More computationally expensive.
● Used mainly for visualization, not for full modeling.

4. Suitable Algorithms for Real-World Applications

● Random Forest → Fraud detection, medical diagnosis (high accuracy & robustness).
● SVM → Spam detection, sentiment classification (works well with high-dimensional data).
● Linear/Ridge Regression → Sales prediction, housing prices (interpretability matters).
● Decision Tree → Loan approvals (easy explanation to stakeholders).
● K-Means → Customer segmentation, market analysis.
● DBSCAN → Anomaly detection, noise filtering (arbitrary cluster shapes).

5. Real-World Recommendation Logic

94
Activity Based Learning - 1SP22IC02
BIC703 6

Requirement Recommended Algorithm

Highest prediction accuracy Random Forest / SVM
Easy explanation Linear Regression / Decision
Tree Discover hidden groups K-Means
Detect anomalies/noise DBSCAN
Reduce dimensionality PCA
Visualize complex data t-SNE

6. Final Conclusion
● No single model is best for every task.
● Supervised models excel when labeled data is available.
● Unsupervised models reveal hidden patterns where labels are unknown.
● PCA and t-SNE are essential when visualizing high-dimensional data.
● Real-world choice depends on:
o dataset size
o data type
o interpretability needs
o computational resources

95