MACHINE LEARNING

LAB
BCSL606
Program 1

Develop a program to create histograms for all numerical features and

analyze the distribution of each feature. Generate box plots for all
numerical features and identify any outliers. Use California Housing
dataset.

14-05-2026 Sandeep Kini 2

# Import required libraries
import pandas as pd
import [Link] as plt
import seaborn as sns

pd.set_option('display.max_columns', None)
pd.set_option('[Link]', None)
df = pd.read_csv("housing [Link]")

14-05-2026 Sandeep Kini 3

# Display basic information
print("Dataset Loaded Successfully!")
print([Link]())

# Select only numerical columns

num_cols = df.select_dtypes(include=['int64', 'float64']).columns
print("\nNumerical Features:")
print(list(num_cols))

14-05-2026 Sandeep Kini 4

# -----------------------------
# HISTOGRAMS FOR NUMERICAL FEATURES
# -----------------------------
print("\nGenerating Histograms...")

df[num_cols].hist(bins=30, figsize=(15, 10), color='skyblue', edgecolor='black')

[Link]("Histograms of Numerical Features", fontsize=16)
plt.tight_layout()
[Link]()

14-05-2026 Sandeep Kini 5

# -----------------------------
# BOXPLOTS FOR NUMERICAL FEATURES
# -----------------------------
print("\nGenerating Boxplots...")

[Link](figsize=(15, 10))
for i, column in enumerate(num_cols):
[Link](len(num_cols) // 3 + 1, 3, i + 1)
[Link](x=df[column], color='lightgreen')
[Link](f"Boxplot of {column}")

plt.tight_layout()
[Link]()
14-05-2026 Sandeep Kini 6
# -----------------------------
# SUMMARY STATISTICS
# -----------------------------
print("\nSummary Statistics:")
print(df[num_cols].describe())

14-05-2026 Sandeep Kini 7

14-05-2026 Sandeep Kini 8
14-05-2026 Sandeep Kini 9
14-05-2026 Sandeep Kini 10
14-05-2026 Sandeep Kini 11
14-05-2026 Sandeep Kini 12
Heatmap

• Strong positive relationships

• Strong negative relationships

• Weak relationships

5/14/2026 Sandeep Kini 13

Pair Plot

• Linear relationships

• Clusters

• Outliers

• Distribution patterns

• Positive/negative trends
5/14/2026 Sandeep Kini 14
✔ Helps select important features

✔ Understand feature dependency

✔ Improve model accuracy

✔ Remove redundant features

5/14/2026 Sandeep Kini 15

5/14/2026 Sandeep Kini 16
5/14/2026 Sandeep Kini 17
5/14/2026 Sandeep Kini 18
5/14/2026 Sandeep Kini 19
5/14/2026 Sandeep Kini 20
Correlation is a standardized measure of the strength and direction of the linear
relationship between two variables. It is derived from covariance and ranges between
-1 and 1. Unlike covariance, which only indicates the direction of the relationship,
correlation provides a standardized measure.

5/14/2026 Sandeep Kini 21

5/14/2026 Sandeep Kini 22
5/14/2026 Sandeep Kini 23
Program 2

Develop a program to compute the correlation matrix to understand

the relationships between pairs of features. Visualize the correlation
matrix using a heatmap to know which variables have strong
positive/negative correlations. Create a pair plot to visualize pairwise
relationships between features. Use the California Housing dataset.

14-05-2026 Sandeep Kini 24

# Import necessary libraries
import pandas as pd
import seaborn as sns
import [Link] as plt

pd.set_option('display.max_columns', None)
pd.set_option('[Link]', None)
df = pd.read_csv("housing [Link]")

print("Dataset Loaded Successfully!")

print([Link]())

14-05-2026 Sandeep Kini 25

# ---------------------------------------
# SELECT ONLY NUMERICAL COLUMNS
# ---------------------------------------
num_cols = df.select_dtypes(include=['int64', 'float64']).columns
print("\nNumerical Columns:")
print(list(num_cols))

# ---------------------------------------
# COMPUTE THE CORRELATION MATRIX
# ---------------------------------------
corr_matrix = df[num_cols].corr()

print("\nCorrelation Matrix:")
print(corr_matrix)
14-05-2026 Sandeep Kini 26
# ---------------------------------------
# VISUALIZE CORRELATION MATRIX (HEATMAP)
# ---------------------------------------
[Link](figsize=(12, 8))
[Link](corr_matrix, annot=True, cmap="coolwarm", fmt=".2f",
linewidths=0.5)
[Link]("Correlation Heatmap of Numerical Features")
[Link]()

14-05-2026 Sandeep Kini 27

# ---------------------------------------
# PAIR PLOT VISUALIZATION
# ---------------------------------------
print("\nGenerating Pair Plot... (This may take some time for large datasets)")
[Link](df[num_cols], diag_kind="kde")
[Link]("Pair Plot of Numerical Features", y=1.02)
[Link]()

14-05-2026 Sandeep Kini 28

14-05-2026 Sandeep Kini 29
14-05-2026 Sandeep Kini 30
14-05-2026 Sandeep Kini 31
14-05-2026 Sandeep Kini 32
Program 3

Develop a program to implement Principal Component

Analysis (PCA) for reducing the dimensionality of the

Iris dataset from 4 features to 2.

14-05-2026 Sandeep Kini 33

5/14/2026 Sandeep Kini 34
5/14/2026 Sandeep Kini 35
5/14/2026 Sandeep Kini 36
# Import necessary libraries
import pandas as pd
from [Link] import StandardScaler
from [Link] import PCA
import [Link] as plt
import seaborn as sns

pd.set_option('display.max_columns', None)
pd.set_option('[Link]', None)
df = pd.read_csv("[Link]")

print("Dataset Loaded Successfully!")

print([Link]())
14-05-2026 Sandeep Kini 37
# ---------------------------------------
# SELECT FEATURES
# ---------------------------------------
features = df.select_dtypes(include=['float64', 'int64']).columns
X = df[features]

# Standardize the features

scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

14-05-2026 Sandeep Kini 38

# ---------------------------------------
# APPLY PCA (Reduce 4 → 2 components)
# ---------------------------------------
pca = PCA(n_components=2)
principal_components = pca.fit_transform(X_scaled)

# Create DataFrame for PCA output

pca_df = [Link](data=principal_components, columns=['PC1', 'PC2'])

# If the dataset contains species, keep it for visualization

if 'species' in [Link]:
pca_df['species'] = df['species']

print("\nPCA Transformation Completed!")

print(pca_df.head())
14-05-2026 Sandeep Kini 39
# ---------------------------------------
# APPLY PCA (Reduce 4 → 2 components)
# ---------------------------------------
pca = PCA(n_components=2)
principal_components = pca.fit_transform(X_scaled)

# Create DataFrame for PCA output

pca_df = [Link](data=principal_components, columns=['PC1', 'PC2'])

# If the dataset contains species, keep it for visualization

if 'species' in [Link]:
pca_df['species'] = df['species']

print("\nPCA Transformation Completed!")

print(pca_df.head())
14-05-2026 Sandeep Kini 40
# ---------------------------------------
# Explained Variance
# ---------------------------------------
print("\nExplained Variance Ratio:")
print(pca.explained_variance_ratio_)

14-05-2026 Sandeep Kini 41

Sandeep Kini 5/14/2026 42
5/14/2026 Sandeep Kini 43
Sandeep Kini 5/14/2026 44
5/14/2026 Sandeep Kini 45
14-05-2026 Sandeep Kini 46
Program 4

For a given set of training data examples stored in a .CSV file,

implement and demonstrate the Find-S algorithm to output a

description of the set of all hypotheses consistent with the training

examples.

14-05-2026 Sandeep Kini 47

import pandas as pd

df = pd.read_csv("[Link]")

print("Dataset Loaded Successfully!")

print(df)

# Attribute columns (all except last)

attributes = [Link][:-1]
target = [Link][-1]

14-05-2026 Sandeep Kini 48

# -------------------------------
# FIND-S ALGORITHM
# -------------------------------

# Step 1: Start with the first positive example

hypothesis = None

for i in range(len(df)):
if df[target][i].lower() == "yes": # find first positive example
hypothesis = [Link][i, :-1].tolist()
break

14-05-2026 Sandeep Kini 49

# Step 2: Generalize for each positive example

for i in range(len(df)):

if df[target][i].lower() == "yes":

instance = [Link][i, :-1].tolist()

for j in range(len(hypothesis)):

if hypothesis[j] != instance[j]:

hypothesis[j] = '?' # generalization

14-05-2026 Sandeep Kini 50

# -------------------------------

# OUTPUT FINAL HYPOTHESIS

# -------------------------------

print("\nFinal Hypothesis (Find-S Result):")

print(hypothesis)

14-05-2026 Sandeep Kini 51

Output:

14-05-2026 Sandeep Kini 52

Program 5
5. Develop a program to implement k-Nearest Neighbour algorithm to
classify the randomly generated 100 values of x in the range of [0,1].
Perform the following based on dataset generated.

a. Label the first 50 points {x1,……,x50} as follows: if (xi ≤ 0.5),

then xi ε Class1, else xi ε Class2

b. Classify the remaining points, x51,……,x100 }using KNN.

Perform this for k=1,2,3,4,5,20,30

14-05-2026 Sandeep Kini 53
KNN algorithm

K-Nearest Neighbors (KNN) algorithm is a simple, supervised machine learning

method used for both classification and regression tasks.

• majority class (for classification)

• average value (for regression)

• lazy learner algorithm

14-05-2026 Sandeep Kini 54

• proximity and majority voting to make predictions.

• K -> nearby points or neighbors

• Noise or outliers- larger k can make the predictions more stable

• Underfitting- k is too large the model may become too simple and miss

important patterns. (Underfitting = High Bias + Low Variance)

• Overfitting- k is too small. (Overfitting = Low Bias + High Variance)

• Distance
14-05-2026
Metrics: Euclidean or Manhattan
Sandeep Kini
Distance 55
Applications

• Recommendation Systems

• Spam Detection

• Customer Segmentation

• Speech Recognition

14-05-2026 Sandeep Kini 56

5/14/2026 Sandeep Kini 57
5/14/2026 Sandeep Kini 58
5/14/2026 Sandeep Kini 59
14-05-2026 Sandeep Kini 60
5/14/2026 Sandeep Kini 61
import numpy as np
from [Link] import KNeighborsClassifier

# Generate 100 random values in the range [0, 1]

data = [Link](100)

# Label the first 50 points

labels = [Link](100)
labels[:50] = [Link](data[:50] <= 0.5, 1, 2)

14-05-2026 Sandeep Kini 62

# Separate data into training and testing sets

train_data = data[:50].reshape(-1, 1)

train_labels = labels[:50]

test_data = data[50:].reshape(-1, 1)

# Perform KNN classification for different values of k

k_values = [1, 2, 3, 4, 5, 20, 30]

14-05-2026 Sandeep Kini 63

for k in k_values:
knn = KNeighborsClassifier(n_neighbors=k)
[Link](train_data, train_labels)
predicted_labels = [Link](test_data)
print(f"K = {k}")
print("Test Value\tPredicted Label")

for val, label in zip(test_data.flatten(), predicted_labels):

print(f"{val:.3f}\t\t{int(label)}")

print()

14-05-2026 Sandeep Kini 64

Output:
K = 1, K = 2, K = 3
Test Value K1 Pred K1 Test Value K2 Pred K2 Test Value K3 Pred K3

0.908 2 0.908 2 0.908 2

0.240 1 0.240 1 0.240 1
0.145 1 0.145 1 0.145 1
0.489 1 0.489 1 0.489 2
0.986 2 0.986 2 0.986 2
0.242 1 0.242 1 0.242 1
0.672 2 0.672 2 0.672 2
0.762 2 0.762 2 0.762 2
14-05-2026 Sandeep Kini 65
 0.238 1 0.238 1 0.238 1
0.728 2 0.728 2 0.728 2
0.368 1 0.368 1 0.368 1
0.632 2 0.632 2 0.632 2
0.634 2 0.634 2 0.634 2
0.536 2 0.536 2 0.536 2
0.090 1 0.090 1 0.090 1
0.835 2 0.835 2 0.835 2
0.321 1 0.321 1 0.321 1
0.187 1 0.187 1 0.187 1
0.041 1 0.041 1 0.041 1
5/14/2026 Sandeep Kini 66
 0.591 2 0.591 2 0.591 2
0.678 2 0.678 2 0.678 2
0.017 1 0.017 1 0.017 1
0.512 2 0.512 2 0.512 2
0.226 1 0.226 1 0.226 1
0.645 2 0.645 2 0.645 2
0.174 1 0.174 1 0.174 1

5/14/2026 Sandeep Kini 67

0.691 2 0.691 2 0.691 2
0.387 1 0.387 1 0.387 1
0.937 2 0.937 2 0.937 2
0.138 1 0.138 1 0.138 1
0.341 1 0.341 1 0.341 1
0.113 1 0.113 1 0.113 1
0.925 2 0.925 2 0.925 2
0.877 2 0.877 2 0.877 2
0.258 1 0.258 1 0.258 1
0.660 2 0.660 2 0.660 2
0.817 2 0.817 2 0.817 2
0.555 2 0.555 2 0.555 2
0.530 2 0.530 2 0.530 2

5/14/2026 Sandeep Kini 68

0.242 1 0.242 1 0.242 1
0.093 1 0.093 1 0.093 1
0.897 2 0.897 2 0.897 2
0.900 2 0.900 2 0.900 2
0.633 2 0.633 2 0.633 2
0.339 1 0.339 1 0.339 1
0.349 1 0.349 1 0.349 1
0.726 2 0.726 2 0.726 2
0.897 2 0.897 2 0.897 2
0.887 2 0.887 2 0.887 2
0.780 2 0.780 2 0.780 2
5/14/2026 Sandeep Kini 69
K = 4, K = 5, K = 20
Test Value K4 Pred K4 Test Value K5 Pred K5 Test Value Pred K20
K20

0.908 2 0.908 2 0.908 2

0.240 1 0.240 1 0.240 1
0.145 1 0.145 1 0.145 1
0.489 2 0.489 2 0.489 1
0.986 2 0.986 2 0.986 2
0.242 1 0.242 1 0.242 1
0.672 2 0.672 2 0.672 2
0.762 2 0.762 2 0.762 2
5/14/2026 Sandeep Kini 70
0.238 1 0.238 1 0.238 1
0.728 2 0.728 2 0.728 2
0.368 1 0.368 1 0.368 1
0.632 2 0.632 2 0.632 2
0.634 2 0.634 2 0.634 2
0.536 2 0.536 2 0.536 1
0.090 1 0.090 1 0.090 1
0.835 2 0.835 2 0.835 2
0.321 1 0.321 1 0.321 1
0.187 1 0.187 1 0.187 1
5/14/2026 Sandeep Kini 71
0.041 1 0.041 1 0.041 1
0.591 2 0.591 2 0.591 2
0.678 2 0.678 2 0.678 2
0.017 1 0.017 1 0.017 1
0.512 2 0.512 2 0.512 1
0.226 1 0.226 1 0.226 1
0.645 2 0.645 2 0.645 2
0.174 1 0.174 1 0.174 1
0.691 2 0.691 2 0.691 2
0.387 1 0.387 1 0.387 1
0.937 2 0.937 2 0.937 2
5/14/2026 Sandeep Kini 72
0.138 1 0.138 1 0.138 1
0.341 1 0.341 1 0.341 1
0.113 1 0.113 1 0.113 1
0.925 2 0.925 2 0.925 2
0.877 2 0.877 2 0.877 2
0.258 1 0.258 1 0.258 1
0.660 2 0.660 2 0.660 2
0.817 2 0.817 2 0.817 2
0.555 2 0.555 2 0.555 2
0.530 2 0.530 2 0.530 1
5/14/2026 Sandeep Kini 73
0.242 1 0.242 1 0.242 1
0.093 1 0.093 1 0.093 1
0.897 2 0.897 2 0.897 2
0.900 2 0.900 2 0.900 2
0.633 2 0.633 2 0.633 2
0.339 1 0.339 1 0.339 1
0.349 1 0.349 1 0.349 1
0.726 2 0.726 2 0.726 2
0.897 2 0.897 2 0.897 2
0.887 2 0.887 2 0.887 2
0.780 2 0.780 2 0.780 2
5/14/2026 Sandeep Kini 74
K = 30
Test Value Pred K30
0.908 2
0.240 1
0.145 1
0.489 1
0.986 2
0.242 1
0.672 2
0.762 2
0.238 1
0.728 2
0.368 1
0.632 2
5/14/2026 Sandeep Kini 75
0.634 2
0.536 1
0.090 1
0.835 2
0.321 1
0.187 1
0.041 1
0.591 2
0.678 2
0.017 1
0.512 1
0.226 1
5/14/2026 Sandeep Kini 76
0.645 2
0.174 1
0.691 2
0.387 1
0.937 2
0.138 1
0.341 1
0.113 1
0.925 2
0.877 2
0.258 1
0.660 2
0.817 2
5/14/2026 Sandeep Kini 77
0.555 1
0.530 1
0.242 1
0.093 1
0.897 2
0.900 2
0.633 2
0.339 1
0.349 1
0.726 2
0.897 2
0.887 2
0.780 2
5/14/2026 Sandeep Kini 78
14-05-2026 Sandeep Kini 79
5/14/2026 Sandeep Kini 80
14-05-2026 Sandeep Kini 81
Program 6

6. Implement the non-parametric Locally Weighted Regression

algorithm to fit data points. Select an appropriate data set for

your experiment and draw graphs.

14-05-2026 Sandeep Kini 82

Regression:

• Regression in machine learning is a supervised learning technique used to

predict continuous numerical values by learning relationships between input
variables (features) and an output variable (target).

• Is widely used in forecasting, risk analysis, decision-making and trend

estimation.

14-05-2026 Sandeep Kini 83

Types of Regression:

1. Simple Linear Regression

2. Multiple Linear Regression

3. Polynomial Regression

4. Logistic Regression

14-05-2026 Sandeep Kini 84

14-05-2026 Sandeep Kini 85
Sandeep Kini 5/14/2026 86
5/14/2026 Sandeep Kini 87
14-05-2026 Sandeep Kini 88
Sandeep Kini 5/14/2026 89
5/14/2026 Sandeep Kini 90
14-05-2026 Sandeep Kini 91
5/14/2026 Sandeep Kini 92
5/14/2026 Sandeep Kini 93
5/14/2026 Sandeep Kini 94
5/14/2026 Sandeep Kini 95
Sandeep Kini 5/14/2026 96
14-05-2026 Sandeep Kini 97
14-05-2026 Sandeep Kini 98
import [Link] as plt #plotting graphs
import pandas as pd #load and handle dataset
import numpy as np #matrix operations and numerical computations

# -------------------------------
# Gaussian Kernel Function
# -------------------------------
def kernel(point, xmat, k): #Computes weights for all training points
m, n = [Link](xmat)
weights = [Link]([Link](m)) #Creates a diagonal weight matrix

14-05-2026 Sandeep Kini 99

 for j in range(m):
diff = point - X[j] #computes diff b/n current query point & sample
dist = float(diff @ diff.T) # convert 1×1 matrix → scalar
(Computes Squared Euclidean distance)
weights[j, j] = [Link](dist / (-2 * k * k)) #Gaussian weight formula, k is
bandwidth parameter (small & large distance)

return weights
# -------------------------------
# Compute Local Weights
# -------------------------------
def localWeight(point, xmat, ymat, k): # calculates theta(regression parameters) for
a given point
wei = kernel(point, xmat, k) #creates weight matrix
theta = (X.T @ (wei @ X)).I @ (X.T @ (wei @ ymat.T)) #weighted least squares equation
return theta
14-05-2026 Sandeep Kini 100
# -------------------------------
# LWR Regression
# -------------------------------
def localWeightRegression(xmat, ymat, k):
m, n = [Link](xmat)
ypred = [Link](m) #stores predicted values

for i in range(m): #for every datapoint

theta = localWeight(xmat[i], xmat, ymat, k) #fits local regression model
ypred[i] = float(xmat[i] @ theta) # convert 1×1 to scalar

return ypred
14-05-2026 Sandeep Kini 101
# -------------------------------

# Load dataset

# -------------------------------

data = pd.read_csv("[Link]")

bill = [Link](data["total_bill"])

tip = [Link](data["tip"])

14-05-2026 Sandeep Kini 102

# -------------------------------

# Prepare matrices (NumPy 2.0 compatible)

# -------------------------------

mbill = [Link](bill)

mtip = [Link](tip)

m = [Link][1]

one = [Link]([Link](m)) #creates column of ones

14-05-2026 Sandeep Kini 103
# Add bias term
X = [Link]((one.T, mbill.T))

# -------------------------------
# Run LWR
# -------------------------------
k = 0.5 #bandwidth parameter
ypred = localWeightRegression(X, mtip, k)

# -------------------------------
# Sort for smooth plot
# -------------------------------
X_array = [Link](X)
SortIndex = X_array[:, 1].argsort()

xsort = X_array[SortIndex, 1]
14-05-2026 Sandeep Kini 104
# -------------------------------
# Plot
# -------------------------------
[Link](bill, tip, color="green", label="Data Points")
[Link](xsort, ypred[SortIndex], color="red", linewidth=3, label="LWR Curve")

[Link]("Total Bill")
[Link]("Tip")
[Link]("Locally Weighted Regression (NumPy 2.0 Safe)")
[Link]()
[Link](True)
[Link]()
14-05-2026 Sandeep Kini 105
Output:

14-05-2026 Sandeep Kini 106

Program 7a

7a . Develop a program to demonstrate the working of Linear

Regression. Use Boston Housing Dataset for Linear Regression.

14-05-2026 Sandeep Kini 107

import pandas as pd
import [Link] as plt
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from [Link] import mean_squared_error #Measures prediction error

# ---------------------------------------------------
# Load your Boston Housing dataset
# ---------------------------------------------------
df = pd.read_csv("housing_boston.csv")

14-05-2026 Sandeep Kini 108

# Feature and Target

X = df[["RM"]].values # Average number of rooms

y = df["MEDV"].values # House price

# Train-test split

X_train, X_test, y_train, y_test = train_test_split(

X, y, test_size=0.2, random_state=42)

14-05-2026 Sandeep Kini 109

# Linear Regression Model

model = LinearRegression()

[Link](X_train, y_train) #trains using training data

# Prediction

y_pred = [Link](X_test)
14-05-2026 Sandeep Kini 110
# ---------------------------------------------------
# Plot Actual vs Predicted
# ---------------------------------------------------
[Link](X_test, y_test, color="blue", label="Actual Prices")
[Link](X_test, y_pred, color="red", label="Predicted Prices")
[Link]("Average Rooms per Dwelling (RM)")
[Link]("Median House Value (MEDV)")
[Link]("Linear Regression – Boston Housing Dataset")
[Link]()
[Link](True)
[Link]()

print("Linear Regression MSE:", mean_squared_error(y_test, y_pred))

14-05-2026 Sandeep Kini 111
Output:
Linear Regression MSE: 46.144775347317264

14-05-2026 Sandeep Kini 112

Program 7b

7b . Develop a program to demonstrate the working Polynomial

Regression. Use Auto MPG Dataset (for vehicle fuel efficiency

prediction) for Polynomial Regression.

14-05-2026 Sandeep Kini 113

import pandas as pd

import numpy as np

import [Link] as plt

from [Link] import PolynomialFeatures #Converts linear features into

polynomial features

from sklearn.linear_model import LinearRegression

from sklearn.model_selection import train_test_split

from [Link] import mean_squared_error #Measures model error

df = pd.read_csv("auto_mpg.csv")
14-05-2026 Sandeep Kini 114
# Clean invalid entries

df = [Link]("?", [Link])

df = [Link](subset=["horsepower", "mpg"])

df["horsepower"] = df["horsepower"].astype(float)

# Feature and Target

X = df[["horsepower"]].values

y = df["mpg"].values
14-05-2026 Sandeep Kini 115
# Polynomial Transformation (degree = 3)

poly = PolynomialFeatures(degree=3)

X_poly = poly.fit_transform(X)

# Train-Test Split

X_train, X_test, y_train, y_test = train_test_split(

X_poly, y, test_size=0.2, random_state=42)

14-05-2026 Sandeep Kini 116

# Train Polynomial Regression Model
model = LinearRegression()
[Link](X_train, y_train)

# Prediction
y_pred = [Link](X_test)

# ---------------------------------------------------
# Polynomial Curve
# ---------------------------------------------------
[Link](X, y, color="green", label="Actual Data")

14-05-2026 Sandeep Kini 117

# Smooth curve
x_line = [Link](min(X), max(X), 200).reshape(-1, 1)
x_poly_line = [Link](x_line)
y_line = [Link](x_poly_line)

[Link](x_line, y_line, color="red", linewidth=3, label="Polynomial Fit (Degree3)")

[Link]("Horsepower")
[Link]("Miles Per Gallon (MPG)")
[Link]("Polynomial Regression – Auto MPG Dataset")
[Link]()
[Link](True)
[Link]()
print("Polynomial Regression MSE:", mean_squared_error(y_test, y_pred))
14-05-2026 Sandeep Kini 118
Output:
Polynomial Regression MSE: 18.460267222145085

14-05-2026 Sandeep Kini 119

Program 8

8. Develop a program to demonstrate the working of the decision tree

algorithm. Use Breast Cancer Data set for building the decision tree and

apply this knowledge to classify a new sample.

14-05-2026 Sandeep Kini 120

import pandas as pd

import numpy as np

from sklearn.model_selection import train_test_split

from [Link] import DecisionTreeClassifier, plot_tree

from [Link] import accuracy_score

import [Link] as plt

14-05-2026 Sandeep Kini 121
# ---------------------------------------------------------
# LOAD YOUR BREAST CANCER CSV
# ---------------------------------------------------------
df = pd.read_csv("[Link]")

# Drop ID column (not useful)

df = [Link](columns=["id"])

# Encode diagnosis: M → 1 (Malignant), B → 0 (Benign)

df["diagnosis"] = df["diagnosis"].map({"M": 1, "B": 0})

14-05-2026 Sandeep Kini 122

# Separate features and label

X = [Link](columns=["diagnosis"])

y = df["diagnosis"]

# ---------------------------------------------------------

# TRAIN-TEST SPLIT

# ---------------------------------------------------------

X_train, X_test, y_train, y_test = train_test_split(

X, y, test_size=0.2, random_state=42)
14-05-2026 Sandeep Kini 123
# ---------------------------------------------------------
# DECISION TREE MODEL
# ---------------------------------------------------------
model = DecisionTreeClassifier(criterion="gini", max_depth=5,
random_state=42)
[Link](X_train, y_train)

# ---------------------------------------------------------
# PREDICTION & ACCURACY
# ---------------------------------------------------------
y_pred = [Link](X_test)
acc = accuracy_score(y_test, y_pred)

print("Decision
14-05-2026 Tree Accuracy:", acc) Sandeep Kini 124
# ---------------------------------------------------------
# CLASSIFY A NEW SAMPLE
# ---------------------------------------------------------
# Example new sample (use real values)
new_sample = [Link]([[
17.99, 10.38, 122.8, 1001, 0.1184, 0.2776, 0.3001, 0.078, 0.242, 0.07871,
1.095, 0.9053, 8.589, 153.4, 0.0064, 0.049, 0.0537, 0.03003, 0.00619,
0.00466,
25.38, 17.33, 184.6, 2019, 0.1622, 0.6656, 0.7119, 0.2654, 0.4601, 0.1189
]])

14-05-2026 Sandeep Kini 125

prediction = [Link](new_sample)

if prediction[0] == 1:

print("The new sample is classified as: MALIGNANT")

else:

print("The new sample is classified as: BENIGN")

14-05-2026 Sandeep Kini 126

# ---------------------------------------------------------

# OPTIONAL — VISUALIZE TREE

# ---------------------------------------------------------

[Link](figsize=(18, 10))

plot_tree(model, feature_names=[Link], class_names=["Benign",

"Malignant"], filled=True)

[Link]("Decision Tree for Breast Cancer Classification")

[Link]()
14-05-2026 Sandeep Kini 127
Output:

Decision Tree Accuracy: 0.9473684210526315

The new sample is classified as: MALIGNANT

14-05-2026 Sandeep Kini 128

5/14/2026 Sandeep Kini 129
Program 9

9. Develop a program to implement the Naive Bayesian classifier

considering Olivetti Face Data set for training. Compute the accuracy of

the classifier, considering a few test data sets.

14-05-2026 Sandeep Kini 130

import numpy as np
import [Link] as plt
from [Link] import fetch_olivetti_faces
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import GaussianNB
from [Link] import accuracy_score

# -----------------------------------------------------------
# Load Olivetti Face Dataset
# -----------------------------------------------------------
faces = fetch_olivetti_faces()

14-05-2026 Sandeep Kini 131

X = [Link] # Flattened pixel data (400 × 4096)
y = [Link] # Person ID (0–39)

print("Dataset Loaded.")
print("Total samples:", [Link][0])
print("Features per sample:", [Link][1])

# -----------------------------------------------------------
# Train-test split
# -----------------------------------------------------------
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42
)
14-05-2026 Sandeep Kini 132
# -----------------------------------------------------------
# Naive Bayes Classifier
# -----------------------------------------------------------
model = GaussianNB()
[Link](X_train, y_train)

# -----------------------------------------------------------
# Predictions & Accuracy
# -----------------------------------------------------------
y_pred = [Link](X_test)
accuracy = accuracy_score(y_test, y_pred)

print("\nNaive Bayes Classifier Accuracy:", round(accuracy * 100, 2), "%")

14-05-2026 Sandeep Kini 133
# -----------------------------------------------------------
# Show a few test images with predictions
# -----------------------------------------------------------
fig, ax = [Link](2, 5, figsize=(10, 5))
[Link]("Olivetti Faces - Predictions by Naive Bayes")

for i in range(10):
ax[i//5, i%5].imshow(X_test[i].reshape(64, 64), cmap='gray')
ax[i//5, i%5].set_title(f"True:{y_test[i]}\nPred:{y_pred[i]}")
ax[i//5
[Link](), i%5].axis("off")

14-05-2026 Sandeep Kini 134

Output:
Naive Bayes Classifier Accuracy: 83.75 %

14-05-2026 Sandeep Kini 135

Program 10

10. Develop a program to implement k-means clustering using the

Wisconsin Breast Cancer data set and visualize the clustering result.

14-05-2026 Sandeep Kini 136

import [Link] as plt
from [Link] import load_breast_cancer
from [Link] import StandardScaler
from [Link] import KMeans
from [Link] import PCA

# Load the dataset

data = load_breast_cancer()
X = [Link] # Features
y = [Link] # Labels (not used in clustering)

14-05-2026 Sandeep Kini 137

# Standardize the features
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

# Apply K-Means clustering

kmeans = KMeans(n_clusters=2, random_state=42, n_init=10)
[Link](X_scaled)
labels = [Link]

14-05-2026 Sandeep Kini 138

# Reduce dimensions using PCA for visualization
pca = PCA(n_components=2)
X_pca = pca.fit_transform(X_scaled)

# Plot the clusters

[Link](figsize=(8, 6))
[Link](X_pca[:, 0], X_pca[:, 1], c=labels, cmap='viridis', alpha=0.7)
[Link]('Principal Component 1')
[Link]('Principal Component 2')
[Link]('K-Means Clustering of Breast Cancer Dataset')
[Link](label='Cluster')
[Link]()
14-05-2026 Sandeep Kini 139
Output:

14-05-2026 Sandeep Kini 140