Programme Name: MCA Semester III
Course Name & Code: Data Science & MCA37114
Class: MCA2024
Academic Session: 2025-26
Study Material
Module 5: Classification and Regression
1. Introduction to Classification and Regression
1.1 What is Classification?
Classification is a type of supervised machine learning where the goal is to predict a categorical output (e.g.,
yes/no, spam/not spam, disease/no disease) based on input features.
1.2 What is Regression?
Regression is used when the output variable is continuous (e.g., predicting temperature, price, age). It
estimates the relationship between input variables and a numeric target.
1.3 Difference Between Classification and Regression
Aspect Classification Regression
Type of Output Categorical (discrete labels or classes) Continuous (real-valued numbers)
Assign input to one of the predefined Predict a numeric value based on input
Goal
categories features
Email spam detection, disease diagnosis House price prediction, temperature
Examples
(yes/no), image recognition forecasting
Evaluation Accuracy, Precision, Recall, F1 Score, AUC- Mean Squared Error (MSE), RMSE, R-
Metrics ROC squared (R²)
Fits a line or curve to best represent the
Decision Boundary Finds boundaries between classes
data trend
Output
Class labels or probabilities of each class Predicted numeric values
Interpretation
Common Logistic Regression, k-NN, Decision Tree, Linear Regression, Polynomial
Algorithms SVM Regression, Ridge, Lasso
A line/curve showing trend or
Visual Output Data points grouped into categories
relationship
Often predicts class probability (e.g., 70% class
Use of Probability Directly predicts a numerical estimate
A)
Data
Requires labelled class data Requires labelled numeric target data
Requirements
2. Logistic Regression
2.1 What is Logistic Regression?
Logistic Regression is a classification algorithm used to predict a binary outcome (two classes), such as
"yes/no" or "1/0". Despite its name, it is used for classification, not regression.
It uses the logistic (sigmoid) function to convert linear predictions into probabilities between 0 and 1.
Prepared by the faculties of CSS dept Brainware University, Kolkata
1
Programme Name: MCA Semester III
Course Name & Code: Data Science & MCA37114
Class: MCA2024
Academic Session: 2025-26
2.2 Sigmoid Function
If the output probability is greater than 0.5, we predict class 1; otherwise, class 0.
2.3 Applications
Disease diagnosis (positive/negative)
Credit approval (approve/reject)
Spam detection
2.4 Implementation of Logistic Regression in Python
Using scikit-learn, a popular Python library for machine learning:
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from [Link] import load_iris
from [Link] import accuracy_score
# Load sample data
data = load_iris()
X = [Link]
y = ([Link] == 0).astype(int) # Convert to binary
# Split the data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)
# Build the model
model = LogisticRegression()
[Link](X_train, y_train)
# Make predictions
y_pred = [Link](X_test)
# Evaluate accuracy
print("Accuracy:", accuracy_score(y_test, y_pred))
3. k-Nearest Neighbors (k-NN)
3.1 What is k-NN?
k-Nearest Neighbors is a simple, non-parametric classification algorithm. It classifies a new data point
based on the majority class among its ‘k’ closest neighbors in the training data. It works based on distance
(usually Euclidean distance).
3.2 How k-NN Works
Prepared by the faculties of CSS dept Brainware University, Kolkata
2
Programme Name: MCA Semester III
Course Name & Code: Data Science & MCA37114
Class: MCA2024
Academic Session: 2025-26
Choose the number of neighbors k
Measure distance between the new point and all other points
Select the k nearest points
Predict the class that is most common among those neighbors
3.3 Applications
Recommender systems
Image recognition
Customer segmentation
3.4 Implementation of k-NN in Python
from [Link] import KNeighborsClassifier
from [Link] import load_iris
from sklearn.model_selection import train_test_split
from [Link] import accuracy_score
# Load data
data = load_iris()
X = [Link]
y = [Link]
# Split data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)
# Build k-NN model
knn = KNeighborsClassifier(n_neighbors=3)
[Link](X_train, y_train)
# Predict and evaluate
y_pred = [Link](X_test)
print("Accuracy:", accuracy_score(y_test, y_pred))
4. k-Means Clustering
4.1 What is k-Means Clustering?
k-Means is an unsupervised learning algorithm used for clustering. It groups data into k clusters based on
similarity. Unlike classification, clustering is used when the data does not have predefined labels.
4.2 How k-Means Works
1. Choose the number of clusters k
2. Randomly initialize k cluster centroids
3. Assign each data point to the nearest centroid
4. Update centroids based on the average of assigned points
5. Repeat steps 3 and 4 until centroids stop changing
4.3 Applications
Prepared by the faculties of CSS dept Brainware University, Kolkata
3
Programme Name: MCA Semester III
Course Name & Code: Data Science & MCA37114
Class: MCA2024
Academic Session: 2025-26
Customer segmentation
Market basket analysis
Image compression
4.4 Implementation of k-Means Clustering in Python
from [Link] import KMeans
from [Link] import load_iris
import [Link] as plt
# Load data
data = load_iris()
X = [Link]
# Apply k-means clustering
kmeans = KMeans(n_clusters=3)
[Link](X)
# Print cluster centers
print("Cluster Centers:\n", kmeans.cluster_centers_)
# Plot clustering result
[Link](X[:, 0], X[:, 1], c=kmeans.labels_, cmap='viridis')
[Link]("k-Means Clustering")
[Link]("Feature 1")
[Link]("Feature 2")
[Link]()
5. Summary of Key Concepts
Concept Type Goal Output
Logistic Regression Classification Predict binary outcomes 0 or 1
k-Nearest Neighbors Classification Classify using neighbor votes Class label
k-Means Clustering Clustering Group similar data points Cluster ID
Prepared by the faculties of CSS dept Brainware University, Kolkata
4