Department of Computer Engineering
Roll no.- B1-518
Date: 24/9/2025
Experiment No.-9(b)
Aim: To study and implement clustering, classification and association rule algorithms.
Theory:
Classification is a supervised data analysis task that constructs a model to distinguish between
predefined classes or concepts. It predicts the category to which a new observation belongs,
based on a training set with known class labels.
Key Concepts
Classes and Training Set:
A class is a label or category (e.g., “Safe” vs. “Risky”). The training set consists of observations
whose class membership is known.
Two-Step Process:
1. Learning (Training) Phase
o Construct the classification model by applying algorithms to the training set.
o The model “learns” patterns that distinguish classes to enable accurate predictions.
2. Classification (Testing) Phase
o Apply the model to test data to predict class labels.
o Estimate model accuracy using the known labels in the test set.
Example:
Before approving a project, a classifier predicts whether the project is Safe or Risky based on
historical project data.
Training and Testing Analogy
Training: Like a person learning to move away when a fan starts falling.
Testing: Observing whether the person moves aside when a heavy object is falling.
Similarly, the data model must be trained on known examples and then tested on new data to validate
its predictive ability.
Data Types and Attributes
An attribute (feature) represents a characteristic of an object. Common attribute types
include:
Binary: Two possible values (e.g., True/False).
– Symmetric: Both values equally important (Yes/No survey response).
– Asymmetric: Values have different significance (e.g., presence vs. absence of disease).
Nominal: More than two categories, unordered (e.g., colors: Red, Green, Black, Yellow).
Ordinal: Ordered categories (e.g., grades: A, B, C, D).
Continuous: Infinite values in a range (e.g., weights: 50.0, 51.5, 52.3 kg).
Discrete: Finite values (e.g., marks: 65, 70, 75, 80, 90).
Decision Trees and the ID3 Algorithm
A Decision Tree is a flowchart-like structure used for classification and prediction:
Root Node: Represents the entire dataset.
Internal Nodes: Decision points based on attribute tests.
Branches: Outcomes of attribute tests leading to further nodes.
Leaf Nodes: Final class labels or predictions.
ID3 (Iterative Dichotomiser 3)
A greedy, top-down decision tree algorithm for categorical data that uses Information Gain to
select the best attribute at each node.
3. Building the Tree:
a. Compute entropy of the current dataset.
b. For each attribute, compute information gain.
c. Choose the attribute with highest gain as the decision node.
d. Split the dataset by that attribute’s values.
e. Recursively apply steps a–d to each subset until
o All examples in a subset belong to the same class, or
o No attributes remain.
Example (Play Tennis):
Deciding whether to play tennis based on Outlook, Temperature, Humidity, and Wind.
Calculate entropy of the full dataset (3 “Yes”, 2 “No”).
Compute information gain for each attribute; Outlook has the highest gain and becomes the root node.
Continue recursively for each branch until a complete tree is formed.
Implementation:
Dataset:
Conclusion:
Classification assigns data to predefined classes using supervised learning. Decision trees,
especially the ID3 algorithm, provide an intuitive, effective, and interpretable method by
selecting attributes that maximise information gain and reduce uncertainty