Decision Tree Algorithm
Learning Objective
To develop a Decision Tree-based classification model and refine its results by adjusting key
hyperparameters.
Learning Context
This program offers a detailed understanding of the Decision Tree algorithm for classification
problems using the Iris dataset. The Iris dataset includes measurements of different flower
species based on features such as sepal length, sepal width, petal length, and petal width. The
main objective is to guide learners in building, training, and evaluating a decision tree
classification model.
The process begins with loading the dataset and dividing it into training and testing sets using the
train_test_split function. This step is essential for validating the model’s performance and
reducing the risk of overfitting. After splitting the data, the Decision Tree classifier is trained using
the training dataset. The model’s performance is then assessed using evaluation metrics such as
accuracy score, confusion matrix, and classification report. These metrics help learners
understand important performance measures including precision, recall, F1-score, and overall
model accuracy.
Furthermore, the program includes visualization of the decision tree using the plot_tree function.
This visualization helps learners clearly see how the algorithm makes decisions by splitting the
dataset based on feature values. The program also covers hyperparameter tuning using
GridSearchCV, allowing learners to test different parameters such as criterion, max_depth, and
min_samples_split. By experimenting with these parameters, learners can identify the best
combination that improves the model’s accuracy and generalization ability.
Exercise
Here is the Decision Tree implementation using the Iris dataset with clear explanation for
lab/practical use
Solution
#Step 1: Import Required Libraries
import pandas as pd
import numpy as np
from [Link] import load_iris
from sklearn.model_selection import train_test_split
from [Link] import DecisionTreeClassifier
from [Link] import accuracy_score, confusion_matrix, classification_report
#Step 2: Load Iris Dataset
iris = load_iris()
X = [Link] # Features
y = [Link] # Target labels
feature_names = iris.feature_names
target_names = iris.target_names
print("Feature Names:", feature_names)
print("Target Names:", target_names)
# Step 3: Split Data into Training & Testing
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.3, random_state=42
)
#Step 4: Create Decision Tree Model
model = DecisionTreeClassifier(
criterion='gini', # Split using Gini Index
max_depth=3, # Control tree depth
random_state=42
)
#Step 5: Train the Model
[Link](X_train, y_train)
#Step 6: Make Predictions
y_pred = [Link](X_test)
#Step 7: Evaluate Model
print("Accuracy:", accuracy_score(y_test, y_pred))
print("\nConfusion Matrix:\n", confusion_matrix(y_test, y_pred))
print("\nClassification Report:\n", classification_report(y_test, y_pred))
03/03/2026, 19:34 Decision Tree Algorithm
In [1]: !pip install pandas
Requirement already satisfied: pandas in d:\software\lib\site-packages (2.3.3)
Requirement already satisfied: numpy>=1.26.0 in d:\software\lib\site-packages (from
pandas) (2.3.5)
Requirement already satisfied: python-dateutil>=2.8.2 in d:\software\lib\site-packag
es (from pandas) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in d:\software\lib\site-packages (from p
andas) (2025.2)
Requirement already satisfied: tzdata>=2022.7 in d:\software\lib\site-packages (from
pandas) (2025.2)
Requirement already satisfied: six>=1.5 in d:\software\lib\site-packages (from pytho
n-dateutil>=2.8.2->pandas) (1.17.0)
In [7]: #Step 1: Import Required Libraries
import pandas as pd
import numpy as np
from [Link] import load_iris
from sklearn.model_selection import train_test_split
from [Link] import DecisionTreeClassifier
from [Link] import accuracy_score, confusion_matrix, classification_report
In [8]: #Step 2: Load Iris Dataset
iris = load_iris()
X = [Link] # Features
y = [Link] # Target labels
feature_names = iris.feature_names
target_names = iris.target_names
print("Feature Names:", feature_names)
print("Target Names:", target_names)
Feature Names: ['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal
width (cm)']
Target Names: ['setosa' 'versicolor' 'virginica']
In [9]: # Step 3: Split Data into Training & Testing
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.3, random_state=42
)
In [10]: #Step 4: Create Decision Tree Model
model = DecisionTreeClassifier(
criterion='gini', # Split using Gini Index
max_depth=3, # Control tree depth
random_state=42
)
In [11]: #Step 5: Train the Model
[Link](X_train, y_train)
localhost:8888/doc/tree/Decision Tree Algorithm .ipynb 1/2
03/03/2026, 19:34 Decision Tree Algorithm
Out[11]: ▾ DecisionTreeClassifier i ?
Parameters
In [12]: #Step 6: Make Predictions
y_pred = [Link](X_test)
In [13]: #Step 7: Evaluate Model
print("Accuracy:", accuracy_score(y_test, y_pred))
print("\nConfusion Matrix:\n", confusion_matrix(y_test, y_pred))
print("\nClassification Report:\n", classification_report(y_test, y_pred))
Accuracy: 1.0
Confusion Matrix:
[[19 0 0]
[ 0 13 0]
[ 0 0 13]]
Classification Report:
precision recall f1-score support
0 1.00 1.00 1.00 19
1 1.00 1.00 1.00 13
2 1.00 1.00 1.00 13
accuracy 1.00 45
macro avg 1.00 1.00 1.00 45
weighted avg 1.00 1.00 1.00 45
localhost:8888/doc/tree/Decision Tree Algorithm .ipynb 2/2