DATA7703 – Machine Learning for Data Scientists S2 - 2025
Tutorial – Week 2
Supervised Learning - Classification
The below tutorial problems are meant to be solved using pen and paper.
1. Decision Trees
Decision trees have many attractive properties.
• Easy to understand the decision process
• Can handle both discrete and continuous features
• Highly flexible, decision trees can represent increasingly complex decision boundaries as the
depth (or number of nodes) of the tree increases.
Given below is a number of data instances belonging to class 0 or class 1. We have also been
provided with a decision tree (trained by someone) to classify these data points.
a) What is the dimension of the feature space? 2
b) What is the depth of the decision tree? 3
c) How are the two data points underlined in red in the left figure below classified by the
decision tree? Both classified correctly
d) Draw the decision boundary of the decision tree. See figure below
e) What is the classification accuracy of this decision tree? 16/19 ≈ 84%
2. Naïve Bayes Classifier
You have been hired by Amazon to work on their recommendation system for books.
Your first task is to see if you can predict the recommendations of a single user, using the
following two attributes:
Genre (Romance, Self-Help, or Thriller) and
Price (High, Medium, or Low).
The classification is whether the user recommended the book or not.
You collect the following training set:
Book Genre Price Class
B1 Romance Low Recommended
B2 Romance Medium Recommended
B3 Thriller Low Recommended
B4 Thriller High Recommended
B5 Self-Help Low Recommended
B6 Self-Help High Not Recommended
B7 Romance High Not Recommended
Show (using Bayes rule) how a Naïve Bayes classifier, using add-1 (α=1) Laplacian smoothing,
would classify the following new examples as “Recommended” or “Not Recommended”
Book Genre Price Class
B8 Self-Help Medium ?
B9 Romance High ?
Solution:
B8:
5 2
P(Recommended) = 7 P(Not Recommended) = 7
1 2 1 2
P(Self-Help | Recommended) = 5 → 8 P(Medium | Recommended) = 5 → 8
1 2 0 1
P(Self-Help | Not Recommended) = 2 → 5 P(Medium | Not Recommended) = 2 → 5
P(Rec | Self-Help, Medium) =
= P(Self-Help, Medium | Rec) P(Rec ) / P(Self-Help, Medium) =
= P(Self-Help | Rec ) P(Medium | Rec ) P(Rec ) / P(Self-Help, Medium) =
225
= / P(Self-Help, Medium) = 0.045 / P(Self-Help, Medium)
887
P(Not Rec | Self-Help, Medium) =
= P(Self-Help, Medium | Not Rec) P(Not Rec ) / P(Self-Help, Medium) =
= P(Self-Help | Not Rec ) P(Medium | Not Rec ) P(Not Rec ) / P(Self-Help, Medium) =
212
= / P(Self-Help, Medium) = 0.023 / P(Self-Help, Medium)
557
Book B8 is Recommended
B9:
5 2
P(Recommended) = 7 P(Not Recommended) = 7
2 3 1 2
P(Romance | Recommended) = → P(High | Recommended) = →
5 8 5 8
1 2 2 3
P(Romance | Not Recommended) = → P(High | Not Recommended) = →
2 5 2 5
P(Rec | Romance, High) =
= P(Romance, High | Rec) P(Rec ) / P(Romance, High) =
= P(Romance | Rec ) P(High | Rec ) P(Rec ) / P(Romance, High) =
325
= / P(Romance, High) = 0.066 / P(Romance, High)
887
P(Not Rec | Romance, High) =
= P(Romance, High | Not Rec) P(Not Rec ) / P(Romance, High) =
= P(Romance | Not Rec ) P(High | Not Rec ) P(Not Rec ) / P(Romance, High) =
232
= / P(Romance, High) = 0.068 / P(Romance, High)
557
Book B9 is Not Recommended