Scribd
Upload
65 views7 pages

Decision Tree Example: Weather Data Analysis

The document presents a decision tree example using weather data to determine whether to play or not based on attributes like Outlook, Temperature, Humidity, and Windy. It calculates information gain for each attribute to select the best one for splitting the data, with 'Outlook' showing the highest gain of 0.247 bits. The final decision tree is constructed based on these calculations, noting that not all leaves need to be pure.

Uploaded by

debrituandarsa
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
65 views7 pages

Decision Tree Example: Weather Data Analysis

The document presents a decision tree example using weather data to determine whether to play or not based on attributes like Outlook, Temperature, Humidity, and Windy. It calculates information gain for each attribute to select the best one for splitting the data, with 'Outlook' showing the highest gain of 0.247 bits. The final decision tree is constructed based on these calculations, noting that not all leaves need to be pure.

Uploaded by

debrituandarsa
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd

Decision Tree

A complete Example
Weather Data: Play or not Play?
Play
Outlook Temperature Humidity Windy ?
sunny hot high false No
sunny hot high true No
overcast hot high false Yes
rain mild high false Yes
rain cool normal false Yes
rain cool normal true No
overcast cool normal true Yes
sunny mild high false No
sunny cool normal false Yes
rain mild normal false Yes
sunny mild normal true Yes
overcast mild high true Yes
overcast hot normal false Yes
rain mild high true No
Which attribute to select?

Play
Outlook Temperature Humidity Windy ?
sunny hot high false No
sunny hot high true No

overcast hot high false Yes


rain mild high false Yes
rain cool normal false Yes
rain cool normal true No

overcast cool normal true Yes


sunny mild high false No
sunny cool normal false Yes
rain mild normal false Yes
sunny mild normal true Yes

overcast mild high true Yes

overcast hot normal false Yes


rain mild high true No
Example: attribute “Outlook”

• “Outlook” = “Sunny”:
info([2,3]) entropy(2/5,3/5)  2 / 5 log(2 / 5)  3 / 5 log(3 / 5) 0.971 bits
Note: log(0) is
• “Outlook” = “Overcast”: not defined, but
info([4,0]) entropy(1,0)  1log(1)  0 log(0) 0 bits we evaluate
0*log(0) as zero
• “Outlook” = “Rainy”:
info([3,2]) entropy(3/5,2/5)  3 / 5 log(3 / 5)  2 / 5 log(2 / 5) 0.971 bits
• Expected information for attribute:
info([3,2], [4,0],[3,2]) (5 / 14) 0.971  (4 / 14) 0  (5 / 14) 0.971
0.693 bits
Computing the information gain

• Information gain:
(information before split) – (information after
split)
gain(" Outlook") info([9,5]) - info([2,3], [4,0], [3,2]) 0.940 - 0.693
0.247 bits
• Information gain for attributes from weather
data: gain(" Outlook" ) 0.247 bits
gain("Temperature" ) 0.029 bits
gain(" Humidity" ) 0.152 bits
gain(" Windy" ) 0.048 bits
Continuing to split

gain(" Temperatur e" ) 0.571 bits

gain(" Windy" ) 0.020 bits

gain(" Humidity") 0.971 bits


The final decision tree

• Note: not all leaves need to be pure; sometimes


identical instances have different classes
 Splitting stops when data can’t be split any further
(or perhaps sooner to avoid overfitting).

You might also like