Scribd
Upload
11 views27 pages

Decision Tree Construction Explained

A decision tree is a powerful supervised learning tool used for classification and regression, constructed by recursively splitting training data based on attribute values. The algorithm selects the best attribute to split the data using metrics like entropy or Gini impurity to maximize information gain. The document also provides examples of information gain calculations for various weather attributes.

Uploaded by

pallelaharish02
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
11 views27 pages

Decision Tree Construction Explained

A decision tree is a powerful supervised learning tool used for classification and regression, constructed by recursively splitting training data based on attribute values. The algorithm selects the best attribute to split the data using metrics like entropy or Gini impurity to maximize information gain. The document also provides examples of information gain calculations for various weather attributes.

Uploaded by

pallelaharish02
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

DECESION TREE USING DIVIDE &

CONQUER METHOD
• A decision tree is one of the most powerful tools
of supervised learning algorithms used for both
classification and regression tasks.
• It is constructed by recursively splitting the
training data into subsets based on the values of
the attributes until a stopping criterion is met.
• During training, the Decision Tree algorithm
selects the best attribute to split the data based
on a metric such as entropy or Gini impurity,
which measures the level of impurity
• The goal is to find the attribute that maximizes
the information gain or the reduction in
impurity after the split.
• The problem of constructing a decision tree
can be expressed recursively. First, select an
attribute to place at the root node and make
one branch for each possible value.
• The construction of a decision tree classifier
does not require any domain knowledge or
parameter setting and therefore is appropriate
for exploratory knowledge discovery.
• The only thing left to decide is how to
determine which attribute to split on, given a
set of examples with different classes.
• The measure of purity that we will use is
called the information and is measured in
units called bits.
• Gain(S,outlook)=0.94-5/14x0.971-4/14x0-
5/14x0.971 = 0.247
• Gain(S,Temp)=0.94-4/14x1.0-6/14x0.9183-
4/14x0.8113 = 0.029
• Information gain for attributes from weather
data:
• gain("Outlook") = 0.247 bits
gain("Temperature") = 0.029bits
gain("Humidity") = 0.152 bits
• gain("Windy") = 0.048 bits

You might also like