Scribd
Upload
9 views59 pages

Decision Tree and Bayesian Learning Guide

Decision tree learning is a supervised learning technique used for both classification and regression. It involves constructing a decision tree from training data where internal nodes represent attributes, branches represent attribute values, and leaf nodes represent target class labels or values. The best attribute is selected at each node using an attribute selection measure like entropy, which calculates the uncertainty in outcomes. The training data is then split into subsets based on the outcome of the test attribute. This splitting process is repeated recursively until the leaf nodes are pure or a stopping criteria is reached. Bayesian learning is another supervised learning method discussed in the document.

Uploaded by

lavanyalkaroshi
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
9 views59 pages

Decision Tree and Bayesian Learning Guide

Decision tree learning is a supervised learning technique used for both classification and regression. It involves constructing a decision tree from training data where internal nodes represent attributes, branches represent attribute values, and leaf nodes represent target class labels or values. The best attribute is selected at each node using an attribute selection measure like entropy, which calculates the uncertainty in outcomes. The training data is then split into subsets based on the outcome of the test attribute. This splitting process is repeated recursively until the leaf nodes are pure or a stopping criteria is reached. Bayesian learning is another supervised learning method discussed in the document.

Uploaded by

lavanyalkaroshi
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

AIML_Module4

Topics:
1. Decision Tree Learning
2. Bayesian Learning
Decision Tree Learning
Is used for both classification and regression tasks. Decision tree is a concept
tree which summarises the information contained in the training dataset in the
form of a tree structure. The tree can be used to classify both categorical target
values and continuous valued target variables. Here the model for the training
data, the hypothesis function f(x) is a decision tree. It consists of a root node,
internal or decision nodes, branches and terminal nodes/leaf nodes. Figure
below illustrate the graphical symbol used for constructing decision tree:
Symbols used in Decision Tree Example:

Building the tree

Advantages of Decision Trees


Disadvantages of Decision Trees

Example :

Example
Fundamentals of Entropy
The decision tree for training data set with multiple features or attributes is
constructed by selecting best attribute at each level. The best feature is selected
based on the amount of information among attributes which in turn is calculated
using Entropy which is calculated based on the probability of events.
Entropy is the amount of uncertainty or randomness in the outcome of a random
variables or an event. For homogeneous data Entropy is 0. For heterogeneous
data the entropy varies from 0 to 1.
General Algorithm for Decision Trees
1. Find the best attribute from the training dataset using an attribute
selection measure and place it at the roo of the tree
2. Split the training dataset into subsets based on the outcome of the test
attribute.
3. Repeat steps 1 and 2 until we end up with leaf nodes in all brances of tree
4. This splitting process is recursive until stopping criteria is reached
1. If data instances are homogeneous and entropy becomes zero
2. When node become leaf node
3. Maximum tree depth is reached.

Decision Tree Induction Algorithms


Bayesian Learning

You might also like