Scribd
Upload
107 views3 pages

Rule-Based Classification in Data Mining

Rule-based classification uses IF-THEN rules to classify data. A rule consists of a condition (IF) and a prediction (THEN). Rules can be extracted from decision trees by creating one rule per path from root to leaf. Sequential covering algorithms learn one rule at a time to cover examples of a target class, removing covered examples and repeating. Rule pruning removes conditions to improve accuracy on new data, keeping pruned rules that perform better.

Uploaded by

Dr B Anjanadevi
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
107 views3 pages

Rule-Based Classification in Data Mining

Rule-based classification uses IF-THEN rules to classify data. A rule consists of a condition (IF) and a prediction (THEN). Rules can be extracted from decision trees by creating one rule per path from root to leaf. Sequential covering algorithms learn one rule at a time to cover examples of a target class, removing covered examples and repeating. Rule pruning removes conditions to improve accuracy on new data, keeping pruned rules that perform better.

Uploaded by

Dr B Anjanadevi
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
  • Rule-Based Classification
  • Rule Induction Using Sequential Covering Algorithm
  • Rule Pruning

11/28/22, 8:20 PM Data Mining - Rule Based Classification

Data Mining - Rule Based Classification

IF-THEN Rules
Rule-based classifier makes use of a set of IF-THEN rules for classification. We can express a
rule in the following from −

IF condition THEN conclusion

Let us consider a rule R1,

R1: IF age = youth AND student = yes

THEN buy_computer = yes

Points to remember −

The IF part of the rule is called rule antecedent or precondition.

The THEN part of the rule is called rule consequent.


The antecedent part the condition consist of one or more attribute tests and these tests are
logically ANDed.
The consequent part consists of class prediction.

Note − We can also write rule R1 as follows −

R1: (age = youth) ^ (student = yes))(buys computer = yes)

If the condition holds true for a given tuple, then the antecedent is satisfied.

Rule Extraction
Here we will learn how to build a rule-based classifier by extracting IF-THEN rules from a
decision tree.

Points to remember −

To extract a rule from a decision tree −

One rule is created for each path from the root to the leaf node.

To form a rule antecedent, each splitting criterion is logically ANDed.

[Link] 1/3
11/28/22, 8:20 PM Data Mining - Rule Based Classification

The leaf node holds the class prediction, forming the rule consequent.

Rule Induction Using Sequential Covering Algorithm


Sequential Covering Algorithm can be used to extract IF-THEN rules form the training data. We
do not require to generate a decision tree first. In this algorithm, each rule for a given class
covers many of the tuples of that class.

Some of the sequential Covering Algorithms are AQ, CN2, and RIPPER. As per the general
strategy the rules are learned one at a time. For each time rules are learned, a tuple covered by
the rule is removed and the process continues for the rest of the tuples. This is because the path
to each leaf in a decision tree corresponds to a rule.

Note − The Decision tree induction can be considered as learning a set of rules simultaneously.

The Following is the sequential learning Algorithm where rules are learned for one class at a
time. When learning a rule from a class Ci, we want the rule to cover all the tuples from class C
only and no tuple form any other class.

Algorithm: Sequential Covering

Input:

D, a data set class-labeled tuples,

Att_vals, the set of all attributes and their possible values.

Output: A Set of IF-THEN rules.

Method:

Rule_set={ }; // initial set of rules learned is empty

for each class c do

repeat

Rule = Learn_One_Rule(D, Att_valls, c);

remove tuples covered by Rule form D;

until termination condition;

Rule_set=Rule_set+Rule; // add a new rule to rule-set

end for

return Rule_Set;

Rule Pruning
The rule is pruned is due to the following reason −

The Assessment of quality is made on the original set of training data. The rule may perform
well on training data but less well on subsequent data. That's why the rule pruning is
required.

[Link] 2/3
11/28/22, 8:20 PM Data Mining - Rule Based Classification

The rule is pruned by removing conjunct. The rule R is pruned, if pruned version of R has
greater quality than what was assessed on an independent set of tuples.

FOIL is one of the simple and effective method for rule pruning. For a given rule R,

FOIL_Prune = pos - neg / pos + neg

where pos and neg is the number of positive tuples covered by R, respectively.

Note − This value will increase with the accuracy of R on the pruning set. Hence, if the
FOIL_Prune value is higher for the pruned version of R, then we prune R.

[Link] 3/3

Common questions

Powered by AI

Assessing rule quality on the original set of training data is significant as it provides an initial benchmark for rule performance, highlighting discrepancies between training and real-world data application. It allows researchers to identify rules that perform well on training data but may generalize poorly to new data, thus necessitating pruning to enhance real-world applicability .

The FOIL_Prune metric is used to evaluate a rule's quality during the pruning process. It is calculated as FOIL_Prune = (pos - neg) / (pos + neg), where pos and neg represent the number of positive and negative tuples covered by the rule, respectively. A higher FOIL_Prune value indicates better accuracy on the pruning set, suggesting that the rule should be pruned if the pruned version has a higher value .

Rule pruning is necessary because although a rule may perform well on training data, it may perform less well on new data. The pruning process improves the generalization capability of the rules. FOIL (First Order Inductive Learner) is a common and effective method for rule pruning, which evaluates the quality of a rule using a pruning set .

In standard decision tree methods, forming a rule antecedent involves using each path from the root to a leaf node where each node's splitting criterion is logically ANDed. In contrast, the Sequential Covering Algorithm forms rule antecedents directly from training data without a tree, learning rules for one class at a time and modifying the dataset by removing covered tuples iteratively .

A rule-based classifier uses IF-THEN rules for classification where the IF part is called the rule antecedent or precondition, and the THEN part is called the rule consequent. The antecedent part consists of one or more attribute tests, which are logically ANDed together. The consequent part consists of a class prediction .

A rule-based classifier might be preferred when interpretability of the model is crucial, as the IF-THEN rules are easy to understand and explain. It is also beneficial when dealing with datasets where rules can be intuitively mapped to domain knowledge. Additionally, rule-based classifiers can be advantageous in scenarios with a strong need for domain-specific customization or when dealing with imbalanced datasets .

Challenges in rule pruning include determining the optimal level of pruning to avoid overfitting and underfitting, ensuring that pruned rules maintain their robustness across diverse datasets, and handling the computational cost associated with evaluating rules against a pruning set. Additionally, selecting appropriate metrics that accurately reflect rule performance in a given context can also be difficult .

To extract IF-THEN rules from a decision tree, rules are created for each path from the root to a leaf node. Each splitting criterion in the path forms the rule antecedent by being logically ANDed together, while the leaf node holds the class prediction, forming the rule consequent .

The Sequential Covering Algorithm offers the advantage of directly generating rule sets from training data without the need to first create a decision tree. This method allows for focusing on one class at a time, removing covered tuples for more efficient processing. Additionally, it avoids the complexity and overhead associated with building a complete decision tree, potentially leading to more concise and targeted rule sets .

The Sequential Covering Algorithm is used to extract IF-THEN rules directly from training data without first generating a decision tree. It covers data by learning one rule at a time for a given class, removing tuples covered by each rule, and repeating the process until a termination condition is met. This method contrasts with decision tree induction, which simultaneously learns a set of rules through the tree structure .

11/28/22, 8:20 PM Data Mining - Rule Based Classification https://www.tutorialspoint.com/data_mining/dm_rbc.htm 1/3 Data Mini
11/28/22, 8:20 PM Data Mining - Rule Based Classification https://www.tutorialspoint.com/data_mining/dm_rbc.htm 2/3 The leaf
11/28/22, 8:20 PM Data Mining - Rule Based Classification https://www.tutorialspoint.com/data_mining/dm_rbc.htm 3/3 The rule

You might also like