0% found this document useful (0 votes)
506 views17 pages

Find-S Algorithm in Machine Learning

The Find-S algorithm is a concept learning algorithm that finds the most specific hypothesis that fits all the positive training examples by starting with the most specific hypothesis and generalizing it for each positive example where attributes do not match. It initializes the hypothesis to the most specific representation and replaces attribute values with '?' for positive examples where the attribute value does not match the hypothesis to generalize it, ignoring negative examples. The final hypothesis after processing all examples fits all positive examples in the most general way.
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
506 views17 pages

Find-S Algorithm in Machine Learning

The Find-S algorithm is a concept learning algorithm that finds the most specific hypothesis that fits all the positive training examples by starting with the most specific hypothesis and generalizing it for each positive example where attributes do not match. It initializes the hypothesis to the most specific representation and replaces attribute values with '?' for positive examples where the attribute value does not match the hypothesis to generalize it, ignoring negative examples. The final hypothesis after processing all examples fits all positive examples in the most general way.
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
  • Find S Algorithm Overview
  • Steps Involved in Find-S
  • Examples and Application
  • Conceptual Explanation
  • Algorithm Implementation
  • How Does It Work
  • Limitations of Find-S
  • Implementation of Find-S
  • Resources and Further Reading

Find S Algorithm

The find-S algorithm is a basic concept learning


algorithm in machine learning. The find-S algorithm
finds the most specific hypothesis that fits all the positive
examples. We have to note here that the algorithm
considers only those positive training example.

Introduction :
The find-S algorithm starts with the most specific
hypothesis and generalizes this hypothesis each time it
fails to classify an observed positive training data.
Hence, the Find-S algorithm moves from the most
specific hypothesis to the most general hypothesis.
Important Representation :

1. ? indicates that any value is acceptable for the


attribute.
2. specify a single required value ( e.g., Cold ) for
the attribute.
3. ϕindicates that no value is acceptable.
4. The most general hypothesis is represented
by: {?, ?, ?, ?, ?, ?}
5. The most specific hypothesis is represented
by: {ϕ, ϕ, ϕ, ϕ, ϕ, ϕ}
Steps Involved In Find-S :

1. Start with the most specific hypothesis.


h = {ϕ, ϕ, ϕ, ϕ, ϕ, ϕ}
2. Take the next example and if it is negative, then
no changes occur to the hypothesis.
3. If the example is positive and we find that our
initial hypothesis is too specific then we update
our current hypothesis to a general condition.
4. Keep repeating the above steps till all the training
examples are complete.
5. After we have completed all the training examples
we will have the final hypothesis when can use to
classify the new examples.
Example :
Consider the following data set having the data about
which particular seeds are poisonous.

First, we consider the hypothesis to be a more specific


hypothesis. Hence, our hypothesis would be :
h = {ϕ, ϕ, ϕ, ϕ, ϕ, ϕ}

Consider example 1 :
The data in example 1 is { GREEN, HARD, NO,
WRINKLED }. We see that our initial hypothesis is
more specific and we have to generalize it for this
example. Hence, the hypothesis becomes :
h = { GREEN, HARD, NO, WRINKLED }
Consider example 2 :
Here we see that this example has a negative outcome.
Hence we neglect this example and our hypothesis
remains the same.
h = { GREEN, HARD, NO, WRINKLED }
Consider example 3 :
Here we see that this example has a negative outcome.
Hence we neglect this example and our hypothesis
remains the same.
h = { GREEN, HARD, NO, WRINKLED }
Consider example 4 :
The data present in example 4 is { ORANGE, HARD,
NO, WRINKLED }. We compare every single attribute
with the initial data and if any mismatch is found we
replace that particular attribute with a general case ( ” ?
” ). After doing the process the hypothesis becomes :
h = { ?, HARD, NO, WRINKLED }
Consider example 5 :
The data present in example 5 is { GREEN, SOFT, YES,
SMOOTH }. We compare every single attribute with the
initial data and if any mismatch is found we replace that
particular attribute with a general case ( ” ? ” ). After
doing the process the hypothesis becomes :
h = { ?, ?, ?, ? }
Since we have reached a point where all the attributes in
our hypothesis have the general condition, example 6 and
example 7 would result in the same hypothesizes with all
general attributes.
h = { ?, ?, ?, ? }
Hence, for the given data the final hypothesis would be
:
Final Hyposthesis: h = { ?, ?, ?, ? }

Algorithm :

1. Initialize h to the most specific hypothesis in H

2. For each positive training instance x


For each attribute constraint a, in h

If the constraint a, is satisfied by x

Then do nothing

Else replace a, in h by the next more general


constraint that is satisfied by x

3. Output hypothesis h

How To Implement Find-S Algorithm In Machine


Learning?

In Machine Learning, concept learning can be termed as


“a problem of searching through a predefined space of
potential hypothesis for the hypothesis that best fits the
training examples” – Tom Mitchell. In this article, we
will go through one such concept learning algorithm
known as the Find-S algorithm. The following topics are
discussed in this article.
• What is Find-S Algorithm in Machine Learning?
• How Does it Work?
• Limitations of Find-S Algorithm
• Implementation of Find-S Algorithm
• Use Case

What is Find-S Algorithm in Machine Learning?

In order to understand Find-S algorithm, you need to


have a basic idea of the following concepts as well:

1. Concept Learning
2. General Hypothesis
3. Specific Hypothesis

1. Concept Learning

Let’s try to understand concept learning with a real-life


example. Most of human learning is based on past
instances or experiences. For example, we are able to
identify any type of vehicle based on a certain set of
features like make, model, etc., that are defined over a
large set of features.

These special features differentiate the set of cars, trucks,


etc from the larger set of vehicles. These features that
define the set of cars, trucks, etc are known as concepts.

Similar to this, machines can also learn from concepts to


identify whether an object belongs to a specific category
or not. Any algorithm that supports concept learning
requires the following:

• Training Data
• Target Concept
• Actual Data Objects

2. General Hypothesis

Hypothesis, in general, is an explanation for something.


The general hypothesis basically states the general
relationship between the major variables. For example, a
general hypothesis for ordering food would be I want a
burger.
G = { ‘?’, ‘?’, ‘?’, …..’?’}

3. Specific Hypothesis

The specific hypothesis fills in all the important details


about the variables given in the general hypothesis. The
more specific details into the example given above would
be I want a cheeseburger with a chicken pepperoni
filling with a lot of lettuce.

S = {‘Φ’,’Φ’,’Φ’, ……,’Φ’}

Python Machine Learning Certification Training

• Instructor-led Live Sessions


• Real-life Case Studies
• Assignments
• Lifetime Access

Explore Curriculum
Now ,let’s talk about the Find-S Algorithm in Machine
Learning.

The Find-S algorithm follows the steps written below:

1. Initialize ‘h’ to the most specific hypothesis.


2. The Find-S algorithm only considers the positive
examples and eliminates negative examples. For
each positive example, the algorithm checks for
each attribute in the example. If the attribute value
is the same as the hypothesis value, the algorithm
moves on without any changes. But if the attribute
value is different than the hypothesis value, the
algorithm changes it to ‘?’.

Now that we are done with the basic explanation of the


Find-S algorithm, let us take a look at how it works.

How Does It Work?


1. The process starts with initializing ‘h’ with the most
specific hypothesis, generally, it is the first positive
example in the data set.
2. We check for each positive example. If the example
is negative, we will move on to the next example but
if it is a positive example we will consider it for the
next step.
3. We will check if each attribute in the example is
equal to the hypothesis value.
4. If the value matches, then no changes are made.
5. If the value does not match, the value is changed to
‘?’.
6. We do this until we reach the last positive example
in the data set.

Limitations of Find-S Algorithm

There are a few limitations of the Find-S algorithm listed


down below:

1. There is no way to determine if the hypothesis is


consistent throughout the data.
2. Inconsistent training sets can actually mislead the
Find-S algorithm, since it ignores the negative
examples.
3. Find-S algorithm does not provide a backtracking
technique to determine the best possible changes
that could be done to improve the resulting
hypothesis.

Implementation of Find-S Algorithm


To understand the implementation, let us try to
implement it to a smaller data set with a bunch of
examples to decide if a person wants to go for a walk.

The concept of this particular problem will be on what


days does a person likes to go on walk.

Weath Temperat Compa Humid Go


Time Wind
er ure ny ity es
Morni Stron
Sunny Warm Yes Mild Yes
ng g
Eveni Norm
Rainy Cold No Mild No
ng al
Morni Norm
Sunny Moderate Yes Normal Yes
ng al
Eveni Stron
Sunny Cold Yes High Yes
ng g
Looking at the data set, we have six attributes and a final
attribute that defines the positive or negative example. In
this case, yes is a positive example, which means the
person will go for a walk.
So now, the general hypothesis is:

Next
h0 = {‘Morning’, ‘Sunny’, ‘Warm’, ‘Yes’, ‘Mild’,
‘Strong’}

This is our general hypothesis, and now we will consider


each example one by one, but only the positive examples.

h1= {‘Morning’, ‘Sunny’, ‘?’, ‘Yes’, ‘?’, ‘?’}

h2 = {‘?’, ‘Sunny’, ‘?’, ‘Yes’, ‘?’, ‘?’}

We replaced all the different values in the general


hypothesis to get a resultant hypothesis. Now that we
know how the Find-S algorithm works, let us take a look
at an implementation using Python.

Use Case
Let’s try to implement the above example using Python.
The code to implement the Find-S algorithm using the
above data is given below.

1 import pandas as pd
2 import numpy as np
3
4 #to read the data in the csv file
5 data = pd.read_csv("[Link]")
6 print(data,"n")
7
8 #making an array of all the attributes
9 d = [Link](data)[:,:-1]
10print("n The attributes are: ",d)
11
12#segragating the target that has positive and negative examples
13target = [Link](data)[:,-1]
14print("n The target is: ",target)
15
16#training function to implement find-s algorithm
17def train(c,t):
18 for i, val in enumerate(t):
19 if val == "Yes":
20 specific_hypothesis = c[i].copy()
21 break
22
23 for i, val in enumerate(c):
24 if t[i] == "Yes":
25 for x in range(len(specific_hypothesis)):
26 if val[x] != specific_hypothesis[x]:
27 specific_hypothesis[x] = '?'
28 else:
29 pass
30
31 return specific_hypothesis
32
33#obtaining the final hypothesis
34print("n The final hypothesis is:",train(d,target))
Output:
U Tube link

1. (2817) Find-S Algorithm (concept) | Machine


Learning (2018) - YouTube
2. (2817) FIND S Algorithm | Finding A Maximally
Specific Hypothesis | Solved Example - 1 by
Mahesh Huddar - YouTube
3. (2817) Machine Learning | Find-S Algorithm -
YouTube( EASY)

Common questions

Powered by AI

The Find-S Algorithm handles variability in data attributes by replacing incongruent attribute values with '?' during hypothesizing. This substitution means that any example with mismatched attributes becomes generalized, working towards a common hypothesis covering all positive instances' potential variability. Such a treatment of attribute variability implies that the final hypothesis can often be overly general, as any difference across initial specific-to-specific comparisons induces a generalization. This promotes simplification but risks loss of precision if the attributes were crucial for distinction .

The Find-S Algorithm is limited because it only considers positive examples and ignores negative examples, making it vulnerable to inconsistencies in the training data. This singular focus can mislead the algorithm into forming hypotheses that are overgeneralized, which may not correctly classify all relevant negative instances. Furthermore, it lacks a mechanism for backtracking to refine the hypothesis based on negative examples, potentially leading to suboptimal or incorrect generalizations. Consequently, its practical application is limited to datasets where negative instances do not provide crucial information, reducing its usability in more complex or nuanced machine learning tasks .

The emergence of diverse positive training instances in the Find-S Algorithm inherently drives the hypothesis towards greater generality because each difference in attribute values among examples leads to a generalization ('?') in the hypothesis. When examples introduce varied attributes, the hypothesis is forced to generalize those attributes to retain consistent inclusion of all positive instances. As a result, while adapting to diverse attributes, the algorithm incrementally replaces specific values with more general placeholders, potentially culminating in a hypothesis overly broad yet sufficient to categorize every positive instance as intended .

The Find-S Algorithm implementation involves initializing the hypothesis 'h' to the most specific state, represented by {ϕ, ϕ, ϕ, ϕ, ϕ, ϕ}. For each positive training example, if an attribute value matches the current hypothesis, it is left unchanged. If there is a mismatch, the attribute in the hypothesis is replaced by the more general '?'. This process is repeated for all positive instances, ensuring the hypothesis becomes more general as necessary to fit all positive examples. However, the algorithm only considers positive examples and ignores negative ones, potentially leading directly to the most general hypothesis: {?, ?, ?, ?, ?, ?} if all attributes differ across examples .

In the Find-S Algorithm, a specific hypothesis is one that precisely specifies all attributes for the observed data, initially represented as {ϕ, ϕ, ϕ, ϕ, ϕ, ϕ}. A general hypothesis, conversely, uses '?' to denote any possible value for an attribute, indicating greater generality in matching data. Specific hypotheses are narrowed or constrained, while general hypotheses cover a broader range of examples. These hypotheses are essential in concept learning as they allow the algorithm to gradually adapt a hypothesis that accurately categorizes all positive instances without overfitting the model to specific cases .

Upon encountering a new positive example, the Find-S Algorithm compares each attribute of the example against the current hypothesis. If an attribute matches, no action is taken; if it does not, the attribute is generalized to '?'. This sequence cumulatively refines the hypothesis to accommodate all observed positive data, ensuring it remains consistently inclusive across different examples. The process is integral as it gradually changes the hypothesis from very specific to adequately general, seeking balance between fitting the maximum number of positive instances and maintaining specificity .

Initialization in the Find-S Algorithm is crucial as it begins the hypothesis with the most specific state {ϕ, ϕ, ϕ, ϕ, ϕ, ϕ}, representing absolute specificity. This starting point is integral because it forms the baseline from which generalization occurs. Each step involves modifying specific attributes only as needed to accommodate positive instances, ensuring precision in potentially chaotic data sets and facilitating progressive refinement. The initial specificity inherently guides the development of the hypothesis, constraining changes to strictly necessary adaptions, thus influencing the final balance between complexity and simplicity achieved .

The Find-S Algorithm simplifies hypothesis creation by maintaining a straightforward decision rule that strictly evaluates positive examples. Unlike other complex models involving iterative adjustments, backtracking, or error corrections, Find-S deterministically progresses from specificity to generality, reducing computational complexity. The trade-off is a lack of robustness to errors from negative instances and potential over-reliance on solely positive indications, which can lead to inadequately generalized hypotheses or failure to account for noise or outliers in data. This simplicity ensures fast processing and ease of implementation but constrains its versatility in dynamic or mixed datasets .

The Find-S Algorithm is best suited for scenarios where negative examples are less significant, such as identifying consistent traits across positively characterized datasets like user preferences in demographic research or consistent patterns in quality control where deviations are rare. Its primary limitation in real-world applications arises from overlooking negative instances, potentially leading to overgeneralizations. Yet, its effectiveness is enhanced in environments where only consistent positive categorization is actionable, providing straightforward hypothesis outputs without complex algorithmic decision-making, instrumental in rapid concept establishment tasks .

Concept learning refers to the process of learning to distinguish instances of a concept based on their features, essentially forming a hypothesis that best categorizes the examples. The Find-S Algorithm is a specific implementation within concept learning that identifies the most specific hypothesis matching all positive examples without revisiting negative ones. While traditional concept learning might involve iterative reevaluation of both types of instances, Find-S bypasses this by focusing purely on positive reinforcement, thereby simplifying hypothesis formation but at the expense of comprehensiveness. This straight-line approach resolves differences through specificity without accounting for error correction mechanisms for misclassified negatives .

Find S Algorithm 
The find-S algorithm is a basic concept learning 
algorithm in machine learning. The find-S algorithm 
find
4. The most general hypothesis is represented 
by: {?, ?, ?, ?, ?, ?} 
5. The most specific hypothesis is represented 
by: {ϕ
Consider the following data set having the data about 
which 
particular 
seeds 
are 
poisonous.  
  
 
First, we consider th
example. 
Hence, 
the 
hypothesis 
becomes 
:  
h = { GREEN, HARD, NO, WRINKLED }  
Consider 
example 
2 
:  
Here we see tha
Consider 
example 
5 
:  
The data present in example 5 is { GREEN, SOFT, YES, 
SMOOTH }. We compare every single attribute w
For each attribute constraint a, in h 
        If the constraint a, is satisfied by x 
        Then do nothing 
        E
• What is Find-S Algorithm in Machine Learning? 
• How Does it Work? 
• Limitations of Find-S Algorithm 
• Implementation of
features like make, model, etc., that are defined over a 
large set of features. 
These special features differentiate the se
G = { ‘?’, ‘?’, ‘?’, …..’?’} 
3. Specific Hypothesis 
The specific hypothesis fills in all the important details 
about the v
Now ,let’s talk about the Find-S Algorithm in Machine 
Learning. 
The Find-S algorithm follows the steps written below: 
1. I

You might also like