Rosenblatt’s Perceptron
Perceptron
• In the simplest form of the perceptron, there are two decision regions separated by a
hyperplane, which is defined by
• Bayes classifier is one of the popular approach for the pattern classification.
• The Bayes classifier is a method in machine learning that uses probability to make
decisions.
• Because it uses probability distributions of the features for each class to make
decisions. Instead of making a strict yes/no decision based solely on thresholds or
rules, it calculates the likelihoods and makes the decision based on which class is
most probable.
For example:
• In "spam" emails, certain words like "free" or "offer" might appear more
frequently.
• In "not spam" emails, words like "meeting" or "schedule" might be more
common.
• The Bayes classifier uses these patterns (probability distributions) to figure out how
likely it is that a new email belongs to the "spam" or "not spam" category based on
the words it contains.
• The Perceptron is indeed the most basic or elementary form of a neural network.
• It was one of the earliest models proposed for machine learning and is the foundation
of more complex neural network architectures
• The Perceptron consists of a single neuron with adjustable weights and a bias.
• It takes a set of input features, applies weights to them, sums them up, adds a bias
term, and then passes the result through an activation function (usually a step
function).
• Mathematical Representation:
• The Perceptron creates a linear decision boundary in the feature space, meaning it can
only solve problems where the classes are linearly separable.
• Bayes classifier is based on the “Average Risk Minimization” (minimizing average
misclassifications)
• Means: weighted sum of correct classification + weighted sum of misclassification.
Because it is 2- classifier problem,
C11, C12, C21, C22.
• What is our Objective?
When we want to take the likelihood ratio, we have to consider the following
equation
So, in this case what is going to be the decision boundary?
Is that the equation is linear????
• Fundamental condition for the perceptron is “pattern classes must be linearly
separable”.
• Bayes classifier for the gaussian distribution only it is taking the linear form
(linearly separable like perceptron). But otherwise, they are non-liner in
nature.
Differences between Perceptron and
Bayes classifier
Batch perceptron algorithm
• Perceptron algorithm works in an "online mode," it means that the
algorithm updates its weights incrementally, one training example at a
time, rather than processing all the training data at once.
• The Batch Perceptron refers to a variant of the Perceptron algorithm that
updates its weights based on the entire training dataset rather than on
individual examples
or
• Instead of adjusting the weight every time in an online manner, adjust
the weight at one short based upon the set of misclassified points.