SOFT COMPUTING

UNIT-1
DR. PALLAVI.M

Dr. Pallavi Malavath

 SYLLABUS
• UNIT-I
• Introduction to Soft Computing: Evolutionary
Computing, Hard Computing Vs Soft Computing,
• Soft Computing Methods, Recent Trends in Soft
Computing
• Foundations of Biological Neural Networks:
Model of Biological Neuron, Mathematical Model
of Neuron, ANN Architecture, Learning Rules,
Learning Paradigms, Perceptron Network
Dr. Pallavi Malavath
 INTRODUCTION
• Hard Computing: Hard computing uses traditional
mathematical methods to solve problems, such as algorithms
and mathematical models. It is based on deterministic and
precise calculations and is ideal for solving problems that
have well-defined mathematical solutions.
• Advantages
• Accurate solutions can be obtained
• Faster
• Disadvantages
• Not suitable for real world problems
• Cannot handle imprecision and partial truth
• Fusion of soft and hard computing techniques are also useful
in applications such as robotics.

Dr. Pallavi Malavath

• Hard computing involves the use of precise,
deterministic algorithms to solve complex
computational problems, particularly where
accuracy, performance, and predictability are critical.
It's often based on binary logic, mathematical
models, and conventional computational methods.
• Hard computing methods are widely used in
applications where strict accuracy is essential, as
opposed to soft computing approaches, which allow
for approximation and handling of uncertainty.

Dr. Pallavi Malavath

 APPLICATIONS
• Air Traffic Control (ATC)
• determines flight paths, sequencing landings, and
avoiding collisions.
• Ensures flight safety.
• Weather Forecasting and Climate Modeling
• predicts temperature, rainfall, and wind patterns.
• analyzes data from satellites, radar, and weather
stations.
• Accurate, real-time forecasts help in planning and
preparing for severe weather events, crucial for public
safety.

Dr. Pallavi Malavath

• Security and Surveillance Systems (e.g., Facial Recognition,
license plate recognition, and other security monitoring
applications.)
• Embedded Systems in Consumer Electronics (e.g., Mobile
Phones and Game Consoles)
• Medical Imaging and Diagnostic Equipment
(used in CT scans, MRI machines, and X-rays for image
reconstruction, noise reduction, and enhancement.
• Telecommunication Systems (e.g., Network Routing and
Traffic Management) data packets are transmitted through
the optimal path with minimal delay, preventing network
congestion and maintaining communication quality, essential
for real-time applications like video streaming and VoIP

Dr. Pallavi Malavath

 INTRODUCTION
• Soft computing is the reverse of hard (conventional)
computing. It refers to a group of computational techniques
that are based on artificial intelligence (AI) and natural
selection.
• It provides cost-effective solutions to the complex real-life
problems for which hard computing solution does not exist.
• Lotfi Zadeh of the University of California at Berkeley first
advanced the idea of fuzzy logic in the 1960s.
• The objective of soft computing is to provide precise
approximation and quick solutions for complex real life
problems.
Dr. Pallavi Malavath
 Soft computing techniques

Soft computing techniques:

• fuzzy logic
• neural networks
• genetic algorithms
• probabilistic reasoning.

Dr. Pallavi Malavath

 Applications
• Washing machines
Fuzzy logic can be used to regulate a washing machine's water
intake, wash time, spin speed, and washing process.
• Fraud Detection in Banking
Soft computing detects fraudulent activities in transactions by
identifying unusual patterns
• Smart Home Automation Systems
Soft computing controls various smart home features like lighting,
temperature control, and security systems based on fuzzy input.
• Robotics and Human-Robot Interaction
Soft computing methods are utilized in robots to interpret human
commands, navigate complex environments, and interact with
people.

Dr. Pallavi Malavath

• Autonomous Vehicles and Driver Assistance
Systems
• used in self-driving cars for tasks like object
recognition, decision-making in uncertain conditions,
and traffic patterns and road conditions.
• Image and Speech Recognition
• used in facial recognition, voice-activated assistants,
and other pattern recognition tasks.
• Medical Diagnosis and Decision Support Systems
• used in diagnostic systems to assess patient
symptoms, medical history, and other uncertain
factors to suggest possible diagnoses.
Dr. Pallavi Malavath
 Fuzzy Logic

• The term fuzzy refers to things that are not clear or

are vague.
• Sometimes, we cannot decide in real life that the
given problem or statement is either true or false.
• At that time, this concept provides many values
between the true and false and gives the flexibility
to find the best solution to that problem.

Dr. Pallavi Malavath

• Fuzzy logic contains the multiple logical values and
these values are the truth values of a problem between
0 and 1. This concept was introduced by Lofti
Zadeh in 1965 based on the Fuzzy Set Theory.
• In the Boolean system, only two possibilities (0 and 1)
exist, where 1 denotes the absolute truth value and 0
denotes the absolute false value. But in the fuzzy
system, there are multiple possibilities present
between the 0 and 1, which are partially false and
partially true.
• The Fuzzy logic can be implemented in systems such as
micro-controllers, It can also be implemented in both
hardware or software.
Dr. Pallavi Malavath
Difference between Boolean logic and fuzzy logic

Dr. Pallavi Malavath

 Evolutionary Computing
• Evolutionary computation is a subfield of artificial intelligence
(AI) and soft computing that uses algorithms to mimic
biological evolution to solve complex problems. It's used in
soft computing to help with data mining, robotics,
optimization, and engineering methods.
• It uses algorithms that simulate the principles of natural
selection, reproduction, and survival of the fittest to solve
complex problems.

Dr. Pallavi Malavath

Evolutionary Computing

Dr. Pallavi Malavath

Evolutionary Algorithms

Dr. Pallavi Malavath

 Hard computing Vs Soft computing
HARD COMPUTING SOFT COMPUTING
It uses Binary Logic It uses Fuzzy Logic
It can only deal with exact data It can deal with noisy data
It gives exact answers It gives approximate answers
It allows sequential computing It allows parallel computing
It is also known as conventional It is also known as computational
intelligence intelligence

It needs accuracy/precise It needs robustness

High computational cost Low computational cost
Low machine intelligence quotient(MIQ) High machine intelligence quotient(MIQ)

Dr. Pallavi Malavath

 Soft Computing Methods
• A soft computing method is an approach within
computational intelligence that enables
approximate, adaptive, and flexible solutions to
complex problems.
• Unlike traditional "hard" computing, which relies on
precise, deterministic algorithms, soft computing is
designed to handle uncertainty, imprecision, and
partial truths.

Dr. Pallavi Malavath

 Key Characteristics of Soft
Computing Methods
• Approximate Reasoning: They produce solutions that
may not be exact but are satisfactory and practically
useful.
• Tolerance for Uncertainty and Imprecision: Soft
computing methods accept inputs that are incomplete
and still yield reasonable results.
• Adaptability: Many methods are adaptive and can
improve performance over time by learning from data or
experience.
• Human-Like Decision-Making: They mimic aspects of
human cognition and can deal with problems that lack a
clear, deterministic solution.

Dr. Pallavi Malavath

 Soft Computing Methods
• Fuzzy Logic: Handles degrees of truth rather than binary
true/false, useful for control systems and decision-making.
• Neural Networks: Modeled after the brain, these networks
learn from data and are good at pattern recognition.
• Genetic Algorithms: Use evolutionary strategies for
optimization problems by simulating natural selection.
• Probabilistic Reasoning: Uses probability to make decisions in
uncertain environments, as in Bayesian networks.
• Rough Sets: Analyzes imprecise or incomplete data, useful in
data mining and classification.

Dr. Pallavi Malavath

 Artificial Neural Networking
• Artificial neural networks (ANNs) are a type of machine
learning model inspired by the human brain's structure
and function.
• They’re designed to recognize patterns, learn from data,
and make decisions based on that learning.
• ANNs are widely used in fields like image recognition,
natural language processing, and game-playing AI.
• Neural Networks are capable of learning by examples,
can remember their past experiences, perform parallel
processing, fault tolerance, and also adaptable to new
environment.

Dr. Pallavi Malavath

Biological Neuron

Dr. Pallavi Malavath

Artificial Neural Network

Dr. Pallavi Malavath

 Basic Structure of Artificial Neural
Networks
• The basic building block of an ANN is the neuron (or node),
which mimics a biological neuron. Neurons are arranged in
layers, and a typical ANN consists of three types of layers:
• Input Layer: Receives the raw data (e.g., an image’s pixel
values).
• Hidden Layers: These are intermediate layers that process
the inputs. There can be multiple hidden layers, making the
network "deep" (hence "deep learning").
• Output Layer: Produces the final output, like a classification
label or a prediction value.
• Each neuron in one layer is connected to neurons in the
next layer. These connections have weights that determine
the strength of the connection.

Dr. Pallavi Malavath

 Fuzzy Sets and Fuzzy Logic
• Fuzzy set theory is particularly designed to
mathematically handle uncertainty and vagueness.
• Fuzzy was the first successful tool that can handle
imprecise knowledge.
• The basic idea of fuzzy set is that it allows graded
membership of elements instead of bi-valued and
triple-valued logic.
• Fuzzy logic enables approximate reasoning
capabilities like humans, which provide strength to
handle uncertainties and vagueness.

Dr. Pallavi Malavath

 [Link] Sets
• The notion of fuzzy sets was introduced by L.A. Zadeh in the year 1965, in
an attempt to extend the notion of crisp sets and capture uncertanity
among data by the way of introducing the graded membership of
elements.

Dr. Pallavi Malavath

 [Link] logic
• Fuzzy logic uses fuzzy set theory as a basic tool.
• Basic mathematical ideas for fuzzy logic evolve from
the infinite-valued logic.
• Fuzzy logic can be considered as an extension of
infinite-valued-logic in the sense of incorporating fuzzy
sets and fuzzy relations into the system of infinite-
valued logic.

Dr. Pallavi Malavath

 Fuzzy logic
• Fuzzy logic focuses on linguistic variable in natural language
and aims to provide foundations for approximate reasoning
with imprecise propositions.
• Linguistic hedges are special linguistic terms by which other
linguistic are modified.
• Linguistic terms such as very, reasonably, quite, highly, and
lowly are examples of hedges.
• Example: The linguistic term “strong” can be modified by the
hedge “very "to generate “very strong "or the linguistic term
“tall "can be modified by using the hedge “quite "to generate
“quite tall”.

Dr. Pallavi Malavath

 [Link] Fuzzy Sets
• An intuitionistic fuzzy set is a fuzzy set extension where
decision makers can specify both the degree of membership
and non-membership of elements in the set, allowing for
uncertainty in decision-making processes.
• Intuitionistic fuzzy set (IFS) is very useful in providing a
flexible model to elaborate uncertainty and vagueness
involved in decision making.
• Fuzzy set theory helps make decisions when the information
isn’t clear or exact. It uses fuzzy logic, which allows us to work
with vague or uncertain data, like human opinions or
estimates.
• This makes it easier to handle real-life situations, so decisions
can be more adaptable and realistic.

Dr. Pallavi Malavath

Intuitionistic Fuzzy Sets

Dr. Pallavi Malavath

 [Link] Set Theory
• A rough set is a mathematical tool to deal with uncertainty
and vagueness in data analysis, introduced by Zdzisław Pawlak
in the early 1980s.
• It is part of soft computing and is particularly useful in
knowledge discovery, machine learning, and decision support
systems.
• The rough set comprises a combination of two sets: the lower
and upper approximation. The lower approximation is made
up of elements that definitely belong to the set, whereas the
upper approximation is composed of elements that possibly
belong to the set.

Dr. Pallavi Malavath

 [Link] Set Theory
• It focuses on classifying data into groups based
on available information, even if some details are
missing or unclear.
• Rough set theory helps us make sense of messy
or incomplete data by grouping similar pieces
together and highlighting what’s certain and
what’s uncertain.
• We can use a rough set approach to discover
structural relationships within imprecise and
noisy data.
Dr. Pallavi Malavath
Foundations of Biological Neural Networks:
Model of Biological Neuron
• Human brain receives signals from various sense
organs.
• Brain consists of huge number of neurons connected
through the axon and synapses.
• The signals between the links are electrochemical
signals that pass though the axon and synapses to
reach the neuron .
• The nucleus part of the neurons consists of soma to
which the filaments are attached called dendrites.
• Dendrites are the input channels through which
inputs from other neurons are received.
• Axons are the output channels of the neuron that
passes uniform electrical signals to other neurons via
terminal buttons.
• Terminal buttons are connected to other neurons by a
small gap called synapse that generate the signal that
may inhibit or exhibit the signals.
Functional units of Biological Neuron
• The neurons are able to
respond to the total number
of inputs aggregated within a
short time interval called
latent summation
• Neurons respond when the
membrane’s potential reaches
a certain threshold value and
hence fires.
• This neural structure helps the
human to learn and recall the
information when required.

Dr. Pallavi Malavath

 Mathematical model of Neuron
• Artificial neuron uses mathematical model that simulates the
behavior of the brain.
• Its functional units are similar to the biological model ie; it
receives signals from sensors, performs summation of inputs,
process the input data by applying activation function, and
checks with the threshold value to inhibit or excite.
• The signals are the weights that pass through the connection
between different nodes(neural units).

Dr. Pallavi Malavath

 Mathematical model of Neuron
• ANN models have at least two layers, the input layer and the processing
layer(or the output layer).
• The input layer receives the input from the sensors and passes the signal
to the processing layer.
• The input layer does not perform any processing.
• The processing layer receives the summation of inputs and applies
activation function.
• The output of the activation function is checked with the threshold value
which then performs inhibitory(weakens) or excitatory(strenghthens).
• This is the ouput signal of the ANN. The signals transferred are in the form
of weights which can be binary, bipolar, or real values.

Dr. Pallavi Malavath

Mathematical model of Neuron

Dr. Pallavi Malavath

Mathematical model of Neuron
 ANN Architecture
• ANN architecture is all about the arrangement of
neurons in layers and their interconnections within
or between the layers and nodes. The basic
categories of ANN architecture are:
• 1. Single-Layer feedforward
• 2. Multilayer feedforward
• 3. Competitive net
• 4. Recurrent network

Dr. Pallavi Malavath

Single-Layer Feedforward Network
• A single-layer feed forward network has directed synaptic
interconnections which carry weight from the input layer to
the output layer .
• Therefore it is called as feed forward network.
• Although it has 2 layers, it is named as single layer because
the neurons in the output layer alone can process the
information.
• The neurons in the input layer can receive the input from the
sensor and pass along its weight to the output layer.

Dr. Pallavi Malavath

Dr. Pallavi Malavath
 Multilayer Feedforward Network
• Since it has multiple processing layers, it is called Multilayer feedforward
network. It is a feedforword network with more than one intermediate
layers between the input and the output layers called the hidden layers.
• The hidden layers can have any number of processing neurons and they
are connected to their adjacent layers through synaptic interconnections.
• The input and the weights in the input layers are passed to the hidden
layers where they are processed and their output is feed as the input to
the next layer. This process continues in every layer until it reaches the
output layer.
• As the number of nodes or hidden layer increases the computation
increases, resulting in better accuracy.

Dr. Pallavi Malavath

Dr. Pallavi Malavath
 Competitive Net
• This is an unsupervised learning where all the output neurons are
involved in the competition of identifying the patterns.
• The neuron which is strongly activated will win.
• Hence, it is a competitive net.
• All the neurons on the output layers are connected for competition.
• Sometimes the neighborhood neurons alone are involved in competition.
• The patterns are clustered based on the similarity.
• The node with the highest similarity between the input patterns wins the
competition, and the weights of the corresponding node are adjusted.

Dr. Pallavi Malavath

Dr. Pallavi Malavath
 Recurrent Network
• The network with at least one feed back connection
is called as Recurrent network.
• In these networks, the output is fed as the input to
the network as the feedback.
• It may be a self-loop or may be loop between the
layers.
• The networks varies by the number of layers(single
or multiple) and their feedback connections.

Dr. Pallavi Malavath

Dr. Pallavi Malavath
 Learning Rules
• Learning rules in soft computing are the algorithms or
procedures used to update weights and improve the
performance of models based on training data.
• These rules play a critical role in adapting the model to
perform tasks such as classification, prediction, optimization,
and control.
• Learning in ANN happens through the synaptic weight
adjustment. This adjusted weight helps to learn different
patterns despite change of state.
• Learning rule is all about how the weights are adjusted for
different ANN networks.

Dr. Pallavi Malavath

 [Link] Learning Rule(Hebbian learning rule)
• This is the most famous weight adjustment method.
• The hebbian change in weight can be computed by finding the
cross product of input with the output and is defined by.
• The learning requires a small random value assigned to weight
initially say 0.
• If the cross product is positive, the weight increases otherwise
decreases.
• The hebbain learning is also known as correlation learning
since the output is strengthened for every input.
• Hebb rule is feedforward and can be applied to unsupervised
learning.

Dr. Pallavi Malavath

 [Link] Learning
• What is Perceptron?
• Perceptron is a type of neural network that performs
binary classification that maps input features to an
output decision, usually classifying data into one of two
categories, such as 0 or 1.
• Perceptron rule is the supervised learning of a neural
network. The learning signal can be computed by finding
the difference between the actual computed output and
the desired output(target).
• Let ‘n’ be the number of input patterns, xi be the i-th
input pattern and ti be the target of the i-th input. Then
the output yi can be computed by

Dr. Pallavi Malavath

Dr. Pallavi Malavath
 [Link] Learning Rule
• Delta rule can also be called as Widrow-Hoff rule or
least mean square(LMS) rule.
• According to the delta rule, the weight adjustment is
proportional to the product of error and input signal.
• The objective of this rule is to minimize the error
value. Therefore, if error increases, the weight is
increased, otherwise the weight decreases.

Dr. Pallavi Malavath

Delta rule for single output unit

Dr. Pallavi Malavath

Delta rule for multiple output units

Dr. Pallavi Malavath

 [Link] Take All learning rule
• Winner take all learning rule is also known as competitive
learning, where more than one neuron will compete with
each other in order to become active.
• the neuron with the maximum response for the given input
will win the competition.
• This principle is called Winner Takes All.
• Out of all units, only one unit is selected to become active at a
time.
• The output of winning unit is set to 1 and all the other units
will become 0.

Dr. Pallavi Malavath

•The output of winning unit is set to 1 and all the other units will
become 0.

Dr. Pallavi Malavath

 Outstar Learning Rule
• Outstar rule is well suited for the networks
with multiple layers.
• This rule is suitable for supervised learning.
• Although the weight calculation procedure is
same , it is only suitable for supervised
learning.

Dr. Pallavi Malavath

 Boltzmann Learning Rule
• This rule is applicable for Recurrent Network.
• The learning in this rule is characterized by
energy function.

• The neurons are taken at random to calculate

energy function.
• This type of learning is applicable in repetitive
network.
Dr. Pallavi Malavath
 Memory-Based Learning Rule
• This type of learning remembers the previous
experience of correctly learned patterns.
• The input and output mapping is recalled when
similar situation occurs.
• The learning rule is applied for the training in the
nearest neighborhood.
• This type of learning is used in most popular
technique called k- Nearest Neighborhood.

Dr. Pallavi Malavath

 Learning Paradigms
• Soft computing encompasses a set of methodologies that deal
with imprecision, uncertainty, and partial truth, mimicking the
human mind's decision-making process. In the context of
learning paradigms, soft computing employs various
approaches to train systems to adapt and generalize from
data. The primary learning paradigms in soft computing are:
• 1) Supervised Learning Algorithm( learn by example)
• 2) Unsupervised Learning (learn by experience)
• 3) Reinforcement Learning (Learn by analogy)

Dr. Pallavi Malavath

 Supervised Learning Algorithm
• The system learns from labeled examples, where the desired
output is provided for each input.
• In the learning process, the networks are trained with a set of
input patterns to produce the expected output.
• During the learning process, the network will be trained to
produce the output closer to the desired output.
• The learning process is repeated until the desired and the
actual output are closer to each other .
• The difference between the desired and the actual output is
called the error.
• Applications: Pattern recognition, speech recognition, and
predictive analytics.

Dr. Pallavi Malavath

• Key Techniques:
• Artificial Neural Networks (ANNs): E.g.,
Backpropagation for error correction.
• Fuzzy Logic-Based Systems: Incorporating expert-
labeled fuzzy sets and rules.
• Support Vector Machines (SVMs): Classification
and regression with soft margin optimization.
• Applications: Pattern recognition, speech
recognition, and predictive analytics.

Dr. Pallavi Malavath

 Unsupervised Learning
• The system learns from unlabeled data by identifying
patterns, structures, or features.
• In this method, the learning is based on this experience. The
target value is not known during the training. It learns by
discovering patterns by adapting various features.
• Key Techniques:
• Clustering Algorithms: E.g., k-means, Self-Organizing Maps
(SOM).
• Dimensionality Reduction: E.g., Principal Component Analysis
(PCA).
• Neuro-Fuzzy Systems: Adaptation without predefined labels.
• Applications: Market segmentation, anomaly detection, and
data compression.

Dr. Pallavi Malavath

 Reinforcement Learning
• Learning by interacting with an environment to achieve a
goal, guided by feedback in the form of rewards or
penalties.
• In this learning, the network will get a reinforce signal
that will guide the learning process.
• This reinforce signal can only direct whether the learning
is moving in the right direction or not.
• This doesn’t help to identify the error and rectify it,
rather it can guide by giving reward(positive signal) or
penalty(negative signal) when learning is correct or not.
• Applications:Robotics, game AI

Dr. Pallavi Malavath

 Perceptron Network
• Perceptron network is a supervised single-
layer feed forward network with three layers:
• Sensor unit
• Associator unit
• Response unit
• Sensor layer receives the input from the
environment and passes the input signal as 1’s
and 0’s to the network through the associator
unit.
Dr. Pallavi Malavath
• The output is received through response unit.
• The sensor unit consists of 400 photodetectors arranged in a
2-D matrix.
• These detectors are connected randomly to the associated
unit.
• They receive the input images and pass them as binary signal
to the associator unit.
• The electrical signal 1 is sent if the input signal exceeds the
threshold value, otherwise it sends 0.
• The associator unit has feature predicated that can detect the
specific feature from the input pattern.
• The response unit consists of pattern recognizers or
perceptron. The weights in the sensor and associate units are
fixed, and the weights in the response unit are adjustable.
Dr. Pallavi Malavath
Dr. Pallavi Malavath
Dr. Pallavi Malavath
 Perceptron Architecture
• Perceptron architecture uses perceptron learning rule.
• Binary activation function is used in the sensor and associator
units.
• Binary step function (output ->O or 1)with fixed threshold
value is used in the activation function of associator unit.
• Bipolar activation (output -1 or 1) is used between the
associator and the response unit and it uses perceptron
learning rule for weight updation.
• The error is computed by comparing the target value and the
calculated value.
• The weight updation is done if there is any error, otherwise
the weights remains the same.
Dr. Pallavi Malavath
Single layer Perceptron Architecture

Dr. Pallavi Malavath

Multi layer Perceptron Architecture

Dr. Pallavi Malavath

Soft Computing
UNIT -2

Dr. Pallavi Malavath

 Syllabus
• Feed Forward Neural Network: Back
Propagation Network, Parameter Selection in
BPN, Local Minima and Global Minima, Merits
and Demerits of Back Propagation, Variants of
Back Propagation, Applications of BPN, Radial
Basis Function
 [Link] Forward Neural Network
• What is a Feedforward Neural Network?
• A Feedforward Neural Network (FNN) is a type of
artificial neural network where connections between
the nodes do not form cycles. This characteristic
differentiates it from recurrent neural networks
(RNNs).
• The network consists of an input layer, one or more
hidden layers, and an output layer. Information
flows in one direction—from input to output—hence
the name "feedforward."
Feed Forward Neural Network
 Feed Forward Neural Network
• In the feed back network, the output is fed as input
and thus it forms a cycle.
• In feed forward network, the information flows in
one direction from the input node to the hidden
node.
• The synaptic links carrying weights are connected in
unidirectional flow in the forward direction.
• During the learning process, there is no backward
movement of signals between the layers.
• This network doesn’t have any loops.
 [Link] Propagation Network
• BPN is capable of handling non-linear separable problems
with high mathematical foundation.
• It is a popular feed forward network that could compute
weight adjustment based on the error term.
• The change in the weight is computed based on LMS(least
mean square) that minimizes the error In each of the
iterations.
• It uses Gradient Descent (derivative of error) for modification
of weight, which progresses in shifting of error location,
thereby avoiding more oscillation in convergence.
Back Propagation Network
 Back Propagation Network
• The network recognizes the given input pattern by
supervised learning method. The network is trained
to learn the patterns so that it responds correctly to
the one that are similar to the input patterns.
• The aim of this algorithm is to obtain stability in
identifying similar patterns. The network is tuned to
learn the pattern at the training phase, and the
learning ability of the system is identified at the
testing phase.
 2.1 Training and Testing
• Back propagation alg comes under supervised learning. In this
input patterns along with their outputs are fed to the network
for learning.
• This phase of alg is called training.
• It facilitates the network to learn the pattern based on the
similarity between the given inputs and the existing
knowledge.
• The error is calculated based on the distance between the
new pattern and the accumulated knowledge and can be
reduced by adjusting the weights.
• Now the network is tuned to recognize the new pattern other
than the available in the training set is called learning.
 2.1 Training and Testing
• The learning accuracy is estimated based on the networks
response to the test pattern.
• In general, 70% of the dataset is used as training set,
remaining 30% of the dataset is used as test dataset.
• To get statistically meaningful results, cross-validation
methods is applied called 10-fold cross validation.
• One portion is held out for testing and remaining 9 portions
are used for training purpose.
• This procedure is repeated 10times such that partition is used
once for testing.
• The average error of 10 iterations is the final error
 2.2Delta rule
• Delta rule
• 2.3 Back propagation Training Algorithm
• 2.4 Example problems for BPN

• Notes for these topics are

attached at the end
 Parameter Selection in BPN
• Epoch
• Batch size.
• Convergence
• Learning rate
• Momentum Factor
• Local Minima and Global Minima
• Under fitting and over Fitting
• Step size.
• Epoch: One full pass of the entire training
dataset.
 Batch Size
• How It Works
• In each training iteration, the model processes a
subset of the dataset called a batch.
• After processing the batch, the loss is computed, and
gradients are used to update the weights.
• Once all batches in the dataset are processed, it
completes one epoch.
• Common batch sizes: 32, 64, 128, or [Link] with
32 or 64 for a balance of speed and performance
• Convergence
• Convergence refers to the point during training when the
neural network's performance stabilizes, and further training
does not significantly reduce the loss or improve accuracy. It
indicates that the model has found an optimal (or near-
optimal) set of weights that minimizes the loss function.
• The model's parameters (weights and biases) are adjusted
iteratively using gradient descent. As training progresses, the
loss decreases, and the gradients become smaller.
• Convergence occurs when the loss stops decreasing or
decreases very slowly.
• The model's performance on a validation set stabilizes.
 Indicators of Convergence
• Training Loss and Validation Loss:
– Loss becomes stable and does not decrease further.
• Gradients:
– Gradients approach zero (indicating the minimum of
the loss function).
• Model Performance:
– Accuracy or another performance metric stabilizes on
the validation/test set.
• No Significant Weight Updates:
– Weight updates become very small.
• Learning Rate
• The learning rate (denoted as η\eta or α\alpha) is a
hyperparameter that controls the step size at which
the neural network updates its weights during training.
It determines how quickly or slowly the model learns.
• Choosing the Right Learning Rate
• A small learning rate (e.g., 0.0001):
– Leads to slow convergence.
– May get stuck in local minima.
• A large learning rate (e.g., 1.0):
– Causes the model to overshoot the minimum.
– May result in unstable training or prevent convergence.
• Momentum Factor:
• The momentum factor helps speed up the training of a neural
network by smoothing out weight updates. Instead of using
only the current gradient to update weights, momentum also
considers the previous updates. This helps the model move
faster in the right direction and reduces unnecessary
zigzagging.
• Momentum helps make the updates smooth and faster to
reach the minimum loss.
• Momentum is like a "push" that helps the model keep moving in
the same direction as before.
• It makes training faster. It prevents the model from oscillating or
getting stuck.
• Momentum is usually set to 0.9. This means 90% of the previous
update is carried forward.
• Step Size: In deep learning, step size (often referred to as the learning rate)
is a crucial hyper parameter that controls how much the model weights are
adjusted during each iteration of the optimization process.
• The step size determines the size of the steps taken in the direction of the
gradient during optimization. A larger step size results in larger updates to
the model parameters, while a smaller step size results in smaller updates.
• Under Fitting & Over Fitting:
Over fitting and under fitting are two
common problems in machine learning that
affect the performance of models.
• Over fitting occurs when a model learns not
only the underlying patterns in the training
data but also the noise and random
fluctuations.
• High accuracy on training data.
• Poor performance on validation/test data.
• Under fitting occurs when a model is too
simple to capture the underlying patterns in
the data. It performs poorly on both training
and unseen data.
• Low accuracy on both training and
validation/test data.
• Model fails to capture the complexity of the
data.
 Parameter selection in BPN
• For best results we need to fix the parameters
carefully.
• Initial weights: initial weights are randomly
chosen. If the initial weights are high, it leads to
the faster convergence. If it is low, the
convergence is too late.
• Also, high weight will push the system to local
minima. Choosing of initial weight has a great
impact over convergence . Hence, an optimal
selection of initial weights is essential.
• The weights can be updated after
computation of the output of every pattern or
weights can be updated after all the patterns
are presented for the output calculation i.e.;
after one epoch called the batch mode.
• Learning Rate: It determines the change of weight which in turn
affects the rate of convergence.
• Poor selection of learning rate value ends up with more oscillation
 Local Minima and Global Minima

• Minima and Maxima are the extreme values i.e.; the

smallest and the largest value, respectively, found in
the given neighborhood or may be in the entire set.
• When training a neural network, the goal is to
minimize the loss function. The terms local minima
and global minima describe different points in the
loss surface where the loss value is low.
• Local minima and local maxima: it is the smallest values
compared to the neighborhood values. Similarly, the
maximum value, called, called local maxima, compared to the
neighbors.
• Global minima and global maxima: the smallest value and the
largest value in the entire universe is called global minima or
global maxima
• The back propagation algorithm in NN gets stuck into the local
minima. The occurrence of such state is due to the
disharmony in weight adjustments between the hidden and
output layers.
• Momentum Factor:
• To avoid oscillation to extreme values duriong
learning , a small constant (momentum
value)is added to the gradient descent . Kt
smoothens the descent path toward the
positive direction and also improves the
convergence rate.
• Generally, the value 0.5-0.9 is used that allows
faster learning rate without much oscillation.
Example for Momentum
• Number of Hidden layer Nodes:
• Generally, the number of nodes in the hidden layer will
be between the number of nodes in the input layer and
the output layer.
• Too few neurons: Model underfits and fails to capture
the patterns.
• Too many neurons: Model overfits and generalizes
poorly.
• Start with a reasonable number (e.g., between input
size and output size).
• Use rules of thumb, such as "2/3 the size of input layer
+ output layer".
• Tune using cross-validation or tools like Grid
Search/Random Search.
 Merits and Demerits
• Merits:
• Easy to implement.
• It doesn’t require any knowledge on special features,
only mathematical formula is applied for learning
and can be applied to any network.
• Convergence of the learning process will be faster if
the initial weights are set to the minimum value.
• Demerits: Every time the results may vary, hence it
is fuzzy output or non-numeric output.
• It may get stuck into local minima.
 Variants of Back Propagation
• 1) Back Propagation with Genetic Algorithm
• Genetic algorithm is combined with back propagation in
training phase to improve the efficiency of classification.
• Back propagation is a gradient-based method that updates
network weights by minimizing error through iterative
adjustments using the chain rule.
• It is efficient but can struggle with highly non-linear search
spaces or getting stuck in local minima.
• Genetic Algorithms are inspired by the principles of natural
evolution and work by generating a population of potential
solutions, evaluating their performance, and using
operations like selection, crossover, and mutation to evolve
better solutions over generations.
2) Adaptive Back Propagation Algorithm:
• The parameter that influence the learning during the
training phase of back propagation is momentum
and learning rate.
• It should be carefully chosen so that the learning rate
is neither too fast nor too slow.
• The faster learning will not produce global optimal
solution and has higher chance of getting stuck into
the local optima.
• To overcome these drawbacks, the momentum ,
learning rate should be altered
dynamically(Adaptively)
• The learning rate can be set to higher value, if it is
far away from global optima.
• Similarly, it can be minimized if it is closer to the
global optima, so that it will not cross the optimal
solution.
• Momentum can be dynamically fixed so that more
oscillation in gradient descent can be avoided.
• The aim is to smoothen the descent path so that it
converges fast.
• Another adaptive procedure is to fix the optimal number of
hidden nodes.
• If there are more number of hidden nodes, the
computational completely increases and the network is over
trained .
• If there are less number of hidden nodes, the network is
under trained to recognize the pattern correctly.
• Optimal number of nodes in the hidden layer can be fixed by
adding one node at a time for one epoch.
• When the results are good, stop adding the nodes and fix the
weights and network architecture for testing.
3) Back Propagation with Fuzzy Technique:
• It integrates the gradient-based optimization of back propagation
and handling of uncertainty offered by fuzzy logic.
• This hybrid approach is particularly beneficial in problems where
data is noisy, vague, or uncertain, and traditional neural networks
may struggle to deliver robust performance.
• Back propagation with fuzzy techniques combines the gradient-
based optimization of neural networks with the interpretability
and uncertainty-handling capabilities of fuzzy logic.
• Back propagation, widely used for training neural networks, adjusts
weights to minimize prediction errors but often struggles with
noisy, uncertain, or imprecise data.
• Fuzzy logic addresses this by representing inputs and
outputs as degrees of membership to linguistic
categories like "low," "medium," or "high," enabling
the system to process uncertainty effectively.
• This hybrid approach can be implemented in two ways:
fuzzy neural networks (FNNs), where fuzzy principles
are embedded directly into the network and optimized
using back propagation, or neural network-fuzzy
systems, where fuzzy logic preprocesses inputs or
interprets outputs from the neural network.
 Applications of BPN
Image processing
Optical character recognition
Face Recognition
Forecasting- Weather, stock Market.
Pollution Prediction
Handwriting Recognition.
Medical diagnosis
Fault Detection
 Radial Basis Function
• Radial Basis Function (RBF) Neural Networks are a
specialized type of Artificial Neural Network (ANN) .
• RBF Networks offer faster learning speeds and efficient
performance in classification and regression problems.
• What are Radial Basis Functions?
• Radial Basis Functions (RBFs) are a special category of feed-
forward neural networks comprising three layers:
• Input Layer: Receives input data and passes it to the hidden
layer.
• Hidden Layer: The core computational layer where RBF
neurons process the data.
• Output Layer: Produces the network’s predictions, suitable
for classification or regression tasks.
 How Do RBF Networks Work?
• RBF Networks are conceptually similar to K-Nearest
Neighbor (k-NN) models, though their implementation is
distinct. The fundamental idea is that an item's predicted
target value is influenced by nearby items with similar
predictor variable values. Here’s how RBF Networks
operate:
• Input Vector: The network receives an n-dimensional
input vector that needs classification or regression.
• RBF Neurons: Each neuron in the hidden layer represents
a prototype vector from the training set. The network
computes the Euclidean distance between the input
vector and each neuron's center.
• Activation Function: The Euclidean distance is
transformed using a Radial Basis Function (typically a
Gaussian function) to compute the neuron’s
activation value. This value decreases exponentially
as the distance increases.
• Output Nodes: Each output node calculates a score
based on a weighted sum of the activation values
from all RBF neurons. For classification, the category
with the highest score is chosen.
Radial Basis Function
Different RBFs
Difference between RBF and
MLP(Multi layer Perceptron)
 Applications of RBF
• Face Recognition.
• ECG analysis.
• Modeling a 3D shape.
• Diagnosis of diabetes .
• Currency exchange rates forecasting.
• Network Security.
Soft Computing
Unit -3
Dr. Pallavi Malavath

Dr. Pallavi Malavath

 Syllabus
• Associative Memory: Auto associative
Memory, Hetero-associative Memory,
Bidirectional Associative Network,
Applications of Associative Memory
• Associate Models: Hopfield Network,
Boltzmann Network, Simulated Annealing,
Applications of Networks

Dr. Pallavi Malavath

 Associative Memory
• Associative memory is a pattern warehouse that stores
a set of patterns as memories.
• The network will recall the pattern associated with it
from the memory when an input pattern is given.
• The pattern may be the same as the key pattern or it
can be a different pattern.
• On the basis of pattern retrieval, the associative
network can be classified into auto associative memory
and Hetero associative memory also known as auto-
correlators and hetero-correlators.

Dr. Pallavi Malavath

• These type of memories are called content
addressable memories(CAM).
• The input pattern is correlated to the patterns stored
in the memory.
• The correlation helps to retrieve the pattern closer to
the input pattern .
• The memory should consist of unique patterns so
that there may not be any ambiguity in the retrieval
of the patterns

Dr. Pallavi Malavath

• This is Hetero-
associative memory
because input pattern x
is different from the
recall pattern y.
• If x and y are same,
then the model is an
auto-associative
memory.

Dr. Pallavi Malavath

• These models use feed forward network or
recurrent network and are called as neural
associative network.
• The selection of network depends on the
association capability.
• Hebbian rules are used for learning.
• Auto correlators are useful for the retrieval of
images even when they are distorted( noisy).

Dr. Pallavi Malavath

Dr. Pallavi Malavath
 Auto-associative Memory
• Associative net uses Hebb rule, which is the product of input
xi and output yi or an outer product rule, which is an
alternate method of Hebb rule.
• These associative net has same number of neurons in the
input and output layers as the training and the target output
vectors are same.
• The association between the patterns is stored by their
weights.
• The performance of the net is based on its recall of patterns
from the noisy input.
• The initial weights are set to ‘0’.

Dr. Pallavi Malavath

Dr. Pallavi Malavath
Dr. Pallavi Malavath
Dr. Pallavi Malavath
 Hetero-associative Memory
• Hetero-associative Memory retrieves the
pattern from the memory where the input
and the target patterns are different.
• These networks are trained using the Hebb or
delta rule.
• The correlation between the input pattern and
the output pattern is determined by the
weights.

Dr. Pallavi Malavath

• These networks respond well in noisy and
incomplete patterns.
• The input layer has n nodes and the output layer has
m nodes representing the patterns.
• The interconnection between the nodes in the layers
are linked by weights that are defined using the Hebb
rule.
• The training algorithm is similar to the auto
associative memory.

Dr. Pallavi Malavath

Dr. Pallavi Malavath
 Bidirectional Associative Memory
(BAM)
• Bidirectional Associative Memory (BAM) is
a supervised learning model in Artificial Neural
Network.
• This is hetero-associative memory, for an input
pattern, it returns another pattern which is
potentially of a different size. This phenomenon is
very similar to the human brain.
• A Recurrent Neural Network (RNN) is needed to
receive a pattern of one set of neurons as an
input and generate a related, but different,
output pattern of another set of neurons.

Dr. Pallavi Malavath

 Why BAM is required?
• The main objective to introduce such a network
model is to store hetero-associative pattern pairs.
This is used to retrieve a pattern given a noisy or
incomplete pattern.
• BAM Architecture: When BAM accepts an input of n-
dimensional vector X from set A then the model
recalls m-dimensional vector Y from set B. Similarly
when Y is treated as input, the BAM recalls X.

Dr. Pallavi Malavath

BAM Architecture

Dr. Pallavi Malavath

• BAM can be classified into 3 types:
• 1) Discrete BAM
• 2) Continuous BAM
• 3) Exponential BAM
• The BAM net resembles the single layer
perceptron network and the interconnection
between the layers in the net is bi-
directional.

Dr. Pallavi Malavath

• Signals can be sent from the layer X to layer Y with W
as the weight matrix and from the layer Y to layer X
with the weight
. matrix Wt.
• The layers are not called as the input and output
layers as the input/output pattern can be given at
the either side.
• During the training process, the weights are adjusted
in both forward and backward movement of signals
and this oscillation will converge when they reach
the equilibrium

Dr. Pallavi Malavath

Dr. Pallavi Malavath
Dr. Pallavi Malavath
• A step function is used as an activation
function.
• The bipolar value improves the performance
of Hebbain net.
• Activation function at layer Y:

Dr. Pallavi Malavath

• This signal is send to layer X.
• Activation function at layer X:

Dr. Pallavi Malavath

 Continuous BAM
• Continuous Bidirectional Associative Memory (BAM)
is an extension of the original BAM model, which is
a type of neural network designed to associate
patterns bi-directionally.
• Continuous BAM is particularly useful for
applications where the input and output patterns
are represented by continuous values rather than
binary or discrete ones.

Dr. Pallavi Malavath

Dr. Pallavi Malavath
 Features of Continuous BAM
• Bidirectional Mapping: The network learns
associations in both directions between two
pattern sets, X and Y. Once trained, it can
retrieve Y given X, or vice versa.
• Continuous Activation Functions: Unlike the
traditional BAM, which uses binary thresholds,
the continuous version employs smooth
activation functions, such as sigmoid or
hyperbolic tangent, to handle real-valued
inputs and outputs.
Dr. Pallavi Malavath
• Energy Function: Continuous BAM networks
define an energy function that decreases
during the association process.
• This guarantees convergence to a stable state
making the network suitable for memory
retrieval tasks.

Dr. Pallavi Malavath

• Learning Rule: The associative weights W
between X and Y are updated using a
generalized Hebbian learning rule:

•Feedback Dynamics: The continuous BAM

uses feedback loops to iteratively refine the
states of neurons in the X-layer and Y-layer
until they converge to stable patterns.

Dr. Pallavi Malavath

 Applications
• Continuous BAM networks are applied in
fields where continuous-valued pattern
associations are necessary, such as:
• Signal processing
• Control systems
• Pattern recognition
• Data compression
• Robotics

Dr. Pallavi Malavath

 Exponential Bidirectional Associative
Memory
• Exponential Bidirectional Associative Memory
(EBAM) is a neural network model that
extends the traditional Bidirectional
Associative Memory (BAM).
• It uses an exponential function to encode and
decode associative patterns, enhancing the
network's ability to store and retrieve
information efficiently.

Dr. Pallavi Malavath

 Features of EBAM:
• Bidirectional Mapping: EBAM, like BAM, supports
two-way (bidirectional) associations between input
patterns (X) and output patterns (Y). It allows
retrieving X from Y and vice versa.
• Exponential Function: The core enhancement in
EBAM is its use of an exponential activation function.
This improves the network's ability to handle non-
linear associations, increasing its capacity and
robustness in pattern recognition.

Dr. Pallavi Malavath

• Energy-Based Model: Similar to BAM, EBAM is often
described in terms of an energy function. The exponential
function helps minimize this energy during learning, leading
to faster convergence and improved stability.
• Learning Rule: The Hebbian learning rule (or its variants) is
typically used to train the associative weights. In EBAM, the
weights (WWW) are updated to strengthen the relationship
between co-occurring patterns:

Dr. Pallavi Malavath

• Storage Capacity: The exponential modification
enhances storage capacity compared to
traditional BAM, allowing EBAM to store and
retrieve a larger number of patterns.
• Applications: EBAM is particularly useful in fields
like:
– Pattern recognition
– Associative memory systems
– Data classification and clustering
– Signal processing

Dr. Pallavi Malavath

• Advantages of EBAM:
• Improved Accuracy: Handles complex and
non-linear relationships better than BAM.
• Faster Convergence: The exponential
dynamics allow quicker stabilization during
retrieval.
• Increased Capacity: Capable of storing a larger
number of associative patterns.

Dr. Pallavi Malavath

 Applications of Associative Memory
• Character recognition.
• Fault detection.
• Categorization and classification problems.
• Feature extraction
• Analysis of document images and identification of
object within the image.
• Detection analogies in a structural safety
management.
• Design for VLSI implementation
Dr. Pallavi Malavath
 Hopfield Network
• A Hopfield Network is a type of recurrent
artificial neural network that serves as an
associative memory system.
• It is generally used in performing auto-
association and optimization tasks. It is
calculated using a converging interactive
process and it generates a different response
than our normal neural nets.

Dr. Pallavi Malavath

• Discrete Hopfield Network
• It is a fully interconnected neural network where each
unit is connected to every other unit. It behaves in a
discrete manner, i.e. it gives finite distinct output,
generally of two types:
• Binary (0/1)
• Bipolar (-1/1)
• The weights associated with this network are symmetric
in nature and have the following properties.
• wij=wji
• wii=0

Dr. Pallavi Malavath

 Structure & Architecture of Hopfield Network

• Each neuron has an inverting and a non-

inverting output.
• Being fully connected, the output of each
neuron is an input to all other neurons but not
the self.

Dr. Pallavi Malavath

Dr. Pallavi Malavath
 Boltzmann Network
• What is a Boltzmann machine?
• A Boltzmann machine is a type of stochastic neural
network that can learn complex data distributions
through its network of interconnected nodes.
• How does a Boltzmann machine work?
• It works by using a learning algorithm to adjust the
weights of the connections, optimizing the network
to represent complex patterns in data through
probability distribution.

Dr. Pallavi Malavath

Dr. Pallavi Malavath
 Overview of Boltzmann Machines
• Introduced by Geoffrey Hinton and Terry
Sejnowski in 1985.
• Uses a network of neurons (or units) that are
symmetrically connected.
• The network follows an energy-based model
inspired by the Boltzmann distribution in
thermodynamics.
• It operates using stochastic (randomized)
updates to find optimal solutions.
Dr. Pallavi Malavath
 Structure of a Boltzmann Machine
A Boltzmann Machine consists of:
• Visible Units (V): Represent the input and output of
the system.
• Hidden Units (H): Used to capture dependencies and
patterns in the data.
• Bidirectional Weighted Connections: Every unit is
connected to others (fully connected network).
• Energy Function (E): The probability of a state is
determined by its energy level.

Dr. Pallavi Malavath

Dr. Pallavi Malavath
 Learning in Boltzmann Machines
• Learning occurs through adjusting weights to
minimize energy and maximize the likelihood
of observed data.
• Uses Gibbs Sampling and Contrastive
Divergence (CD) for training.
• The network updates neuron states using the
sigmoid activation function.

Dr. Pallavi Malavath

 Applications in Soft Computing
• Optimization Problems (e.g., Travelling
Salesman Problem, Job Scheduling).
• Feature Learning & Dimensionality
Reduction.
• Image Recognition & Processing.
• Natural Language Processing (NLP).
• Recommendation Systems.

Dr. Pallavi Malavath

 Pros & Cons of Boltzmann Machines (BMs)
• Pros (Advantages)
• Efficient in Optimization – Useful for solving
complex combinatorial problems(computationally
difficult- Traveling Salesman Problem, Sudoku &
Puzzles, Knapsack Problem)
• Supports Unsupervised Learning – Can learn
patterns without labeled data.
• Probabilistic Model – Handles uncertainty and
stochastic learning well.
• Good for High-Dimensional Data – Effective in
feature extraction & dimensionality reduction.

Dr. Pallavi Malavath

• Cons (Disadvantages)
• High Computational Cost – Full BMs require
exponential time to train.
• Slow Convergence – Training with Gibbs
Sampling is inefficient.
• Difficult to Train – Gradient calculations are
complex and unstable.
• Requires Large Datasets – Needs significant
data to perform well.

Dr. Pallavi Malavath

 Simulated Annealing
• What is Simulated Annealing?
• Simulated Annealing (SA) is a probabilistic
optimization algorithm inspired by the annealing
process in metallurgy, where materials are heated
and then cooled slowly to remove defects and
achieve a stable structure.
• In computing, SA is used to find near-optimal
solutions for difficult combinatorial and optimization
problems.

Dr. Pallavi Malavath

Simulated Annealing

Dr. Pallavi Malavath

• SA searches for the global optimum by
allowing occasional moves to worse solutions
to escape local optima.
• It gradually reduces the probability of
accepting worse solutions as the
"temperature" decreases.

Dr. Pallavi Malavath

 How Simulated Annealing Works?

• Initialize: Start with an initial solution and set

a high temperature (T).
• Generate New Solution: Make a small random
change to the current solution.
• Evaluate the Change:
– If the new solution is better, accept it.

Dr. Pallavi Malavath

• If the new solution is worse, accept it with a
probability

•Cool Down: Gradually decrease the temperature T

(e.g., using T=0.9TT = 0.9TT=0.9T).
•Repeat Steps 2–4 until T ≈ 0 or a stopping criterion is
met.

Dr. Pallavi Malavath

 Applications of Associative Memory
• Recalling and Reconstruct corrupted patterns.
• Large-Scale computational intelligence
systems.
• Handwriting recognition software.
• Medical image processing.
• Classification and prediction of data analysis.
• VLSI design, Routing in Network etc…

Dr. Pallavi Malavath

 UNIT -4
By
Dr. Pallavi Malavath
 Syllabus
• Fuzzy Logic: Basic concepts, Fuzzy set theory, Basic
operations, Fuzzification and Defuzzification, Fuzzy
decision making Particle Swarm Optimization,
Genetic Algorithms
• Rough Set Theory: Rough Sets, Differences between
Rough sets and fuzzy sets, Decision Tables, Rule
Induction, Discernibility Matrix
 Crisp vs. Fuzzy Logic

• Crisp Logic (Classical Logic): Values are strictly

true (1) or false (0).
– Example: "The water is hot" → Either True (hot) or
False (not hot).
• Fuzzy Logic: Values range between 0 and 1,
representing degrees of truth.
– Example: "The water is warm" → It could be 0.6
hot and 0.4 cold.
• Membership Functions (MF)
• A Membership Function assigns a degree of belonging (0
to 1) to each input in a fuzzy set.
• Fuzzy Rules (IF-THEN Rules)
• Fuzzy logic systems use IF-THEN rules to make decisions.
• Example: Temperature Control System
• Rule 1: IF temperature is cold, THEN turn heater on high.
• Rule 2: IF temperature is warm, THEN turn heater on
low.
• Rule 3: IF temperature is hot, THEN turn heater off.
Fuzzy set theory
 Basic Fuzzy operations
• Fuzzy operations are performed on fuzzy sets,
whereas crisp operations are performed
on crisp sets. Fuzzy operations are very useful
in the design of a Fuzzy Logic Controller.
• The three basic operations are complement,
intersection, union.
 Fuzzification and Defuzzification.
• Fuzzy logic is widely used in systems where decisions
are made under uncertainty. The two key processes
in fuzzy logic systems are Fuzzification and
Defuzzification.
1. Fuzzification (Crisp to Fuzzy Conversion)
• Fuzzification is the process of converting a crisp
(precise) input into a fuzzy value. This allows
systems to handle imprecise and uncertain data.
• Steps in Fuzzification:
• Identify input variables – Determine the crisp
numerical values that need to be fuzzified.
• Define fuzzy sets (membership functions) –
Represent different linguistic terms such as "Low,"
"Medium," and "High" using membership functions.
• Assign membership values – Each crisp input value is
mapped to a fuzzy set based on its degree of
belonging.
• Example of Fuzzification:
• Imagine a temperature control system where:
• Crisp Input: Temperature = 35°C
• Fuzzy Sets:
– "Cold" (0 to 25°C)
– "Warm" (20 to 40°C)
– "Hot" (30 to 50°C)
• Using a triangular membership function, 35°C belongs:
• 0.25 to "Warm“, 0.75 to "Hot“
 Defuzzification (Fuzzy to Crisp Conversion)
• Defuzzification is the process of converting fuzzy outputs into a
single crisp value, which can be used as a control action in real-
world applications.
• Methods of Defuzzification:
• Centroid Method (Center of Gravity –COG).
• Max Membership Principle (Mean of Maximum - MOM)- takes the
value with the highest membership.
• Weighted Average Method- Averages the fuzzy values weighted by
their membership.
• Mean max membership.
• Center of Sums
• Center of largest area.
• First(or last) of maxima
• [Link] membership principle:
• This is also known as height method. In this
case the defuzzified value is obtained as the
element having the highest membership value
in the domain
• [Link] method: This procedure (also
called of area or the center of gravity) is the
mostly used method of all the defuzzification
methods.
• [Link] average method:
• This is most widely used and computationally
efficient defuzzification methods, it is only
applicable to the symmetrical output
membership functions.
• It takes the weighted arithmetic mean of the
elements in the domain, where the weights are
the membership values of the elements .
• [Link] max membership:
• This is the generalized version of the first
method. Instead of a single point, the
maximum membership may occur at a set of
points forming a plateau in the graph of the
output function.
• This method is also called middle-of-maxima
• [Link] of sums:
• This method is faster and it is not restricted to
symmetric membership functions..
• The computation of the centroids of the
individual membership functions is required in
this process given in the following equation.
6. Center of largest area:
• If the output fuzzy set has at least two convex
sub-regions, obtain the defuzzified value “a”
of the output .
• [Link](or last) of maxima:
In this method the domain Ci with the
maximized membership degree among the
output fuzzy sets Ck, k=1,2…….n, is
determined by using the overall output, which
is nothing but the union of all the individual
fuzzy sets
 Rough Set Theory
(for more info refer 1st unit also)
• Rough Set Theory is a method for dealing with
uncertain data by defining sets with clear and possible
members.
• Rough Set Theory (RST) is a mathematical approach
used to deal with incompleteness in data analysis. It
was introduced by Zdzisław Pawlak in the early 1980s.
• The main advantage of rough set theory in data
analysis is that it does not need any preliminary or
additional information about data − like probability in
statistics.
The proposed approach
•provides efficient algorithms for finding hidden
patterns in data,
•finds minimal sets of data (data reduction),
•evaluates significance of data,
•generates sets of decision rules from data,
•it is easy to understand,
•offers straightforward interpretation of obtained
results,
•most algorithms based on the rough set theory are
particularly suited for parallel processing.
Rough sets consists of:
• Information System: A dataset represented as a table
where rows correspond to objects and columns to
attributes.
• Indiscernibility Relation: Objects with the same values
for a subset of attributes are considered indiscernible.
• Lower and Upper Approximation:
– Lower Approximation: The set of objects that definitely
belong to a concept.
– Upper Approximation: The set of objects that possibly
belong to a concept.
• Boundary Region: The difference between upper and
lower approximations, representing uncertainty.
• Reducts: The minimal subset of attributes preserving
the classification capability of the original dataset.
 Differences between Rough sets and
fuzzy sets
• Rough sets and fuzzy sets are both mathematical tools
used for dealing with uncertainty and imprecision in
data, but they differ fundamentally in their approaches
and applications.
• Example:
• Rough Set Approach: Suppose we are classifying emails
as spam or not spam. If some emails have unclear
classification, they fall into the boundary region of the
rough set.
• Fuzzy Set Approach: In a fuzzy system, an email could be
60% spam and 40% not spam, based on membership
values.
Data Requirements Works well with Requires a predefined
categorical data and membership function
does not require for each fuzzy set.
additional information
like a membership
function.
Application Areas Used in machine Used in control systems,
learning, feature decision-making, and
selection, and approximate reasoning
knowledge discovery where gradual
where uncertainty arises transitions are needed.
due to incomplete
information.
 Differences Between Rough Sets and
Fuzzy Sets
FEATURE ROUGH SETS FUZZY SETS
Based on approximation Based on degrees of
Concept of sets using lower and membership in a set.
upper bounds.
Handling Uncertainty Deals with vagueness by Deals with vagueness by
using boundary regions allowing partial
(lower and upper membership of
approximations). elements.
Membership Membership is crisp Membership is gradual,
(either belongs or does ranging from 0 to 1.
not belong to lower or
upper approximation).
 Decision Table in Rough Set Theory

• A decision table is a tabular representation of

rules or conditions used for decision-making.
In soft computing, decision tables are often
used in rough set theory to extract patterns,
make classifications, and reduce data
dimensionality.
 Components of a Decision Table:

• Condition Attributes – Features or criteria

that influence decision-making.
• Decision Attributes – The outcome or
classification based on condition attributes.
• Objects (Rows) – Instances or cases described
by condition attributes.
• Decision Rules – If-then rules derived from
the table.
 Decision Table in Rough Set Theory:
• Lower Approximation: Objects that definitely belong
to a decision class.
• Upper Approximation: Objects that possibly belong
to a decision class.
• Boundary Region: Objects that cannot be precisely
classified.
• By analyzing inconsistencies in the decision table,
rough set theory helps identify essential attributes
and remove redundant ones.
 Rule Induction
• Rule induction is a process of extracting decision
rules from datasets, commonly used in machine
learning, data mining, expert systems, and rough
set theory.
• It helps in knowledge discovery by identifying
patterns and relationships in data.
 Types of Rule Induction:
• Direct Rule Induction
– Rules are derived directly from the dataset by
identifying patterns.
– Used in decision trees, rough set theory, and
association rule mining.
• Indirect Rule Induction
– Involves learning an intermediate model (neural
networks) and extracting rules from it.
– Used in fuzzy logic systems, neural networks, and
genetic algorithms.
• Rule Induction in Rough Set Theory:
• In rough sets, rule induction is performed
using decision tables where:
• Condition attributes are used to describe
objects.
• Decision attributes determine the outcome.
• Reducts help remove redundant attributes
while preserving classification power.
 Discernibility Matrix
• A discernibility matrix is a tool used in rough
set theory to identify important attributes in a
dataset.
• It helps determine which attributes distinguish
objects from each other, allowing for attribute
reduction while maintaining classification
ability.
• Construction of a Discernibility Matrix
• Given:
• A decision system S = (U, A ∪ {d}), where:
– U is the universe of objects (data instances)
– A is the set of conditional attributes
– d is the decision attribute (class label)
• Each entry M(i, j) contains a set of attributes
that differentiate object x_i from x_j.
• If two objects have the same decision value
(d(x_i) = d(x_ j)), then M(i, j) = ∅ (they are
indiscernible).
• The discernibility matrix is used to construct
reducts, which are minimal subsets of
attributes that preserve classification ability.
• Applications
• Feature Selection: Helps find minimal
attribute subsets that preserve classification
ability.
• Data Preprocessing: Identifies redundant or
irrelevant attributes.
• Rough Set Theory: Used for computing
reducts and core attributes in knowledge
reduction.
 UNIT 5
Integration of Soft Computing Techniques: Fuzzy Neural Network, Rough Fuzzy Sets, Rough
intuitionistic Fuzzy Sets, Neuro Fuzzy Systems, Fuzzy Genetic algorithms

Applications of Soft computing techniques: Pattern Recognition, Image Processing, Soft computing in
information retrieval and sematic web, Soft computing in software Engineering

 [Link] Neural Network (FNN):

• A fuzzy neural network (FNN) combines fuzzy logic and neural networks to create a
hybrid system that can handle uncertainty and learn from data, offering advantages in
pattern recognition, regression.

• Neural Networks: Inspired by the human brain, neural networks consist of

interconnected nodes (neurons) arranged in layers, capable of learning from data through
an iterative process.

• FNNs leverage the strengths of both fuzzy logic and neural networks, combining the
ability to handle imprecise information with the learning capabilities of neural networks.

Neuro-Fuzzy System (NFS):

• This is an another term for FNN, often used to emphasize the integration of neural
networks and fuzzy systems.

How FNNs Work:

• Fuzzy Neurons:

• FNNs are composed of fuzzy neurons, which are processing units that incorporate fuzzy
logic concepts.

• Parameter Learning:

• Neural network techniques are used to determine the parameters of a fuzzy system, such
as fuzzy sets and fuzzy rules.

• Synergistic Collaboration: ‘]o;p-

• FNNs aim to combine the uncertainty handling of fuzzy systems with the learning ability
of neural networks.
• Interpretability:

• FNNs can maintain a level of interpretability, allowing for the extraction of knowledge
from the network topology.

• Fuzzification: Inputs are converted into fuzzy values (membership degrees) using fuzzy
sets.

• Rule Evaluation: The fuzzy rules are applied to the inputs to generate fuzzy outputs.
These rules typically follow an "IF-THEN" structure.

• Defuzzification: The fuzzy output is converted back into a crisp value using a
defuzzification method like the centroid method.

• Features of Fuzzy Neural Networks:

• Inputs are fuzzy numbers.

• Outputs are fuzzy numbers.

• Weights are fuzzy numbers.

• Weighted inputs of each neuron are not aggregated by summation, but by some other
aggregation operation.
  2,Rough Fuzzy Sets
Rough fuzzy sets combine rough set theory and fuzzy set theory to handle uncertainty, capturing both
vagueness and indiscernibility through lower and upper approximations, offering a powerful tool for
processing imprecise and incomplete data.

 Rough Set Theory: This theory, developed by Pawlak, deals with knowledge
representation and data analysis, particularly when dealing with incomplete or inconsistent
data. It approximates crisp sets (sets with clear boundaries) using equivalence classes, which
represent objects that are indiscernible based on certain attributes.

 Fuzzy Set Theory: This theory, introduced by Zadeh, deals with vagueness and
uncertainty by assigning membership degrees (values between 0 and 1) to elements within a
set, allowing for partial membership.
 Rough Fuzzy Sets: By combining these two, rough fuzzy sets extend the capabilities of
both theories, allowing for the representation of knowledge that is both vague and imprecise.
 Lower and Upper Approximations: Rough sets use lower and upper approximations
to represent a concept or set. The lower approximation contains elements that are definitely in
the set, while the upper approximation contains elements that could potentially be in the set.
 Fuzzy Membership: In a rough fuzzy set, the membership of an element in the lower
and upper approximations can be represented using fuzzy membership values, allowing for a
more nuanced representation of uncertainty.
Definition:
Let (U,R) be an approximation space and U/R={X 1,X2.......Xn}. Then for any X
2.1 Rough Fuzzy C-Means (RFCM) Algorithm

 The Rough Fuzzy C-Means (RFCM) Algorithm is a clustering technique that integrates fuzzy
membership (to handle overlapping clusters) and rough set theory (to handle uncertainty and
vagueness) into the traditional C-Means algorithm.

Concepts Used in RFCM

1. Fuzzy Sets: Assigns each object a degree of belonging to multiple clusters (values
between 0 and 1).
2. Rough Sets: Uses lower and upper approximations to handle uncertainty in
classification.

Algorithm Steps

1. Initialize the cluster centers randomly.

2. Compute Membership Values using both fuzzy and rough set principles.
3. Update Cluster Centers by considering objects in both lower and upper approximations.
4. Recalculate Memberships based on new cluster centers.
5. Iterate Until Convergence (i.e., when cluster centers stop changing significantly).

Objective Function


 Incorporating both fuzzy and rough sets, a new C-means algorithm called rough fuzzy c-
means (RFCM) .This algorithm add the concept of fuzzy sets, and lower and upper
approximations of rough sets into C-means algorithm. While the members of fuzzy sets
enables efficient handling of overlapping partitions, the rough sets deal with uncertainty,
vagueness and incompleteness in class definition.
Steps of the Rough Fuzzy C-Means (RFCM) Algorithm:
  3. Rough intuitionistic Fuzzy Sets(RIFS)

Definition
Rough Intuitionistic Fuzzy Sets (RIFS) combine Rough Sets, Fuzzy Sets, and Intuitionistic
Fuzzy Sets to model uncertainty, vagueness, and hesitation in decision-making.

 Rough Sets handle imprecise or incomplete data by using lower and upper
approximations.
 Fuzzy Sets allow partial membership values between 0 and 1.
 Intuitionistic Fuzzy Sets (IFS) introduce degree of hesitation, making them more
flexible than standard fuzzy sets.
3.1 Rough Intuitionistic Fuzzy C-Means (RIFCM) Algorithm:

RIFCM is a clustering algorithm that combines rough sets, fuzzy sets, and intuitionistic fuzzy
sets. It is an enhanced version of Intuitionistic Fuzzy C-Means (IFCM) and Rough C-Means
(RCM).

Features of RIFCM:

 Uses lower and upper approximations (from rough sets) to define cluster boundaries.
 Fuzzy membership is used for better uncertainty handling.
 Non-membership and hesitation values from intuitionistic fuzzy sets improve decision-
making.
 Helps deal with overlapping partitions and uncertain data effectively.

Each cluster in RIFCM is identified by:

1. A centroid
2. A lower approximation (definite members)
3. An intuitionistic fuzzy boundary (uncertain members)

Membership Rules:

 If an object fully belongs to a cluster, its membership is 1 and hesitation is 0.

 If an object is in the boundary region, it has partial membership and partial
hesitation.
 This allows objects in overlapping regions to belong to multiple clusters.
Steps of the RIFCM Algorithm:
[Link] Fuzzy Systems

 Neuro-fuzzy (NF) computing is a hybrid framework that combines neural networks

(NNs) and fuzzy logic to solve complex problems involving uncertainty and non-
linearity. This approach is widely used in research and practical applications due to its
ability to integrate learning capabilities of NNs with reasoning capabilities of fuzzy
logic.
Features of Neuro-Fuzzy Systems:

 Combining Neural Networks & Fuzzy Logic

 Neural networks can model nonlinear relationships and are useful for classification
problems.
 Fuzzy logic addresses the imprecision of inputs and outputs by defining them as fuzzy
sets.
 Neural networks lack precision in outputs, but fuzzy logic provides interpretability
and accuracy.

 Training and Adaptability

 Neural networks require large training datasets to learn patterns.

 Fuzzy logic helps in knowledge-based decision-making, reducing dependency on
massive training data.

 Integration of Linguistic Rules & Learning Capabilities

 Fuzzy Information System (FIS): Uses linguistic rules and fuzzy operators to make
decisions.
 Artificial Neural Networks (ANNs): Learn from data using training algorithms and
model complex functions.
 The drawbacks of both approaches are complementary, making their integration
highly effective.

Advantages of the Hybrid Neuro-Fuzzy Model:

✔Handles Uncertainty: Can model and process imprecise data effectively.

✔Mathematical Relationship Handling: Useful for complex dynamic systems with multiple
variables.
✔Adaptability: Neural networks help fuzzy systems learn and adapt to changing
environments.
✔Expressiveness: Improves the ability to represent knowledge using fuzzy rules.
✔Better than Conventional Control Systems: Can control nonlinear systems, which
traditional methods struggle with.

Two Perspectives on Hybridization:

1. Adding Fuzzy Capabilities to Neural Networks

o Enhances the ability of NNs to handle uncertainty and imprecision.
o Allows for more flexible learning in dynamic environments.
 2. Adding Learning Capabilities to Fuzzy Systems
o Helps fuzzy logic-based models adapt over time.
o Enables self-learning fuzzy systems that refine rules dynamically.

This hybrid approach is also called Neural-Network-Driven Fuzzy Reasoning.

4.1Types of Neuro-Fuzzy Systems (NF Systems):

Neuro-fuzzy systems combine the learning capabilities of neural networks with the
interpretability of fuzzy logic systems (FIS). The image describes three main types of neuro-
fuzzy systems:

1. Cooperative Neuro-Fuzzy Systems

 How it works:
o The neural network learns FIS membership functions or fuzzy rules from training data.
o Once the fuzzy parameters are set, the neural network is no longer needed.
o Fuzzy rules are determined using clustering techniques (e.g., fuzzy clustering).
o ANN helps in defining fuzzy membership functions.
 Use Case:
o Used as a preprocessing step where the ANN initializes the fuzzy system, after which it
is no longer used.

2. Concurrent Neuro-Fuzzy Systems

 How it works:
o The ANN and FIS work together throughout the process.
o The ANN continuously adjusts FIS parameters, especially when inputs are hard to
measure.
o In some cases, FIS outputs may not be usable directly, and ANN helps in post-
processing.
 Use Case:
o Applied in adaptive control systems, where continuous updates are required.

3. Fused Neuro-Fuzzy Systems

 How it works:
o The ANN and FIS share a common structure instead of working separately.
o ANN directly determines the FIS parameters.
 o The fuzzy system is represented using an ANN-like model, making the learning process
seamless.
o However, conventional ANN training cannot be directly applied because fuzzy inference
involves non-differentiable functions.
 Solution:
o Use differentiable functions in fuzzy inference or modify the learning algorithm.
 Use Case:
o Deep learning-based fuzzy models for complex decision-making.

[Link] Genetic Algorithms (FGA)

Genetic Algorithms are inspired by natural evolution and mimic biological processes such as
selection, crossover, and mutation to find optimal solutions.

 GAs perform random searches in a defined space to find the best solution based on a
fitness function.
 The fitness function determines how good a solution is.

Genetic Algorithms differ from conventional optimization methods in four key ways:

1. Uses Encoding of Parameters Instead of Direct Values

 GA works with a coded representation (e.g., binary, real numbers) of parameters instead of
using the actual parameters directly.

2. Searches in a Population Instead of a Single Point

 Unlike traditional optimization, which improves a single solution at a time, GA explores a

population of possible solutions.

3. Uses Only Objective Function Information

 GA does not require derivative information like gradient-based methods. It works with
just the fitness value of solutions.

4. Uses Probabilistic Rules Instead of Deterministic Rules

 GA introduces randomness in selection, crossover, and mutation to explore different

solutions instead of following a fixed path.
 Fuzzy logic helps GA handle uncertain, imprecise, or vague fitness functions, making FGA useful
in:
✔ Complex optimization problems
 ✔ Machine learning and AI
✔ Control systems and robotics

Key Components of Genetic Algorithms:

Genetic algorithms use the following elements in optimization:

✔Genetic representation of solutions (chromosomes).
✔Initial pool of solutions (population).

✔Fitness function to evaluate solutions.

✔Genetic operators (crossover & mutation) to evolve solutions.

✔GA parameters (mutation rate, crossover probability, etc.).

 GA requires a finite set of alternatives for optimization.

 If the problem domain is continuous or infinite, it must be discretized into a finite subset.
 Solutions are represented as chromosomes, which are strings of symbols from a finite
alphabet.
 Chromosomes are made up of genes, and the complete set of genes is called the gene pool

Connection Between Fuzzy Systems & GA

The relationship between Genetic Algorithms and Fuzzy Systems is bidirectional:

1. GA Used to Optimize Fuzzy Systems:

 GA can optimize fuzzy inference rules in fuzzy controllers.

 Helps improve decision-making in adaptive fuzzy systems.

2. Fuzzifying Genetic Algorithms:

 Classical GA can be fuzzified, allowing fuzzy values in chromosomes instead of crisp values.
 This improves GA’s ability to handle uncertainty and imprecise solutions.

Two Ways to Fuzzify Genetic Algorithms

1. Fuzzify the Gene Pool & Coding of Chromosomes

 Traditional GA uses binary representation (e.g., {0,1}).

 Instead of discrete {0,1} values, we extend the gene pool to a continuous range [0,1], allowing
partial membership.
 2. Fuzzify the Genetic Operations on Chromosomes

 Selection, Crossover, and Mutation can be modified to incorporate fuzzy logic.

 For example, mutation may have a fuzzy probability instead of a fixed rate.

Applications of Soft computing techniques

1. Pattern Recognition:

 The capability of recognizing and classifying patterns is one of the most

fundamental characteristics of human intelligence. It plays a key role in
perceptron as well as the various levels of cognition.
 The term pattern refers to the task of placing some objects to a correct class based
on features of the object.
 A pattern recognition is a classification of input data via extraction of important
features from a lot of noisy data.

A system which classifies a given set of objects on the basis of certain parameters
depicting upon their characteristics is called pattern recognition system.

 A pattern recognition system has three major components. These components take care of
sensing, feature extraction and classification.
 Some of the applications such as hand written recognition seem to be easy for humans but
difficult for machines. These characters are written differently by different people and
sometimes it also differs for a single person.
 Syntactic Pattern Recognition (SPR) is an approach to pattern recognition that models
patterns as structured objects rather than simple feature vectors. It is based on formal
grammars and languages, making it useful for complex and hierarchical pattern structures.
Syntactic Pattern Recognition (SPR) is an approach to pattern recognition that models
patterns as structured objects rather than simple feature vectors. It is based on formal
grammars and languages, making it useful for complex and hierarchical pattern structures.
 Syntactic Pattern Recognition models patterns as structured objects using formal
grammars.
 Parsing techniques are used for recognition, making it ideal for speech, handwriting, and
bioinformatics.
 Neural pattern recognition is yet another type of pattern recognition process which uses
artificial neural networks to solve pattern recognition problems
 1.1 Fuzzy Pattern Recognition:
 Fuzzy Pattern Recognition is an advanced pattern recognition method that integrates
fuzzy set theory to handle uncertainty, vagueness, and imprecise data more
effectively than traditional crisp models.

 Traditional pattern recognition methods using crisp logic often fail in real-life situations
with imprecise or overlapping patterns.
 The introduction of fuzzy sets by Zadeh (1965) revolutionized clustering and
classification methods, making them more robust to vagueness.
 Since human thinking is inherently fuzzy, fuzzy logic is more suitable for complex
decision-making systems.

Three Key Problems in Pattern Recognition

1. Sensing Problem (Input Data Representation)

 The measurement of input data from objects that need to be recognized.

 Each object is represented as a vector of attributes (pattern vector).

 Mathematical Representation:

a=(a1,a2,…,ar)

where each ai is an attribute of the object.

 Some features in the vector may not be significant and can be removed.
 Feature reduction helps in improving system efficiency.

2. Feature Extraction Problem

 Identifying important features that distinguish pattern classes.

 Selecting only the most relevant attributes improves classification accuracy.

 Reduce dimensionality of data.

 Improve pattern discrimination between classes.

3. Classification Problem (Decision Making)

 Assigning an object to a specific class based on its features.

  Uses discrimination functions to classify patterns.

 A real number (score) is assigned to each pattern by a discrimination function.

 The pattern is classified into the class with the highest score

 Reduce dimensionality of data.

Methods of Fuzzy Pattern Recognition

Fuzzy pattern recognition can be generalized from two classical approaches:

1. Membership-Roster Methods
2. Syntactic Methods

1. Membership-Roster Methods

 In this approach, each pattern class is defined by a set of stored patterns.

 A new pattern is compared with stored patterns, and if it matches, it is classified into that class.
 Example:
o Biometric systems (e.g., fingerprint recognition in employee identification).
o Fingerprint patterns of employees are stored in a database, and an incoming fingerprint
is matched against stored ones.
 Limitation:
o Requires storing many patterns to capture the full variation within a class.

2. Syntactic Methods (Grammar-Based Recognition)

 A pattern is represented as a sequence of sub-patterns called primitives.

 These primitives act like an alphabet in a formal language.
 Patterns are classified based on grammar rules that define how primitives can be combined.
 Example:
o Recognizing handwritten words where strokes or shapes form characters based on
predefined grammar rules.
o If an unknown pattern follows the same grammatical structure, it is classified into the
corresponding pattern class.

2. Image Processing:

 Image processing is connected with pattern recognition in the sense that data in many
pattern recognition problems are given in terms of digital images. Examples are
 printed and handwritten character recognition, automatic classification of X-ray
images, fingerprints recognition, target identification and human face recognition.
 In general a gray-tone image is a discrete approximation of a black and white picture,
which is essential for computer processing. It is represented by mxn matrix.

 There are basically two classes of method for image processing: frequency domain
methods and spatial domain methods.
 In frequency domain methods, the image is processed in terms of its Fourier transform.
 In spatial domain methods, the processing directly involves the pixels of matrix R.

Components of an Image Processing System

1. Image Sensors – Captures images in either analog or digital form.
2. Digitizers – Converts analog images into digital format using sampling and quantization.
 3. Image Processor – Performs image enhancement, restoration, compression, and analysis.
4. Hard Copy Unit – Prints images using laser printers, film cameras, etc.
5. Image Display Unit – Displays processed images on a monitor.
6. Main Storage Device – Stores large image data, requiring compression for efficient handling

Functionality of Each Component

Image Sensors:

 Capture images using cameras, scanners, or sensors.

 Analog images are captured on photographic films (e.g., traditional cameras).
 Digital sensors directly capture digital images.

Digitizers (Conversion of Analog to Digital):

 Converts an analog image into a digital image.

 Uses sampling (selecting intensity values at points) and quantization (assigning pixel
intensity levels).

Image Processor:

 A general-purpose or specialized computer used for processing images.

 Performs image enhancement, filtering, feature extraction, segmentation, etc.
 Requires large storage capacity (e.g., a 1024×1024 image with 8-bit intensity per pixel
needs megabytes of memory).

Storage & Display:

 Processed images must be stored for display or printing.

 Compression techniques help reduce storage requirements.
 Display cards in computers help render high-quality images.

Hard Copy & Transmission:

 Printers and film cameras create hard copies.

 For network transmission, high bandwidth is required to send large image files.

2.1 Fuzzy Image Processing:

What is Fuzzy Image Processing (FIP)?

 FIP is a specialized branch of fuzzy pattern recognition.

 It applies fuzzy set theory to traditional digital image processing techniques.
 Images, segments, or features are processed as fuzzy sets instead of crisp values.
  The processing method depends on the problem definition and fuzzy methodology
used.

2.1.2 Image Processing through Clustering

What is Image Processing Through Clustering?

 Image segmentation is a process that divides an image into its constituent regions or
objects.
 The goal is to extract meaningful objects from an image by detecting similarities or
discontinuities.
 Clustering is a widely used technique in image segmentation, helping to group pixels
based on shared characteristics.

Two Main Approaches to Image Segmentation

1. Discontinuity-Based Segmentation (Edge Detection):

  Focuses on abrupt intensity changes in an image.
 Identifies edges and boundaries between different regions.
 Example: Sobel, Canny, Prewitt edge detection algorithms.

2. Similarity-Based Segmentation (Clustering):

 Partitions an image into regions based on similar intensity or texture.

 Uses clustering techniques to group similar pixels.
 Example: K-Means, Fuzzy C-Means (FCM), Mean-Shift, Watershed Algorithm.

Clustering in Image Segmentation

 Clustering helps determine which pixels "belong together" based on features like
intensity, texture, or color.
 There are two main clustering approaches in image segmentation:
✅ Partitioning Methods: Divide data into k predefined clusters (e.g., K-Means).
✅ Grouping Methods: Clusters naturally emerge based on similarity (e.g., Mean-
Shift).

Role of Uncertainty-Based Clustering Models

 Many soft computing techniques, like fuzzy logic and rough sets, handle uncertainty in
image segmentation.
 Fuzzy clustering (e.g., Fuzzy C-Means) is useful when boundaries between objects are
not well-defined.
 These models improve accuracy and adaptability in complex image segmentation tasks.
Application of Soft Computing Techniques in MRI Image Processing

🔹 Overview

 The given brain MRI image is segmented using different soft computing techniques to
highlight patterns in medical imaging.
 The image segmentation technique groups similar pixel intensities to enhance
visualization of brain structures.
 Rough-Set-Based Clustering Algorithms Used
 1. RCM (Rough C-Means) – Handles lower approximation objects effectively.
2. RFCM (Rough Fuzzy C-Means) – Uses both rough and fuzzy memberships for better
segmentation.
[Link] (Rough Intuitionistic Fuzzy C-Means) – Further refines boundary regions by
considering hesitation degrees.

Observations from MRI Image Processing

 Figure 17.3(a): Shows the original MRI image (raw scan).

 Figure 17.3(b): RCM version, where the brain structure is enhanced using rough
clustering.
 Figure 17.3(c): FCM applied, outlining the brain better by grouping similar pixel
values.
  Figure 17.3(d): IFCM is applied, making the inner brain outline clearer than FCM.
 Figure 17.3(e): RCM overlaid on FCM, further improving edge detection and
segmentation clarity.

Different Segmentation Techniques Used

1. Figure 17.3(c) - FCM (Fuzzy C-Means) Version:

 Groups pixels based on similarity.

 Produces a smooth and continuous segmentation but lacks well-defined boundaries.

2. Figure 17.3(d) - IFCM (Intuitionistic Fuzzy C-Means) Version:

 Enhances segmentation by considering hesitation degrees.

 Better edge detection than FCM, making the brain's outline clearer.

3. Figure 17.3(e) - RFCM (Rough Fuzzy C-Means) Version:

 Handles uncertainty better by using lower and upper approximations.

 Improves segmentation compared to standard FCM.

4. Figure 17.3(f) - RIFCM (Rough Intuitionistic Fuzzy C-Means) Version:

 Combines advantages of RFCM and IFCM.

 Produces the most refined segmentation, accurately identifying various brain regions.

Note:

✔FCM provides smooth segmentation but lacks strong boundary detection.

✔IFCM improves edge detection, making brain structures more distinct.
✔RFCM and RIFCM perform better in handling uncertainty and providing accurate
segmentation.
✔RIFCM gives the best result by refining the segmentation while preserving details.
Comparison of Soft Computing Techniques in Coin Image Processing:
Observations from Different Techniques

1. Figure 17.4(a) - Original Image:

 Shows the original penny coin with clear details.

2. Figure 17.4(b) - RCM (Rough C-Means) Version:

 Only two major clusters formed instead of four, as required.

 Some details are lost due to rough clustering.

[Link] 17.4(c) - FCM (Fuzzy C-Means) Version:

 Improves edge detection and captures subtle color differences.

 Provides smoother segmentation but may still mix similar regions.

[Link] 17.4(d) - IFCM (Intuitionistic FCM) Version:

 Enhances the face features on the coin.

 More accurate color shading detection than FCM.

[Link] 17.4(e) - Combination of RCM and FCM:

 Fixes color variation issues and enhances cluster separation.

  Face shape is more accurately defined compared to Figure 17.4(b).

[Link] 17.4(f) - Combination of RCM and IFCM (RFCM/RIFCM):

 The best refinement, but the face shape is still slightly unclear.
 Handles uncertainty better than previous methods.
 Horizontal dark shade across the coin is still visible.

 Soft Computing in Information Retrieval and Semantic Web

Soft computing techniques are extensively used in the semantic web to handle incomplete,
uncertain, and vague real-life data. Fuzzy ontology is a major approach for modeling and
recognizing human behavior. For example, Fuzzy OWL2 helps manage uncertainty and
analyzes temporal relationships in human activities using fuzzy logic. Compared to crisp
techniques, fuzzy-based knowledge representation improves decision-making.

Applications of Soft Computing in Human Activity Recognition

 Ambient Assisted Living & Smart Homes

o MOSES: Monitors staff, reminds deadlines, and manages pending tasks.
o iShopFloor: Helps in industrial planning and control.
o EasyMeeting & Aml: Optimizes office meetings and official tasks.
 Smart Office Environments
o Tools like AIRE, iRoom, NIST enhance smart meeting spaces and consider
emotional factors of participants.

Why Soft Computing for Ontology?

 Soft computing handles missing, uncertain, and vague values better than crisp models.
 It enables approximate reasoning, allowing intelligent systems to function like humans.
 Fuzzy ontology models effectively address real-world data inconsistencies, while crisp
ontology-based models struggle to do so.

Example: Coffee Break Recognition

A fuzzy ontology can define an event like a coffee break by assigning weights to its component
actions (e.g., 0.3 TakeMug, 0.3 Take-CoffeePan, 0.4 TakeMilk).

 If one action is skipped due to sensor failure or an exception (e.g., milk runs out), the
activity can still be recognized to a lower degree.
 However, in crisp ontology, missing any one of these elements fails recognition
completely.
Ontology-Based Fuzzy Model for Human Behavior Recognition

Figure 17.7 illustrates the relationship between user, activity, and environment in an
ontology model for recognizing human behavior. The relationships are defined using fuzzy
membership functions such as triangular, trapezoidal, left shoulder, and right shoulder
functions.

Rule 1: Coffee Making Activity

If a person opens the cupboard to take coffee, turns on the coffee machine, holds the coffee jar,
and possibly takes milk from the fridge, then the person is engaged in making coffee.

(define-concept antecedent1

(g-and User (some performsAction MakeCoffee))

(define-concept MakeCoffee

(w-sum (0.1 OpenCupboard)

(0.5 TurnCoffeeMachineOn)

(0.3 MoveCoffeeJar)

(0.1 OpenFridge)))

)
(define-concept consequent1

(g-and User (some performsActivity MakingCoffee))

(define-concept Rule1

(l-implies antecedent1 consequent1))

 The antecedent1 checks if a User performs MakeCoffee actions.

 The MakeCoffee activity is fuzzily defined with different weight contributions (e.g., 0.5
for turning on the coffee machine, 0.1 for opening the fridge).
 If the antecedent holds true, the consequent1 determines that the User is performing
the MakingCoffee activity.

Rule 2: Meeting Room Activity

If a meeting room has the lights ON, it implies that a meeting is happening.

Fuzzy Rule Definition:

(define-concept antecedent2

(g-and MeetingRoom (some hasLights (some hasStatus ON)))

(define-concept consequent2

(g-and MeetingRoom (some hasOccupancyStatus HoldingAMeeting))

(define-concept Rule2

(l-implies antecedent2 consequent2))

 The antecedent2 checks if the MeetingRoom lights are ON.

 If lights are ON, the MeetingRoom is occupied and a meeting is in progress

(HoldingAMeeting).
  Fuzzy logic is used to model uncertainties in human behavior.
 Activities are not strictly defined, allowing partial recognition based on observed
actions.
 Weight-based fuzzy rules help model real-world uncertainty, unlike crisp logic.

Lunch Activity Recognition

If a person uses a bottle, plate, fork, spoon, and knife, then it is highly certain that they are
having lunch.
(define-concept antecedent3

(g-and User (some performsAction Lunch))

(define-concept Lunch

(w-sum (0.1 UseBottle)

(0.4 UsePlate)

(0.2 UseFork)

(0.1 UseSpoon)

(0.1 UseKnife)))

(define-concept consequent3

(g-and User (some performsActivity HaveLunch))

(define-concept Rule3

(l-implies antecedent3 consequent3))

 Antecedent3 checks if a User is performing Lunch-related actions.

 The Lunch activity is modeled fuzzily with different weight contributions (e.g., 0.4 for
using a plate, 0.2 for using a fork).
 If the antecedent holds true, the consequent3 confirms that the User is engaged in
having Lunch.

Hierarchical-Data Divisive Soft Clustering (H2D-SC) Algorithm

This algorithm is proposed by Bordogna for text classification and efficiently handles
overlapping, multi-label classes and multi-objective problems.
Steps in H2D-SC Algorithm

Step 1:

 All documents are initially considered as members of a single large cluster C (the whole
collection).
 Membership value = 1 for all documents in C.

Step 2:

 Decision is made whether to split C into sub-clusters, based on cluster quality assessment.

Step 3:

 If splitting is required, the number of sub-clusters (KC) is determined based on cluster

properties.

Step 4:

 The FCM algorithm (Fuzzy C-Means) is applied to generate soft clusters at the next hierarchical
level.

Step 5:

 The process iterates until no further cluster splitting is needed.

 Fuzzy techniques in the semantic web enable intelligent information retrieval.
 Fuzzy description logic improves search, annotation, security, and medical
diagnosis.
 H2D-SC algorithm is useful for hierarchical clustering and text classification.
 The Hierarchical-Data Divisive Soft Clustering (H2D-SC) model organizes text
documents into hierarchical fuzzy clusters using a soft clustering approach (like Fuzzy
C-Means (FCM)).
Input: Documents → The process starts with full-text indexing, extracting content features
from the text.

Quality Measures Assessment → Evaluates the quality of clustering before deciding whether
a node should be split further.

User's Input Parameters

 MAXL → Maximum number of hierarchy levels.

 Dmin & Dmax → Minimum and maximum cluster sizes.
 Decides Node's Splitting → Based on the quality assessment, the model decides whether to
split a node into sub-clusters.

Decides Number of Nodes → Determines how many child nodes (sub-clusters) should be
created at each level.

FLAT Soft Clustering Application (FCM) → If splitting occurs, Fuzzy C-Means (FCM) is
used to create soft clusters at the next hierarchical level.

Final Hierarchical Clustering Structure → The process continues iteratively until no further
splitting is needed.

Key Features of H2D-SC

 Fuzzy Soft Clustering → Instead of hard clustering, documents can belong to multiple
clusters with different membership values.
 Hierarchical Organization → Generates a tree-like structure, useful for text
classification, document retrieval, and topic modeling.
 User Control → Allows customization of clustering behavior using MAXL, Dmin, and
Dmax parameters.

 Soft Computing in Software Engineering

Soft computing (SC) techniques are widely used in software engineering for tasks such as
software documentation, design, maintenance, re-engineering, refactoring, and software
quality assessment.
Key Applications of SC in Software Engineering:

1. Decision Making & Problem Solving: SC helps engineers tackle complex problems, handle
constraints, resolve conflicts, and make optimal decisions in both functional and non-functional
aspects.
2. Search-Based Software Engineering: SC techniques find optimal solutions for software
engineering problems, especially combinatorial search problems.
3. Meta-Heuristic Search Techniques: Techniques like genetic algorithms, hill climbing, particle
swarm optimization, and simulated annealing are used for design optimization and object-
oriented analysis.
4. Class Responsibility Assignment & Architecture Optimization: SC methods improve
modifiability, help in UML-based design, and optimize software packages.

Fuzzy Neural Models in Software Quality Assessment:

 Traditional Boolean logic is replaced by multi-valued fuzzy logic to assess software quality.
 Fuzzy neural networks are used to model relationships between factors, criteria, and software
metrics using max-min composition.
  A rough neural network model is also used, showing better results compared to the fuzzy
neural model.
 The lower and upper bound estimation determine the number of software module changes.
 A developer can make judgments about the number of required changed relative to certain and
uncertain knowledge about software quality.

These techniques enhance software quality evaluation, automate decision-making, and

optimize software design efficiently.