UNIT IV: Probabilistic Neural Network

Syllabus: Probabilistic Neural Network: Hopfield Net, Boltzmann machine, RBMs, Sigmoid net,
Auto encoders
Outline the basic features of Probabilistic Neural Networks. How did they implement
memorizing concepts?
Probabilistic Neural Networks (PNNs) are a type of neural network that falls under the category
of probabilistic models. They are primarily used for pattern recognition and classification tasks.
Here are the basic features of Probabilistic Neural Networks and how they implement the
memorization of concepts:
Architecture:

PNNs consist of four layers: input layer, pattern layer, summation layer and decision/output
layer.
Input Layer: p neurons represent the input vector and distribute it to the next layer. p equals the
number of input features.
Pattern Layer: This layer applies the kernel to the input. It organizes the learning set by
representing each training vector by a hidden neuron that records the features of this vector.
During inference, each neuron calculates the Euclidean distance between the input test vector
and the training sample, then applies the radial basis kernel function. In this way, it encodes the
PDF centered on each training sample or pattern. For class j, we have nj neurons, for j in [1, m].
Summation Layer: This layer computes the average of the output of the pattern units for each
class. There's one neuron for each class. Each class neuron is connected to all neurons in the
pattern layer of that class.
Output Layer: This layer selects the maximum value from the summation layer, and the
associated class label is determined accordingly.
Radial Basis Function (RBF) Activation: The pattern layer uses RBF activation functions, which
measure the similarity between the input pattern and stored patterns. The RBF activation for a
neuron j in the pattern layer is given by: A j=exp(−2σ2/∥X−Wj∥2) where X is the input pattern, Wj
is the weight vector associated with the j-th neuron, and σ is a width parameter controlling the
spread of the activation.
Pattern Storage: PNNs memorize concepts by storing patterns in the pattern layer. Each neuron
in the pattern layer corresponds to a stored pattern or prototype. The weight vectors Wj
associated with each neuron represent the stored patterns.
Probabilistic Decision Layer: The decision layer computes the class probabilities based on the
output of the pattern layer. The class probability P(Ci∣X) for class Ci given input pattern X is
proportional to the sum of the RBF activations of the neurons in the pattern layer associated with
class Ci.
Classification: PNNs use a winner-takes-all strategy for classification. The class with the highest
computed probability is chosen as the output class.
Training: PNNs do not undergo a traditional training process involving iterative weight updates.
Instead, training involves storing prototype patterns in the pattern layer during a training phase.
The width parameter σ is often determined through cross-validation or other heuristics.
Kernel Density Estimation: PNNs can be viewed as a form of kernel density estimation, where
the RBF activations act as kernels centered at the stored patterns. The decision layer computes
class probabilities based on the estimated density of the input pattern within each class.
Memory and Conceptual Learning: PNNs memorize concepts by storing exemplar patterns in the
pattern layer. When a new input pattern is presented, the network evaluates the similarity
between the input pattern and stored patterns to make a probabilistic decision.
Advantages
 Multilayer perceptron networks are substantially slower than PNNs.
 PNNs have the potential to outperform multilayer perceptron networks in terms of
accuracy.
 Outliers aren’t as noticeable in PNN networks.
 PNN networks predict target probability scores with high accuracy.
 PNNs are getting close to Bayes’s optimum classification.
Disadvantages:
 When it comes to classifying new cases, PNNs are slower than multilayer perceptron
networks.
 PNN requires extra memory to store the mod.
Applications:
 Probabilistic Neural Networks can be applied to class prediction of Leukemia and Central
Nervous System Embryonal Tumors.
 Probabilistic Neural Networks can be used to identify ships.
 Management of sensor setup in a wireless ad hoc network can be done using a
probabilistic neural network.
 It can be applied to Remote-sensing image classification.
 Character Recognition is also an important application of Probabilistic Neural Networks.
There are many more applications of PNNs.
In summary, Probabilistic Neural Networks operate by computing the similarity between input
patterns and stored patterns using RBF activations. The memorization of concepts is achieved
through the storage of prototype patterns in the pattern layer, allowing for probabilistic decision-
making based on the learned patterns. PNNs are simple yet effective models, particularly suitable
for certain pattern recognition tasks.
Explain the architecture of Hopfield Net
A Hopfield Network is a type of recurrent artificial neural network that serves as a content-
addressable memory system. It is characterized by its associative memory properties, where the
network is trained to memorize patterns and recall them even when presented with incomplete or
noisy versions of the stored patterns. The architecture of a Hopfield Network is relatively simple
and consists of the following components:
Neurons (Nodes): A Hopfield Network consists of a set of binary neurons (nodes). Each neuron
can take on one of two states: +1 or -1. These states are used to represent the presence or absence
of a feature in a pattern.
Activation (Output) State: The
activation state of a neuron is
typically binary, either +1 or -1.
This binary activation is determined
by a threshold function.
Connection Weights: The
connections between neurons are
represented by symmetric weights,
often denoted by a weight matrix W.
The weight between neuron i and
neuron j is denoted as Wij. The
weights are used to store the
information or patterns that the
network has learned.
Recurrence (Feedback): Hopfield
Networks are recurrent neural
networks, meaning that neurons are
connected in a loop, allowing for feedback. The recurrent connections enable the network to
update its state over multiple iterations.
Energy Function: The energy function of a Hopfield Network is a crucial concept. It is used to
measure the stability or goodness of a particular configuration of neuron states. The energy E of
a network configuration S is often defined as: E(S)=−2/∑i∑jWijSiSj where Si and Sj are the
states of neurons i and j, respectively.
Activation Function (Thresholding): The activation function of a Hopfield Network is typically a
simple thresholding function. Neurons activate (change state) based on the sign of the net input.
If the net input is positive, the neuron becomes active (+1); if negative, the neuron becomes
inactive (-1).
Learning (Weight Update) Rule: Hopfield Networks are trained to store patterns using a Hebbian
learning rule. The weight between neurons i and j is updated based on the product of the states of
i and j in the training patterns: Wij=N1∑pSip⋅Sjp where N is the number of neurons, p indexes
the training patterns, and Sip and Sjp are the states of neurons i and j in pattern p.
Pattern Retrieval: Hopfield Networks can be used to retrieve stored patterns from incomplete or
noisy inputs. The network iteratively updates its state based on the weights until it converges to a
stable state or reaches a limit on the number of iterations.
The dynamics of a Hopfield Network involve updating neuron states in a sequential or
asynchronous manner until the network reaches a stable state. Once stable, the network state can
represent a memorized pattern or the closest approximation to a stored pattern given the input.
Hopfield Networks have limitations, such as capacity constraints and the possibility of spurious
states, but they remain a fundamental model in the study of associative memory and content-
addressable memory systems.
Explain the working principle of Hopfield network
The working principle of a Hopfield Network involves two key processes: training and pattern
retrieval. Hopfield Networks are recurrent neural networks designed for associative memory,
allowing them to store and recall patterns based on partial or noisy input. Here's a step-by-step
explanation of how a Hopfield Network works:
1. Training:
Initialization: Start with an empty or randomly initialized Hopfield Network composed of N
binary neurons (N is the number of neurons in the network).
Define a set of training patterns, each represented by the states of the neurons (+1 or −1).
Hebbian Learning: Train the Hopfield Network using a Hebbian learning rule to adjust the
connection weights between neurons.
The weight between neurons i and j is updated based on the product of their states in the training
patterns: Wij=N/∑pSip⋅Sjp where N is the number of neurons, p indexes the training patterns,
and Sip and Sjp are the states of neurons i and j in pattern p.
Symmetry: Ensure that the weight matrix W is symmetric (Wij=Wji), reflecting the symmetric
nature of the Hebbian learning rule.
2. Pattern Retrieval:
Input Presentation:
Present an incomplete or noisy pattern as input to the Hopfield Network by setting the states of
some neurons.
Iterative Update: Iteratively update the states of the neurons until the network reaches a stable
state or a limit on the number of iterations is reached.
The update rule for neuron i is based on the net input vi, calculated as the weighted sum of the
inputs and the activation function: vi=∑jWijSj and Si(t+1)=sgn(vi)
The activation function sgn(⋅) assigns a binary state (+1 or −1) based on the sign of the net input.
Energy Minimization: The network updates its state to minimize the energy function, which is
defined as: E(S)=−2/∑i∑jWijSiSj
The energy should decrease as the network converges to a stable state.
Pattern Retrieval: The stable state reached by the network represents the recalled or closest
approximation to a stored pattern.
The retrieved pattern may correspond to the stored pattern that is most similar to the input.
3. Associative Memory:
Hopfield Networks exhibit associative memory, meaning that presenting a partial or noisy
version of a stored pattern can trigger the retrieval of the complete pattern.
The network has the ability to generalize and recall patterns even when presented with
incomplete or distorted inputs.
Key Considerations:
Capacity Limitations: Hopfield Networks have limitations on the number of patterns they can
effectively store (limited capacity).
The network may exhibit spurious states, representing patterns that were not part of the training
set.
Dynamics: The network's dynamics involve the iterative updating of neuron states, and the
convergence to stable states is influenced by the network's weight matrix.
Hopfield Networks, while simple, have been influential in the study of associative memory and
content-addressable memory systems. They provide insights into how recurrent neural networks
can be used for pattern retrieval and offer a foundational understanding of associative memory
principles.
Discuss the working Sigmoid net with suitable example.
A "Sigmoid net" typically refers to a neural network that uses the sigmoid activation function in
its neurons. The sigmoid function is commonly employed in the hidden layers of a neural
network to introduce non-linearity and enable the model to learn complex relationships in the
data. The sigmoid function squashes the output to a range between 0 and 1, making it useful for
binary classification problems.
Here is an explanation of the working of a neural network with sigmoid activation, along with a
suitable example:
Working of Sigmoid Net:
Input Layer: The network begins with an input layer where each node represents a feature of the
input data.
Weighted Sum: The input values are multiplied by weights, and the weighted sums are calculated
for each neuron in the hidden layer.
The weighted sum for the i-th neuron in the hidden layer is given by: zi=∑jwij⋅xj+bi where wij is
the weight connecting the j-th input node to the i-th hidden neuron, xj is the input value at node j,
and bi is the bias term for the i-th neuron.
Activation Function (Sigmoid): The sigmoid activation function is applied to the weighted sum
1
to introduce non-linearity. The sigmoid function is defined as: σ(z)= −z
1+ e
The output of the i-th hidden neuron (ai) is the result of applying the sigmoid activation function
to zi: ai=σ(zi)
Propagation to Output Layer (if applicable): For multi-layer networks, the process is repeated
for subsequent hidden layers until the output layer is reached.
The output layer may use the sigmoid activation function for binary classification or a softmax
function for multi-class classification.
Loss Function: A loss function is used to measure the difference between the predicted output
and the actual target values. Common loss functions for binary classification with sigmoid
activation include binary cross-entropy.
Backpropagation: The backpropagation algorithm is employed to update the weights and biases
in the network to minimize the loss. The gradients are computed with respect to the weights and
biases, and the parameters are adjusted in the opposite direction of the gradient.
Training: The network is trained on a labeled dataset, and the weights and biases are adjusted
iteratively using an optimization algorithm such as stochastic gradient descent.
Prediction: Once trained, the network can make predictions on new, unseen data by feeding it
through the network. The sigmoid function in the output layer can be used to obtain probabilities
for binary classification.
Example:
Let's consider a binary classification problem where the task is to predict whether a student
passes (y=1) or fails (y=0) based on two exam scores (features: x1 and x2).
Input Layer: Two nodes representing exam scores (x1 and x2).
Hidden Layer: Two neurons with sigmoid activation.
Output Layer: One neuron with sigmoid activation.
The network is trained on a dataset with labeled examples. The sigmoid activation at the output
layer is used to produce a probability that the student passes the exam. The weights and biases
are adjusted during training to minimize the prediction error.

Compare and contrast the features of Hopfield net and sigmoid net
Hopfield Networks and Sigmoid Networks (which I assume refers to networks using sigmoid
activation functions, often seen in feedforward neural networks) are distinct types of neural
networks with different architectures and purposes. Here's a comparison of their features:
Hopfield Network:
Architecture: Hopfield networks are recurrent neural networks with symmetric connections.
They have a single layer of neurons where each neuron is connected to every other neuron.
Neuron Activation: Binary Activation: Neurons in Hopfield networks typically have binary
activation states (+1 or -1).
Training: Hebbian Learning: Hopfield networks are trained using Hebbian learning, where the
weights are adjusted based on the correlations between the states of connected neurons in the
training patterns.
Memory: Associative Memory: Hopfield networks serve as associative memory systems,
allowing for the recall of entire patterns based on partial or noisy inputs.
Energy Function: Energy Minimization: The network operates by iteratively minimizing an
energy function, and the stable states reached during this process represent stored patterns.
Use Case: Pattern Recognition: Hopfield networks are commonly used for pattern recognition
and associative memory tasks, where the goal is to recall stored patterns from incomplete or
distorted inputs.
Sigmoid Network:
Architecture: Sigmoid networks are feedforward neural networks with an input layer, one or
more hidden layers, and an output layer.
Neuron Activation: Sigmoid Activation: Neurons in the hidden layers of sigmoid networks
typically use the sigmoid activation function, which squashes the output to a range between 0
and 1.
Training: Backpropagation: Sigmoid networks are trained using backpropagation, an iterative
optimization algorithm that adjusts the weights and biases to minimize a predefined loss
function.
Memory: No Explicit Memory: Sigmoid networks do not have explicit memory of specific
patterns. They learn to generalize from the training data and make predictions on new, unseen
examples.
Energy Function: Not Applicable: Sigmoid networks do not involve explicit energy functions
like Hopfield networks. The focus is on minimizing prediction errors during training.
Use Case: Generalization and Prediction: Sigmoid networks are widely used for tasks such as
classification, regression, and other predictive modeling tasks. They excel at generalizing from
data to make predictions on new, unseen examples.
Comparison:
Architecture: Hopfield networks have a simpler architecture with symmetric connections, while
sigmoid networks are typically more complex, allowing for hierarchical feature representations.
Learning: Hopfield networks use Hebbian learning for memorization, while sigmoid networks
use backpropagation for learning from labeled data.
Memory: Hopfield networks explicitly store patterns as stable states, serving as content-
addressable memory. Sigmoid networks do not have explicit memory storage.
Use Cases: Hopfield networks are suited for pattern recognition and associative memory tasks.
Sigmoid networks are versatile and widely used for various tasks, including classification and
regression.
Activation Function: Hopfield networks use binary activation, while sigmoid networks use the
sigmoid activation function to introduce non-linearity.
Training Approach: Hopfield networks are trained to memorize specific patterns, while sigmoid
networks are trained to generalize from examples and make predictions on new data.
In summary, Hopfield networks are specialized for associative memory tasks, while sigmoid
networks are more general-purpose and commonly used for supervised learning tasks in machine
learning. Their architectures, learning mechanisms, and applications differ significantly.
Explain the architecture/ working of the Autoencoder with an example.

An autoencoder is a type of artificial neural network designed for unsupervised learning and
dimensionality reduction. The architecture of an autoencoder consists of an encoder and a
decoder, and the network is trained to reconstruct its input data. The key idea is to learn a
compact and meaningful representation (encoding) of the input data in a lower-dimensional
space. Here's a breakdown of the architecture with an example:
Architecture of an Autoencoder:
Encoder: The encoder is the first part of the autoencoder and is responsible for mapping the
input data to a lower-dimensional representation. It consists of one or more layers of neurons that
reduce the input dimensionality.
Bottleneck (Latent Space): The reduced-dimensional representation is known as the bottleneck or
latent space. It represents a compressed and informative encoding of the input data.
Decoder: The decoder is the second part of the autoencoder and is responsible for reconstructing
the input data from the bottleneck representation. It mirrors the structure of the encoder but in the
reverse order.
Symmetry: The autoencoder architecture is often designed to be symmetrical, with the encoder
and decoder having a similar or mirrored structure. This symmetry encourages the network to
learn a meaningful representation in the bottleneck.
Loss Function: The training objective of the autoencoder is to minimize the reconstruction error
between the input data and the reconstructed output. Common loss functions include mean
squared error (MSE) or binary cross-entropy, depending on the nature of the data.
Example of an Autoencoder: Let's consider a simple example using a one-dimensional input
(e.g., a single feature) and a bottleneck of lower dimensionality. This is often referred to as a
"undercomplete" autoencoder.
In this example, the autoencoder has one input neuron, one neuron in the bottleneck layer, and
one output neuron. The activation functions and the number of neurons can be adjusted based on
the complexity of the task and the nature of the data.
The training process involves minimizing the mean squared error between the input and the
reconstructed output. After training, the autoencoder can be used to encode data into the
bottleneck representation. The example above is a basic illustration, and autoencoders can be
extended to handle higher-dimensional input data and more complex architectures.
Explain the working of the Boltzmann machine with an example.
A Boltzmann Machine is a type of stochastic recurrent neural network with an undirected graph
structure. It was introduced by Geoffrey Hinton and Terry Sejnowski in the 1980s. Boltzmann
Machines are designed to model the joint probability distribution of binary variables. The
working of a Boltzmann Machine involves a process of sampling and learning. Here's an
overview of how it works along with a simple example:
Working of a Boltzmann Machine:
Graph Structure: A Boltzmann Machine has a fully connected undirected graph structure,
meaning that each neuron is connected to every other neuron in the network.
Neurons and Binary States: Neurons in a Boltzmann Machine are binary units, meaning they can
take on two states: 0 or 1. These binary states represent the presence or absence of a feature.
Energy Function: The probability distribution over the states of the neurons is determined by an
energy function. The energy of a configuration is given by a weighted sum of the product of
neuron states and connection weights.
E(s)=−∑i∑j wijsisj− ∑i bisi
where E(s) is the energy of the configuration s, wij is the weight between neurons i and j, bi is
the bias of neuron i, and si is the state of neuron i.
Probability Distribution: The probability of a configuration s is defined using the Boltzmann
distribution:
−E (s)/ T
e
P(s)= ∑ e−E (t )/ T
t

where T is the temperature parameter that

controls the randomness of the system. As T
approaches zero, the distribution becomes
deterministic.
Gibbs Sampling: Gibbs sampling is used to
generate samples from the probability
distribution. It involves iteratively updating
the state of each neuron based on the states
of its neighbors. The probability of a neuron
flipping its state is determined by the
conditional probability given the states of its
neighbors.
Learning (Training): The learning in Boltzmann Machines involves adjusting the connection
weights to better capture the underlying data distribution. This is typically done using a form of
Hebbian learning where the weight update is proportional to the correlation between the states of
connected neurons:
Δwij=η(⟨sisj⟩data−⟨sisj⟩model)
where ⟨sisj⟩data is the correlation in the training data, ⟨sisj⟩model is the correlation in the
model's samples, and η is the learning rate.
Example of a Boltzmann Machine: Let's consider a simplified example where we have a
Boltzmann Machine with three binary neurons (0 or 1). The task is to model the distribution of
possible states. We'll perform Gibbs sampling to generate samples.
In this example, Gibbs sampling is used to generate samples from the Boltzmann Machine. The
network's parameters (weights and biases) are randomly initialized. The resulting samples should
represent the distribution of possible states according to the Boltzmann distribution.
This is a simplified illustration, and in practice, Boltzmann Machines can be more complex with
larger numbers of neurons and advanced training techniques. They were historically challenging
to train due to the computational demands of Gibbs sampling, and more modern neural network
architectures like Restricted Boltzmann Machines (RBMs) have been developed to address these
challenges.
Restricted Boltzmann Machines
Introduction: Invented by Geoffrey Hinton in 1985, Restricted Boltzmann Machine which falls
under the category of unsupervised learning algorithms is a network of symmetrically connected
neuron-like units that make stochastic decisions. This deep learning algorithm became very
popular after the Netflix Competition where RBM was used as a collaborative filtering technique
to predict user ratings for movies and beat most of its competition. It is useful for regression,
classification, dimensionality reduction, feature learning, topic modelling and collaborative
filtering.
Restricted Boltzmann Machines are stochastic two layered neural networks which belong to a
category of energy based models that can detect inherent patterns automatically in the data by
reconstructing input. They have two layers visible and hidden. Visible layer has input nodes
(nodes which receive input data) and the hidden layer is formed by nodes which extract feature
information from the data and the output at the hidden layer is a weighted sum of input layers. It
might seem strange but they don’t have any output nodes and they don’t have typical binary
output through which patterns are learnt. The learning process happens without that capability
which makes them different. We only take care of input nodes and don’t worry about hidden
nodes. Once the input is provided, RBM’s automatically capture all the patterns, parameters and
correlation among the data. That’s the beauty of the Restricted Boltzmann Machine.
Features of Restricted Boltzmann Machine
 They use recurrent and symmetric structure.
 RBMs in their learning process try to associate high probability with low energy states
and vice-versa.
 There are no intra layer connections.
 It is an unsupervised learning algorithm i.e. it makes inferences from input data without
labeled responses.
Let’s understand how a Restricted Boltzmann Machine is different from a Boltzmann Machine.
Both the algorithms have two layers visible and hidden. In the Boltzmann Machine each neuron
in the visible layer is connected to each neuron in the hidden layer as well as all neurons are
connected within the layers. However, RBM is a special case of Boltzmann Machine with a
restriction that neurons within the layer are not connected i.e., no intra-layer communication
which makes them independent and easier to implement as conditional independence means that
we need to calculate only marginal probability which is easier to compute. Below diagrams will
help us understand the same:

Working of RBM: As mentioned earlier Restricted Boltzmann Machine is an unsupervised

learning algorithm, so how does it learn without using any output data? We have a visible layer
of neurons that receives input data which is multiplied by some weights and added to a bias
value at the hidden layer neuron to generate output. Then the output value generated at the
hidden layer neuron will become a new input which is then multiplied with the same weights and
then bias of the visible layer will be added to regenerate input. This process is called
reconstruction or backward pass. Then the regenerated input will be compared with the original
input if it matches or not. This process will keep on happening until the regenerated input is
aligned with the original input.
For Example, we have a six set of movies Avatar, Oblivion, Avengers, Gravity, Wonder Woman
and Fast & Furious 7. So, movies will become visible neurons and the latent features which we
are trying to learn will become hidden neurons. Below diagram shows the Restricted Boltzmann
Machine formed.
Here Avatar, Oblivion and Gravity will fall under Sci-Fi movie genre and remaining will fall
under thriller. So, Thriller and Sci-Fi will become hidden neurons of hidden layers which are the
features extracted from our input (set of movies).
If a person has told us her set of movie preferences, then our RBM can activate the hidden
neurons of her preferences. So, those sets of movies will send messages to hidden neurons to
update themselves for that user.
Conversely if a user likes Thriller movies then, our RBM can find movies which are turned on by
the hidden neuron “Thriller” and hidden neurons will send messages to visible neurons to update
their states and this will work as a movie recommendation engine.
Training of RBM: RBM is trained using Gibbs Sampling and Contrastive Divergence.
 In statistics, Gibbs sampling is a Markov chain Monte Carlo algorithm for obtaining a
sequence of observations which are approximately from a specified multivariate
probability distribution, when direct sampling is difficult.
If input is represented by v and hidden value by h then, p(h|v) is the prediction. Knowing the
hidden values, p(v|h) is used for prediction of regenerated input values. Say this process is
repeated k times and after k iterations v_k is obtained from initial input value v_0.
 Contrastive Divergence is an approximate Maximum-Likelihood learning algorithm to
approximate the graphical slope representing the relationship between a network’s
weights and its error, called the gradient. It is used in a situation where we can’t evaluate
a function or set of probabilities directly, some form of inference model is needed to
approximate the algorithm’s learning gradient and decide which direction to move
towards.
In CD, weights are being updated. First gradient is calculated from reconstructed input and then
delta is added to old weights to get new weights.
Advantages:
 Expressive enough to encode any distribution and computationally efficient.
 Faster than traditional Boltzmann Machine due to the restrictions in terms of connections
between nodes.
 Activations of the hidden layer can be used as input to other models as useful features to
improve performance
Disadvantages:
 Training is more difficult as it is difficult to calculate the Energy gradient function.
 CD-k algorithm used in RBMs is not as familiar as the back propagation algorithm.
 Weight Adjustment
Applications of RBM
There are many real time business use cases where RBM is used like
 Pattern recognition: RBM is used for feature extraction in pattern recognition problems
where the challenge is to understand the hand written text or a random pattern.
 Recommendation Engines: RBM is widely used for collaborating filtering techniques
where it is used to predict what should be recommended to the end user so that the user
enjoys using a particular application or platform. For example: Movie Recommendation,
Book Recommendation
 Radar Target Recognition: Here, RBM is used to detect intra pulse in Radar systems
which have very low SNR and high noise.
To summarize, Restricted Boltzmann Machines are unsupervised two layered neural models that
learn from the input distribution. There are many variations and improvements on RBM and the
algorithms used for their training and optimization. They are trained using contrastive divergence
and after training they can generate novel samples from the training dataset. There are also some
modifications which are being made to original RBM’s in order to make them more efficient and
representable like in the case of Fuzzy RBM, Infinite RBM.