Associative memory is also known as content addressable
memory (CAM) or associative storage or associative array. It is a
special type of memory that is optimized for performing searches
through data, as opposed to providing a simple direct access to
the data based on the address.
It can store the set of patterns as memories when the associative
memory is being presented with a key pattern, it responds by
producing one of the stored pattern which closely resembles or
relates to the key pattern.
It can be viewed as data correlation here. input data is
correlated with that of stored data in the CAM.
It forms of two type:
1. auto associative memory network : An auto-associative
memory network, also known as a recurrent neural
network, is a type of associative memory that is used to
recall a pattern from partial or degraded inputs. In an
auto-associative network, the output of the network is
fed back into the input, allowing the network to learn and
remember the patterns it has been trained on. This type
of memory network is commonly used in applications
such as speech and image recognition, where the input
data may be incomplete or noisy.
2. hetero associative memory network : A hetero-
associative memory network is a type of associative
memory that is used to associate one set of patterns with
another. In a hetero-associative network, the input
pattern is associated with a different output pattern,
allowing the network to learn and remember the
associations between the two sets of patterns. This type
of memory network is commonly used in applications
such as data compression and data retrieval.
Training input and target output vectors are different
Uses Hebb’s rule/Delta Rule/Outer Product Rule to find the weight
matrix.
Answer:
2. Train a Hetero associative Memory Network Hebb Rule
to Store input to output
Training input & target output vectors are same.
Uses Hebb’s rule/ Outer Product Rule to find the weight matrix.
Question:
Answer
Bidirectional Associative Neural Networks
Bidirectional Associative Memory (BAM) is a type of recurrent neural netw
ork that allows for the storage and retrieval of hetero-associative patterns.
BAM is capable of responding to inputs from both the input and
output layers, making it bidirectional. This unique feature enables BAM to
store and
recall patterns that are potentially of different sizes, similar to human me
mory. The
architecture of BAM consists of two fully connected layers of neurons, whic
h are
interconnected bidirectionally. This design allows BAM to store and retriev
e data
efficiently, making it suitable for various applications, including pattern rec
ognition and decision-making processes in AI systems.
Bidirectional Associative Memory (BAM) is a supervised
learning model in Artificial Neural Network. This is hetero-associative
memory, for an input pattern, it returns another pattern which is
potentially of a different size. This phenomenon is very similar to the
human brain. Human memory is necessarily associative. It uses a chain of
mental associations to recover a lost memory like associations of faces
with names, in exam questions with answers, etc. In such memory
associations for one type of object with another, a Recurrent Neural
Network (RNN) is needed to receive a pattern of one set of neurons as an
input and generate a related, but different, output pattern of another set
of neurons. Why BAM is required? The main objective to introduce such
a network model is to store hetero-associative pattern pairs. This is used
to retrieve a pattern given a noisy or incomplete pattern. BAM
Architecture: When BAM accepts an input of n-dimensional
vector X from set A then the model recalls m-dimensional vector Y from
set B. Similarly when Y is treated as input, the BAM recalls X.
A BAM contains two layers of neurons, which we shall denote X and Y.
Layers X and Y are fully connected to each other. Once the weights have
been established, input into layer X presents the pattern in layer Y, and
vice versa.
The layers can be connected in both directions (bidirectional) with the
result the weight matrix sent from the X layer to the Y layer is W and the
T
weight matrix for signals sent from the Y layer to the X layer is W .
Thus, the weight matrix is calculated in both directions.
Procedure
Learning
Imagine we wish to store two associations, A1:B1 and A2:B2.
A1 = (1, 0, 1, 0, 1, 0), B1 = (1, 1, 0, 0)
A2 = (1, 1, 1, 0, 0, 0), B2 = (1, 0, 1, 0)
These are then transformed into the bipolar forms:
X1 = (1, -1, 1, -1, 1, -1), Y1 = (1, 1, -1, -1)
X2 = (1, 1, 1, -1, -1, -1), Y2 = (1, -1, 1, -1)
T T
From there, we calculate M =∑ X i Y iwhere X i denotes the transpose.
So,
[ ]
2 0 0 −2
0 −2 2 0
M= 2 0 0 −2
−2 0 0 2
0 2 −2 0
−2 0 0 2
min(m, n) nm
, where " " is the number of units in the Y layer.[3]
The internal matrix has n x p independent degrees of freedom, where n is
the dimension of the first vector (6 in this example) and p is the dimension
of the second vector (4). This allows the BAM to be able to reliably store
and recall a total of up to min(n,p) independent vector pairs, or min(6,4) =
4 in this example. The capacity can be increased above by sacrificing
reliability (incorrect bits on the output).
Stability
A pair ( A , B)defines the state of a BAM. To store a pattern, the energy
function value for that pattern has to occupy a minimum point in the
energy landscape.
The stability analysis of a BAM is based on the definition of Lyapunov
E ( A , B)( A , B)
function (energy function) , with each state is
presented to BAM, the neurons change states until a bi-directionally stable
state ( A f , B f )is reached, which Kosko proved to correspond to a local
minimum of the energy function. The discrete BAM is proved to converge
to a stable state.
T
The Energy Function proposed by Kosko is E( A , B)=− AM B for the
bidirectional case, which for a particular case A=B corresponds
T
to Hopfield's auto-associative energy function.[3] (i.e. E( A , B)=− AM A ).