UNIT 2 — NETWORKS (12 Hours)

Hopfield Networks – Energy – Hamming Network – RAM – Boltzmann Machine –

Instar / Outstar Network – ART – Kohonen Network Recognition

1. HOPFIELD NETWORK

1.1 Introduction

A Hopfield Network is a type of recurrent artificial neural network (RNN) introduced

by John J. Hopfield in 1982. It is one of the earliest neural network models designed to
function as an associative memory system, capable of storing and retrieving patterns.
Unlike feedforward neural networks, where information flows in only one direction, a
Hopfield Network has feedback connections, allowing information to circulate among
neurons until the network reaches a stable state.

The primary purpose of a Hopfield Network is pattern storage and pattern retrieval. It
can recall a complete stored pattern even when the input pattern is incomplete, noisy,
or partially corrupted. This capability makes it similar to the way human memory works,
where a small clue can help recall an entire memory.

Hopfield Networks are called auto-associative memory networks because the input
pattern and the desired output pattern are the same. During training, patterns are stored
in the network by adjusting the connection weights. During recall, the network
compares the input with stored memories and converges to the closest matching
pattern.

The network consists of interconnected neurons with symmetric weights. Each neuron
can have one of two possible states:

• Binary representation: 0 or 1

• Bipolar representation: -1 or +1

The network repeatedly updates neuron states until no further changes occur. At this
point, the network reaches a stable equilibrium known as an attractor state, which
corresponds to one of the stored patterns.

Hopfield Networks have applications in:

• Pattern recognition

• Image restoration

• Error correction

• Associative memory
 • Optimization problems (such as the Traveling Salesman Problem)

• Signal processing

Although modern deep learning methods have replaced Hopfield Networks in many
practical applications, they remain an important milestone in the history of artificial
intelligence and neural network research.

1.2 Characteristics

The Hopfield Network has several important characteristics that distinguish it from
other neural network models.

1. Binary or Bipolar Neurons

Each neuron can take only two possible values:

• Binary: 0 or 1

• Bipolar: -1 or +1

The bipolar representation is more commonly used because it simplifies mathematical

calculations.

2. Fully Connected Network

Every neuron is connected to every other neuron in the network. This complete
interconnection enables all neurons to influence each other's states during
computation.

3. Symmetric Weight Matrix

The weights satisfy the condition:

𝑊𝑖𝑗 = 𝑊𝑗𝑖

This symmetry ensures that the network's energy function always decreases during
updates, guaranteeing convergence.

4. No Self-Connections

A neuron does not have a connection to itself.

𝑊𝑖𝑖 = 0

This prevents unnecessary feedback loops.

5. Energy Function
The Hopfield Network is governed by an energy function that decreases whenever a
neuron updates its state. Eventually, the energy reaches a minimum value, indicating a
stable solution.

6. Asynchronous Updating

Neurons update one at a time rather than simultaneously. This asynchronous updating
improves stability and prevents oscillations.

7. Content Addressable Memory

Instead of searching memory by address, the network retrieves stored information

based on similarity between the input and stored patterns.

8. Stable Attractor States

Each stored pattern represents a stable equilibrium point. Any noisy version of that
pattern eventually converges to the original stored memory.

9. Limited Storage Capacity

A Hopfield Network can reliably store approximately:

0.138𝑁

patterns, where N is the number of neurons. Exceeding this limit introduces retrieval
errors and false memories (spurious states).

1.3 Architecture Diagram

The Hopfield Network consists of fully interconnected neurons with symmetric weights.

(v1)

●────────● (v2)

/\ /\

/ \____/ \

/ /\ \

/ / \ \

(v3) ●───●────● (v4)

Fully Connected Hopfield Network

A clearer representation with four neurons is shown below:

● (v1)

/| \ \

/| \\

●---●----●

(v2) | (v3)

\| /

\| /

(v4)

Each neuron is connected to every other neuron.

Components of Architecture

• Input neurons receive the pattern.

• Weighted connections connect every neuron to every other neuron.

• Symmetric weights satisfy 𝑊𝑖𝑗 = 𝑊𝑗𝑖 .

• No self-loop, so 𝑊𝑖𝑖 = 0.

• The network repeatedly updates neuron states until convergence.

1.4 Working of Hopfield Network

The operation of a Hopfield Network consists of two major phases:

Phase 1: Training (Memory Storage)

1. Training patterns are converted into bipolar form.

2. The weight matrix is calculated using the Hebbian learning rule.

3. All patterns are stored within the network.

Phase 2: Recall (Pattern Retrieval)

1. A noisy or incomplete pattern is supplied as input.

2. Each neuron computes its weighted input.

 𝑁𝑒𝑡𝑖 = ∑ 𝑊𝑖𝑗 𝑋𝑗
𝑗

3. The neuron updates its state according to the sign function.

+1, 𝑁𝑒𝑡𝑖 > 0
𝑋𝑖 = {
−1, 𝑁𝑒𝑡𝑖 < 0

4. One neuron updates at a time (asynchronous updating).

5. After each update, the network energy decreases.

6. Updating continues until the neuron states stop changing.

7. The final stable state corresponds to the stored pattern closest to the input.

Working Steps

1. Input pattern is applied to the network.

2. The weighted sum is calculated for each neuron.

3. Neurons update one after another.

4. Energy decreases after every update.

5. The network reaches a minimum energy state.

6. The final output becomes the retrieved stored pattern.

1.5 Hopfield Learning Rule

The Hopfield Network uses the Hebbian Learning Rule, which is based on the
principle:

"Neurons that fire together, wire together."

The connection weights are determined from the training patterns.

For bipolar inputs (+1 and -1), the weight between neuron 𝑖and neuron 𝑗is:
𝑃
1 (𝑝) (𝑝)
𝑊𝑖𝑗 = ∑ 𝑋𝑖 𝑋𝑗 , 𝑖 ≠ 𝑗
𝑁
𝑝=1

where:

• 𝑁= Number of neurons
 • 𝑃= Number of stored patterns
(𝑝)
• 𝑋𝑖 = State of neuron 𝑖in pattern 𝑝

• 𝑊𝑖𝑗 = Weight between neurons 𝑖and 𝑗

The diagonal elements are always zero:

𝑊𝑖𝑖 = 0

Weight Matrix

The complete weight matrix is:

0 𝑊12 𝑊13 𝑊14
𝑊21 0 𝑊23 𝑊24
𝑊=[ ]
𝑊31 𝑊32 0 𝑊34
𝑊41 𝑊42 𝑊43 0

The matrix is symmetric:

𝑊𝑖𝑗 = 𝑊𝑗𝑖

Energy Function

The stability of the Hopfield Network is measured by the energy function:

1
𝐸 = − ∑ ∑ 𝑊𝑖𝑗 𝑋𝑖 𝑋𝑗
2
𝑗
𝑖

As neurons update, the energy continually decreases until a minimum value is reached.
This minimum corresponds to a stable stored memory.

Advantages

• Simple architecture.

• Efficient associative memory.

• Can recover noisy or incomplete patterns.

• Guaranteed convergence due to the energy function.

• Useful for optimization and pattern recognition tasks.

Limitations
 • Limited storage capacity.

• May converge to incorrect or spurious states.

• Computationally expensive for large networks.

• Not suitable for learning complex nonlinear relationships.

• Performance degrades when too many patterns are stored.

Conclusion:
The Hopfield Network is a foundational recurrent neural network model that
demonstrates how information can be stored and retrieved using associative memory.
Its concepts of energy minimization, symmetric weights, and attractor dynamics have
significantly influenced the development of modern neural networks and optimization
techniques.

2. ENERGY FUNCTION

2.1 Energy Concept

The Energy Function is one of the most important concepts in a Hopfield Network. It is
a mathematical function used to measure the stability of the network. During the
operation of the network, the energy continuously decreases as neurons update their
states. The network eventually reaches a minimum energy level, where no further
neuron state changes occur. This minimum energy state represents a stable stored
pattern (attractor).

The energy function acts like a "guide" that directs the network toward the correct
memory. Whenever an input pattern is provided, especially if it is noisy or incomplete,
the network modifies the neuron outputs step by step, reducing the energy after each
update until a stable solution is reached.

The concept is similar to many physical systems in nature, where objects naturally
move toward the lowest possible energy state. Likewise, a Hopfield Network always
evolves toward a minimum energy configuration.

The general energy equation for a Hopfield Network is:

1
𝐸 = − ∑ ∑ 𝑊𝑖𝑗 𝑉𝑖 𝑉𝑗 + ∑ 𝜃𝑖 𝑉𝑖
2
𝑗 𝑖
𝑖

where:

• E = Energy of the network

 • 𝑊𝑖𝑗 = Weight between neuron i and neuron j

• 𝑉𝑖 = Output (state) of neuron i

• 𝜃𝑖 = Threshold (bias) of neuron i

If all thresholds are zero, the equation becomes:

1
𝐸 = − ∑ ∑ 𝑊𝑖𝑗 𝑉𝑖 𝑉𝑗
2
𝑗
𝑖

Importance of the Energy Function

• Measures the stability of the network.

• Determines whether the network has reached a stored memory.

• Guarantees convergence to a stable state.

• Prevents continuous oscillations during neuron updates.

• Enables associative memory retrieval.

2.2 Energy Minimization

The Hopfield Network always attempts to minimize its energy during computation. This
process is known as energy minimization.

Initially, when a noisy or incomplete pattern is applied, the network may have a
relatively high energy value. Each neuron calculates the weighted sum of its inputs and
updates its output based on the activation rule. Every update either decreases or
maintains the current energy level; it never increases.

As neurons continue updating asynchronously (one neuron at a time), the energy

gradually decreases until the network reaches its minimum value. At this point, the
network becomes stable and no neuron changes its state. The final output corresponds
to one of the stored patterns in memory.

The energy minimization process can be summarized as follows:

1. A noisy or incomplete pattern is applied.

2. The initial energy is relatively high.

3. Neurons update one at a time.

4. Each update reduces or maintains the network energy.

 5. Energy continues decreasing.

6. The network reaches the lowest energy state.

7. The final stable state represents the stored memory.

Relation to Physical Systems

The energy function of a Hopfield Network is inspired by the Ising model in statistical
physics. In the Ising model, magnetic particles naturally align themselves to achieve the
lowest energy configuration. Similarly, neurons in a Hopfield Network update their
states until the overall network energy reaches a minimum.

This analogy explains why Hopfield Networks are highly stable and reliable for
associative memory applications.

Advantages of Energy Minimization

• Guarantees convergence to a stable state.

• Ensures reliable memory retrieval.

• Reduces the effect of noisy inputs.

• Makes the network robust against small disturbances.

• Provides a mathematical measure of network stability.

2.3 Graphical Representation

The energy landscape of a Hopfield Network can be visualized as valleys and hills. High-
energy states are unstable, while low-energy valleys correspond to stored memories.

High Energy

/\

/ \

/ \

/ \

/ \

● ●

/ \

/ \
 ____/________________\____

● ●

Stored Memory Stored Memory

(Low Energy) (Low Energy)

Network State

Moves downhill by reducing energy

until it reaches the nearest stable state.

Energy Landscape Explanation

• High energy region: Unstable neuron configurations.

• Downhill movement: Network updates reduce the energy.

• Low energy valleys: Stable attractor states representing stored memories.

• Final state: The network converges to the nearest stored pattern.

Conclusion

The Energy Function is the foundation of the Hopfield Network's operation. It provides a
mathematical way to evaluate network stability and ensures that every neuron update
moves the system toward a lower-energy configuration. Through the process of energy
minimization, the network successfully retrieves stored patterns from noisy or
incomplete inputs, making Hopfield Networks an effective model for associative
memory and optimization problems.

3. HAMMING NETWORK

3.1 Introduction

A Hamming Network is a type of artificial neural network designed for pattern

recognition and classification. It identifies the stored pattern that is most similar to a
given input pattern by using the Hamming distance as a similarity measure. The
network is especially suitable for recognizing binary patterns, where each element of
the pattern is represented by either 0 and 1 or -1 and +1.

The Hamming Network was developed to efficiently compare an input pattern with a set
of stored reference patterns. It determines which stored pattern differs from the input
by the fewest number of bits. The pattern with the smallest Hamming distance (or
equivalently, the highest similarity score) is selected as the output.

Unlike the Hopfield Network, which retrieves patterns through energy minimization, the
Hamming Network performs classification by comparing the input directly with stored
templates and selecting the best match.

Hamming Distance

Hamming Distance is defined as the number of bit positions at which two binary
patterns differ.

It is calculated as:

Hamming Distance = Number of differing bit positions

Example

Pattern 1:

11001

Pattern 2:

11101

Comparison:

Position 1 2 3 4 5

Pattern 1 1 1 0 0 1

Pattern 2 1 1 1 0 1

Different? No No Yes No No

Only one bit is different.

Therefore,

Hamming Distance = 1

Features of Hamming Network

• Designed for binary pattern recognition.

• Uses Hamming distance for similarity measurement.

• Consists of two processing layers.

 • Performs fast classification.

• Produces a single winning output (Winner-Takes-All).

• Suitable for noisy binary inputs.

3.2 Network Structure

The Hamming Network consists of two layers:

1. Feedforward Layer

The feedforward layer compares the input pattern with every stored pattern.

Functions of this layer:

• Receives the input pattern.

• Computes the similarity score between the input and stored patterns.

• Produces one score for each stored pattern.

• Higher score indicates greater similarity.

The neuron corresponding to the stored pattern with the highest similarity receives the
largest activation.

2. Recurrent Layer

The recurrent layer performs competitive learning using the Winner-Takes-All (WTA)
mechanism.

Functions of this layer:

• Receives similarity scores from the feedforward layer.

• Neurons inhibit one another through lateral inhibition.

• Only the neuron with the highest activation remains active.

• Suppresses all other neurons.

• Declares the winning class.

This layer ensures that only the best-matching stored pattern is selected.

Overall Working

1. Binary input pattern is applied.

 2. Feedforward layer compares the input with stored patterns.

3. Similarity scores are calculated.

4. Scores are passed to the recurrent layer.

5. Winner-Takes-All competition begins.

6. Highest scoring neuron survives.

7. The winning neuron identifies the recognized pattern.

3.3 Diagram

Basic Architecture

Input Pattern

+------------------------+

| Feedforward Layer |

| (Similarity Computation)|

+------------------------+

+------------------------+

| Recurrent Layer |

| (Winner-Takes-All) |

+------------------------+

Winner Class

Detailed Representation

Input Pattern

│
 ---------------------

│ │ │

▼ ▼ ▼

N1 N2 N3

Feedforward Layer

│ │ │

└────────┼─────────┘

Recurrent Layer

(Competition)

Winning Neuron

Recognized Pattern

3.4 Applications

The Hamming Network is widely used in applications involving binary pattern

comparison and classification.

1. Character Recognition

Recognizes printed or handwritten characters by comparing the input character with

stored templates.

2. Binary Pattern Matching

Finds the closest stored binary pattern to a given input pattern.

3. Codeword Identification

Detects and identifies valid binary codewords in communication systems.

4. Error Detection and Correction

Helps detect transmission errors by measuring the Hamming distance between
received and valid codewords.

5. Optical Character Recognition (OCR)

Used in OCR systems to recognize letters and numbers from scanned documents.

6. Image Processing

Recognizes binary images and simple visual patterns.

7. Digital Communication

Used for signal detection and decoding in digital communication systems.

8. Pattern Classification

Classifies unknown binary inputs into predefined categories based on similarity.

Advantages

• Simple and easy to implement.

• Fast pattern recognition.

• Efficient for binary data.

• Robust against small amounts of noise.

• Winner-Takes-All mechanism provides clear classification.

• Low computational complexity.

Limitations

• Suitable mainly for binary patterns.

• Cannot learn new patterns dynamically after training.

• Limited to template-based classification.

• Performance decreases if patterns are highly similar.

• Less effective for complex, high-dimensional data compared to modern neural

networks.

Conclusion

The Hamming Network is an efficient neural network model for binary pattern
recognition and classification. It measures similarity using the Hamming distance,
computes similarity scores in the feedforward layer, and selects the best matching
pattern through the Winner-Takes-All recurrent layer. Due to its simplicity, speed, and
reliability, it is widely used in applications such as character recognition, binary
pattern matching, codeword identification, error detection, and digital
communication. Although modern neural networks have largely replaced it in complex
applications, the Hamming Network remains an important foundational model in
pattern recognition and artificial intelligence.

4. RAM NETWORK (Random Access Memory Network)

4.1 Introduction

A RAM Network (Random Access Memory Network) is a type of weightless neural

network in which information is stored directly in memory locations (RAM cells) rather
than in weighted connections between neurons. Unlike conventional neural networks
such as the Hopfield Network or Multilayer Perceptron, a RAM Network does not require
weight adjustment or lengthy training algorithms.

The basic idea behind a RAM Network is simple: the input pattern is used as an address
to access a memory location. During training, the network stores the desired output in
the corresponding RAM location. During testing, the same input pattern accesses the
stored memory, allowing the network to retrieve the associated output almost instantly.

Since the network stores patterns directly into memory, learning is extremely fast and
usually requires only a single training pass. This makes RAM Networks suitable for
applications requiring rapid learning and retrieval of binary patterns.

RAM Networks are commonly used in:

• Boolean function classification

• Pattern recognition

• Pattern storage and retrieval

• Digital signal processing

• Image recognition

• Data classification

Unlike traditional neural networks that learn by repeatedly adjusting weights through
backpropagation or Hebbian learning, RAM Networks simply memorize input-output
associations.

4.2 Key Points

The RAM Network has several unique characteristics that distinguish it from other
neural network models.

1. Weightless Neural Network

A RAM Network does not use connection weights. Instead, knowledge is stored
directly in RAM memory cells.

2. Input Bits Form Address

The binary input pattern is interpreted as a memory address. Different input

combinations correspond to different RAM locations.

Example:

Input Pattern: 1010

Binary Address = 10 (Decimal)

The network accesses memory location 10.

3. Address Decoder

An address decoder converts the input bits into a unique memory address. Only one
RAM cell corresponding to that address is activated at a time.

4. Direct Memory Storage

During training, the desired output is stored in the selected RAM location. During recall,
the stored value is read directly from memory.

5. Fast Learning

Learning is extremely fast because the network simply writes data into memory without
iterative weight updates. Most RAM Networks can be trained in a single pass.

6. Fast Retrieval

Once trained, retrieving stored information requires only a memory access, making the
response time very fast.

7. Binary Data Processing

RAM Networks are particularly effective for binary inputs and Boolean functions.

Working Steps

1. Binary input pattern is applied.

2. Input bits are converted into a memory address.

 3. The address decoder selects the corresponding RAM cell.

4. Stored information is read from the RAM.

5. The output is produced immediately.

4.3 Diagram

Basic Architecture

Input Bits

+------------------+

| Address Decoder |

+------------------+

+------------------+

| RAM Cell |

| (Memory Storage) |

+------------------+

Output

Detailed Representation

Binary Input

(1 0 1 1 0 1)

+------------------+

| Address Decoder |
 +------------------+

Memory Address

+------------------+

| RAM Memory |

| [Stored Data] |

+------------------+

Classified Output

Applications

RAM Networks are widely used in applications that require fast pattern storage and
retrieval.

• Boolean classification

• Pattern storage

• Pattern recognition

• Character recognition

• Image classification

• Digital communication systems

• Signal processing

• Fault detection systems

• Real-time decision-making applications

Advantages

• Very fast learning (single-pass training).

 • No weight calculations are required.

• Simple architecture.

• High-speed memory retrieval.

• Easy implementation in hardware.

• Suitable for binary pattern classification.

• Low computational complexity.

Limitations

• Requires a large amount of memory for complex problems.

• Best suited for binary input data.

• Does not generalize well to unseen patterns.

• Memory usage increases rapidly with input size.

• Less effective for continuous or high-dimensional data.

Conclusion

A RAM Network (Random Access Memory Network) is a weightless neural network

that stores knowledge directly in memory instead of using weighted connections. Input
bits act as memory addresses, and an address decoder selects the appropriate RAM
cell to retrieve the stored output. Due to its fast learning, simple architecture, and
rapid memory access, the RAM Network is widely used in Boolean classification,
pattern storage, character recognition, and digital signal processing. Although it
requires considerable memory for large datasets, it remains an important neural
network model for applications that demand high-speed pattern matching and retrieval.

5. BOLTZMANN MACHINE

5.1 Introduction

A Boltzmann Machine (BM) is a type of stochastic recurrent neural network

introduced by Geoffrey Hinton and Terry Sejnowski in the early 1980s. It is an energy-
based neural network that uses probabilistic neuron activation instead of
deterministic activation. Unlike the Hopfield Network, where neuron states are updated
deterministically, neurons in a Boltzmann Machine change their states based on
probability. This enables the network to explore many possible solutions and avoid
getting trapped in poor local minima.
The Boltzmann Machine is designed to learn complex probability distributions from
data. It can discover hidden structures and relationships within datasets, making it a
powerful model for unsupervised learning. The network stores information in the form
of an energy function, and learning aims to reduce the energy associated with observed
data while increasing the energy of incorrect patterns.

A Boltzmann Machine consists of two types of neurons:

• Visible units, which receive input data.

• Hidden units, which learn useful internal representations of the input.

During training, the network repeatedly updates neuron states based on probabilities
until it reaches thermal equilibrium. This process allows the machine to learn the
statistical properties of the training data.

Boltzmann Machines laid the foundation for several important deep learning models,
including the Restricted Boltzmann Machine (RBM) and the Deep Belief Network
(DBN). Although full Boltzmann Machines are computationally expensive, their
simplified versions are still widely used in machine learning and artificial intelligence.

Features

• Stochastic recurrent neural network.

• Probabilistic neuron activation.

• Energy-based learning model.

• Uses visible and hidden units.

• Learns probability distributions.

• Basis for several modern deep learning architectures.

5.2 Characteristics

The Boltzmann Machine has several important characteristics that distinguish it from
other neural networks.

1. Visible and Hidden Units

The network consists of:

• Visible units that represent the input data.

• Hidden units that capture complex features and relationships within the data.
Hidden neurons enable the network to learn abstract representations that are not
directly observable.

2. Symmetric Weights

The connection weights satisfy:

𝑊𝑖𝑗 = 𝑊𝑗𝑖

This symmetry ensures that the network follows a well-defined energy function.

3. Probabilistic Neuron Activation

Unlike deterministic networks, neurons update their states according to a probability.

The activation probability is given by the sigmoid function:

1
𝑃(𝑉𝑖 = 1) =
1 + 𝑒 −Δ𝐸/𝑇

where:

• 𝑃(𝑉𝑖 = 1)= Probability that neuron 𝑖becomes active

• Δ𝐸= Change in energy

• 𝑇= Temperature parameter

Higher probability means the neuron is more likely to become active.

4. Energy-Based Learning

The network learns by reducing the energy of correct patterns while increasing the
energy of incorrect patterns.

5. Stochastic Learning

Neuron updates are random but guided by probability, allowing the network to explore
multiple possible states.

6. Gradient-Based Learning

Training adjusts the connection weights using stochastic gradient descent or similar
optimization methods to maximize the likelihood of the training data.

7. Unsupervised Learning

The network discovers useful features without requiring labeled output data.

8. Slow Training
Because of repeated sampling and probabilistic updates, Boltzmann Machines require
significant computational time compared to simpler neural networks.

5.3 Architecture Diagram

A Boltzmann Machine consists of visible units and hidden units connected through
symmetric weights.

Basic Architecture

Visible Units

●────●────●

\ │ /

\ │/

\│/

●────●

Hidden Units

Detailed Representation

Visible Layer

V1 ●────● V2────● V3

│\ │ /│

│ \│ / │

│ \│/ │

H1 ●────● H2

Hidden Layer

Components

• Visible Units: Receive the input data.

• Hidden Units: Learn latent features.

• Symmetric Connections: Store knowledge in connection weights.

 • No Direction: Connections are bidirectional.

5.4 Energy Function

Like the Hopfield Network, the Boltzmann Machine is governed by an energy function.
However, because neuron activations are probabilistic, the network represents a
probability distribution over possible states.

The energy of a network state is defined as:

𝐸(𝑣, ℎ) = − ∑ 𝑎𝑖 𝑣𝑖 − ∑ 𝑏𝑗 ℎ𝑗 − ∑ ∑ 𝑊𝑖𝑗 𝑣𝑖 ℎ𝑗
𝑖 𝑗 𝑗
𝑖

where:

• 𝑣𝑖 = Visible unit

• ℎ𝑗 = Hidden unit

• 𝑎𝑖 = Bias of visible unit

• 𝑏𝑗 = Bias of hidden unit

• 𝑊𝑖𝑗 = Weight between visible and hidden units

Relationship Between Energy and Probability

The probability of a state is given by the Boltzmann distribution:

𝑒 −𝐸(𝑣,ℎ)
𝑃(𝑣, ℎ) =
𝑍

where:

• 𝑃(𝑣, ℎ)= Probability of the state

• 𝐸(𝑣, ℎ)= Energy of the state

• 𝑍= Partition function (normalization constant)

States with lower energy have higher probability, while high-energy states occur less
frequently.

Learning Process

The training process consists of:

 1. Apply input to visible units.

2. Hidden units activate probabilistically.

3. Compute network energy.

4. Update weights using stochastic gradient descent.

5. Repeat until the probability distribution matches the training data.

The objective is to assign low energy to observed data and high energy to unlikely
patterns.

5.5 Applications

Boltzmann Machines have many applications in machine learning and artificial

intelligence.

1. Feature Learning

Automatically extracts useful features from raw data without manual intervention.

2. Sampling

Generates new samples that resemble the training data using probabilistic sampling
techniques.

3. Deep Learning Foundations

Restricted Boltzmann Machines (RBMs) are building blocks for Deep Belief Networks
(DBNs) and influenced the development of modern deep learning.

4. Pattern Recognition

Recognizes patterns in images, speech, and text.

5. Image Processing

Used for image reconstruction, denoising, and feature extraction.

6. Recommendation Systems

Learns user preferences to recommend products, movies, or music.

7. Data Compression

Finds compact representations of large datasets.

8. Anomaly Detection

Identifies unusual or abnormal data patterns.

Advantages

• Learns complex probability distributions.

• Performs unsupervised feature learning.

• Can escape local minima using stochastic updates.

• Hidden units capture meaningful latent features.

• Forms the basis of several deep learning models.

• Effective for generative modeling.

Limitations

• Computationally expensive to train.

• Slow convergence due to repeated sampling.

• Difficult to scale to very large datasets.

• Requires careful tuning of learning parameters.

• Full Boltzmann Machines are rarely used in practice because of high

computational cost.

Conclusion

A Boltzmann Machine is a powerful stochastic recurrent neural network that learns

by modeling probability distributions over data. Unlike deterministic networks, it
updates neuron states probabilistically and uses an energy function to guide learning
toward low-energy, high-probability states. With its visible and hidden units,
symmetric weights, and stochastic learning process, the Boltzmann Machine became a
cornerstone of unsupervised learning. Its simplified form, the Restricted Boltzmann
Machine (RBM), played a major role in the evolution of Deep Belief Networks (DBNs)
and modern deep learning, making the Boltzmann Machine an important milestone in
the history of artificial intelligence.

6. INSTAR NETWORK

6.1 Introduction

An Instar Network is a simple unsupervised learning neural network used to learn

and recognize input patterns by continuously adjusting its connection weights toward
the input vector. The Instar learning rule was introduced by Stephen Grossberg as part
of his Adaptive Resonance Theory (ART). It is commonly used for pattern recognition,
clustering, and feature detection.

The main idea of an Instar Network is that when a neuron becomes active, its
connection weights move closer to the current input pattern. As learning continues,
the neuron gradually becomes specialized in recognizing similar input patterns. This
process allows the network to automatically organize and group similar data without
requiring target outputs.

Unlike supervised learning methods, the Instar Network does not need desired output
values. Instead, it learns directly from the input data by modifying the weights according
to the Instar learning rule.

The Instar learning rule is expressed as:

Δ𝑤 = 𝜂(𝑥 − 𝑤) 𝑦

where:

• Δw = Change in weight

• η (eta) = Learning rate

• x = Input vector

• w = Current weight vector

• y = Neuron activation (output)

The equation shows that the weight vector moves toward the input vector whenever the
neuron is activated. The greater the neuron activation, the larger the weight adjustment.

Features of Instar Network

• Uses unsupervised learning.

• Learns input patterns automatically.

• Adjusts weights toward the input pattern.

• Simple and computationally efficient.

• Suitable for clustering and feature extraction.

• Uses only local learning information.

6.2 Working
The Instar Network learns by modifying the weights of an activated neuron so that they
become similar to the presented input pattern.

Learning Process

1. An input vector is applied to the network.

2. The neuron computes its activation based on the input and current weights.

3. If the neuron becomes active, the Instar learning rule is applied.

4. The weights move toward the input vector.

5. Repeated presentation of similar inputs gradually stabilizes the weights.

6. The neuron becomes specialized in recognizing that group of input patterns.

Instar Learning Rule

The weight update equation is:

Δ𝑤 = 𝜂(𝑥 − 𝑤) 𝑦

where:

• η controls the learning speed.

• (x − w) represents the difference between the input and current weights.

• y determines whether learning occurs. If the neuron is not activated (y = 0), the
weights remain unchanged.

The new weight is calculated as:

𝑤new = 𝑤old + Δ𝑤

Working Example

Suppose:

• Input vector = (1, 0, 1)

• Weight vector = (0.6, 0.2, 0.5)

• Learning rate = 0.1

• Neuron activation = 1

Using the Instar rule, the weights move slightly toward the input vector after each
learning step. Repeated presentations of similar inputs gradually make the weight
vector closely match the input pattern.
Advantages of Instar Learning

• Learns continuously from incoming data.

• Adapts to changing input patterns.

• Detects similarities among input vectors.

• Produces efficient feature detectors.

6.3 Diagram

Basic Architecture

Input Pattern

x₁ x₂ x₃ x₄

│ │ │ │

└────┼────┼────┘

▼
+----------------+

| Single Neuron |

| (Instar) |

+----------------+

Weight Update

Δw = η(x − w)y

Output

Working Diagram

Input Vector

(x₁, x₂, x₃)

 │

┌─────────────┐

│ Neuron y │

└─────────────┘

Activated?

Yes ─┘

Update Weights

Toward Input

Only the activated neuron updates its weights, while inactive neurons remain
unchanged.

Applications

Instar Networks are used in several pattern recognition and machine learning
applications, including:

• Pattern recognition

• Clustering

• Feature detection

• Vector quantization

• Image processing

• Speech recognition

• Signal processing

• Adaptive learning systems

• Prototype learning

Advantages
 • Simple and easy to implement.

• Fast learning process.

• Suitable for unsupervised learning.

• Learns representative features automatically.

• Low computational complexity.

• Efficient for clustering similar input patterns.

Limitations

• Limited to simple pattern learning tasks.

• Sensitive to the learning rate.

• May not perform well on highly complex datasets.

• Cannot solve complex nonlinear classification problems alone.

• Often combined with other neural network models for advanced applications.

Conclusion

The Instar Network is an unsupervised learning neural network that learns by moving
its connection weights toward the presented input patterns. Using the Instar learning
rule,

Δ𝑤 = 𝜂(𝑥 − 𝑤) 𝑦

the network gradually adapts to recognize similar inputs and develops neurons that act
as feature detectors. Because of its simplicity, fast learning, and ability to perform
clustering and feature extraction, the Instar Network remains an important foundational
model in neural network learning and adaptive pattern recognition.

7. OUTSTAR NETWORK

7.1 Introduction

An Outstar Network is a type of unsupervised neural network used for pattern

reproduction and output pattern learning. It was introduced by Stephen Grossberg
as the counterpart of the Instar learning model. While the Instar Network learns input
patterns by adjusting incoming weights toward the input vector, the Outstar Network
learns desired output patterns by adjusting outgoing weights from an activated
neuron.

The Outstar Network is designed so that when a central neuron becomes active, it
reproduces a complete output pattern through its outgoing connections. Over repeated
learning, the outgoing weights gradually approach the desired output values. As a
result, activating the central neuron later reproduces the learned output pattern
automatically.

The Outstar learning rule is expressed as:

Δ𝑤 = 𝜂(𝑦 − 𝑤) 𝑥

where:

• Δw = Change in weight

• η (eta) = Learning rate

• y = Desired output pattern

• w = Current weight

• x = Activation of the input (central neuron)

The equation indicates that the outgoing weights move toward the desired output
whenever the input neuron is active.

Features of Outstar Network

• Performs output pattern learning.

• Opposite of the Instar Network.

• Uses unsupervised/adaptive learning.

• Learns outgoing connection weights.

• Stores and reproduces output patterns.

• Simple and computationally efficient.

7.2 Working

The Outstar Network operates by learning the desired output pattern associated with an
activated neuron.

Learning Process
 1. An input activates the central neuron.

2. The desired output pattern is presented.

3. The outgoing weights are compared with the desired output.

4. The Outstar learning rule updates the weights.

5. The weights gradually move toward the desired output values.

6. After repeated learning, activating the central neuron reproduces the stored
output pattern.

Outstar Learning Rule

The weight update equation is:

Δ𝑤 = 𝜂(𝑦 − 𝑤) 𝑥

where:

• η = Learning rate

• y = Desired output vector

• w = Current weight vector

• x = Activation of the input neuron

The updated weight becomes:

𝑤new = 𝑤old + Δ𝑤

When the input neuron is active (x = 1), the outgoing weights move toward the desired
output pattern. If the neuron is inactive (x = 0), the weights remain unchanged.

Working Example

Suppose:

• Desired output = (1, 0, 1)

• Current weights = (0.5, 0.3, 0.6)

• Learning rate = 0.1

• Input neuron activation = 1

Using the Outstar learning rule, the outgoing weights are adjusted slightly toward the
desired output. After repeated training, the stored weights closely match the desired
output pattern, allowing the network to reproduce it whenever the central neuron is
activated.

Advantages of Outstar Learning

• Learns complete output patterns.

• Produces smooth adaptive learning.

• Stores associations between neurons and output responses.

• Suitable for motor control and associative memory.

7.3 Diagram

Basic Architecture

Central Neuron

/|\

/| \

▼ ▼ ▼

● ● ●

Output Pattern Units

Detailed Representation

Input Signal

▼
+----------------+

| Central Neuron |

+----------------+

│ │ │

▼ ▼ ▼

O1 O2 O3

Output Pattern Units

Outgoing weights are updated

using the Outstar learning rule.

Applications

Outstar Networks are used in various adaptive learning and pattern reproduction tasks,
including:

• Pattern reproduction

• Associative memory

• Motor control systems

• Adaptive control

• Robotics

• Speech production models

• Pattern generation

• Artificial intelligence systems

Advantages

• Simple and easy to implement.

• Learns output patterns efficiently.

• Fast convergence for simple problems.

• Supports adaptive learning.

• Useful for pattern reproduction and motor learning.

• Low computational complexity.

Limitations

• Suitable mainly for simple output learning tasks.

• Limited ability to model highly complex data.

• Performance depends on proper learning rate selection.

 • Often combined with other neural network models for advanced applications.

• Cannot independently solve complex nonlinear learning problems.

Difference Between Instar and Outstar Networks

Feature Instar Network Outstar Network

Learning
Learns input patterns Learns output patterns
Direction

Weight Update Incoming weights Outgoing weights

Learning Rule Δ𝑤 = 𝜂(𝑥 − 𝑤)𝑦 Δ𝑤 = 𝜂(𝑦 − 𝑤)𝑥

Feature detection and Pattern reproduction and associative

Purpose
clustering output

Weight
Toward input vector Toward desired output vector
Movement

Conclusion

The Outstar Network is a neural network model that specializes in output pattern
learning and reproduction. It works as the counterpart of the Instar Network, updating
outgoing connection weights according to the Outstar learning rule:

Δ𝑤 = 𝜂(𝑦 − 𝑤) 𝑥

When the central neuron is activated, it gradually learns to reproduce the desired
output pattern through its outgoing connections. Due to its simplicity and adaptive
learning capability, the Outstar Network is widely used in associative memory, motor
control, robotics, pattern generation, and adaptive control systems. Together with
the Instar Network, it forms an important foundation for understanding adaptive neural
learning mechanisms.

8. ART NETWORK (Adaptive Resonance Theory)

8.1 Introduction

Adaptive Resonance Theory (ART) is a family of unsupervised neural networks

proposed by Stephen Grossberg in collaboration with Gail Carpenter. ART networks are
designed to recognize patterns, classify data, and learn continuously without forgetting
previously learned information.
One of the major advantages of ART networks is their ability to solve the stability–
plasticity dilemma, which is a common problem in neural networks.

• Stability means retaining previously learned knowledge.

• Plasticity means learning new patterns from incoming data.

Most neural networks suffer from catastrophic forgetting, where learning new
information destroys previously learned knowledge. ART networks overcome this
problem by creating new categories whenever the input pattern differs significantly
from existing categories while preserving old memories.

ART networks compare incoming patterns with stored prototype patterns. If the
similarity is sufficiently high, the network enters a resonance state and updates the
existing category. Otherwise, it creates a new category to represent the unfamiliar
pattern.

Because of this adaptive learning mechanism, ART networks are widely used in pattern
recognition, clustering, image processing, speech recognition, robotics, and real-
time learning systems.

Stability–Plasticity Dilemma

The ART network successfully balances two important learning requirements:

Stability

• Retains previously learned categories.

• Prevents destruction of old knowledge.

• Ensures reliable long-term memory.

Plasticity

• Learns new input patterns.

• Creates new categories when necessary.

• Adapts continuously to changing environments.

Features of ART Networks

• Unsupervised learning.

• Incremental (online) learning.

• Solves catastrophic forgetting.

• Creates new categories automatically.

• Uses competitive learning.

 • Suitable for clustering and classification.

8.2 Architecture

The ART network consists of different versions depending on the type of input data.

ART1

• Works with binary input patterns (0 and 1).

• Used for binary pattern classification.

• Simpler architecture.

ART2

• Works with continuous (analog) input patterns.

• Suitable for real-valued data.

• More complex than ART1.

Main Components of ART Network

The ART architecture contains the following components:

1. F1 Layer (Feature Layer)

The Feature Layer (F1) receives the input pattern.

Functions:

• Accepts input vectors.

• Preprocesses the input.

• Compares input with stored prototypes.

• Sends activation signals to the F2 layer.

2. F2 Layer (Category Layer)

The Category Layer (F2) stores learned categories.

Functions:

• Performs competitive learning.

• Selects the best matching category.

 • Stores prototype patterns.

• Creates new categories when required.

3. Gain Control Unit

The gain control mechanism regulates signal flow between layers.

Functions:

• Controls neuron activation.

• Prevents unnecessary updates.

• Stabilizes learning.

4. Reset Mechanism

The reset mechanism ensures that only sufficiently similar patterns are accepted.

Functions:

• Measures similarity.

• Rejects poor matches.

• Searches for another category.

• Creates a new category if no suitable match exists.

5. Vigilance Parameter (ρ)

The vigilance parameter determines how similar the input must be to an existing
category.

• High vigilance (ρ close to 1):

o More categories created.

o High classification accuracy.

o Strict matching.

• Low vigilance (ρ close to 0):

o Fewer categories.

o More generalized classification.

 o Flexible matching.

8.3 Working

The operation of an ART network consists of repeated comparison and learning.

Step 1: Input Pattern

A binary (ART1) or continuous (ART2) input pattern is applied to the F1 layer.

Step 2: Feature Processing

The F1 layer processes the input and forwards it to the F2 layer.

Step 3: Category Selection

The F2 layer compares the input with stored prototype patterns.

The category having the highest similarity becomes the winning neuron.

Step 4: Vigilance Test

The winning category is checked using the vigilance parameter.

• If similarity ≥ vigilance threshold:

o Resonance occurs.

o Learning continues.

• If similarity < vigilance threshold:

o Reset signal is generated.

o The category is rejected.

Step 5: Resonance State

If the similarity is acceptable, the network enters the resonance state.

During resonance:

• Weights are updated.

• Prototype pattern is improved.

 • Learning occurs.

Step 6: New Category Creation

If no stored category satisfies the vigilance criterion, the network automatically creates
a new category for the input pattern.

This allows ART networks to learn continuously without overwriting previously learned
information.

Working Algorithm

1. Input pattern enters the F1 layer.

2. F1 sends the pattern to the F2 layer.

3. F2 finds the best matching category.

4. Vigilance parameter measures similarity.

5. If similarity is sufficient → Resonance.

6. If similarity is insufficient → Reset.

7. Search for another category.

8. If no category matches → Create a new category.

9. Store the learned prototype.

8.4 Diagram

Basic Architecture

+----------------+

Input Pattern -->| F1 Layer |------+

| (Feature Layer)| |

+----------------+ |

| |

▼ |

+----------------+ |

| F2 Layer |------+
 |(Category Layer)|

+----------------+

Output Category

+----------------+

| Reset Mechanism|

+----------------+

Simplified ART Architecture

Input

+-------------+

| F1 Layer |

+-------------+

+-------------+

| F2 Layer |

+-------------+

Output Category

│
Reset Mechanism

(Vigilance Check)

Applications

ART networks are widely used in machine learning and artificial intelligence
applications.

1. Pattern Recognition

Recognizes and classifies input patterns without supervision.

2. Clustering

Groups similar data into meaningful categories.

3. Image Processing

Used for image segmentation, object recognition, and image classification.

4. Speech Recognition

Identifies speech patterns and speaker characteristics.

5. Medical Diagnosis

Classifies diseases and medical records based on patient data.

6. Robotics

Supports adaptive learning and real-time decision-making.

7. Data Mining

Discovers hidden patterns in large datasets.

8. Anomaly Detection

Detects unusual or abnormal observations in data.

Advantages

• Solves the stability–plasticity dilemma.

• Learns continuously without catastrophic forgetting.

• Automatically creates new categories.

• Performs online (incremental) learning.

 • Suitable for supervised and unsupervised extensions.

• Robust to changing environments.

Limitations

• Performance depends on the vigilance parameter.

• High vigilance may create too many categories.

• Low vigilance may merge different patterns into one category.

• ART2 architecture is more complex than ART1.

• Requires careful parameter tuning.

Difference Between ART1 and ART2

Feature ART1 ART2

Input Type Binary data Continuous (analog) data

Data Values 0 and 1 Real-valued inputs

Complexity Simple More complex

Applications Binary pattern recognition Continuous pattern recognition

Processing Speed Faster Slightly slower

Conclusion

The Adaptive Resonance Theory (ART) Network is a powerful unsupervised learning

model developed by Stephen Grossberg to address the stability–plasticity dilemma.
By combining the F1 Feature Layer, F2 Category Layer, vigilance parameter, and
reset mechanism, ART networks can continuously learn new patterns while preserving
previously acquired knowledge. The ability to automatically create new categories
without catastrophic forgetting makes ART networks highly effective for pattern
recognition, clustering, image processing, speech recognition, robotics, and real-
time adaptive learning, making them one of the most significant neural network
architectures in unsupervised learning.
9. KOHONEN SELF-ORGANIZING MAP (SOM)

9.1 Introduction

The Kohonen Self-Organizing Map (SOM) is an unsupervised neural network

developed by Teuvo Kohonen in the early 1980s. It is a type of competitive learning
network that organizes high-dimensional input data into a two-dimensional (2D)
topographic map while preserving the relationships among the input patterns.

Unlike supervised neural networks, a Self-Organizing Map does not require target
outputs. Instead, it automatically discovers similarities in the input data and groups
similar patterns into neighboring neurons on the map. This makes SOM one of the most
effective neural network models for clustering, visualization, and dimensionality
reduction.

The SOM consists of a grid of neurons, where each neuron has an associated weight
vector. When an input vector is presented, all neurons compete to determine which
one most closely matches the input. The neuron with the smallest distance to the input
is called the Best Matching Unit (BMU) or winning neuron.

After the BMU is selected, not only is its weight updated, but the weights of its
neighboring neurons are also adjusted. This neighborhood learning allows the network
to preserve the topological structure of the input data, ensuring that similar input
patterns are mapped to nearby neurons.

Because of its ability to convert complex, high-dimensional data into a simple two-
dimensional representation, the Kohonen SOM is widely used in data mining, pattern
recognition, speech processing, image analysis, and market research.

Main Uses of SOM

• Data clustering

• Data visualization

• Dimensionality reduction

• Feature extraction

• Pattern recognition

9.2 Characteristics

The Kohonen Self-Organizing Map has several unique characteristics.

1. Unsupervised Learning
The SOM learns directly from input data without requiring target outputs or labeled
examples.

2. Competitive Learning

All neurons compete to represent the input pattern.

Only the neuron that best matches the input becomes the Best Matching Unit (BMU).

3. Grid-Based Organization

Neurons are arranged in a regular grid, usually:

• One-dimensional

• Two-dimensional

• Hexagonal

• Rectangular

The two-dimensional grid is the most commonly used structure.

4. Best Matching Unit (BMU)

The neuron whose weight vector is closest to the input vector becomes the winner.

The Euclidean distance is commonly used:

𝑑 = √∑( 𝑥𝑖 − 𝑤𝑖 )2
𝑖

where:

• 𝑥𝑖 = Input vector

• 𝑤𝑖 = Weight vector

The neuron with the minimum distance is selected as the BMU.

5. Neighborhood Function

Unlike ordinary competitive learning, the neighboring neurons around the BMU are also
updated.

This preserves the topology of the input space and creates smooth clusters.

6. Topology Preservation

Input patterns that are similar remain close together on the map.

7. Adaptive Learning
Learning rate and neighborhood size decrease gradually during training, leading to
stable convergence.

9.3 SOM Architecture Diagram

The architecture of a Kohonen Self-Organizing Map consists of an input layer connected

to a competitive grid of neurons.

Basic Architecture

Input Vector

+--------------------+

| Weight Vector Bank |

+--------------------+

Competition Stage

Best Matching Unit

(Winner)

Neighborhood Update

Grid Representation

Input Layer

x1 x2 x3 x4

\ || /

\| |/

\| |/
 +-------------+

| SOM Grid |

|●●●● |

|●●●● |

|●●●● |

+-------------+

Winning neuron and its neighbors

update their weights.

Components

• Input Layer: Receives input vectors.

• Weight Vector Bank: Stores neuron weights.

• Competition Stage: Finds the BMU.

• Neighborhood Function: Updates nearby neurons.

• Output Grid: Forms a two-dimensional map.

9.4 Learning Steps

The SOM learning algorithm consists of repeated competition and adaptation.

Step 1: Initialize Weights

Assign small random values to the weight vectors of all neurons.

Step 2: Present Input Pattern

An input vector is applied to the network.

Step 3: Compute Similarity

Calculate the distance between the input vector and every neuron's weight vector.

The Euclidean distance is usually used.

Step 4: Select the BMU

Choose the neuron with the minimum distance.

This neuron becomes the Best Matching Unit (BMU).

Step 5: Update Weights

Update both:

• BMU

• Neighboring neurons

Weight update equation:

𝑤(𝑡 + 1) = 𝑤(𝑡) + 𝛼(𝑡) ℎ(𝑡) [𝑥 − 𝑤(𝑡)]

where:

• 𝑤(𝑡)= Current weight

• 𝑥= Input vector

• 𝛼(𝑡)= Learning rate

• ℎ(𝑡)= Neighborhood function

Step 6: Reduce Learning Rate

Gradually decrease:

• Learning rate

• Neighborhood radius

This improves convergence.

Step 7: Repeat

Repeat the above steps for all training samples until the weight vectors stabilize.

SOM Learning Algorithm

1. Initialize neuron weights randomly.

 2. Present an input vector.

3. Compute similarity between the input and all neurons.

4. Select the Best Matching Unit (BMU).

5. Update the BMU and neighboring neurons.

6. Reduce learning rate and neighborhood size.

7. Repeat until convergence.

9.5 Applications

The Kohonen Self-Organizing Map is widely used in machine learning and data analysis.

1. Data Clustering

Groups similar data into meaningful clusters without supervision.

2. Speech Recognition

Recognizes speech patterns and phonemes.

3. Facial Feature Mapping

Identifies and organizes facial characteristics for recognition systems.

4. Market Segmentation

Groups customers based on purchasing behavior and demographics.

5. Image Processing

Performs image segmentation, compression, and visualization.

6. Medical Diagnosis

Clusters patient records and disease patterns.

7. Data Visualization

Converts high-dimensional data into a two-dimensional map for easy interpretation.

8. Anomaly Detection

Detects unusual patterns in network traffic, finance, and industrial systems.

9. Bioinformatics

Analyzes gene expression data and protein classification.

10. Robotics
Supports environment mapping and autonomous navigation.

Advantages

• Performs unsupervised learning.

• Preserves the topology of input data.

• Reduces high-dimensional data into a 2D map.

• Excellent for clustering and visualization.

• Learns continuously from data.

• Easy to interpret graphically.

• Robust to noisy input patterns.

Limitations

• Training may require many iterations.

• Sensitive to the initial weight values.

• Choosing the neighborhood size is important.

• Performance decreases with poor parameter selection.

• Not suitable for supervised classification tasks.

Difference Between SOM and ART Network

Feature Kohonen SOM ART Network

Learning Type Unsupervised Unsupervised

Main Purpose Clustering and visualization Adaptive pattern classification

Structure 2D neuron grid F1 and F2 layers

Learning Method Competitive learning Resonance learning

Organized through neighborhood Created automatically using

New Categories
updates vigilance
Feature Kohonen SOM ART Network

Topology
Yes No
Preservation

Conclusion

The Kohonen Self-Organizing Map (SOM) is a powerful unsupervised neural network

developed by Teuvo Kohonen for clustering, visualization, and dimensionality
reduction. By using competitive learning, the Best Matching Unit (BMU), and the
neighborhood function, the network organizes complex, high-dimensional data into a
meaningful two-dimensional topographic map while preserving the relationships
among data points. Due to its ability to discover hidden patterns and provide intuitive
visual representations, the SOM is widely used in data clustering, speech recognition,
facial feature mapping, market segmentation, image processing, bioinformatics,
and data visualization, making it one of the most influential neural network models for
unsupervised learning.