0% found this document useful (0 votes)
24 views6 pages

Soft Computing: Neural Network Insights

The document discusses Kohonen self-organizing neural networks, including their architecture, properties, and algorithms. It also covers adaptive resonance theory (ART) neural networks, describing different ART types and their basic architecture.

Uploaded by

Nitesh Nema
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
24 views6 pages

Soft Computing: Neural Network Insights

The document discusses Kohonen self-organizing neural networks, including their architecture, properties, and algorithms. It also covers adaptive resonance theory (ART) neural networks, describing different ART types and their basic architecture.

Uploaded by

Nitesh Nema
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

McCulloch-Pitts Neuron Model Numerical questions follow this link

([Link]

Question - Implement AND function using McCulloch-Pitts Neuron (take binary data).
OR
Question - Implement XOR function using McCulloch-Pitts Neuron (take binary data).

KOHONEN SELF ORGANISING NEURAL NETWORK


Artificial Neural Network (ANN) is based on a collection of connected nodes called artificial
neurons, which models the neurons like a human brain. The self-organizing map (SOM) or
Kohonen Network is a type of Artificial Neural Network that is trained by a unsupervised
learning .

Kohonen Self-Organizing Neural Network:


Kohonen Map or Self-Organizing Map (SOM) is a type of neural network. This was developed
by Tuevo Kohonen in 1982. The name Self-organizing because it does not require supervision.
This follows an unsupervised learning approach and its network is trained through a competitive
learning algorithm.
The major characteristic of this algorithm is that the input data that are close in high dimensional
data are mapped to the nearby nodes in the 2 dimensional spaces (2D). This technique is used in
the method called dimensionality reduction, as it maps the high-dimension input to the low
discredited representation. The advantage is the nodes are self-organizing, so that supervision is
not needed.
Feature Maps: - The self-organizing maps (SOM) are also called as feature maps, as they are
retaining the feature of the input data, and training to define the similarities between the nodes.
This makes SOM useful for visualization by creating low-dimensional views of high-dimension
and even representing their relationships between them.
Vector quantization: - Vector quantization is one of the properties of Self-Organizing Maps,
which is a compression technique. It provides a way to represent multi-dimensional data in a
lower dimensional space typically in one or two dimensions. The SOM utilizes competitive
learning instead of error correction, to modify the weights. It implies only on an individual node.
Let’s discuss about the architectural structure of self-organizing neural network.

Architecture of Self-Organizing Maps:


The self-organizing Neural Network differs from the ANNs both in architectural and algorithmic
properties.

This self-organizing neural network consists of a single layer linear 2D grid of neurons, rather
than a series of layers. All the nodes on this lattice are associated directly to the input vector. The
SOM network consists of 2 layers input layer and the output layer.
The weight gets updated on the basis of weights as a function of the input data. The grid itself
maps the coordinates at each iteration as a function of the input data. Only single node is
activated at each iteration in which the features of an instance of the input vector are presented to
the neural network as all nodes respond to the input.
Stages of operations:
The function of self-organizing neural network is divided into three stages:

Construction: - The self-organizing network consists of few basic elements. The input signals
are stimulated in a matrix of neurons. These signals are grouped and transferred to every neuron.
Learning: - This mechanism defines the similarities between the every neurons and the input
signal. This assigns the neurons with shortest distance as the winner. At the start of process the
wages are of small random numbers, after learning those wages are modified and show the
internal structure of input data.
Identification:- Thus at the final stage the net values of winning neurons and its neighbors are
get adapted and the net topology is defined by determining the neighbors of every input neurons.

Properties:
Some of the properties to be known are:
Best Matching Unit (BMU):The node is chosen by determining the between the current
input values and all the nodes in the network.
Distance from input= i=0i-n(Ii-Wi)2
Where I- current input vector
W- Node’s weight vector
N=number of weights
Algorithm:
Step:1
Initialize the weights wij. And initialize random weights
Step:2
Choose a random input vector x.
Step:3
Repeat steps 4 and 5 for all nodes on the map.
Step:4
Calculate the Euclidean distance between weight vector wij and the input vector x(t), and
calculate the square of the distance
Step:5
Track the node that generates the smallest distance t and generate the winning weight using
formula.
Step:6
Calculate the overall Best Matching Unit (BMU). It means the node with the smallest distance
from all calculated ones.
Step:7
Discover topological neighborhood of BMU in Kohonen Map.
Step:8
Repeat for all nodes in the BMU neighborhood:
Update the winning weight of the first node in the neighborhood of the BMU by including a
fraction of the difference between the input vector x(t) and the weight w(t) of the neuron.
Step:9
Repeat the complete iteration until reaching the iteration.
Here, step 1 represents initialization phase, while step 2 to 9 represents the training phase.

Advantages:
 It is easily interpreted and understood.

 The reduction of dimensionality.

 Grid clustering makes it easy to observe similarities in the data.

Disadvantages:
 It does not build a generative model for the data

 It relies on a predefined distance in feature space (a problem shared by most clustering


algorithms, to be fair)

 The magnification factors not well understood (at least to my best knowledge)

 The 1D (proven) topological ordering property does not extend to 2D

 slow training, hard to train against slowly evolving data

 It not so intuitive : neurons close on the map (topological proximity) may be far away in
feature space

 It does not behave so gently when using categorical data, even worse for mixed data
Adaptive Resonance Theory (ART)

Adaptive resonance theory is a type of neural network technique developed by Stephen


Grossberg and Gail Carpenter in 1987. The basic ART uses unsupervised learning technique.
The term “adaptive” and “resonance” used in this suggests that they are open to new
learning(i.e. adaptive) without discarding the previous or the old information(i.e. resonance). The
ART networks are known to solve the stability-plasticity dilemma i.e., stability refers to their
nature of memorizing the learning and plasticity refers to the fact that they are flexible to gain
new information. ART networks implement a clustering algorithm. Input is presented to the
network and the algorithm checks whether it fits into one of the already stored clusters.

Types of Adaptive Resonance Theory (ART)

Carpenter and Grossberg developed different ART architectures as a result of 20 years of


research. The ARTs can be classified as follows:
 ART1 – It is the simplest and the basic ART architecture. It is capable of clustering binary
input values.
 ART2 – It is extension of ART1 that is capable of clustering continuous-valued input data.
 Fuzzy ART – It is the augmentation of fuzzy logic and ART.
 ARTMAP – It is a supervised form of ART learning where one ART learns based on the
previous ART module. It is also known as predictive ART.
 FARTMAP – This is a supervised ART architecture with Fuzzy logic included.

Basic of Adaptive Resonance Theory (ART) Architecture

The adaptive resonant theory is a type of neural network that is self-organizing and competitive.
It can be of both types, the unsupervised ones(ART1, ART2, ART3, etc) or the supervised
ones(ARTMAP). Generally, the supervised algorithms are named with the suffix “MAP”.
But the basic ART model is unsupervised in nature and consists of :
 F1 layer or the comparison field(where the inputs are processed)
 F2 layer or the recognition field (which consists of the clustering units)
 The Reset Module (that acts as a control mechanism)
The F1 layer accepts the inputs and performs some processing and transfers it to the F2 layer that
best matches with the classification factor.

There exist two sets of weighted interconnection for controlling the degree of similarity between
the units in the F1 and the F2 layer.

The F2 layer is a competitive [Link] cluster unit with the large net input becomes the
candidate to learn the input pattern first and the rest F2 units are ignored.

The reset unit makes the decision whether or not the cluster unit is allowed to learn the input
pattern depending on how similar its top-down weight vector is to the input vector and to the
decision. This is called the vigilance test.
Thus we can say that the vigilance parameter helps to incorporate new memories or new
information. Higher vigilance produces more detailed memories, lower vigilance produces more
general memories.

Advantage of Adaptive Resonance Theory (ART)


 It exhibits stability and is not disturbed by a wide variety of inputs provided to its network.
 It can be integrated and used with various other techniques to give more good results.
 It can be used for various fields such as mobile robot control, face recognition, land cover
classification, target recognition, medical diagnosis, signature verification, clustering web
users, etc.
 It has got advantages over competitive learning (like bpnn etc). The competitive learning
lacks the capability to add new clusters when deemed necessary.
 It does not guarantee stability in forming clusters.

Limitations of Adaptive Resonance Theory


Some ART networks are inconsistent (like the Fuzzy ART and ART1) as they depend upon the
order in which training data, or upon the learning rate.

Common questions

Powered by AI

The learning mechanism in Adaptive Resonance Theory (ART) differs from traditional competitive learning models by incorporating mechanisms that resolve the stability-plasticity problem. Traditional competitive learning updates neurons based on a winner-take-all approach within a layer, which can lead to overwriting past knowledge when new data is introduced. In contrast, ART utilizes a vigilance parameter to control how closely new inputs must match current patterns before modifying existing structures or forming new categories . This allows ART networks to learn new information adaptively without forgetting stable patterns. Additionally, ART employs a reset mechanism that dynamically recalibrates the network, ensuring that learning is both plastic and stable as new data patterns emerge .

Self-Organizing Maps (SOM) use a single-layer, linear 2D grid of neurons trained with competitive learning to map high-dimensional data to a lower-dimensional space without any supervision . In contrast, Adaptive Resonance Theory (ART) networks typically involve multiple layers, such as F1 and F2 layers for comparison and recognition fields, respectively, as well as a reset module to control learning based on a vigilance parameter . ART networks solve the stability-plasticity dilemma by maintaining consistent learning while adapting to new data inputs by adding new units if necessary, unlike SOMs, which generally rely on preset grid sizes .

Adaptive Resonance Theory offers several advantages in machine learning applications, including the ability to maintain stability without being disrupted by varying inputs, making it suitable for dynamic environments . ART networks integrate well with other techniques, enhancing their performance across diverse applications such as mobile robot control, medical diagnosis, and pattern recognition. The ability to adjust vigilance allows ART to create either general or detailed memories, providing flexibility in learning and clustering tasks . Additionally, ART networks can solve the stability-plasticity dilemma by dynamically updating categories with new information without losing existing memory, a challenge in traditional neural networks .

Adaptive Resonance Theory (ART) achieves stability and plasticity through its unique architecture and mechanisms, such as the vigilance parameter and the reset module. Stability is maintained as the network preserves learned information over time, resisting disruptive changes from new data inputs . Plasticity, on the other hand, is achieved by allowing the network to adapt to new information through the vigilance parameter, which controls the specificity of category recognition. High vigilance results in more specific categories and new node creation, while low vigilance allows for broader generalization . Additionally, the reset mechanism ensures that only suitable neurons learn the new patterns, contributing to effective adaptability without losing stability .

The Kohonen Self-Organizing Map (SOM) approaches dimensionality reduction by mapping high-dimensional input data to a two-dimensional grid of neurons using a competitive learning algorithm. This is done by calculating which neuron of the grid is the closest to an input vector, thus preserving the topological properties of the input space. The implication for data visualization is significant as it reduces complexity, allowing patterns and structures in high-dimensional space to be observed more easily in a two-dimensional space. This makes SOMs particularly useful for visualization because they create a low-dimensional view while retaining information about similarities between nodes .

Kohonen Self-Organizing Maps' limitations include their reliance on a predefined distance in feature space, a common issue with many clustering algorithms . They do not build a generative model for data, and their magnification factors are not well understood, limiting their explanatory power. Additionally, the topological ordering property does not extend effectively from 1D to 2D spaces. SOMs also face challenges with slow training, especially with slowly evolving data, and have difficulty handling categorical or mixed data types effectively .

The Best Matching Unit (BMU) in a Kohonen Self-Organizing Map is the neuron whose weight vector is closest to the current input vector, as determined by the smallest Euclidean distance. The identification of the BMU is crucial as it determines how the weights of the neurons are updated; the BMU, along with its neighbors, will have its weights adjusted towards the input vector, which is a fundamental operation of competitive learning in SOM. This process helps the network to organize itself by mapping similar input vectors to nearby neurons on the grid, thereby creating a topology that reflects the input space's structure .

Using categorical and mixed data in Kohonen Self-Organizing Maps presents significant challenges primarily because SOMs are modeled to work with numerical data where Euclidean distances are meaningful. The algorithm's reliance on these distances for determining node associations and weight updates does not translate well for categorical data, which lacks a clear magnitude or order . Furthermore, when mixed data types are involved, it becomes difficult to unify the inputs into a single metric space, complicating the competitive algorithm's ability to create accurate input representations. This leads to potential distortions in the resulting topological map, affecting the reliability of the patterns and clusters formed by the SOM .

One might prefer using an Adaptive Resonance Theory (ART) network over a Kohonen Self-Organizing Map (SOM) when dealing with applications requiring stability and the ability to learn new patterns dynamically without disrupting already established categories. ART networks maintain stability while accommodating new data, making them ideal for applications where input conditions can change over time, such as mobile robot control or adaptive systems . ART's ability to maintain different resolutions of memory through the vigilance parameter also allows for more flexible adaptation than SOMs, which have a fixed topology and might struggle with dynamic data. Therefore, ART is preferable in environments needing robust category adaptation and maintenance of detailed historical data .

Vector quantization in the Kohonen Self-Organizing Map is used as a compression technique that transforms multi-dimensional data into lower-dimensional representations, typically visualized on a 2D grid . The process involves each input vector being assigned to the nearest neuron, or the Best Matching Unit (BMU), based on the smallest Euclidean distance between them. This allows the SOM to reduce the dimensionality of the data, retaining essential structures and relationships while discarding redundancy. The importance of this process lies in its ability to simplify complex datasets, making it easier to visualize and interpret patterns or clusters in large-scale data environments, which is particularly helpful in applications like feature mapping and clustering .

You might also like