Tutorial-1
Evaluation of Various Activation Functions
Aim
To implement and evaluate various activation functions used in artificial neural networks.
Theory
Activation functions play a crucial role in artificial neural networks. A neuron in a neural network
receives inputs, multiplies them with weights, sums them, and passes the result through an activation
function to produce the final output. Without activation functions, the neural network would behave
like a simple linear model regardless of the number of layers used.
Activation functions introduce non-linearity into the model, allowing neural networks to learn
complex patterns and relationships present in real-world data. This capability is essential for solving
tasks such as image recognition, speech processing, pattern classification, and natural language
processing.
Different types of activation functions are used depending on the type of neural network and the
problem being solved. Some of the most commonly used activation functions include:
1. Binary Step Function
The binary step function is one of the simplest activation functions. It outputs either 0 or 1 depending
on whether the input value crosses a threshold. It is commonly used in perceptron models.
Although simple, this function is rarely used in deep neural networks because it is not differentiable,
which makes gradient-based learning difficult.
2. Sigmoid Function
The sigmoid activation function maps any input value into the range 0 to 1. It produces an S-shaped
curve.
Sigmoid functions are widely used in binary classification problems and output layers of
neural networks.
Source Code:-
Output:-
Tutorial- 2
Kohonen Self Organizing Map (SOM) for Clustering
Aim
To write a program for training and testing of Kohonen Self Organizing Map for clustering
application.
Theory
A Self Organizing Map (SOM) is an unsupervised neural network introduced by Teuvo Kohonen. It
is used for clustering, visualization, and dimensionality reduction of high-dimensional data.
Unlike supervised learning algorithms, SOM does not require labeled data. Instead, it organizes data
based on similarity using competitive learning.
In SOM, neurons compete with each other to represent input data. The neuron whose weight vector is
closest to the input vector is called the Best Matching Unit (BMU) or winning neuron.
The training process follows these steps:
1. Initialize weight vectors randomly.
2. Present an input vector to the network.
3. Calculate the distance between the input and each neuron weight.
4. Select the neuron with minimum distance as the winner.
5. Update the winner's weight to move closer to the input vector.
Weight update rule:
SOM is widely used in:
● Data clustering
● Pattern recognition
● Image processing
● Market segmentation
● Data visualization
In this experiment, a simple SOM model is implemented to cluster input patterns.
Explanation
1. Input dataset contains 2 clusters of points.
2. Random weights represent neurons.
3. Euclidean distance is calculated between input and weights.
4. The neuron with minimum distance becomes the winning neuron.
5. The weight of the winning neuron is updated to move closer to the input vector.
6. After several iterations, clusters are formed.
Output
Final Weights:
[[0.18 0.19]
[0.86 0.87]]
Interpretation:
Cluster 1 → points near (0.1,0.2)
Cluster 2 → points near (0.9,0.8)
Source code:-
Output:-
Tutorial- 3
Fuzzy Set Operations (AND, OR, De Morgan's Theorem)
Aim
To implement fuzzy set operations AND, OR and verify De Morgan’s theorem.
Theory
In fuzzy logic, elements belong to a set with a degree of membership between 0 and 1.
A fuzzy set is defined as:
where μA(x) represents the membership value.
Important fuzzy operations include:
This theorem verifies the consistency of fuzzy set operations.
Fuzzy set operations are used in:
● Fuzzy controllers
● Decision making systems
● Expert systems
● Pattern recognition
Explanation
1. Two fuzzy sets A and B are defined.
2. Intersection is calculated using minimum function. 3.
Union is calculated using maximum function. 4.
Complement is calculated using 1 − membership value. 5.
De Morgan’s theorem is verified by comparing both sides.
Source code:-
Output :-
Tutorial - 4
Fuzzy Relations and Composition
Aim
To write a program to find fuzzy relations and their compositions.
Theory
A fuzzy relation describes the relationship between elements of two fuzzy sets. It is represented using
a membership matrix.
Example:
Each value represents the degree of relation between elements of the sets.
One of the most important operations on fuzzy relations is composition.
The most commonly used rule is the Max-Min composition.
Where:
R → relation between A and B
S → relation between B and C
T → relation between A and C
Fuzzy relation composition is used in:
● Fuzzy inference systems
● Control systems
● Expert systems
● Decision support systems
Explanation
1. Two fuzzy relations R and S are defined as matrices.
2. A new matrix T is created to store the result.
3. For each element of T, the max-min composition rule is applied.
4. The final matrix represents the composed fuzzy relation.
Interpretation:
The resulting matrix represents the combined relationship between the two fuzzy relations.
Source code:-
Output:-