0% found this document useful (0 votes)
19 views31 pages

Activation Functions in Neural Networks

The document serves as a lab manual for soft computing experiments, detailing various activation functions used in neural networks, including Sigmoid, Tanh, ReLU, Softmax, and Leaky ReLU. It also covers the implementation of a single perceptron for binary classification and an M-Pits neuron for the AND gate, providing code examples and explanations for each. The manual emphasizes the importance of activation functions in learning complex patterns and includes practical coding exercises using Python and relevant libraries.

Uploaded by

anveshasingh1910
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
19 views31 pages

Activation Functions in Neural Networks

The document serves as a lab manual for soft computing experiments, detailing various activation functions used in neural networks, including Sigmoid, Tanh, ReLU, Softmax, and Leaky ReLU. It also covers the implementation of a single perceptron for binary classification and an M-Pits neuron for the AND gate, providing code examples and explanations for each. The manual emphasizes the importance of activation functions in learning complex patterns and includes practical coding exercises using Python and relevant libraries.

Uploaded by

anveshasingh1910
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

Soft Computing lab experiment

LAB MANUAL

Experiment 1
Plot Different types of activation Functions.
Solution :- Activation function decides whether a neuron should be activated or not by
calculating the weighted sum and further adding bias to it. It introduces non-linearity to the
model, allowing it to learn complex patterns and relationships in the data. Activation functions
determine the output of a node (or neuron) in a neural network, and they play a crucial role in the
learning process.
Here are some common types of activation functions:
1. Sigmoid Function:
 Formula: A = 1/(1 + e-x)
 Nature : Non-linear. Notice that X values lies between -2 to 2, Y values are very
steep. This means, small changes in x would also bring about large changes in
the value of Y.
 It is a function which is plotted as ‘S’ shaped graph.
 Range: (0, 1)
 Use Case: Commonly used in output layer of a binary classification, where
result is either 0 or 1, as value for sigmoid function lies between 0 and 1 only so,
result can be predicted easily to be 1 if value is greater than 0.5 and 0 otherwise.
2. Hyperbolic Tangent (tanh) Function: The activation that works almost always better
than sigmoid function is Tanh function also known as Tangent Hyperbolic function. It’s
actually mathematically shifted version of the sigmoid function. Both are similar and
can be derived from each other.
 Formula: f(x) = tanhx = 2/(1 + e-2x)-1
 Nature :- non-linear
 Range: (-1, 1)
 Use Case: Similar to the sigmoid, tanh is often used in the hidden layers of a
neural network hence the mean for the hidden layer comes out be 0 or very
close to it, hence helps in centering the data by bringing mean close to 0. This
makes learning for the next layer much easier.
3. Rectified Linear Unit (ReLU): It Stands for Rectified linear unit. It is the most
widely used activation function. Chiefly implemented in hidden layers of Neural
network.
 Formula: A(x) = max(0,x). It gives an output x if x is positive and 0 otherwise.
 Range: [0, +∞(inf))
 Nature :- non-linear, which means we can easily backpropagate the errors
and have multiple layers of neurons being activated by the ReLU function.
 Use Case: ReLu is less computationally expensive than tanh and sigmoid because
it involves simpler mathematical operations. At a time only a few neurons are
activated making the network sparse making it efficient and easy for
computation.

4. Softmax Function:-
The softmax function is also a type of sigmoid function but is handy when we are trying to handle
multi- class classification problems.
Nature :- non-linear
Uses :- Usually used when trying to handle multiple classes. the softmax function was commonly
found in the output layer of image classification [Link] softmax function would
squeeze the outputs for each class between 0 and 1 and would also divide by the sum of the
outputs.
Output:- The softmax function is ideally used in the output layer of the classifier where we are
actually trying to attain the probabilities to define the class of each input.
The basic rule of thumb is if you really don’t know what activation function to use, then simply use
RELU as it is a general activation function in hidden layers and is used in most cases these days.
If your output is for binary classification then, sigmoid function is very natural choice for output
layer.
If your output is for multi-class classification then, Softmax is very useful to predict the probabilities
of each classes.
Leaky Rectified Linear Unit (Leaky ReLU) Activation Function:
Advantages:
Addresses the "dying ReLU" problem where neurons could become inactive during training.
Allows a small, non-zero gradient for negative inputs, preventing neurons from being completely
turned off.

Software: Python with libraries:


 TensorFlow and Keras for neural networks.
 Scikit-fuzzy for fuzzy logic.
 DEAP for evolutionary algorithms.
 Pandas, NumPy, and SciPy for general scientific computing tasks.

Code:- import numpy as np


import [Link] as plt

# Define the activation functions


def sigmoid(x):
return 1 / (1 + [Link](-x))

def tanh(x):
return [Link](x)
def relu(x):
return [Link](0, x)

def softmax(x):
exp_x = [Link](x - [Link](x))
return exp_x / exp_x.sum(axis=0)

# Generate x values
x = [Link](-5, 5, 100)

# Plot sigmoid activation


function [Link](figsize=(12,
4))
[Link](1, 4, 1)
[Link](x, sigmoid(x),
label='Sigmoid') [Link]('Sigmoid
Activation Function') [Link]()

# Plot tanh activation function


[Link](1, 4, 2)
[Link](x, tanh(x), label='Tanh')
[Link]('Hyperbolic Tangent Activation Function')
[Link]()

# Plot ReLU activation function


[Link](1, 4, 3)
[Link](x, relu(x), label='ReLU')
[Link]('Rectified Linear Unit (ReLU) Activation Function')
[Link]()

# Plot softmax activation function (for multiple classes)


x_softmax = [Link]([1, 2, 3])
[Link](1, 4, 4)
[Link](range(len(x_softmax)), softmax(x_softmax))
[Link]('Softmax Activation Function')
[Link](range(len(x_softmax)), ['Class 1', 'Class 2', 'Class 3'])

# Adjust layout and show the plots


plt.tight_layout()
[Link]()

Explanation:
 Import Libraries: Import necessary libraries, NumPy which is used for numerical
operations (import numpy as np) and Matplotlib is used for plotting (import
[Link] as plt) where
-----The pyplot module provides a collection of functions for creating plots and
visualizations.
-----as plt: Assigns the alias plt to the imported [Link] module.
-----For example, you can use [Link]() to create a plot, [Link]() to set the x-axis
label, and so on.

 Define Activation Functions: Functions for sigmoid (sigmoid), hyperbolic


tangent (tanh), Rectified Linear Unit (relu), and softmax (softmax) activation
functions are defined.

 Generate x Values: [Link](-5, 5, 100) generates 100 equally spaced values from -5 to
5 and stores them in the variable x.

 Plot Activation Functions: The code then uses Matplotlib to create a figure with
subplots for each activation function. It plots the functions using the generated x
values, sets titles, and displays legends.

 For sigmoid:

[Link](1, 4, 1), [Link](x, sigmoid(x), label='Sigmoid'), [Link]('Sigmoid Activation


Function')

 For tanh:

[Link](1, 4, 2), [Link](x, tanh(x), label='Tanh'), [Link]('Hyperbolic Tangent


Activation Function')

 For ReLU:

[Link](1, 4, 3), [Link](x, relu(x), label='ReLU'), [Link]('Rectified Linear Unit


(ReLU) Activation Function')
 For softmax:

[Link](1, 4, 4), [Link](range(len(x_softmax)), softmax(x_softmax)),


[Link]('Softmax Activation Function')

 Show the Plots: plt.tight_layout() adjusts the layout for better spacing, and [Link]()
displays the entire set of subplots.

Experiment No-2
Program for single perceptron.
 Perceptron networks come under single-layer feed-forward networks and this is
also known as simple perceptron.
 Perceptron is one of the simplest Artificial neural network architectures. It
was introduced by Frank Rosenblatt in 1957s.
 It is the simplest type of feedforward neural network, consisting of a single layer of
input nodes that are fully connected to a layer of output nodes.
 It can learn the linearly separable patterns. It uses slightly different types of
artificial neurons known as threshold logic units (TLU).
 It was first introduced by McCulloch and Walter Pitts in the 1940s.
 A single perceptron is the simplest form of a neural network. It takes multiple
inputs, each multiplied by a weight, and produces a single output by applying an
activation function to the weighted sum of inputs.
----The output y of a single perceptron with n inputs can be represented as:

y=activation(∑n i=1 wi⋅xi+b)


Where:

 xi is the i-th input,

 wi is the weight corresponding to the i-th input,

 b is the bias term,


 activation is the activation function (typically a step function for binary
classification problems).
The perceptron learns by adjusting its weights and bias based on the error in its predictions
during training, using techniques like gradient descent.

Code:
import numpy as np
class Perceptron:
def init (self, num_inputs, learning_rate=0.01, epochs=100):
[Link] = [Link](num_inputs)
[Link] = 0
self.learning_rate = learning_rate
[Link] = epochs #An epoch means training the neural network with
all the training data for one cycle.
def activation_function(self, x):
return 1 if x > [Link] else 0
def train(self, X_train, y_train):
for _ in range([Link]):
for inputs, label in zip(X_train,
y_train): prediction =
[Link](inputs)
error = label - prediction
[Link] += self.learning_rate * error * inputs
def predict(self, inputs):
weighted_sum = [Link](inputs, [Link])
return self.activation_function(weighted_sum)
# Sample training data (OR gate)
X_train = [Link]([[0, 0], [0, 1], [1, 0], [1, 1]])
y_train = [Link]([0, 1, 1, 1])
# Create and train the perceptron
perceptron = Perceptron(num_inputs=2)
[Link](X_train, y_train)
# Test the perceptron
test_inputs = [Link]([[0, 0], [0, 1], [1, 0], [1, 1]])
for inputs in test_inputs:
prediction = [Link](inputs)
print(f"Input: {inputs}, Predicted Output: {prediction}")
Explanation:
 Import Libraries: We import the NumPy library, which provides support for
numerical operations in Python.
 Perceptron Class: We define a class called Perceptron to encapsulate the
functionality of a single-layer perceptron.
 Initializer Method: This method initializes the perceptron object with specified
parameters:
 num_inputs: Number of input features.
 learning_rate: Learning rate for weight updates during training (default is 0.01).
 epochs: Number of training epochs (default is 100).
 Weights Initialization: We initialize the weights randomly using NumPy's
[Link] function.
 Threshold Initialization: We set the threshold value to 0.
 Learning Rate and Epochs: We assign the learning rate and number of epochs to
class attributes.
 Activation Function: This method defines the activation function. It returns 1 if the
input x exceeds the threshold, otherwise 0.
 Training Method: This method trains the perceptron using the provided training data
X_train (input features) and y_train (labels).
 Prediction Method: This method makes predictions using the trained perceptron on
new input inputs.
 Training Loop: We iterate through the training data for the specified number of
epochs.
 For each input-output pair, we compute the prediction, calculate the error, and update
the weights using the perceptron learning rule.
 Sample Training Data: We define sample training data for the logical OR gate.
 Perceptron Initialization and Training: We create an instance of the perceptron
and train it using the sample training data.
 Testing and Output: We test the trained perceptron on new input data and print the
predicted outputs.
Output:
Input: [0 0], Predicted Output: 0
Input: [0 1], Predicted Output: 1
Input: [1 0], Predicted Output: 1
Input: [1 1], Predicted Output: 1
The output demonstrates that the trained perceptron correctly implements the OR gate logic,
producing the expected outputs for the given input data.
zip(X_train, y_train): This function takes two iterables (X_train and y_train) and returns an
iterator that aggregates elements from each iterable into tuples. For example, if X_train
contains [x1, x2, x3] and y_train contains [y1, y2, y3], zip(X_train, y_train) would yield
[(x1, y1), (x2, y2), (x3, y3)].

for inputs, label in ...: This is a for loop construct in Python that iterates over each element
of the iterator returned by zip(X_train, y_train). In each iteration, it unpacks the tuple into
variables inputs and label, where inputs represents the input features (e.g., a single data
sample or instance from X_train) and label represents the corresponding label (e.g., the
corresponding label from y_train).
Threshold: In neural networks, a threshold is a value used to determine whether a neuron in the
network should activate or not. It's a crucial concept in models like the perceptron, where the
output of the neuron is binary (typically 0 or 1) based on whether the input signal exceeds the
threshold.

Experiment-3
Program for M-Pits neuron for AND Gate
This program includes code for forward propagation, backpropagation, and updating weights
using gradient descent.

Code:
import numpy as np
class MPitsNeuron:
def init (self, input_size, learning_rate=0.01, m=1):
[Link] = [Link](input_size) # Initialize weights randomly
[Link] = [Link](1) # Initialize bias randomly
self.learning_rate = learning_rate
self.m = m
def activate(self, x):
return [Link](0, x) ** self.m # M-Pits activation function
def forward_propagation(self, inputs):
z = [Link](inputs, [Link]) + [Link]
return [Link](z)
def backward_propagation(self, inputs, output, expected_output):
error = output - expected_output
derivative = [Link](inputs > 0, self.m * (inputs ** (self.m - 1)), 0) # Derivative of M-
Pits function
gradient = error * derivative
[Link] -= self.learning_rate * gradient * inputs
[Link] -= self.learning_rate * error
def train(self, inputs, expected_outputs, epochs):
for epoch in range(epochs):
for i in range(len(inputs)):
output = self.forward_propagation(inputs[i])
self.backward_propagation(inputs[i], output,
expected_outputs[i])
def predict(self, inputs):
return self.forward_propagation(inputs)
# Example usage for AND gate:
if name == " main ":
# Input data
inputs = [Link]([[0, 0], [0, 1], [1, 0], [1, 1]])
expected_outputs = [Link]([0, 0, 0, 1])
# Create and train the neuron
neuron = MPitsNeuron(input_size=2, learning_rate=0.1, m=2)
[Link](inputs, expected_outputs, epochs=1000)

# Test the trained neuron


for i in range(len(inputs)):
prediction = [Link](inputs[i])
print(f"Input: {inputs[i]}, Predicted Output: {prediction}")
Output:
Input: [0 0], Predicted Output: [0]
Input: [0 1], Predicted Output: [0]
Input: [1 0], Predicted Output: [0]
Input: [1 1], Predicted Output: [1]

Explanation:
 The program defines a class MPitsNeuron that represents a neuron with the M-
Pits activation function.
 The forward_propagation method calculates the output of the neuron given inputs.
 The backward_propagation method updates the weights and bias of the neuron
using gradient descent.
 The train method trains the neuron using input-output pairs for a specified number
of epochs.
 The predict method predicts the output for given inputs after training.
 In the example usage section, an AND gate problem is solved using the implemented
neuron. The input data consists of AND gate truth table inputs, and the expected output
is the corresponding AND gate truth table outputs. The neuron is trained for 1000
epochs. Finally, the trained neuron is used to predict outputs for each input, and the
results are printed.

Experiment-4
Write a program to implement hetero-associative memory using hebb rule.
CODE:
import numpy as np
class HeteroAssociativeMemory:
def init (self, input_patterns, output_patterns):
self.input_patterns = [Link](input_patterns)
self.output_patterns = [Link](output_patterns)
[Link] = [Link]((self.input_patterns.shape[1], self.output_patterns.shape[1]))
def train(self):
for input_pattern, output_pattern in zip(self.input_patterns, self.output_patterns):
[Link] += [Link](input_pattern, output_pattern)
def recall(self, input_pattern):
return [Link](input_pattern, [Link])
# Example usage:
input_patterns = [[1, 1, 0],
[1, 0, 1],
[0, 1, 1]]
output_patterns = [[1, 0],
[0, 1],
[1, 1]]
ham = HeteroAssociativeMemory(input_patterns, output_patterns)
[Link]()
input_pattern = [1, 0, 0]
retrieved_output = [Link](input_pattern)
print("Retrieved Output:", retrieved_output)

OUTPUT:
Given input patterns:
[[1, 1, 0],
[1, 0, 1],
[0, 1, 1]]
Output patterns: = [[1, 0],
[0, 1],
[1, 1]]
The weight matrix after training: [[1, 1], [1, 1], [1, 1]]
Note: Now, for the input pattern [1, 0, 0], the retrieved output will be: [1,1]
So, the output: [1,1]
Explanation:
1) First we have to import the library named as NumPy library, which is used for numerical
computing.
2) Defines a class named hetero associative memory, which will be used to create
objects representing hetero-associative memory.
3) def init (self, input_patterns, output_patterns): This line defines the constructor
method for the `HeteroAssociativeMemory` class. It initializes the object with input patterns
and output patterns provided as arguments.
4) self.input_patterns = [Link](input_patterns)
self.output_patterns =
[Link](output_patterns)
For both patterns: These lines convert the input and output patterns provided as lists into
NumPy arrays and store them as attributes of the hetero associative memroy object.
5) [Link] = [Link]((self.input_patterns.shape[1], self.output_patterns.shape[1])) :
This line initializes the weights matrix with zeros. The size of the matrix is determined by the
number of input and output neurons.
6) def train(self): Defines a method named train which will be used to train the hetero-
associative memory using the Hebbian learning rule.
7) for input_pattern, output_pattern in zip(self.input_patterns, self.output_patterns): -
This line iterates over each pair of input and output patterns using the zip function, which
combines elements from two or more iterables into tuples.
8) return [Link](input_pattern, [Link]): - This line computes the dot product
between the input pattern and the weights matrix, which gives the retrieved output pattern.
9) input_patterns = [[1, 1, 0], [1, 0, 1], [0, 1, 1]] output_patterns = [[1, 0], [0, 1], [1, 1]]
For above two patterns: It defines the input and output patterns that will be used to train the
hetero-associative memory.
10) [Link](): calls the train method to train the hetero-associative memory using
the provided input and output patterns.
11) retrievd_output= [Link](input_pattern): calls the recall method to retrieve the
output pattern for the given input pattern.

Experiment -5
Write a program in MATLAB to perform Union, Intersection and Complement operations.
Program:
# Define two sets
set1 = {1, 2, 3, 4, 5}
set2 = {4, 5, 6, 7, 8}
# Union operation
union_set = [Link](set2)
print("Union:", union_set)
# Intersection operation
intersection_set = [Link](set2)
print("Intersection:", intersection_set)
# Complement operation
complement_set1 = [Link](set2)
complement_set2 = [Link](set1)
print("Complement set1:", complement_set1)
print("Complement set2:", complement_set2)

Output:
Union: {1, 2, 3, 4, 5, 6, 7, 8}
Intersection: {4, 5}
Complement set1: {1, 2, 3}
Complement set2: {8, 6, 7}

Experiment-6
Write a program in PYTHON to plot various membership functions,\.

Code:
import numpy as np
import [Link] as plt
# Define the universe of discourse (x-axis range)
x = [Link](0, 10, 1000)

# Membership functions

# Triangular membership function


def triangular(x, a, b, c):
return [Link](0, [Link]((x - a) / (b - a), (c - x) / (c - b)))

# Trapezoidal membership function


def trapezoidal(x, a, b, c, d):
return [Link](0, [Link]([Link]((x - a) / (b - a), 1 - (x - c) / (d - c)), 1))
# Gaussian membership function
def gaussian(x, mean, std_dev):
return [Link](-((x - mean) / std_dev) ** 2)

# Plotting membership functions


[Link](figsize=(10, 6))
# Triangular membership function
[Link](3, 1, 1)
[Link](x, triangular(x, 2, 4, 7), label='Triangular MF')
[Link]('x')
[Link]('Membership Degree')
[Link]('Triangular Membership Function')
[Link]()
# Trapezoidal membership function
[Link](3, 1, 2)
[Link](x, trapezoidal(x, 2, 4, 7, 9), label='Trapezoidal MF')
[Link]('x')
[Link]('Membership Degree')
[Link]('Trapezoidal Membership Function')
[Link]()
# Gaussian membership function
[Link](3, 1, 3)
[Link](x, gaussian(x, 5, 1), label='Gaussian MF')
[Link]('x')
[Link]('Membership Degree')
[Link]('Gaussian Membership Function')
[Link]()
plt.tight_layout()
[Link]()

Output:
Explanation:

1. import numpy as np: Imports the NumPy library, which is used for numerical computations.

2. import [Link] as plt: Imports the Matplotlib library for plotting graphs.

x = [Link](0, 10, 1000): Creates an array of 1000 equally spaced points between 0 and 10.
This serves as the universe of discourse.
3. Membership functions:

triangular(): Defines a triangular membership function with parameters a, b, and c.


trapezoidal(): Defines a trapezoidal membership function with parameters a, b, c, and d.
gaussian(): Defines a Gaussian membership function with mean and standard deviation.
4. Plotting membership functions:

Uses [Link]() to create subplots for each membership function.


[Link]() is used to plot each membership function.
[Link](), [Link](), and [Link]() set labels and titles for the plots.
[Link]() adds a legend to each plot.
plt.tight_layout(): Automatically adjusts subplot parameters to give specified padding.
[Link](): Displays the plots.

Experiment-7

Write a program for complete Genetic Algorithm cycle.

We'll use a genetic algorithm to evolve a population of candidate solutions to a target solution.
In this example, each candidate solution is a string of characters. The algorithm will aim to
evolve a population of random strings to match a given target string.
Code:
import random
# Constants
TARGET = "GENETIC ALGORITHMS"
POP_SIZE = 100
MUT_RATE = 0.01
GENS = 1000

# Functions for individual creation, fitness calculation, crossover, and mutation


create_individual = lambda: ''.join([Link]("ABCDEFGHIJKLMNOPQRSTUVWXYZ ")
for _ in TARGET)
fitness = lambda ind: sum(1 for i, j in zip(ind, TARGET) if i == j) / len(TARGET)
crossover = lambda p1, p2: ''.join(p1[i] if [Link]() > 0.5 else p2[i] for i in
range(len(p1)))
mutate = lambda ind: ''.join(c if [Link]() > MUT_RATE else
[Link]("ABCDEFGHIJKLMNOPQRSTUVWXYZ ") for c in ind)

# Initialize population
population = [create_individual() for _ in range(POP_SIZE)]

# Main genetic algorithm loop


for gen in range(GENS):
[Link](key=fitness, reverse=True)
best = population[0]
print(f"Gen {gen + 1}: {best} (Fitness: {fitness(best):.4f})")
if best == TARGET:
print("Target reached!")
break
top_half = population[:POP_SIZE // 2]
population = [mutate(crossover([Link](top_half), [Link](top_half))) for _ in
range(POP_SIZE)]

Output:
Gen 1: GRUYXOOSNDZZSBCIIQ (Fitness: 0.1579)
Gen 2: GQLEQPIHAAUQRRBVMJ (Fitness: 0.2105)
Gen 3: GEXETIHMXIDRITHMJO (Fitness: 0.2632)
...
Gen 45: GENETIC ALBOKITHMS (Fitness: 0.8947)
Gen 46: GENETIC ALGORITHMS (Fitness: 1.0000)

Explanation:

 Constants define the target string, population size, mutation rate, and max generations.
 Functions: create_individual, fitness, crossover, and mutate are
implemented with lambda functions for brevity.
 Population Initialization: Starts with random individuals.
 Main Loop: Sorts population by fitness, prints the best individual each generation,
checks if the target is reached, and uses selection, crossover, and mutation to create the
next generation.

Experiment-8

Write a program for all types of crossover methods.

crossover methods: Single-Point, Two-Point, Uniform, Arithmetic, Partially Matched


(PMX), and Order Crossover (OX). Each method is implemented as a function that takes two
parent chromosomes and returns an offspring.

For simplicity, we use a pair of parent chromosomes for each crossover method

Code:
import random
# Parents for demonstration
parent1 = [1, 2, 3, 4, 5, 6, 7, 8]
parent2 = [8, 7, 6, 5, 4, 3, 2, 1]

# Single-Point Crossover
def single_point_crossover(p1, p2):
point = [Link](1, len(p1) - 1)
return p1[:point] + p2[point:]

# Two-Point Crossover
def two_point_crossover(p1, p2):
point1 = [Link](1, len(p1) - 2)
point2 = [Link](point1 + 1, len(p1) - 1)
return p1[:point1] + p2[point1:point2] + p1[point2:]

# Uniform Crossover
def uniform_crossover(p1, p2):
return [p1[i] if [Link]() > 0.5 else p2[i] for i in range(len(p1))]

# Arithmetic Crossover (for real-valued chromosomes)


def arithmetic_crossover(p1, p2, alpha=0.5):
return [(alpha * p1[i] + (1 - alpha) * p2[i]) for i in range(len(p1))]

# Partially Matched Crossover (PMX) for permutations


def pmx_crossover(p1, p2):
point1, point2 = sorted([Link](range(len(p1)), 2))
child = [None] * len(p1)
child[point1:point2] = p1[point1:point2]
for i in range(point1, point2):
if p2[i] not in
child: j = i
while child[j] is not None:
j = [Link](p1[j])
child[j] = p2[i]
for i in range(len(p1)):
if child[i] is None:
child[i] = p2[i]
return child

# Order Crossover (OX) for permutations


def order_crossover(p1, p2):
point1, point2 = sorted([Link](range(len(p1)), 2))
child = [None] * len(p1)
child[point1:point2] = p1[point1:point2]
fill = [gene for gene in p2 if gene not in child]
for i in range(len(child)):
if child[i] is None:
child[i] = [Link](0)
return child

# Demonstration of each crossover method


print("Parent 1:", parent1)
print("Parent 2:", parent2)
print("Single-Point Crossover:", single_point_crossover(parent1,
parent2)) print("Two-Point Crossover:", two_point_crossover(parent1,
parent2))
print("Uniform Crossover:", uniform_crossover(parent1, parent2))
print("Arithmetic Crossover:", arithmetic_crossover(parent1, parent2))
print("PMX Crossover:", pmx_crossover(parent1, parent2))
print("Order Crossover:", order_crossover(parent1, parent2))

Output:
Parent 1: [1, 2, 3, 4, 5, 6, 7, 8]
Parent 2: [8, 7, 6, 5, 4, 3, 2, 1]
Single-Point Crossover: [1, 2, 3, 4, 4, 3, 2, 1]
Two-Point Crossover: [1, 2, 6, 5, 4, 6, 7, 8]
Uniform Crossover: [8, 2, 6, 4, 5, 3, 2, 8]
Arithmetic Crossover: [4.5, 4.5, 4.5, 4.5, 4.5, 4.5, 4.5, 4.5]
PMX Crossover: [1, 2, 3, 5, 4, 6, 7, 8]
Order Crossover: [4, 5, 6, 4, 5, 6, 7, 8]

Explanation:
 Single-Point Crossover: A single point is chosen, and offspring are created by
combining segments from both parents at that point.
 Two-Point Crossover: Two points are chosen, and the offspring is formed by
combining segments between those points.
 Uniform Crossover: Each gene is randomly selected from one of the parents.
 Arithmetic Crossover: For real-valued genes, each gene is calculated as a weighted sum
of both parents.
 Partially Matched Crossover (PMX): Suitable for permutations, this method ensures
each gene appears only once by filling in segments and handling duplicates.
 Order Crossover (OX): Also for permutations, OX selects a segment from one parent
and fills the rest with non-repeating genes from the other parent.
Experiment-9

Program for composition on Fuzzy and Crisp Relations.

In fuzzy logic and soft computing, the composition of relations allows us to combine fuzzy or crisp
relations to form new relationships. Composition operations are particularly useful in situations
that involve chaining relationships or inference in fuzzy rule-based systems.

Here’s a Python program to demonstrate the composition of fuzzy and crisp relations using the
Max-Min Composition and Max-Product Composition methods.

Program: Composition of Fuzzy and Crisp Relations

We’ll create two relations, R and S, where:

 R is a relation from set XXX to YYY.


 S is a relation from set YYY to ZZZ.

The goal is to find the composition T=R∘ST = R \circ ST=R∘S from set XXX to ZZZ.

import numpy as np

# Define relations R and S as numpy arrays


# Relation R (from X to Y) - Example Fuzzy Matrix R
= [Link]([
[0.8, 0.4, 0.3],
[0.5, 0.7, 0.2],
[0.6, 0.9, 0.5]
])

# Relation S (from Y to Z) - Example Fuzzy Matrix S =


[Link]([
[0.9, 0.3, 0.4],
[0.5, 0.8, 0.7],
[0.6, 0.4, 0.9]
])

# Max-Min Composition of Fuzzy Relations


def max_min_composition(R, S):
T = [Link](([Link][0], [Link][1]))
for i in range([Link][0]):
for j in range([Link][1]):
T[i, j] = [Link]([Link](R[i, :], S[:, j]))
return T

# Max-Product Composition of Fuzzy Relations


def max_product_composition(R, S):
T = [Link](([Link][0], [Link][1]))
for i in range([Link][0]):
for j in range([Link][1]):
T[i, j] = [Link](R[i, :] * S[:, j])
return T

# Perform Max-Min and Max-Product compositions


T_max_min = max_min_composition(R, S)
T_max_product = max_product_composition(R, S)

# Output results
print("Relation R:")
print(R)
print("Relation S:")
print(S)
print("Max-Min Composition (R ∘ S):")
print(T_max_min)
print("Max-Product Composition (R ∘ S):")
print(T_max_product)

Output:
Relation R:
[[0.8 0.4 0.3]
[0.5 0.7 0.2]
[0.6 0.9 0.5]]
Relation S:
[[0.9 0.3 0.4]
[0.5 0.8 0.7]
[0.6 0.4 0.9]]
Max-Min Composition (R ∘ S):
[[0.8 0.4 0.7]
[0.7 0.7 0.7]
[0.9 0.8 0.9]]
Max-Product Composition (R ∘ S):
[[0.72 0.32 0.72]
[0.63 0.56 0.63]
[0.81 0.72 0.81]]

Explanation:

 Relations R and S: These are represented as matrices (NumPy arrays), where each element
is a degree of relation (fuzzy values between 0 and 1).

 Max-Min Composition:
 For each pair (i,j)(i, j)(i,j) in the resulting relation T, we calculate the maximum of
the minimum values from each row in R and each column in S.
 [Link](R[i, :], S[:, j]) finds the element-wise minimum between row i of R
and column j of S, and [Link](...) gives the maximum of these minimum values.

 Max-Product Composition:

 Similar to Max-Min, but instead of taking the minimum, we take the product for
each pair and then the maximum.

Experiment-10

Write a program to implement Defuzzification methods such as Centroid, Centre of Sum and
Mean of Maxima.

Defuzzification is the process of converting fuzzy values into a crisp output, which is necessary
for making decisions based on fuzzy inference systems. Here’s a Python program to demonstrate
three defuzzification methods: Centroid, Center of Sum, and Mean of Maxima. Each method
works with a fuzzy set represented as a membership function.

Code:

import numpy as np

# Define a fuzzy set (example membership function for a fuzzy output

variable) # x_values represents the discrete universe of discourse

x_values = [Link](0, 10, 100)

# membership_values is the corresponding membership degree at each x in x_values

membership_values = [Link]([max(0, 1 - abs(x - 5) / 5) for x in x_values])

# Centroid Defuzzification
def centroid_defuzzification(x_values, membership_values):

numerator = [Link](x_values * membership_values)

denominator = [Link](membership_values)

return numerator / denominator if denominator != 0 else 0

# Center of Sum Defuzzification

def center_of_sum_defuzzification(x_values, membership_values):

scaled_membership = membership_values / [Link](membership_values)

return [Link](x_values * scaled_membership)

# Mean of Maxima Defuzzification

def mean_of_maxima_defuzzification(x_values, membership_values):

max_value = [Link](membership_values)

max_indices = [Link](membership_values == max_value)[0]

return [Link](x_values[max_indices])

# Calculate crisp outputs

centroid_result = centroid_defuzzification(x_values, membership_values)

center_of_sum_result = center_of_sum_defuzzification(x_values, membership_values)

mean_of_maxima_result = mean_of_maxima_defuzzification(x_values, membership_values)

# Output results

print("Centroid Defuzzification:", centroid_result)


print("Center of Sum Defuzzification:", center_of_sum_result)

print("Mean of Maxima Defuzzification:", mean_of_maxima_result)

Output:

Centroid Defuzzification: 5.0


Center of Sum Defuzzification: 5.0
Mean of Maxima Defuzzification: 5.0

Explanation:

 Input Fuzzy Set:

 x_values: A range of x-values representing the variable’s universe of discourse (e.g.,


from 0 to 10).
 membership_values: Membership degrees of each x-value. Here, we use a
simple triangular shape centered at 5 for demonstration.

 Defuzzification Methods:

 Centroid (Center of Area): Calculates the center of gravity of the membership


function by taking the weighted average of the x-values.
o numerator: Sum of x * membership_values.
o denominator: Sum of membership_values.
 Center of Sum: Divides each membership value by the total sum of memberships
to normalize, then finds the sum of x * normalized memberships.
 Mean of Maxima (MOM): Finds the average of x-values where the membership is at its
maximum.
Practical Title: Understanding Artificial Neural Networks (ANN)

Objective:
To study the basic structure, functionality, and implementation of Artificial Neural Networks (ANN)
and understand their significance in solving real-world problems.

Introduction:
Artificial Neural Networks (ANNs) are a subset of machine learning algorithms inspired by the human
brain's structure and functionality. They consist of interconnected layers of nodes, known as neurons,
that process and transmit information. ANNs are widely used in various fields such as image
recognition, natural language processing, and predictive analytics due to their ability to model complex
patterns and relationships in data.

This practical focuses on implementing an ANN and observing its learning process through a guided
example. The practical also highlights key concepts such as activation functions, forward propagation,
backpropagation, and the role of weights and biases in network training.

For a detailed walkthrough of the implementation, refer to the video demonstration available at the
following link:

Artificial Neural Network Practical - Video Demonstration:

[Link]

Learning Outcomes:

1. Understand the architecture and working principles of an Artificial Neural Network.


2. Implement a simple ANN using Python or a relevant framework (e.g., TensorFlow, PyTorch).
3. Analyze the training process and evaluate the model's performance.

Practical Title: Understanding Convolutional Neural Networks (CNN)

Objective:
To study the architecture, functionality, and implementation of Convolutional Neural Networks (CNNs)
and understand their application in image processing and computer vision tasks.

Introduction:
Convolutional Neural Networks (CNNs) are a specialized type of artificial neural network designed to
process structured data, particularly images. By leveraging layers such as convolutional layers, pooling
layers, and fully connected layers, CNNs can effectively detect and extract features from images,
enabling tasks like image classification, object detection, and segmentation.

This practical explores the foundational concepts of CNNs, including convolution operations, filters,
feature maps, and pooling mechanisms. It also provides a hands-on demonstration of implementing a
CNN model and analyzing its behavior.

For step-by-step guidance and demonstrations, refer to the following video resources:

1. Introduction to CNN - Part 1


[Link]

2. CNN Implementation and Concepts - Part 2

[Link]

Learning Outcomes:

1. Understand the fundamental building blocks of a Convolutional Neural Network.


2. Implement a basic CNN for image classification using Python or relevant libraries (e.g., TensorFlow,
Keras, or PyTorch).
3. Analyze how convolutional and pooling layers contribute to feature extraction.
4. Evaluate the performance of a CNN model on a sample dataset.

Practical Title: Understanding Fuzzy Logic Systems

Objective:
To understand the principles of fuzzy logic, its components, and its application in decision-making and
control systems through practical implementation.

Introduction:
Fuzzy logic is a mathematical approach to handle uncertainty and imprecision, mimicking human
reasoning by allowing intermediate values between absolute true and false. Unlike classical binary logic,
fuzzy logic enables decision-making based on degrees of truth, making it highly applicable in real-world
scenarios such as control systems, pattern recognition, and artificial intelligence.

This practical focuses on the implementation of a fuzzy logic system, exploring key concepts like fuzzy
sets, membership functions, rules, and inference mechanisms. The session provides insights into how
fuzzy logic can be used to solve problems involving ambiguous or imprecise data.

For a detailed walkthrough and demonstration of fuzzy logic implementation, refer to the following
video resource:

Fuzzy Logic Practical - Video Demonstration

[Link]

Learning Outcomes:

1. Understand the fundamentals of fuzzy logic and its difference from classical logic.
2. Design a fuzzy logic system, including defining fuzzy sets and rules.
3. Implement a fuzzy inference system using tools such as MATLAB or Python.
4. Analyze the output of a fuzzy logic system and its application in decision-making.

Practical Title: Understanding Genetic Algorithms (GA)


Objective:
To study the principles and implementation of Genetic Algorithms (GAs) for solving optimization
problems by simulating the process of natural selection.

Introduction:
Genetic Algorithms (GAs) are a class of optimization techniques inspired by the process of natural
evolution. GAs work by generating a population of candidate solutions and iteratively improving them
using genetic operators like selection, crossover, and mutation. These algorithms are particularly useful
for solving complex optimization problems where traditional methods may struggle.

This practical focuses on understanding the working of GAs, including the creation of an initial
population, evaluation of fitness, and application of genetic operators. A hands-on implementation
demonstrates how GAs can be applied to solve a sample optimization problem.

For a detailed walkthrough and demonstration of Genetic Algorithm implementation, refer to the
following video resource:

Genetic Algorithm Practical - Video Demonstration

[Link]

Learning Outcomes:

1. Understand the basic principles of Genetic Algorithms and their role in optimization.
2. Implement a Genetic Algorithm to solve a given optimization problem.
3. Analyze the effects of different genetic operators (selection, crossover, mutation) on the solution quality.
4. Evaluate the performance of the algorithm in terms of convergence and computational efficiency.

Common questions

Powered by AI

A single-layer perceptron learns by adjusting weights based on the error between predicted outcomes and actual labels for linearly separable data. The learning involves adjusting weights directly using techniques like gradient descent to minimize this error. In contrast, a multi-layer neural network uses backpropagation, a more complex learning process that involves computing gradients through multiple layers. Errors are propagated back through the network from the output to the input layer, allowing each layer to adjust its parameters. This process enables learning of complex patterns beyond linear separability, unlike the simpler perceptron .

Different genetic crossover methods offer various trade-offs in terms of maintaining diversity and convergence in the population. Single-point crossover, by combining segments from both parents at one point, provides good mixing but might limit diversity if the same point is commonly chosen. Two-point crossover allows multiple segments, offering better diversity but can complicate offspring structure. Uniform crossover, selecting each gene randomly from any parent, maximizes diversity but can slow convergence as it often disrupts high-quality solutions. Each method balances exploration of the solution space (diversity) against exploiting known good solutions (convergence) differently .

In neural networks like the M-Pits neuron model, forward propagation is the process where inputs are passed through the network to produce an output. This output is then compared with the expected output to gauge the performance of the network. Forward propagation involves calculating neuron activations starting from the input layer through to the output layer, using weights and biases at each step. This process is critical for determining the error during training, which is later minimized through backward propagation and weight adjustments .

Studying and implementing Convolutional Neural Networks (CNNs) results in several educational outcomes, including understanding deep learning architectures specifically designed for structured grid data like images. CNNs' ability to automatically and adaptively learn spatial hierarchies of features makes them essential in image processing, as they efficiently detect patterns through convolutional and pooling layers. This reduces the need for manual feature extraction, enhances classification accuracy, and advances computer vision tasks such as image recognition, detection, and segmentation, thereby underpinning modern technological advancements in AI and ML .

The Hetero-associative memory model using Hebb's rule differs from traditional memory models primarily through its associative capability, where it learns to map input patterns to distinct output patterns by reinforcing synaptic weights when co-activation occurs. A core feature of Hebbian learning is its simplicity and biological plausibility, as it strengthens connections between simultaneously active neuron pairs, encoding direct associations. This enables the memory model to recognize and retrieve patterns efficiently when given partial or noisy inputs, offering robustness against input variations, unlike more rigid traditional memory models .

Fuzzy logic systems are instrumental in real-world decision-making, particularly in scenarios where information is ambiguous or imprecise. Unlike classical binary logic, fuzzy logic allows for varying degrees of truth, which makes it suitable for control systems, pattern recognition, and AI applications. Such systems are used in consumer electronics for adaptive control (like washing machines adjusting cycles based on load and type), in vehicle stability controls, and decision support systems where human reasoning must be emulated. The adaptability and interpretability of fuzzy logic enable it to manage uncertain data effectively, thereby improving operational decision-making .

Centroid defuzzification works by calculating the center of mass of the fuzzy set, effectively providing a single crisp value that best represents the entire fuzzy information. This involves a weighted average of all the x-values in the universe of discourse, each weighted by its membership function value. The practical benefits of centroid defuzzification include its ability to provide smooth, consistent outputs that are particularly useful in control systems where gradual responses are necessary to prevent abrupt changes, and its intuitive representation of the 'center of gravity,' which can be easily understood .

The primary advantage of using the Leaky ReLU activation function over the standard ReLU is that it addresses the "dying ReLU" problem. In standard ReLU, neurons can become inactive during training if they output zero for all inputs, essentially making such neurons untrainable. Leaky ReLU maintains a small, non-zero gradient for negative inputs, which prevents neurons from being completely turned off and allows for continued learning .

In perceptron models, the threshold acts as a decision boundary that determines whether the neuron should activate or not. If the weighted sum of inputs exceeds the threshold, the neuron fires (outputs 1); otherwise, it remains inactive (outputs 0). This binary activation based on the threshold is crucial for the model's capacity to classify inputs into two distinct classes . The threshold ensures that only significant input signals activate the neuron, thus affecting the overall pattern of neuronal activity within the network.

The rule of thumb regarding activation functions suggests using the sigmoid function for binary classification outputs as it naturally maps inputs to probabilities between 0 and 1 for a two-class problem. In contrast, the softmax function is recommended for multi-class classification problems because it squashes a vector of arbitrary real-valued scores into a vector of values between 0 and 1 that sum to 1, representing the probability distribution over multiple mutually exclusive classes .

You might also like