Expt:no:01 Date:30-07-2025
Implementation of XOR gate through a Neural network
Aim:
To program and obtain the output of the XOR Gate through XOR Neural network.
Software required:
Visual studio code (VS Code)
Program:
import numpy as np
import [Link] as plt
class XORNeuralNetwork:
def __init__(self):
[Link](42)
# Network structure
self.input_size = 2
self.hidden_size = 2
self.output_size = 1
# Initialize weights and biases
self.W1 = [Link](-1, 1, (self.input_size, self.hidden_size))
self.b1 = [Link]((1, self.hidden_size))
self.W2 = [Link](-1, 1, (self.hidden_size, self.output_size))
self.b2 = [Link]((1, self.output_size))
#Learning rate
[Link] = 0.1
def sigmoid(self, x):
return 1 / (1 + [Link](-x))
def sigmoid_derivative(self, x):
return x * (1 - x)
def binary_cross_entropy(self, y_true, y_pred):
# Clip predictions to avoid log(0)
y_pred = [Link](y_pred, 1e-9, 1 - 1e-9)
return -[Link](y_true * [Link](y_pred) + (1 - y_true) * [Link](1 - y_pred))
def forward_pass(self, X):
self.z1 = [Link](X, self.W1) + self.b1
self.a1 = [Link](self.z1)
self.z2 = [Link](self.a1, self.W2) + self.b2
[Link] = [Link](self.z2)
return [Link]
def backward_pass(self, X, y):
# Binary cross-entropy derivative
output_error = [Link] - y
output_delta = output_error * self.sigmoid_derivative([Link])
hidden_error = output_delta.dot(self.W2.T)
hidden_delta = hidden_error * self.sigmoid_derivative(self.a1)
# Update weights and biases
self.W2 -= [Link](output_delta) * [Link]
self.b2 -= [Link](output_delta, axis=0, keepdims=True) * [Link]
self.W1 -= [Link](hidden_delta) * [Link]
self.b1 -= [Link](hidden_delta, axis=0, keepdims=True) * [Link]
# Return Binary Cross Entropy Cost
return self.binary_cross_entropy(y, [Link])
def train(self, X, y, epochs=100000):
costs = []
for epoch in range(epochs):
self.forward_pass(X)
cost = self.backward_pass(X, y)
[Link](cost)
if epoch % 10000 == 0:
print(f"Epoch {epoch}, Cost: {cost:.5f}")
return costs
# XOR Data
X = [Link]([[0, 0],
[0, 1],
[1, 0],
[1, 1]])
y = [Link]([[0], [1], [1], [0]])
# Train model
model = XORNeuralNetwork()
costs = [Link](X, y)
# Plot cost function
[Link](costs)
[Link]("Cross-Entropy Cost over Epochs")
[Link]("Epochs")
[Link]("Cross-Entropy Cost")
[Link](True)
[Link]()
# Final predictions
print("Final Predictions:")
print([Link](model.forward_pass(X), 3))
Result:
Cost function
Predicted output:
Epoch 10000, Cost: 0.10366
Epoch 20000, Cost: 0.04643
Epoch 30000, Cost: 0.03391
Epoch 40000, Cost: 0.02786
Epoch 50000, Cost: 0.02415
Epoch 60000, Cost: 0.02159
Epoch 70000, Cost: 0.01969
Epoch 80000, Cost: 0.01820
Epoch 90000, Cost: 0.01701
Final Predictions:
[[0.015]≈ 0
[0.983]≈ 1
[0.983]≈ 1
[0.014]]≈ 0
Input (x1, x2) Expected Predicted
(0, 0) 0 0.015
(0, 1) 1 0.983
(1, 0) 1 0.983
(1, 1) 0 0.014
Expt:no:00 Date:30-07-2025
Implementation of Activation function
Aim:
To program and plot the output of the activation functions.
Software required:
Visual studio code (VS Code)
Program:
Sigmoid Function:
# Import necessary libraries
import [Link] as plt
import numpy as np
# Initiate a list of random variables
r_list = [Link](-10,10,100)
# Define sigmoid function
def sigmoid(x):
return 1/(1+[Link](-x))
# plotting for Sigmoid function
[Link]([Link](r_list), sorted(sigmoid([Link](r_list))))
[Link]('Values of x')
[Link]('sigmoid(x)')
[Link]("Sigmoid Function")
[Link]()
[Link]('[Link]')
ReLu Activation Function:
# define Relu function
def relu(x):
return [Link](0, x)
# plotting for Relu function
[Link]([Link](r_list), [Link](relu([Link](r_list))))
[Link]('Values of x')
[Link]('Relu(x)')
[Link]("Relu Function")
[Link]()
[Link]('[Link]')
Swish function:
# define Swish function
def swish(x, b):
return x*sigmoid(b*x)
# plotting for Swish function
# select arbitrary beta list for the swish function
b_list = [0.1,0.2,0.5,1,1.5,2]
fig, ax = [Link](3,2, figsize = (10,8))
ax = [Link]()
for i,b in enumerate(b_list):
ax[i].plot([Link](r_list), (swish([Link](r_list),b)))
ax[i].set_title(f'Swish function with b = {b}')
ax[i].set_xlabel('Values of x')
ax[i].set_ylabel('Swish function')
plt.tight_layout()
[Link]()
Leaky ReLu:
def leakyrelu(A,x):
return [Link](x < 0, A * x, x)
[Link]((-10,10))
[Link]((-10,10))
[Link]([0,0],[-10,10],color='blue')
[Link]([-10,10],[0,0],color='blue')
[Link]([Link](r_list),[Link](leakyrelu(0.2,([Link](r_list)))))
[Link]('Values of x')
[Link]('LeakyRelu(x)')
[Link]("LeakyRelu Function")
[Link]()
Softmax Activation function:
import numpy as np
import [Link] as plt
# Define softmax function
def softmax(logits):
exp_logits = [Link](logits - [Link](logits, axis=1, keepdims=True))
return exp_logits / [Link](exp_logits, axis=1, keepdims=True)
# Generate input range
x = [Link](-5, 5, 100)
# Create logits for 3 classes
logits = [Link]([[0, 1, 2]])
softmax_outputs = []
for i in x:
test_logits = [Link]([[i, 1, 2]])
probs = softmax(test_logits)
softmax_outputs.append(probs[0])
softmax_outputs = [Link](softmax_outputs)
# Plotting
[Link](x, softmax_outputs[:, 0], label='Class 1')
[Link](x, softmax_outputs[:, 1], label='Class 2')
[Link](x, softmax_outputs[:, 2], label='Class 3')
[Link]("Varying logit (Class 1)")
[Link]("Softmax Output Probability")
[Link]("Softmax Output vs Varying Logit of Class 1")
[Link]()
[Link](True)
[Link]()
tanh Activation function:
import numpy as np
import [Link] as plt
# Define the tanh function
def tanh(x):
return [Link](x)
# Generate input range
x = [Link](-10, 10, 500)
# Compute tanh output
y = tanh(x)
# Plot the tanh function
[Link](x, y, label='tanh(x)', color='purple')
[Link]("Hyperbolic Tangent (tanh) Activation Function")
[Link]("Input")
[Link]("Output")
[Link](True)
[Link](0, color='blue', linestyle='--')
[Link](0, color='blue', linestyle='--')
[Link]()
[Link]()
Exponential Linear unit (ELU):
import numpy as np
import [Link] as plt
# Define the ELU function
def elu(x, alpha=1.0):
return [Link](x >= 0, x, alpha * ([Link](x) - 1))
# Generate input range
x = [Link](-10, 10, 500)
# Compute ELU output
y = elu(x, alpha=1.0)
# Plot the ELU function
[Link](x, y, label='ELU(x)', color='darkorange')
[Link]("Exponential Linear Unit (ELU) Activation Function")
[Link]("Input")
[Link]("Output")
[Link](True)
[Link](0, color='blue', linestyle='--')
[Link](0, color='blue', linestyle='--')
[Link]()
[Link]()
Result:
a) Sigmoid function b) ReLu
c) Leaky ReLu d) tanh
e) Swish Activation function
f) ELU g) Softmax