Artificial Neural Networks: Forward and Backpropagation Practice Set
Numerical Practice Problems for 4th Semester Curriculum
Problem 1: Single Perceptron with Sigmoid Activation
Consider a single perceptron with two inputs x1 = 0.6 and x2 = 0.4. The initial weights are w1 = 0.5 and
w2 = −0.3, with a bias b = 0.2. The activation function is the Sigmoid function σ (z) = 1+e1 −z . Calculate the
final output y and the Mean Squared Error E = 12 (t − y)2 given a target output t = 1.
x1
w1
=0
.5 b = 0.2
y
∑ |σ
.3
= −0
w 2
x2
Final Answer:
Output y ≈ 0.5939
Error E ≈ 0.0825
Problem 2: Error Gradient Calculation
For the network below, the output neuron uses a Sigmoid activation. Input x = 0.8, weight w = 0.5, bias
b = −0.1, and the target t = 0. Using the loss function E = 12 (t − y)2 , find the gradient of the error with
respect to the weight, ∂∂ Ew , and calculate the updated weight after one step of Gradient Descent with learning
rate η = 0.1.
b = −0.1
w = 0.5 y
x = 0.8 σ
Final Answer:
Gradient ∂∂ Ew ≈ 0.1123
Updated weight w ≈ 0.4888
1
Problem 3: Multi-Layer Perceptron (MLP) Forward Pass
A neural network has a 2-2-1 architecture. All hidden and output nodes use the ReLU activation f (z) =
max(0, z). Inputs: x1 = 1.0, x2 = −1.0.
Weights (Input to Hidden): w13 = 0.4, w23 = 0.6, w14 = −0.2, w24 = −0.5.
Biases (Hidden): b3 = 0.5, b4 = 0.1.
Weights (Hidden to Output): w35 = 0.8, w45 = 0.9.
Bias (Output): b5 = −0.2.
Calculate the final output y of the network.
w13
X1 3
w3
5
w 45
X2 w24 4
Final Answer:
Output y = 0.04
Problem 4: Hidden Layer Delta Calculation
In a network using Sigmoid activation for all nodes, the error term (delta) at the output node O is given as
δO = 0.15. The weights from the two hidden nodes to the output node are w1O = 0.5 and w2O = −0.4. The
outputs of the hidden nodes during the forward pass were h1 = 0.7 and h2 = 0.3. Calculate the error term δ1
for hidden node 1.
h1
w1O
= 0.
5
O δO = 0.15
0.4
=−
w 2O
h2
Final Answer:
δ1 = 0.01575
2
Problem 5: Full Forward Pass with Biases (Matrix Representation)
Calculate the final output y for the following network using Sigmoid activation for all nodes. Inputs: X1 =
0.5, X2 = 0.8.
(1) 0.2 −0.1 (1)
Weights from input to hidden layer W = (where Wi j is the weight from input i to hidden
0.4 0.3
j), b(1) = [0.1, −0.2].
Weights from hidden to output W (2) = [0.5, −0.5], b(2) = 0.2.
(1)
W11
X1 H1
W (2)
1
W (
12 1)
)
O
(1
W 21
(2)
W2
X2 H2
(1)
W22
Final Answer:
Output y ≈ 0.5658
Problem 6: ReLU Dead Neuron Scenario
Consider a single neuron with three inputs x1 = 2, x2 = −3, x3 = 1 and weights w1 = 0.2, w2 = 0.5, w3 =
−0.1. The bias is b = 0. Calculate the output y using the ReLU activation. If the target is t = 1 and learning
rate η = 0.1, what is the updated weight for w2 ?
x1
w1
w2 y
x2 ReLU
w3
x3
Final Answer:
Output y = 0
Updated weight w2 = 0.5 (Gradient is 0 due to inactive ReLU)
3
Problem 7: Cross-Entropy Loss Calculation
A network with a 2-2-1 architecture produces a final predicted probability y = 0.8 at the output node (using
a Sigmoid activation). If the true target label is t = 0, calculate the Cross-Entropy Loss E = −[t ln y + (1 −
t) ln(1 − y)].
Final Answer:
Error E ≈ 1.6094
Problem 8: Linear Perceptron Update
A neuron with a linear activation function (identity) has inputs x1 = 0.5, x2 = 0.5 with weights w1 = 0.4, w2 =
0.4 and bias b = −0.2. Given a target t = 1.0 and learning rate η = 0.2, calculate the updated weight w1
using Stochastic Gradient Descent. Loss function E = 12 (y − t)2 .
Final Answer:
Updated weight w1 = 0.48
Problem 9: Sigmoid Derivative Evaluation
During backpropagation, the weighted sum at a hidden node is found to be z = 1.5. Evaluate the output of
the node h = σ (z) and the derivative of the activation function with respect to z, given by σ ′ (z).
Final Answer:
h ≈ 0.8176
σ ′ (z) ≈ 0.1491
Problem 10: Tanh Activation Forward Pass
z −e−z
A hidden neuron uses the hyperbolic tangent activation function tanh(z) = eez +e −z . It receives inputs x1 =
0.8, x2 = −0.4 with respective weights w1 = 1.5, w2 = 2.0. The bias is b = 0.1. Calculate the final output h
of this neuron.
x1
w1 =
1.5 b = 0.1
h
tanh
2.0
w2 =
x2
Final Answer:
Output h ≈ 0.4621