ê
Given - x =0.05
x 2 = 0·10
W4
W1 =0.15, W2 = 0.2, W3 = 0.25, W1
W5=0·4, W6= 0.45; W7=0.5
C Forward Pass
-;) Shi
Outh, =
120
43 20.25
the W2=012
W 0.55
Sha
11
1
1+e-03775
=
WG=0.47
6 Out b2
8
5055 Mi
;) Soi
Expected Output
01 = 0·01, 02 =
0.99 = 0.3, b, =
0.35,0
)
, Wg=0.55, b2 = 0.6
w1 x + W2x2 +b,
=
= 0.15 × 0.05 + 0.2 X 0.10 +0.35 =
0.3775
0.5932
Wz x + Wy x 2 + bj = 0.25 × 0.05 +0.3×0·1
+0.35
0.3925
1
Ite
0.3925
= 0·59689
W5, +wo h2 + b2 = 0.4x0.5933 +
0.45x
1.1059
0.5968970x
Outo
1
=
èl·1059
07514
14) So2 = with, +ligh 2 + b2 =
0.5x0.5933+0·55x0·59689
='1'
112250
+0.6
Out 02
S
1+1.225
Calculating total error
-
E
tota
l
EO,
Eoi
E02
E02
=
=
= 0·7729
ди
= '/1⁄2
Squared Frror fu
(target - putput)"
1⁄2 (target 0,- Out o)2 =
2 (0.01 -0.7514)2
2
= 0.2748
E total = Eoi +E02
=
1 1⁄2 (target 02 - Out o2)2 =
1/2 (0.99-0.772
0.02356
1.
= 0.2748 +0.02356 . Etotal =
0.298
Backward pass
Goal is to update each weight in
My
so that they cause actual off to be closer to
target off
error for each off neuron.
thereby minimizing i) so ws, we want to know how
much change in w
I Exokal (ive of Exotal [Link] partial derivat affects the total
error de
(this is
Gradient w.r.t. Ws)
ο ως
W5)
By applying chain rule, we know
that:
703
Exotal
Etotal
*
d out o
*
二
awp
dout of
Iso,
2W5
O
Now, need to figure out each piece in this equat.
First, how much does the total error change
with
respect to the Olp ?
Estota
l
い
1⁄2 (target - Out,)2 +
1⁄2 (target -Out (2)2
)
*
I Etoral £0, 2 + 1 (target-Outo,)
2 =" + -1 +0 constant 2 Outo = -
(target - Outo1)
= - (target - Outo1) =
-(0.01 -0.7514)
A 0.7414
ilp
Next, how much does op of O, change
w.r.t its `/p
*. SOT.
The partial derivative of the sigmoid
this ex
Outo,
1
Ité So,
Doutor
So,
Outo, (1 - Outo1) =
0.7314(-0.7514)
1
= 0.1868
Finally. How much does total i/p of 0,
change wirtus
two
6
So1 = wsouth, + wg Out h2 +
b2 * @ So1 = 1x Outh, x
w(i-1)
Jws
=
1x Outh, x W{ > +0 +0 =
Outh,
0.5933
20:0
Futting it all together
in eqn
E total
aws
0.7414 * 0.18680-59336
=0.0822
Now, to decrease the error, we subtract this value
from the current weight (multiplied with learning
rate = 0.5 consider)
Ws - n d Exotal
w5
1
or
Ws
new
= ws = 2 *
ws
Ws
ne
new
θως
=0·4- 0.5× 0.0822
= 0·3589
with repeating above steps to get we, w,
& Wg: -
WE new =
W7 new
W8 new =
news
= 0.4087
0.5113
0.5614
ii) Hidden
Layer
1
2861
we continew backpropagat for
these
with original weights & all are updated
after getting all new
weights!
Now we calculate new values for w,, w2, &
W3 &W,
I E Letal
I
Estotal
*
d out hi
a Shi
*
2
dw,
ωι
Jouth,
Here we use similar process but
slightly diff has accout the fact that the
Olp of each hidden layer neuron contributes
to the opp of multiple olp neurons. We
know that Outb. affects both Out
& Out 02 Therefore the
Etotal need to take into
consideration its
Out b
effect on the both off neurons:
Etotal 2 out hi
1
DEOL +
Jouth,
d to2
Out he
Jouth,
starting with 1st part of
equation
JEO, aso,
*
Jouth,
2
we can calculate
EOL
Eor
using
values we
calculated
earli
slova Eo,
aso,
2
* Outo,
0.7416 × 0.1868
=
Iso,
Outo
a 30,
0.1385
05:0
And 50, is equal to
ws:
= Ws "Outh, + wo⋅ Out b2 + b2
*
до
Out),
So,
aso1 =
Douthi
w6"
280
Ws to to
+0+05
0.45
804-0
Putting them in eq" 04
Outhe
Followi
ng
same process for
@EO2,
༡
egn
Eo
= 0.1385 × 0.4
= 0.0554
JEO2, we get! Jout 12
E total
JEO,
+
@
Outh,
2 E02
a
Outbr
2 Out 2
= −0·0191
2 Out hi
0.0554 + -0.0191 = 0.0364
Now, that we have IE total
20uth,
&
the a
20uth,
s
ashi
2 Outh,
J Shi
J
, we need to find
Two for each weight
cut the
Outh =
He-sh,
Out (1- Outh,) = 0.59
33 (1-0.5933)
h1
= 0.2413
We calculate the partial
derivative of total silp to "I w.r.t. W,
the same way we did for the off news,
= W1.x, + Wz⋅ x2 +
b1
Shi asn
B
2, +0 +0 = 0.05
it all together
Dw.
Puttin
g
I Etotal
Etotal x
2 Outh, x
asni
二
dash, dw,
деб
до
1
op
=X
Out hi
0.0364 0.2413 * 0.05
= 0.00044
We can now update w:
W1 new
= w1 - n. a Etotal
=
1
0.1498
| W
= 0.15-0.5 × 0.00044
W3
Repeating this for W2, wz &
w4=
W2
new
0.1996
Ws new
=
0·2498
8.2995
W4 new
=
we update all
weights! when we
ard the 0.05 & 0
Now, we
forward
On
the new was
fed
i/p originally, the erro
was 0.29837! After applying
this 1st round of backpropagat error is
now down to 0.29103.. But after
repeating proces 1000's time we get the
opp as expected.