Tutorial 0809: Multilayer Perceptron (MLP)
Course: Machine learning
2024–08–20
1 Một số hàm cần thiết:
• Activation
1
g(z) = sigmoid(x) =
1 − ex
• Loss
L(y, ŷ) = y − ŷ
2 Thuật toán MLP với Input Layer (2 node) - Hidden layer
(4 neutral) - Output layer (1 node)
Bước 1: Thực hiện lan truyền tiến -> tính gía trị đầu ra của mạng
• layer1 = X ∗ w1 + b1
• activate1 = g(layer1)
• layer2 = activate1 ∗ w2 + b2
• ŷ = activate2 = g(layer2)
Bước 2: Tính độ lỗi
• error = y − ŷ
Bước 3: Thực hiện lan truyền ngược
1
• w′ 2 = activate1T ∗ (error ∗ g ′ (layer2))
• b′ 2 = Σ(error ∗ w2 T )∗g ′ (layer1)
• error_hidden = error ∗ (w2T ) ∗ g ′ (layer1)
• w′ 1 = X.T ∗ (error_hidden)
• b′ 1 = Σ(error_hidden)
Bước 4: Cập nhật trọng số cho mô hình
• w2 = w2 − lr ∗ w′ 2
• b2 = b2 − lr ∗ b′ 2
• w1 = w1 − lr ∗ w′ 1
• b1 = b1 − lr ∗ b′ 1
3 Code mẫu:
1 import numpy as np
2 from sklearn import datasets
3
4 # load iris dataset
5 iris = datasets . load_iris ()
6 X = iris [ " data " ][: , (2 , 3) ] # petal length, petal width
7 y = ( iris [ " target " ] == 2) . astype ( int ) # 1 if Iris-Virginica, else 0
8 y = y . reshape ([150 ,1])
1 def sigmoid ( z ) :
2 return 1 / (1 + np . exp ( - z ) )
3
4 def s ig mo id_ de ri vat iv e ( z ) :
5 s = sigmoid ( z )
6 return s * (1 - s )
1 class MLP :
2 def __init__ ( self , input_size , hidden_size , output_size , learning_rate
=0.01) :
3 self . input_size = input_size
4 self . hidden_size = hidden_size
5 self . output_size = output_size
6 self . learning_rate = learning_rate
7
8 # initialize weights randomly
9 self . weights1 = np . random . randn ( self . input_size , self . hidden_size )
10 self . weights2 = np . random . randn ( self . hidden_size , self . output_size )
11
12 # initialize biases to 0
13 self . bias1 = np . zeros ((1 , self . hidden_size ) )
14 self . bias2 = np . zeros ((1 , self . output_size ) )
15
16 def fit ( self , X , y , epochs =1000) :
2
17 for epoch in range ( epochs ) :
18 # feedforward
19 layer1 = X . dot ( self . weights1 ) + self . bias1
20 activation1 = sigmoid ( layer1 )
21 layer2 = activation1 . dot ( self . weights2 ) + self . bias2
22 activation2 = sigmoid ( layer2 )
23
24 # backpropagation
25 error = activation2 - y
26
27 d_weights2 = activation1 . T . dot ( error * si gm oid _d er iva ti ve ( layer2 ) )
28 d_bias2 = np . sum ( error * si gm oi d_d er iv at ive ( layer2 ) , axis =0 ,
keepdims = True )
29 error_hidden = error . dot ( self . weights2 . T ) * s igm oi d_ der iv at ive (
layer1 )
30 d_weights1 = X . T . dot ( error_hidden )
31 d_bias1 = np . sum ( error_hidden , axis =0 , keepdims = True )
32
33 # update weights and biases
34 self . weights2 -= self . learning_rate * d_weights2
35 self . bias2 -= self . learning_rate * d_bias2
36 self . weights1 -= self . learning_rate * d_weights1
37 self . bias1 -= self . learning_rate * d_bias1
38
39 def predict ( self , X ) :
40 layer1 = X . dot ( self . weights1 ) + self . bias1
41 activation1 = sigmoid ( layer1 )
42 layer2 = activation1 . dot ( self . weights2 ) + self . bias2
43 activation2 = sigmoid ( layer2 )
44 return ( activation2 > 0.5) . astype ( int )
1 # create an instance of the MLP class
2 mlp = MLP ( input_size =2 , hidden_size =4 , output_size =1)
3
4 # train the MLP on the training data
5 mlp . fit (X , y )
6
7 # make predictions on the test data
8 y_pred = mlp . predict ( X )
9
10 # evaluate the accuracy of the MLP
11 accuracy = np . mean ( y_pred == y )
12 print ( f " Accuracy : { accuracy :.2 f } " )
13
14 >> Accuracy : 0.97
4 Một số link tham khảo
• Sklearn
• Keras
-- Hết --