Scribd
Upload
11 views3 pages

Neural Network Training Loop Example

The document outlines a Python implementation of a neural network training loop using a spiral dataset with 500 points and 3 classes. It includes the creation of a neural network with two hidden layers, the use of the Adam optimizer, and the training process over 10,000 epochs, calculating loss and accuracy at intervals. Finally, it visualizes the training data and the predictions made by the neural network.

Uploaded by

22146119
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
11 views3 pages

Neural Network Training Loop Example

The document outlines a Python implementation of a neural network training loop using a spiral dataset with 500 points and 3 classes. It includes the creation of a neural network with two hidden layers, the use of the Adam optimizer, and the training process over 10,000 epochs, calculating loss and accuracy at intervals. Finally, it visualizes the training data and the predictions made by the neural network.

Uploaded by

22146119
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

# Example of Training in loop: running for 10000 times

import numpy as np
import Activation as activation # type: ignore
import Dense as layer # type: ignore
import Optimizer as optimizer # type: ignore
import Points as point # type: ignore
import [Link] as plt

n_points = 500
n_classes = 3
Training_Data = [Link](n_points,n_classes,2)

# Create the NN with 2 inputs, 2 hidden layers (32 neurons) and 3 outputs:
Dense1 = [Link](2,32)
activation1 = activation.Activation_ReLU()
Dense_dropout1 = layer.Dense_Dropout(0.0)
Dense2 = [Link](32,32)
activation2 = activation.Activation_ReLU()
Dense_dropout2 = layer.Dense_Dropout(0.0)
Dense3 = [Link](32,n_classes)
loss_activation = activation.Activation_Softmax_Loss_CategoricalCrossEntropy()
[Link]()

# Create the optimizer (to updates the NN's weights and biases)

decay = 0.00001
momentum = 0.4

# Adam parameters
learning_rate = 0.05
decay = 1e-3
epsilon = 1e-7
beta1 = 0.5
beta2 = 0.5

#optimizer = optimizer.Optimizer_SGD_Decay_Momentum(1,decay,momentum)
optimizer = optimizer.Optimizer_Adam(learning_rate, decay, epsilon, beta1, beta2)

# Train in loop
for epoch in range(10001):
# Perform a forward pass
[Link](Training_Data.P)
[Link]([Link])
Dense_dropout1.forward([Link])
[Link](Dense_dropout1.output)
[Link]([Link])
Dense_dropout2.forward([Link])
[Link](Dense_dropout2.output)

# Calculate loss & accuracy


loss = loss_activation.forward([Link], Training_Data.L)

predictions = [Link](loss_activation.output, axis = 1)


if len(Training_Data.[Link]) == 2:
Training_Data.L = [Link](Training_Data.L, axis=1)
accuracy = [Link](predictions == Training_Data.L)

# print if epoch % 100 = 0


if not epoch % 100:
print(f'epoch: {epoch}, ' +
f'acc: {accuracy:.3f}, ' +
f'loss: {loss:.3f} ' +
f'LR: {optimizer.current_learning_rate:.03f}')

# Back propagation
loss_activation.backward(loss_activation.output,Training_Data.L)
[Link](loss_activation.dinputs)
Dense_dropout2.backward([Link])
[Link](Dense_dropout2.dinputs)
[Link]([Link])
Dense_dropout1.backward([Link])
[Link](Dense_dropout1.dinputs)
[Link]([Link])

# Update weights and biases using optimizer


optimizer.pre_update_params()

optimizer.update_params(Dense1)
optimizer.update_params(Dense2)
optimizer.update_params(Dense3)

optimizer.post_update_params()

# Final result of classification of data


print(predictions) # Contain the information of classification of the NN
print(Training_Data.L) # The training result vector

# Visualization of data (2D):


[Link](1)
[Link](Training_Data.P[:, 0], Training_Data.P[:, 1], c=Training_Data.L,
s=20, cmap=[Link])
[Link]("Training data")

[Link](2)
[Link](Training_Data.P[:, 0], Training_Data.P[:, 1], c=predictions, s=20,
cmap=[Link])
[Link]("Data predicted by the NN")
[Link]()

You might also like