Scribd
Upload
9 views3 pages

DNA Neural Network Calculations Guide

The document outlines various computational tasks related to neural networks, including calculations for batch size, activation functions, RNN outputs, CNN layer dimensions, and LSTM predictions. It also discusses the total number of trainable parameters in different architectures and the application of softmax activation for classification. Additionally, it explains concepts like computational graphs and the forward pass in neural networks.

Uploaded by

helladolla14
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
9 views3 pages

DNA Neural Network Calculations Guide

The document outlines various computational tasks related to neural networks, including calculations for batch size, activation functions, RNN outputs, CNN layer dimensions, and LSTM predictions. It also discusses the total number of trainable parameters in different architectures and the application of softmax activation for classification. Additionally, it explains concepts like computational graphs and the forward pass in neural networks.

Uploaded by

helladolla14
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

DNA Problems

1. Batch size = 250, total dataset size = 3000. Compute the number of iterations per epoch. To
process the dataset, we split it into batches.
● Batch size: Number of samples the model processes before updating weights.
● Iterations per epoch: Number of batches needed to cover the entire dataset.
Formula:

Substitute: 3000/250
2. Apply ReLU activation function on the flattened feature vector [-501, 32, -43, 14, 0].
3. Detect anomalies in a sequence [10, 20, 30, 300, 40, 50] using an RNN model.
4. Compute the output of an RNN where the input is [1, 2], weights are Win = 0.5, Whidden =
0.2, and activation is linear.
Explanation:
This is a vanilla RNN with:
● Linear activation (i.e., no non-linearity like ReLU or sigmoid)
● Initial hidden state h0=0h_0 = 0h0=0
● Equation:

5. Compute the hidden states of a vanilla RNN for the input sequence [-1,0,1], where the hidden
state at each step is calculated as ht=σ(W⋅xt+U⋅ht−1+b). Assume W=0.4, U=0.3, b=0.2, and σ
is the sigmoid function.
6. Apply the input X = [Link]([100, 10, 1],[10, 1, 100], [1,10,100]) to a trained model with
final weight matrix of the fully connected layer given as [Link]([[0.1, 0.2, 0.3],
7. A CNN model has the following layers: Input layer: 250×250×3. Conv2D Layer 1: Kernel
size: 3×3,Filters: 16,Stride: 1, Padding: "same". MaxPooling Layer 1: Pool size: 2×2, Stride:
2. Conv2D Layer 1: Kernel size: 3×3,Filters: 32,Stride: 2, Padding: "valid". Flatten
Layer:1,Fully Connected Layer: 500 neurons. Output Layer: 200 neurons (Softmax activation).
Calculate the size of the feature maps after each layer.
8. Evaluate the total number of trainable parameters in the entire architecture.
9. A CNN network is designed to take an input patch of size 5X5 given by I = [Link]([100
120 200 100 100], [110 121 201 101 101], [101 110 210 110 110], [200 220 220 120 100],[200
120 220 120 120]). The following CNN model is designed with Conv2D layer with a filter
kernel of dimension 3, K = [Link]([-1 -1 -1], [0 0 0], [-1 -1 -1]). Calculate the output feature
at the end of layer 1 with stride 1 and padding = 1. Also compute the dimensions of the output
features.
10. If an extra layer is added in the form of MaxPooling with a stride 2, Now calculate the
output.
11. A normalized weather data is defined as an input sequence: X = [0.3, ,0.5 ,0.7,0.9, 1.0]. The
target prediction is y=1.4. Apply the LSTM cell on the input sequence for training with to
predict the target variable. a. Calculate the output of the forget gate in the LSTM cell given the
hidden state and cell state initial values as [0,0]. Consider initial values of weights and biases
as [0.2; 0.2] and [0.1;0.2].
12. Calculate the loss after 1st iteration if the output of the LSTM cell is predicted as 1.45.
13. Given One feature, one sample, sequence length = 1, X = [0.4] that has to target a value of
y = 1.0. The following inputs an RNN module with initial hidden state 0, with Wx = 0.7, Wh
= 0.4, b = 0.2, Wy = 1.2, and by = 0.2. Calculate hidden state output during the forward pass.
13. Use the above computed hidden state output to calculate the output of the RNN cell.
14. Compute the output of an RNN where the input is [1, 2] weights are Win=0.6 and
Whidden=0.4, and the activation is linear.
15. Apply the input X = [Link]([100, 10, 1],[10, 1, 100],[1,10,100]) to a trained model with
final weight matrix of the fully connected layer given as [Link]([[0.1, 0.2, 0.3],
16. Compute the hidden states of a vanilla RNN for the input sequence [-1,0,1], where the
hidden state at each step is calculated as ht=σ(W⋅xt+U⋅ht−1+b). Assume W=0.5, U=0.4, b=0.1,
and σ is the sigmoid function.
17. A CNN model has the following layers: Input layer: 252×252×3. Conv2D Layer 1: Kernel
size: 3×3,Filters: 16,Stride: 1, Padding: "same". MaxPooling Layer 1: Pool size: 2×2, Stride:
2. Conv2D Layer 1: Kernel size: 3×3, Filters: 32,Stride: 2, Padding: "valid". Flatten
Layer:1,Fully Connected Layer: 504 neurons. Output Layer: 200 neurons (Softmax activation).
Calculate the size of the feature maps after each layer.
18. Evaluate the total number of trainable parameters in the entire architecture:
Given One feature, one sample, sequence length = 1, X = [0.3] that has to target a value of y =
2.0. The following inputs an RNN module with initial hidden state 0, with Wx = 0.8, Wh = 0.5,
b = 0.1, Wy = 1.2, and by = 0.2. Calculate hidden state output during the forward pass. Use the
above computed hidden state output to calculate the output of the RNN cell.
19. Classify the sequence [0,1,0,1] into two categories using a trained LSTM.
20. The logits for a multi class classification are given by Logits:[2.0, 1.0, 0.1]. Apply softmax
activation and calculate the probabilities.
21. Apply the GRU training model on a sequence [0.8,0.4,0.1]and compute the predictions for
the next state. Randomly assume weights and biases.
22. A sequence-to-sequence model consists of: Encoder: LSTM with input size = 15, hidden
size = 30. Decoder: LSTM with input size = 30, hidden size = 30. Calculate the total trainable
parameters in the encoder-decoder architecture
23. An RNN model has: Input size = 10 (number of features in each time step). Hidden size =
20 (number of neurons in the hidden layer). Calculate the total number of trainable parameters
in the RNN, including weights and biases.
24. Given One feature, one sample, sequence length = 1, X =[0.5] that has to target a value of
y = 1.0. The followinginputs an RNN module with initial hidden state 0, with Wx =0.8, Wh =
0.5, b = 0.1, Wy = 1.2, and by = 0.2.
25. Calculate hidden state output during the forward pass.
26. Use the above computed hidden state output to calculate the output of the RNN cell.
27. Explain the concept of a computational graph in deep learning.
28. Walk through the forward pass of a simple 2-layer Neural Network.
29. An Inception module uses: 1×1: 64 filters, input channels = 3 3×3: 128 filters, input from
1×1 (64 channels) 5×5: 32 filters, input from 1×1 (64 channels) Max pooling: 1×1 convolution
with 32 filters, input from max pooling (3 channels) Calculate the total trainable parameters in
the module. If the above module is used for image classification, what will be the dimensions
of the features at the end of the module.

Common questions

Powered by AI

The softmax function converts logits into probabilities: P(class i) = exp(logit_i) / sum(exp(logits)). For logits [2.0, 1.0, 0.1], the computations yield probabilities approximately [0.659, 0.242, 0.099], indicating the likelihood of each class .

With padding of 1 and stride 1, a 3x3 kernel outputs the dimensions as (original_size + 2*padding - kernel_size) / stride + 1 = 5. Applying the kernel computes each output element by convolving the kernel with the input elements it covers, contributing to the resulting feature map .

To calculate the number of iterations per epoch, divide the total dataset size by the batch size: 3000 / 250 = 12. Therefore, 12 iterations are required per epoch .

RNNs model sequential dependencies and can predict expected values. Anomalies are detected by significant deviations from predicted sequences, like '300' in [10, 20, 30, 300, 40, 50], indicating potentially aberrant behavior. RNNs learn temporal patterns, allowing identification and analysis of anomalies in context .

LSTM cells are more effective due to their ability to maintain long-term dependencies using gating mechanisms that control the cell state and hidden state updates. This alleviates the vanishing gradient problem often experienced by vanilla RNNs, which have difficulties learning long-term patterns .

After the first Conv2D layer with 'same' padding and 16 filters, the output size remains 250×250×16. After the MaxPooling layer with 2×2 pool size and stride 2, the dimensions reduce to 125×125×16. For the following Conv2D with 3×3 kernels, 'valid' padding, and stride 2, the resulting size is 62×62×32 .

The ReLU activation function outputs the input directly if it is positive; otherwise, it will output zero. Thus, applying ReLU to the feature vector [-501, 32, -43, 14, 0] results in [0, 32, 0, 14, 0].

In a 2-layer neural network, the input passes through the first layer where a weighted sum of inputs is computed, an activation function is applied, and then passed to the second layer. This second layer performs similar computations with its weights and activation function, producing the final output .

For the forget gate: f_t = σ(W_f * [h_t-1, x_t] + b_f). Given hidden and cell states start at 0, input X, and weights W and biases b as specified in the problem, calculations follow these steps. By substituting the values in, you find f_t for the input sequence, yielding the value according to LSTM dynamics .

To compute the trainable parameters, consider input-to-hidden (Wx), hidden-to-hidden (Wh), and hidden biases (b): Wx = 10 * 20 = 200 parameters, Wh = 20 * 20 = 400 parameters, and biases = 20 parameters. Adding these gives 620 total parameters .

You might also like