Scribd
Upload
4 views5 pages

CNN Notes2

This document presents a numerical example of a Convolutional Neural Network (CNN) using a 5x5 grayscale input image, detailing the convolutional layer with two filters, ReLU activation, max-pooling, and a fully connected layer. The CNN predicts Class 2 with a probability of approximately 0.4853 after processing the input through various layers. Additionally, a Python implementation using PyTorch is provided to replicate the CNN operations.

Uploaded by

sreeja
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
4 views5 pages

CNN Notes2

This document presents a numerical example of a Convolutional Neural Network (CNN) using a 5x5 grayscale input image, detailing the convolutional layer with two filters, ReLU activation, max-pooling, and a fully connected layer. The CNN predicts Class 2 with a probability of approximately 0.4853 after processing the input through various layers. Additionally, a Python implementation using PyTorch is provided to replicate the CNN operations.

Uploaded by

sreeja
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Convolutional Neural Network Numerical Example

This document provides a detailed numerical example of a Convolutional Neural Network (CNN) with
minimal dimensions: input image size = 5 × 5 (grayscale, single channel), batch size = 1, convolutional
layer with 2 filters (3 × 3 kernel, stride = 1, no padding), ReLU activation, 2 × 2 max-pooling (stride =
2), and a fully connected layer with 3 output classes. General formulas are provided before numerical
computations, followed by a Python implementation using PyTorch tensors.

1 Input Image
The input is a 5 × 5 grayscale image (single channel):
 
1.0 0.5 0.3 0.7 0.2
0.4 1.2 0.8 0.6 0.9
 
X= 0.3 0.7 1.1 0.5 0.4
0.8 0.6 0.4 1.0 0.3
0.2 0.9 0.5 0.7 1.3

2 Convolutional Layer
The convolutional layer uses 2 filters, each with a 3 × 3 kernel, stride = 1, no padding. The output size
is calculated as:
Win − K + 2P Hin − K + 2P
Wout = ⌊ ⌋ + 1, Hout = ⌊ ⌋+1
S S
where Win = Hin = 5 (input width/height), K = 3 (kernel size), P = 0 (padding), S = 1 (stride). Thus:
5−3+0
Wout = Hout = ⌊ ⌋+1=3
1
Output size: 3 × 3 per filter.

2.1 Filter Weights


• Filter 1 (W1 ):  
0.1 0.2 0.3
W1 = 0.4 0.5 0.6
0.7 0.8 0.9

• Filter 2 (W2 ):  
−0.1 −0.2 −0.3
W2 =  0.2 0.3 0.4 
−0.5 −0.6 −0.7

• Biases: b1 = 0.1, b2 = 0.2

2.2 Convolution Operation


For a filter W and input region Xi,j of size K × K, the convolution output at position (i, j) is:

∑ K−1
K−1 ∑
F (i, j) = X(i + m, j + n) · W (m, n) + b
m=0 n=0

where b is the bias.

1
2.2.1 Filter 1
For position (1,1):
   
X(1, 1) X(1, 2) X(1, 3) 1.0 0.5 0.3
Region = X(2, 1) X(2, 2) X(2, 3) = 0.4 1.2 0.8
X(3, 1) X(3, 2) X(3, 3) 0.3 0.7 1.1

∑
2 ∑
2
F1 (1, 1) = X(1 + m, 1 + n) · W1 (m, n) + b1
m=0 n=0

= (1.0 · 0.1 + 0.5 · 0.2 + 0.3 · 0.3 + 0.4 · 0.4 + 1.2 · 0.5 + 0.8 · 0.6 + 0.3 · 0.7 + 0.7 · 0.8 + 1.1 · 0.9) + 0.1
= (0.1 + 0.1 + 0.09 + 0.16 + 0.6 + 0.48 + 0.21 + 0.56 + 0.99) + 0.1 = 3.29 + 0.1 = 3.39
Computing for all positions, the output feature map is:
 
3.39 3.05 2.92
F1 = 3.07 3.38 2.79
2.67 2.98 2.54

2.2.2 Filter 2
For position (1,1):
∑
2 ∑
2
F2 (1, 1) = X(1 + m, 1 + n) · W2 (m, n) + b2
m=0 n=0

= (1.0·(−0.1)+0.5·(−0.2)+0.3·(−0.3)+0.4·0.2+1.2·0.3+0.8·0.4+0.3·(−0.5)+0.7·(−0.6)+1.1·(−0.7))+0.2
= (−0.1 − 0.1 − 0.09 + 0.08 + 0.36 + 0.32 − 0.15 − 0.42 − 0.77) + 0.2 = −0.86 + 0.2 = −0.66
Output feature map:  
−0.66 −0.51 −0.47
F2 = −0.70 −0.62 −0.45
−0.59 −0.53 −0.48

2.3 ReLU Activation


The ReLU activation function is:
f (x) = max(0, x)
Applied element-wise to each feature map:
• For F1 (all positive, unchanged):
 
3.39 3.05 2.92
F1 = 3.07 3.38 2.79
2.67 2.98 2.54

• For F2 (negative values become 0):


   
max(0, −0.66) max(0, −0.51) max(0, −0.47) 0 0 0
F2 = max(0, −0.70) max(0, −0.62) max(0, −0.45) = 0 0 0
max(0, −0.59) max(0, −0.53) max(0, −0.48) 0 0 0

2
3 Max-Pooling Layer
Apply 2 × 2 max-pooling with stride = 2. The output size is:
Win − K Hin − K
Wout = ⌊ ⌋ + 1, Hout = ⌊ ⌋+1
S S
where Win = Hin = 3, K = 2, S = 2:
3−2
Wout = Hout = ⌊ ⌋+1=1
2
Output size: 1 × 1 per feature map. The max-pooling operation is:
K−1 K−1
P (i, j) = max max F (i · S + m, j · S + n)
m=0 n=0

• For F1 : ][
3.39 3.05
Region =
3.07 3.38
P1 (1, 1) = max(3.39, 3.05, 3.07, 3.38) = 3.39
Output: [3.39]
• For F2 : [ ]
0 0
Region =
0 0
P2 (1, 1) = max(0, 0, 0, 0) = 0
Output: [0]
• Pooled output (flattened): [3.39, 0]

4 Fully Connected Layer


Map the pooled output to 3 classes using a fully connected layer. For input vector x (size N ), weight
matrix Wfc (size N × C), and bias bfc (size C), the output is:
y = x · Wfc + bfc
where N = 2 (pooled features), C = 3 (classes).
• Weight Matrix: [ ]
0.2 0.3 0.4
Wfc =
0.1 0.5 0.6

• Bias: bfc = [0.1, 0.2, 0.3]


• Input: x = [3.39, 0]
• Compute: [ ]
0.2 0.3 0.4
y = [3.39, 0] · + [0.1, 0.2, 0.3]
0.1 0.5 0.6
= [3.39 · 0.2 + 0 · 0.1, 3.39 · 0.3 + 0 · 0.5, 3.39 · 0.4 + 0 · 0.6] + [0.1, 0.2, 0.3]
= [0.678, 1.017, 1.356] + [0.1, 0.2, 0.3] = [0.778, 1.217, 1.656]

• Softmax:
exp(yi )
Softmax(yi ) = ∑C
j=1 exp(yj )

exp([0.778, 1.217, 1.656]) ≈ [2.177, 3.377, 5.237], Sum = 10.791


[ ]
2.177 3.377 5.237
, , ≈ [0.2018, 0.3129, 0.4853]
10.791 10.791 10.791
• Highest probability: Class 2 (0.4853).

3
5 Summary
For the input 5×5 grayscale image, the CNN predicts Class 2 with a probability of approximately 0.4853.

6 PyTorch Implementation
Below is a Python implementation using PyTorch tensors to perform the CNN operations described
above, without using the nn module. The code implements convolution, ReLU, max-pooling, and the
fully connected layer with the same tensor values.

import torch

# Input image (5x5, single channel)


X = [Link]([[
[1.0, 0.5, 0.3, 0.7, 0.2],
[0.4, 1.2, 0.8, 0.6, 0.9],
[0.3, 0.7, 1.1, 0.5, 0.4],
[0.8, 0.6, 0.4, 1.0, 0.3],
[0.2, 0.9, 0.5, 0.7, 1.3]
]])

# Filters (2 filters, 3x3 kernel)


W1 = [Link]([[
[0.1, 0.2, 0.3],
[0.4, 0.5, 0.6],
[0.7, 0.8, 0.9]
]])
W2 = [Link]([[
[-0.1, -0.2, -0.3],
[0.2, 0.3, 0.4],
[-0.5, -0.6, -0.7]
]])
filters = [Link]([W1, W2]) # Shape: (2, 1, 3, 3)
biases = [Link]([0.1, 0.2]) # Biases for 2 filters

# Convolution (stride=1, no padding)


def conv2d(X, filters, biases):
out_h = [Link][1] - [Link][2] + 1 # 5-3+1=3
out_w = [Link][2] - [Link][3] + 1 # 5-3+1=3
out = [Link](([Link][0], out_h, out_w))
for k in range([Link][0]): # For each filter
for i in range(out_h):
for j in range(out_w):
region = X[:, i:i+3, j:j+3]
out[k, i, j] = (region * filters[k]).sum() + biases[k]
return out

conv_out = conv2d([Link](0), filters, biases) # Shape: (2, 3, 3)

# ReLU
relu_out = [Link](conv_out, [Link](0.0))

# Max-Pooling (2x2, stride=2)


def maxpool2d(X, kernel_size=2, stride=2):
out_h = ([Link][1] - kernel_size) // stride + 1 # (3-2)//2+1=1
out_w = ([Link][2] - kernel_size) // stride + 1 # (3-2)//2+1=1
out = [Link](([Link][0], out_h, out_w))
for k in range([Link][0]):

4
for i in range(0, [Link][1] - kernel_size + 1, stride):
for j in range(0, [Link][2] - kernel_size + 1, stride):
out[k, i//stride, j//stride] = X[k, i:i+kernel_size, j:j+kernel_size].max()
return out

pool_out = maxpool2d(relu_out) # Shape: (2, 1, 1)


pool_flat = pool_out.flatten() # Shape: (2,)

# Fully Connected Layer


W_fc = [Link]([
[0.2, 0.3, 0.4],
[0.1, 0.5, 0.6]
])
b_fc = [Link]([0.1, 0.2, 0.3])
fc_out = pool_flat @ W_fc + b_fc

# Softmax
probs = [Link](fc_out) / [Link](fc_out).sum()

# Output probabilities
print(probs) # Expected: ~[0.2018, 0.3129, 0.4853]

You might also like