Scribd
Upload
9 views2 pages

DATA7703 Week 9 Tutorial Solutions

The document discusses deep architectures in neural networks, highlighting their biological inspiration, efficiency in function representation, and reduced need for feature engineering. It explains convolution operations, including filter application and parameter calculations, as well as the relationship between average pooling and convolutional filters. The tutorial solutions provide detailed mathematical examples and explanations related to these concepts.

Uploaded by

karandeepsinghsaluja963
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
9 views2 pages

DATA7703 Week 9 Tutorial Solutions

The document discusses deep architectures in neural networks, highlighting their biological inspiration, efficiency in function representation, and reduced need for feature engineering. It explains convolution operations, including filter application and parameter calculations, as well as the relationship between average pooling and convolutional filters. The tutorial solutions provide detailed mathematical examples and explanations related to these concepts.

Uploaded by

karandeepsinghsaluja963
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

DATA7703 Week 9 Tutorial Solutions

2025 Semester 2

1. First, deep architectures are found in biological neural networks, and this provides in-
spirations for how to construct deep artificial neural networks to solving problems that
biological neural networks are good at solving, such as vision problems.
Second, from the perspective of representing functions, deep neural networks can some-
times use much fewer neurons to represent a function as compared to shallower networks.
Third, from the perspective of developing machine learning algorithms that requires little
handcrafting, the additional layers in deep neural networks provides the possibility to
learn a hierarchy of increasing more abstract features from data, while in the case of
shallow networks, significant feature engineering is required.

2. (a) Convolution slides an array called the filter array through the input array, and com-
putes the inner product between the filter array with each input subarray encoun-
tered. In general, a bias may be added to the inner product.
(b) 3 − 1 − 2) + 1 = 1. We can similarly
i. The first entry in the output is given by(︂(1 + )︂
2 2
work out other entries. The answer is .
2 2
(︀ )︀
ii. The output is a 1x1 matrix 2 , as only the first 2 × 2 sub-array is considered.
(︀ )︀
iii. The output is a 1 × 1 matrix 5 , as (1 + 7 − 1 − 3) + 1 = 5.
⎛ ⎞
0 0 0 0 0
⎜0 1 2 3 0⎟
⎜ ⎟
iv. After padding, we obtain ⎜ ⎜0 1 3 5 0⎟.
⎟
⎝0 1 4 7 0⎠
0 0 0 0 0 (︂ )︂
2 2
Apply the filter with a stride of 2, we obtain .
1 2
⎛ ⎞
4 5 −2
v. ⎝ 3 5 −1⎠.
−2 −3 4
⌊︀ 𝑁1 −𝐹1 ⌋︀
(c) ⌊︀Using the formula in the lecture slides for both dimensions, we have 𝑆
+ 1 ×
𝑁2 −𝐹2
⌋︀
𝑆
+ 1 .
(d) There are 6 5 × 5 filters in C1. Each has 5 × 5 + 1 = 26 parameters, thus there are
6 × 26 = 156 parameters in total.
S2 does not have any parameter.
(︀ )︀
(e) i. 7/4 19/4 .
(︀ )︀
ii. 3 7 .
(f ) Average pooling is the same as a convolutional filter with bias 0, and weight array of
the same kernel size as average pooling, and consisting of numbers which are equal

1
and sum to 1. In addition, the stride of the convolutional filter at each dimension is
equal to the size of the kernel along that dimension. For example, (︂ a 2 × 3 average
)︂
1/6 1/6/ 1/6
pooling corresponds to a convolutional filter with weight matrix ,
1/6 1/6/ 1/6
bias 0, vertical stride 2, and horizontal stride 3.

You might also like