DEEP
LEARNING
Convolution Operations
Interactive chart
2
Convolution Kernels
Also known as filters, are small matrices used for convolution operation.
These kernels slide over across the input data, performing element-wise multiplication with
the corresponding pixels and producing a feature map that highlights specific patterns or
features in the input.
Convolution Operation
The process of applying a convolution kernel to the input data is convolution operation.
This operation involves sliding the kernel across the input, computing the dot product at each
position, and generating an output feature map.
Features
In the context of CNNs, features refer to the meaningful patterns or characteristics
extracted from the input by the convolutional layers.
These features represent different aspects of the input, such as edges, textures, or more
complex structures.
Each convolutional kernel is responsible for detecting specific features in the input data.
Features
Convolutional kernels (also known as filters) are updated during the backpropagation
process in CNNs.
Features
We can add zero padding around the entire image in order to calculate the kernel
convolution of the image without losing information.
Convolution Kernels
A kernel is a small 2-D matrix whose contents are based upon the operations to be
performed.
A kernel maps on the input image by simple matrix multiplication and addition, the output
obtained is of lower dimensions and therefore easier to work with.
Identity Kernel
Let the original image is:
• A = [[ 52, 55, 61],
• [ 63, 59, 55],
• [ 62, 59, 68]]
If we apply convolution kernel with contents:
• K = [[ 0, 0, 0],
• [ 0, 1, 0],
• [ 0, 0, 0]]
Identity Kernel
• K⊙A = 0×52 + 0×55 + 0×61 + 0×63 + 1×59 + 0×55 + 0×62 + 0×59 + 0×68 = 59
• Result (centre pixel): 59 → same as original
• No change to the image
• It essentially copies the input directly to the output
• Acts as the identity in convolution operations
Gaussian Blur Kernel
Let the original image is:
• A = [[ 52, 55, 61],
• [ 63, 59, 55],
• [ 62, 59, 68]]
If we apply convolution kernel with contents:
• K = 1/16 [[ 1, 2, 1],
• [ 2, 4, 2],
• [ 1, 2, 1]]
Gaussian Blur Kernel
• Blurred value = 1/16 × (1×52 + 2×55 + 1×61 + 2×63 + 4×59 + 2×55 + 1×62 + 2×59 +
1×68) =1/16 × (52 + 110 + 61 + 126 + 236 + 110 + 62 + 118 + 68) = 94316≈58.94
• (centre pixel): 58.94 → Its bit blurred.
• Smoothens the image
• Since it's a 3×3 image, applying a full 3×3 kernel gives only 1 output pixel (center), unless
padding is used.
Sharpen Kernel
Let the original image is:
• A = [[ 52, 55, 61],
• [ 63, 59, 55],
• [ 62, 59, 68]]
If we apply convolution kernel with contents:
• K = [[ 0, -1, 0],
• [ -1, 5, -1],
• [ 0, -1, 0]]
Sharpen Kernel
• Sharpened value = 0×52 + (−1)×55 + 0×61 + (−1)×63 + 5×59 + (−1)×55 + 0×62 +
(−1)×59 + 0×68 = −55 −63 + 295 −55 −59 = 63
• (centre pixel): 63
• This enhances contrast and sharp edges.
Edge Detection Kernel
Let the original image is:
• A = [[ 52, 55, 61],
• [ 63, 59, 55],
• [ 62, 59, 68]]
If we apply convolution kernel with contents:
• Sobel X = [[ -1, 0, 1],
• [ -2, 0, 2],
• [ -1, 0, 1]]
Edge Detection Kernel
Let the original image is:
• A = [[ 52, 55, 61],
• [ 63, 59, 55],
• [ 62, 59, 68]]
If we apply convolution kernel with contents:
• Sobel Y = [[ -1, -2, -1],
• [ 0, 0, 0],
• [ 1, 2, 1]]
Edge Detection Kernel (Cont..)
• Gx = (−1×52) + (0×55) + (1×61) + (−2×63) + (0×59) +(2×55) + (−1×62) + (0×59) + (1×68) = −52 + 0
+ 61 − 126 + 0 +110 − 62 + 0 + 68 = −1
• Gy = (−1×52) + (−2×55) + (−1×61) + (0×63) + (0×59) + (0×55) + (1×62) + (2×59) + (1×68) = − 52
−110 − 61 + 0 + 0 + 0 + 62 + 118 + 68 = −43
• Result (centre pixel): 43
• This shows the strength of the edge at the center.
Convolution Kernels (Cont..)
Above is an example of a kernel for applying Gaussian blur (to smoothen the image), sharpen
image (enhance the depth of edges) and edge detection.
Convolution Kernels (Cont..)
The shape of a kernel is heavily dependent on the input shape of the image and architecture of the
entire network, mostly the size of kernels is (MxM) i.e a square matrix.
The movement of a kernel is always from left to right and top to bottom.
Convolution Kernels (Cont..)
Stride defines by what step does to kernel move, for example stride of 1 makes kernel slide by one
row/column at a time and stride of 2 moves kernel by 2 rows/columns.
Multiple kernels aka filters with stride=1
Convolution Kernels (Cont..)
In the context of Convolutional Neural Networks (CNNs), a “filter” is another term used
interchangeably with “convolution kernel” or “convolution matrix.”
It refers to a small matrix of weights that is convolved with the input data to perform feature
extraction.
When a filter is applied to the input data using the convolution operation, it slides or convolves
across the input, computing the dot product between its weights and the corresponding pixels of the
input data.
This process generates a feature map that highlights specific patterns or characteristics present in
the input.
Sliding Window Protocol
1. The kernel gets into position at the top-left corner of the input matrix.
2. Then it starts moving left to right, calculating the dot product and saving it to a new matrix until it
has reached the last column.
3. Next, kernel resets its position at first column but now it slides one row to the bottom. Thus,
following the fashion left-right and top-bottom.
4. Steps 2 and 3 are repeated till the entire input has been processed.
• For a 3D input matrix, the movement of the kernel will be from front to back, left to right and top
to bottom.
Mathematical Understanding
• For a grayscale image, let's say the image size is 24x24x1 (24 is the width, 24 is the height, 1 is the
number of color channels, since its a grayscale image, it will have 1 channel.
• If it would have been a colored image, number of channels would have been 3).
• And the kernel size is 3x3x1x64 (3 is the width, 3 is the height, 1 is the number of color channel of
the kernel, 64 is the number of such kernels).
• Then after the convolution operation the size of the output image will be 22x22x64 ((24–
3+1)x(24–3+1)x64)
Mathematical Understanding
• Convolution operation happening on a grayscale image where size of the image is n x n x 1 and
size of the kernel is f x f x 1 x c.
• Here c is the number of kernels and not the number of channels
Mathematical Understanding
• If the image is a colored one, lets consider the size of the image as 24 x 24 x 3.
• For performing convolution operation on a colored image, the kernel should also have 3 channels),
kernel size as 4 x 4 x 3 (where 3 is the number of channels).
• The size of the image after the convolution operation will be (21 x 21 x 1).
• Here 1 denotes the number of color channels, which means after the convolution operation,
grayscale image is obtained.
• This is a common scenario in convolutional neural networks, where intermediate layers may
reduce the number of channels while preserving spatial information.
Mathematical Understanding
• NOTE: Here only 1 kernel is being used
• If there would have been multiple kernels let's say 64, then the size of the image would have been
21 x 21 x 1 x 64.
Residual/Skip Connections
Many modern deep nets contain a very large number of layers
In general, just stacking lots of layer doesn’t necessarily help a deep learning
model
Vanishing/exploding gradient may make learning difficult
Skip connections or “residual connections” help if we want very deep networks
This idea was popularized by “Residual Networks”* (ResNets) which can have hundreds of
layers
Basic idea: Don’t force a layer to learn everything about a mapping
Add footer here 27
May need to perform an
additional
projection/adjustment to that
the sizes of 𝑥 and 𝑔(𝑥) match
Added a “residual branch” or
“short-cut” connection to
connect 𝑥 to the residual output
𝑔(𝑥) of these layers
These layers
trying to learn Reducing their burden by
some function just asking them to learn
𝑓(𝑥) the “residual” 𝑔 𝑥
= 𝑓 𝑥 −𝑥
Add footer here 28
29
Add footer here 30
Efficient Convolution
• Normal Convolution operation
• Input data of size Df x Df x M. Image size (Df x Df )
• Suppose there are N filters/kernels of size Dk x Dk x M.
• The output size will be Dp x Dp x N.
The number of multiplications in 1 convolution operation = size of filter = Dk x Dk x M.
The total number of multiplications become N x Dp x Dp x (Multiplications per convolution).
Total number of multiplications = N x Dp2 x Dk2 x M.
31
32
33
• Depth-Wise Convolutions
• In depth-wise operation, convolution is applied to a single channel at a time unlike standard CNN's
in which it is done for all the M channels.
• So here the filters/kernels will be of size Dk x Dk x 1.
• Given there are M channels in the input data, then M such filters are required.
• Output will be of size Dp x Dp x M.
34
• A single convolution operation require Dk x Dk multiplications.
• The total number of multiplications is equal to M x Dp x Dp x Dk x Dk.
• Total no of multiplications = M x Dk2 x Dp2 .
Add footer here 35
• Point-Wise Convolutions
• In point-wise operation, a 1x1 convolution operation is applied on the M channels.
• So the filter size for this operation will be 1 x 1 x M. Say we use N such filters, the output size
becomes Dp x Dp x N.
• A single convolution operation require 1 x M multiplications.
• The total number of multiplications is equal to M x Dp x Dp x (no. of filters).
• Total no of multiplications = M x Dp2 x N
36
• Total multiplications = Depth wise conv. multiplications + Point wise conv.
multiplications
•
• Total multiplications = M * Dk2 * Dp2 + M * Dp2 * N = M * Dp2 * (Dk2 + n).
• Depth wise separable convolution operation:
• Total no of multiplications = M * Dp2 * (Dk2 + N).
37
Grouped Convolution.
1. Grouped convolution was first introduced in 2012 in the AlexNet paper.
2. A deep neural network with multiple kernels per layer that lead to multiple channels per layer.
This resulted in a wider neural network.
3. Each kernel filter convolves on all the feature maps obtained on the previous layer, resulting in
lots of convolutions.
4. We may use a smaller batch size which would make the overall convergence difficult.
5. Filter grouping decreases the complexity of CNN and thus facilitating the training of the larger
neural networks.
38
• The third dimension of convolution filter, as we go deeper into the convolution this dimension increases
very rapidly thus increases complexity.
• The spatial dimensions have some degree of effect on the complexity but in deeper layers, they are not
really the cause of concern.
39
• The convolution with 2 filter groups that are created from one, each of the 2 filter groups
convolves with only half of the channel dimension of the previous layer.
• Let's consider a convolution layer that has input and output channel depth of c in and cout.
• If the number of filter groups is n, then the set of filter shapes of a convolution layer with the
logarithmic filter grouping would be-
40
41
42
43
Thank you