Feb 20, 2025
Class #3:
Related Terms
Dense Layer - A dense layer (also called a fully connected layer) is a
fundamental building block in artificial neural networks (ANNs). It is widely used
in architectures such as Feedforward Neural Networks (FNNs), Convolutional
Neural Networks (CNNs), and Transformers.
Convolutional - In Artificial Intelligence (AI), the term "convolutional" refers to
the mathematical operation of convolution, which is extensively used in
Convolutional Neural Networks (CNNs) for processing images, videos, and other
spatially structured data.
CNN - A Convolutional Neural Network (CNN) is a type of deep learning model
specifically designed for processing structured grid data, such as images and
videos. CNNs are widely used in computer vision tasks like image classification,
object detection, facial recognition, and medical image analysis.
Edge Detection Matrix
In AI and image processing, edge detection is a fundamental technique used to
identify boundaries and object structures within an image. Matrix dot products
(convolutions) play a crucial role in detecting edges.
Vertical Edge Detection
-1 0 1 | 1 0 -1
-1 0 1 | 1 0 -1
-1 0 1 | 1 0 -1
Horizontal Edge Detection
-1 -1 -1 | 1 1 1
0 0 0 | 0 0 0
1 1 1 | -1 -1 -1
Filter/Kernal - A filter (kernel) in a Convolutional Neural Network (CNN) is a
small matrix used to extract important features such as edges, textures, and
patterns from an image. Filters slide over the input image and perform
convolution operations, transforming the input data into a more useful
representation for classification, detection, or segmentation.
Stride - In Convolutional Neural Networks (CNNs), stride refers to the step
size at which the filter (kernel) moves across the input image during the
convolution operation. It determines how much the filter shifts at each step and
influences the output size of the feature map.
Dimensionality Reduction - In computer vision, dimensionality reduction can
be understood as a process that reduces the resolution or complexity of an
image while preserving important structural information. Blurring is one such
technique that reduces dimensionality by smoothing high-frequency details,
such as edges and textures, leading to a more simplified representation.
Auto Encoder
Latent Space - In encoder-decoder architectures, the latent space (also
called bottleneck representation or embedding space) is the intermediate
compressed representation of input data, capturing its most essential features
before reconstruction or transformation by the decoder.
Channel in Image
In computer vision and image processing, a channel represents a separate
component of an image that stores intensity values for a specific colour or
feature. The number of channels in an image depends on its colour model and
depth.
Grayscale - Single-Channel, Pixel values range from 0 (black) to 255 (white)
(for 8-bit images).
RGB - 3 channels, Red (R), Green (G), and Blue (B). Each pixel is a
combination of the three colour intensities.
RGBA - 4 Channels, includes an Alpha (A) channel for transparency. Used in
images requiring transparency effects.
CMYK - 4 Channels, Cyan (C), Magenta (M), Yellow (Y), and Black (K). Used in
printing applications.
Multi-Channel - Some images (e.g., hyperspectral images) have dozens to
hundreds of channels. Used in scientific imaging, medical scans, and satellite
images.
Formula and Notations:
The general formula to calculate the result of a single neuron in a layer is:
z = ( w1*x1 + w2*x2 + ... + wn*xn ) + b
Where:
z: The weighted sum of the inputs plus the bias. This is the value that will be
passed to the activation function.
w1, w2, ..., wn: The weights connecting the neuron to the inputs from the
previous layer.
x1, x2, ..., xn: The inputs from the previous layer.
b: The bias term.
—-----------------------------------------------------------------------
In matrix form:
Z=W*X+B
Where:
Z = Weighted sum (before activation)
W = Weight matrix
X = Input vector
B = Bias vector
Calculation of Number of Parameters:
Consider a layer with:
1 input layer with 784 neurons ( from an image of 28x28 pixels )
2 hidden layers with 16 neurons each
1 output layer with 10 neurons
Weights: ( neurons of input layer x neurons of hidden layer 1 + neurons of hidden layer
1 x neurons of hidden layer 2 + neurons of hidden layer 2 x neurons of output layer )
= ( 784 x 16 + 16 x 16 + 16 x 10 ) = 12544 + 256 + 160 = 12960
Bias: ( 16 + 16 + 10 ) = 42
Total number of parameters : 12960 + 42 = 13002
weights and biases in a convolutional neural network (CNN) :
In ANN we have weight and biases but in CNN we have filters and biases.
Key point:
Unlike dense layers where each neuron has its own bias, in convolutional layers, each filter
shares a single bias term across all spatial locations.