Chapter 5 Notes

Computer Vision

Computer Vision
Computer vision is a field of artificial intelligence that enables machines to see, interpret and analyse
visual data like images & videos. Computer Vision uses AI algorithms to recognize objects, detect patterns
and extract meaningful information.

Computer Vision vs. Image Processing

Computer Vision Image Processing
• Computer vision deals with extracting meaningful • Image processing deals with processing
information from the input images/videos and the input images to enhance them or
using that to predict outcomes or detect patterns. prepare them for further processing tasks.
• Computer vision is a superset of image processing. • Image processing is a subset of CV.
• Examples – object detection, handwriting • Examples – rescaling image, correcting
recognition. brightness.

Applications of Computer Vision

• Self-Driving vehicles – Computer Vision is the fundamental technology behind the development of
autonomous vehicles. Using hands-free technology in cars involves the process of identifying objects,
getting navigational routes and environment monitoring.

• Facial Recognition - Security and surveillance systems use Computer Vision for facial recognition. It
can be either guest recognition or log maintenance of the visitors. It also finds its application in
unlocking phones and in schools for attendance systems based on facial recognition of students.

• Augmented reality (face filters) - Modern-day apps like Instagram and Snapchat have a lot of features
based on the usage of computer vision. The application of face filters is one among them. Through the
camera, the machine or the algorithm is able to identify the facial dynamics of the person and applies
the facial filter selected.

• Retail - Retailers can use CV techniques to track customers' movements through stores, analyse
navigational routes and detect walking patterns. Through security camera image analysis, a Computer
Vision algorithm can generate a very accurate estimate of the items available in the store, thus
helping in Inventory Management. Also, it can analyse the use of shelf space to identify suboptimal
configurations and suggest better item placement.
• Medical imaging and Diagnostics – Computer vision aids in medical imaging for accurate diagnostics.
It helps to create and analyse images, and helps doctors with their interpretation. The application is
used to read and convert 2D scan images into interactive 3D models that enable medical professionals
to gain a detailed understanding of a patient's health condition.

• Google's Search by Image - This uses Computer Vision as it compares different features of the input
image to the database of images and gives us the search result while at the same time analysing
various features of the image.

• Google Translate App - When you point your phone’s camera at a sign written in a foreign language,
the Google Translate app uses CV to understand and translate the text almost instantly. First, the app
uses Optical Character Recognition (OCR), a Computer Vision technique, to see and recognize the text
in the image. OCR detects the letters, words, and sentences from the signboard. Next, the recognized
text is sent to the language translation system, which converts the foreign language into your
preferred language. Finally, Augmented Reality (AR) is used to overlay the translated text back onto
the original image in real time. This makes it appear as if the sign itself has been replaced with the
translated version on your screen.

Computer Vision Tasks

CV tasks or algorithms are a set of computational techniques designed to analyze, interpret and
understand visual data captured by cameras or other imaging devices.
The tasks used in a computer vision application are:

For Single Objects

Classification – Classification is the process of identifying or categorizing objects or images into different
classes or categories. A label is assigned to an image based on its content.
Classification is done using a classifier algorithm, ie, a specialized
algorithm that analyzes the visual features of the input image and
predicts the most appropriate label or category.
Classifiers can be binary classifiers and multiple classifiers.
A binary classifier is trained to predict two classes of an object,
while multiple classifiers can predict more than two classes.
Localization – Localization refers to accurately determining the
location and position of objects within an image.
The output of image localization is shown by
a bounding rectangular box around the object. This box indicates the spatial coordinates of the object
withing the image.

For Multiple Objects

Object Detection – Object detection is a CV algorithm which handles both classification and localization of
multiple objects present in an image. The algorithm is trained to identify and categorize various objects
into their respective classes or labels. It also provides the position and coordinates of the object within
the image by putting the object in bounding boxes.

Image Segmentation (Instance Segmentation) - An image is a

collection of different pixels. Dividing an image into different
regions or segments based on certain common characteristics of
the pixels is called Image segmentation. Image segmentation
gives a more precise and detailed shape of the object.

Basics of Images
Pixel
The word "pixel" means a picture element. Every photograph, in digital form, is made up of pixels. They
are the smallest unit of information that make up a picture. Usually round or square, they are typically
arranged in a 2-dimensional grid. The more pixels you have, the more closely the image resembles the
original.

Resolution
The number of pixels in an image is called the resolution. It refers to the amount of detail an image, video,
or display can show. Higher resolution images have more pixels creating high quality sharp images.
For example, a monitor resolution of 1280x1024, means there are 1280 pixels from one side to the other,
and 1024 from top to bottom. A 5 mega pixel camera (a megapixel is a million pixels) means the pixels
along the width multiplied by the pixels along the height of the image taken by the camera equals 5
million pixels.

Pixel Value
Pixel Value refers to the numerical representation of the colour or intensity of a specific pixel in an image.
The most common pixel format is the byte image, where this number is stored as an 8-bit integer giving a
range of possible values from 0 to [Link], zero is to be taken as no colour or black and 255 is taken
to be full colour or white.

Grayscale images - Grayscale images are images that contain shades of gray, without apparent colour.
In a grayscale image, each pixel stores 1 byte. The pixel value represents the brightness or intensity of the
pixel, and ranges from 0-255. The darkest possible shade is black, which is the total absence of colour or
zero value of pixel. The lightest possible shade is white, which is the total presence of colour or 255 value
of a pixel. Intermediate shades of gray are represented by equal brightness levels of the three primary
colours. Grayscale images have only one channel and so has one 2D array of pixels. The size of a grayscale
image is defined as the Height x Width of that image.
RGB images - Each colour image is stored in the form of three different channels, namely R channel, G
channel and B channel, each having different colour intensities. All three channels combine to form the
colour we see. Each pixel stores 3 bytes and each channel uses 1 byte, with values ranging from 0-255.
Each pixel has a set of three different values which together give colour to that particular pixel. An RGB
image is represented as a 3D array consisting of three 2D arrays for red, green, and blue channels.

Image Features
In computer vision and image processing, a feature is a piece of information that is relevant for solving
the computational task related to a certain application. Features may be specific structures in the image
such as points, edges, or objects.
Edges: CV algorithms identify the edges between different objects or regions in an image by analysing the
contrast (changes in colour or intensity) between adjacent pixels.
Corners: CV can also identify points on an image where the intensity changes significantly in different
directions.
Textures: CV algorithms analyse variations in pixel values such as smooth, rough or granular textures.

The blue patch is a flat area and difficult to find and track. Wherever you
move the blue patch it looks the same.
The black patch has an edge. Moved along the edge (parallel to edge), it
looks the same.
The red patch is a corner. Wherever you move the patch, it looks
different, therefore it is unique.

Convolution
Convolution is a common tool used for image editing. A convolution is an element-wise multiplication of
image arrays and another array called the kernel followed by sum.

A Kernel or a Filter is a matrix, which is slid across the image and multiplied with the input such that the
output is enhanced in a certain desirable manner. Each kernel has a different value for different kinds of
effects that we want to apply to an image. In Image processing, we use the convolution operation to
extract the features from the images which can later be used for further processing especially in
Convolution Neural Network (CNN).
Convolution Neural Networks
A Convolutional Neural Network (CNN) is a Deep Learning algorithm that can take in an input image,
assign importance (learnable weights and biases) to various aspects/objects in the image, and be able to
differentiate one from the other. CNNs focus on edges, textures and object shapes when recognizing
familiar objects, like the human brain.

In the diagram, we give an input

image, which is then processed
through a CNN and then gives a
prediction based on the label
given in the particular dataset.

Layers of CNN
The different layers of a Convolutional Neural Network (CNN) are as follows:

1. Convolutional Layer
It is the first layer of a CNN. The objective of the Convolution Operation is to extract the high-level
features such as edges, from the input image. CNN need not be limited to only one Convolutional
Layer. The first Convolution Layer is responsible for capturing the Low-Level features such as edges,
colour, gradient orientation, etc. With added layers, the High-Level features is captured as well.
In the convolution layer, several kernels are used to perform convolution operations to produce
several features. The output of this layer is called the feature map or the activation map.

Uses of Feature maps

• We reduce the image size so that it can be processed more efficiently.
• We only focus on the features of the image that can help us in processing the image further.
For example, you might only need to recognize someone's eyes, nose, and mouth to recognize the
person. You might not need to see the whole face.
2. Rectified linear unit (ReLU)
The next layer in the Convolution Neural Network is the Rectified Linear Unit function or the ReLU
layer. After we get the feature map, it is then passed onto the ReLU layer. This layer gets rid of all the
negative numbers in the feature map and lets the positive number stay as it is.

The process of passing it to the ReLU

layer introduces non - linearity in the
feature map.

After applying the ReLu function, we

can see a more abrupt color change
which makes the edges more obvious
and acts as a better feature for the
further layers in a CNN as it enhances the activation layer.

3. Pooling Layer
The Pooling layer is responsible for reducing the spatial size of the Convolved Feature while still
retaining the important features.
Two types of pooling can be performed on an image.
1. Max Pooling: Max Pooling returns the maximum value from the
portion of the image covered by the Kernel.
2. Average Pooling: Average Pooling returns the maximum value
from the portion of the image covered by the Kernel.

Uses of Pooling
1. Makes the image smaller and more manageable.
2. Makes the image more resistant to small transformations, distortions, and translations in the input
image.

4. Fully Connected Layer

The final layer in the CNN is the Fully Connected Layer (FC layer). The objective of a fully connected
layer is to take the results of the convolution/pooling process and use them to classify the image into
a label.
The output of convolution/pooling is flattened into a single vector of values, each representing a
probability that a certain feature belongs to a label.
For example, if the image is of a cat, features representing things like whiskers or fur should have high
probabilities for the label "cat".