Chapter 1

Introduction to Image Processing

and Computer Vision
Computer vision is concerned with modelling and replicating
human vision using computer software and hardware.
Formally, computer vision is a discipline that studies how to reconstruct, interpret and
understand a 3D scene from its 2D images .

AI helps computers think, while computer vision helps them

perceive and understand the environment.
It imitates the human eye and is used to train models to
perform various functions with the help of cameras, algorithms,
and data rather than optic nerves, retinas, and visual cortex.
Evolution of CVIP

Computer vision (CV) and image processing are two closely related fields that utilize
techniques from artificial intelligence (AI) and pattern recognition to derive
meaningful information from images, videos, and other visual inputs.

Image processing is at the helm of the technical revolution that is being called Industry
4.0, laying the framework for new technologies in image data processing and object
recognition.

Image processing algorithms are used to extract information from images, restore
and compress image and video data, and build new experiences in virtual and
augmented reality.

Image Processing and Computer

Vision
 Computer vision uses image processing to recognize and categorize image
data. In fact, CV is becoming more adept at identifying patterns from images
than the human visual cognitive system.

CV technology is being used across industries from healthcare and media to

gaming and transportation.

For example, vehicles use CV to track objects and surroundings and to

enable driver assistance system and autonomous navigation.

Security systems use facial recognition and object detection in video

surveillance. Medical devices use image classification to identify and
diagnose conditions.

In sports, object and motion tracking is used to follow balls and players.
Drones with sophisticated cameras are used in agriculture to monitor the
health and yield of crops.

Describe the concept of computer vision does it require in image processing.

Computer vision is an interdisciplinary field that focuses on enabling computers to

interpret and understand images and videos like humans. It involves analyzing and
processing visual data from digital images and videos to extract meaningful
information that can be used for a wide range of applications, including object
recognition, face detection, medical imaging, and autonomous navigation.

Computer vision requires image processing, which involves the manipulation of

digital images using various techniques to extract useful features or enhance image
quality. Image processing techniques used in computer vision may include image
filtering, edge detection, feature extraction, image segmentation, and object
recognition. These techniques are used to identify and classify objects in an image,
track their movements, and predict their future behavior.

Overall, the concept of computer vision relies heavily on image processing

techniques to extract high-level information from digital images and videos,
enabling computers to perform complex tasks that would otherwise require
human intervention.

Describe various elements of digital image processing system.

1. Image Acquisition: This is the first step in which a digital image is captured
using cameras, scanners, or other imaging devices.

2. Pre-processing: In this stage, the acquired image is corrected for

distortions, noise reduction, and enhancement of features in
the image.

3. Segmentation: This is the process of dividing the image into different regions
based on the characteristics of the image's pixels.

4. Feature extraction: This involves identifying and extracting relevant

information from the segmented regions to produce accurate results.

5. Object Recognition: Object recognition is the process of identifying certain

objects or features in the image.

6. Image Analysis: The process of using mathematical techniques to evaluate data

Image Processing and Computer
Vision in order to uncover hidden patterns and relationships.
 7. Visualization: This is the final stage in which the processed image is presented
to the end- user in a visually pleasing way, such as image enhancement,
filtering, or modification.

8. Implementation: The final step involves implementing the digital image

processing system, which may involve automating or integrating the
system into an existing system or workflow.

IP vs CV

Image processing studies image to image transformation.

The input and output of image processing are both images.

Computer vision is the construction of explicit, meaningful descriptions of

physical objects from their image.

The output of computer vision is a description or an interpretation of

structures and 3D scene.

CV Models

A computer vision (CV) model is a processing block that takes uploaded inputs,
like images or videos, and predicts or returns pre-learned concepts or labels.

Examples of this technology include image recognition, visual recognition,

and facial recognition.

Image Classification

Image classification attempts to identify the most significant object class in

the image; in computer vision, we refer to each class as a label. For example,
we can use a general classification model, such as ‘alwaysai/googlenet’

Object Detection

When the location of the object is of importance, object detection DNNs [Deep
neural network] are typically used. These networks return a set of coordinates,
called a bounding box, that specify an area of the input image containing an
object, along with confidence value for that bounding box and a label.

Image Segmentation

As we described above, in some tasks it is important to understand the exact

shape of the object. This requires generating a pixel level boundary for each
object, which is achieved through image segmentation.

Object Landmark Detection

Object landmark detection is the labeling of certain ‘keypoints’ in images that

capture important features in the object. For our virtual wardrobe, we could
use a pose estimation model, such as ‘alwaysai/human-pose’, that identifies
body keypoints such as hips, shoulders and elbows, similar to the image
shown below, to help our users accessorize.

Training models

When you train a model, use images from the target environment, or from a
test environment that closely resembles the target environment. Consider the
following
Image Processing factors that can affect model performance:
and Computer
Vision
 Lighting

Resolution

Image distortion

Computer vision
model lifecycle

Image Filtering

Filtering is a technique for modifying or enhancing an image. For example, you

can filter an image to emphasize certain features or remove other features.
Image processing operations implemented with filtering include smoothing,
sharpening, and edge enhancement.

Smoothing
Smoothing is often used to reduce noise within an image.

Image smoothing is a key technology of image enhancement, which can

remove noise in images. So, it is a necessary functional module in various
image-processing software.

Image smoothing is a method of improving the quality of

images. Smoothing is performed by spatial and

frequency filters
Spatial
Image Processing andfiltering
Computer
Vision
 Spatial filtering term is the filtering operations that are performed directly
on the pixels of an image. The process consists simply of moving the filter
mask from point to point in an image.

Smoothing spatial

filters Sharpening

spatial filters

Smoothing Spatial Filters

Smoothing filters are

used for noise reduction
and blurring operations.

It takes into account the pixels surrounding it in order to make a

determination of a more accurate version of this pixel.

By taking neighboring pixels into consideration, extreme “noisy” pixels can

be filtered out.

Unfortunately, extreme pixels can also represent original fine details, which
can also be lost due to the smoothing process

Image Representations

In computer science, the representation of an image can take many forms.

Most of the time, it refers to the way that the conveyed information, such as
color, is coded digitally and how the image is stored, i.e., how is structured an
image file.

Several open or patented standards were proposed to create, manipulate

store and exchange digital images. They describe the format of image files,
the algorithms of image encoding such as compression as well as the format
of additional information often called metadata.

Differently, the visual content of the image can also take part in its
representation. This more recent concept has provided new approaches of
representation and new standards, gathered together into the discipline
named content-based image indexing.

Image Statistics Recognition Methodology: Conditioning, Labeling, Grouping,

Extracting, and Matching,

Conditioning
In an image, there are features which are uninteresting, either because
they were introduced into the image during the digitization process as noise,
or because they form part of a background.

An observed image is composed of informative patterns modified by

uninteresting random variations.

Conditioning suppresses, or normalizes, the uninteresting variations

in the image, effectively highlighting the interesting parts of the image.

Applied uniformly and in context-independent.

Image Processing and Computer
Vision
Labeling
 Image labeling is a type of data labeling that focuses on identifying and
tagging specific details in an image.

In computer vision, data labeling involves adding tags to raw data such as
images and videos. Each tag represents an object class associated with the
data.

Supervised machine learning models employ labels when learning to identify

a specific object class in unclassified data. It helps these models associate
meaning to data, which can help train a model.

Image annotation is used to create datasets for computer vision models, which
are split into training sets, used to initially train the model, and test/validation
sets used to evaluate model performance.

Data scientists use the dataset to train and evaluate their model, and then
the model can automatically assign labels to unseen, unlabelled data.

Why is Image Labeling Important for AI and Machine Learning?

Image labeling is a key component of developing supervised models with

computer vision capabilities. It helps train machine learning models to label
entire images, or identify classes of objects within an image. Here are several
ways in which image labeling helps:

Developing functional artificial intelligence (AI) models—image labeling tools

and techniques help highlight or capture specific objects in an image. These
labels make images readable by machines, and highlighted images often
serve as training data sets for AI and machine learning models.

Improving computer vision—image labeling and annotation helps improve

computer vision accuracy by enabling object recognition. Training AI and
machine learning with labels helps these models identify patterns until
they can recognize objects on their own.

Grouping
Grouping can turn edges into lines by determining that different edges belong
to the same spatial event.

A grouping operation, where edges are grouped into lines, is called line-fitting.

The first 3 operations represent the image as a digital image data structure
(pixel information), however, from the grouping operation the data structure
needs also to record the spatial events to which each pixel belongs.

This information is stored in a logical data structure.

Extracting

Image Processing and Computer

Vision
 Matching

Image Processing and Computer

Vision
 Morphological Image Processing: Introduction

Morphology is a broad set of image processing operations that process images

based on shapes.

In a morphological operation, each pixel in the image is adjusted based on

the value of other pixels in its neighborhood.

By choosing the size and shape of the neighborhood, you can construct a
morphological operation that is sensitive to specific shapes in the input image.

Morphological image processing (MIP) is a group of non-linear operations that

correlate to the shape or morphology of features in an image. The
morphological operations depend only on the comparative arrangement of pixel
values, not their numerical values. They are particularly suitable for processing
binary images.

Grayscale images also benefit from morphological operations because their

optical transfer functions are unknown, and their absolute pixel values are of
little or no interest.

Types of Morphological operations:

Dilation: Dilation adds pixels on the object boundaries.

Erosion: Erosion removes pixels on object boundaries.

Open: The opening operation erodes an image and then dilates the
eroded image, using the same structuring element for both operations.

Close: The closing operation dilates an image and then erodes the
dilated image, using the same structuring element for both operations.

Image Processing and Computer

Vision
 Dilation

Dilation expands the image pixels, or it adds pixels on object boundaries. First,
we traverse the structuring element over the image object to perform an
dilation operation, as shown in Figure 7. The output pixel values are calculated
using the following equation.

The value of the output pixel is the maximum value of all pixels in the
neighborhood. In a binary image, a pixel is set to 1 if any of the neighboring
pixels have the value 1.

Morphological dilation makes objects more visible and fills in small holes in
objects. Lines appear thicker, and filled shapes appear larger.

Pixel (output) = 1 {if HIT}

Pixel (output) = 0

{otherwise}
Image Processing and Computer
Vision
 Properties:

1. It can repair breaks (Figure

9).

2. It can repair intrusions

(Figure 9).

Erosion

Erosion shrinks the image pixels, or erosion removes pixels on object

boundaries. First, we traverse the structuring element over the image object to
perform an erosion operation, as shown in Figure 4. The output pixel values are
calculated using the following equation.

The value of the output pixel is the minimum value of all pixels in the
neighborhood. In a binary image, a pixel is set to 0 if any of the neighboring
pixels have the value 0.

Image Processing and Computer

Vision
 Morphological erosion removes floating pixels and thin lines so that only
substantive objects remain. Remaining lines appear thinner and shapes
appear smaller.

Pixel (output) = 1 {if FIT}

Pixel (output) = 0

{otherwise}

Properties:

1. It can split apart joint objects

(Figure 6).

2. It can strip away extrusions (Figure

6).

Closing & Opening

💡 Closing: Closing is a morphological operation that is used to remove

small gaps or holes in an object within an image. The closing operation
combines dilation and erosion to remove small gaps and smooth out the
edges of an object.

Image Processing and Computer

Vision
 Compound Operations
Most morphological operations are not performed using either dilation or erosion;
instead, they are performed by using both. Two most widely used compound
operations are:

1. Closing (by first performing dilation and then erosion), and

2. Opening (by first performing erosion and then dilation).

Application: Edge Extraction of an Object

Extracting the boundary is an important process to gain information and understand

the feature of an image. It is the first process in preprocessing to present the image’s
characteristics. This process can help the researcher to acquire data from the image.
We can perform boundary extraction of an object by following the below steps.

Step 1. Create an image (E) by erosion process; this will shrink the image slightly.
The kernel size of the structuring element can be varied accordingly.

Step 2. Subtract image E from the original image. By performing this step, we get
the boundary of our object.

Image Processing and Computer

Vision
 Hit-or-Miss transformation

The hit-and-miss transform is a general binary morphological operation that can

be used to look for particular patterns of foreground and background pixels in
an image.

It is actually the basic operation of binary morphology since almost all the
other binary morphological operators can be derived from it.

As with other binary morphological operators it takes as input a binary

image and a structuring element, and produces another binary image as
output.

Image Processing and Computer

Vision
 Morphological algorithm operations on binary images

The following figure illustrates the dilation of a binary image. The structuring
element defines the neighborhood of the pixel of interest, which is circled. The
dilation function applies the appropriate rule to the pixels in the neighborhood and
assigns a value to the corresponding pixel in the output image. In the figure, the
morphological dilation function sets the value of the output pixel to 1 because one
of the elements in the neighborhood defined by the structuring element is on.

Image Processing and Computer

Vision
 Morphological algorithm operations on gray-scale images

The following figure illustrates this processing for a grayscale image. The dilation
function applies the rule to the neighborhood of the circled pixel of interest. The
value of the corresponding pixel in the output image is assigned as the highest
value among all neighborhood pixels. In the figure, the value of the output pixel is
16 because it is the highest value in the neighborhood defined by the structuring
element.

Discuss morphological algorithm operations performed on gray scale images.

Morphological algorithms operate on grayscale images using operations like dilation,

erosion, opening, and closing. These operations are commonly used in image
processing applications to filter noise, extract features, segment image regions, and
enhance image quality.

1. Dilation: Dilation is an operation that adds pixels to the boundaries of objects in

an image. It is performed by scanning the image with a structuring element
that defines the shape and size of the dilation operation. The structuring
element identifies the pixels to be added to the boundary of an
object. A grayscale dilation operation increases the brightness of an image by
adding brighter pixels to the object boundary.

2. Erosion: Erosion is an operation that removes pixels from the boundaries of

objects in an image. It is performed by scanning the image with a structuring
element that defines the shape and size of the erosion operation. The
structuring element identifies the pixels to be removed from the
boundary of an object. A grayscale erosion operation decreases the
brightness of an image by removing brighter pixels from the object boundary.

3. Opening: Opening is an operation that combines an erosion operation followed

by a dilation operation. It is used to remove small objects and thin structures in
an image while preserving the integrity of larger objects. Opening is
performed by first eroding the image to remove small objects, followed by
dilation to fill in the gaps between objects.

4. Closing:
Image Processing Closing is an operation that combines a dilation operation followed by
and Computer
Vision an erosion operation. It is used to fill in gaps and holes in objects while
preserving the object's overall
 shape. Closing is performed by first dilating the image to fill in gaps, followed by
an erosion operation to remove any excess pixels added by dilation.

In summary, morphological algorithms operate on grayscale images using

dilation, erosion, opening, and closing operations to filter noise, extract features,
segment image regions, and enhance image quality. These operations are useful
in a variety of image processing applications, including medical imaging, remote
sensing, and industrial inspection.

Thinning

Thinning is a morphological operation that is used to remove selected foreground

pixels from binary images, somewhat like erosion or opening. It can be used for
several applications, but is particularly useful for skeletonization. In this mode it is
commonly used to tidy up the output of edge detectors by reducing all lines to
single pixel thickness. Thinning is normally only applied to binary images, and
produces another binary image as output.

Thinning is a morphological operation in image processing that reduces the

thickness of the foreground objects in a binary image to a single-pixel width while
preserving their topology. The operation is used to simplify the shape of objects and
to extract features such as lines, curves, and endpoints.

Thinning is performed by iteratively scanning the binary image and removing pixels
that meet certain conditions. The process continues until no more pixels can be
deleted without changing the topology of the objects. The conditions for pixel
deletion depend on the structuring element used for the operation. The structuring
element is a small matrix of values that defines the neighborhood around each
pixel. The most commonly used structuring elements for thinning are the 3x3 and
5x5 square structuring elements.

The thinning process involves the following steps:

1. Create a binary image where the foreground objects are represented by white
pixels and the background is represented by black pixels.

2. Choose a structuring element for the thinning operation.

3. Convolve the structuring element with the binary image such that the center
pixel of the structuring element is aligned with each pixel in the image.

4. Check if the center pixel of the structuring element meets the conditions for
deletion. If so, mark the pixel for deletion.

5. Repeat steps 3 and 4 for each pixel in the image.

6. Delete all the marked pixels.

7. Repeat steps 3 to 6 until no more pixels can be deleted without changing the
topology of the objects.

After the thinning operation, the foreground objects in the binary image will have a
single-pixel width. The thinning operation can be applied iteratively to achieve
different levels of thinning. In addition, thinning can be used in conjunction with
other morphological operations, such as erosion and dilation, to achieve more
complex image processing tasks such as feature extraction, object recognition, and
Image Processing and Computer
image segmentation.
Vision
 Thickening

Thickening is a morphological operation that is used to grow selected regions of

foreground pixels in binary images, somewhat like dilation or closing. It has several
applications, including determining the approximate convex hull of a shape, and
determining the skeleton by zone of influence. Thickening is normally only applied to
binary images, and it produces another binary image as output.

Thickening is a morphological dual of a thinning and is defined as

where B is a structuring element suitable for thickening. As in thinning,

thickening can be defined as a sequentional operation:

The structuring element used for thickening has the same form as one used for
thinning but with all ones and zeros interchanged.

However, the usual procedure is to thin the background of the set to be processed
and then complement the result. Therefore, to thicken a set A, we form its
complement, thin it, and then complement the result.

Image Processing and Computer

Vision
 Depending on the nature of A, the thickening procedure may result in
disconnected points. Therefore, this method is usually followed by post-processing
to remove disconnected points.

Region growing

Region growing is a technique used in image processing for segmentation, or

dividing an image into distinct regions or objects. The idea behind region
growing is to start with a seed pixel or a small region of pixels, and then
iteratively add neighboring pixels that have similar properties or characteristics,
such as intensity or color.

The region growing algorithm starts by selecting a seed pixel or region, which is
typically chosen manually or through an automated process. The selected seed
pixel or region is considered as the starting point of the segmentation process.
The algorithm then examines the neighboring pixels of the seed pixel or region
and decides whether to add them to the segmented region based on certain
criteria.

The most common criteria for adding pixels to the segmented region are
intensity or color similarity. If the intensity or color of a neighboring pixel is
within a certain threshold of the intensity or color of the pixels already in the
segmented region, the pixel is added to the region. This process is repeated for
all neighboring pixels, and the segmented region grows iteratively.

The region growing algorithm stops when there are no more neighboring
pixels that meet the criteria for adding to the segmented region. The resulting
region is a connected component of the image that shares similar properties
or characteristics.

Region growing can be applied to both grayscale and color images. The output
of the algorithm is a binary image, where the pixels in the segmented region are
assigned a value of 1, and the rest of the pixels are assigned a value of 0.
Region growing can be used in a wide range of image processing tasks,
including object detection, object tracking, and medical imaging.

An initial set of small areas are iteratively merged according to similarity

constraints.
Image Processing and Computer
Vision
 Start by choosing an arbitrary seed pixel and compare it with neighbouring
pixels

Region is grown from the seed pixel by adding in neighbouring pixels that
are similar, increasing the size of the region.

When the growth of one region stops we simply choose another seed pixel
which does not yet belong to any region and start again.

This whole process is continued until all pixels belong to

some region. A bottom up method.

Region growing methods often give very good segmentations that correspond
well to the observed edges.

Region shrinking / Region splitting & merging

The basic idea of region splitting is to break the image into a set of disjoint
regions which are coherent within themselves:

Initially take the image as a whole to be the area of interest.

Look at the area of interest and decide if all pixels contained in the region
satisfy some similarity constraint.

Image Processing and Computer

Vision
 If TRUE then the area of interest corresponds to a region in the image.

If FALSE split the area of interest (usually into four equal sub-areas) and
consider each of the sub-areas as the area of interest in turn.

This process continues until no further splitting occurs. In the worst case this
happens when the areas are just one pixel in size.

This is a divide and conquer or top down method.

If only a splitting schedule is used then the final segmentation would probably
contain many neighbouring regions that have identical or similar properties.

Thus, a merging process is used after each split which compares adjacent regions
and merges them if necessary. Algorithms of this nature are called split and merge
algorithms.

To illustrate the basic principle of these methods let us consider an imaginary

image.

Image segmentation

Image segmentation is a branch of digital image processing which focuses on

partitioning an image into different parts according to their features and
properties.

The primary goal of image segmentation is to simplify the image for easier
analysis.

Image Processing and Computer

Vision
 In image segmentation, you divide an image into various parts that have
similar attributes. The parts in which you divide the image are called Image
Objects.

Different Types of Image Segmentation Techniques

1. Thresholding Segmentation

a. Simple Thresholding

b. Otsu’s Binarization

c. Adaptive Thresholding

2. Edge-Based Segmentation

a. Search-Based Edge Detection

3. Region-Based Segmentation

a. Region Growing

b. Region Splitting and Merging

4. Watershed Segmentation

5. Clustering-Based Segmentation Algorithms

a. K-means Clustering

6. Neural Networks for Segmentation

Define digital image. What do you mean by image sampling and quantization?

A digital image is a representation of a picture that is stored on a computer in

digital form. It is made up of a grid of tiny squares called pixels, each of which
has a color or grayscale value assigned to it. Digital images can be photographs,
drawings, or any other type of visual content.

To create an image which is digital, we need to covert continuous data into

digital form. There are two steps in which it is done.

7. Sampling

8. Quantization

Image sampling is the process of dividing the continuous tone of an image into
a limited set of discrete values that can be stored digitally. This involves taking
a continuous tone image and converting it into a pixelated image made up of
discrete values. The process of image sampling results in a loss of quality of
the original image.

Image Processing and Computer

Vision
 Sampling is the process of converting an analog signal into discrete values.

Image quantization

After sampling the analog signal, we will apply quantization. Quantization

digitizes the amplitude of the sampled signal. Quantization is done by
rounding off the amplitude of each sample and then assigning a different
value according to its amplitude. Each value will represent a different color
tone.

Difference between Sampling vs Quantization

Sampling establishes the number of pixels in a digital image, whereas

quantization establishes the color of each pixel.

Sampling digitalizes an analog signal's x-axis, whereas quantization

digitalizes its y- axis.

Sampling is carried out before quantization, whereas quantization is

carried out following sampling
UNIT 2 - Image Representation and Description
Representation schemes

Region representation:

based on external characteristics (its boundary)

based on internal characteristics (pixels comprising the region)

Region description:

boundary descriptors, such as boundary length, diameter,

curvature, etc. regional descriptors, such as area, perimeter,

compactness, mean value, etc.

💡 Generally, an external representation is chosen when a primary focus is

on shape characteristics. An internal representation is selected when a
primary
Chain codes focus is on reflectivity properties, such as color or texture.

1. represent a boundary by a connected sequence of straight-line segments

of specified length and direction

2. the direction of each segment is coded by using a numbering scheme

Image Processing and Computer

Vision
 Polygonal approximations

1. a digital boundary can be approximated by a polygon

2. minimum perimeter polygons

a. enclose a boundary by a set of concatenated cells and produce a

minimum perimeter that fits the cell strip

Image Processing and Computer

Vision
 The skeleton of a region

1. medial axis transformation (MAT) generates a "skeleton" of a region

2. MAT algorithm:

a. for each point in the region we find its closest point in boundary,

b. if a point has more than one such a neighbor — > a point belongs to the
medial axis (skeleton) of the region

Boundary descriptors

Image Processing and Computer

Vision
 Boundary descriptors in image processing are mathematical representations of the
boundaries or contours of an object in an image. These descriptors are used to
extract important features from the boundary of an object, which can be used for a
variety of image analysis tasks, such as object recognition and classification, shape
analysis, and object tracking.

Boundary descriptors can be either global or local. Global descriptors represent

the entire boundary of an object with a single value. Examples of global
descriptors include perimeter, area, and compactness. Local descriptors, on the
other hand, represent different segments of the boundary using multiple values.
Examples of local descriptors include curvature, tangent angle, and length of the
segment.

Boundary descriptors are essential tools in image processing and can be used for a
wide range of applications such as object recognition, tracking, segmentation, and
analysis.

Region descriptors

Image Processing and Computer

Vision
 Region descriptors in image processing refer to features that describe the
characteristics of an image region. These regions are typically defined by
segmenting an image into separate regions based on color, texture, or other visual
attributes.

Some common region descriptors include:

1. Area: the size of the region in pixels.

2. Perimeter: the length of the boundary of the region.

3. Compactness: a measure of how closely the region is packed.

4. Convexity: how closely the region follows a convex shape.

5. Eccentricity: the ratio of the major and minor axes of an ellipse that best fits
the region.

6. Orientation: the angle of the major axis of the best-fit ellipse.

7. Moments: statistical measurements of the region's shape and size.

8. Texture: statistical measurements of the spatial arrangement of image

pixels within the region.

Region descriptors are used in a wide range of applications, including object

recognition, image segmentation, and feature matching. They are particularly useful
for identifying and characterizing regions of interest within an image, and for
distinguishing between different types of objects based on their visual features.

Binary Machine Vision

Problem definition

Objective: Design a vision system to “see” a “flat”

world Page of text: text, line drawings, etc.

Side panel of a truck

Objects on an inspection belt

X-ray image of separated potatoes

Microscopic image of blood

cells

General approach to

recognition/inspection Acquire gray

scale image using camera

Reduce to black and white image -
black objects on white background

Find individual black objects and measure their

properties Compare those properties to object

models

Image segmentation by Thresholding

Image Processing and Computer
Vision Image thresholding is a simple, yet effective, way of partitioning an image into a
foreground and background.
 This image analysis technique is a type of image segmentation that isolates
objects by converting grayscale images into binary images. Image
thresholding is most effective in images with high levels of contrast.

Common image thresholding algorithms include histogram and multi-level

thresholding.

If the objects and background occupy different ranges of gray levels, we can
“mark” the object pixels by a process called thresholding:

Let F(i,j) be the original, gray level image

B(i,j) is a binary image (pixels are either 0 or 1) created by

thresholding F(i,j) B(i,j) = 1 if F(i,j) < t [t=threshold]

B(i,j) = 0 if F(i,j) >= t

We will assume that the 1’s are the object pixels and the 0’s are
the background pixels

Explain global and adaptive thresholding techniques.

Global thresholding is a technique used in image processing to convert a

grayscale image into a binary image. It involves selecting a threshold value
that divides the grayscale image into two classes: foreground and background.
All pixel values below the threshold value are assigned to the background class,
and all pixel values above the threshold value are assigned to the foreground
class. The threshold value is usually fixed for the entire image.

Adaptive thresholding, on the other hand, is a technique that varies the

threshold value for different regions of the image. In this method, the threshold
value is not fixed but is calculated based on the local image properties. The
idea behind the adaptive thresholding is to handle variations in lighting
conditions and image noise.

Image Processing and Computer

Vision
 There are different types of adaptive thresholding techniques such as Mean
Thresholding, Gaussian Thresholding, and Sauvola Thresholding. For instance,
Mean thresholding calculates the threshold value based on the mean of the pixel
values in a local region of the image. Gaussain thresholding calculates the
threshold value based on a weighted average of the pixel values in the
neighborhood. And, Sauvola thresholding takes into account both the local mean
and standard deviation of the image in defining the threshold value.

In summary, the global thresholding technique assigns a single threshold value

to the entire image, while adaptive thresholding techniques vary the threshold
value depending on the local characteristics of the image.

Image Segmentation

Image segmentation is a commonly used technique in digital image

processing and analysis to partition an image into multiple parts or
regions, often based on the characteristics of the pixels in the image.

Image segmentation is the process of grouping perceptually similar pixels

into regions.

Image segmentation could involve separating foreground from background,

or clustering regions of pixels based on similarities in color or shape.

For example, a common application of image segmentation in medical imaging

is to detect and label pixels in an image or voxels of a 3D volume that
represent a tumor in a patient’s brain or other organs.

Ideally, object pixels would be black (0 intensity) and background pixels white
(maximum intensity)

But this rarely happens

pixels overlap regions from both the object and the background, yielding
intensities between pure black and white - edge blur

cameras introduce “noise” during imaging - measurement “noise”

Image Processing and Computer

Vision
 Connected component labeling

This is a computer vision algorithm to detect and count the number of

connected regions — also called blobs — in a binary image. Often, this is done
after a segmentation algorithm.
The resulting segmentation (binary) mask is then run through the connected
component labelling algorithm to count the number of distinct regions.

Goal

The input of the algorithm is a binary image. The goal is to label each
connected component (or blob) with the same unique label. Because each
blob will labelled, we can infer the total number of individual blobs. It is
important to note that the output will differ with respect to the
representation/connectivity you use.

Different connectivities

There are two common ways of defining whether or not a component is

connected. One is stating that a pixel only has 4 neighbours (sometimes
called 4-connectivity). The other is stating that a pixel has 8 neighbours. See
Figure 1.

Hierarchal segmentation

A hierarchical image segmentation is a series of image segmentations at different

detail levels
such that the segmentation at a given level can be produced by merging regions
of the segmentation at the previous level. As a result, in such hierarchical
representation, any region

Image Processing and Computer

Vision
 that belongs to a finer detail level is included (or nested) within a region that
belongs to a
coarser level.
Image segmentation is the process of grouping perceptually similar pixels into
regions.

A hierarchical image segmentation is a set of image segmentations at

different detail levels in which the segmentations at coarser detail levels can
be produced from simple merges of regions from segmentations at finer
detail levels.

Therefore, the segmentations at finer levels are nested with respect to those
at coarser levels.

Hierarchical methods have the interesting property of preserving spatial and

neighboring information among segmented regions.

Spatial clustering

Spatial clustering in image processing refers to grouping or clustering similar

pixels together based on their spatial proximity or neighborhood in an image.
This technique is used for segmentation, object recognition, and image
compression.

The spatial clustering algorithm starts by selecting a pixel, which becomes the
center of a cluster. Then, neighboring pixels are added to the cluster based on
their distance from the center pixel and a predefined similarity measure. The
process continues until a stopping criterion is met, which could be a maximum
cluster size or a certain similarity threshold.

The output of spatial clustering is a labeled image where each cluster is

assigned a unique label. The labels can then be used for further analysis, such
as object identification or texture analysis.

Examples of spatial clustering algorithms include k-means clustering, mean shift

clustering, hierarchical clustering, and fuzzy clustering. Each algorithm has its
own strengths and weaknesses and is suitable for different types of images and
applications.

Image Processing and Computer

Vision
 Split & merge

Split and merge segmentation is an image processing technique used to

segment an image. The image is successively split into quadrants based on a
homogeneity criterion and similar regions are merged to create the segmented
result.

The technique incorporates a quadtree data structure, meaning that there is a

parent-child node relationship. The total region is a parent, and each of the four
splits is a child

The basic idea of region splitting is to break the image into a set of disjoint
regions which are coherent within themselves:

Initially take the image as a whole to be the area of interest.

Look at the area of interest and decide if all pixels contained in the region
satisfy some similarity constraint.

If TRUE then the area of interest corresponds to a region in the image.

If FALSE split the area of interest (usually into four equal sub-areas) and
consider each of the sub-areas as the area of interest in turn.

This process continues until no further splitting occurs. In the worst case this
happens when the areas are just one pixel in size.

This is a divide and conquer or top down method.

If only a splitting schedule is used then the final segmentation would probably
contain many neighbouring regions that have identical or similar properties.

Thus, a merging process is used after each split which compares adjacent regions
and merges them if necessary. Algorithms of this nature are called split and merge
algorithms.

To illustrate the basic principle of these methods let us consider an imaginary

image.

Image Processing and Computer

Vision
 Rule-based Segmentation

Rule-based image segmentation is a method of dividing an image into different

parts or segments based on predefined rules or criteria. The segmentation
process involves defining specific features such as edges, colors, or textures
that distinguish different objects or regions in the image.

For example, in a medical image of a lung, a rule-based algorithm can be

designed to segment the image based on specific features such as the
brightness levels in specific regions, the distance between adjacent regions,
and the shape of the different structures present in the image.

While rule-based image segmentation provides accurate results when the

rules are well- defined, it has limitations in dealing with complex images and
object interactions.

In addition, rule-based methods require a significant amount of manual

intervention and are less flexible than other segmentation methods, such as
machine learning-based approaches.

Motion-based segmentation.

Motion Segmentation is the task of identifying the independently moving

objects (pixels) in the video and separating them from the background motion.

Image Processing and Computer

Vision
 If the background consists of a plane, then we can register the various
frames onto a common frame perfectly, using projective transformations.

The areas of the image that do not register well; belong to the moving
objects.

If the registration of all frames is perfect, we can take the image difference of
two registered images. The pixels that have high intensity difference can be
classified as moving objects.

However, this simple scheme has a lot of false alarms since registration is
not always perfect.

In the presence of slight misregistration, the pixels near prominent edges

usually flag up as moving pixels. In order to reduce the false alarms, we resort
to a detection scheme that combines evidences from

1. image differences, and

2. optical flow discrepancy.

Example 5.9 shows a frame marking the moving object pixels in the video
sequence detected in that particular frame.

Area Extraction / Feature extraction

Area Extraction

Area extraction is a process of identifying and separating different regions or

objects present in an image. It is an important step in image processing and
computer vision applications, as it enables us to isolate and analyze different parts
of an image separately.

There are various techniques used for area extraction, depending on the
characteristics of the image and the specific application requirements. Some of the
commonly used techniques are:

1. Thresholding: It is a simple technique where we define a threshold value and

classify each pixel in the image as foreground or background based on whether
its intensity is above or below the threshold value.

Image Processing and Computer

Vision
 2. Edge detection: This technique is used to identify edges or boundaries
between different objects in an image. It is done by detecting
sudden changes in intensity or color in the image.

3. Region growing: In this technique, we start with a seed point and grow
the region by including neighboring pixels that have similar intensity or
color.

4. Clustering: It involves grouping pixels into clusters based on their similarity in

color, texture or other features.

5. Template matching: It is used to match a predefined template or object with

the image and extract the corresponding area.

Area extraction has many applications such as object recognition, segmentation,

image analysis, and medical imaging. It plays a crucial role in identifying and
understanding the different components of an image, which helps in making
accurate decisions and predictions.

Concepts

Area extraction refers to the process of identifying and extracting specific

information from a given area or region.

How do Machines Store Images?

Let’s understand this with a simple example. Look at the image below:

We have an image of the number 8. Look really closely at the image – you’ll
notice that it is made up of small square boxes. These are called pixels.

Machine store images in the form of numbers. Have a look at the image
below:

These numbers, or the pixel values, denote the intensity or brightness of the
pixel. Smaller numbers (closer to zero) represent black, and larger
numbers (closer to 255) denote white.

Image Processing and Computer

Vision
 In the case of a colored image, there are three Matrices (or channels) – Red,
Green, and Blue. Each matrix has values between 0-255 representing the
intensity of the color for that pixel. Consider the below image to understand
this concept:

This can include extracting land cover information, identifying land-use patterns,
tracking changes in vegetation cover or urban development, and mapping
natural resources such as forests, water bodies, or wetlands.

Area extraction techniques are commonly used in remote sensing and GIS
applications, where satellite data or aerial imagery is analyzed to identify and
extract spatial data about a particular region.

This information is then used for environmental management, urban

planning, disaster response, and other applications.

Data-structures

Traditional image data structures

1. Matrices

a. Most common data structure for low level image representation Elements of
the matrix are integer numbers

b. Image data of this kind are usually the direct output of the image capturing
device, e.g., a scanner.

2. Chains

a. Chains are used for description of object borders

b. Symbols in a chain usually correspond to the neighborhood of primitives in

the image.

Image Processing and Computer

Vision
 c. Chains can be represented using static data structures (e.g., 1D arrays);
their size is the longest length of the chain expected.

Hierarchical data structures

1. Pyramids

a. M-pyramid - Matrix pyramid ... is a sequence {ML, ML-1, ..., M0} of images

b. M-pyramids are used when it is necessary to work with an image at different

resolutions simultaneously.

Often it is advantageous to use several resolutions simultaneously rather

than to choose just one image from the M-pyramid.

Such images can be represented using tree pyramids ... T-

pyramids. T-pyramid is a tree, every node of the T-pyramid

has 4 child nodes.

2. Quadtrees

a. Quadtrees are modifications of T-pyramids.

b. Every node of the tree except the leaves has four children (NW: north-
western, NE: north-eastern, SW: south-western, SE: south-eastern).

c. Similarly to T-pyramids, the image is divided into four quadrants at each

hierarchical level, however it is not necessary to keep nodes at all
levels.

d. If a parent node has four children of the same value (e.g.,

brightness), it is not necessary to record them.

Image Processing and Computer

Vision
 Edge detection

As we know, the computer sees the images in the form of matrices.

Edge detection is an image processing technique for finding the boundaries of

an object in the given image.

Now the question is do we have to sit and manually compare these values
to find the edges. Well, obviously not. For the task, we can use a matrix
known as the kernel and perform the element-wise multiplication.

Let’s say, in the selected portion of the image, I multiply all the numbers on left
with -1, all the numbers on right with 1. Also all the numbers in the middle row
with 0. In simple terms, I am trying to find the difference between the left and
right pixels. When this difference is higher than a threshold, we can conclude
it’s an edge.

Image Processing and Computer

Vision
 In the above case, the number is 31 which is not a large number. Hence this
pixel doesn’t lie on edge.

To summarize, Pixels on the edge have a significant difference in values. We can

compare neighboring pixel values to find the edge. Also, a matrix or a kernel is
used to compare the values. The higher the difference between the right and left
pixels, the target is closer to the edge. Similarly, Lower the difference- pixel is
not at the edge.

Line-Linking

Line-linking in image processing refers to the process of connecting or

merging separate line segments to form longer lines.

This is typically done to make the overall image more visually appealing, reduce
noise, and make it easier to interpret.

The process of line-linking involves analyzing the image for individual line
segments and then determining which segments should be merged to create
longer lines that span the length or width of the image.

This can be done through various methods, such as edge detection, Hough
transforms, and computer vision algorithms.

Once the line segments have been identified, they are analyzed based on their
proximity to one another, their orientation, and other factors, to determine if
they should be merged into a longer line. The resulting connected lines can
create a more cohesive image and make it easier to identify patterns or other
features within the image.

Line-linking is often used in image processing applications such as object

recognition, image segmentation, and feature extraction. It is particularly
useful when dealing with images that contain a large number of separate
or overlapping lines, such as medical images, satellite imagery, or maps.

Hough transform

The Hough transform (HT) is a feature extraction approach in image analysis,

computer vision, and digital image processing

The Hough Transform is an algorithm patented by Paul V. C. Hough

Image Processing and Computer

Vision
 originally invented to recognize complex lines in photographs

Since its inception, the algorithm has been modified and enhanced to be able
to recognize other shapes such as circles and quadrilaterals of specific types

It uses a voting mechanism to identify bad examples of objects inside a

given class of forms. This voting mechanism is carried out in parameter
space. Object candidates are produced as local maxima in an accumulator
space using the HT algorithm.

How Does it Work?

The Hough approach is effective for computing a global description of a

feature(s) from (potentially noisy) where the number of solution classes does
not need to be provided before.

For example, the Hough approach for line identification is motivated by the
assumption that each input measurement reflects its contribution to a
globally consistent solution (e.g., the physical line which gave rise to that
image point).

A line can be described analytically in a variety of ways. One of the line

equations uses the parametric or normal notion: xcosθ+ysinθ=r. where r is
the length of a normal from the origin to this line and θ is the orientation as
given in Figure 5.

The known variables (i.e., xi,yi) in the Image are constants in the parametric
line equation, whereas r and are the unknown variables we seek. Points in
cartesian image space correspond to curves (i.e., sinusoids) in the polar
Hough parameter space if we plot the potential (r, θ) values specified by
each. The Hough transformation for straight

Image Processing and Computer

Vision
 lines is this point-to-curve transformation. Collinear spots in the cartesian
image space become clearly obvious when examined in the Hough
parameter space because they provide curves that overlap at a single (r, θ)
point.

Where a and b are the circle’s center coordinates, and r is the radius.
Because we now have three coordinates in the parameter space and a 3-D
accumulator, the algorithm’s computing complexity increases. (In general,
the number of parameters increases the calculation and the size of the
accumulator array polynomially.) As a result, the fundamental Hough
approach described here is only applicable to straight lines.

Advantages

The HT benefits from not requiring all pixels on a single line to be

contiguous. As a result, it can be quite effective when identifying lines
with small gaps due to noise or when objects are partially occluded.

Disadvantages

The HT has the following drawbacks:

It can produce deceptive results when objects align by accident;

Rather than finite lines with definite ends, detected lines are infinite lines
defined by their (m,c) values.

The Hough transform is widely used in computer vision and image processing
applications such as edge detection, object detection, and image segmentation. It is
a robust technique that can accurately detect patterns even in noisy or low-
resolution images. However, it is computationally intensive and may require
optimization techniques to improve its performance.

Line fitting

Line fitting in image processing is the process of fitting a line to a set of data
points in an image.

This can be used to extract information about the orientation and position of
objects in the image.

Line fitting can be performed using various algorithms, such as least-squares

regression, Hough transform, or RANSAC.

The output of line fitting can be used for various applications, including
edge detection, object recognition, and image segmentation.

Curve fitting (Least-square fitting).

Curve fitting, also known as least-square fitting, is a statistical technique used

to find a mathematical model that best fits a set of data points. The goal of
curve fitting is to find the equation of a curve that passes through the data
points as closely as possible.

There are many methods to perform curve fitting, but the most commonly
used method is the least-square method. In this method, the sum of the
squared differences between the observed data points and the predicted
Image Processing andof
values Computer
the model is minimized.
Vision
 The curve fitting process involves selecting a type of curve that is expected to
fit the data, such as a linear, quadratic, exponential, or logarithmic curve, and
adjusting the curve's parameters to find the best fit. This is done using
mathematical optimization techniques, such as gradient descent or the Gauss-
Newton algorithm.

Once the curve fitting is complete, the resulting model can be used to make
predictions and extrapolations beyond the data set. Curve fitting is widely used
in many fields, including finance, engineering, biology, and physics, to analyze
and model real-world data.

UNIT 3
Region Analysis
Region is a segment produced by connected component labeling or signature
segmentation

Region Analysis

Region analysis in image processing refers to the process of identifying and

classifying different regions in an image based on their characteristics. It is a
critical step for many applications, such as object recognition, segmentation,
and tracking.

Region analysis involves extracting meaningful features from an image, such as

color, texture, and shape, and then clustering pixels or groups of pixels based
on those features. The resulting clusters can then be analyzed and classified
based on predefined criteria, such as size, shape, and texture.

There are many techniques and algorithms used for region analysis, including
thresholding, watershed segmentation, and clustering algorithms such as K-
means and Fuzzy C-Means. The choice of algorithm typically depends on the
specific application and the characteristics of the image being processed.

Overall, region analysis is an important tool in image processing and helps to

extract meaningful information from images, leading to a better
understanding of the underlying patterns and structures.

Region properties

Many properties can be extracted from an image region

Image Processing and Computer

Vision
 area

length of

perimeter

orientation etc.

These properties can be used for many

tasks object recognition

"dimensioning" (measuring sizes of physical

objects) to assist in higher-level processing

Extremal points

In region analysis, extremal points are points that lie on the boundary of a
region and have the property that any straight line passing through them either
enters or exits the region.

Image Processing and Computer

Vision
 These points are also known as boundary points or corner points.

They can be as many as eight distinct extermal points.

For example, in a square region, the four corners are the extremal points.
In a circular region, any point on the circumference is an extremal point. In
a polygonal region, the vertices of the polygon are the extremal points.

Extremal points are important in region analysis because they help to

determine the shape and size of the region. They are also used in optimization
problems to find the maximum or minimum value of a function over a given
region, since the maximum or minimum is often found at an extremal point.

Image Processing and Computer

Vision
 Spatial moments

Moments in image processing are average values from the single pixels`
intensities of an image. With this moments physical properties like orientation,
eccentricity, the area or the centroid of an object in the image can be
identified.

Mathematically, the image moment Mij of order (i,j) for a greyscale

image with pixel intensities I(x,y) is calculated as

Area:

For a binary image, the zeroth order moment corresponds to the area.

Centroid:

Centroid simply is the arithmetic mean position of all the points. In

terms of image moments, centroid is given by the relation

Let’s take a simple example to understand how to calculate image moments

for a given image.

Below are the area and centroid calculation for the

above image

Image Processing and Computer

Vision
 In summary, spatial moments are an important tool in region analysis, and are
used to compute various properties of a region, such as its area, centroid,
orientation, and moments of inertia. These moments are used to characterize
the shape and structure of the region, and to compare it with other regions or
templates.

Mixed spatial gray-level moments

Mixed spatial gray-level moments are a type of feature extraction method used in
region analysis. They provide information about the spatial relationship between
the gray-level values in a region.

Mixed spatial gray-level moments are calculated using a combination of two

different parameters: the order of the moment and the distance between the
pixels. The order of the moment determines the degree of accuracy of the
representation of the region, while the distance parameter sets the scope of the
analysis.

These moments are calculated by first defining a reference point in the region, and
then measuring the relationship of the gray-level values from each pixel to that
point. The reference point is typically defined as the centroid of the region. This
allows for a more accurate representation of the spatial distribution of the gray-
level values in the region.

Mixed spatial gray-level moments can be used to extract important features of a

region, such as its shape and texture. They can also be used to classify regions
based on these features, allowing for automated analysis of large datasets.

Boundary analysis
Boundary analysis

Boundary analysis in image processing refers to the task of identifying and

analyzing the edges or contours in an image. Edges are sudden changes in pixel
intensity
Image Processing values and are
and Computer
Vision
 generally a result of changes in the underlying image characteristics, such as
changes in color, texture, or depth.

Boundary analysis is an essential step in many image processing

applications, including object recognition, segmentation, and tracking. It
involves techniques such as edge detection, contour following, and geometric
analysis to isolate and extract useful information about an image's
boundaries.

Edge detection is one of the most commonly used techniques in boundary

analysis

Overall, boundary analysis plays an important role in image processing and

helps to extract valuable information from images for various applications.

Inner boundary = original image - eroded image

Outer boundary = dilated image - original image.

Signature properties

1. Identifies boundary values: Boundary analysis focuses on identifying the values

that lie on the edge of a range of input values. These boundary values are
critical as they can have a significant impact on the behavior of the system.

2. Covers both extremes of the input range: Boundary analysis covers both
extremes of the input range. This is because extreme values are
more likely to cause errors and faults in the software system.

3. Reveals input errors: Boundary analysis helps to reveal input errors that
might not be detected by other testing techniques. For example, if the
system is designed to accept values between 1 and 100, the
boundary values would be 1 and 100. If a user outside of this
range enters a value of 0 or 101, this would be an input error that boundary
analysis can reveal.

4. Enhances test coverage: By identifying boundary values, boundary analysis

enhances test coverage. This means that testers can focus on
testing the areas of the system that are most likely to cause errors.

5. Improves software quality: With the help of boundary analysis, testers can
identify and fix potential errors and defects at an early stage of the
software development cycle. This improves the overall quality of the
software and reduces the risk of failures and errors in the final
product.

Shape numbers

Shape numbers in boundary analysis are the numbers that represent the shape
of the boundary of the input parameter. The shape numbers signify the
boundary values and the direction of the boundary values. The shape numbers
are essential in boundary analysis as they help in identifying the errors that
occur at the boundary values of input parameters.
The shape numbers are usually denoted as S1, S2, S3, S4, and so on,
depending on the number of boundary conditions. The shape numbers are
used
Image Processing andto Computer
represent different types of boundary conditions, such as endpoints,
Vision interior points, and values outside the range.
 Shape numbers refer to numerical values assigned to different shapes, such as
circles, squares, triangles, etc., in image processing. These numeric values can
be used to quantify and compare the characteristics of different shapes in an
image, such as area, perimeter, and aspect ratio.

Shape numbers are typically calculated by applying mathematical algorithms

to the pixel values of an image, using techniques such as edge detection,
thresholding, and morphological operations.

For example, the circularity of a shape can be calculated by dividing the area
of the shape by the area of its circumscribed circle. Similarly, the aspect ratio
of a shape can be calculated by dividing its width by its height.

In image processing applications such as object recognition and tracking,

shape numbers play an important role in identifying and distinguishing
different objects based on their geometric features. They can also be used to
detect and quantify changes in the shape of objects over time, which can be
useful for monitoring and analyzing dynamic processes.

General Frame Works For Matching

Matching

Matching in image processing refers to the process of comparing two images

and finding the degree of similarity between them. This is important in a
variety of applications, including computer vision, object recognition, and
image retrieval.

Matching in image processing can be used for a variety of applications, such as

identifying objects in an image, recognizing faces, and tracking objects in a
video stream. It is an important tool for improving automation and efficiency in
a range of industries, from manufacturing to healthcare.

Distance relational approach

The distance relational approach in image processing is a technique used for

analyzing the spatial relationships between objects in an image.

It involves measuring the distances between different objects or regions in an

image and using these distances to represent the relationships between
them.

Image Processing and Computer

Vision
 The basic idea behind the distance relational approach is to represent an image as a
set of points in a vector space, where each point corresponds to a specific region of
the image.

The distance between two points in the vector space represents the spatial
relationship between the two corresponding regions.

This approach is often used for segmentation, object recognition, and

classification tasks in image processing.

By analyzing the distance relationships between different objects in an image, it

is possible to identify and classify different objects and regions based on their
spatial characteristics.

The distance relational approach is particularly useful for analyzing complex

images with multiple objects, as it allows for a more detailed representation of
the spatial relationships between these objects.

It can also be combined with other image processing techniques, such as

feature extraction and machine learning, to improve the accuracy and efficiency
of image analysis tasks.

Ordered structural matching

Ordered structural matching is a process used in computer vision and image

processing to compare two images and determine the similarity between them.
The process involves comparing the structural information of the images based
on their positions.

In ordered structural matching, the images are first preprocessed to extract their
structural information based on the positions of the pixels. This information is
then compared between the two images to determine the degree of similarity.

The comparison process typically involves computing a similarity score between the
two images based on a set of predefined rules. These rules may consider factors
such as the position of the pixels, their intensity values, and the size and shape of
the structures in the images.

Image Processing and Computer

Vision
 Ordered structural matching is commonly used in applications such as object
recognition, face recognition, and fingerprint recognition, where the position and
structure of the features in the images are important for accurate matching.
The technique can also be used for image retrieval and content-based image
retrieval, where similar images are found based on their structural similarity.

View class matching

View class matching in image processing refers to the process of identifying

objects or regions in an image that belong to a certain class or category.

This can be achieved using various algorithms and techniques such as

supervised or unsupervised learning, feature extraction, template
matching, etc.

For example, in a medical image, view-class matching can be used to detect

specific abnormalities such as tumors, cysts or nodules. Similarly, in a
surveillance image, view- class matching can be used to identify specific
objects such as vehicles, pedestrians, or animals.

View class matching is an important step in many applications of computer

vision and image processing such as object detection, recognition, tracking,
and segmentation. It enables automatic and efficient analysis of large
datasets, leading to better decision- making and improved outcomes.

Image Processing and Computer

Vision
 Models database organization

💡 Image Database (IDB) is a logically coherent collection of im- ages with

some inherent meaning. The images usually belong to a specific
application domain. An IDB is designed, built, and populated with
images for specifics purpose and
represents some aspects of the real world.

Image Database Management System (IDBMS) is a collection of programs that

enable the user to define, construct and manipulate an IDB for various
applications.

There are several possible ways to organize a database for image processing,
depending on the specific application and the available resources. One possible
model is the following:

1. Data acquisition: The first step is to collect the raw data, which can come from
various sources such as cameras, sensors, scanners, or external files. The
data can be in different formats such as JPEG, PNG, TIFF, BMP, or RAW, and
may have different characteristics such as resolution, color depth,
dynamic range, or spatial orientation.

2. Preprocessing: The next step is to apply some basic operations to the raw data
to enhance its quality or extract some useful features. This can include
operations such as noise reduction, contrast enhancement, color
normalization, edge detection, or feature extraction. The preprocessed data
can be saved as separate images or as metadata attached to the original
data.

3. Annotation: The next step is to add some metadata or labels to the images,
either manually or automatically. This can include information such
as the object class, the position, the size, the shape, or the orientation of
the objects in the image. The annotations can be stored in separate files
or as part of the image metadata.

4. Storage: The final step is to store the processed data in a database that can
efficiently query and retrieve the images based on various criteria. The
database can be organized in different ways such as by object
UNIT class,
4 by image features, by time, or by location. The database can use
Facet Model types
various Recognition
of indexing structures such as B-trees, hashes, or spatial
Facetindexes
models to
areoptimize
used in the search
image performance.
processing to recognize similar patterns and
structures
Overall, in an provides
this model image. The facetframework
a basic model is a for
mathematical model
organizing an that is used to
image
describedatabase,
processing structures that
but thehave repeating
details patterns.
can vary These
depending onpatterns may be found
the specific
in a variety
requirements of contexts,
and such
constraints as application.
of the in images of cells or crystals.

Image Processing and Computer

Vision
 The idea behind the facet model is to identify a repeating pattern in an image
and then use that pattern to identify other similar structures. This can be done
by creating a template or "facet" of the repeating pattern and then matching it
to other parts of the image.

Facet models are particularly useful in materials science, where they can be
used to identify crystal structures and other repeating patterns in materials.
They can also be used in computer vision applications, such as object
recognition in images.

To use a facet model for image recognition, the image must first be processed
to identify the repeating patterns. This can be done using techniques such as
Fourier analysis or wavelet transforms. Once the repeating patterns have been
identified, a template or facet can be created and used to match other similar
patterns in the image.

Facet models are a powerful tool for image recognition, but they can be
computationally intensive and require specialized software and hardware to
implement effectively. However, with the growing complexity of modern images,
facet models are becoming an increasingly important tool for identifying
patterns and structures in images.

Labeling lines

Understanding line drawings

Classification of shapes by labeling of edges

Recognition of shapes

Consisting labeling problem

Back-tracking Algorithm
Discuss back tracking
algorithm by an example.

The backtracking algorithm is a recursive algorithm that is used to solve problems

by keeping track of all possible solutions and eliminating those solutions which are
not feasible. Here is an example of how this algorithm works:

Suppose we have a Sudoku puzzle to solve, which is a famous puzzle of numbers

arranged in a 9x9 grid. The objective is to fill each cell of the grid with a number
from 1 to 9, such that each row, column, and 3x3 sub-grid contains all of the digits
from 1 to 9.

To solve this problem using backtracking algorithm, we can start by selecting an

empty cell in the grid, and then try filling it with a number from 1 to 9. After that,
we check if the number is valid, which means it does not repeat in the same row,
column or sub-grid. If the number is valid, then we move on to fill the next empty
cell, and repeat the process until we have filled all the cells. If we reach a point
where we cannot fill the next cell with a valid number, we backtrack to the previous
cell and try filling it with a different number.

Here is the algorithm in more detail:

1. Select an empty cell in the grid.

Image Processing and Computer
Vision 2. Try filling the cell with a number from 1 to 9.

3. Check if the number is valid, i.e., it does not repeat in the same row, column, or
 4. If the number is valid, move on to fill the next empty cell. If there are no more
empty cells, the puzzle is solved.

5. If the number is not valid, try filling the cell with a different number.

6. If no valid number is found, backtrack to the previous cell and try filling it
with a different number.

7. Repeat steps 2 to 6 until the puzzle is solved.

In this way, the backtracking algorithm uses a recursive process to explore all
possible solutions until the correct one is found. It is important to note that
backtracking algorithms can be computationally expensive, especially for large
problem spaces, but they are often the most efficient way to solve certain types of
problems.

Perspective Projective geometry

Inverse perspective Projection

Inverse perspective projection, also known as reverse perspective or Byzantine

perspective, is a type of perspective drawing method that is used to create
distorted images that appear as if they are receding into the image plane rather
than projecting outwards. In this drawing method, the lines of the subject are
drawn as if they are emanating from a point behind the picture plane, creating
a distorted image that appears to be tilted upwards.

The inverse perspective projection method was commonly used in Byzantine art
and architecture, where it was employed to depict religious and spiritual
scenes. The technique was also used in the art of the Renaissance, where it was
used to create unique and dramatic effects.

One of the most famous examples of the inverse perspective projection can be
found in the painting titled "The Last Supper" by Leonardo da Vinci. In this
painting, the lines of the room are drawn in a way that they appear to be
receding into the background, creating a dramatic and otherworldly effect.

Overall, inverse perspective projection is a unique and creative method of

creating art that allows artists to experiment with spatial distortion and create
intriguing and captivating images that can transport viewers into a different
world.

Compare weak perspective projection and orthographic projection in affine projection

models.

Affine projection models are mathematical models used to represent 3D

objects on a 2D surface. Within these models, there are two main types of
projections: weak perspective projection and orthographic projection.

Weak perspective projection, also known as a scaled orthographic projection or

oblique projection, is a type of projection where the distance between the camera
and the object is finite, but not large enough to be considered as a perspective
projection. In weak perspective projection, objects appear smaller as they move
away from the camera, but their shape is not distorted. This type of projection is
commonly used in architectural drawings, where it is necessary to show objects in
3D space
Image Processing andinComputer
a 2D drawing.
Vision
 Orthographic projection, also known as a parallel projection, is a type of projection
where the viewing plane is parallel to the object being projected. In orthographic
projection, all lines that are parallel in the 3D space remain parallel in the 2D
representation. This projection is often used in technical and engineering drawings,
where it is important to accurately depict the shape and size of objects.

Image Processing and Computer

Vision
 In summary, weak perspective projection is used to show objects in 3D space
from a finite distance, while orthographic projection is used to accurately
depict the shape and size of objects from a viewing plane that is parallel to the
object being projected.

Photogrammetric -from 2D to 3D,

Image matching: Intensity matching of ID signals, Matching of 2D image, Hierarchical

image matching.

Object Models And Matching: 2D representation, Global vs. Local features

UNIT 5
Describe various computer vision models and their applications.

There are several computer vision models that are used in various applications.
Some of these models are:

1. Convolutional Neural Networks (CNN): CNNs are deep learning models that can
analyze visual imagery. These models are used in image and video recognition
applications such as object recognition, face recognition, and gesture
recognition.

Image Processing and Computer

Vision
 2. Recurrent Neural Networks (RNN): RNNs are deep learning models that can
analyze sequential data. These models are used in applications such as
speech recognition, sentiment analysis, and handwriting recognition.

3. Optical Character Recognition (OCR): OCR is a computer vision model that can
recognize text in an image or video stream. This model is used in applications
such as document scanning, license plate recognition, and handwriting
recognition.

Overall, computer vision models have various applications in several domains

such as healthcare, automotive, security, and entertainment.

Knowledge Based Vision

Knowledge Based Vision

Knowledge-based vision refers to the ability of computer systems to recognize

objects, scenes, and other visual data using pre-existing knowledge bases. This
means that rather than requiring thousands of examples of each object or
scene to be trained on, the computer system can rely on prior knowledge of
the object or scene in question to recognize it more efficiently.

This type of vision system typically involves the use of artificial intelligence,
machine learning, and other advanced technologies to analyze images and
other visual data. It is
Image Processing and Computer
Vision
 commonly used in applications such as computer vision, autonomous
vehicles, robotics, and more.

By incorporating knowledge bases into vision systems, these systems can

become more accurate, faster, and more efficient. Additionally, they can be
more adaptable to varying situations and environments, making them ideal
for use in a variety of industries and applications.

Knowledge representation

Knowledge representation in computer vision refers to the process of

representing visual information in a structured and systematic way that can be
easily interpreted and processed by a computer. This involves defining a set of
rules and algorithms that can be used to extract features and objects from
images or videos, and organizing this information into a meaningful format that
can be analyzed and understood by both humans and machines.

There are various techniques used for knowledge representation in computer

vision, including symbolic reasoning, logic-based inference, probabilistic
modeling, and deep learning. Symbolic reasoning involves representing visual
information as symbols and using rules and algorithms to manipulate these
symbols to derive meaning.

Logic-based inference uses mathematical logic to represent and reason about

visual information, while probabilistic modeling uses statistical methods to
model the uncertainty and variability of visual data. Deep learning, on the
other hand, involves training neural networks to recognize and classify visual
features and objects through a process of learning from large datasets.

Overall, knowledge representation plays a key role in enabling computers to

understand and interpret visual information, and is an essential component in
enabling the development of advanced computer vision systems for a variety of
applications.

Control-strategies

Control strategies in computer vision aim to achieve specific objectives, such

as object recognition, segmentation, tracking, and classification. These
strategies typically involve a combination of software and hardware
components designed to achieve a particular task.

Some common control strategies used in computer vision include:

1. Object detection and recognition: This involves identifying specific objects or

patterns within an image or video stream. It typically involves
creating a model or template of the object of interest and then using
algorithms to detect and classify it in real-time.

2. Segmentation: This involves dividing an image into distinct regions or segments,

which are then analyzed separately. One common use of segmentation is in
medical imaging, where it is used to identify specific structures, such as
tumors.

3. Tracking: This involves following the movement of objects across a series of

frames in a video stream. It is often used in surveillance, robotics, and
Image Processing and Computer
autonomous vehicles to track moving targets and predict their
Vision
movements.
 sensing and satellite imagery to detect and classify land use patterns.

Overall, control strategies in computer vision are an essential part of developing

effective and efficient systems for image and video analysis. They enable us to
extract meaningful information from visual data and make decisions based on that
information.

Information Integration

Information integration in computer vision refers to the process of combining

information from multiple sources, such as sensors, cameras, and databases,
to generate a complete and accurate understanding of a scene or an object.
This process often involves several steps, including data fusion, feature
extraction, and pattern recognition.

Data fusion involves combining data from multiple sensors or sources to

create a more complete and accurate representation of a scene or object.
For example, combining information from a depth sensor and a camera can
provide a more detailed view of an object’s geometry and texture.

Feature extraction involves identifying relevant features or characteristics of an

object or scene, such as edges, shapes, and textures, that can be used to
identify and classify objects. These features are often extracted using machine
learning algorithms that analyze large amounts of data to identify patterns.

Pattern recognition involves using these features to identify and classify

objects. This process often involves comparing the extracted features with a
database of known objects to identify the best match.

Overall, information integration is an important process in computer vision

that allows machines to generate more accurate and reliable visual
information and make more informed decisions.

Object recognition
Object recognition

Object recognition in computer vision refers to the ability of a computer or

machine to identify and categorize objects within digital images or video frames.
The recognition process typically involves using algorithms and machine learning
techniques to analyze image features such as edges, corners, textures, and colors.
Some common applications of object recognition include face recognition, text
recognition, and object tracking. Object recognition is an important component of
many computer vision systems, particularly those used in autonomous vehicles,
robotics, and surveillance.

Hough transforms and other simple object recognition methods

Hough transforms are a common method for detecting simple geometric shapes
like lines, circles, and ellipses in an image. This algorithm works by converting
the image space into a transformed space (a Hough space) where each point
represents a parameter set of the desired shape.

For example, in the case of detecting lines, each point in the Hough space
represents a pair of parameters (ρ,θ) that define a line in the original image.
Image Processing and Computer
The algorithm then
Vision
 searches for peaks in the Hough space, which correspond to the most likely
lines in the original image.

This technique is straightforward and computationally efficient, but it has

limitations. Hough transforms require the shape to be clearly defined and have a
consistent set of parameters, such as a line with a fixed slope and intercept.
When the object has more complex shapes or varies in size, orientation, or
illumination, more sophisticated techniques are necessary.

Other simple object recognition methods include template matching and

feature-based methods. Template matching involves searching for a known
pattern in the image by comparing it to a pre-defined reference image. This
technique is effective when the pattern is known and has little variation.

Feature-based methods involve identifying key points in the image and

describing them based on their attributes, such as edges, corners, or texture.
These features are then matched to a database of known features to recognize
the object. Feature-based methods are more versatile than template matching
because they can handle more complex shapes and variations.

Overall, simple object recognition methods like Hough transforms, template

matching, and feature-based methods are useful for detecting basic shapes
and patterns but may not be suitable for more complex recognition tasks. More
advanced techniques, such as deep learning, are necessary for more robust
and accurate object recognition.

Shape correspondence and shape matching

Shape correspondence and shape matching are two important tasks in computer
vision that involve finding similarities and correspondences between geometric
shapes.

Shape correspondence aims to establish correspondences between different

instances of the same object or different objects with similar shapes. This is an
important task in fields such as object recognition, scene analysis, and shape
modeling. Establishing correspondences between shapes allows us to transfer
knowledge across datasets and to learn shape representations that are
invariant to various transformations.

Shape matching, on the other hand, involves finding the correspondence

between two or more shapes that may be slightly different in size, rotation, or
deformation. This task is important in areas such as medical imaging, robotics,
and 3D modeling, where it is necessary to align shapes to perform tasks such
as segmentation, registration, tracking, and reconstruction.

Various methods have been proposed to address these tasks, such as point-
based, feature-based, and deep learning-based approaches. These methods
use a combination of geometric and statistical techniques to extract relevant
features, match correspondences, and establish correspondences between
shapes. The performance of these methods depends on the quality of the input
data, the type of problem being addressed, and the accuracy of the feature
extraction and matching algorithms.

Image Processing and Computer

Vision
 Principal component analysis

Principal component analysis (PCA) is a commonly used technique in computer

vision for feature extraction and data reduction. It is a statistical method that
analyzes the relationships between variables in a dataset and identifies the
underlying patterns or structures.

In computer vision, PCA is used to analyze the variability of pixel values in an

image dataset and identify the most important features or components.
These components are then used for tasks such as face recognition, object
recognition, and image retrieval.

PCA can also be used for data compression, as it can reduce the
dimensionality of high- dimensional image data by removing noise and
redundancy. This can make it easier to work with large datasets and increase
the efficiency of machine learning algorithms.

Overall, PCA is a powerful tool in computer vision for analyzing and extracting
meaningful patterns from complex image data.

Image Processing and Computer

Vision
 Feature extraction

In computer vision, feature extraction refers to the process of detecting and

extracting specific image patterns or features that can be used to classify,
recognize, or understand an image.
Certain visual patterns or spatial cues can help identify objects or regions in an
image, such as edges, corners, shapes, textures, or colors. Feature extraction
involves selecting relevant features that can represent or capture the essential
characteristics of an image, while filtering out irrelevant or redundant information.

Feature extraction algorithms typically analyze the pixel values of an image and
transform them into a set of numerical features or descriptors that can be compared
or clustered to identify similarities or differences between images. Some common
techniques for feature extraction in computer vision include:

1. Edge detection - Identifying sharp boundaries or transitions between regions

of high and low intensity in an image.

2. Corner detection - Identifying points where the curvature or gradient of an

image changes abruptly.

3. Scale-invariant feature transform (SIFT) - Extracting distinctive local

features that are invariant to changes in scale, rotation, and
affine transformation.

4. Histogram of oriented gradients (HOG) - Computing the distribution of gradient

orientation and magnitude across an image to represent the shape or
structure of an object.

5. Convolutional neural networks (CNNs) - Learning hierarchical features by

applying convolutional layers to extract features at different levels
of abstraction.

Feature extraction is an important step in many computer vision tasks, such as

object detection, facial recognition, image retrieval, and scene analysis. It can help
reduce and
Image Processing the dimensionality
Computer and complexity of image data, and enable efficient and
Vision accurate processing of large-scale visual datasets.
 Neural network and Machine learning for image shape recognition

Neural network and machine learning are powerful tools for image shape
recognition. Neural networks are a type of machine learning algorithm that can
be trained on large amounts of data to recognize complex patterns and make
predictions. Convolutional neural networks (CNNs) are a type of neural network
that is specifically designed for image recognition tasks.

The process of image shape recognition using neural networks and machine
learning involves several steps. First, the input image is preprocessed to extract
relevant features, such as edges and corners. This feature extraction process is
typically done using specific filters, such as the Sobel filter or the Canny filter.

Next, the preprocessed image is passed through a neural network for

classification. The neural network is trained on a set of labeled images, where
the correct label is known for each image. During training, the weights of the
neural network are adjusted to minimize the difference between the predicted
labels and the actual labels.

Once the neural network has been trained, it can be used to classify new
images. The input image is preprocessed and then passed through the neural
network, which predicts the shape of the object in the image. The predicted
shape can then be used for further analysis or as input to another algorithm or
system.

Overall, neural networks and machine learning are powerful tools for image
shape recognition. With the right training data and neural network
architecture, it is possible to achieve high levels of accuracy in recognizing
complex shapes in images.

Explain the use of neural network structures for pattern recognition with an example.

Image Processing and Computer

Vision
 Neural network structures are used for pattern recognition in many different fields,
from computer vision to speech recognition to natural language processing. One
example of neural network structures for pattern recognition is a convolutional
neural network (CNN) used for object recognition in images.

A CNN is a type of neural network that is specifically designed to work with images.
It consists of layers of interconnected nodes called neurons that process and
analyze the image data. The first layer of a CNN detects simple features such as
edges and corners, while subsequent layers combine those features to detect more
complex patterns, such as shapes and objects.

For example, a CNN trained to recognize cats might start with simple features like
edges and corners, and gradually learn to recognize more complex patterns such as
fur and whiskers, until it is able to identify an entire cat in an image. The network is
trained on a large dataset of images labeled with the object they contain, and it
adjusts the neuron weights and biases over time to improve its accuracy at
identifying those objects.

Once the CNN is trained, it can be used to recognize objects in new images that it
has never seen before. The image is fed into the network, and the activations of
the neurons in the final layer are compared to the patterns learned during training
to determine which object the image contains. The output of the network is a
probability distribution over the possible object categories, indicating which one the
network believes is most likely.

Overall, neural network structures like CNNs are powerful tools for pattern
recognition across a wide range of applications, allowing machines to learn to
recognize and classify complex patterns with high accuracy.

How to evaluate extracted shape descriptors in 3D vision? Discuss.

In 3D vision, shape descriptors are used to summarize and represent the geometry
of an object or scene. These descriptors can be defined based on different
properties such as curvature, volume, or surface area. They are widely used in
various applications such as object recognition, matching, and registration.
However, evaluating the performance of shape descriptors is crucial to ensure their
effectiveness and reliability. Here are some methods to evaluate extracted shape
descriptors in 3D vision:

1. Object recognition: Object recognition is one of the common applications of

shape descriptors. The performance of a descriptor can be evaluated
by applying it to different datasets of the same class and measuring its
accuracy. The accuracy can be calculated using metrics such as precision,
recall, F1-score, and mean average precision (mAP).

2. Shape comparison: Shape comparison is another important application of

shape descriptors. The quality of a descriptor can be evaluated by
comparing it with other descriptors using different metrics such as the
Jaccard distance, Hausdorff distance, and Chamfer distance.

3. Robustness to noise and occlusion: In real-world scenarios, 3D data is often

noisy or occluded. The performance of a descriptor can be evaluated by
analyzing its robustness to these factors. One way to test this is by adding
Image Processing and Computer
Vision noise or occlusion to the 3D data and comparing the results.
 4. Computational efficiency: Shape descriptors are often used in real-time
applications such as robotics and autonomous vehicles. Therefore,
the computational efficiency of a descriptor is a critical factor. The
speed and memory usage of a descriptor can be evaluated and
compared with other descriptors.

5. Human perception: Human perception is an important evaluation criteria for

shape descriptors. Human perception experiments can be conducted
where humans are asked to evaluate the similarity of different
shapes based on the descriptors.

In conclusion, evaluating the performance of extracted shape descriptors in 3D

vision is critical to ensure their effectiveness and reliability. The evaluation methods
depend on the application and the requirements of the system. Therefore, it is
important to choose the appropriate evaluation method based on the application
and analyze the results carefully.

Explain the identifying road signs in vehicle vision system.

Identifying road signs in a vehicle vision system involves using various sensors to
detect and analyze the images of road signs. The process involves the following
steps:

1. Image acquisition: The system uses cameras or other sensors to capture

images of road signs. These images are sent to the system's
processing unit for analysis.

2. Pre-processing: The images are first pre-processed to remove any noise or

unwanted artifacts. This helps to improve the accuracy of the system.

3. Object detection: Using computer vision algorithms and machine learning

techniques, the system identifies the presence of road signs in the
captured images.

4. Classification: Once a road sign is detected, the system classifies it by

identifying its shape, color, and any text or symbols on it.

5. Interpretation: Based on the classification, the system interprets the meaning

of the road sign and provides the driver with relevant information or
warnings.

Some of the road signs that a vehicle vision system can identify include speed limit
signs, stop signs, yield signs, no parking signs, and pedestrian crossing signs. The
system can also detect temporary signs such as construction zone signs and school
zones signs.

Discuss about accurate centre location by using Hough transform.

Image Processing and Computer

Vision
 The Hough transform is a technique used for detecting shapes in an image. It is
particularly useful in detecting lines and circles. One application of the Hough
transform is to accurately determine the centre location of objects in an image.

To use the Hough transform for centre location, the following steps can be
followed:

1. Threshold the image to obtain a binary image.

2. Apply edge detection to obtain a set of edge points.

3. For each edge point, generate a set of circles that pass through that point.
These circles should have radii that range from a minimum value
(corresponding to the smallest object in the image) to a maximum value
(corresponding to the largest object in the image).

4. For each circle, increment a counter in a Hough accumulator matrix at

the location corresponding to the circle's centre.

5. Once all circles have been checked, the cell in the accumulator matrix with
the highest count value will correspond to the centre location of the
object in the image.

6. If multiple objects need to be located, repeat the process for each object.

The Hough transform is a powerful tool for accurately locating the centres of
objects in an image. Its ability to detect circles of varying radii allows it to detect
objects of different sizes, making it a versatile tool in image processing and
computer vision applications.

Image Processing and Computer

Vision