CHAPTER III.

EDGE DETECTION AND OBJECT DETECTION

ALGORITHMS

Overview of Edge detection methods

Introduction
Edge detection is basically an image segmentation technique that divides
spatial domain, on which the image is defined into meaningful parts or regions [1].
Edges characterize boundaries of objects in an image and object recognition is of
fundamental importance in image processing. Edges typically occur on the
boundary between two different regions in an image, where there is a more or less
abrupt change in gray level or texture indicating the end of one region and the
beginning of another. Edge detection allows user to observe those boundaries.
Edge detection therefore finds practical applications in medical imaging,
computer-guided surgery, locate object in satellite images, face recognition, and
finger print recognition, automatic traffic controlling systems, study of anatomical
structure etc. Several edge detection techniques have been developed such as
gradient based standard operators like Robert, Prewitt, Sobel. However, they have
not given sharp edges and have been highly sensitive and provided poor
performance in noisy images [1]. Laplacian based Marr Hildrith operators also
suffers from two limitations: high probability of detecting false edges and the
localization error may be severe at curved edges but the algorithm proposed by
Canny in 1986 is considered as the ideal edge detection algorithm for images that
are corrupted with noise [2]. Canny’s aim is to discover the optimal edge detection
algorithm which reduces the probability of detecting false edges, and gives sharp
edges. Wavelet analysis is an efficient tool for edge detection. Deterministic
methods are inappropriate when noise models are applied. In images with low
SNR, the noise must be accounted with a noise model, multiplicative or additive
[3]. An important issue in edge detection is the scale of detection filter. Small-
scaled filters are sensitive to edge signals but also proneto noise, whereas large-
scaled filters are robust to noise but could filter out fine details. As suggested by
Marr and Hildreth [4], multiple scales could be employed to describe and
synthesize the varieties of edge structures. The idea of scale multiplication was
first exploited in [5], where it was shown that the scale products could improve the
edge localization. Mallat [6] illustrated mathematically that signals and noise had
different singularities and edge structures present observable magnitudes along the
scales, while noise decreased rapidly. With this observation, Xu et al. [7] was
given a wavelet-based spatially selective filtering technique by multiplying the
adjacent scales. Sadler and Swami [8] applied the wavelet-multiscale-products to
step detection and estimation and Bao and Zhang [9] presented a denoising scheme
by thresholding the multiscale products.

Edge detection is the process of determining where edges of objects fall

within an image. Figure 1 shows schematic diagram for the standard edge
detection method. To sense and detect abrupt change at edges, several operators
have been constructed based on different ideas. In the following section, a brief
review of some operators is presented.

First Order Edge Detector

The gradient method detects the edges by looking for the maximum and
minimum in the first derivative of the image. Thus, consider the two dimensional
function f (x , y ) to represent the input image then image gradient is given by the
following formula [1]:

Figure 1. Standard edge detector.

[ ]
∂ f (x, y)

[]
∇ f =G[f ( x , y)]=
fx
fy
=
∂x
∂ f (x, y)
∂y
(1)

The gradient magnitude can be computed by the formula:

 ¿ ∇ f ∨¿ √ f 2x + f 2y =
√( ∂x )( )
∂f 2 ∂f 2
+
∂y
≈|f x|+|f y| (2)

The gradient direction can be computed by the formula:

−1
θ=tan ⁡( f y / f x ) (3)

The magnitude of the gradient computed by Equation (2) gives edge strength
and the gradient direction is always perpendicular to the direction of the edge.
Robert, Sobel, and Prewitt operators are classified as standard first order derivative
operators which are easy to operate but highly sensitive to noise [1].

Robert Operator

The input image is convolved with the default kernels of operator and
gradient magnitude and directions are computed. It uses the following 2 ×2 two
kernels [10]:

G x=
[ +10 −10 ] and G =[−10 +10 ]
y (4)

The plus factor of this operator is its simplicity but having small kernel it is
highly sensitive to noise and not compatible with today's technology [10].

Sobel Operator

It convolves the input image with kernel and computes the gradient
magnitude and direction. It uses the following 3 ×3 two kernels:

[ ] [ ]
−1 0 +1 +1 +1 +1
G x = −1 0 +1 and G y = 0 0 0 (5)
−1 0 +1 −1 −1 −1

As compared to Robert operator, Sobel operator has slow computation.

However it is less sensitive to noise as compared to Robert operator.

Prewitt Operator
 Prewitt edge operator gives better performance than that of Sobel operator.
The function of Prewitt edge detector is almost the same as Sobel detector but have
different kernels:

[ ] [ ]
−1 0 +1 +1 +1 +1
G x = −1 0 +1 and G y = 0 0 0 (6)
−1 0 +1 −1 −1 −1

Second Order Edge Detector

Any definition of a second derivate 1) must be zero in flat areas; 2) must be

nonzero at the onset and end of a gray level step and ramp; and 3) must be zero
along ramps of constant slope [11]. The Laplacian operator ∇ 2 for a 2D image
f (x , y ) is defined by the following formula [10]:

2 2
2 ∂ ∂
∇ f ( x , y)= 2
f (x , y )+ 2 f (x , y ) (7)
∂x ∂y

There are several ways to define digital Laplacian using neighborhoods.

Whatever the definition, it has to satisfy the properties of a second derivate
outlined in Equation (11). The following notation is used for second order
derivative in the x and y directions [11]
2
∂ f
2 2
=f ( x+1 , y ) + f ( x−1 , y )−2 f ( x , y ) (8)
∂ x
2
∂ f
2 2
=f ( x , y+ 1 )+ f ( x , y−1 )−2 f ( x , y ) (9)
∂ y
∇ f = [ f ( x+1 , y ) + f ( x−1 , y ) + f ( x , y+ 1 )+ f ( x , y−1 ) ] −4 f ( x , y ) (10)
2

Furthermore, when the first derivative is at a maximum, the second

derivative is zero [12].

Laplacian

Laplacian of an image f (x , y ) is defined by:

2
L[f ( x , y )]=∇ f [f (x+1 , y )+ f (x−1 , y )+ f (x , y +1)+ f (x , y −1)]−4 f (x , y) (11)

In this case, Laplacian kernel given by the following equation is adopted.

 [ ]
0 1 0
G xy = 1 −4 1 (12)
0 1 0

Laplacian of Gaussian (Marr-Hildreth Edge Detector)

The Laplacian of Gaussian (LoG) follows the following four steps [12]:

1. Smooth the image using Gaussian filter which is defined by:

( )
2 2
x +y
−
1 2 σ2
(13)
G(x , y)= e
√2 π σ 2
2. Enhance the edges using Laplacian operator which given by this
equation [12]:

[ ]
2 2
2 2 −x + y
−1 x +y 2

log ⁡(x , y )= 4 1− 2
e 2σ
(14)
πσ 2σ

3. Estimate the zero crossings denote the edge location.

4. Use linear interpolation to determine the sub-pixel location of the edge.

Greater the value of σ , broader the Gaussian filter, more is the smoothing.
Too much smoothing may make the detection of edges difficult; also called the
Maxican Hat operator.

Difference of Gaussian

To reduce the computational requirements, the LoG is approximated by the

DoG. The width of the edge can be adjusted by changing σ 1 and σ 2. The difference
of Gaussian (DoG) operator of an image f (x , y ) is defined by:

e
−
( x 2+ y2
2 π σ1
2
) e
( −
x2 + y 2
2 πσ2
2
)
DoG ⁡(x , y )= − (15)
2 2
2π σ 1 2π σ 2

LoG requires large computation time for a large edge detector mask. Second
Order Derivative Methods Properties. The following are the three most important
properties of the second order derivatives:
 1. Second order derivative methods especially Laplacian, are very sensitive to
noise.

2. Probability of false and missing edges remains.

3. Localization is better than gradient operators.

Canny Edge Detector

The Canny edge detection algorithm is known to many as the optimal edge
detector. Canny's intentions were to enhance the many edge detectors already out
at the time he started his work. He was very successful in achieving his goal and
his ideas and methods can be found [2]. The first and most obvious is low error
rate. It is important that edges occurring in images should not be missed and that
there be no responses to non-edges. The second criterion is that the edge points be
well localized. In other words, the distance between the edge pixels as found by the
detector and the actual edge is to be at a minimum. A third criterion is to have only
one response to a single edge. This was implemented because the first two were
not substantial enough to completely eliminate the possibility of multiple responses
to an edge.

Based on these criteria, the Canny edge detector first smoothies the image to
eliminate the noise. It then finds the image gradient to highlight regions with high
spatial derivatives. The algorithm then tracks along these regions and suppresses
any pixel that is not at the maximum (non-maximum suppression). The gradient
array is now further reduced by hysteresis. Hysteresis is used to track along the
remaining pixels that have not been suppressed. Hysteresis uses two thresholds and
if the magnitude is below the first threshold T 1, it is set to zero (made a non-edge).
If the magnitude is above the high threshold T 2, it is made an edge. And if the
magnitude is between the two thresholds, then it is set to zero unless there is a path
from this pixel to a pixel with a gradient above the second threshold T 2. In order to
implement the canny edge detector algorithm, a series of steps must be followed
[13].
 Step 1: Smooth the image with a two dimensional Gaussian filters.

Step 2: Take the gradient of the image.

Step 3: The direction of the edge is computed using the gradient in the x and
y directions.

Step 4: relate the edge direction to a direction that can be traced in an image.

Step 5: Non-maximum suppression.

Step 6: Hysteresis.

Flowchart of edge detection algorithm

Advantages and Disadvantages of Standard Edge Detection Methods

Each edge detection method has its advantages and disadvantages. Table 1
summarize the main advantages and disadvantages of each method [12].
 Edge Detection Using Wavelets

2D Discrete Wavelet Transform Edge Detection

First define a Gaussian equation in two dimensions as [14] [15]

(( ) ( ) )
x−μ x 2 y− μy 2
1
+
1 2 σx σy
(16)
θ(x , y)= e
2 π σxσ y

Table 1. Advantages and disadvantages of edge detector ion methods.

Method Advantages Disadvantages

Simplicity, detection of
Standard (Sobel, Prewitt, Sensitivity to noise,
edges
Kirsch) inaccurate
and their orientations
Detection of edges and
Responding to some of
Zero crossing their orientations.
the
(Laplacian, 2nd Having fixed
existing edges, sensitivity
directional derivative) characteristics in all
to noise
directions
Malfunctioning at the
corners, curves and
Finding the correct where the gray level
Laplacian of Gaussian places of edges, intensity function
(LoG)-(Marr-Hildreth) testing wider area around varies. Not finding the
the pixel orientation of edge
because of using the
Laplacian filter
Using probability for
finding error rate,
localization of response. Complex computations,
Gaussian (Canny, Shen-
Improving signal to noise false zero crossing,
Castan)
ratio, better detection time consuming
specially in noise
conditions

where, σ is standard deviation of the signal, and assume that μ x =μ y =0 .

Define two wavelet functions Ψ 1 (x , y) and Ψ 2s (x , y) such that
 ∂θ(x, y) ∂θ(x, y)
Ψ 1 ( x , y )= and Ψ 2 ( x , y )= (17)
∂x ∂y
1 1
( )
x y
Ψ s ( x , y )= 2 Ψ 1 , (18)
s s s
1
( )
x y
Ψ 2s ( x , y )= 2 Ψ 2 , (19)
s s s

The wavelet transform of f (x , y ) at the scale s has two components defined

by
1 1 2 2
W s f ( x , y )=f ∗Ψ s ( x , y ) and W s f ( x , y )=f ∗Ψ s ( x , y ) (20)

( )
∂
( f ∗θ s ) (x , y )
( )
1
ws f (x , y ) ∂x
2
=s =s ∇ ( f ∗θ s ) ( x , y ) (21)
W s f (x , y) ∂
∂y
( f ∗θs ) (x , y)

Wavelet Edge Detector Implementation

Wavelet edge detection can be demonstrated as in Figure 2. Each process

and the corresponding output will be discussed in the following sections.

The Smoothing

Smooth the image and detect the sharp variation points from their first order
derivative. The smoothing function is any function whose integral is equal to 1 and
that converges to 0 at infinity. One can choose smoothing function equal to a
Gaussian.

(( ) ( ) )
x− μx 2 y− μy 2
−1
+
1 2 σx σy
(22)
θ(x , y)= e
2 π σxσ y

Assume σ x =σ y ; so the rotation variation is ignored i.e. μ x =μ y =0 [14].

By convolving the image with smoothing function removes small signal

fluctuation, then sharp variations will only be detected. The value of σ can be
changed from image to image according to the amount of edges and detail. Linear
filtering of an image is accomplished through an operation called convolution. In
convolution, the value of an output pixel is computed as a weighted sum of
neighboring pixels. The matrix of weights is called the convolution kernel, also
known as the filter. In general case the wavelet domain in 2-dimension has R4
(Four dimensions) namely ( x , y , scale and rotation). At each step, the image is
convolved with a wavelet to obtain the coefficients. It is then smoothed with a
Gaussian. Both the wavelet and Gaussian filtering are done using separate 1-D
filters vertically and horizontally.

Figure 2. Standard wavelet edge detection method.

Local Modulus Maxima

In order to compute the local maxima of the modulus of the image in DWT
need to find: 1 -the modulus and 2- the angle of the DWT at each scale. The
modulus maxima are then defined as the points of the modulus images that are
larger than the two neighbors whose positions are in the direction indicated by the
angle value at the corresponding location. Let's see how to get the modulus of an
image:

√ 2 2
M s f (x , y )= |W s f ( x , y)| +|W s f (x , y)|
1 2
(23)

Then get the angle of the DWT. There are only four possible directions
when describing the surrounding pixels: 0 degree (in the horizontal direction), 45
degree (along the positive diagonal), 90 degree (in the vertical direction), or 135
degree (along the negative diagonal). From the modulus image and the angular
DWT can find the local modulus maxima.

Chain the Edge

 The edge points can then be gathered together into chains. For point that is a
maximum, can join it with the point closest to it if they have similar angles. They
thus define the multi-scale image edges.

Thresholding

Although the maxima of the modulus of the discrete wavelet transform are a
good approximation of the edges in an image, even in the absence of noise often
there are many false edges as can be seen. Therefore a criterion such as
thresholding must be determined to separate the "true" edges from the "false"
edges.

A Preprocessed Edge Detection Approach

Figure 3 shows schematic diagram for the reviewed preprocessed edge

detection method. In this method the image is processed and the resulting one is
applied to standard edge detection method for the objective to enhance the final
edge detection method performance.

The reviewed preprocessing involves computation of the histogram, finding

out the total number of peaks and suppressing irrelevant peaks. From the intensity
values corresponding to peaks, threshold values are obtained. From these threshold
values, optimal thresholds are calculated using the Otsu method for multilevel
image thresholds [16]. This preprocessing can be implemented in the following
steps demonstrated by an image example:

1. Filter the original image Figure 4 using a Gaussian filter as shown in

Figure 5.

2. Convert the image to grayscale as shown in Figure 6.

3. Use an averaging mask over the image and resize the image. To limit
computation time, resize the image to have 256 rows and columns that are
proportion to it i.e. the original aspect ratio is maintained as shown in
Figure 7.
4. A histogram of the image is generated. Thus, the result is of the form
[intensity level, number of pixels with that intensity level] as shown in
Figure 8.

Figure 3. Reviewed technique for edge detection.

Figure 4. Original image.

Figure 5. Gaussian filtered image.

Figure 6. Gray image.

Figure 7. Resized image.

 Figure 8. Image histogram.

5. From the histogram data, created a set of values from which the local peaks
are counted 58 elements.

6. From the above set, created a set where the peaks that are within 1 % of the
maximum value (of the number of pixels corresponding to the same
intensity level) are counted and others are discarded 57 elements. This factor
has been statistically obtained from assessing many images [17].

7. From the set created in the previous step, remove the peaks that correspond
to intensities that are within 20 levels of each other 7 elements. The number
20 is again found effective after experiments by [17]. The number of peaks
which are thresholds that can consider from the image. They represent the
significant peaks in the histograms shown in Figure 9.

8. Using the threshold values above, find out the optimal threshold using Otsu's
method; Thresh = [53 74 98 128162190 218].

9. Based in the thresholds obtained from Otsu's method, segment the resized
image as shown in Figure 10.
 Simulation Results

This section presents a comparison between different edge detection models.

For this purpose MATLAB 8.1 (R2013a) program has been adopted to investigate
the difference between edge detection models. Namely, BSD (Berkeley
Segmentation Dataset) images [18] and respective ground truths are used for
experimentation. It has been found that the best model is wavelet edge detection
model. This comparison between Roberts, Prewitt, LoG, Canny and Wavelet edge
detection methods is done using ground truth of images. The performance
parameters used are Performance Ratio (PR), Peak Signal to Noise Ratio (PSNR),
False Alarm Count (FAC), Miss Count (MSC), and Figure of Merit (FoM). PR is
defined as the ratio of true to false edges [19].

True Edges ( Edge pixels identified as Edges )

PR= ×100 (24)
False Edges + Edge pixels identified as non − edge pixels

The FoM is defined by

IA
1 1
FoM = ∑ ❑
max ( I A , I I ) i=1 1+d i ∝2
(25)

where, false edges are the non-edge pixels identified as edges, I A is the
detected edges, I I is the ideal edges, d is the distance between actual and ideal
edges and ∝ is the penalty factor for displaced edges. (PR, PSNR, FoM), higher the
value of these parameters better the result is. (MSC, FAC), lower the value of these
parameters better the result is. Table 2 summarizes the performance measures of
standard edge detection methods mentioned in this paper with and without
preprocessing approach.

As shown in Table 2, the considered edge detection methods provide better

performance measures than the standard ones. However, the considered edge
detection methods take more computation time due to the preprocessing. Figure 11
shows the original image used in simulation and its ground truth and the output of
standard and considered methods mentioned in Table 2.
 53 977
78 550
105 697
126 244
193 970
227 167
254 159

Figure 9. Otsu threshold.

Figure 10. Otsu image.

(a) (b)

(c) (d)

(e) (f)

(g) (h)

(i) (j)
(k) (l)
Figure 11. Comparison between various edge detection models (a) Original image;
(b) Ground truth; (c) Robert; (d) Reviewed by Robert; (e) Prewitt; (f) Reviewed by
Prewitt; (g) LoG; (h) Reviewed by LoG; (i) Canny; (j) Reviewed by Canny; (k)
Wavelet; (l) Reviewed by wavelet.

Table 2. Comparison between the reviewed by considered and standard edge

detection methods

Edge Method PR PSNR db FoM FAC MSC

Detection
Standard 9.8844 17.2154 0.083204 1482 3934
Roberts
Considered 9.9116 17.2153 0.086603 1478 3926
Standard 9.5487 17.2104 0.094635 2004 3922
Prewitt
Considered 9.6984 17.2106 0.095899 1954 3921
Standard 11.6908 17.2118 0.12455 4792 3952
Zero Cross
(LoG) Considered 11.8603 17.2120 0.12633 5086 3960
Standard 17.1933 17.2235 0.11672 4948 3969
Canny
Considered 17.2174 17.2240 0.11936 6116 3957
Standard 63.5240 17.2272 0.047461 0 2024
Wavelet
Considered 63.5461 17.2272 0.048007 0 2012

The considered approach consists of computation of the image histogram,

finding out the total number of peaks and suppressing irrelevant peaks. From the
intensity values corresponding to relevant peaks, threshold values are obtained.
From these threshold values, optimal multilevel thresholds are calculated using the
Otsu method, then multilevel image segmentation is carried out. Finally, a standard
edge detection method can be applied to the resultant segmented image. The
considered approach has been summarized in steps and demonstrated by an image
example. Simulation results have shown that the considered approach is used with
standard edge detection methods to enhance their performance. Results have also
shown that wavelet edge detection with the considered approach provides the
superior performance among other standard methods with and without
preprocessing approach.
 Object detection methods techniques

Introduction and Background

Object detection is a fundamental problem in computer vision, with

numerous applications spanning fields such as surveillance, robotics, autonomous
vehicles, augmented reality, and human-computer interaction. The primary goal of
object detection is to recognize and localize instances of objects belonging to
predefined classes in images or videos. In recent years, significant progress has
been made in the development of object detection algorithms, mainly due to the
emergence of deep learning and Convolutional Neural Networks (CNNs). These
advancements have led to impressive performance improvements in various
benchmark datasets, such as PASCAL VOC, ImageNet, and MS COCO. Despite
these successes, object detection in unconstrained environments remains a
challenging task. Unconstrained environments are characterized by variations in
lighting conditions, viewpoint changes, occlusions, object deformations, scale
changes, and the presence of cluttered backgrounds. These factors can severely
affect the performance of object detection algorithms, making it difficult to achieve
high detection accuracy and robustness.

In recent years, significant progress has been made in object detection,

particularly in the area of deep learning and Convolutional Neural Networks
(CNNs) [20]. These techniques have significantly improved the performance of
object detection algorithms, particularly in unconstrained environments where
objects may appear at different scales, angles, and orientations. Region-based
object detectors, such as Region-based Convolutional Neural Networks (R-CNN)
[21], operate by first generating region proposals using a selective search
algorithm, which generates around 2000 regions per image. Each region is then
passed through a CNN to generate a fixed-length feature vector, which is fed into a
support vector machine (SVM) [22] to classify the region and predict its bounding
box coordinates. Finally, non-maximum suppression is applied to eliminate
redundant detections. While R-CNN was a significant breakthrough in object
detection, it has several limitations, such as slow training and inference times.

To address these issues, researchers have seen several variants of R-CNN,

such as Fast R- CNN [23], which shares convolutional features across region
proposals, and Faster R-CNN , which introduces a Region Proposal Network
(RPN) to generate region proposals in an end-to-end manner. These variants
significantly improve the speed and accuracy of R-CNN, making it a popular
choice for object detection in unconstrained environments.

Challenges in Object Detection in Unconstrained Environments

Illumination Changes

Variations in lighting conditions, such as shadows and overexposure, can

significantly impact the appearance of objects, making it difficult for detection
algorithms to identify and localize them accurately. Variations in lighting
conditions, such as shadows, overexposure, or underexposure, can significantly
impact the appearance of objects in images [24]. These changes can make it
difficult for detection algorithms to identify and localize objects accurately. To
address this issue, several approaches have been considired, including color
constancy techniques [25] and deep learning-based methods that can learn
illumination invariant features [26].

Viewpoint Variation

Changes in the viewpoint or camera angle can alter the object's appearance,
causing the detection algorithm to fail in recognizing the object or produce
inaccurate bounding boxes [27]. Several methods have been considired to tackle
this issue, such as viewpoint invariant features and multi- view object detectors
[27].

Occlusion
 Objects in the scene may be partially or entirely occluded by other objects,
making it challenging for the detection algorithm to identify and localize them
correctly [28]. To address occlusion, some methods employ part-based models [29]
or leverage context information from surrounding regions.

Datasets

Object detection is a vital task in computer vision that involves identifying

the presence and location of objects in an image or video. To evaluate the
performance of object detection techniques in unconstrained environments, several
benchmark datasets have been created. These datasets provide a standardized set of
images with labeled objects, enabling researchers to compare the accuracy and
speed of different algorithms. Some popular datasets include:

Pascal VOC

The PASCAL VOC (Visual Object Classes) dataset is one of the oldest and
most popular datasets for object detection. It contains 17,125 images with 20
object classes, such as person, car, and dog. The dataset provides bounding box
annotations for each object in the image. PASCAL VOC has been used as a
benchmark dataset for several years, and many state-of-the-art object detection
techniques have been evaluated on this dataset.

ImageNet

The ImageNet dataset is a massive dataset that contains 1.2 million images
with 1,000 object classes. Unlike PASCAL VOC, ImageNet does not provide
annotations for object detection. However, many researchers have used this dataset
to pre-train their models on a large amount of data before fine-tuning them on
smaller object detection datasets.

COCO

The COCO (Common Objects in Context) dataset is a newer dataset that

contains 330,000 images with 80 object classes. COCO provides more detailed
annotations than PASCAL VOC, including segmentation masks for each object in
the image. This makes COCO a more challenging dataset for object detection
algorithms to perform well on.

Open Images

The Open Images dataset is another large-scale dataset that contains 1.7
million images with 600 object classes. It provides both bounding box and
segmentation mask annotations and has been used as a benchmark for object
detection algorithms that require large amounts of training data.

These datasets vary in size, number of classes, and annotation types,

allowing researchers to test their algorithms on a wide range of scenarios. The
following table summarizes some key information about the four popular
benchmark datasets used for evaluating object detection techniques:

Table 3. Summary of key information about benchmark datasets for object

detection

Number of
Dataset Name Number of Classes Annotation Type
Images

PASCAL VOC[30] 17,125 20 Bounding Boxes

ImageNet [31] 1.2 million 1,000 Bounding Boxes

COCO [32] 330,000 80 Bounding Boxes

Open Images [33] 1.7 million 600 Mask RCNN

State-of-the-art Object Detection Techniques

The state-of-the-art object detection techniques are categorized by us into

two main groups: two-stage detectors and single
 Figure 12. Milestones of object detection [34].

Two-stage detectors

Two-stage detectors consist of a region proposal stage followed by a

classification stage. Some prominent two-stage detectors include:

R-CNN

R-CNN (Region-based Convolutional Neural Networks) is an object detection model

that was proposed in 2014 by Ross Girshick et al. R-CNN is a two-stage object
detection framework that uses a region proposal mechanism to generate potential
object regions in an image and then applies a convolutional neural network (CNN) to
classify and refine these regions.

Figure 13. R-CNN – Object Detection Algorithm

The R-CNN framework consists of the following steps:

 Region Proposal: The first stage of R-CNN generates potential object
regions by using a selective search algorithm that combines low-level features,
such as color and texture, with high-level cues, such as edges and corners.
Selective search generates around 2,000 region proposals for each image.

1. Feature Extraction: In the second stage, each region proposal is warped to a

fixed size and fed through a pre-trained CNN, such as Alex Net or VGG, to
extract a feature vector for that region.

2. Object Classification and Refinement: The feature vector for each region

proposal is then fed into a set of fully connected layers that perform object
classification and bounding box regression. The classification layer outputs
the probability of each region proposal containing a particular object class,
while the regression layer outputs the refined bounding box coordinates for
that object class.

Fast R-CNN

Faster R-CNN (Region-based Convolutional Neural Networks): Faster R-

CNN is a two-stage object detection model that uses a Region Proposal Network
(RPN) to generate object proposals and a Fast R-CNN network to classify and
refine the proposals. The RPN generates region proposals by sliding a small
network over the convolutional feature map and predicting abjectness scores and
bounding box offsets. Faster R-CNN is known for its accuracy and has been
widely used in object detection tasks.
 Figure 14. Faster R-CNN – Object Detection Algorithm

Single-stage detectors

Single-stage detectors directly predict object bounding boxes and class

probabilities from an image. Some popular single-stage detectors include:

YOLO

YOLO is another one-stage object detection model that predicts object class
scores and bounding box offsets directly from the entire image. YOLO divides the
image into a grid of cells and predicts the class and bounding box for each cell.
YOLO uses a single neural network to make predictions and is known for its speed
and real-time performance.

Figure 15. YOLO – Object Detection Algorithm

 SSD

The Single Shot MultiBox Detector (SSD) extends the concept of YOLO by
predicting bounding boxes and class probabilities at multiple scales, which
improves the detection of objects with varying sizes. SSD uses a feature extractor
to generate convolutional feature maps and applies a set of convolutional filters to
predict class scores and offsets for each default box. SSD is known for its speed
and efficiency and has been used in real-time object detection applications.

Figure 16. SSD – Object Detection Algorithm

Retina Net

Retina Net introduces the Focal Loss, which addresses the issue of class
imbalance by down weighting the contribution of easy examples and focusing on
hard examples during training. This results in improved detection performance,
particularly for small objects. Retina Net uses a novel focal loss function that
assigns higher weights to hard examples and reduces the effect of easy examples
during training. Retina Net also uses a Feature Pyramid Network (FPN) to handle
objects at different scales and has achieved state-of-the-art performance on several
object detection benchmarks.
 Figure 17. RetineNet – Object Detection Algorithm

Figure 18. One stage vs two stage object detection.

The below table summarizes some key features of these state-of-the-art

object detection techniques:
 Table 4. Key features of state-of-the-art object detection techniques

Training Inference Number of AP on

Technique
Time Time Parameters COCO

Faster R-CNN
Long Medium High 39.3
[35]

SSD [36] Medium Fast Low 31.2

YOLO [37] Short Very Fast Low 28.2

Retina Net [38] Long Medium High 39.1

Comparative Analysis

In this section, compared the performance of various object detection

techniques on the COCO dataset [5]. The results are summarized in Table 3.

Table 5: Comparison of object detection techniques on the COCO dataset

Method Average Precision (AP) Speed (fps)

R-CNN 53.3 0.5

Fast R-CNN 70.0 5

Faster R-CNN 73.2 7

YOLOv8 95.1 0.8

SSD 72.1 19

RetinaNet 74.8 12