[Link].

org © 2022 IJCRT | Volume 10, Issue 5 May 2022 | ISSN: 2320-2882

AUTOMATIC LICENSE PLATE

RECOGNITION USING YOLOV4 AND
TESSERACT OCR
Abhay Baheti, Jatin Meshram, Sparsh Raj and Mrs. Suvarna Pawar*
Department of Computer Science Engineering, Savitribai Phule Pune University, Sinhgad College of
Engineering, Vadgaon(Bk), Pune, Maharashtra, India

ABSTRACT
In modern times the quantity of on road vehicles is expanding very quickly. Most of the time, it is
important to verify the identity of these vehicles for authorization of the transit regulation, overseeing
parking garages. it is hard to check this colossal number of moving vehicles physically. Subsequently,
building up a precise automatic license plate recognition model (ALPR) including character recognition is
important to ease the issues mentioned above. We have developed a model based on multiple types of
license plates from different countries. The dataset of images was trained using Yolov4 which uses CNN
architectures. Character recognition was done using the Tesseract OCR after multiple image pre-
processing techniques and morphological transformations. The proposed program has obtained an
accuracy of 92% in license plate detection and 81% in character recognition.

Keywords: recognition, accuracy, detection General Terms- Automatic License plate recognition (ALPR),
tesseract-OCR, image processing, Yolov4

Cite this Article: Abhay Baheti, Jatin Meshram, Sparsh Raj and Mrs. Suvrana Pawar, Automatic License
Plate Recognition using Yolov4 and Tesseract OCR,

1. INTRODUCTION
The objective of ALPR is to separate the vehicle number from pictures of moving vehicles. ALPR
incorporates two significant steps; detecting the license plate region using bounding boxes and recognition
of the characters using image pre-processing techniques and tesseract- OCR.
The paper intends to build up another and effective ALPR approach for multiple license plates. The
proposed approach is based on deep learning to solve plate detection and recognition problems. Efficient CNN
architectures are proposed in plate detection and recognition stages. The CNN models depend on YOLO4
CNN design. The YOLO4 CNN design is altered to a Automatic License Plate Recognition using Yolov4
and Tesseract OCR

shallow CNN design to distinguish and perceive little items (characters of tag), the quantity of layers is little
thought about with YOLOv4, which thus diminishes the running time. YOLO is short for You Only Look
Once. It is a real-time object recognition system that can recognize multiple objects in a single frame. YOLO
recognizes objects more precisely and faster than other recognition systems. It can predict up to 9000 classes
and even unseen classes. The real- time recognition system will recognize multiple objects from an image
and also make a boundary box around the object. It can be easily trained and deployed in a production
system. YOLO is based on a single Convolutional Neural Network (CNN). The CNN divides an image into
regions and then it predicts the boundary boxes and probabilities for each region. It simultaneously predicts
IJCRT2205389 International Journal of Creative Research Thoughts (IJCRT) [Link] d485
 [Link] © 2022 IJCRT | Volume 10, Issue 5 May 2022 | ISSN: 2320-2882
multiple bounding boxes and probabilities for those classes. YOLO sees the entire image during training and
test time so it implicitly encodes contextual information about classes as well as their appearance. Hence,
facilitating the detection of the license plate. The recognition of characters is done using the Tesseract OCR
software after image pre-processing techniques are done on the detected license plate, using Python language.

2. RELATED WORK
Computer vision and character recognition, algorithms for license plate recognition play an important role in
video analysis of the number plate image. Therefore they form the core modules in any ALPR system.
Nijhuis et al. [3] combined neural networks and fuzzy logic in recognition of car number plate for the case of
the Dutch number plates. ANN models was also used for training and detection, along the character
recognition using image pre-processing techniques and Tesseract-OCR by Antonius Herusutopo et al.[5].
Moreover, CNN architectures using YOLOv3 were implemented by Salah Alghyaline [6]. Sindh standard
number plate recognition is done by Quadri and Asif [7] where the number plate region is cornered with the
help of yellow color identification, later using smearing algorithm the plate is segmented, then the Optical
Character Recognition (OCR) is used to identify the characters. However the type of OCR algorithm used is
not mentioned and the accuracy rate is also not given. Tejas et al. [8] proposed Indian number plate detection
and recognition using techniques like Sobel edge detection, bounding box segmentation, and neural
networks for recognition .

3. PROPOSED METHODOLOGY
The method used can be divided in 3 main phases:
 Firstly we have gathered a dataset of images containing cars and their respective license plate. We have trained the
dataset using YoloV4 which is based on a single Convolutional Neural Network (CNN). The CNN divides an image
into regions and then it predicts the boundary boxes and probabilities for each region. In this case, we will train the
dataset in order to recognize the license plates and form bounding boxes around them. The weights obtained from
training the dataset is the converted to Tensorflow format for compatibility with python.
 Secondly, we have used image processing techniques, namely; grayscaling, Gaussian blur, Otsu’s thresholding and
binarization method being pre-processing techniques applied to the detected license plate region, followed by
morphological transformations and application of contours around desired characters based on the dimensions of the
characters and spatial localization. This is done using OpenCV.
 Finally, the characters are segmented and recognition is done using the Tesseract-OCR.

IJCRT2205389 International Journal of Creative Research Thoughts (IJCRT) [Link] d486

 [Link] © 2022 IJCRT | Volume 10, Issue 5 May 2022 | ISSN: 2320-2882

Figure 1. Proposed methodology Flow Diagram

3.1. Training Dataset Using Yolov4 and Detecting License Plate

Yolov4 is an object detection model. Object detection models are usually trained to look at an image and
search for a subset of object classes. These object classes are enclosed in a bounding box and their class is
identified. Yolov4 is a one-stage object detection model.
In contrast, a two stage detector uses a preliminary stage where regions of importance are detected and then
is classified to see if the object has been detected in these areas. The main upside of a one stage detector is the
speed it is able to make predictions quickly for real time use[8].

Figure 2 Structure of One-stage detector (YoloV4)[8]

Backbone
The YoloV4 backbone architecture is made up of three parts:
 Bag of freebies ; they are set of methods that only increase the cost of training or change the training strategy while
leaving the cost of inference low. Some of those methods are data augmentation, photometric distortion, geometric
distortion, mix p augmentation and Cut mix.
 Bag of specials: Bag of special methods are the set of methods which increase inference cost by a small amount but can
significantly improve the accuracy of object detection. It consists of mish activation function. Mish avoids saturation,
which generally causes training to slow down due to near-zero gradients drastically.

IJCRT2205389 International Journal of Creative Research Thoughts (IJCRT) [Link] d487

 [Link] © 2022 IJCRT | Volume 10, Issue 5 May 2022 | ISSN: 2320-2882
Automatic License Plate Recognition using Yolov4 and Tesseract OCR

 CSPDarknet53: The Cross Stage Partial architecture is derived from the DenseNet architecture which uses the previous
input and concatenates it with the current input before moving into the dense layer.

Neck (detector)
The main role of the neck is to collect feature maps from different [Link] structure of the latter will consist
of a Spatial Pyramid Pooling Layer which will allow to generate fixed size features whatever the size of our
feature maps

Figure 3. Structure for SPP layer

Head (detector)
The role of the head in the case of a one stage detector is to perform dense prediction. The dense prediction is the
final prediction which is composed of a vector containing the coordinates of the predicted bounding box
(center, height, width), the confidence score of the prediction and the label which in our case, the bounding
box will be around the license plate.

Figure 4. License plate detected in bounding box

As we see in Fig 4. the license plate has been detected with an accuracy of 92%.

Image Processing and Segmentation

The next phase after training the dataset of images and detecting the license plate is to apply pre-processing
techniques i.e. gray scaling, Gaussian smoothing, thresholding using Otsu’s method.

IJCRT2205389 International Journal of Creative Research Thoughts (IJCRT) [Link] d488

[Link] © 2022 IJCRT | Volume 10, Issue 5 May 2022 | ISSN: 2320-2882
Cropping the license plate from the bounding box
First step of the process is taking the bounding box coordinates from YOLOv4 detection phase and simply
taking the sub image region within the bounds of the box.

Figure 5. Resized image of license plate

Grayscaling
The importance of grayscaling is dimension reduction for example, in RGB images there are three color
channels and has three dimensions while grayscaled images are single dimensional. Grayscaling also reduces
the complexity of processing of the image On the other hand, the same neural network will need only 100 input
node for grayscaled images.

Applying Gaussian Smoothing

In Gaussian smoothing, every point of the input array is convolved using the Gaussian equation as it is shown in
equation 1 below. The output array is obtained by the summation of all such points.
1 −𝑥2
𝑥(𝑥) = 𝑥2𝑥2 (1)
√2𝑥𝑥2
In the case of an Image, a two-dimensional version of this function is used, which is just the product of two
one-dimensional functions. Mathematically, it can be expressed as:
2 2)
1 −(𝑥 +𝑥
𝑥(𝑥, 𝑥) = 𝑥 2𝑥2 (2)
2𝑥𝑥2
where x & y are the distances from the origin in the horizontal axis and vertical axis
respectively and the standard deviation of the Gaussian distribution is denoted by σ.[12]
That results in a decrease of computational complexity when compared to its two- dimensional counterpart

Thresholding and binarization using Otsu’s method

The image is then thresholded to white text with black background and has Otsu's method also applied. This
white text on black background helps to find contours of image.
Otsu’s binarization distinguishes the foreground from background and turns the latter black as it is effective
on bimodal images.
𝑥 2 (𝑥) = 𝑥 (𝑥)𝑥 2 (𝑥) + 𝑥 (𝑥)𝑥 2 (𝑥)
𝑥 1 1 2 2
Where

IJCRT2205389 International Journal of Creative Research Thoughts (IJCRT) [Link] d489

[Link] © 2022 IJCRT | Volume 10, Issue 5 May 2022 | ISSN: 2320-2882
Automatic License Plate Recognition using Yolov4 and Tesseract OCR

𝑥 𝑥

𝑥1 (𝑥) = ∑ 𝑥(𝑥) 𝑥1 (𝑥) = ∑ 𝑥(𝑥)

𝑥=1 𝑥=𝑥+1
𝑥 𝑥
𝑥𝑥(𝑥) 𝑥𝑥(𝑥)
𝑥1 (𝑥) = ∑ 𝑥2 (𝑥) = ∑ 𝑥 (𝑥)
𝑥 (𝑥) 1 2
𝑥=1 𝑥=𝑥+1
𝑥 𝑥(𝑥) 𝑥
𝑥 2 (𝑥) = ∑ [𝑥 − 𝑥 (𝑥)]2 𝑥 2 (𝑥) = ∑ [𝑥 − 𝑥(𝑥)
1 (𝑥)]2
1
𝑥1 (𝑥) 2 1
𝑥2 (𝑥)
𝑥=1 𝑥=𝑥+1
It finds a value of t which lies in between two peaks such that variances to both classes are minimum.
For 2 classes, minimizing the intra-class variance is equivalent to maximizing inter-class variance:
𝑥 2 (𝑥) = 𝑥 2 − 𝑥 2 (𝑥) = 𝑥 (𝑥 − 𝑥 )2 + 𝑥 (𝑥 − 𝑥 )2
𝑥 𝑥0 0 𝑥1 1 𝑥
= 𝑥0 (𝑥)𝑥1 (𝑥)[𝑥0 (𝑥) − 𝑥1 (𝑥)]2
which is expressed in terms of class probabilities 𝑥 and class means 𝑥, where the class means 𝑥0(𝑥), 𝑥1(𝑥)
and 𝑥𝑥 are:

∑𝑥−1 𝑥𝑥(𝑥)
𝑥0 𝑥=0 (𝑥) =
𝑥 0 (𝑥)
∑𝑥−1 𝑥𝑥(𝑥)
𝑥=𝑥
𝑥1(𝑥) =
𝑥1 (𝑥)
𝑥−1

𝑥𝑥 = ∑ 𝑥𝑥(𝑥)
𝑥=0

Figure 6. Image after Otsu’s binarization

Morphological Transformations
The image then undergoes dilation using OpenCV in order to make contours more visible and be picked up in
future step.

Figure 7. Image after Dilation process

Application of contours and segmentation

We now use OpenCV properties in Python to apply contours in the form of rectangular boxes around the
characters and sort them left to right.

IJCRT2205389 International Journal of Creative Research Thoughts (IJCRT) [Link] d490

 [Link] © 2022 IJCRT | Volume 10, Issue 5 May 2022 | ISSN: 2320-2882

Figure 8. Contours applied in rectangular boxes form

Figure [Link] around desired charatcters

The individual characters of the license plate number are now the only regions of interest left. We segment
each sub image and apply a bitwise_not mask to flip the image to black text on white background which
Tesseract is more accurate with.
Finally we will apply a small median blur to eliminate any remaining noise.

Figure 10. Segmented characters of the image

3.2. Recognition of character using Tesseract OCR

Pre-processing techniques are required though for the accurate use of the tesseract-OCR. It can be used to
recognize both structured and unstructured data.

Figure 11. Structure of the Tesseract-OCR

This neural network architecture implements and combines feature extraction, sequence modeling, and
transcription into a unified framework. This model does not need character segmentation. The CNN extracts
features from the input image(text detected region). The deep Automatic License Plate Recognition using Yolov4
and Tesseract OCR bidirectional recurrent neural network predicts label sequence with some relation between
the characters.

IJCRT2205389 International Journal of Creative Research Thoughts (IJCRT) [Link] d491

 [Link] © 2022 IJCRT | Volume 10, Issue 5 May 2022 | ISSN: 2320-2882

Figure 12 Image with recognized characters

4. EXPERIMENTAL RESULT AND DISCUSSION

Images trained with Yolov4 using R-CNN and an input of 8000 iterations had a validation accuracy of 98% and
an error rate of less than 1.

Figure 13. Image with more unwanted texts in the license plate
License plate Recognition : 80% Characters on license plate: V0DKAA Characters read : V0DKAA
Character recognition was: 100%
We can see here that the license plate was smaller and full of unwanted texts.

Figure 14. Slightly noisy and blurry image from dashboard

License plate Recognition : 88% Characters on license plate:KR696969 Characters read : KR696969 Character recognition
was: 100%

IJCRT2205389 International Journal of Creative Research Thoughts (IJCRT) [Link] d492

 [Link] © 2022 IJCRT | Volume 10, Issue 5 May 2022 | ISSN: 2320-2882
4.1. Result using video
We can see below a screenshot from a video where license plate recognition of both of the cars are done as they
are moving , even when they are in the same frame.

Figure 15. Result from 1 frame of a video

As the video proceeds, the license plates and the characters are detected. It depends on the frame they are at
which moment.

Table 1. Accuracy of proposed ALPR model

ALPR Model License plate Character
recognition Recognition
Ours 98% 81%
[9] 85% 80

5. CONCLUSION AND FUTURE WORK

The program that has been developed here using YoloV4 to train images has had 98% validation rate with an error
rate of less than 1. This license plate detection model enables detection and recognition of characters in different
types of environments and on multiple types of license plates. Pre-processing techniques have been used such as
gray scaling, Gaussian smoothing, Thresholding by Otsu’s method and other morphological transformations in
order to make the recognition of characters in the license plates easier. We have tested the program with further
30 samples of images and obtained 92% of accuracy in license plate detection and 81% of accuracy in detection
of characters. Our future works will be to enhance the character recognition program by training individual
characters so that the Tesseract-OCR would work more efficiently.
Automatic License Plate Recognition using Yolov4 and Tesseract OCR

IJCRT2205389 International Journal of Creative Research Thoughts (IJCRT) [Link] d493

[Link] © 2022 IJCRT | Volume 10, Issue 5 May 2022 | ISSN: 2320-2882
REFERENCES
[1] J. A. G. Nijhuis, M. H. Ter Brugge, K. A. Helmholt, J. P. W. Pluim, L. Spaanenburg, R. S.
Venema and M. A. Westenberg. (1995) “Car
[2] license plate recognition with neural networks and fuzzy logic,” Proceedings of ICNN'95 - International Conference
on Neural Networks, Perth.
[3] Herusutopo, Antonius, et al. "Recognition Design of License Plate and Car Type Using Tesseract Ocr and Emgucv."
Communication and Information Technology Journal, vol. 6, no. 2, 2012, pp. 76-84
[4] Alghyaline, Salah. (2020). Real-time Jordanian license plate recognition using deep learning. Journal of King Saud
University - Computer and Information Sciences.
[5] Muhammad Tahir Qadri and Muhammad Asif. (2009) “Automatic Number Plate Recognition System for Vehicle
Identification Using Optical Character Recognition,” 2009 International Conference on Education Technology and
Computer, Singapore.
[6] K Tejas, K Ashok Reddy, D Pradeep Reddy, K P Bharath, R Karthik and M. R. Kumar, (2018) “Efficient License
Plate Recognition System with Smarter Interpretation Through IoT,” Bansal J., Das K., Nagar A., Deep K., Ojha A. (eds)
Soft Computing for Problem Solving. Advances in Intelligent Systems and Computing, 817: 207-220.
[7] Md Yeasir Arafat, Anis Salwa Mohd Khairuddin, Uswah Khairuddin and Raveendran Paramesran, (2019)
“Systematic review on vehicular license plate recognition framework in intelligent transport systems,” IET Intelligent
Transport Systems.
[8] Bochkovskiy, Alexey & Wang, Chien-Yao & Liao, Hong-yuan. (2020). YOLOv4: Optimal Speed and Accuracy of
Object Detection.
[9] M M Shidore, and S P Narote. (2011) “Number Plate Recognition for Indian Vehicles”
International Journal of Computer Science and Network Security 11(2): 143-146.

IJCRT2205389 International Journal of Creative Research Thoughts (IJCRT) [Link] d494