2016 14th International Conference on Control, Automation, Robotics & Vision

Phuket, Thailand, 13-15th November 2016 (ICARCV 2016) Su33.6

Automated License Plate Detection

using a Support Vector Machine

Shigeharu Miyata Kenji Oka

School of Engr., Hiroshima, School of Engr., Hiroshima,
Dept. of Robotics, Kinki U., Dept. of Robotics, Kinki U.,
Higashi-Hiroshima, JAPAN Higashi-Hiroshima, JAPAN
miyata@[Link] redstar_8_boy@[Link]

Abstract— This paper proposes a new method of detecting license License plate recognition is a three step process. First, a
plates in images of vehicles where the license plate is shown, and detection process is performed to extract only the license plate
reports the detection results when this method was applied to area from a vehicle image which includes the license plate. Next,
detection of license plates on vehicles in Japan. This license plate the parts of the license plate image containing letters and
detection process detects only the edge vertical components, and numbers are extracted, and finally recognition of the letters and
the candidate license plates are narrowed down using the contours numbers in each of these areas is performed. Because the
obtained by dilation and erosion processing and region fill accuracy of license plate recognition depends on the accuracy of
processing. A SVM (Support Vector Machine) based on negative each step, improving the accuracy of the processing at any step
and positive examples is used to determine whether or not a
will improve the accuracy of license plate recognition overall.
candidate area is a license plate, and finally the position of the
license plate is identified. This study examined how the license For detection of the license plate, methods which use the
plate detection results in license plate and non-license plate images degree of similarity to a license plate based on edge information
were affected by differences in aspect ratios, differences in are widely used. Other proposed methods for identifying the
brightness between the vehicle body and license plate, and the license plate position include a method [9] of detecting the edges
number of positive and negative examples used for learning. The of the license plate holder by means of a Hough transform, a
effectiveness of this method was confirmed to yield a license plate method [10] of identifying the license plate position by detecting
detection rate of approximately 90%.
and projecting only the edge vertical components (because the
Keywords-License plate detection; Support vector machine;
license plate area contains many edge vertical components), a
Machine learning; Image processing method [11] using neural network, and a template matching
method [12] using predetermined rules for the positioning of
license plates.
I. INTRODUCTION
For extraction of the letters and numbers, proposed methods
Systems for recognizing vehicle license plates using image
include a method [13] of creating a binary image and estimating
sensors are effective in a variety of applications because they do
the character size based on advance information, a method [14]
not require installation of a special device on the vehicle. For
using grayscale template matching.
example, a travel time measurement system collects images of
the vehicles which pass by a camera installed on the road, Proposed methods for character recognition include a binary
extracting and analyzing the license plate information in order to image template matching method [13], a method [15] using
measure the traffic flow and provide information such as the cross-count and template matching. OCR technologies have also
expected time of arrival to the drivers. A variety of other been broadly applied to character recognition.
applications are also possible, including preventing
unauthorized vehicles from entering a contracted parking area, Previous methods for license plate detection mostly assume
checking for suspicious vehicles, automated guidance to the that the plate shape and character layout follow certain
correct parking area for each vehicle type in large-size parking predetermined rules, and therefore cannot be used with widely
facilities, automatic monitoring for speeding vehicles, and varying license plates. This report describes the application of a
searching for stolen vehicles. Some of these systems have machine learning method that uses a SVM (Support Vector
already been commercialized and are a widespread part of Machine) to detection of license plates, and that is capable of
everyday life. Many hardware and software improvements are detecting a wide variety of license plates. This license plate
being proposed for the systems used in these various detection process detects only the edge vertical components, and
applications in order to achieve high recognition rates and low the candidate license plates are narrowed down using the
reading error rates [1]-[8]. These include development of contours obtained by dilation and erosion processing and region
advanced imaging units and image processing boards, as well as fill processing. A SVM based on negative and positive examples
new image processing algorithms. is used to determine whether or not a candidate area is a license
plate, and finally the position of the license plate is identified.

978-1-5090-3549-6/16/$31.00 ©2016 IEEE

This study applied this method to Japanese vehicles, and Language) file. Some samples of the 50 license plate images and
examined how the license plate detection results for license plate 50 non-license plate images are shown in Fig. 1.
and non-license plate images were affected by differences in
aspect ratios, differences in brightness between the vehicle body In order to test whether or not the machine learning method
and license plate, and the number of positive and negative has an effect on detection, 5 sets of different numbers of license
examples used for learning. plate and non-license plate learning images (total 20, 40, 60, 80,
and 100 images) were created for the classifier SVM.
Section 2 describes the processing procedure and specific
processing method that were used for license plate extraction.
Section 3 presents the detection results when this method was
applied to a variety of images taken in different imaging
environments, and examines the effect of the learning method
and image characteristics on the detection rate. Section 4
presents the conclusions and remaining issues.

II. OVERVIEW OF THE LICENSE PLATE EXTRACTION

METHOD

A. Creation of the learning images

This method for detecting the license plate in vehicle images
uses a SVM that corresponds to supervised machine learning for
pattern recognition. One excellent feature of a SVM is that it
searches for a maximum-margin hyper-plane among multiple
candidate planes to separate disjoint data sets by a gap. Based on
the learning samples which it is given, it learns the boundary that
can most clearly classify the samples.
The machine learning for a classifier SVM follows the
procedure below. The size of the license plate on an Italian
vehicle is 110 mm horizontal × 520 mm vertical (aspect ratio
4.72). The size of the license plate on a mid-sized Japanese
vehicle is 165 mm horizontal × 330 mm vertical (aspect ratio
2.0), and on a compact vehicle is 125 mm horizontal × 230 mm
vertical (aspect ratio 1.84).
When creating the positive and negative example learning
Fig. 1. License plate and non-license plate learning images
images, two methods were used in order to compare the effect
from differences in the method of image creation. With the first
method, we created learning images by converting the 1.84
aspect ratio of Japanese vehicle license plates to the 4.72 aspect B. License plate detection process
ratio of Italian vehicles, an aspect ratio which is significantly
An example of the image processing for license plate
different from the Japanese one. With the second method, we
detection is described below. The color image showing the
created learning images based on the 1.84 aspect ratio of
vehicle (Fig. 2(a)) was subject to a smoothing process of size 5
Japanese vehicle license plates. The license plate learning image
pixels × 5 pixels to remove noise (Fig. 2(b)). Because the license
size for the positive examples was 144 pixels horizontal by 33
plate area contains a large number of vertical edge components
pixels vertical for the 4.72 aspect ratio, and 144 pixels horizontal
in the frame and numbers, the vertical edge components in the
by 78 pixels vertical for the 1.84 aspect ratio. For the learning
image are extracted (Fig. 2(c)). For image binarization, the
images which do not contain license plates (the negative
grayscale image is converted to a binary image based on the
examples), images of the same size as each of the two aspect
known Otsu method (Fig. 2(d)). The resulting image is then
ratios were prepared.
subjected to a dilation and erosion process to remove fine noise
In Japan, the license plate colors are different for large-size, and reduce the image to its primary areas (Fig. 2(e)). From these
mid-size, and compact vehicles. In order to eliminate differences areas, only the contours which are within the predetermined
between vehicle types due to color, both the license plate and allowable aspect ratio range are extracted (Fig. 2(f)). In this
non-license plate learning images were converted from color example, 4 contours were obtained. If the 3-channel BGR
images to grayscale images. brightness value of the pixels near the center coordinate in each
contour is within the predetermined tolerance range, then the
The number of prepared learning images was 50 for license area is considered a connected component. For each area which
plate images and 50 for non-license plate images. So that it could is identified as a connected component, a surrounding mask
be used in SVM recognition, the pixel values and information image composed of the minimum containing rectangle is created
identifying the images as positive or negative examples for all of (Fig. 2(g)). The mask images which are within the allowable
the 100 images were stored in an XML (Extensible Markup range of aspect ratios for license plates are identified as license
plate candidate areas (Fig. 2(h)). The SVM is then applied C. Effects of body color on detection
individually to each of these candidate areas to determine For the detection process explained in Section II.B, it was
whether or not it is a license plate (Fig. 2(i)). Sample results from found that when setting the allowable brightness value range,
application of the above method to test images (different from finding the connected components, and creating the mask
the learning images) are shown in Fig. 3. The results show that images, it was not possible in some cases to correctly detect the
the license plate area was correctly detected. license plate unless the allowable range was adjusted
appropriately when the body color was near-white and when it
was a darker color such as gray. The following is an example.
When the allowable range for brightness values was set to 30
for finding connected components in order to create the mask
images, the license plate was not detected for a white vehicle as
shown in Fig. 4. However when the allowable brightness range
was set to 5, it became possible to detect the license plate for the
same vehicle as shown in Fig. 5. When the allowable brightness
(a) (b)㻌 (c) range was left at 5 and processing was performed of a gray
vehicle, the license plate was not detected as shown in Fig. 6.
The above shows that there is room for improvement by
automatically determining the difference in brightness between
the body color and license plate, or the brightness of the loaded
image or similar characteristic, at the stage when the mask
images are created in order to eliminate false detection and failed
detection.
(d)㻌 (e)㻌 (f)

(g)㻌

Fig. 4. License plate was not detected when allowable brightness range was set
to 30.

(h)㻌 (i)㻌
Fig. 2. (a) vehicle color image, (b) smoothed image, (c) extraction of vertical
edges, (d) binalized image, (e) image after dilation and erosion processing,
(f) extracted profiles within the allowable aspect ratio range, (g) mask
images, (h) license plate candidate areas, (i) detected license plate.

Fig. 5. License plate was detected in same vehicle when allowable brightness
range was set to 5.

Fig. 3. Detection result for license plate in test image.

Fig. 6. License plate was not detected on a gray vehicle when same processing
was performed with allowable brightness range left at 5.
 III. LICENSE PLATE DETECTION RESULTS TABLE 1. Detection rate for each of the 10 test images, and the average detection
rate, for each number of learning images when the learning images
This section describes the detection rate results that were were set to the Italian license plate aspect of 4.72.
obtained using the license plate detection method described
above with 10 test images. 10 test images were prepared, and Number of learning images
processing was performed 30 times for each test image. Image 20 40 60 80 100
Detection was considered successful when only the license plate No.
was detected, and the detection rate for each test image was 1 80 90 90 96 96
calculated. Calculation of the detection rate was performed for 2 0 36 93 60 80
each of the 5 classifier SVM that were described in Section II.A, 3 15 96 100 100 100
and the average detection rate of the 10 test images was 4 100 100 100 100 100
calculated for each classifier SVM. This result was considered
5 100 100 100 100 100
the detection rate for that classifier SVM.
6 15 100 100 100 100
As explained regarding the creation of learning images in 7 45 96 100 100 100
Section II.A, two types of learning images were created: images 8 45 90 83 96 96
with the Italian vehicle aspect ratio and images with the Japanese 9 65 30 50 73 86
vehicle aspect ratio. This was done in order to investigate the 10 30 90 100 100 100
effect on the license plate detection rate of the different license
Average 50 83 92 93 96
plate shapes in different countries. The detection rate results for
each learning image type are as shown below. value

• Case (1): The aspect ratio of Japanese vehicle license

plates was converted to the aspect ratio of Italian
vehicles when the learning images were created. 100 91.6 92.5 95.8
DETECTION RATE (%)

Details of the success rate are shown in TABLE 1. 82.8

Changes in the detection rate for each classifier are 80
shown in Fig. 7.
60 49.5
• Case (2): The aspect ratio of Japanese license plates was
left unchanged when the learning images were created. 40
Details of the success rate are shown in TABLE 2.
Changes in the detection rate for each classifier are 20
shown in Fig. 8.
0
Comparison of Case (1) and Case (2) shows that when the 20 40 60 80 100
number of images used in learning is small, then in order to
obtain a stable and high detection rate, the learning image aspect NUMBER OF LEARNING IMAGES
ratio should be approximately the same as the license plate
aspect ratio in that country. This is because when the allowable
Fig. 7. Average detection rate for each number of learning images when the
aspect ratio range was set to +/-60%, the allowable aspect ratio learning images were set to the Italian vehicle license plate aspect ratio of
range for Japanese vehicles with the 1.84 aspect ratio (Japanese 4.72.
vehicle aspect ratio) was 0.74 - 2.56, however the allowable
TABLE 2. Detection rate for each of the 10 test images, and the average detection
aspect ratio for Italian vehicles with the 4.73 aspect ratio (Italian rate, for each number of learning images when the learning images
vehicle aspect ratio) was 1.89 - 7.57, and the results using the were set to the Japanese license plate aspect of 1.84.
Japanese vehicle aspect ratio excluded many candidate areas
which were not the license plate, resulting in stable detection Number of learning images
even when the number of learning images was small. In Case Image 20 40 60 80 100
(2), increasing the number of learning images showed little No.
effect on the detection success rate. The areas which were false 1 58 93 90 100 100
detections were mostly areas of the background and not parts of 2 100 100 100 100 100
the vehicle. This is thought to be due to the fact that many of the
3 62 76 83 86 76
non-license plate images used at the learning stage were images
of parts of the vehicle and because no images of background 4 60 76 66 76 83
areas at all were used in this experiment. 5 70 100 86 96 96
6 98 90 93 96 86
7 100 100 96 100 100
8 100 100 100 100 100
9 100 93 73 80 83
10 52 83 56 90 60
Average 80 91 84 92 88
value
 ACKNOWLEDGMENT
100 91.1 92.4 88.4 We would like to thank Miss Naomi Somyo and Miss Yuki
84.3
DETECTION RATE (%)

80.0 Nakaya who provided great support for this study.

80

60
REFERENCES
40 [1] Yasuo Ogiuchi, Masakatsu Higashikubo, Sakda Panwai, Elarin
Luenagvilai, “Automatic license plate detection and recognition in
20 Thailand,” SEI technical review, 184:p.34-39, 2014.
[2] Shan Du, Mahmoud Ibrahim, Mohamed Shehata, Wael Badawy.
0 “Automatic license plate recognition: a state-of-the-art review,” IEEE
transactions on circuits and systems for video technology, 23:2: p.311-
20 40 60 80 100 325, 2013.
NUMBER OF LEARNING IMAGES [3] Rinku Solanki, Naitik Kapadia, Viral Patel, “The review on automatic
license plate recognition,” Int. Jour. of research in engineering and
Fig. 8. Average detection rate for each number of learning images when the technology, 2:2: p.178-183, 2013.
learning images were set to the Japanese vehicle license plate aspect ratio [4] Wengang Zhou, Houqiang Li, Yijuan Lu, Qi Tian, “Principal visual word
of 1.84. discovery for automatic license plate detection,” IEEE transactions on
image processing, 21:9: p.4269-4279, 2012.
IV. CONCLUSION [5] Bharat Raju Dandu, Abhinav Chopra, “Vehicular number plate
recognition using edge detection and characteristic analysis of national
This paper proposed a method of using machine learning for number plates,” Int. Jour. of computational engineering research, 2:3:
license plate detection. It is a method which is not dependent on p.795-799, 2012.
license plate shapes and character layout information, as [6] Yasuo Ogiuchi, Masakatsu Higashikubo, “Detection method for various
previous methods are, and which can be used with a wide variety license plates,” SEI technical review, 181:p.95-98, 2012.
of license plates. We applied this method to Japanese vehicles, [7] Takehiko Kato, Kayo Tanaka, Masatoshi Asada, Hiroyuki Yasuhara,
and identified the effects that differences in the image features Kanae Tachiki, Michihiro Kodama, Yusuke Yasuhara, Toshihiro Asai,
Yasuo Ogiuchi, “Development of automatic license plate recognition
during machine learning have on license plate detection. device,” SEI technical review, 181:p.99-103, 2012.
Regarding the number of positive example (license plate) [8] Halina Kwasnicka, Bartosz Wawrzyniak, “License plate localization and
and negative example (non-license plate) images used during recognition in camera pictures,” Symposium on methods of artificial
intelligence (AI-METH 2002), November, 2002.
machine learning, the results were the following. When the
[9] Takeshi Agui, Hyung Jin Choi, Masayuki Nakajima, “An extracting
images were created using the Italian vehicle aspect ratio, the method of car number plate by image processing (in Japanese),” IEICE
resulting detection rate for Japanese vehicle license plates Trans. D-II, 70:3: p.560-565, 1987.
improved when the number of images used during learning was [10] Yuji Sawada, Mitsuaki Nakamura, Tomoaki Sumida, Masamichi
larger. However when the Japanese license plate aspect ratio was Kawanaka, Hisashi Saiga, Yoshihiro Kitamura, “Development of license
used, the detection results for Japanese vehicle license plates plate recognition system (in Japanese),” Technical report of IEICE,
showed little dependence on the number of images used during PRMU96-46, 96:46: p.65-72, 1996.
learning, and good detection results were achieved in all cases. [11] Hironobu Fujiyoshi, Taizo Umezaki, Tomohiko Imamura, Takeo Kanade,
“Area extraction of the license plate using artificial neural network (in
In this case, most of the areas which were falsely detected as Japanese),” IEICE Trans. D-II, 80:6: p.1627-1634, 1997.
license plates were in parts of the background and not parts of
[12] Katsuyoshi Tanabe, Harumi Kawashima, Eisaku Marubayashi, Tadashi
the vehicle. As a result, we conclude that it is necessary to Nakanishi, Akio Shio, Sakuichi Ohtsuka, “Car License plate extraction
include background area images in the non-license plate learning based on character alignment model (in Japanese),” IEICE Trans. D-II,
images. Based on the above, we have confirmed that increasing 81:10: p.2280-2287, 1998.
the number of images used in machine learning is a technique [13] Takashi Naito, Toshihiko Tsukada, Kei-ichi Yamada, Shin Yamamoto,
which can produce a high detection rate for the license plates of “License plate recognition method for passing vehicles with robust
Japanese vehicles without being dependent on advance sensing device against varied illumination condition (in Japanese),”
IEICE Trans. D-II, 81:9: p.2019-2026, 1998.
information regarding the license plate shape (in this report, the
[14] Yasuko Takahashi, Hisako Tanaka, Akira Suzuki, Akio Shio, Sakuichi
aspect ratio). Ohtsuka, “License plate recognition using a gray image template
It was also found that the detection rate is affected by the matching with noise reduction filters and character alignment (in
Japanese), IEICE Trans. D-II, 87:7: p.1451-1461, 2004.
difference in brightness between the vehicle body and license
[15] Yosuke Shinohara, Nobuo Funabiki, Jun Kawashima, Junichi Takeuchi,
plate at the stage when the detection process mask images are Masayuki Ishizaki, “A license plate recognition algorithm using cross
created. In order to further improve the detection rate, it will be count and template matching methods (in Japanese),” Technical report of
necessary to automatically determine the vehicle color, IEICE, PRMU2004-170, 2004:170: p.39-44, 2005.
brightness, and other features, and to incorporate them in the
processing.