We propose a novel method to precisely detect car license plates by locating main characters, which are printed with large font size. The regions of the main characters are directly detected without detecting the plate region boundaries, so that license regions can be detected more precisely than by other existing methods. To generate a binary image, multiple thresholds are applied, and segmented regions are selected from multiple binarized images by a criterion of size and compactness. We do not employ any character matching methods, so that many candidates for main character groups are detected; thus, we use a neural network to reject non-main character groups from the candidates. The relation of the character regions and the intensity statistics are used as the input to the neural network for classification. The detection performance has been investigated on real images captured under various illumination conditions for 1000 vehicles. 980 plates were correctly detected, and almost all non-detected plates were so stained that their characters could not be isolated for character recognition. In addition, the processing time is fast enough for a commercial automatic license plate recognition system. Therefore, the proposed method can be used for recognition systems with high performance and fast processing.
Automatic license plate recognition (ALPR) is used in various applications for vehicle identification, e.g., traffic enforcement, parking management and electronic toll collection,in intelligent transportation systems (ITS)
. In ALPR systems, car images are acquired from a camera,and the images are processed by various image processing and optical pattern recognition methods
. Commonly, an ALPR system consists of three main processes: detection of plate regions, character isolation and character recognition.Prior to the character recognition step, the license plate (LP) regions must be detected and the character regions must be isolated. Since overall performance and processing speed are influenced by these tasks, they are the most significant step in ALPR systems
In the literature, there are a large number of methods to detect LPs, where features such as texture, frame edges and color are used
. The texture-based methods use distinctive gray levels such as local variances
, gradient density
. These methods show high detection performance, but many non plate regions are detected. To overcome this drawback, combined methods with morphological steps were proposed in
The methods using frame edges
detect frame boundaries by edge or line information. These methods may detect frame regions correctly; however the plates which have boundaries with low contrast cannot be detected.
The color-based methods
use a unique color or color sequence of the plate regions. These colors vary greatly due to different illumination conditions, and vehicles may have a color similar to that of the LPs. Thus, color information cannot be directly employed to detect LP regions.
In most applications, character isolation is performed by projections
or connected region analyses
of a binarized LP image after the LP detection. So, even if LPs were correctly detected, characters may not be isolated.
To develop the best appropriate LP detection method for ALPR, we propose a novel method which can detect LP regions precisely. In the proposed method, we detect LP regions by locating main characters (MCs), which are printed with large font size. To detect MCs, we binarize an input image with multiple thresholds by a criterion of size and compactness. MCs are, then, grouped as candidates, and a neural network using the relation of the MCs and the intensity statistics is employed to reject non LP regions.Therefore, our method can detect LP regions precisely, and is the most appropriate method for character recognition because MC regions are isolated in the detection process.
II. ORGANIZATION OF PROPOSED METHOD
The overall scheme of our method is shown in
.We first apply a DC notch filter, which compensates the variation of illumination, as shown in
. The DC notch filtering is calculated by
are the original and the filtered images,respectively, and
is the local DC (direct current, average intensity) component at (
). The size of the window that calculates the DC may be somewhat larger than the largest character expected to be detected
MC regions are detected by selecting from isolated binary regions by multiple thresholds and shape information.Then, we group the MC regions into main character group(MCG) candidates, where an MCG is a group of MCs on a LP. MCGs are detected without any character matching,so that many MCG candidates are detected. Characters of MCG, however, are arranged with predefined regulation,so we use a neural network to reject non MCGs from candidates, where the relation of the character regions and
Overall scheme of proposed method.
the intensity statistics are used as input features of the neural network.
III. DETECTION OF MCG CANDIDATES
Vehicle images, which contain LPs, are acquired in various places at various times; thus, a simple threshold cannot segment character regions correctly. For this reason, we apply multiple thresholds, and then select character regions.
To detect MCGs, we first segment isolated MC candidates as shown in
. Binary images are generated using multiple thresholds, and we select character regions, which are the largest regions less than a limited size for detecting characters, from among the binary images. In addition,we use a compactness to reject non character regions. The process proceeds according to the following steps.
The characters may be observed with frames of LPs because of the camera angle. Thus, we delete long horizontal lines connected pixels whose length is greater than
in step 3 before labeling. In step 5, the compactness(
) is defined as the ratio of the character regions to the bounding box regions. The compactness is calculated by
Step 1: Initialize threshold levelsT,T, …,T[n]associated with higher percentages of the intensity histogram of the image.
Step 2: Binarize the image withT,T, …,T[n].
Step 3: (For each kth binary image) Delete long horizontal lines.
Step 4: (For each kth binary image) Perform the image labeling on the binary image. Then store the bounding boxes of connected regions.
Step 5: (For each kth binary image) Select the largest regions satisfying the size of a MC. If the selected region has a lower compactness than the region which is constructed by the lower threshold level (k?1) and included in the selected regions, select the region constructed byk?1 instead of the selected region.
Step 6: Generate a binary image from the labels selected from all binary images at Step 5.
Step 7: Label the binary image and then store the bounding boxes in bright character regions (Cib).
Step 8: For threshold levels associated with lower percentages of the histogram, repeat step 1 to step 7, and store dark character regions (Cib).
is the number of pixels in the segmented regions, and
is the area of the bounding box enclosing the segmented region. The label merged with the boundary of the plate has a low compactness, so it is rejected by comparing to the compactness of the included region constructed in the lower threshold level.
(c) to (h), the gray color shows the regions deleted as long horizontal lines, and the regions surrounded by dotted boxes show the rejected regions as non character regions because they have the compactness lower than the compactness of the included region constructed in the lower threshold levels.
shows an example for detection of MC candidates.
(d) are the generated binary images with higher and lower percentages of the histogram, respectively.
In common LPs including those of Korea, the MCG is four isodistant numbers. Thus, we detect MCGs by grouping four horizontally adjacent MCs as follows:
Step 1: For all bright character regions (Cib), extend group regions toECibsuch as
The examples of detection of MCG candidates are shown in
does not have three regions to be included in
, so the MCG cannot be created; however, three MCGs
whereHECibandWECibdenote the height and the width ofECib, respectively, andHCibis the height ofCib.ECibis aligned left of the vertical middleCibposition.
Step 2: Detect the three nearest character regions included inECib, and group the union of the three rectangle regions andCibas a kth MCG candidate for bright characters (Gkb).
Step 3: For all dark character regions (Cid), repeat step 1 and step 2, and create akth MCG candidate for dark characters (Gkd).
Multiple thresholds: (a) original image (b) binaryimage and detected MC regions (c) (d) (e) (f) (g) and (h)binary images by thresholds associated with higher 1.0%2.5% 5.0% 7.5% 10.0% and 20.0% of the intensityhistogram respectively.
Detection of MC candidates: (a) original image (b)filtered image (c) and (d) generated binary images withhigher and lower percentages of the intensity histogramrespectively.
Detection of MCG candidates.
Detected MCG candidates of Fig. 3(a).
are created by
A number of MCG candidates may be detected by many horizontally adjacent regions as shown in
. For this reason, we apply a neural network to reject non MCGs.
IV. MCG CLASSIFICATION BY NEURAL NETWORK
Although the LP is oriented with small angles in the image, the horizontal distance between neighbor characters is seen to have regular arrangement. Therefore, we extract the ratios of the horizontal distance between adjacent characters and the width of the MCG, and they are used as the input features of classification and are calculated by
where W denotes the width of the MCG, and
denotes the horizontal distances between adjacent characters, as shown in
(a). Because the vertical positions of the characters have similar characteristics, we use also three ratios (
) of the vertical distances between adjacent characters and the height of the MCG as the input features of a neural network. The ratios are calculated by
denotes the height of the MCG, and
denotes the vertical distances between adjacent characters.
The intensities of the background and the character regions are different, and both the background and the character regions are characterized by uniform intensity, as shown in
(b). Therefore, we extract the average intensity (
)and the standard deviation (
) of the background region,and the average intensity (
) and the standard deviation(
) of the character region. The type of the MCG (1 for
and 0 for
) is used as the feature (
) to determine whether MGs are bright or dark.
Features for classification.
Classification of MCGs.
By using extracted input features, we employ a neural network to reject non MCGs, as shown in
, where the input nodes are the features, and the output nodes classify whether the region is a MCG or not.
V. EXPERIMENTAL RESULTS
Our method was tested on 1000 vehicle images taken in Korea. Four hundred forty-five images were taken as training images: 115 LP images and 330 non LP images, where many non LP images are needed for high performance,because non LP images which are similar to LP images are very varied. The original image size is 1024×1024. Then the images are down sized to 512×512 and the upper region is truncated, so the final image has a size of 512×256 when it is processed.
shows an example of the results.
(a) shows non MCGs rejected by a neural network,and
(b) shows a detected MCG.
shows correctly detected MCGs with whole plate regions for various types of LPs.
In the experiments, various types of LP regions with poor quality were correctly detected as shown in
.Out of the 5664 candidates detected, 4650 candidates were rejected, and 34 non MCGs were not rejected (false posi
MCG classification of Fig. 5: (a) non MCGs and (b)detected MCG.
Correctly detected MCGs.
Detected MCG images.
tive rate: 0.6%). Out of the 987 MCGs detected, 980 MCGs were correctly detected (detection rate: 98.0%), and 7 false rejections were observed (true negative rate: 0.1%). The rejected non MCGs are shown in
. The images which have undetected MCGs have poor image quality, as shown in
. The number of false detected MCGs is 34. The false detected MCGs are exemplified in
. These regions may be rejected by detecting whole license characters.
shows MCGs falsely rejected by the neural network. Character isolation and recognition can be correctly performed by additional processes for these images; however,overall performance may worsen because the additional
Rejected MCG candidate images.
Non-detected MCG images.
False detected MCG images.
False rejected MCG images.
processes may influence clean images. Although various types of plates are being used in Korea as shown in
, high detection performance was achieved because the employed neural network nonlinearly classifies the feature space.
On a Pentium PC with 2.4 GHz, the average processing time of a single image was about 57.5 ms. This is fast enough to realize a commercial ALPR system.
There are many methods to detect LPs; however, their performance rates are often overestimated for promotional reasons. In the literature, the detection rates are often introduced as shown in
, but the data sets are limited with small quantities and restricted image qualities.For this reason, it is very difficult to objectively compare performance. Although the detection rate of the proposed method is less than the results of some methods, the proposed method has advantages over other methods as follows; stained LP regions which cannot be processed for character isolation were not detected, and MC regions are located in LP detection. Therefore, our method may be robust and precise.
Comparison of detection performance.
Comparison of detection performance.
In this paper, we proposed a novel method which detects car LPs by locating MCGs. For considering various operating conditions, multiple thresholds are used to detect MCs.To reject non MCGs for candidates, the relation of the character regions and the intensity statistics are applied to input nodes of a neural network.
If characters cannot be segmented, it is impossible to recognize a LP even though the LP is detected. Character isolation is the most important part of ALPR. Common ALPR systems employ both detection of LPs and character isolation. However, we directly detect MCGs by detecting LP regions. The detection rate is 98.0%; however, almost all non-detected plates are so stained that their characters cannot be isolated for character recognition, and the average processing time is about 57.5 ms. Therefore, the proposed method can be applied to ALPR systems with high performance and fast processing time. Moreover, when the number of grouping characters and training data are modified, this method is appropriate for multinational LP detection.
Park S. W
Hwang S. C
Park J. W
The extraction of vehicle license plate region using edge directional properties of wavelet subband
IEICE Trans. Inf. & Syst.
Anagnostopoulos C.-N. E
Anagnostrpoulos I. E
Psoroulas I. D
License plate recognition from still images and video sequences: a survey
IEEE Trans. Intell. Transp. Syst.
DOI : 10.1109/TITS.2008.922938
Automatic license plate recognition: a review
Proc. Int. Conf. Imaging ScienceSystems and Technology
Las Vegas USA
Gao D. S
Car license plates detection form complex scene
Proc. 5th Int. Conf. Signal Processing
Real-time license plate detection under various conditions
Lecture Notes in Computer Science
DOI : 10.1007/11833529_20
Adaptive license plate image extraction
Proc.5th Int. Conf. Computer Systems and Technologies
Claudio E. D. D
Car plate recognition by neural networks and image processing
Proc. IEEE Int. Symp. Circuits and Systems
DOI : 10.1109/ISCAS.1998.703970
Alba J. L
New methods for automatic reading of VLP’s (Vehicle License Plates)
Proc. IASTED Int. Conf. SPPRA
A hybrid license plate extraction method based on edge statistics and morphology
Proc. Int. Conf. Pattern Recognition
DOI : 10.1109/ICPR.2004.44
Kasaei S. H. M
Kasaei S. M. M
Monadjemi S. A
A novel morphological method for detection and recognition of vehicle license plates
Amer. J. of Appl. Sci.
DOI : 10.3844/ajassp.2009.2066.2070
Extraction and tracking of the license plate using Hough transform and voted block matching
Proc. IEEE Intelligent Vehicles Symp.
Hilton Hotel Columbus USA
DOI : 10.1109/IVS.2003.1212916
Ahmed M. J
Ghazi S. A
Saudi Arabian license plate recognition system
Proc. 2003 Int. Conf.Geometric Modeling and Graphics
DOI : 10.1109/GMAG.2003.1219663
Kim Y. D
An approach to Korean license plate recognition based on vertical edge matching
Proc 2000 IEEE Int. Conf. Systems Man and Cybernetics
DOI : 10.1109/ICSMC.2000.884453
Duan T. D
Du T. L. H
Phuoc T. V
Hoang N. V
Building an automatic vehicle license-plate recognition system
Proc Int. Conf. Computer Science
Can Tho Vietnam
Kim K. I
Kim J. H
Color texture-based object detection: an application to license plate localization
Lecture Notes in Computer Science
DOI : 10.1007/3-540-45665-1_25
Kim S. K
Kim D. W
Kim H. J
A recognition of vehicle license plate using a genetic algorithm based segmentation
Proc. Int. Conf. Image Processing
DOI : 10.1109/ICIP.1996.560964
An adaptive approach to vehicle license plate localization
Proc 29th Annual Conf.IECON
Hotel Roanoke and Conference Center USA
DOI : 10.1109/IECON.2003.1280330
An efficient method of vehicle plate detection based on HSI color model and histogram
Lecture Notes in Computer Science
DOI : 10.1007/978-3-642-02568-6_7
Automatic license plate recognition
IEEE Trans. Intell.Transp. Syst.
DOI : 10.1109/TITS.2004.825086
Robust license plate recognition method for passing vehicles under outside environment
IEEE Trans.Vehic. Techn.
DOI : 10.1109/25.901900
The detection and segmentation of blobs in infrared images
IEEE Trans. Sys. Man and Cybern.
DOI : 10.1109/TSMC.1981.4308652