Advanced
Precise Detection of Car License Plates by Locating Main Characters
Precise Detection of Car License Plates by Locating Main Characters
Journal of the Optical Society of Korea. 2010. Dec, 14(4): 376-382
Copyright ©2010, Optical Society of Korea
  • Received : August 08, 2010
  • Accepted : September 09, 2010
  • Published : December 25, 2010
Download
PDF
e-PUB
PubReader
PPT
Export by style
Share
Article
Author
Metrics
Cited by
TagCloud
About the Authors
Daeho Lee
nize@khu.ac.kr
Jin Hyuk Choi
Abstract
We propose a novel method to precisely detect car license plates by locating main characters, which are printed with large font size. The regions of the main characters are directly detected without detecting the plate region boundaries, so that license regions can be detected more precisely than by other existing methods. To generate a binary image, multiple thresholds are applied, and segmented regions are selected from multiple binarized images by a criterion of size and compactness. We do not employ any character matching methods, so that many candidates for main character groups are detected; thus, we use a neural network to reject non-main character groups from the candidates. The relation of the character regions and the intensity statistics are used as the input to the neural network for classification. The detection performance has been investigated on real images captured under various illumination conditions for 1000 vehicles. 980 plates were correctly detected, and almost all non-detected plates were so stained that their characters could not be isolated for character recognition. In addition, the processing time is fast enough for a commercial automatic license plate recognition system. Therefore, the proposed method can be used for recognition systems with high performance and fast processing.
Keywords
I. INTRODUCTION
Automatic license plate recognition (ALPR) is used in various applications for vehicle identification, e.g., traffic enforcement, parking management and electronic toll collection,in intelligent transportation systems (ITS) [1 , 2] . In ALPR systems, car images are acquired from a camera,and the images are processed by various image processing and optical pattern recognition methods [3] . Commonly, an ALPR system consists of three main processes: detection of plate regions, character isolation and character recognition.Prior to the character recognition step, the license plate (LP) regions must be detected and the character regions must be isolated. Since overall performance and processing speed are influenced by these tasks, they are the most significant step in ALPR systems [3] .
In the literature, there are a large number of methods to detect LPs, where features such as texture, frame edges and color are used [4] . The texture-based methods use distinctive gray levels such as local variances [5] , gradient density [6 - 8] , frequencies [9] and uniformity [3] . These methods show high detection performance, but many non plate regions are detected. To overcome this drawback, combined methods with morphological steps were proposed in [10 - 12] .
The methods using frame edges [13 - 17] detect frame boundaries by edge or line information. These methods may detect frame regions correctly; however the plates which have boundaries with low contrast cannot be detected.
The color-based methods [18 - 22] use a unique color or color sequence of the plate regions. These colors vary greatly due to different illumination conditions, and vehicles may have a color similar to that of the LPs. Thus, color information cannot be directly employed to detect LP regions.
In most applications, character isolation is performed by projections [8] or connected region analyses [22 , 23] of a binarized LP image after the LP detection. So, even if LPs were correctly detected, characters may not be isolated.
To develop the best appropriate LP detection method for ALPR, we propose a novel method which can detect LP regions precisely. In the proposed method, we detect LP regions by locating main characters (MCs), which are printed with large font size. To detect MCs, we binarize an input image with multiple thresholds by a criterion of size and compactness. MCs are, then, grouped as candidates, and a neural network using the relation of the MCs and the intensity statistics is employed to reject non LP regions.Therefore, our method can detect LP regions precisely, and is the most appropriate method for character recognition because MC regions are isolated in the detection process.
II. ORGANIZATION OF PROPOSED METHOD
The overall scheme of our method is shown in Fig. 1 .We first apply a DC notch filter, which compensates the variation of illumination, as shown in Fig. 1 . The DC notch filtering is calculated by
PPT Slide
Lager Image
where f and g are the original and the filtered images,respectively, and fm is the local DC (direct current, average intensity) component at ( x, y ). The size of the window that calculates the DC may be somewhat larger than the largest character expected to be detected [24] .
MC regions are detected by selecting from isolated binary regions by multiple thresholds and shape information.Then, we group the MC regions into main character group(MCG) candidates, where an MCG is a group of MCs on a LP. MCGs are detected without any character matching,so that many MCG candidates are detected. Characters of MCG, however, are arranged with predefined regulation,so we use a neural network to reject non MCGs from candidates, where the relation of the character regions and
PPT Slide
Lager Image
Overall scheme of proposed method.
the intensity statistics are used as input features of the neural network.
III. DETECTION OF MCG CANDIDATES
Vehicle images, which contain LPs, are acquired in various places at various times; thus, a simple threshold cannot segment character regions correctly. For this reason, we apply multiple thresholds, and then select character regions.
To detect MCGs, we first segment isolated MC candidates as shown in Fig. 1 . Binary images are generated using multiple thresholds, and we select character regions, which are the largest regions less than a limited size for detecting characters, from among the binary images. In addition,we use a compactness to reject non character regions. The process proceeds according to the following steps.
  • Step 1: Initialize threshold levelsT[0],T[1], …,T[n]associated with higher percentages of the intensity histogram of the image.
  • Step 2: Binarize the image withT[0],T[1], …,T[n].
  • Step 3: (For each kth binary image) Delete long horizontal lines.
  • Step 4: (For each kth binary image) Perform the image labeling on the binary image. Then store the bounding boxes of connected regions.
  • Step 5: (For each kth binary image) Select the largest regions satisfying the size of a MC. If the selected region has a lower compactness than the region which is constructed by the lower threshold level (k?1) and included in the selected regions, select the region constructed byk?1 instead of the selected region.
  • Step 6: Generate a binary image from the labels selected from all binary images at Step 5.
  • Step 7: Label the binary image and then store the bounding boxes in bright character regions (Cib).
  • Step 8: For threshold levels associated with lower percentages of the histogram, repeat step 1 to step 7, and store dark character regions (Cib).
The characters may be observed with frames of LPs because of the camera angle. Thus, we delete long horizontal lines connected pixels whose length is greater than τl in step 3 before labeling. In step 5, the compactness( α ) is defined as the ratio of the character regions to the bounding box regions. The compactness is calculated by
PPT Slide
Lager Image
where N is the number of pixels in the segmented regions, and M is the area of the bounding box enclosing the segmented region. The label merged with the boundary of the plate has a low compactness, so it is rejected by comparing to the compactness of the included region constructed in the lower threshold level.
In Fig. 2 (c) to (h), the gray color shows the regions deleted as long horizontal lines, and the regions surrounded by dotted boxes show the rejected regions as non character regions because they have the compactness lower than the compactness of the included region constructed in the lower threshold levels.
Figure 3 shows an example for detection of MC candidates. Figures 3 (c) and 3 (d) are the generated binary images with higher and lower percentages of the histogram, respectively.
In common LPs including those of Korea, the MCG is four isodistant numbers. Thus, we detect MCGs by grouping four horizontally adjacent MCs as follows:
  • Step 1: For all bright character regions (Cib), extend group regions toECibsuch as
PPT Slide
Lager Image
  • whereHECibandWECibdenote the height and the width ofECib, respectively, andHCibis the height ofCib.ECibis aligned left of the vertical middleCibposition.
  • Step 2: Detect the three nearest character regions included inECib, and group the union of the three rectangle regions andCibas a kth MCG candidate for bright characters (Gkb).
  • Step 3: For all dark character regions (Cid), repeat step 1 and step 2, and create akth MCG candidate for dark characters (Gkd).
The examples of detection of MCG candidates are shown in Fig. 4 . Cb2 does not have three regions to be included in ECb2 , so the MCG cannot be created; however, three MCGs
PPT Slide
Lager Image
Multiple thresholds: (a) original image (b) binaryimage and detected MC regions (c) (d) (e) (f) (g) and (h)binary images by thresholds associated with higher 1.0%2.5% 5.0% 7.5% 10.0% and 20.0% of the intensityhistogram respectively.
PPT Slide
Lager Image
Detection of MC candidates: (a) original image (b)filtered image (c) and (d) generated binary images withhigher and lower percentages of the intensity histogramrespectively.
PPT Slide
Lager Image
Detection of MCG candidates.
PPT Slide
Lager Image
Detected MCG candidates of Fig. 3(a).
are created by C1b , Cb3 and Cb4 .
A number of MCG candidates may be detected by many horizontally adjacent regions as shown in Fig. 5 . For this reason, we apply a neural network to reject non MCGs.
IV. MCG CLASSIFICATION BY NEURAL NETWORK
Although the LP is oriented with small angles in the image, the horizontal distance between neighbor characters is seen to have regular arrangement. Therefore, we extract the ratios of the horizontal distance between adjacent characters and the width of the MCG, and they are used as the input features of classification and are calculated by
PPT Slide
Lager Image
where W denotes the width of the MCG, and Dih denotes the horizontal distances between adjacent characters, as shown in Fig. 6 (a). Because the vertical positions of the characters have similar characteristics, we use also three ratios ( I4 , I5 and I6 ) of the vertical distances between adjacent characters and the height of the MCG as the input features of a neural network. The ratios are calculated by
PPT Slide
Lager Image
where H denotes the height of the MCG, and Diw denotes the vertical distances between adjacent characters.
The intensities of the background and the character regions are different, and both the background and the character regions are characterized by uniform intensity, as shown in Fig. 6 (b). Therefore, we extract the average intensity ( I7 )and the standard deviation ( I8 ) of the background region,and the average intensity ( I9 ) and the standard deviation( I10 ) of the character region. The type of the MCG (1 for Gb and 0 for Gd ) is used as the feature ( I11 ) to determine whether MGs are bright or dark.
PPT Slide
Lager Image
Features for classification.
PPT Slide
Lager Image
Classification of MCGs.
By using extracted input features, we employ a neural network to reject non MCGs, as shown in Fig. 7 , where the input nodes are the features, and the output nodes classify whether the region is a MCG or not.
V. EXPERIMENTAL RESULTS
Our method was tested on 1000 vehicle images taken in Korea. Four hundred forty-five images were taken as training images: 115 LP images and 330 non LP images, where many non LP images are needed for high performance,because non LP images which are similar to LP images are very varied. The original image size is 1024×1024. Then the images are down sized to 512×512 and the upper region is truncated, so the final image has a size of 512×256 when it is processed. Figure 8 shows an example of the results. Figure 8 (a) shows non MCGs rejected by a neural network,and Fig. 8 (b) shows a detected MCG. Figure 9 shows correctly detected MCGs with whole plate regions for various types of LPs.
In the experiments, various types of LP regions with poor quality were correctly detected as shown in Fig. 10 .Out of the 5664 candidates detected, 4650 candidates were rejected, and 34 non MCGs were not rejected (false posi
PPT Slide
Lager Image
MCG classification of Fig. 5: (a) non MCGs and (b)detected MCG.
PPT Slide
Lager Image
Correctly detected MCGs.
PPT Slide
Lager Image
Detected MCG images.
tive rate: 0.6%). Out of the 987 MCGs detected, 980 MCGs were correctly detected (detection rate: 98.0%), and 7 false rejections were observed (true negative rate: 0.1%). The rejected non MCGs are shown in Fig. 11 . The images which have undetected MCGs have poor image quality, as shown in Fig. 12 . The number of false detected MCGs is 34. The false detected MCGs are exemplified in Fig. 13 . These regions may be rejected by detecting whole license characters. Figure 14 shows MCGs falsely rejected by the neural network. Character isolation and recognition can be correctly performed by additional processes for these images; however,overall performance may worsen because the additional
PPT Slide
Lager Image
Rejected MCG candidate images.
PPT Slide
Lager Image
Non-detected MCG images.
PPT Slide
Lager Image
False detected MCG images.
PPT Slide
Lager Image
False rejected MCG images.
processes may influence clean images. Although various types of plates are being used in Korea as shown in Figs.9 and 10 , high detection performance was achieved because the employed neural network nonlinearly classifies the feature space.
On a Pentium PC with 2.4 GHz, the average processing time of a single image was about 57.5 ms. This is fast enough to realize a commercial ALPR system.
There are many methods to detect LPs; however, their performance rates are often overestimated for promotional reasons. In the literature, the detection rates are often introduced as shown in Table 1 , but the data sets are limited with small quantities and restricted image qualities.For this reason, it is very difficult to objectively compare performance. Although the detection rate of the proposed method is less than the results of some methods, the proposed method has advantages over other methods as follows; stained LP regions which cannot be processed for character isolation were not detected, and MC regions are located in LP detection. Therefore, our method may be robust and precise.
Comparison of detection performance.
PPT Slide
Lager Image
Comparison of detection performance.
VI. CONCLUSIONS
In this paper, we proposed a novel method which detects car LPs by locating MCGs. For considering various operating conditions, multiple thresholds are used to detect MCs.To reject non MCGs for candidates, the relation of the character regions and the intensity statistics are applied to input nodes of a neural network.
If characters cannot be segmented, it is impossible to recognize a LP even though the LP is detected. Character isolation is the most important part of ALPR. Common ALPR systems employ both detection of LPs and character isolation. However, we directly detect MCGs by detecting LP regions. The detection rate is 98.0%; however, almost all non-detected plates are so stained that their characters cannot be isolated for character recognition, and the average processing time is about 57.5 ms. Therefore, the proposed method can be applied to ALPR systems with high performance and fast processing time. Moreover, when the number of grouping characters and training data are modified, this method is appropriate for multinational LP detection.
References
Park S. W , Hwang S. C , Park J. W 2003 The extraction of vehicle license plate region using edge directional properties of wavelet subband IEICE Trans. Inf. & Syst. E86-D 664 - 669
Anagnostopoulos C.-N. E , Anagnostrpoulos I. E , Psoroulas I. D , Loumos V , Kayafas E 2008 License plate recognition from still images and video sequences: a survey IEEE Trans. Intell. Transp. Syst. 9 377 - 391
Jia W , Zhang H , He X 2007 Region-based license plate detection J. Netw. Comput. Appl. 30 1324 - 1333
Jia W , He X , Piccardi M Automatic license plate recognition: a review Proc. Int. Conf. Imaging ScienceSystems and Technology Las Vegas USA Jun. 2004 43 - 49
Gao D. S , Zhou J Car license plates detection form complex scene Proc. 5th Int. Conf. Signal Processing Beijing China Aug. 2000 1409 - 1414
Zhang H , Jia W , He X , Wu Q 2006 Real-time license plate detection under various conditions Lecture Notes in Computer Science 4159 192 - 199
Shapiro V , Dimov D , Bonchev S , Velichkov V , Gluhchev G Adaptive license plate image extraction Proc.5th Int. Conf. Computer Systems and Technologies RousseBulgaria Jun. 2004 1 - 7
Zheng D , Zhao Y , Wang J 2005 An efficient method of license plate location Pattern Recognit. Lett. 26 2431 - 2438
Parisi R , Claudio E. D. D , Lucarelli G , Orlandi G Car plate recognition by neural networks and image processing Proc. IEEE Int. Symp. Circuits and Systems Monterey USA May 1998 195 - 198
Martin F , Garcia M , Alba J. L New methods for automatic reading of VLP’s (Vehicle License Plates) Proc. IASTED Int. Conf. SPPRA Crete Greece Jun. 2002
Hongliang B , Changping L A hybrid license plate extraction method based on edge statistics and morphology Proc. Int. Conf. Pattern Recognition Cambridge UK Aug. 2004 831 - 834
Kasaei S. H. M , Kasaei S. M. M , Monadjemi S. A 2009 A novel morphological method for detection and recognition of vehicle license plates Amer. J. of Appl. Sci. 6 2066 - 2070
Abolghasemi V , Ahmadyfard A 2007 A fast algorithm for license plate detection Lecture Notes in Computer Science 4782 468 - 477
Yanamur Y , Goto M , Nishiyama D Extraction and tracking of the license plate using Hough transform and voted block matching Proc. IEEE Intelligent Vehicles Symp. Hilton Hotel Columbus USA Jun. 2003 9 - 11
Sarfraz M , Ahmed M. J , Ghazi S. A Saudi Arabian license plate recognition system Proc. 2003 Int. Conf.Geometric Modeling and Graphics London England Jul.2003 36 - 41
Yu M , Kim Y. D An approach to Korean license plate recognition based on vertical edge matching Proc 2000 IEEE Int. Conf. Systems Man and Cybernetics Nashville USA Oct. 2000 2975 - 2980
Duan T. D , Du T. L. H , Phuoc T. V , Hoang N. V Building an automatic vehicle license-plate recognition system Proc Int. Conf. Computer Science Can Tho Vietnam Feb. 2005 59 - 63
Kim K. I , Jung K , Kim J. H 2002 Color texture-based object detection: an application to license plate localization Lecture Notes in Computer Science 2388 321 - 335
Kim S. K , Kim D. W , Kim H. J A recognition of vehicle license plate using a genetic algorithm based segmentation Proc. Int. Conf. Image Processing LausanneSwitzerland Sep. 1996 661 - 664
Cao G , Chen J , Jiang J An adaptive approach to vehicle license plate localization Proc 29th Annual Conf.IECON Hotel Roanoke and Conference Center USA Nov.2003 1786 - 1791
Deb K , Lim H , Kang S.-J , Jo K.-H 2009 An efficient method of vehicle plate detection based on HSI color model and histogram Lecture Notes in Computer Science 5579 66 - 75
Chang S.-L , Chen L.-S , Chung Y.-C , Chen S.-W 2004 Automatic license plate recognition IEEE Trans. Intell.Transp. Syst. 5 42 - 53
Naito T , Tsukada T , Yamada K , Kozuka K , Yamamoto S 2000 Robust license plate recognition method for passing vehicles under outside environment IEEE Trans.Vehic. Techn. 49 2309 - 2319
Minor L.G , Sklansky J 1981 The detection and segmentation of blobs in infrared images IEEE Trans. Sys. Man and Cybern. SMC-11 194 - 201