Download - ZhengHuiHu_presentation031907
-
8/2/2019 ZhengHuiHu_presentation031907
1/35
California Car License Plate
Recognition System
ZhengHui Hu
Advisor: Dr. Kang
-
8/2/2019 ZhengHuiHu_presentation031907
2/35
Introduction
A License Plate Recognition System
(LPRS) is a system to automatically
detect, recognize and identify avehicle plate.
It involves low-level image
processing techniques with higherlevel artificial intelligence
techniques.
-
8/2/2019 ZhengHuiHu_presentation031907
3/35
Applications
Mainly for monitoring, surveillanceand security. For example,
Entrance/Exit monitoring for parking lotstructures
Part of surveillance system for gatedcommunities
Control gateways for vehicle passage
Security Systems for high traffic
Law Enforcement
-
8/2/2019 ZhengHuiHu_presentation031907
4/35
Challenges
Image Capturing Vehicle speed
Lighting condition
Occlusion
Processing speed
Heavy trafficRecognition accuracy
High correctness
-
8/2/2019 ZhengHuiHu_presentation031907
5/35
Recognition Stages
Plate Localization Locate plate region out of
car and/or background
Character Segmentation Segment each
character/number out ofplate
Character Recognition Recognize each character
on the plate
Similar to OCR process
Plate Localization
Character
Segmentation
Character Recognition
-
8/2/2019 ZhengHuiHu_presentation031907
6/35
Previous Work
Many difference solutions have
already been proposed for each
stage of recognitionPlate localization
Use edge statistics to locate the plate
Fuzzy clustering algorithms
-
8/2/2019 ZhengHuiHu_presentation031907
7/35
Previous Work
Character Segmentation Vertical/horizontal projection Adaptive Clustering
Optical Character Recognition Template matching
Neural network
Feature analysis
-
8/2/2019 ZhengHuiHu_presentation031907
8/35
Approach
Target: California State License
Plates The word California appears at the
top-center on the plate in red and italic. The plate number starts with a digit(0-9),
followed by three English characters (A-
Z) and three more digits (0-9). The plate background is a light shade of
gray while its characters are of dark
blueish color.
-
8/2/2019 ZhengHuiHu_presentation031907
9/35
Input Images
Captured using adigital camera Different distance
Different lightingconditions
Different angles
Original size2048X1536
Resized to 800X600for faster process
-
8/2/2019 ZhengHuiHu_presentation031907
10/35
Base Knowledge
No assumption on size nor the
possible location.
Helpful knowledge light background color and dark
foreground
rectangular shape with same width andheight proportion high in edge concentration
-
8/2/2019 ZhengHuiHu_presentation031907
11/35
Plate Localization
Noise Filtering andBrightness normalization
Extract Edge Information
Filter using Colorand Edge Information
Connected Component
Analysis
Candidate(s) Found
Continue to
Character Segmentation
> Filter thresholdFailed
-
8/2/2019 ZhengHuiHu_presentation031907
12/35
-
8/2/2019 ZhengHuiHu_presentation031907
13/35
Extract Edge Information
Create a difference
image using
equationimg =threshold(close(src)-src)
High frequency
features areenhanced by the
operation
-
8/2/2019 ZhengHuiHu_presentation031907
14/35
Filtering
Select plate
background color
using two criteria Next to edge pixels
detected in
previous step
A light shade ofgray
-
8/2/2019 ZhengHuiHu_presentation031907
15/35
Connected Component
AnalysisCreate connected
components
Find candidateregions
High concentration
of edge pixels
Width/Height ratiosimilar to plate
-
8/2/2019 ZhengHuiHu_presentation031907
16/35
Candidate Regions
Multiple candidate
regions can be
found at this stage
All of them will be
submitted to next
stage
-
8/2/2019 ZhengHuiHu_presentation031907
17/35
Character Segmentation
Filter and Threshold
Find componentsof similar size
regionunprocessed ?
No
Count Components
Resizecomponents
7
Interpolate boxes
< 2> 2 and < 7
Crop Picture
Continue to Neural NetworkRecognition
Failed
-
8/2/2019 ZhengHuiHu_presentation031907
18/35
Filter and Threshold
Color filter the plate to remove foreignelements
Apply inverse binary threshold using thefollowing function
fimgij=255, if imgijthreshold
0, otherwise
-
8/2/2019 ZhengHuiHu_presentation031907
19/35
Character Extraction
Find a series of boxes with similar shapeand size. These are the individual
characters If number of boxes found is less than 7then an interpolation is performed
-
8/2/2019 ZhengHuiHu_presentation031907
20/35
Character Recognition
Two neural networks are used. One for recognition of digits (0-9) One for characters (A-Z)
Both networks were trained by using ahybrid method combining traditionalBack-propagation algorithm with a
Simulated Annealing process
-
8/2/2019 ZhengHuiHu_presentation031907
21/35
Artificial Neural Network
Artificial NeuralNetworks (ANN)
are modeled afterthe human brain Network of
processing units
called Neurons Good for solving
classificationproblems
InputLayer
HiddenLayer
OutputLayer
-
8/2/2019 ZhengHuiHu_presentation031907
22/35
Neurons
Neurons process information byreceiving and firing signal according to
internal function Two different types are used Threshold based step function
Sigmoid function
f=1, if wijthreshold0, otherwise
f=1
1et
-
8/2/2019 ZhengHuiHu_presentation031907
23/35
ANN Training
ANN must be trained by example beforeuse.
Supervised Training ANN receive set of Input data, and output ANN is adjusted according to the error
produced Repeat with different set of data
Back propagation (B-P) algorithm is aclassical supervised training method
-
8/2/2019 ZhengHuiHu_presentation031907
24/35
Limitation of B-P training
Same limitation ofany gradient
descent algorithm Lengthy flatplateau travel
Local minimatrap
Localmaximum
GlobalOptimum
FlatPlateau
-
8/2/2019 ZhengHuiHu_presentation031907
25/35
Simulated Annealing (SA)
Probabilistic heuristic for locating globaloptimum in large search space Invented
by Kirkpatrick, S., Gelatt, C.D., andVecchi, M.P. in 1983 Inspired by metallurgic annealing
process in which metal is cool down
gradually to get the best configuration Inner random selection allows it to
escape from local minima trapping
-
8/2/2019 ZhengHuiHu_presentation031907
26/35
Hybrid B-P SA Training
Train neighbors
Find neighbors
Create new ANN, astraining candidate
Train candidate
NN with least
error is new candidate
reduce T
End Training
< thresholdOR T > 0
-
8/2/2019 ZhengHuiHu_presentation031907
27/35
Experimental Results
Hardware used
Pentium 4 1.0 GHz processor and 512
MB of RAMImages
50 Images were acquired using a digital
cameraOriginal image size: 2048X1536
Test image size: 800X600
-
8/2/2019 ZhengHuiHu_presentation031907
28/35
Experimental Results
Plate Localization
Character Segmentation
Total Images Plate located Failed to locate Success rate
50 48 2 96%
Total Images Character
segmented
Failed Success rate Cumulative success
rate
48 45 3 93.75% 90%
-
8/2/2019 ZhengHuiHu_presentation031907
29/35
Experimental Results
Optical Character Recognition Training 5 images as basis for each digit Create 9 variations by adding noise, altering
column/rows, distortion, etc.. Recognition rate of digits
Recognition rate on training data 80%
Recognition on actual images extracted from LPRS 60%
-
8/2/2019 ZhengHuiHu_presentation031907
30/35
Experiment Results
Processing time
Average processing time from image
input to result: ~ 300ms
-
8/2/2019 ZhengHuiHu_presentation031907
31/35
Conclusion
Preprocessing
Proposed algorithm's performance are
satisfactoryNeural network training
New method combining Back-
Propagation algorithm with SimulatedAnnealing process
-
8/2/2019 ZhengHuiHu_presentation031907
32/35
Future Work
Extend current system to alsorecognize uncharacteristic plates andadditional character set.
Improve recognition ratio by usingalternative ANN configurations
-
8/2/2019 ZhengHuiHu_presentation031907
33/35
References
1.William K. Pratt, Digital Image Processing, Third Edition, JohnWiley & Sons, 20012.R. Parisi, E.D.Di Claudio, G.Lucarelli, and G. Orlandi, Car PlateRecognition by neural networks and image processing, Proceedingsof the 1998 IEEE International Symposium on Circuits and Systems,
(ISCAS '98).3.Leonard G. C. Hamey, Colin Priest, Automatic Number PlateRecognition for Australian Coditions, Proceedings of the DigitalImaging Computing: Techniques and Applications (DICTA 2005)4.Bai Hongliang, Liu Changping, A Hybrid License Plate ExtractionMethod Based on Edge Statistics and Morphology, Proceedings of the
17th
International Conference on Pattern Recognition(ICPR'04)5.Choudhury A. Rahman, Wael Badawy, Ahmad Radmanesh, A RealTime Vehicle's License Plate Recognition System, Proceedings of theIEEE Conference on Advanced Video and Signal Based Surveillance(AVSS'03)
-
8/2/2019 ZhengHuiHu_presentation031907
34/35
References
6.Takashi Naito, TOshihiko Tsukada, Keiichi Yamada, Kazuhiro Kozuka,Shin Yamamoto, Robust License-Plate Recognition Method forPassing Vehicles Under Outside Environment, IEEE Transactions onVehicular Technology, 20007.Rodolfo Zunino, Stefano Rovetta, Vector Quantization for License
Plate Location and Image coding, IEEE transactions on IndustrialElectronics, Vol 47, No. 1, 20008.Mi-Ae Ko, Young-Mo Kim, License Plate Surveillance System UsingWeighted Template Matching , Proceedings of the 32ndAppliedImagery Pattern Recognition Workshop (AIPR' 03)9.Feng Yang, Zheng Ma, Vehicle License Plate Location Based on
Histogram and Mathematical Morphology, Proceedings of the FourthIEEE Workshop on Automatic Identification Advanced Technologies(AutoID'05)10.Shyang-Lih Chang, Li-Shien Chen, Yun-Chung Chung, Sei-WanChen, Automatic License Plate Recognition, IEEE transactions onIntelligent Transportation Systems, Vol 5, No. 1, 2004
-
8/2/2019 ZhengHuiHu_presentation031907
35/35
References
11.Juntanasub, R., Sureerattanan, N., Car license plate recognitionthrough Hausdorff distance technique, Proceedings of the 17th IEEEInternational Conference on Tools with Artificial Intelligence, (ICTAI' 05)12.Timothy Masters, Advanced Algorithms for neural networks?, JohnWiley & Sons, 1995
13.Cornelius T. Leondes (Editor), ?Algorithms and Architectures(Neural Networks Systems Techniques and Applications), AcademicPress, 199814.David E. Rumelhart, Geoffrey E. Hinton & Ronald J. Williams,Learning representations by back-propagating errors, Nature 323, 533- 536 (09 October 1986)
15.Kirkpatrick, S., Gelatt, C.D., and Vecchi, M.P., Optimization bySimulated Annealing, Science, Volume 220, Number 4598, 13 May1983, pp. 671680.