vision-based multiple vehicle detection and tracking for driver assistant system
DESCRIPTION
Vision-Based Multiple Vehicle Detection and Tracking for Driver Assistant System. Qing Ming [email protected] May. 11. 2012. Vision Based Driver Assistant System. Problem setting. Main Goal : Multiple vehicle detection and tracking. Challenge work:. - PowerPoint PPT PresentationTRANSCRIPT
Qing [email protected]
May. 11. 2012
Vision-Based Multiple Vehicle Detection and Tracking for Driver
Assistant System
2Intelligent Systems
Lab.
Vision Based Driver Assistant System
3Intelligent Systems
Lab.
Problem setting
Main Goal: Multiple vehicle detection and tracking
Challenge work:-Different environment(illumination, distance, background…)-Tracking windows scale dynamic adjustment -Vehicle partial occlusion-Vehicle temporary missing
4Intelligent Systems
Lab.
…
…
yes
System architecture
Frame 1
Vehicle detection unit
Detectednew vehicle?
yes
Frame 2
Vehicle detection unit
Detectednew vehicle?
yes
Frame n
Vehicle detection unit
Detectednew vehicle?
Particle filter 1
Internal storage
Particle filter 2
Particle filter m
Vehicle Tracking Unit Return to Vehicle
detection unit
5Intelligent Systems
Lab.
Offline process
Vehicle Detection Algorithm
Image Sequence
Gabor FeatureExtraction
BP Neural NetworkTraining
BP Neural NetworkClassifier
Test Image
Vehicle Candidate Detection
Vehicle Candidate Verification
Detected Vehicle
6Intelligent Systems
Lab.
Color segmentation
Morphological operation
Original image
Vehicle candidate detection
+
Threshold obtained by tail light image statistical value
7Intelligent Systems
Lab.
Vehicle candidate generation
hc2
1 2min maxc cw w w 1 2c ch h d
wc1c2: width between vehicle light pair.
hc1, hc2: height of C1 and C2
d : a constant which depends on image size
hc1
wc1c2
8Intelligent Systems
Lab.
Gabor feature2 2
2 212 ( )1(x,y, , ) exp exp
2x y
x y
j x y
x y
G
8 orientations5 scale
Tai Sing LEE, “Image Representation using 2D Gabor Wavelets,”IEEE Transactions on Parrern Analysis and Machine Intelligence,Vol 18,No.10, pp959-971,October 1996
9Intelligent Systems
Lab.
…
…
Training
Back propagationNeural network
Non-vehicle database
vehicle database
-1
1
Stuart Russell and Peter Norvig, “Artificial Intelligence A Modern Approach”. p. 578. 1969
10Intelligent Systems
Lab.
training
…
(No)
(Yes)
(Yes)BPNNclassifier
Gabor feature set
Vehicle candidate verification
11Intelligent Systems
Lab.
Vehicle Tracking
Frame t
Frame t+1
Detected vehicle
Histogram generation
Colorhistogram
Particle generation
Particle selection
Similaritycomputation
TrackingWindow
estimation
Histogram updating
Updatedhistogram …
12Intelligent Systems
Lab.
Target Vehicle Representation
Frame t
Color space representation
Detected vehicle
Histogram representation
Split into uniform Histogram bins
13Intelligent Systems
Lab.
Color PDF
Particle Generation
Frame t+1 Particle selection Each Particle is
consider as one pixel
• Randomly generate particles at the position of tracking window in previous frame
14Intelligent Systems
Lab.
Similarity Computing
……
•Bhattacharyya coefficient•Mean state of the particle set
Each selected Particle is consider as one region
Dorin Comaniciu, Visvanathan Ramesh, P eterMeer, "Real-Time Tracking of Non-Rigid Objects using Mean Shift, " IEEE Conference on Computer Vision and Pattern Recognition, June 13 -15, Hilton Head, SC, USA, 2000
15Intelligent Systems
Lab.
Partial occlusion and temporary missing
Frame 1 Frame 45 Frame 25
Frame 120 Frame 80 Frame 148
Target vehicle Partial occlusion Temporary missing
Temporary missing Partial occlusion Target vehicle re-tracking
16Intelligent Systems
Lab.
Temporary missing
Frame 80
Particles are generated nearby the covering vehicle bounding box
effective particles are searched
Frame 125
When enough effective particles are searched, the missing vehicle start Tracking again
17Intelligent Systems
Lab.
Color Histogram Updating
Target vehicle color histogram changing under different condition
(ex: different distance, different illumination…)
18Intelligent Systems
Lab.
Color Histogram Updating
Frame 1 Frame 55 Frame 62 Frame 115
Frame 1 Frame 55 Frame 62 Frame 115
Tracking without color histogram updating
Tracking with color histogram updating
19Intelligent Systems
Lab.
Detection result
High way Urban Road Campus
20Intelligent Systems
Lab.
Multiple vehicle detection result
Image originHigh way Urban road Campus
Total number of vehicles 56 42 36
Number of vehicle Correct detection 43 34 30
Number of vehicle fail detection 13 8 6
Detect rate (%) 76.8 80.1 83.3
21Intelligent Systems
Lab.
Tracking result
22Intelligent Systems
Lab.
Tracking result
Horizontal trajectory Vertical trajectory
23Intelligent Systems
Lab.
Tracking result
Trajectory in image plane Tracking error
24Intelligent Systems
Lab.
Partial occlusion and temporary missing result
25Intelligent Systems
Lab.
Multiple vehicle tracking result
Video origin
High way Urban Road Campus
Total number of frames 5254 952 1326
Image Size 640×480 1920×1080 640×480
Tracking window size (in pixel) 40×32-130×104 60×48-320×256 32×25-130×104
AVG. of tracking trajectory error (in pixels) 10 23 7
Total number of moving vehicle 12 5 6
Number of miss tracking vehicle 3 1 0
26Intelligent Systems
Lab.
Conclusion
Detected multiple vehicles in different environment (different light condition, different size vehicle, different speed) Tracked partial occluded vehicle Re-tracked temporary missing vehicle Tracking windows dynamically adapt according to target vehicle scale changing Color histogram self-updating
Advantage
Disadvantage Only color model based multiple vehicle tracking is not suitable for same color Vehicle occlusion problem
Future works -Camera stabilization for smooth trajectory generation-Combine with odometry information to predict dangerous situations
27Intelligent Systems
Lab.
Publications Qing Ming, Kang-Hyun Jo, “Vehicle Detection Using Tail Light Segmentation,” International Forum on Strategic Technology, August 22-24, Harbin, 2011.Ming Qing and Kang-Hyun Jo, “Vehicle Detection and Scale-adaptive Tracking Using Tail Light Segmentation,” proc. of image and vision computing New Zealand, pp. 115-119, 2011.Ming Qing and Kang-Hyun Jo, “A novel particle filter implementation for Multiple-Vehicle Detection and Tracking System using Tail Light Segmentation”, International Journal of Control, Automation, and Systems (Reviewing)Ming Qing and Kang-Hyun Jo, “Vision Based Multiple Vehicle Detection and Tracking Using Tail Light Segmentation”, IECON Montreal (reviewing), October 25-28, 2012 Mecatronics 2012, Paris, Date line: 30 May,2012
28Intelligent Systems
Lab.
29Intelligent Systems
Lab.
BPNNBack propagation nerual network:
Xi : input Zi :output t1: expect output wij : weights between input layer and hidden layer wjk : weights between hidden layer and output layer. The input goes through the neural network in order to obtain the forward propagation’s output. Compare with expect output, difference value backward propagation through network to update weights
30Intelligent Systems
Lab.
HSV color model
H: HueS: SaturationV: Value
31Intelligent Systems
Lab.
Particle filter
Particle filter:
PropagateParticle generation evaluation …
p(xt|zt) p(xt+1|zt+1)p(xt+1|zt)
Zt+1
Zt: observation
P(xt|Zt): particle state under current observation
P(xt+1|Zt): particle state prediction
Propagate
p(xt+2|zt+1)
32Intelligent Systems
Lab.
Bhattacharyya coefficient
Bhattacharyya coefficient is an approximate measurement of the amount of overlap between two statistical samples. This coefficient can be used to describe the similarity of two discrete and normalized distributions.
1
[ , ]m
u uu
p q p q