robust nighttime vehicle detection by tracking and grouping headlights qi zou, haibin ling, siwei...
DESCRIPTION
Outline INTRODUCTION RELATED WORKS OFFLINE PROCESS ONLINE PROCESS EXPERIMENT CONCLUSIONTRANSCRIPT
Robust Nighttime Vehicle Detection by Tracking
and Grouping HeadlightsQi Zou, Haibin Ling, Siwei Luo, Yaping Huang, and Mei Tian
Outline• INTRODUCTION• RELATED WORKS• OFFLINE PROCESS• ONLINE PROCESS• EXPERIMENT• CONCLUSION
Outline• INTRODUCTION• RELATED WORKS• OFFLINE PROCESS• ONLINE PROCESS• EXPERIMENT• CONCLUSION
INTRODUCTION
vehicle counting and velocity estimation traffic behavior analysis and prediction
Reflections?
INTRODUCTION
Maximal Weighted Independent Set
Outline• INTRODUCTION• RELATED WORKS• OFFLINE PROCESS• ONLINE PROCESS• EXPERIMENT• CONCLUSION
RELATED WORKS• Headlight Detection Methods• Rule-based methods• Huang et al. [2] : block-based contrast• Chen et al. [8] : width-height-ratios and areas
• Physical-model-based methods• Zhang et al. [10] : light attenuation law
• Machine learning techniques• AdaBoost
[2] K. Huang, L.Wang, T. Tan, and S.Maybank, “A real-time object detectingand tracking system for outdoor night surveillance,” Pattern Recognit.,vol. 41, no. 1, pp. 432–444, Jan. 2008.
[8] Y.-L. Chen, B.-F. Wu, H.-Y. Huang, and C.-J. Fan, “A real-time visionsystem for nighttime vehicle detection and traffic surveillance,” IEEETrans. Ind. Electron., vol. 58, no. 5, pp. 2030–2044, May 2011.
[10] W. Zhang, Q. M. J. Wu, G. Wang, and X. You, “Tracking and pairingvehicle headlight in night scenes,” IEEE Trans. Intell. Transp. Syst.,vol. 13, no. 1, pp. 140–153, Mar. 2012.
RELATED WORKS• Headlight Pairing Methods• Motion based pairing• Chen et al. [8]
• Vanishing points + Bidirectional reasoning• Zhang et al. [10]
• Graphical model
Learning Headlight Detector• AdaBoost+Haar object detection framework [20]• Discriminates headlights from non-headlights• 180 dimensional feature vector • 5 types of Haar features at 4 scales and 9 positions on a patch
• Contrast patterns between center and periphery
[20] P. Viola and M. J. Jones, “Robust real-time face detection,” Int. J. Comp.Vis., vol. 57, no. 2, pp. 137–154, May 2004.
Learning Geometric Model• Perspective projection of the 3D world onto a 2D plane• Between-headlight distance is large when they are close to the
camera• OXYZ is camera-centered• Two headlights of the same vehicle are (, , ) and (, , ) • Their projected image coordinates are (, ) and (, )
Learning Geometric Model
• The two headlights of a vehicle are at the same height as well as the same vertical distances H from the camerai.e. Y1 ≈ Y2 = H, so y1 ≈ y2 = y
Δx and y are linear
Headlight Detection• ROI : a region of interest• Reduce the disturbances from street lamps and environments• Automatic ROI detection is feasible using offline learning
In this paper, however, we skip this part and focus on robust headlight pairing and tracking
Headlight Detection• First step :• Get all possible headlight candidates1. Extract bright blobs from ROIs by thresholding the luminance2. Some postprocessing by morphological operations is performed
to remove noises• Second step :• Classify the extracted bright blobs as headlights or not [20]
Headlight Tracking by Context-Based MultipleObject Tracking• An image sequence : {}• The headlight candidates detected in frame : {}• Each headlight feature vector : {, , , }
• To cope with false detections and missing objects, for each headlight we add a dummy element into the association affinity matrix
position area shape(ratios)
Headlight Tracking by Context-Based MultipleObject Tracking• FindΠ={} ϵ to maximize certain energy function :
Headlight Tracking by Context-Based MultipleObject Tracking
• The prediction is based on either a constant velocity model or a constant acceleration model• If velocity is unavailable such as in the first two frames or when new
objects enter the scene, we predict positions using a velocity prior that is learned from training data
Headlight Tracking by Context-Based MultipleObject Tracking
• The optimization in (6) can be solved by the Hungarian algorithm [22]• However, in a congested traffic or high-speed situation, ambiguity in
association increases greatly
Headlight Tracking by Context-Based MultipleObject Tracking
assignment
Headlight Tracking by Context-Based MultipleObject Tracking large γ is used for vehicles with large velocity change
Headlight Tracking by Context-Based MultipleObject Tracking• Context based association (CA)• Association without context (NoCA)• Association by overlapping area based tracking
(OAA, the tracking strategy used in [8])
cannot be matched in ↓
Keep constant velocity↓
Die out if larger than a threshold
cannot be matched↓
Start new trajectories
Pairing by Maximum Weighted Independent Set
Pairing by Maximum Weighted Independent Set
Pairing by Maximum Weighted Independent Set• The set of headlights ={}• Candidate pairs =(, ), , Є = {}• G = {V, E, W}• V= • E= {}
M subsets with connecting only conflicting pairsi.e. a conflict subset with respect to • W={:(i,j) ЄV}
Pairing by Maximum Weighted Independent Set
small λ is used when headlight size is less reliable
Pairing by Maximum Weighted Independent Set
• Greedy Randomized Adaptive Search Procedure (GRASP) [23]
vertex packing
Pairing by Maximum Weighted Independent Set• Since our assumption that a pair of headlights form a vehicle, it does
not account for vehicles which possess four headlights• We groups two pairs into one pair if these two pairs are well aligned
and the distance between them is smaller than the length of a vehicle
Pairing by Maximum Weighted Independent Set
About 5 iterations
Outline• INTRODUCTION• RELATED WORKS• OFFLINE PROCESS• ONLINE PROCESS• EXPERIMENT• CONCLUSION
EXPERIMENT• Five videos :• a congested highway (Seq.1)• an urban traffic scene (Seq.2)• a busy street (Seq.3a:without rain; Seq.3b:rain)• a bridge traffic in a rainy night (Seq.4)• another rain night traffic (Seq.5)
EXPERIMENT• Measurement :• Jaccard coefficient :
• CLEAR metrics :• Miss Rate (MR)• False Positive Rate (FPR)• Mismatch Rate (MMR)
Headlight Detection
Reflection Intensity Maps
[8] [10]
Headlight Detection
Headlight Tracking and Pairing
[8]
[2]
allows unpaired single headlights(motorbikes)
local contrast
Outline• INTRODUCTION• RELATED WORKS• OFFLINE PROCESS• ONLINE PROCESS• EXPERIMENT• CONCLUSION
CONCLUSION• We proposed a nighttime traffic tracking system :
• The main contribution of the proposed system lies in its learning-based detection
Thanks for listening!