my research work new
TRANSCRIPT
-
8/9/2019 My Research Work New
1/40
Tracking with 3D Data
M. Usman Butt
Department of Electrical and Computer Engineering
10 July, 2012
-
8/9/2019 My Research Work New
2/40
-
8/9/2019 My Research Work New
3/40
Objectives
To propose a novel solution for robust objectdetection and tracing using 3D !depth" informationand other features# if necessar$# such as te%ture#edges etc&
To design and develop a program implementing thescheme&
To investigate and identif$ the problems for real'time implementation of the scheme for objectdetection and tracing&
To develop the scheme on an ()*+ or a *),platform for real'time implementation&
To evaluate and validate the performance of thes$stem b$ comparing with e%isting state'of'the'art
s$stems&
-
8/9/2019 My Research Work New
4/40
Previous Work
Stereo with-o Intensit$ and binar$ template(Beymer et al., Frame-rate, 1999)
o.olour and pattern(Darell et al., IJCV, 2000)
o.olour and motion(Yang et al., IJIST, 2005)
o Intensit$# 3D world point clouds !plan view"(Baha!r" et al., #$$l. Intell"gen%e, 200&)
oSparse set of feature points and ernel'basedclustering
(Ca" et al., 'attern e%!gn"t"!n, 2010)
-
8/9/2019 My Research Work New
5/40
M !""roach
Onl$ 3D information from a high resolution disparit$ mapgenerated b$ implementation of S$mmetric D$namic)rogramming Stereo !SD)S" algorithm on ()*+
Object detection from SD)S disparit$ map
Tracing using the 0alman 1lter
+ssumptions + suitable bacground disparit$ map is available
Objects of interest can be an$ rigid !e&g& vehicles" or non'rigid !e&g&human" object# but object sies must lie in a pre'de1ned range
ach traced object 1rst appears as an isolated object and moves
upright Object speed is less than 45 m s '4 !Ol$mpic sprinter" for human
tracing
The ground plane is basicall$ 6at and 1%ed
.ameras are tilted at 1%ed angle to the ground plane
-
8/9/2019 My Research Work New
6/40
#mmetric Dnamic Programming#tereo $#DP#%Matches scan line b$ scan line
)roduces disparit$ map seen b$ a.$clop8an e$e
9as a nice mathematical propert$o +n$ change in disparit$ along a
scan line is accompanied b$ onemonocular point for each unitchange in disparit$• : *+ 1, 2, , * are M; or M< pi%els
Objects separated from their
bacground are outlined with bandsof M; or M< points
-
8/9/2019 My Research Work New
7/40
#mmetric Dnamic Programming#tereo $#DP#%
+n object>s 3D structure is represented b$ severalcontours
+ e$ problem
oDetermine the contour best outlining the object
Major Drawbac
o
Streaing artefacts
Disparit$ map .ontoursSphere
-*
-
8/9/2019 My Research Work New
8/40
#tereo &ardware #stem
SD)S algorithm implemented on ()*+ as wellas *),
4 Mpi%els at 35 fps
Disparit$ range 5'42?
-
8/9/2019 My Research Work New
9/40
'a(man)i(ter
#DP#!(gorithm$)P*! or
*PU%
Object Detection and Tracking
<image
;image
Dispmap
Occl
map
Thresholding disp
map
MOOA
.ontour*eneration
3DObjectmodel
DetermineBC !object"contours
-
8/9/2019 My Research Work New
10/40
'a(man)i(ter
#DP#!(gorithm
$)P*!%
Object Detection and Tracking
<image
;image
Dispmap
Occl
map
Thresholding disp
map
MOOA
.ontour*eneration
3DObjectmodel
DetermineBC !object"contours
-
8/9/2019 My Research Work New
11/40
'a(man)i(ter
#DP#!(gorithm
$)P*!%
Object Detection and Tracking
<image
;image
Dispmap
Occl
map
Thresholding disp
map
MOOA
.ontour*eneration
3DObjectmodel
DetermineBC !object"contours
-
8/9/2019 My Research Work New
12/40
'a(man)i(ter
#DP#!(gorithm
$)P*!%
Object Detection and Tracking
<image
;image
Dispmap
Occl
map
Thresholding disp
map
MOOA
.ontour*eneration
3DObjectmodel
DetermineBC !object"contours
-
8/9/2019 My Research Work New
13/40
Mor"ho(ogica( O"en O"eration$MOO%Before MOO +fter MOO
Object Detection and Tracking
MOO
-
8/9/2019 My Research Work New
14/40
'a(man)i(ter
#DP#!(gorithm
$)P*!%
Object Detection and Tracking
<image
;image
Dispmap
Occl
map
Thresholding disp
map
MOOA
.ontour*eneration
3DObjectmodel
DetermineBC !object"contours
-
8/9/2019 My Research Work New
15/40
'a(man)i(ter
#DP#!(gorithm
$)P*!%
Object Detection and Tracking
<image
;image
Dispmap
Occl
map
Thresholding disp
map
MOOA
.ontour*eneration
3DObjectmodel
DetermineBC !object"contours
-
8/9/2019 My Research Work New
16/40
'a(man)i(ter
#DP#!(gorithm
$)P*!%
Object Detection and Tracking
<image
;image
Dispmap
Occlmap
Thresholding disp
map
MOOA
.ontour*eneration
3DObjectmodel
DetermineBC !object"contours
-
8/9/2019 My Research Work New
17/40
Object Detection !(gorithm
.ompare the candidate contours at adjacentdisparities and determine the binocularl$ visible Gtrue>contours representing the object outline using rules-
If the centroid of B,, lies inside B-1, # then B, isenclosed b$ B-1, # then
If the heights of both the contours are same withinthe tolerance# !h, 3h-1, 4 h" and
The internal contour area ratio# #, #-1, 6 0.9, then
B, is an object outline
-
8/9/2019 My Research Work New
18/40
'a(man)i(ter
#DP#!(gorithm
$)P*!%
Object Detection and Tracking
<image
;image
Dispmap
Occlmap
Thresholding disp
map
MOOA
.ontour*eneration
3DObjectmodel
DetermineBC !object"contours
-
8/9/2019 My Research Work New
19/40
'a(man)i(ter
#DP#!(gorithm
$)P*!%
Object Detection and Tracking
<image
;image
Dispmap
Occlmap
Thresholding disp
map
MOOA
.ontour*eneration
3DObjectmodel
DetermineBC !object"contours
-
8/9/2019 My Research Work New
20/40
'a(man)i(ter
#DP#!(gorithm
$)P*!%
Object Detection and Tracking
<image
;image
Dispmap
Occlmap
Thresholding disp
map
MOOA
.ontour*eneration
3DObjectmodel
DetermineBC !object"contours
-
8/9/2019 My Research Work New
21/40
'a(man)i(ter
#DP#!(gorithm
$)P*!%
Object Detection and Tracking
<image
;image
Dispmap
Occlmap
Thresholding disp
map
MOOA
.ontour*eneration
3DObjectmodel
DetermineBC !object"contours
-
8/9/2019 My Research Work New
22/40
Object Detection !(gorithm
(O< +.9 OBH.T I )
-
8/9/2019 My Research Work New
23/40
'a(man)i(ter
#DP#!(gorithm
$)P*!%
Object Detection and Tracking
<image
;image
Dispmap
Occlmap
Thresholding disp
map
MOOA
.ontour*eneration
3DObjectmodel
DetermineBC !object"contours
-
8/9/2019 My Research Work New
24/40
'a(man)i(ter
#DP#!(gorithm
$)P*!%
Object Detection and Tracking
<image
;image
Dispmap
Occlmap
Thresholding disp
map
MOOA
.ontour*eneration
3DObjectmodel
DetermineBC !object"contours
-
8/9/2019 My Research Work New
25/40
ki
-
8/9/2019 My Research Work New
26/40
Tracking
0alman (ilter predicts the object position in the ne%t frame
+deNuate for simple linear model
(aster computationall$ simple )redicted data helps to
o Identif$ heads cut oP due to SD)S streas
o (ind missing objects !occluded or buried in noise" in the frame
Search in a small area around the predicted position for best match withthe template
4e-erence Tracking Point .entroid (: %, Y %, ; % )
o +n obvious choice but# for articulated objects!person"# centroid shifts as the object deforms
(eet (: / , Y / , ; / )o *round contact point changes signi1cantl$ as
people wal
o ,suall$# feet cut oP b$ bacground subtraction
9ead (: h, Y h, ; h )
o +lmost alwa$s visible
-
8/9/2019 My Research Work New
27/40
4e-erence Point +hoice
4e-. Tracking Point 5$ * c , + ( , ( &
670m 6 8 0m 690m
+entroid
5&5/ 5&5R 5&4@
)eet 5&5@ 5&5? 5&47
&ead 5&57 5&5/ 5&4/
-
8/9/2019 My Research Work New
28/40
4esu(ts
#cenario !
-
8/9/2019 My Research Work New
29/40
4esu(ts
#cenario !
-
8/9/2019 My Research Work New
30/40
4esu(ts
#cenario B
-
8/9/2019 My Research Work New
31/40
4esu(ts
#cenario B
-
8/9/2019 My Research Work New
32/40
4esu(ts
#cenario +
-
8/9/2019 My Research Work New
33/40
4esu(ts
#cenario +
'2 '2 '4 '4 5 4 4 2 2
+ +!)red" B B!)red" . .!)red"
7 m
9 m
'5&2
'5&4
5&5
5&45&2
+ B .
)rame :
Deviation$7% m
'2
'4
5
4
2
+!K" +!)redSK"
B!K" B!)redSK"
)rame :
7 m
'5&='5&/'5&2 5&5 5&2 5&/ 5&=5
45
25
35
/57 Devition
+
B
.
'5&2
'5&4
5&5
5&4
5&2
+ B .
)rame :
Deviation$8% m
5
4
4
2
2
3+!L" +!)redSL"
B!L" B!)redSL"
.!L" .!)redSL"
)rame :
8 m
'5&='5&/'5&2 5&5 5&2 5&/ 5&=5
7
45
47 8 Devition
+
B
.
'4&5
'5&7
5&5
5&7
4&5
+ B .
)rame :
Deviation$9% m
45
47
25
27
35 +!M " +!)redSM "B!M " B!)redSM "
)rame :
9 m
'5&='5&/'5&25&5 5&2 5&/ 5&=5
4
2
3
/9 Devition
+
B
.
a" d
"
g"
b" e" h"
c" f" j"
-
8/9/2019 My Research Work New
34/40
4esu(ts
-
8/9/2019 My Research Work New
35/40
4esu(ts
#cenario D
-
8/9/2019 My Research Work New
36/40
4esu(ts
#cenario D
-
8/9/2019 My Research Work New
37/40
4esu(ts
9eight !L"#m
+ctual Measured
Subject + /.;< /.=>
-
8/9/2019 My Research Work New
38/40
4esu(ts
Per-ormance
Time# s
Bacgroundsubtraction
Detection Tracing!0("
Total
Scenario +!Single)erson"
//
-
8/9/2019 My Research Work New
39/40
4esearch Progress to Date
Designed and developed isolated object detection and tracing techniNue
from depth map
Developed and integrated 0alman (ilter !0(" into the tracing scheme
+cNuired video seNuences for several outdoor scenarios with the help of.IT< group to test the developed techniNue
Implemented and tested the techniNue on the video seNuences withisolated objects
Based on anal$sis of results of isolated object tracing# improved thedetection and tracing techniNue to handle the occlusion problem
Tested the improved version on the video seNuences with the presence ofocclusion
+ttended one of the Doctoral Sills )rogramme Induction Da$s
Submitted full thesis proposal
)ublished a paper-
A. . B8tt an J. A!rr", reci$e .racking u$ing /ig( e$olutioneal-time tereo, Image an V""!n C!m$8t"ng Ee ;ealan(IVCE;2011), #8%lan, Ee ;ealan, IVCE;, 2011, $$. 1=@-1=>.
-
8/9/2019 My Research Work New
40/40
)uture Work
Identi1cation of bottlenecs for real'time implementation
)aralleliation of the algorithm for real'time implementation
Stud$ of architecture of diPerent hardware platforms !*),and ()*+" for suitabilit$ of paralleliation
Development of algorithm on suitable platform !()*+ or
*)," for real'time processing Testing of real'time performance of the algorithm
(urther improvement in the algorithm based on resultanal$sis
valuation of the algorithm