rlreal-ti obj t t kitime object tracking system on
TRANSCRIPT
![Page 1: RlReal-Ti Obj t T kiTime Object Tracking System on](https://reader034.vdocument.in/reader034/viewer/2022042807/62688cd842c9be17d23172bc/html5/thumbnails/1.jpg)
R l Ti Obj t T kiReal-Time Object Tracking System on FPGAsSystem on FPGAs
Su Liu1, Alexandros Papakonstantinou2, Hongjun Wang1, Deming Chen2Hongjun Wang , Deming Chen
1School of Information Science and Engineering Shandong1School of Information Science and Engineering, Shandong University, Jinan, China
2Electrical & Computer Engineering Department, University of Illinois, Urbana-Champaign
![Page 2: RlReal-Ti Obj t T kiTime Object Tracking System on](https://reader034.vdocument.in/reader034/viewer/2022042807/62688cd842c9be17d23172bc/html5/thumbnails/2.jpg)
OutlineOutline
• Introduction
• Object Tracking Framework
• Searching Object Map Techniques
• Cascade Boundary Joiner
• Experiment Results
![Page 3: RlReal-Ti Obj t T kiTime Object Tracking System on](https://reader034.vdocument.in/reader034/viewer/2022042807/62688cd842c9be17d23172bc/html5/thumbnails/3.jpg)
OutlineOutline
• Introduction
• Object Tracking Framework
• Searching Object Map Techniques
• Cascade Boundary Joiner
• Experiment Results
![Page 4: RlReal-Ti Obj t T kiTime Object Tracking System on](https://reader034.vdocument.in/reader034/viewer/2022042807/62688cd842c9be17d23172bc/html5/thumbnails/4.jpg)
Applications of Object TrackingApplications of Object Tracking
![Page 5: RlReal-Ti Obj t T kiTime Object Tracking System on](https://reader034.vdocument.in/reader034/viewer/2022042807/62688cd842c9be17d23172bc/html5/thumbnails/5.jpg)
Tasks of Object TrackingTasks of Object Tracking
Obj ki i d f id if i h• Object tracking is used for identifying the trajectory of moving objects in video frame sequences.
• Build the model of what you want to track.
• Use what you know about the previous frames to make estimation about the position of objectsto make estimation about the position of objects in current frames.
![Page 6: RlReal-Ti Obj t T kiTime Object Tracking System on](https://reader034.vdocument.in/reader034/viewer/2022042807/62688cd842c9be17d23172bc/html5/thumbnails/6.jpg)
Real-time object tracking approachesReal time object tracking approaches
• Object detectionObject detection– Point-detection schemes : find interest points in images which have an
expressive texture in their respective localitiesBackground subtraction techniques : Object detection can be achieved– Background subtraction techniques : Object detection can be achieved by building a representation of the scene called the background model and then finding deviations from the model for each incoming frameS t ti titi th i i t t ll i il i– Segmentation : partition the image into perceptually similar regions
• Tracking– Point tracking : Objects detected in consecutive frames are represented
by points– Kernel tracking : Objects are tracked by computing the motion g j y p g
(parametric transformation such as translation, rotation, and affine) of the kernel in consecutive frames
– Silhouette tracking: Such methods use the information encoded insideSilhouette tracking: Such methods use the information encoded inside the object region (appearance density and shape models)
![Page 7: RlReal-Ti Obj t T kiTime Object Tracking System on](https://reader034.vdocument.in/reader034/viewer/2022042807/62688cd842c9be17d23172bc/html5/thumbnails/7.jpg)
Our WorkOur Work
• Propose a highly parallel hardware implementation of an object tracking p j galgorithem
• Improve the object region identification• Improve the object region identification performance of the object tracking l halgorithm
• Hardware implementation achieves up toHardware implementation achieves up to 100X speedup over the software execution
![Page 8: RlReal-Ti Obj t T kiTime Object Tracking System on](https://reader034.vdocument.in/reader034/viewer/2022042807/62688cd842c9be17d23172bc/html5/thumbnails/8.jpg)
OutlineOutline
• Introduction
• Object Tracking Framework
• Searching Object Map Techniques
• Cascade Boundary Joiner
• Experiment Results
![Page 9: RlReal-Ti Obj t T kiTime Object Tracking System on](https://reader034.vdocument.in/reader034/viewer/2022042807/62688cd842c9be17d23172bc/html5/thumbnails/9.jpg)
Main architectureMain architecture
Camera MemoryCamera Input
Memory(DDR2)
Monitor+Pre‐ Tracking
positionMain
+processing
position outputsearching
MC Memory(DDR2)
Monitor+Camerainput
Pre‐ Tracking position
Main
+processing
position outputsearching
![Page 10: RlReal-Ti Obj t T kiTime Object Tracking System on](https://reader034.vdocument.in/reader034/viewer/2022042807/62688cd842c9be17d23172bc/html5/thumbnails/10.jpg)
FPGA-based object tracking systemFPGA based object tracking system
RAW to RGB format
Pre‐processing
CCD CAM DDR2
RAW to RGB format
2D Haar Transformation
BGRND Current 1 1 1 0 0 0 0 0 0 0BRAM BRAM
Classification GeneratorMain
1 1 1 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 1 0 0 0 1 1 1 1 0 0 0 0 0 1 1 1 1 1 0
Search Map
Search Path Splitter
Dilation & ErosionSearching 0 0 0 0 1 1 1 1 1 0 1 1 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 1
Search Path Splitter
Path1
Single Square
Path2
Single Square
Path12
Single Square
...
Path1 BRAM Path2 BRAM Path12 BRAM
Tracking & Display
Five-level boundary joiner
Position Information Output
![Page 11: RlReal-Ti Obj t T kiTime Object Tracking System on](https://reader034.vdocument.in/reader034/viewer/2022042807/62688cd842c9be17d23172bc/html5/thumbnails/11.jpg)
OutlineOutline
• Introduction
• Object Tracking Framework
• Searching Object Map Techniques
• Cascade Boundary Joiner
• Experiment Results
![Page 12: RlReal-Ti Obj t T kiTime Object Tracking System on](https://reader034.vdocument.in/reader034/viewer/2022042807/62688cd842c9be17d23172bc/html5/thumbnails/12.jpg)
Searching Techniques IntroductionSearching Techniques Introduction
1 1 1 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 01 1 1 1 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0
Memory(0,0)(1 0) 001 1 1 1 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0
1 1 1 1 0 0 0 0 0 0 1 1 1 0 1 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 00 1 0 0 0 1 1 1 1 0 0 1 0 0 0 1 1 1 1 0
(1,0)(0,1)(2,0)
2000
3000
40
0 1 0 0 0 1 1 1 1 0 0 1 0 0 0 1 1 1 1 00 0 0 0 1 1 1 1 1 0 0 0 0 0 1 1 1 1 1 01 1 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 11 1 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1
(1,1)(0,2)
0 1000 2000 3000 4000
010
00
(2,1)1 1 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 11 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 10 0 0 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 10 0 0 0 1 1 1 1 1 0 0 0 0 0 1 1 1 1 1 0
(1,2)…………0 0 0 0 1 1 1 1 1 0 0 0 0 0 1 1 1 1 1 0
1 1 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 11 1 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 11 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1
(3,3)……
0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 1
![Page 13: RlReal-Ti Obj t T kiTime Object Tracking System on](https://reader034.vdocument.in/reader034/viewer/2022042807/62688cd842c9be17d23172bc/html5/thumbnails/13.jpg)
Searching Map ImprovementSearching Map Improvement
• Parallel Object Map ProcessingSplit object map into 12 20x20 sub-matricesp j p
3000
4000
0 100 200 300 400
010
020
030
040
0
Index
n
……
0 1000 2000 3000 4000
010
0020
00
0 100 200 300 400
010
020
030
040
0
Index
n
• Dual- Mode Object Identification– Single Mode– Block Mode
![Page 14: RlReal-Ti Obj t T kiTime Object Tracking System on](https://reader034.vdocument.in/reader034/viewer/2022042807/62688cd842c9be17d23172bc/html5/thumbnails/14.jpg)
Object Map1 1 1 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 01 1 1 1 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0
Search Path Splitter
Path1 Path2 Path12...
1 1 1 1 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 01 1 1 1 0 0 0 0 0 0 1 1 1 0 1 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 00 1 0 0 0 1 1 1 1 0 0 1 0 0 0 1 1 1 1 0
Path1 BRAM Path2 BRAM Path12 BRAM
0 1 0 0 0 1 1 1 1 0 0 1 0 0 0 1 1 1 1 00 0 0 0 1 1 1 1 1 0 0 0 0 0 1 1 1 1 1 01 1 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 11 1 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1
3.8 ms1 1 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 11 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 10 0 0 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 10 0 0 0 1 1 1 1 1 0 0 0 0 0 1 1 1 1 1 0
Memory
(0 0)0 0 0 0 1 1 1 1 1 0 0 0 0 0 1 1 1 1 1 01 1 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 11 1 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 11 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1
(0,1)
(0,0)(1,0)
(1,1)
0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 12.66ms
![Page 15: RlReal-Ti Obj t T kiTime Object Tracking System on](https://reader034.vdocument.in/reader034/viewer/2022042807/62688cd842c9be17d23172bc/html5/thumbnails/15.jpg)
OutlineOutline
• Introduction
• Object Tracking Framework
• Searching Object Map Techniques
• Cascade Boundary Joiner
• Experiment Results
![Page 16: RlReal-Ti Obj t T kiTime Object Tracking System on](https://reader034.vdocument.in/reader034/viewer/2022042807/62688cd842c9be17d23172bc/html5/thumbnails/16.jpg)
Cascade Boundary JoinerCascade Boundary Joiner
Path 1 Path 2 Path 3 Path 12…
Physical information recovery
….
FORAM
5-stage boundary joiner
![Page 17: RlReal-Ti Obj t T kiTime Object Tracking System on](https://reader034.vdocument.in/reader034/viewer/2022042807/62688cd842c9be17d23172bc/html5/thumbnails/17.jpg)
Cascade Boundary JoinerCascade Boundary Joiner1 2 3 4 A1 2 3 4
5 6 7 8
9 10 11 12
A
B
C9 10 11 12 C
A1’ B1’ A1’’
A2’ B2’
A3’ B3’
A2’’
A3’’
A1’’’
Complete object position
A1’’’
A2’’’A2
![Page 18: RlReal-Ti Obj t T kiTime Object Tracking System on](https://reader034.vdocument.in/reader034/viewer/2022042807/62688cd842c9be17d23172bc/html5/thumbnails/18.jpg)
OutlineOutline
• Introduction
• Object Tracking Framework
• Searching Object Map Techniques
• Cascade Boundary Joiner
• Experiment Results
![Page 19: RlReal-Ti Obj t T kiTime Object Tracking System on](https://reader034.vdocument.in/reader034/viewer/2022042807/62688cd842c9be17d23172bc/html5/thumbnails/19.jpg)
Experiment SettingExperiment Setting
AVALON
MasterTemplate
Camerainput
DDR2
NMonitor
ki
+Pre‐
processing
Tracking position output
Mainsearching
![Page 20: RlReal-Ti Obj t T kiTime Object Tracking System on](https://reader034.vdocument.in/reader034/viewer/2022042807/62688cd842c9be17d23172bc/html5/thumbnails/20.jpg)
Experimental ResultsTABLE I. PERFORMANCE COMPARISON: SW VERSUS HW
Experimental ResultsObject # SW Exec. Time SW fps HW Exec. Time HW. fps Speedup
0 30ms 33.3 1.45ms 689.6 20.69X 1 79 12 66 3 53 283 3 22 38X1 79 ms 12.66 3.53 ms 283.3 22.38X2 252 ms 3.97 3.87ms 258.4 65.12X 3 392ms 2.55 5.17ms 193.4 75.82X 4 546ms 1.83 5.27ms 189.7 103.6X 5 857ms 1.17 8.73ms 114.5 98.16X 6 1489 ms 0 67 14 4 ms 69 4 103 4X6 1489 ms 0.67 14.4 ms 69.4 103.4X
TABEL II. 20-FRAME PERFORMANCE COMPARISON
Hardware Ex. Time Software Ex. Time Speedup
97.82 ms 6038ms 62X
TABEL II. 20 FRAME PERFORMANCE COMPARISON
![Page 21: RlReal-Ti Obj t T kiTime Object Tracking System on](https://reader034.vdocument.in/reader034/viewer/2022042807/62688cd842c9be17d23172bc/html5/thumbnails/21.jpg)
Conclusions and Future Work
• We presented an FPGA implementation for object tracking in color videotracking in color video– a highly parallel hardware implementation
object tracking through a dual search technique– object tracking through a dual search technique– efficient boundary joining
up to 100X speedup over the software execution– up to 100X speedup over the software execution
• Future workh f h k l h h– improve the sensitivity of the tracking algorithm to the
luminance of the scene3D i t ti– 3D image reconstruction
![Page 22: RlReal-Ti Obj t T kiTime Object Tracking System on](https://reader034.vdocument.in/reader034/viewer/2022042807/62688cd842c9be17d23172bc/html5/thumbnails/22.jpg)
Thank you!Thank you!