vision based single-shot real-time ego- by …
TRANSCRIPT
VISION BASED SINGLE-SHOT REAL-TIME EGO-
LANE ESTIMATION AND VEHICLE DETECTION FOR
FORWARD COLLISION WARNING SYSTEM
BY
MUHAMMAD ANWAR ALHAQ A MATIN
A thesis submitted in fulfilment of the requirement for the
degree of Master of Science (Mechatronics Engineering)
Kulliyyah of Engineering
International Islamic University Malaysia
OCTOBER 2019
ii
ABSTRACT
Vision-based Forward Collision Warning System (FCWS) is a promising assist feature
in a car to alleviate road accidents and make roads safer. In practice, it is exceptionally
hard to accurately and efficiently develop algorithm for FCWS application due to the
complexity of steps involved in FCWS. For FCWS application, multiple steps are
involved namely vehicle detection, target vehicle verification, time-to-collision (TTC).
These involve an elaborated pipeline for the FCWS application using classical computer
vision methods which limits the robustness of the overall system and limits the
scalability of the algorithm. Advancement in deep neural network (DNN) has shown
unprecedented performance for the task of vision-based object detection which opens
up the possibility to be explored as an effective perceptive tool for automotive
application. In this thesis a DNN based single-shot vehicle detection and ego-lane
estimation architecture is presented. This architecture allows simultaneous detection of
vehicles and estimation of ego-lanes in a single-shot. SSD-MobileNetv2 architecture
were used as a backbone network to achieve this. Traffic ego-lanes in this thesis were
defined in two ways; first as a second-degree polynomial and second as semantic
regression points. We collected and labelled 59,068 images of ego-lane datasets and
trained the feature extractor architecture MobileNetv2 to estimate where the ego-lanes
are. Once the feature extractor is trained for ego-lane estimation the meta-architecture
single-shot detector (SSD) was then trained to detect vehicles. This thesis had
demonstrated that this method achieves real-time performance with test results of 88%
total precision on CULane dataset and 91% on our own dataset for ego-lane estimation.
Moreover, we achieve 63.7% mAP for vehicle detection on our own dataset. The
proposed architecture shows that elaborate pipeline of multiple steps to develop
algorithm for FCWS application is eliminated. The proposed method achieves real-time
at 60fps performance on standard PC running on Nvidia GTX1080 proving its potential
to run on embedded device for Forward Collision Warning System.
iii
خلاصة البحث
تعتبر أنظمة تحذير الاصطدام الأمامي للمركبات بالمستشعرات المرئية من التقنيات الواعدة التي ستساعد في جعل الطرقات أكثر أمانا. من الناحية العملية، انه لمن الصعب للغاية تطوير الخوارزميات التي سيعتمد عليها هذه
ية. الخطوات هي ا معقدة للغاالأنظمة نظرا لأن الخطوات التي تنطوي عليهثانيا: التحقق من المركبة المستهدفة ،ثالثا: الوقت المركبة،أولا: كشف ذاتها،. ولأن كل خطوة من هذه الخطوات معقدة بحد الاصطدامالمتبقي قبل
على نطاق قابلة للاستخدام وخوارزميةيحد من تطوير نظام متكامل فهذه الحاسب المرئية بتقنيات الانظمة ذه كننا تصميم هبالرغم من أننا يم واسع
كبير بالمؤثرات يعيبها أنها تتأثر بشكل فإن هذه التقنيات التقليدية، في الآونة الاخيرة، التقنيات محدودة.يجعل استخدامات هذه مما الخارجية
يث ح العميقة الخوارزميات المعتمدة على الشبكات العصبيةحدث تطور في هذه المرئية. الأشياء استخدام هذه الشبكات في الكشف عن من الممكن
التطورات جعلت من الممكن استخدامها كبديل للمستشعرات التقليدية. في هذا البحث، تطرح موضوع الشبكات العصبية العميقة المعتمدة على رصد
. ego-lanes المركبات من لقطة واحدة ومعمارية تقديرد المركبات وقراءة مسارات القيادة في آن واحد. تسمح برص ريةالمعما هذه
استخدمت كشبكة رئيسية للوصول SSD +MobileNetv2 معماريةبطريقتين: الاولى ego-lanes . يمكن ان تعرّف مصطلحه النتائجلهذ
الثانية والثانية تعرف بأنها نقاط انحدار الدرجةتعرّف بأنها متعدد الحدود من من مجموعة ٥٩٬٠٦٨ي هذا البحث تم تجميع وتصنيف الدلالي. ف
وايضا تم تدريب معمارية مستخرج الميزات لتخمين ego-lane بيانات. بعد اتمام تدريب مستخرج الميزات يتم تدريب ego-lane مكان
(single-shot multibox detector (SSD رصد من اجلوتحقق الاداء في الوقت المركبات. هذا البحث قد أثبت أن هذه الطريقة تعمل
% من الدقة الكلية في مجموع ٨٨تبار بنسبة الحقيقي كما اظهرت نتائج الاخمجموع % في ٩١و CULane بياناتفقد توصلنا ل ،إضافة. ego lane estimation بياناتنا٦٣.٧% mAP لرصد المركبات في داخل البيانات الخاصة بنا. هذه
إطار في الثانية في جهاز ٦٠ وقت الحقيقيأداء في ال حققتالطريقة ايضا قابلية استخدام مما يثبت Nvidia GTX1080 حاسوب متوسط تعمل على
هذه الطريقة في الأجهزة المدمجة.
iv
APPROVAL PAGE
I certify that I have supervised and read this study and that in my opinion, it conforms
to acceptable standards of scholarly presentation and is fully adequate, in scope and
quality, as a thesis for the degree of Master of Science (Mechatronics Engineering)
…………………………………..
Hasan Firdaus bin Mohd Zaki Supervisor
…………………………………..
Zulkifli bin Zainal Abidin
Co-Supervisor
…………………………………..
Yasir Mohd. Mustafah
Co-Supervisor
I certify that I have read this study and that in my opinion it conforms to acceptable
standards of scholarly presentation and is fully adequate, in scope and quality, as a thesis
for the degree of Master of Science (Mechatronics Engineering)
…………………………………..
Malik Arman Bin Morshidi
Internal Examiner
…………………………………..
Siti Anom Ahmad
External Examiner
This thesis was submitted to the Department of Mechatronics Engineering and is
accepted as a fulfilment of the requirement for the degree of Master of Science
(Mechatronics Engineering)
…………………………………..
Syamsul Bahrin Abdul Hamid
Head, Department of
Mechatronics Engineering
v
This thesis was submitted to the Kulliyyah of Engineering and is accepted as a
fulfilment of the requirement for the degree of Master of Science (Mechatronics
Engineering)
…………………………………..
Ahmad Faris Ismail
Dean, Kulliyyah of Engineering
vi
DECLARATION
I hereby declare that this thesis is the result of my own investigations, except where
otherwise stated. I also declare that it has not been previously or concurrently submitted
as a whole for any other degrees at IIUM or other institutions.
Muhammad Anwar Alhaq A Matin
Signature ........................................................... Date .........................................
vii
INTERNATIONAL ISLAMIC UNIVERSITY MALAYSIA
DECLARATION OF COPYRIGHT AND AFFIRMATION OF
FAIR USE OF UNPUBLISHED RESEARCH
VISION BASED SINGLE-SHOT REAL-TIME VEHICLE
DETECTION AND EGO-LANE ESTIMATION FOR FORWARD
COLLISION WARNING SYSTEM
I declare that the copyright holders of this thesis are jointly owned by the student
and IIUM.
Copyright © 2019 Muhammad Anwar Alhaq A Matin and International Islamic University
Malaysia. All rights reserved.
No part of this unpublished research may be reproduced, stored in a retrieval system,
or transmitted, in any form or by any means, electronic, mechanical, photocopying,
recording or otherwise without prior written permission of the copyright holder
except as provided below
1. Any material contained in or derived from this unpublished research
may be used by others in their writing with due acknowledgement.
2. IIUM or its library will have the right to make and transmit copies (print
or electronic) for institutional and academic purposes.
3. The IIUM library will have the right to make, store in a retrieved system
and supply copies of this unpublished research if requested by other
universities and research libraries.
By signing this form, I acknowledged that I have read and understand the IIUM
Intellectual Property Right and Commercialization policy.
Affirmed by Muhammad Anwar Alhaq A Matin
……..…………………….. ………………………..
Signature Date
viii
ACKNOWLEDGEMENTS
Firstly, it is my utmost pleasure to dedicate this work to my dear parents and my family,
who granted me the gift of their unwavering belief in my ability to accomplish this goal:
thank you for your support and patience.
I wish to express my appreciation and thanks to those who provided their time,
effort and support for this project. Thank you to my co-supervisor Asst. Prof Dr. Zulkifli
Zainal Abidin and Assoc. Prof. Dr. Yasir Mohd. Mustafah for their support and
guidance. I would also like to extend my gratitude to my lab mate Mr. Syarifuddin
Ahmad Fakhri for his never-failing support in helping me in this project.
Finally, a special thanks to Asst. Prof Dr. Hasan Firdaus bin Mohd. Zaki as
supervisor for his continuous support, encouragement and leadership, and for that, I will
be forever grateful
This thesis project is supported by project collaboration of International Islamic
University Malaysia (IIUM) with Collaborative Research in Engineering, Science &
Technology Centre (CREST) and Delloyd R&D Sdn. Bhd. (Grant ID P11C2-17 &
SP17-029-0291)
ix
TABLE OF CONTENTS
Abstract ................................................................................................................ii
Abstract in Arabic ……………………………………………………………………………iii
Approval Page ......................................................................................................iv Declaration ...........................................................................................................vi
Copyrights ............................................................................................................vii Acknowledgements ..............................................................................................viii
Table of Contents .................................................................................................ix List of Tables........................................................................................................xi
List of Figures ......................................................................................................xii List of Abbreviations ............................................................................................xiv
CHAPTER ONE: INTRODUCTION ................................................................1
1.1 Overview .............................................................................................1 1.2 Statement of the Problem .....................................................................1
1.3 Research Objectives .............................................................................3 1.4 Research Methodology ........................................................................3
1.5 Research Scope ....................................................................................6 1.6 Report Organization .............................................................................6
CHAPTER TWO: LITERATURE REVIEW ...................................................8
2.1 Introduction .........................................................................................8 2.2 Object Detection using Traditional Approach .......................................11
2.3 Deep Learning Based Methods ............................................................12 2.4 Deep Learning Based Feature Extractor ...............................................16
2.5 Deep Learning Based Object Detection ................................................19 2.6 Comparison of classical method vs DNN Method ................................22
2.7 Collision Judgement.............................................................................24 2.7.1 Ego-lane identification ................................................................24
2.7.2 Collision risk identification .........................................................25 2.8 Chapter Summary ................................................................................25
CHAPTER THREE: SYSTEM DESIGN ..........................................................27
3.1 Introduction .........................................................................................27 3.2 Design Development Flowchart ...........................................................27
3.3 System Description ..............................................................................28 3.3.1 Deep Neural Net architecture ......................................................29
3.3.1.1 Ego-lane estimations .......................................................30 3.3.1.2 Vehicle detection ............................................................34
3.3.2 Dataset........................................................................................35 3.3.2.1 Ego-lane prediction dataset .............................................37
3.3.2.2 Vehicle detection dataset ................................................45 3.3.2.3 Overall dataset statistics ..................................................45
3.4 Performance Metric .............................................................................47 3.5 Chapter Summary ................................................................................48
x
CHAPTER FOUR: RESULTS AND DISCUSSION .........................................49
4.1 Introduction .........................................................................................49 4.2 DNN Based Ego-lane Estimation Model Results ..................................49
4.3 Vehicle Detection Training Results ......................................................52 4.4 Real-time Performance Result ..............................................................54
4.5 Implementation for FCWS Application Result .....................................55 4.6 Summary .............................................................................................56
CHAPTER FIVE: CONCLUSION AND RECOMMENDATIONS ................57
5.1 Conclusion ...........................................................................................57 5.2 Contributions .......................................................................................58
5.3 Limitation and Recommendation .........................................................59
REFERENCES ...................................................................................................61
xi
LIST OF TABLES
Table 2.1 Summary of Top-1 accuracy of some popular feature-
extractor CNN
17
Table 2.2 Summary of classical methods vs DNN methods
23
Table 3.1 Statistics of dataset for training ego-lane estimation
architecture
46
Table 3.2 Statistics of dataset for training vehicle detection
architecture
47
Table 4.1 Training loss for 2nd degree polynomial spline based ego-
lane models
50
Table 4.2 Test accuracy for 2nd degree polynomial spline based ego-
lane models
51
Table 4.3 Training loss for point regression based ego-lane models
51
Table 4.4 Test accuracy for point regression based ego-lane models
52
Table 4.5 Model detection accuracy comparison on CULane test
dataset
52
Table 4.6 Total loss of vehicle detection model
53
Table 4.7 Performance comparison of vehicle detection model
53
Table 4.8 Devices specifications to be used to run real-time test
54
Table 4.9 Real time performance on various platform 55
xii
LIST OF FIGURES
Figure 1.1 Flow chart of thesis project
5
Figure 2.1 Features in ADAS. “Increasing reliance on ADAS despite
limitations – Telematicswire ,” 2018
9
Figure 2.2 Illustration of general pipeline for Convolutional Neural
Network (Guo, Y. et al., 2016)
14
Figure 2.3 Shows how IoU are calculated
20
Figure 3.1 Flowchart of system development process
28
Figure 3.2 Overall single-shot vehicle detection and ego-lane
estimation pipeline
29
Figure 3.3 Ego-lane architecture with 2nd degree polynomial output
31
Figure 3.4 Given a resized to 224x224 image point regression model
should output estimation of x-points coordinate of the
fixed y-points.
32
Figure 3.5 Training point regression architecture will require dataset
labelled based on the resized image
33
Figure 3.6 Overall single-shot vehicle detection and ego-lane
estimation using MobileNetv2
34
Figure 3.7 CULane detection with mask labelling. (“CULane
dataset,” 2018)
36
Figure 3.8 Dashcam used to record and collect Malaysian dataset.
“Award of the Administrator KIPO – Good design,” 2016
36
Figure 3.9 Yellow points are the x-y coordinates manually labelled
for the curve lane. The red line is the line that are the
generated outcome of fitting the x-y coordinates to a 2nd
degree polynomial.
38
Figure 3.10 Shows how flipping the x-y axis allows us to get the
curve shape similar to that of an actual curved traffic lane
39
Figure 3.11 (a) Flipped labelled lane (b)The un-flipped labelled
lane
40
xiii
Figure 3.12 Output after labelling
41
Figure 3.13 Original CULane .txt file that contains the lane type
labels
41
Figure 3.14 Original CULane .txt file that contains the lane
coordinates labels
41
Figure 3.15 Samples of data with ego-lane a) 0 1 1 1 lane type b) 0 1
1 0 lane type c) 1 1 1 1 lane type and d) 1 1 1 0 lane type
42
Figure 3.16 Labelling tool to label our own ego-lanes. Yellow
drawings are the manually labelled points on the road.
Red lines are the outcome after fitting it to a polynomial
equation. Green lines are the saved outcome from the
saved labels
43
Figure 3.17 Visualization of ego-lane labels
43
Figure 3.18 Six float numbers corresponding to coefficient values of
two 2nd degree polynomials as labels for two ego-lane
44
Figure 3.19 Variations from the augmentation
44
Figure 3.20 Labelling vehicles using open-source labelling tool
labelimg
45
Figure 3.21 Samples from the augmented dataset for vehicle detection
labels
45
Figure 3.22 Evaluation output showing indicating the estimation lines
that falls on the ground truth lane and those that are not
47
Figure 4.1 Sample output from integrated single-shot vehicle
detection and ego-lane estimation model
56
xiv
LIST OF ABBREVIATIONS
ADAS – Advanced Driver Assistance System
AP – Average Precision
CNN – Convolutional Neural Network
CPU – Central processing unit
DNN – Deep Neural Network
FC – Fully connected layer
FCWS – Forward Collision Warning System
FPS – Frame per second
GPU – Graphical Processing Unit
IoU – Intersection of Union
LDWS – Lane Departure Warning System
MAE – Mean Absolute Error
mAP – Mean Average Precision
MSE – Mean Squared Error
SSD – Single Shot Multibox Detector
TTC – Time to collision
1
CHAPTER ONE
INTRODUCTION
1.1 OVERVIEW
Lack of attention by drivers is identified as the cause for 80% of driver related accidents
(Cui, Liu, Li, & Jia, 2010). With recent advances in technology many applications in
Advanced Driver Assistance Systems (ADAS) are implemented in cars to assist drivers
to ensure safety. With recent advances in Deep Neural Networks (DNN) achieving
highest accuracy in object detections this has become key area of computer vision
especially for ADAS to enable safer roads. Breakthrough in DNN brought opportunities
to bring detection models and to be implemented in vision based ADAS application for
secondary safety or crash protection technologies to deliver large life savings. Among
the potential benefits of high accuracy object detections in ADAS is collision avoidance
systems such as forward collision warning system, reverse collision warning system,
adaptive cruise control and emergency brake assist. Although many of these systems
have been developed using high end sensors, a breakthrough on vision-based detection
using DNN with high accuracy detection has opened opportunities for cheaper
replacement possibilities without compromising performance; moreover,
breakthroughs in DNN offers the possibility to realize the making of autonomous
vehicle.
1.2 STATEMENT OF THE PROBLEM
Developing an elaborate vision-based forward collision warning system (FCWS)
involves the fusion of multiple methods. These methods include: a method for vehicle
detection, a method of target vehicle tracking, and a method of Time to Contact
2
Calculation (TTC). All methods mentioned will be recurring every cycle to first detect
and then finally give decision if a target vehicle is indeed headed to collide with the host
vehicle. Each method has its own specific needs, some of which are required to be fast
and robust for the other methods to be effective.
Consider the problem of vehicle detection method. While many detection methods
prove to show high accuracy detection but noise variance for scenes on road is so high,
we need to make sure our detection model is robust enough to overcome noises and
understand the model’s limitations. Aside from being robust with variance of real road
scenarios, consideration must be made to ensure only the ones that have a suitable
accuracy/speed trade-off is chosen for real-time FCWS application. For FCWS to be
real-time specific detection speed requirement needs to be met before passing the
detections to tracking and TTC methods. This can be evaluated by benchmarking on
current state of the art DNN based model to see how well our current model performs.
Target vehicle tracking and TTC are dependent on each other. Consider a frame
with multiple vehicle detections, while real road scenario shows diverse condition, a
host car approaching a target car on a straight path should be treated differently from a
host car approaching a target car on a curved path. Moreover, cars that are on different
position of different lane should be treated differently. Problem with vision-based
multiple object detection model is that it only gives you the bounding box coordinates
of the object it is detecting. This does not give you the information if the object that is
detected in the previous frame is the same object detected in the next frame. For vision-
based FCWS to work, the algorithm needs to understand what detected cars are more
likely to be in a course to collision to the user’s car by keeping track of its velocity and
its position relative to the host car.
3
This require methods of tracking detected cars and identifying current host car’s
positioning and the target car’s positioning on the road. Defining the host-car’s ego-lane
allows the algorithm to define its driving course on the road. Tracking detected cars
allow the algorithm to predict how likely are they to collide with the host-car by keeping
track of their behavior. Finally, all methods integrated must ensure real-time
performance to achieve real time single shot FCWS.
1.3 RESEARCH OBJECTIVES
The research aims to achieve the following objectives:
1. To investigate and train deep learning based model to perform task of ego-lane
estimation.
2. To develop and integrate vehicle detection and ego-lane estimation so it
achieves simultaneous single-shot output on a PC system.
3. To evaluate the integrated architecture of the single-shot vehicle detection and
ego-lane estimation architecture for accuracy and real-time performance.
1.4 RESEARCH METHODOLOGY
The following methodology will be adopted to achieve the objectives of the project.
1. Extensive literature review on Forward Collision Warning workflow and its
components. Research will be done by collecting information from various
sources such as books, online journals and conference papers.
2. Data collection and train different DNN detection model for vehicle detection.
This is performed to test the detection accuracy and the model’s effectiveness
for FCW application on real road-scene dataset.
4
3. Design and development of experimental set up to calculate the system’s
performance at per-frame. This will be done to measure and record the cost of
performance with variant workflow in the algorithm.
4. Evaluating the performance of the detection algorithm in a simulated
environment. This will be performed on a PC setup on an offline video testing
to test effectiveness of the algorithm.
5. Develop a working FCWS and evaluate the effectiveness of the algorithm on a
real road scenario in a car. Test if the algorithm performs as such compared
with the simulated version.
6. Model testing and validation. Experiments will be conducted to validate the
proposed model.
Figure 1.1 shows the flowchart of the research methodology to be adopted in this
research:
6
1.5 RESEARCH SCOPE
The project aims to focus on developing a vision-based FCWS for ADAS application.
The scope of the project aims at developing a single-shot DNN-based architecture that
is capable to output vehicle detections as well as to output ego-lane estimation for
FCWS application on a single architecture that performs real-time of not less than 24
frames per second. The FCWS is limited to only vision-based input system. Algorithm
testing will be done offline as well as online to test its real-time performance.
1.6 REPORT ORGANIZATION
This research proposal is divided into several chapters.
Chapter 1: Introduction
This chapter discusses the overview of the project which includes research objectives,
problem statements and methodology.
Chapter 2: Literature Review
This chapter will review the literature of the general workflow of FCWS. The review
will cover the DNN based detection suitable for ADAS application as well as other
methods that makes a FCWS. This review will help us extract the important concepts
and help us get the general concept to finally develop an architecture of our own.
Chapter 3: System design
This chapter discusses the design to develop the DNN model for single-shot vehicle
detection and ego-lane estimation. Moreover, this chapter includes discussion on how
we evaluate methods and benchmark the algorithm.
Chapter 4: Results and discussion
7
Results of the individual sub-systems are compared and discussed in this chapter. This
includes all training performance and detections results. Moreover, performance of
algorithm speed will also be discussed here in the chapter.
Chapter 5: Conclusion
This chapter summarizes what was achieved in this project. Moreover, this chapter
discusses the limitations and recommendations for the thesis project.
8
CHAPTER TWO
LITERATURE REVIEW
2.1 INTRODUCTION
Driver’s safety is always a top priority issue for car manufacturers to ensure safe roads
and reduce accidents. ADAS system are rapidly becoming commonplace in a new car
market and this is due to increasing number pressure posed by car safety rating body
such as New Car Assessment Programme (NCAP) and National Highway Traffic Safety
Administration (NHTSA). These bodies set safety standard for new cars to raise safety
standards across the automotive industry. These safety standards pushes car
manufacturers to pursue researches to incorporate technologies that will make their cars
safer and thus makes roads safer for everyone. The ubiquity of camera technologies in
everyday life are pushing manufacturers to push research towards vision-based ADAS
to allow smarter and cheaper alternative to high end sensors. Moreover, a breakthrough
on vision-based detection using DNN with high accuracy detection has opened
opportunities for cheaper replacement possibilities without compromising performance;
moreover, breakthroughs in DNN offers the possibility to realize the making of
autonomous vehicle.
Figure 2.1 shows the different types of ADAS system in a car. Individual ADAS
system are made specifically to run specific task and can run stand alone. Wide variety
of equipment are installed on car to assist in passive ADAS and active ADAS. All which
will assist to protect us from the human factor and human error that cause most traffic
accidents (Ziebinski, Cupek, Grzechca, & Chruszczyk, 2017). Therefore, these systems
9
are equipped to assist drivers in driving and are designed to increase car safety and more
generally road safety.
For object detection, most existing systems still use traditional computer vision
approaches such as Cascaded Haar-like feature detection (Viola & Jones, 2001), HOG-
SVM (Dalal & Triggs, 2005) and Hough transform based line detection (Ballard, 1981).
However, such models are susceptible to environmental noises such as adverse lighting
condition, viewpoint changes, etc. This is due to the fact that these traditional methods
depend on lower level features that is based mostly but not strictly on edges, corners,
symmetries and histograms gradients.
DNN based methods have achieved unprecedented performance in solving
several computer vision problems involving image classification and object detection
which could become a key enabler for highly accurate and robust ADAS application.
The key benefits of DNN as opposed to traditional methods for classification and
detection is because they can extract appropriate features for the tasks. Following the
Figure 2.1 Features in ADAS. “Increasing reliance on ADAS despite limitations –
Telematicswire ,” 2018
10
breakthrough of AlexNet (Krizhevsky, Sutskever, & Hinton, 2017) architecture that led
to the popularity that shows DNN performs better than their traditional approach;
increasing number research focused on using DNN for object detection tasks. This led
to numerous successful DNN object detection architecture with high performances.
DNN based object detection architectures such as R-CNN (Girshick, Donahue,
Darrell, & Malik, 2014), SSD (Liu, et al., 2016), and YOLO (Redmon, Divvala,
Girshick, & Farhadi, 2016) are extension of classification based on DNN. The end goal
is concerned with giving object localization that involves drawing bounding box around
one or multiple objects of multiple classes. The extension architecture for the object
detection are called meta-architecture. Meta-architecture is usually built on top of the
pre-trained classification model, which is sometimes called feature extractor
architecture. This brings an end-to-end solution for object detection.
However, developing forward collision warning system (FCWS) requires
vehicle detection model and ego-lane estimation model which are two separate tasks
which is not practical for real applications in embedded ADAS. The core bottleneck in
such methods is that DNN was trained for each task and combined in a late fusion
manner. Moreover, although DNN constitutes highest performing models for vehicle
detection and ego-lane estimation, it is also known for data hungriness and
computational complexity. Therefore, we propose a unified DNN based single-shot
vehicle detection and ego-lane estimation architecture which allows both tasks to be
performed simultaneously in a single shot.
Breakthrough in DNN brought opportunities to bring detection models to be
implemented in vision-based ADAS application for secondary safety or crash protection
technologies to deliver large life savings. Among the potential benefits of high accuracy
object detections in ADAS is collision avoidance systems such as FCWS, reverse