automated parking space detection using darknet, yolov3
TRANSCRIPT
Automated Parking Space Detection using Darknet,
YOLOv3 and Android
Prathamesh U. Kalebere1, Gandhar M. Deshpande2, Priyanka S. Gavade3,
Harshal G. Pudale4, Ajit S. Mali5and A. M. Jagtap6
1Student at Dept. of Computer Science & Engineering, Rajarambapu Institute of Technology,
Islampur, India. 2Student at Dept. of Computer Science & Engineering, Rajarambapu Institute of Technology,
Islampur, India. 3Student at Dept. of Computer Science & Engineering, Rajarambapu Institute of Technology,
Islampur, India. 4Student at Dept. of Computer Science & Engineering, Rajarambapu Institute of Technology,
Islampur, India. 5Assistant Professor, Dept. of Computer Science & Engineering, Rajarambapu Institute of
Technology, Islampur, India. 6Assistant Professor, Dept. of Computer Science & Engineering, Rajarambapu Institute of
Technology, Islampur, India.
[email protected] [email protected] [email protected]
[email protected] [email protected] [email protected]
---------------------------------------------------------------------------------------------------------------------
Abstract
Finding empty parking slot in less time with fewer efforts is a difficult task for drivers
mostly in metropolitan areas. Traffic congestion may occur due to unavailability of empty
parking slots. This paper represents an approach for smart parking system which facilitates
drivers with available empty parking slots. The system is based on image processing. Using
video surveillance, it captures images and using detection it notifies the vehicle driver whether
the parking lot is available or not. This system employs both software and hardware platform.
This system will make parking of vehicle more efficient and less complex by providing a better
solution to the existing system.
Keywords - Automated parking system, vacant parking slot identification, automated parking
management, Parking space detection; machine learning, YOLO V-3 Object detection, vacant
parking space detection, Smart Parking Management app.
---------------------------------------------------------------------------------------------------------------------
Journal of Information and Computational Science
Volume 10 Issue 5 - 2020
ISSN: 1548-7741
www.joics.org524
1. Introduction
In developing countries like India, there is difficulty in finding the vacant parking space. As
there is increase in number of vehicles, it is more tedious task and takes much of our time in
finding the vacant parking space. Now-a-days much of the parking spaces are managed manually
and there is not a specific automated system for the same which will efficiently work. Generally,
to look for the vacant parking space; the driver has to roam around the parking area. Generally
near the crowded spaces like shopping malls, Hospitals, etc. such kind of problem occurs due the
number parking spaces are less than number of vehicles.
Finding vacant parking spaces is tedious task and it consumes a lot of time consuming as well
as fuels. Many times, due to bad management of available parking spaces, most parking spaces
remained unoccupied which causes traffic jams and congestion near the parking lots.
Improper management of parking spaces and informing the available vacant spaces to the
diver before entering into the parking lots has become a serious issue to look into. To solve this
problem various solutions are already implemented for vehicle slot detection. We have proposed
a system to resolve above issue in efficient manner .A camera is used to provide video of the
parking lots as an input to system and deep learning technique is used to detect the vacant
spaces. The status will get updated at user side as vehicle enters or leaves the parking lot. In this
system we are using YOLOv3 object detection algorithm which detects the parking lot and using
intersection over union principle it detects whether slot is available or not.
2. Literature survey
In today’s world access to parking space is limited and it needs a lot of manpower to maintain
these parking lots. These parking lots are not user friendly and do not provide data regarding
availability of free spaces. Various methods are prevalent for development of autonomous or
intelligent parking systems. Study of these systems shows that these require a little or more
human intervention for the functioning. To overcome the problem of parking in the congested
areas, various methods and techniques have been proposed.
Different methods are proposed for empty parking slot detection. Ming-Yee Chiu et al
proposed an embedded real time vision system. In this vehicles are counted at particular
checkpoints with the help of which no of empty spaces in parking slot can be calculated. For this
purpose, induction loop sensors are installed under the road surface. But the cost of these sensors
is high and also they cause damage to the road. Advantage of using these sensors is they are not
easily affected by environmental conditions and accuracy achieved is high enough. Moreover,
this system is not able to find exact location of free parking slots as this does not give the detail
information; it only records the number of vehicle [15]. Another method was presented by
Vamsee K. Boda et al. This method is based on wireless sensor nodes. The advantage of this
system is it is less costly and wireless sensor nodes can also be implemented at the critical places
such as lane turns, entrance and exit positions of the parking slot. Here the total number of
Journal of Information and Computational Science
Volume 10 Issue 5 - 2020
ISSN: 1548-7741
www.joics.org525
vehicles can be counted by difference of incoming and outgoing cars [13].Zhang Bin et al.
proposed the vision based parking slot detection methods which are very easy to install and the
detector can be easily adjusted according to requirements Moreover; the data obtained from
images is very rich. But, the disadvantage is the accuracy of this system is dependent on position
of camera [12]. R. Yusnita et al. presented a method, in this method a brown colored round patch
was drawn in each parking space manually. In the when the system is initialized it looks for
round patch. If patch is found, then parking slot is considered to be empty. And if the patch is
blocked by vehicle then it is considered to be not available. This system is well enough for
managing parking slots but finds difficulty in rainfall and snow [11].
In our project, majorly concentrating on these issues, we are going to implement an efficient
system as compared to above mentioned methods to search a vacant space for parking system.
3. Methodology
The main flow of the system is as shown in Fig. 1. The video is provided as an input to the
system. The top view of the parking lot is more preferable as the vehicles are more accurately
identified and classified. To enhance the recognition capacity, the video is captured for various
environmental conditions.
Each frame is extracted from a video and further processed to detect the vacant parking
spaces. If the car moves from or occupies the parking space, the green or red mask is place in
that parking space and according to the status of parking space, the database is updated.
Fig. 1 Block Diagram
The proposed methodology is divided into 4 modules. They are discussed as follows:
3.1 Object Detection
3.2 Intersection Over Union
3.3 Updating Database
3.4 Implementation
Journal of Information and Computational Science
Volume 10 Issue 5 - 2020
ISSN: 1548-7741
www.joics.org526
3.1 Object Detection:
Target Detection or Object Detection is also known as Extraction of target from a given
picture. Object Detection is a computer technology which is done with the help of Computer
Vision and Image Processing which detects the object by comparing the semantics of given
object with a certain predefined classes such as human, cars, and buildings. This is done to
identify the required information from an image. The extraction is with the help of specific
algorithms. The pictures are provided as an input to the algorithm and it checks whether required
information is available in given input or not. If it matches with the target semantics, then that
area will get marked. Vehicle Detection also comes under object detection and has been widely
used for intelligent transportation, intelligent monitoring and license plate recognition.
Machine Learning or deep learning based approaches are used for object detection. There
are 2 main requirements for machine learning approaches:
Methods for defining some features
Techniques for classification (like Support vector machine)
Deep learning approaches are based generally on CNN. Without defining the features of the
input, the object detection is done.
3.1.1 Machine Learning techniques:
Histogram of oriented gradients (HOG) features[14]
Scale-invariant feature transform (SIFT)
Viola–Jones object detection
3.1.2 Deep Learning techniques:
Single-Shot Refinement Neural Network for Object Detection (RefineDet) [1].
You Only Look Once (YOLO) [7]
Single Shot MultiBox Detector (SSD) [6]
Region Proposals (R-CNN,[10] Fast R-CNN,[8] Faster R-CNN[9])
Few incremental improvements are established into YOLOv3 over YOLOv2. For example,
DarkNet-53 with shortcut connections, a better feature extractor and object detector with feature
map up sampling and concatenation [3].
Class Prediction
Most of the classifiers assume that the output labels are mutually exclusive. It is only if the
object classes for the outputs are mutually exclusive. Thus, to convert scores into probabilities
that sum up to one, YOLOv2 applies a softmax function. YOLO is a great example of Single
stage detector. This algorithm treat object detection as a regression problem, taking a given input
image and simultaneously learning bounding box coordinates and corresponding class label
probabilities. YOLO v3 uses multi-label classification. For example, the output labels may be
non-exclusive. (The sum of output probabilities can be more than 1).
Journal of Information and Computational Science
Volume 10 Issue 5 - 2020
ISSN: 1548-7741
www.joics.org527
Fig 2: A simplified illustration of the YOLO object detector pipeline
Instead of using the softmax function, YOLOv3 uses independent logistic classifiers to
predict to which to a specific label the input belongs. YOLOv3 uses binary cross-entropy loss for
each label rather than using mean square error for calculation of the classification loss. So by
avoiding the softmax function, it reduces the computation complexity [3]. We have used trained
YOLOV3 model for object detection by using OpenCV on COCO dataset.
3.2 Computing Intersection over Union:
For measuring the accuracy of an object detector, Intersection over Union is an evaluation
metric is used. In applying Intersection over Union evaluation metric we need:
The ground-truth bounding boxes
The predicted bounding boxes
Fig. 3-1 IOU formula
Journal of Information and Computational Science
Volume 10 Issue 5 - 2020
ISSN: 1548-7741
www.joics.org528
IOU = Intersection Space / Union Space
(1)
We have implemented IOU metric for each of the parking slots. IOU is calculated for each
slot and compared with the threshold. Slots having IOU below threshold are considered as a
vacant parking space else the parking lot is considered to be occupied. In order to get accurate
and stable result, the threshold value is set to the highest value. These two functions are used to
calculate the intersection and union of the provided polygon to it.
poly_intersection=poly1_shape.intersection(poly2_shape).area
(2)
poly_union=poly1_shape.union(poly2_shape).area
(3)
IOU=poly_intersection / poly_union
(4)
Fig. 3-2 various examples IOU computation
3.3 Updating the database:
According the IOU status for each parking lot, the database is updated. We have
implemented a Google’s Firebase. It provides a real-time database which is connected to our
android system as well as to our project.
Journal of Information and Computational Science
Volume 10 Issue 5 - 2020
ISSN: 1548-7741
www.joics.org529
Fig. 4 Firebase Database before an update
Fig. 5 Firebase Database after an update
For different region, the status of parking lot is updated in database as per the IOU status. If
IOU is less than the threshold value (IOU < 0.15), then there should have the vacant parking lot
else that parking slot is occupied. The status of each parking slot is either ‘N’ or ‘O’. i.e.
N – Slot which is not vacant,
O – Slot which is vacant. (Fig.4 and Fig.5)
3.4 Implementation:
The front page of the android application is shown in the Fig. 6. We provided an
authentication facility for accessing the app. This activity prompts the user to enter user name
Journal of Information and Computational Science
Volume 10 Issue 5 - 2020
ISSN: 1548-7741
www.joics.org530
and password and then log in to the system if he is already registered else the user will have to
register first. After successful login next page which provides a choice for selecting city will be
displayed.
Fig. 6 Login Page of application
The user will have to register him by entering the appropriate details into the fields (Fig. 7).
Once the user gets registered successfully, then the next ‘Menu’ activity is displayed which
provides a choice to user for selecting the city (Fig. 8 and Fig. 9). For test purpose we have listed
3 cities like Sangli, Miraj, etc. in our app.
Fig. 7 Registration Page of application
Journal of Information and Computational Science
Volume 10 Issue 5 - 2020
ISSN: 1548-7741
www.joics.org531
The user has to select the city first and then next activity is displayed. After selecting the city,
the animated map of that parking lot is displayed with current status of each parking space (Fig.
10). We have created an animated map of the parking lot from a video which is used for testing
purpose.
We have mapped the numbers like 1 – 20 to the parking space and displayed in that animated
map. The red car represents the parking space is occupied and green car represents the parking
space is vacant. The changes in parking lot are displayed in the animated map via network.
The firebase real-time database [2] is connected to the android app which provides current
status of each parking lot. Based on status, the map gets changed for every update.
When a car moves from a parking space, according to IOU, the status of that parking space
gets updated in the database and as a result a green car is displayed over respected parking space
in that animated map (Fig. 10 and Fig. 11 The 7th car from 1st column is updated ).
Fig. 8 Menu of application Fig. 9 City selection activity
Journal of Information and Computational Science
Volume 10 Issue 5 - 2020
ISSN: 1548-7741
www.joics.org532
Fig. 10 Before Update Fig. 11 After update
4. Result and Discussion
Python and OpenCV technologies are used to implement the proposed methodology. For the
real time system, the CCTV camera provides input or video file can be provided as an input to
offline system. The proposed methodology using YOLOV-3 has efficiently detected the
available parking slots. The Slots are marked as green or red indicating non-occupied and
occupied slots respectively.
YOLOv2 uses Darknet-19 and YOLOv3 uses Darknet-53 to extract features. YOLO v2 used a
custom deep architecture darknet-19, an originally 19-layer network supplemented with 11 more
layers for object detection. With 30-layer architecture, YOLO v2 often struggled with small
object detections.Yolov3 - Better at detecting smaller objects. No residual blocks, no skip
connections and no up sampling.
Journal of Information and Computational Science
Volume 10 Issue 5 - 2020
ISSN: 1548-7741
www.joics.org533
Sr. No. Model mAP fps
1. SSD 300 41.2 46
2. SSD 500 46.5 19
3. YOLOv2 48.1 40
4. R-FCNN 91.9 12
5. DSSD513 53.3 6
6. FPN FRCN 59.1 6
7. YOLOv3 55.3 35
Table 1. Comparison between different models.
YOLO v3 incorporates all of these. YOLO v3 uses a variant of Darknet, which originally has
53 layer network trained on Imagenet. For the task of detection, 53 more layers are stacked
onto it, giving us a 106 layer fully convolutional underlying architecture for YOLO v3. This is
the reason behind the slowness of YOLO v3 compared to YOLO v2. [15] [16]
As compared SSD and YOLOv2, the accuracy (i.e. mAP) is less than YOLOv3 and speed of
detection is similar. For other models like R-FCNN, the accuracy is similar but it takes more
time for detection as compared to YOLOv3 (Table 1).
Fig. 12 Parking slot is occupied.
Journal of Information and Computational Science
Volume 10 Issue 5 - 2020
ISSN: 1548-7741
www.joics.org534
We have tested our system on a sample video. The red patch in Fig.12 shows that the
respected parking slot is occupied (because of IOU is higher than threshold). In Fig. 13, the
green patch is shown as that parking slot gets vacant and further it updates into database and app.
Fig. 13 Parking slot is vacant
5. Conclusion
In this paper, we address the issue of parking and present an approach for smart parking
system which is based on image processing. The system that we propose provides real time
information regarding availability of parking slots in a parking area. This paper enhances the
performance of saving users time to locate an appropriate parking space and also helps to resolve
the growing problem of traffic congestion in cities. This system is used to improve the parking
facilities of a city.
6. References
[1] Zhang, Shifeng "Single-Shot Refinement Neural Network for Object Detection".
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
"Firebase Real-time Database". Firebase, Inc dated 18/3/17.
[2] YOLOv3: An Incremental Improvement, Joseph Redmon, Ali Farhadi University of
Washington, 2018
[3] https://www.pyimagesearch.com/2018/11/12/yolo-object-detection-with-opencv
[4] ‘EasyPark Group - Parking App’, https://easyparkgroup.com/, accessed 05-March-2017
Journal of Information and Computational Science
Volume 10 Issue 5 - 2020
ISSN: 1548-7741
www.joics.org535
[5] Liu, Wei (October 2016). SSD: Single shot multibox detector. European Conference on
Computer Vision. Lecture Notes in Computer Science. 9905.
[6] Redmon, Joseph (2016). "You only look once: Unified, real-time object detection".
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
[7] Girschick, Ross (2015). "Fast R-CNN" (PDF). Proceedings of the IEEE International
Conference on Computer Vision
[8] Shaoqing, Ren (2015). "Faster R-CNN". Advances in Neural Information Processing
Systems.
[9] Ross, Girshick (2014). "Rich feature hierarchies for accurate object detection and semantic
segmentation" (PDF). Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition. IEEE: 580–587. arXiv:1311.2524. doi:10.1109/CVPR.2014.81. ISBN 978-1-
4799-5118-5.
[10] Yusnita, R.; FarizaN. ;Norazwinawati B.; “Intelligent Parking Space Detection System
Based on Imag Processing,” International Journal of Innovation, Management and
Technology, Vol. 3, No. 3, June 2012.
[11] Zhang Bin; Jiang Dalin; Wang Fang; Wan Tingting; , "A design of parking space detector
based on video image," Electronic Measurement & Instruments, 2009.
[12] Boda, V.K.; Nasipuri, A.; Howitt, I.;, "Design considerations for a wireless sensor network
for locating parking spaces," SoutheastCon, 2007.
[13] Dalal, Navneet (2005). "Histograms of oriented gradients for human detection" (PDF).
Computer Vision and Pattern Recognition.
[14] Ming-Yee Chiu; Depommier, R.; Spindler, T., "An embedded realtime vision system for 24-
hour indoor/outdoor car-counting applications," Pattern Recognition, 2004
[15] https://towardsdatascience.com/yolo-v3-object-detection-53fb7d3bfe6b
[16] https://medium.com/@venkatakrishna .jonnalagadda/object-detection-yolo-v1-v2-v3-
c3d5eca2312
Journal of Information and Computational Science
Volume 10 Issue 5 - 2020
ISSN: 1548-7741
www.joics.org536