a low cost design to detect drowsiness of …€¦ · driver drowsiness detection system as road...

http://www.iaeme.com/IJCIET/index.asp 1138 [email protected]

International Journal of Civil Engineering and Technology (IJCIET) Volume 8, Issue 9, September 2017, pp. 1138–1149, Article ID: IJCIET_08_09_128

Available online at http://http://www.iaeme.com/ijciet/issues.asp?JType=IJCIET&VType=8&IType=9

ISSN Print: 0976-6308 and ISSN Online: 0976-6316

© IAEME Publication Scopus Indexed

A LOW COST DESIGN TO DETECT

DROWSINESS OF DRIVER

Suganya G*, Premalatha M, Bharathiraja S, Rohan Agrawal

School of Computing Science and Engineering

VIT University, Chennai

ABSTRACT

The aim of the paper is to design an economical system that monitors the

drowsiness of the driver. The system will use a camera to monitor the driver’s eyes. To

achieve this, the system will capture video from a day – night sensitive camera and use

image processing algorithms running on a Raspberry Pi board to detect any

drowsiness in eyes. This system focuses on development in the most economical way

so that it can be implemented in comparatively lower end vehicles too which form the

larger crown on roads and prevent loss of human life through accidents of such types.

Keywords: Raspberri Pi, Image Processing.

Cite this Article: Suganya G, Premalatha M, Bharathiraja S and Rohan Agrawal, A

Low Cost Design to Detect Drowsiness of Driver, International Journal of Civil

Engineering and Technology, 8(9), 2017, pp. 1138–1149.

http://www.iaeme.com/IJCIET/issues.asp?JType=IJCIET&VType=8&IType=9

1. INTRODUCTION

The aim of this paper is to develop a drowsiness detection system. The focus will be placed

on designing a system that will accurately monitor the open or closed state of the driver’s eyes

in real-time. Detection of eyes involves a sequence of images of a face, and the observation of

eye movements. This paper is focused on the localization of the eyes, which involves looking

at the entire image of the face, and determining the position of eyes, by a self-developed

image-processing algorithm. Once the position of the eyes is located, the system is designed

to determine whether the eyes are opened or closed. The paper also compares other

methodologies and the advantages of this method over others. Making the system feasibly

economical is also an important focus.

2. BACKGROUND

Drive or travel for long distance is very tough for every driver. Before the start of journey,

driver should prepare and get the rest at sufficiently so that driver can reach his/her

destination safely. Driver Drowsiness is one of the main causes of Road Accidents. Recent

statistics [7] estimate that annually 1,200 deaths and 76,000 injuries can be attributed to

fatigue related crashes. In order to prevent drivers from drowsy driving, there is a huge

Suganya G, Premalatha M, Bharathiraja S and Rohan Agrawal


demand for safety driving systems. Because of the hazard that drowsiness presents on the

road, methods need to be developed for counteracting its affects.

There are many road safety products designed to prevent road accidents due to driver

drowsiness but most of them are either costly or produce false alarms or are not available

easily in India. The Driver Drowsiness Detection System being implemented in this paper

aims at being easily available, economical and useful for most vehicles.

3. RELATED WORK

Some systems [8] have been suggested and even implemented in some of the commercial

vehicles for drowsiness detection. Still, these systems struggle to find major takers due to the

various drawbacks that each of them have.

Attention Assist is a program that watches a driver's steering input at the beginning of a

trip which contains the pressure of the hands on the steering and also the changes in the

movement of the steering wheel. A pattern is calculated for regular driving behavior of the

user and saved in the database. When there is a drastic change in the current driving behavior

as compared to the preset, the system generates an alarm or warning to alert the driver. The

most important con of this system is that the technology that it uses is very costly and

proprietary as a result of which it is offered only in luxurious cars. The system is also very

difficult to implement and algorithm for pattern understanding is still naïve and debatable.

Anti-Sleep Pilot [9] that is basically psychological implementation with underlying

technology, it makes a general study about the psychological and the physical behavior of the

driver that involves the driver sleeping activities, age and other driver activities that are

needed to be covered in order to generate and set the threshold values. It perhaps presents a

line of ascending lights that show the risk of the driver falling asleep based on the 26 fatigue

factors collected from the driver. The driver sets a risk number on the device, and it calculates

a safe driving time before a break is required in order to drive safe.

Drive Awake (from Cafe Amazon) [10] uses eye-tracking technique to keep the driver

from falling asleep while driving. This iPhone app called Drive Awake uses eye-tracking

technology to keep tabs on driver while driving. If the driver closes eyes for too long, the app

emits a shrieking audible alarm. This App works by using an iPhone’s front-facing camera to

monitor driver’s eye.

Anti-Sleep Alarm [2] measures the conductivity of the skin (electro-dermal activity -

EDA). EDA reflects brain activity. Electrical conductivity of the skin varies, depending on the

activity of the brain. The system takes 3-5 minutes to measure driver’s current brain activity

and then driver fatigue is estimated from this level. A module has to be worn on fingers.

Motion Sensors Alarms are placed over the driver’s ear and it alarm is set on if driver’s

head bobs below a certain angle. It is similar to a mercury switch. If driver dozes off, a loud

alarm will wake driver and passengers.

Issues associated with all these systems

Attention Assist uses a complex algorithm which analysis around 50 factors which helps in

identifying driver’s drowsiness. The algorithm is patented and it requires in depth research to

determine how these factors affect drivers’ drowsiness. It also requires heavy computing.

Electro dermal activity – EDA is a patent technology by STOPSIEOP. So to make this

product, it needs a lot of permissions. The sensors required to set up the system are also rarely

available. Hence it becomes a costly affair.

Motion Sensors Systems are not accurate and their false alarm rate is very high. It triggers

alarm even if driver simply turns his neck.

A Low Cost Design to Detect Drowsiness of Driver


The proposed system’s advantage over all above technologies is that, it is built by open

source utilities and its accuracy is also high with less false alarm rate. Major factor is that its

cost is very less than existing systems like above. So that it can fitted in all classes of vehicles

Driver Drowsiness Detection System as Road Safety Device

The system that has been developed is cheaper as compared to the other systems available in

the market. As for the development of the proposed system, the use of open source libraries

that are easily available on the web without any cost has been done. The system contains a

well proposed algorithm that clearly detects the face and the eye based on the threshold it sets

for the detection. This reduces the overhead and makes it cheaper as compared to systems like

Attention Assist. The efficiency of other systems available in the market depends upon the

exhaustive study of the driver’s behavior which needs large data set that builds up with time.

Real time applications can’t afford to wait for computing large overheads of such systems. On

the other hand system designed here does not require any exhaustive study and it efficiently

manages the driver security and provides the optimum result.

4. TECHNICAL SPECIFICATION

The Driver Drowsiness Detection System uses Raspberry Pi running on 5V power source. It

is the primary processing unit for the proper working of the system and which is suitable for

most of the vehicles. It also requires an optimum light source. In terms of frequency it

corresponds to a band in the vicinity of 400nm to 930nm. The camera should take High

Quality Pictures in order to take the video clearly and the frame rate for the capture of the

video must be more than 70 frames per minute.

The camera which can be connected to GPU of Raspberry Pi would be well suited for the

system to make the processing faster. To overcome this issue, the most suited camera to be

used is Pi Cam or PI NOIR [11]. It can reduce the processing instructions for the processor as

compared to webcam since the graphical processing unit stores the image and can carry out

many Image Processing Operations which do not require main processor. Many Cameras that

are available in the market do contain the cut filter that does not allow the light with

wavelength more than 700nm. PI NOIR is different from normal cameras since it can take the

pictures during the night time in the presence of IR lights.

4.1. COMPONENTS

Raspberri Pi

It uses an ARMV6 microprocessor architecture, contains a 512 MB RAM, 700 MHz clock

frequency processor core which is a high processing speed. It has four USB 2.0 ports, one

Ethernet port and a separate Audio Jack.

Raspberri Pi NOIR

It is an infrared sensitive camera that can take pictures during the night time also since it does

not have an IR cut filter like other camera lenses. Pi NOIR camera uses the dedicated CSI

interface, which was designed specifically for interfacing Raspberry Pi with camera. The CSI

bus is capable of extremely high data transfer rates.



4.2. Training Set

To train the classifiers, two set of images are needed. One set contains an image or scene that

does not contain the object, in this case a facial feature, which is going to be detected. This set

of images is referred to as the negative images. The other set of images, the positive images,

contain one or more instances of the object. The location of the objects within the positive

images is specified by: image name, the upper left pixel and the height, and width of the

object. For training facial features 5,000 negative images with at least a mega-pixel resolution

were used for training. These images consisted of everyday objects, like paperclips, and of

natural scenery, like photographs of forests and mountains.

In order to produce the most robust facial feature detection possible, the original positive

set of images needs to be representative of the variance between different people, including,

race, gender, and age. A good source for these images is National Institute of Standards and

Technology’s (NIST) Facial Recognition Technology (FERET) database [12]. This database

contains over 10,000 images of over 1,000 people under different lighting conditions, poses,

and angles. In training each facial feature, 1,500 images were used. These images were taken

at angles ranging from zero to forty five degrees from a frontal view.

Three separate classifiers were trained, one for the eyes, one for the nose, and one for the

mouth. Since it is not possible to reduce the false positive rate of the classifier without

reducing the positive hit rate. The simplest method is to perform facial detection on the image

first. The area containing the face will also contain facial features. However, the facial feature

cascades often detect other facial features.

The best method to eliminate extra feature detection is to further regionalize the area for

facial feature detection. It can be assumed that the eyes will be located near the top of the

head, the nose will be located in the center area and the mouth will be located near the

bottom. The upper 5/8 of the face is analyzed for the eyes. This area eliminates all other facial

features while still allowing a wide variance in the tilt angle. This area eliminates all but the

upper lip of the mouth and lower eyelid. The lower half of the facial image was used to detect

the mouth. Since the facial detector used sometimes eliminates the lower lip the facial image

was extended by an eighth for mouth detection only.

4.3. Shelf Source

OpenCV libraries [13] have been used to build this system. OpenCV along with Haar

algorithm [14] for the detection of the eye have been chosen. The logic of the software is

developed for Driver Drowsiness Detection which detects the number of frames with the

detected face, the number of frames with open eyes, and the number of frames with closed

eyes. Based on these frame set data, system checks whether driver is sleeping or not and

required steps can be taken to alert the driver.

4.4. Work Flow

The video streaming of the face of the driver is taken by the Pi-Cam and the image is sent to

algorithm through the CSI interface using input/output bus. The system detects the face in the

image using the Haar algorithm features i.e. by using two XML files; face xml and eye xml

file. The algorithm then uses eye xml file to detect the eyes on the face. After eyes are

detected, it checks whether the eyes are closed or open. If eyes are closed for more than 4 out

of 10 frames, it send the signal to output device of Raspberry Pi where the buzzer is attached.

The buzzer rings until the driver is awake. Same procedure as above starts again to detect

driver's drowsiness again.



4.5. Real Time Analysis

Average Eye Blinks of person per 60

seconds 15-20 times

Frames Processed Per Second 1.5 to 2.5

Time to capture and process ten frames 4-6 seconds

Average Eye Blinks of person per 4-6

seconds 1 to 2

Alarm Sounds when more than 5 out of 10 frames do not detect open eyes. Maximum

frames in which eye can remain closed due to natural blink is 2 frames. The system sounds an

alarm when it finds less than 5 frames detecting the eye. It means if the eye is detected only

one, two, three or four times out of ten frames, the system produces an alarm. If a person goes

to sleep, the system will produce an alarm every 5 seconds till the person wakes up. In normal

conditions, an eye will not remain closed for more than this time in 5 seconds. Even if the

blink of the eye is captured both times, it still allows two more frames to pass by. The system

will not be able to detect eye when a person goes to sleep or if the lighting conditions become

extreme.

4.6. Performance

It performs better in all lighting condition.

It gives less false alarm rate

Response time is much better than other available system

5. DESIGN APPROACH

5.1. Architectural Design

There are several different algorithms and methods for eye tracking, and monitoring. Most of

them, in some way, relate to features of the eye within a video image of the driver. One

similarity among all faces is that eyebrows are significantly different from the skin in

intensity, and that the next significant change in intensity, in the y-direction, is the eyes. This

facial characteristic is the center of finding the eyes on the face, which will allow the system

to monitor the eyes and detect long periods of eye closure.

Figure 1 A regular Raspberri Pi Board



Figure 2 A schematic diagram

5.2. Camera Sensor

The Drowsy Driver Detection system consists of a camera that takes images of the driver’s

face. The camera is placed in front of the driver, approximately 30 cm away from the face.

The camera must be positioned such that the following criteria are met:

1. The driver’s face takes up the majority of the image.

2. The driver’s face is approximately at the center of the image.

5.3. Processing Unit

Driver Drowsiness System requires an image processing unit to compute the data obtained

from camera in real time. So, the several processing kits were compared to understand which

one is suitable for the application. A normal micro-controller [5] like Arduino (8 bit

processor) or PIC has a processing power of 16 MHz and a RAM of 64KB whereas

microprocessors like Raspberry Pi Model B (32 Bit) or Beagle Board use 700 MHz processor

and 256 MB RAM. Since, real time image processing is required, a kit like Raspberry Pi or

Beagle Board is necessary. Raspberry Pi is chosen owing to its lower cost and good support

for different input devices.

Figure 3 Flowchart of System

5.4. Warning Unit

The Raspberry Pi will issue an audio warning to driver when it detects closed eye for 3-5

continuous frames. A buzzer will be used as an output to produce a high pitched sound to

wake up and alert the driver.



5.5. Power Unit

A Raspberry Pi requires 5V (1200 mA) battery/USB port to power it up for normal functions.

Additional Camera takes up to 250 mA of current. A USB port is sufficient to power up

Raspberry Pi for normal functions. In case of higher power requirements, an external USB

port can be used. Most of the present day cars have USB port which can be used to power up

the Driver Drowsiness Detection System.

5.6. Alarm

The alarm is attached to the output port of the Raspberry Pi which is triggered when required.

Figure 4 Computing Design

5.7. Computing Design

After a facial image input, pre-processing is first performed by binarizing [6] the image. The

top sides of the face are detected to narrow down the area where the eyes exist. Using the

sides of the face, the center of the face is found, which is used as a reference when comparing

the left and right eyes. Moving down from the top of the face, horizontal averages (average

intensity value for each y coordinate) of the face area are calculated. Large changes in the

averages are used to define the eye area.



6. CONSTRAINTS AND TRADE OFFS

10.1. Constraints

Economic

Execution speed of algorithm is a crucial part since, the code has to execute and give output in

real time. Here comes the crucial part of choosing between different processing units

available in market like Raspberry Pi, Arduino, Arm controller, Beagle boards.

Technical

The most important aspect of implementing a machine vision system is the image acquisition.

Any deficiencies in the acquired images can cause problems with image analysis and

interpretation. Examples of such problems are a lack of detail due to insufficient contrast or

poor positioning of the camera: this can cause the objects to be unrecognizable, so the purpose

of vision cannot be fulfilled.

Illumination

A correct illumination scheme is a crucial part of insuring that the image has the correct

amount of contrast to allow to correctly processing the image. In case of the drowsy driver

detection system, the light source is placed in such a way that the maximum light being

reflected back is from the face. It should also be ensured that light source should not distract

the driver. The algorithm mainly depends on the light source. Different parts of objects are lit

differently, because of variations in the angle of incidence, and hence have different

brightness as seen by the camera. Parts of the background and surrounding objects are in

shadow, and can also affect the brightness values in different regions of the object.

Surrounding light sources (such as daylight) can diminish the effect of the light source on the

object.

Camera Hardware

The next item to be considered in image acquisition is the video camera. Review of several

journal articles [3] reveals that face monitoring systems use an infrared-sensitive camera to

generate the eye images. CCD cameras have a spectral range of 400-1000nm, and peak at

approximately 800nm. CCD camera digitize the image from the outset, although that signal

amplitude represents light intensity – the image is still analog.

Trade Off

Optimal illumination is being achieved by the use of infrared sensitive camera instead of web

camera. Among the various higher end cameras available, to make the paper cost-effective, Pi

NoIR is used. The infrared sensitive Pi NoIR is able to capture images in most lighting

conditions without distracting the driver. This optimality ensures the quality of the system as

well as the cost-effectiveness.

When compared to the different processing units available, Raspberry Pi is preferred. It

has optimal computing power when compared to others.

7. PROTOYPE DEVELOPMENT

A working prototype was developed in the lab and tested to be successful in proper detection

of drowsiness.



Input

The input was the video recording of the driver’s face, which was facing the camera present in

the system.

Output

The system produced an output in the form of an audio signal. One had to connect the speaker

to the audio jack of the system.

Figure 5 Driver Drowsiness Detection Systems

To start the system “Driver Drowsiness Detection”, one requires the power source that can

give the constant 5 volts supply to the system, which is easily available in the car. To provide

the power source to the system, there is a special port on the left hand corner names as “Micro

USB 2.0 Power” port.

The next requirement is to add the camera to the system. Raspberry Pi has a special port

named as “CSI Connector Camera,” that takes the input from the camera and transfers it to the

Raspberry Pi for the processing.

The camera takes the stream of video and is transferred to the pi board, with the help of

the necessary algorithm the video stream is converted into the static frame and the frames are

being continuously checked for the eye detection, if the eye is not detected for 5 frames out of

10 frames that system took into the consideration and if the eye is not detected for the 5

frames, then the output is given to the “Audio Out” which is the special port that is use to give

the output to the Raspberry Pi. This port is used to give the output, so in order to give the

output system is making use of “Buzzer” that is connected to the given port.

Figure 6 Power Supply On

After connecting it to the given manner stated above, one needs to log into the Raspberry

Pi by starting the “VNC Server” that requires the authentication. Once the authentication is



provided one gets into the “Raspberry Pi” and once logged into the system, team choose the

file that contains the program that was created using the “Face Detection” algorithm. Once the

file is chosen, by using the terminal one runs the program and the output can be depicted, if

one is using the Laptop/Pc which acts as the monitor for the “Raspberry Pi” once the program

start running one can clearly see that the program is detecting the face and the eyes are also

encircled. If the eye is not detected, then the sound is produced by using the sound system or

the buzzer.

Figure 7 Overall View in Lab

This can be automated as done in the paper that allows the automatic authentication and

the automatic start of the camera that captures the picture of the driver.

Figure 8 Flow Chart Diagram of camera interfacing with Raspberri Pi

Figure 9 Logical Block Diagram



Figure 10 Working of System in Real Time Environment

8. COMAPRISION OF VARIOUS ALTERNATIVES

8.1. Computing Power

In terms of computing power, the situation seems to be clear as well. Most of the Arduino

boards are equipped with an 8-bit microcontroller from ATMEL, usually the ATmega328

which runs at 16 MHz Of course, it cannot compete with the 700 MHz BCM2835 chip which

powers the Raspberry Pi. Even one the latest board from Arduino, the Arduino Due, can’t

compete with the Pi with its 84 MHz SAM3X8E chip. So if it is computing power that one is

looking for, there is one clear winner: the Raspberry Pi platform.

8.2. Inputs/Outputs

This is nearly an easy one. The Raspberry Pi has some decent inputs and outputs of course,

via the GPIO connector, and supports the I2C and SPI interfaces, but these are all digital

connectors. On the other hand, the Arduino Uno board for example has digital inputs/outputs,

but also PWM outputs, analog inputs, and I2C and SPI interfaces. Plus, some recent boards

like the Arduino Due also have analog outputs, which allow one to play sounds directly from

the Arduino board. Of course, one can easily get analog inputs on the Raspberry Pi by using

Analog-to-Digital Converters, but that’s external components. So in this section, the clear

winner is the Arduino.

8.3. Programming

It is easy to program on the two different platforms Arduino and Raspberry Pi. At first, it

would seem that the Arduino is the clear winner: the processing language is really easy to use,

one can write directly the code on computer in the Arduino IDE, and there are thousands of

tutorials out there about how to program Arduino. On the Raspberry Pi, it is not that easy: one

has to log on the device either with the board itself or via SSH from computer, then write the

code, and run it. But the Raspberry Pi already supports many languages, like Python. Not only

the Python language is really easy to use, but this also opens to door to the use of so many

Python libraries that are available on the web, thus extending the range of application of the

Raspberry Pi board. For this last reason, it is a draw again between the two platforms.

8.4. Price

In terms of price, Arduino Uno is cheaper than the Raspberry Pi. If one counts all the different

accessories needed to make the Raspberry Pi usable, and comparing it to the price of the

single USB cable needed to use the Arduino board, the Arduino platform is much cheaper

than the Raspberry Pi.



9. CONCLUSION

The Driver Drowsiness detection System paper is basically a device proposed in order to save

the life of the drivers that are continuously riding the car and are not provided the sufficient

sleep due which severe accidents take place especially in the developing countries like India,

where the number of running vehicles increases every year. The proposed system safeguards

the driver from any accident that take place because of the drowsiness of the driver. The

proposed system is cheap as compared to other systems that are present only in the luxurious

car models. Also, due to its high portability, it can be installed in old cars easily as well. This

is one of the most important and effective feature of the system that make it practical.

It is able to detect if the eye is closed or open and based on it, issues warning to the driver.

The eye detection capability can be increased using a hybrid of different algorithms which

uses edge detection techniques, machine learning concepts, and good support from open

source libraries like Open CV.

A prototype of the proposed concept was also prepared and proof of concept was created.

The system is cheap, easy to install and much more efficient than any other present systems.

REFERENCES

[1] Jalal A. Nasiri, Sara Khanchi, Hamid R. Pourreza Ferdowsi, Eye Detection Algorithm on

Facial Color Images, University of Mashhad, Mashhad, Iran, IEEE, June 2008

[2] Dr.Suryaprasad J, Sandesh D, Saraswathi V, Swathi D, Manjunath S, Real Time Drowsy

Driver Detection using Haarcascade Samples, PES Institute of Technology-South Campus,

Bangalore, India, Academy and Industry Research Collaboration Centre, 2005

[3] David Balya and Tamas Roska, Face and Eye Detection by CNN Algorithms, Computer and

Automation Research Institute, Budapest, Hungary, Journal of VLSI Signal Processing

Systems - Special issue on spatiotemporal signal processing with analog CNN visual

microprocessors, 1999

[4] Peng Wang, Matthew B. Green, Qiang Ji, Automatic Eye Detection and Its Validation, San

Jose State University, California, United States of America, 2005

[5] Singh Himani Parmar, Mehul Jajal, Yadav Priyanka Brijbhan, Drowsy Driver Warning

System Using Image Processing ,Gujarat Engineering College, Bharuch, Gujarat, India,

International Journal Scientific Engineering and Technology Research , April 2014

[6] Saad A. Sirohey, Azriel Rosenfeld, Eye detection in a face image using linear and nonlinear

filters, University of Maryland, United States of America, Elsevier,2001

[7] Manisha Ruikar, National statistics of road traffic accidents in India, 2016

[8] https://en.wikipedia.org/wiki/Driver_drowsiness_detection#Systems

[9] http://newatlas.com/anti-sleep-pilot-monitors-driver-fatigue/17439/

[10] http://www.cultofmac.com/228393/drive-awake-is-the-first-ios-app-that-will-keep-you-from-

falling-asleep-at-the-wheel/

[11] https://www.raspberrypi.org/products/pi-noir-camera/

[12] http://www.itl.nist.gov/iad/humanid/feret/feret_master.html

[13] http://opencv.org/

[14] https://en.wikipedia.org/wiki/Haar-like_features

[15] P. Dinesh Kumar and Dr. B. Rosiline Jeetha, Unaffected Serial Prophecy based Filter

Technique (USP-FT) for Noise Removal in Facial Expression Recognition Images.

International Journal of Civil Engineering and Technology, 8(4), 2017, pp. 497–506.

[16] A.Hemlata, Mahesh Motwani, Single Frontal Face Detection by Finding Dark Pixel Group

and Comparing Xy-Value of Facial Features, International Journal of Computer Engineering

& Technology (IJCET), Volume 4, Issue 2, March – April (2013), pp. 471-481

a low cost design to detect drowsiness of …€¦ · driver drowsiness detection system as road...

Documents