2007 IEEE International Symposiumon Signal Processing and Information Technology

Drowsy Driver Assistant System

M. ElSabrouty, A. Hamdy, A. Fawky and S. Khalil'1 Faculty ofInformation Engineering and Technology

Communication DepartmentGerman University in CairoNew cairo, Cairo, Egypt.

E-mail. maha.elsabrout 4. uc. edu.egahmed elsayedna~student.guc. edu._!eg

abdeiattah.solimanaa-gdn.uc d.esherifkhail/&student.guc. edu.!eg

Many applications were developed for face detectionAbstract- This paper proposes a new approach to help the which is Neural Networks (NN), Principle Component

drowsy drivers on the roads by developing a real time Analysis (PCA), wavelets, edge detection, Hough transform,computerized system which has the ability to detect the face and ' . ' . ' .

a

to know the status of the eyes, if the eyes are closed the system and template matching and geometrical matching are somestrikes out an alarm to alert the driver. We used PCA and skin techniques taken for major approaches. Neural networks andtone for face detection. For eye detection we used edge detection, matching algorithms needs a huge training set of face andwavelets transform and YCbCr transformation. The algorithm non-face images and take a long time in execution asperforms well regardless the face position, lighting varieties andimmune to any distracting noise. The designed algorithm is able discussed in [2], [7] & [18]. Principle component analysisto attain 90% accuracy when it was tested on different takes only gray scale images as well as it is used fordatabases. recognition as discussed in [5] [6], [10], [13], [15], [20] &

[21]. Edge detection is severely affected by light condition asKeywords-Drowsy driver, edge detection, eye detection, face in [3], [4] & [8]. As for wavelets it uses a big size window so

detection, Principal Component Analysis, skin tone , YCbCr,wavelets transform it cannot detect small face and fail in extreme lighting

condition as discussed in [10] & [22]. In the other hand, eyedetection techniques are not as popular as face detection

I. INTRODUCTION techniques or it only works with certain conditions. So wehad to introduce a new algorithm that could detect the eyes

Sleeping while driving happens more often more than within the face region.most of the people think. According to the United State As our application is online and thus is governed byDepartment of Transportation (USDT) [11I] & [14], execution deadlines, we have proposed a system to improvedrowsiness is responsible for 10% to lO0% percent of the 20millsionesscaraiesontswhicfoccueachto yearcenti the U, the driver's safety by detecting the status of the eye using asmillion car accidents which occur each year in the US, PA&si-oefrac dtcintomiizththerefore, developing an alarm system is a crucial task. It is PCA & skin-tone for face detection to minimize thethus important to develop a fast alarm system that will searching region for the eyes. For eye detection, we havenotify drivers upon drowsiness and yet produce as few false proposed using wavelets transformation and edge detectionalarms as possible. with the addition of YCbCr transformation. At last, we use

The human face can be utilized as a preprocessing step to Kalman filters for tracking the eyes to continuously know thelimit the search for eyes within a limited space and thus eyes position and status. The whole alarm system isreduce false detection result. Face detection is considered as implemented on the DSP processor, ADSP-BF561 Blackfinthe backbone of many systems as in security systems, face Symmetric Multi-Processor for Consumer Multimedia. Therecognition and human interface applications. In Drowsy microprocessor contains 512 MB of RAMs and 8 MB ofDriver Application, face detection has to be combined with flash memory.an eye detection algorithm to get the desired performance.

978-1 -4244-1 835-0/07/$25.00 ©C2007 IEEE 176

Authorized licensed use limited to: Bharat University. Downloaded on December 20, 2009 at 03:19 from IEEE Xplore. Restrictions apply.

The rest of the paper is organized as follows. Section 11, chrominance of the skin to extract some facial features suchpresents the background on the used algorithms. We present as the mouth or the eyes.our modifications and implementation on the used B. Eye Detection Algorithmsalgorithms in Section III. Section IV presents the simulationand implementation results. Finally, we conclude the paperin Section V. 1) Edge detection

II. THEORETICAL BACKGROUND It may be one of the easiest ways for face or eye detection

[16] that comes into our minds at first as it has beenA. Face Detection Algorithms discussed before by many researchers. The main aim of the

edge detection techniques is to find points in an input digital1) Principle Components Analysis image at which the light intensity changes suddenly

(sharply). These sharp changes in the image can reveal someAs discussed in [13], PCA starts with a random vector x important things from an image like:

with n elements, which contain a sample as second order *Non-uniform surface.statistics (SOS). No probability density of the vectors are *Changes in the property of a material.made, it only matters that the first and second order statistics'~~~~~~~~~~~~~~*Luminance change.are known or can be estimated for vector x, so the elements

*~~~~~~~~~~~*Discontinuities in surface orientation.of x are measurements like pixel gray levels or values of a

signal at different time instants. It is important that the The edge detection for an image consequently reduces the

elements are mutually correlated and there is some size of the image and the amount of data in it; it filters out

redundancy in x which makes the compression possible. some unnecessary information. It is easy to be implemented,PCA transformation starts with centering the vector x by because less processing bandwidth is needed, but, it is very

subtracting its mean: affected by the luminance conditions, works only on

x~x,(-E () grayscale images.This mean is estimated from the available sample

x(1), ... ,x(T). After centering the vector, the mean E{x} will 2) Discrete wavelets transformbe zero. Then, x is linearly transformed to another vector y In this method [1] we are suggesting a new method forwith m elements such that m < n, that leads to induce the

redundancyas the corelationsis remove .Thsi.oeb eye recognition using discrete wavelets transform withfidunding yasrathe orthogtional iscordinatesemove. sh tt te

b

'Haar' as the mother wavelet. In spite of many researchers

elementsaoftatxdinrthegonew coordinateystbeome suncorreatede opinions although it is being computational intensive but itelements of x in the new coordinates become uncorrelated.is powerful as it is based on strong mathematical conceptsHowever, the variances of the projections of x on the newand relations that ensures and proves its reliability and its

coordinate axes are maximized so that the first axiscorresponds to the maximal variance; the second axis expected efficiency as it is robust against different lightingcorresponds to the maximal variance in the direction condition, noise and low computational coast. Wavelets are

used not only in eye recognition but also in many other fieldsorthogonal to the first axis and so on.as it facilitates the analysis of a specific signal. Weexperienced some difficulties in dealing with wavelets and in

2) Skin-Tone Extraction applying some algorithms to recognize the eyes in an image.Due to the variation in luminance, different environmental

Many algorithms dealt before with the skin color and how conditions, visual angle and facial expressions; facial organsto detect faces by finding the skin such in [9], [12], [17] & and features are recognized by this order: eyes, nose and[19]. Most of these techniques were based on finding a face mouth. Factors that must be taken in consideration:pattern by comparing it with a certain mask. However, they eyae e' eyelahes, eylidLs movements, head tiltig,_ makeun, glasses.all start by finding the suitable color space such as YCbCr,HWIU or others. YCbCr is the most used color space as mostOf the previously mentioned algorithms operate on the

177

Authorized licensed use limited to: Bharat University. Downloaded on December 20, 2009 at 03:19 from IEEE Xplore. Restrictions apply.

a) Discretised wavelet transforms face edge detection is preformed to detect the edges in theThe main reason for using this type of wavelets face. 4) The output of the edge is taken as an input to the

transformations is to analyze a specified signal to pick just a wavelet transformation.discrete subset of the upper half plane to be able to A. Face Detection Algorithmreconstruct a signal from the corresponding wavelet . .coefficients. The corresponding discrete subset of the half 1) itnitalizationm r The first step iS initialization, by estimating the eigenfaceplane consists of all the points ,. with integers from the database to get the mean position and size of the

while m: nI E Z. The corresponding baby wavelets are faces. First, we load the name of the training set from a textknown as: file. Then, we get the principle component analysis of the

TM = -fTa - (2) eigenfaces resulting from the basic eigenface algorithm. At

a>1, b>O last by manual processing we identify the mean position andFor a reconstruction of any signal x of finite energy by the size of the face.formula: 2) Training

:(t) = E )X,*''6 3, 4} We load the training set from the list file. Then, thenI.EZ-nEZL (3) training set in cropped to meet the criteria from phase 1.

Then, we apply the algorithm discussed in [6]. At the end,b) Mother wavelet we reduce the dimension of the output by choosing the

In real-life practical applications, the designers and the principle components.researchers prefer the most efficient ways, so they use 1. The first step was getting the mean of the trainingcontinuously differentiable functions with compact support set.as mother (prototype) wavelet (functions). Therefore, to Al

satisfy some analytical requirements we should choose the (8)wavelet functions from a subspace of the space. (8)L(R) n2 , in this space the functions are both 2. Then, we centers the training set around zero by

L2R subtracting the mean from it.absolutely and square integrable:

n t m f-rt./~~~~ ~ ~ ~ ~ ~ ~ ~~~~~D = Fi T(t) I' dt< oc ~~~~~~~~~~(9)

-4tH2dt K (4) 3. Afterwards, we calculate the eigenvectors and theJ K(t)dt < (5) eigenvalues of the covariance the matrix.

It can be ensured that we can formulate the conditions of C Ln= '?3+zero mean and square norm one: = AT

.C (10)_00 0(0t dt=O0 (6) Where A-[1 42 .4M1

f/ K(t)Kdt = (7) 4. Then finally, we get the principle components of

In the case of continuous wavelet transform, the T must the eigenfaces depending on its eigenvalues.be a wavelet. The mother wavelet must satisfy an 3) Image Preprocessingadmissibility criterion to get a stably invertible transform. We insert any color image as an input image. This image isIn the case of discrete wavelet transform, the multi- resized to a 256x256 pixel image. Then, the image isresolution analysis is used in many constructions, which normalized to adjust the lightening condition. Afterwards,defines the wavelet by a scaling function. we apply a skin detection algorithm to eliminate the

background from the resized image. At the end, the image isIII. SYSTEM ALGORITHM transformed into a grayscale image ofPGM format.

Our algorithm contains four main techniques: 1) we uses a 4) Face C i ProjectIonskin-tone extraction as a preprocessing step. 2) Then, we use The gray scaled image is scanned for collecting samples.principle component analysis to successfully detect the face These samples are subtracted by the training set mean. Andusing Yale Face Image Database B. 3) After detecting the

178

Authorized licensed use limited to: Bharat University. Downloaded on December 20, 2009 at 03:19 from IEEE Xplore. Restrictions apply.

then, the output samples are projected on the reduced color in the cropped region and we took the highest 10% ofdimension matrix from the training phase. them to lower the number of eye locations

5) Estimation 8) Locating the eyesFor the final phase, the projection matrix is sorted in After getting 10% of the cropped images we then make

ascending order and we get the smallest projection as our sure that there is no eye position close to the other (wefinal output of the face and draw a rectangle of fixed size choose 5 pixels) then we cropped these position 20X20 ,of150x180 pixel. the eye-position in 512X512 colored image ,to look for white

region in this area which will be identified as the eye if notB. Eye Detection Algorithm found an alarm will sound.

1) NormalizationDue to the undesirable effects of the luminance the

YCbCr code gives us undesirable output, so we decided to IV. EXPERIMENTAL RESULTSuse a normalization code to normalize the luminance of the As we can see in the below output image, acceptable eyeinput picture. detection is performed despite some facial conditions like

2) Edge Detection status of the eyes, the existence of glasses, different skinEdge detection as mentioned was our first approach but as colors, and head positions. This algorithm has proven

we witnessed it was not the best approach when it comes to flexibility and reliability. If we use perfect background andeye detection due to luminance and other factors but after all perfect face detection the efficiency will increase up to 9500it gave us the edges of the eye so we decided that it will beour input to the wavelet transform, so that it will make theeye socket detection easier.

3) YCbCr transformationWe developed an algorithm that has the ability to identify thehuman skin color and it turns it to blue based on certainthresholds. Its main idea that it converts the input RGBimage to YCbCr space, using for loops and conditional if- Rstatements it could set the skin color into blue (value: 255),and it cancels any objects at the background (value: 0).

4) DWTThe main aim of this function is to compute the second

order DWT of an input matrix (image). We did it on twosteps, first we calculated the DWT coefficients using 'haar'method, second, we recomputed the second order DWT ofthe computed coefficients, then we took the absolute value Fig 2 Successful eye detectionthe second coefficients in a matrix (Z). V. CONCLUSION AND FUTURE WORK

5) Performing IDWT We have presented an algorithm for Drowsy DriverComputing the inverse DWT of the image. We calculated the Assistant System that detect skins as a pre-processing stepaddition of the selected node from the wavelet tree and its and then apply PCA rules to detect faces successfully. Themaxima. algorithm begins with initialization and then training which

6) Combination Between YCbCr & Wavelets. are applied only once. Then, we take the input image areGiven the output of the wavelet we combined the wavelet preprocess it to get a skin only image in grayscale. The

to YCbCr by adding the two images together. image is then sampled and project on the training space.

7) Cropping the eye region Finally, an estimation phase is applied to detect the face.After combining the YCbCr with wavelet transform all After getting the face, the algorithm performs edge detection

possible eye position is found but that was it not enough we and wavelet transformation to detect the eye position. At last,still had to chose from the many eye position selected .To do we zoom into the eye position to know the status of the eyethat we cropped 1 Oxi 0 image of the YCbCr of every selected by finding the white components in it.eye position, then we calculated the percentage of white

179

Authorized licensed use limited to: Bharat University. Downloaded on December 20, 2009 at 03:19 from IEEE Xplore. Restrictions apply.

For future work, the algorithm will be experimented in [16] Lindeberg, T., "Edge detection and ridge detection withnight condition images. automatic scale selection", International Journal ofComputer

night condition images. Vision, 30, 2, pp 117--154, 1998.[17] Ng, C., and Pun, T. "Statistical Approach to a Color-based Face

Detection Algorithm", 2002.[18] Rowley, H. A., Baluja, S., and Kanade, T. "Neural Network-

ACKNOWLEDGMENT Based Face Detection". IEEE Transactions on PatternWe would like to thank Eng. Mohamed El Mahdy with Analysis andMachine Intelligence, pp. 23-38, January 1998.

[19] Sandeep, K., and Rajagopalan, A. N. "Human Face Detection inthe Computer Science department, faculty of Media Cluttered Color Images Using Skin and Edge Information".Engineering and Technology, German University in Cairo Indian Institute of Technology, Chennai.for all the support and help that he had shown for us. We [20] Tipping, M. E., and Bishop, C. M. "Probabilistic Principle

Component Analysis". Journal of the Royal Statistical Society,would also like to thank all the researches who published 611-622, 1999.their papers on the internet for access. [21] Turk, M., and Pentland, A. "Eigenfaces for Recognition".

Journal ofCognitive Neuroscience, 71-86, 1991.[22] Wu, B., Ai, H., and Liu, R. "Glasses Detection by Boosting

REFERENCES Simple Wavelet Features". Tsinghua University, Beijing.

[1] Addison, P. S., The Illustrated Wavelet Transform Handbook,Institute of Physics, 2002.

[2] Bianchini, M., and Sarti, L. "An Eye Detection System Basedon Neural Autoassociators". Universita degli Studi di Siena,Siena.

[3] Claypoole, R., Lewis, J., Bhashyam, S., and Kelly,K. "Laplacian Edge Detection". Digital ImageProcessing at Rice University, 1997. Available::http://www.owlnet.rice.edu/-elec539/Projects97/morphjrks/laplacian.html

[4] Claypoole, R., Lewis, J., Bhashyam, S., and Kelly, K."Image Processing: Edge Detection". Digital ImageProcessing at Rice University, 1997. Available:http://www.owlnet.rice.edu/belec539/Projects97/morphjrks/moredge.html

[5] Dailey, M. "Matt's Matlab Tutorial Source Code Page".University of California, San Diego, 1997:http://www.cs.ait.ac.th/-mdailey/matlab/

[6] Delac K., Grgic M., and Grgic S. "Independent ComparativeStudy of PCA, ICA, and LDA". International Journal ofImaging Systems and Technology, 252-260, 1995.

[7] El-Bakry, H. M., and Zhao, Q. "Sub-Image Detection Using FastNeural Processors and Image Decomposition". WorldEnformatika transactions on engineering, computing andtechnology, pp. 85-88, February 2001.

[8] Froba, B., and Kuiblbeck, C. "Robust Face Detection at VideoFrame Rate Based on Edge Orientation Features". FraunhoferInstitute for Integrated Circuits, Erlangen.

[9] Garcia, C., and Tziritas, G. "Face Detection Using QuantizedSkin Color Regions Mergin and Wavelet Packet Analysis".IEEE Transactions on Multimedia, pp. 264-277, September1999.

[10] Hamdy, A. "Drowsy Driver Assistant System: PrincipalComponent Analysis". GUC, Cairo, 2007.

[11] Hampton, C., "DANGER: Drowsy Drivers Can Kill". Available:http://www.paturnpike.com/newsletters/summer96/page2-3.htm

[12] Hsu, R.-L., Abdel-Mottaleb, M., and Jain, A. K. "Face Detectionin Color Images". IEEE Transaction on Pattern Analysis andMachine Intelligence, 696-706, 2002.

[13] Hyvarinen, A., Karhunen, J., and Oja, E. IndependentComponent Analysis. John Wiley & son, Inc., NY, 2001

[14] Knipling & Wang, "Drowsy Driving and AutomobileCrashing",, 1995. Available:

dros 1Whtm1[15] Lee, Y. "Face Detection using PCA".

180

Authorized licensed use limited to: Bharat University. Downloaded on December 20, 2009 at 03:19 from IEEE Xplore. Restrictions apply.