Drowsiness Detection System Using Heartbeat Rate in Android-

based Handheld Devices

Advisor : Dr. Kai-Wei KePresenter : D. Jayasakthi

Department of Electrical Engineering and Computer Science

6/24/2013 2

Contents• Introduction• Motivation• Objective• Methodology• Results• Conclusion

6/24/2013 3

Introduction• Driver drowsiness is a major cause of traffic crashes.

• Drowsy driving is a serious issue in our society not only because it affects those who are driving while drowsy, but because it puts all other road users in danger.

• Therefore, the use of assisting systems that monitor a driver’s level of vigilance is important to prevent road accidents.

• These systems should then alert the driver in the case of drowsiness or inattention

6/24/2013 4

Motivation

• A common activity in most people’s life is driving; therefore, making driving safe is an important issue in everyday life.

• Even though the driver’s safety is improving in road and vehicle design, the total number of serious crashes is still increasing.

• Most of these crashes result from impairments of the driver’s attention.

6/24/2013 5

Motivation• Drowsiness detection can be done in various ways based on

the results of different researchers.

• The most accurate technique towards driver fatigue detection is dependent on physiological phenomena like brain waves, heart rate etc.

• Also different techniques based on the behaviors can be used, which are natural and non-intrusive.

• These techniques focus on observable visual behaviors from changes in a human’s facial features like eyes, head and face.

6/24/2013 6

Objective

• The aim of the thesis is develop a prototype for drowsiness detection system.

• The application is developed using the android SDK and it will detect the heart beat signals from the i_Mami-HRM2 heart rate monitoring device.

• ECG signal obtained from the sensor is analyzed in time domain and frequency domain.

6/24/2013 7

Objective• In frequency domain, the power spectral density (PSD)

is found.

• From the PSD the Low Frequency(LF) to High Frequency(HF) ratio is estimated.

• It is found that the LF/HF ratio decreases as the person becomes sleepy.

• As a result the drowsiness of a person can be detected from this power ratio.

6/24/2013 8

How it Works

• Autonomic Nervous System (ANS) activity presents alterations during stress, extreme fatigue and drowsiness.

• Wakefulness states are characterized by an increase of sympathetic activity and/or a decrease of parasympathetic activity.

• Extreme relaxation states are characterized by an increase of parasympathetic activity and/or a decrease of sympathetic activity.

6/24/2013 9

How it Works• The ANS activity can be measured non-invasively from the

Heart Rate Variability (HRV) signal obtained from ECG.

• Power on low frequency (LF) band (0.04-0.15Hz) is considered as a measure of sympathetic activity.

• Power on high frequency (HF) band (0.15-0.4 Hz) is considered of parasympathetic origin in classical HRV analysis.

• Balance between sympathetic and parasympathetic systems is measured by the LF/HF ratio.

6/24/2013 10

Methodology• Various methods that has been implemented are:• Bluetooth module• ECG• Measuring Heart beat• Heart Rate Variability

• Various Signal Processing Methods applied to the ECG signals are:• Decimation• Hamming Window• Fast Fourier Transform• Calculate the low to high frequency ratio

6/24/2013 11

I-Mami HRM2 and Android Phone

I-Mami HRM2 sensor from Microtime Computer Inc.

Garmin Asus A50

6/24/2013 12

Pairing the Sensor with the Mobile

• First the device discovery is done in order to connect the sensor with the mobile.

• If a device is discoverable, it will respond to the discovery request by sharing some information, such as the device name and its unique MAC address.

• Once a connection is made with a remote device for the first time, a pairing request will be automatically presented to the user.

• The user must enter a 4 digit pin number for the device to be paired.

6/24/2013 13

Scan for Bluetooth Devices

Pairing Request

Sensor has been paired with the mobile

6/24/2013 14

Bluetooth Module

A

A

No

Yes

Initialize Bluetooth Socket

Perform a lookup on the remote device in order to match the UUID

UUID - Universally Unique Identifier 

6/24/2013 15

1. Main Screen with all modules

2.Bluetooth Module

3. List of paired device

4. Sensor Connected to the mobile 5. Device not connected

6/24/2013 16

Display ECG signals• As a result of the electrical stimulation a change in potential

of the order of 1mV can be measured during the cardiac cycle.

• This signal is known as the electrocardiogram (ECG).

• The ECG detector works mostly by detecting and amplifying the tiny electrical changes on the skin that are caused during each heartbeat.

• The I-Mami HRM2 heart rate monitoring device is used to fetch the heart rate of a person and it is displayed in the android mobile with the help of programmable application, developed by using android SDK.

6/24/2013 17

2. Select Sensor from menu

3. Displays the paired devices 4. Displays the ECG

signals

1. ECG Module Main Screen

6/24/2013 18

Displaying the Heart Rate• The heart rate is the number of heart beats per minute.

• Normal heart rate of a human being depends on the age. For example, children will have higher heart rates comparing with the adults.

• This measurement can be done in various ways with respect to time.• 60 seconds (no calculation needed) - most accurate• 15 seconds (multiply by 4)• 10 seconds (multiply by 6)• Less than 10 seconds = less precise

6/24/2013 19

1.Heart Rate MeasurementModule

2. Select a device from menu

3. Lists the paired device

4. Displays the heart rate and other values.

6/24/2013 20

Heart Rate Variability• Heart rate variability (HRV), known as the variation of the period between consecutive

heartbeats over time.

• HRV refers to the variations in the beat intervals or correspondingly the instantaneous HR.

• In time domain analysis, based on beat to beat or NN intervals some variables are analyzed. They are

• SDNN: Standard Deviation of all normal to normal intervals index. Often calculated over a 24-hour period.

• SDANN, the standard deviation of the average NN intervals calculated over short periods, usually 5 minutes. SDANN is therefore a measure of changes in heart rate due to cycles longer than 5 minutes.

• NN50: Number of pairs of adjacent NN intervals differing by more than 50 ms in the entire recording

• pNN50: The proportion of NN50 divided by total number of NNs.

• AVNN: Average of all NN intervals.

6/24/2013 21

1. Heart Rate Variability

2. Select a device from menu

3. Select the sensor

4.Displays the HRV

6/24/2013 22

Method to Detect DrowsinessDrowsine

ss Detection

Obtain ECG

signal from

sensorReduce

the sampling rate to 50

Hz

Apply Hamming Window

Apply FFT

Calculate LF/HF ratio

Is Ratio Decreasing

Person Becomes Drowsy

Person is not drowsy

No

Yes

6/24/2013 23

Decimation• Consider a band-limited discrete-time signal x(m) with a base-band spectrum

X(f).

• The sampling rate can be decreased by a factor of L through discarding of L–1 samples for every L samples of x(m).

• Decimation by a factor of L can be achieved through a two-stage process of:

(a) Low-pass filtering of the zero-inserted signal by a filter with a cutoff frequency of Fs/2L, where Fs is the sampling rate.

(b) Discarding of L–1 samples for every L samples

• The decimation factor is simply the ratio of the input rate to the output rate. It is usually symbolized by "M", so input rate / output rate=M.

6/24/2013 24

Decimation

• The sampling frequency of the sensor was 250 Hz which means 250 samples per second.

• It was very high to process the ECG signals.

• So the sampling frequency was reduced by 50 Hz which means 250/50 = 5 samples per second .

• The decimation was done using a low pass filter technique.

6/24/2013 25

Hamming Window Technique• Windowing functions, enhances the ability of an FFT to extract spectral

data from signals.

• Windowing functions act on raw data to reduce the effects of the leakage that occurs during an FFT of the data.

• There are many window functions available.

• For an ECG signal the appropriate window function is the Hamming Window.

• The formula for Hamming window is w(n)=0.54−0.46cos(2πn/N−1).

• If x(n) is the signal ,then we get the windowed signal by multiplying x(n) with the w(n) .

6/24/2013 26

Fast Fourier Transform(FFT)• The FFT is a highly elegant and efficient algorithm, which is still one of the

most used algorithms in speech processing, communications, frequency estimation, etc

• Basic radix-2 algorithm is used which requires N to be a power of 2.

• FFT is applied to the windowed ECG signal.

• By applying FFT , the power spectrum was found .

• LF/HF ratio is calculated every 1 minute .

• If this ratio decreases then the person in becoming drowsy.

6/24/2013 27

Results- While Awake

6/24/2013 28

Results – While Asleep

6/24/2013 29

Conclusion• A non-obstructive, real-time, continuous monitoring method

for determining the drowsiness of the driver has been described .

• From the results it is clear that the LF/HF ratio decreases when the person is sleeping.

• Since ECG is one of the most easy to use physiological signals, a definite relation between drowsiness and HRV may lead to safer driving.

• By applying FFT , the computational complexity is reduced.

6/24/2013 30

Reference• S. Hu and R. Bowlds, "Pulse wave sensor for non-intrusive driver's

drowsiness detection," in Engineering in Medicine and Biology Society, 2009. EMBC 2009. Annual International Conference of the IEEE, Minneapolis, MN, 2009.

• G. Furman, A. Baharav, C. Cahan and S. Akselrod, "Early detection of falling asleep at the wheel: A Heart Rate Variability approach," Computers in Cardiology, pp. 1109-1112, 2008.

• S. Elsenbruch, M. Harnish, and W. C. Orr, “Heart rate variability during waking and sleep in healthy males and females,” Sleep, vol. 22, pp.1067-1071, 1999.

6/24/2013 31

Thank You