![Page 1: Drowsiness Detection using Contactless Heart Rate ......Rate Monitoring and Eye Tracking Kartik Prabhu Bradley Barnhart Arun Seetharaman Motivation: Driving while fatigued is very](https://reader034.vdocument.in/reader034/viewer/2022042600/5f4ea1c6c1f15b4da756a99c/html5/thumbnails/1.jpg)
Drowsiness Detection using Contactless Heart Rate Monitoring and Eye Tracking
Kartik Prabhu Bradley Barnhart Arun Seetharaman
● Motivation: Driving while fatigued is very dangerous and leads to 10-30% of road deaths (Rios-Aguilar et al). Biometric indicators can be used to detect drowsiness in drivers.
● Ocular indicators of drowsiness include blink rate, blink duration, and percentage of eye closure (Ftouni et al).
● Heart rate variability is another useful indicator, which can be measured by analyzing changes in RGB components of images of the face (Rios-Aguilar et al).
![Page 2: Drowsiness Detection using Contactless Heart Rate ......Rate Monitoring and Eye Tracking Kartik Prabhu Bradley Barnhart Arun Seetharaman Motivation: Driving while fatigued is very](https://reader034.vdocument.in/reader034/viewer/2022042600/5f4ea1c6c1f15b4da756a99c/html5/thumbnails/2.jpg)
Eye Detection- Training for fisher imageData Collection▪ OpenCV to segment a bounding box around the eyes ▪ (1) 100 images of open and closed eyes taken at 5 different illuminationsTraining ▪ (2) Global histogram equalization▪ (4) Alignment to “good” image and crop
› Subsequent alignment to (3) meaneye ▪ Top 100 eigenimages ▪ (5) Fisherimage
(1) (2) (3) (4)
(5)
![Page 3: Drowsiness Detection using Contactless Heart Rate ......Rate Monitoring and Eye Tracking Kartik Prabhu Bradley Barnhart Arun Seetharaman Motivation: Driving while fatigued is very](https://reader034.vdocument.in/reader034/viewer/2022042600/5f4ea1c6c1f15b4da756a99c/html5/thumbnails/3.jpg)
Eye Detection- TestingData Collection▪ (1) 100 total images of open and closed eyes Accuracy▪ 100% accuracy, but...▪ More spreading than training case
Training Data Projections
Testing Data Projections
![Page 4: Drowsiness Detection using Contactless Heart Rate ......Rate Monitoring and Eye Tracking Kartik Prabhu Bradley Barnhart Arun Seetharaman Motivation: Driving while fatigued is very](https://reader034.vdocument.in/reader034/viewer/2022042600/5f4ea1c6c1f15b4da756a99c/html5/thumbnails/4.jpg)
Eye Detection- Demonstration
![Page 5: Drowsiness Detection using Contactless Heart Rate ......Rate Monitoring and Eye Tracking Kartik Prabhu Bradley Barnhart Arun Seetharaman Motivation: Driving while fatigued is very](https://reader034.vdocument.in/reader034/viewer/2022042600/5f4ea1c6c1f15b4da756a99c/html5/thumbnails/5.jpg)
General Heart Rate Procedure● Detect face using Viola-Jones detector● Extract cheek area as ROI
○ Currently using dlib’s face landmark detector○ Computationally expensive, will try moving to a simple algorithm
using the face bounding box
![Page 6: Drowsiness Detection using Contactless Heart Rate ......Rate Monitoring and Eye Tracking Kartik Prabhu Bradley Barnhart Arun Seetharaman Motivation: Driving while fatigued is very](https://reader034.vdocument.in/reader034/viewer/2022042600/5f4ea1c6c1f15b4da756a99c/html5/thumbnails/6.jpg)
Heart Rate via ICA▪ Separate color channels into independent signals
› In past experiments one of the signals was BVP▪ Take FFT and look at magnitude to determine dominant frequency▪ Tried bandpass filtering, looking at specific regions and averaging▪ Trained on one video and kept unmixing matrix
![Page 7: Drowsiness Detection using Contactless Heart Rate ......Rate Monitoring and Eye Tracking Kartik Prabhu Bradley Barnhart Arun Seetharaman Motivation: Driving while fatigued is very](https://reader034.vdocument.in/reader034/viewer/2022042600/5f4ea1c6c1f15b4da756a99c/html5/thumbnails/7.jpg)
Heart Rate via ICA
![Page 8: Drowsiness Detection using Contactless Heart Rate ......Rate Monitoring and Eye Tracking Kartik Prabhu Bradley Barnhart Arun Seetharaman Motivation: Driving while fatigued is very](https://reader034.vdocument.in/reader034/viewer/2022042600/5f4ea1c6c1f15b4da756a99c/html5/thumbnails/8.jpg)
Heart Rate via ICA Results▪ Very imprecise
› Measurement varies significantly for a steady video› Standard deviation around 11 bpm for database videos and around 40
bpm for webcam
![Page 9: Drowsiness Detection using Contactless Heart Rate ......Rate Monitoring and Eye Tracking Kartik Prabhu Bradley Barnhart Arun Seetharaman Motivation: Driving while fatigued is very](https://reader034.vdocument.in/reader034/viewer/2022042600/5f4ea1c6c1f15b4da756a99c/html5/thumbnails/9.jpg)
Heart Rate via CHROM● Temporally normalize RGB signals● Assume a standardized skin-tone and project into plane without
specular reflection component● Signal is defined as difference of chrominance signals
G. Haan, V. Jeanne, “Robust Pulse Rate From Chrominance-Based rPPG”
![Page 10: Drowsiness Detection using Contactless Heart Rate ......Rate Monitoring and Eye Tracking Kartik Prabhu Bradley Barnhart Arun Seetharaman Motivation: Driving while fatigued is very](https://reader034.vdocument.in/reader034/viewer/2022042600/5f4ea1c6c1f15b4da756a99c/html5/thumbnails/10.jpg)
Heart Rate via CHROM Results
Time Domain Frequency Domain
Ground Truth: 70bpm
Histogram of BPM
Relatively decent results for both dataset and webcam, within 10bpm
![Page 11: Drowsiness Detection using Contactless Heart Rate ......Rate Monitoring and Eye Tracking Kartik Prabhu Bradley Barnhart Arun Seetharaman Motivation: Driving while fatigued is very](https://reader034.vdocument.in/reader034/viewer/2022042600/5f4ea1c6c1f15b4da756a99c/html5/thumbnails/11.jpg)
Conclusions● Much more difficult than initially thought● Originally planned for Android but canceled● Despite many similar implementations heart rate was very difficult
○ Current implementation cannot determine heart rate variability● Eye detection effective but limited
○ Requires training for individual users ● A good first step toward a drowsiness detector
○ Goal was not to make a detector but to provide tools for it