Dr. Babasaheb Ambedkar Marathwada University
Department of Computer Science and Information Technology
Presentation on Research Work Entitled
Heart Rate Variability (HRV) Based Multimodal
Biometric System for Human Authentication and
Identification
Research Student
Nazneen Akhter Shaikh
Research Guide
Prof. K. V. KaleProfessor
Department of Computer Science and Information Technology,
Dr. Babasaheb Ambedkar Marathwada University, Aurangabad
Work Outline
The entire work presented in thesis is
organized in 8 chapters.
The experimental work is presented in 4
chapters.
Every Chapter is independent and self
explanatory.
Findings and results are presented and
discussed.
2
Presentation Outline
• Introduction
• Background
• Literature Survey
• RR-Interval Measurement System
• HRV Based Unimodal System
• ECG Based Unimodal System
• ECG and HRV Based Multimodal System
• Conclusion and Future Work
• Acknowledgement
• Publications
• References3
When we think of our heart rate, we think of a number between
60 to 90 beats per minute (BPM). This number represents our
average heart rate.
In reality, our heart rate changes from beat to beat.
Heart Rate Variability (HRV) a measure of this naturally
occurring variation in the heart rate.
HRV measurements contain more information than
measurements of heart rate alone.
High HRV, a sign of good health.
Hence HRV arising as a simple, non-invasive measure and HRV
analysis the ability to assess overall cardiac health and the state
of the Autonomic Nervous System (ANS).
Introduction Heart Rate Variability (HRV)
4
Figure 1: ProminentComponents of an ECGSignal
Figure 2: R-R Intervalsin an ECG signal(showing beat to beatvariation)
The interval between two adjacent R-R peak of QRS complex known RR-
Interval Which is the only measurement required for HRV Analysis.
Introduction Heart Rate Variability (HRV)
5
Motivation for using Biosignals
May 2013. A Brazilian lady Doctor arrested for using 6 Silicone Fingers to
Hack Hospital Security System to put attendance of 6 senior colleagues
Limitations of Conventional Systems
• They are unique but not confidential and neither secret to the
individual.
• Traditional biometrics can be conveniently manipulated,
spoofed and counterfeited.
6
Alterations
Image Source Fingerprint Alterations Technical Report 2009. A. K. Jain
Motivation for using Biosignals
7
• Besides uniqueness, the biosignals areconfidential and secure to an individual.
• They are difficult to mimic and hard to be copied.
• Therefore, the identity of an individual is unlikelyto be forged, thus preserving the secrecy andprivacy of the users.
• Unlike anatomical biometric modalities that havetwo-dimensional data representation,
• Biosignals are physiologically low-frequencysignals that have one-dimensional datarepresentation.
Motivation for using Biosignals
8
Motivation for Multimodal Biometrics
Limitations of Unimodal Systems• Inefficiency of uni-modal
Biometrics Systems
• Failure to Enroll/ Failure to Acquire
• Easier to Spoof
• Inefficiencies in performance with
time, environment and sensor
variations
Nov 2013
9
Figure : Advantages of Using Multimodality for Biometric Recognition.
Motivation for Multimodal Biometrics
10
Research Problem
11
Research Problem
• Are these parameters/Features Person Specific?
• Are they capable of Identifying an individual in an
Biometric based Authentication System?
• Can HRV parameters act as independent modality in
an unimodal environment?
• Can HRV parameters act as Supportive modality or
Enhance and Improve the performance/limitation of
any traditional modality in an multimodal
environment?
12
Research Objectives1. To study the existing state of the art in relation to Heart Beat Variability (HRV)
with a perspective on using the same for Biometric purpose.
2. To design and develop instrumentation for heart beat detection, signal
conditioning and data acquisition for data transfer to implement biometrics
recognition based on HRV.
3. Experimental setups will be planned to generate an HRV database for
implementation of HRV based biometric recognition system.
4. Suitable preprocessing techniques, feature set generation techniques will be
judiciously selected and necessary feature extraction techniques will be
developed and implemented.
5. Data from other sources will be used for multimodal fusion of other modalities
with HRV for improving efficiency and robustness of traditional modalities.
6. To compile the outcome of the work.13
Background (Functioning of Heart)At any given time the chambers of the heart may be found in one of two states:
• Systole: During systole, cardiac muscle tissue is contracting to push blood out of the chamber.
• Diastole: During diastole, the cardiac muscle cells relax to allow the chamber to fill with blood.
• Blood pressure increases in the major arteries during ventricular systole and decreases during ventricular diastole.
This leads to the 2 numbers associated with blood pressure—systolic blood pressure is the higher number and diastolic blood pressure is the lower number. For example, a blood pressure of 120/80 describes the systolic pressure (120) and the diastolic pressure (80).
14
• The cardiac cycle includes all of the events thattake place during one heartbeat. There are 3phases to the cardiac cycle: atrial systole,ventricular systole, and relaxation.
• Atrial systole: During the atrial systole phase ofthe cardiac cycle, the atria contract and pushblood into the ventricles.
• Ventricular systole: During ventricular systole,the ventricles contract to push blood into the aortaand pulmonary trunk.
• Relaxation phase: During the relaxation phase,all 4 chambers of the heart are in diastole as bloodpours into the heart from the veins.
Background (Functioning of Heart)
15
• The sounds of a normal heartbeat are known as “lubb” and “dupp” and are caused by blood pushing on the valves of the heart. The “lubb” sound comes first in the heartbeat and is the longer of the two heart sounds. The “lubb” sound is produced by the closing of the AV valves at the beginning of ventricular systole.
• During a normal heartbeat, these sounds repeat in a regular pattern of lubb-dupp-pause.
Background (Heart Sounds During a Beat)
16
Background (Heart Related Measurements)
Sr.
No.Method Signal Type
1 ECG Electrical
2 Pulse Oximeter Optical
3 Blood Pressure Pressure
4 Heart Sounds Acoustic
5Laser Doppler
VibrometerySurface Displacement
6 Radar Doppler
7 Motion Imagery Skin Color Fluctuations
Table 1: Heartbeat Sensing Methods.
17
• Task Force of the European Society of Cardiology.
"Heart rate variability standards of measurement,
physiological interpretation, and clinical use."
European Heart Journal 17 (1996): 354-381.
• Acharya, U. Rajendra, et al. "Heart rate variability: a
review." Medical and biological engineering and
computing 44.12 (2006): 1031-1051.
Literature SurveyInception (Base of Idea)
18
Literature Survey
• Da Silva, H. P., Fred, A., Lourenço, A., & Jain, A. K. (2013,
September). Finger ECG signal for user authentication:
Usability and performance. In Biometrics: Theory,
Applications and Systems (BTAS), 2013 IEEE Sixth
International Conference on (pp. 1-8). IEEE.
• Silva, H., Lourenço, A., Canento, F., Fred, A. L., & Jain, A. K.
(2013, February). ECG Biometrics: Principles and
Applications. In BIOSIGNALS (pp. 215-220).
• Lourenço, A., Silva, H., Fred, A. & Jain, A. K. (2011). Unveilingthe biometric potential of finger-based ECG signals.Computational intelligence and neuroscience, 2011, 5.
Gold References for ECG Biometrics
ECG Literature Review 19
RR-Interval Measurement
System
Device Design Considerations, Testing and
Performance Analysis1. System Description
• Design Considerations• Sensors• Controlling Unit• Acquisition Software
2. System Design Testing • Hardware Ectopy Removal• Software Ectopy Removal• Device Calibrations
3. System Performance Testing• RS- Analysis• Fractal Dimension• Poincare Plot
4. System Working21
The device is made of two parts
Sensor Unit
Controlling and Processing
Unit
Heart Beat (Pulse) SensorControlling & Processing Unit
Device Design Considerations, Testing and
Performance Analysis
22
Pulse Sensors Used
The pulse sensors used in our system
works on the Photoplethysmography PPG
signals
It an electro-optic method of measuring
the cardiovascular heartbeat pulse that
found all through the human body
especially at the peripherals like fingertip
toes or ear lobes.
The pulse wave brought about by the
rhythmic throbs of blood vessels in
synchronism with the heartbeat, causing
changes in the flow of blood and the beat
detected from the changing optical density
or opacity.
Low Cost, Easily Available
Reliable and widely Accepted in Clinical
Practice.
Device Design Considerations, Testing and
Performance Analysis
23
A B
Device Design Considerations, Testing and
Performance Analysis
24
Sensor Testing
Oscilloscope TDS 2024 By Tektronix
Dept. of Physics, Dr. Babasaheb Ambedkar Marathwada University.
25
Device Calibration
USB Based Digital Oscilloscope by Hantek26
Device Calibration
27
Device Calibration
S. No
Actual Time
Interval
(ms)
Time recorded
by the device
(ms)
Difference
between
measured and
true values
(ms)
Absolute
Error (ms)
1 2000 2001 + 1 1
2 2000 1999 – 1 1
3 2000 1999 – 1 1
4 2000 2000 0 0
5 2000 1999 – 1 1
6 2000 2001 + 1 1
7 2000 2000 0 0
8 2000 2000 0 0
9 2000 2001 + 1 1
10 2000 2000 0 0
Mean Absolute
Error (ms)0.6
Table 2: Comparison of actual and recorded time interval
RR-Interval measuring system has an
accuracy of measuring of ± 0.3 ms in 2000 ms
or ± 0.15 ms in 1000 ms. 28
Data Acquisition Software
29
Working of Device and Data Samples
From Device Analysis HRV
Parameters
Features
30
HRV Based Unimodal System
Do HRV Data Have Human
Discriminative Property?
32
Do HRV Data Have Human
Discriminative Property?
33
Workflow for HRV based Unimodal
System
34
Database Specification
100 subjects (57 males and 43 Females)
3000 sequences of RR-Intervals
Each subject 30 samples,10 samples in every session.
Each sample has 64 RR-Intervals
Each sample collection duration 30-60 seconds approximately
Three different sessions
Spread over 6 months
Time interval of three months between each session
No Physiological instructions given and
No emotional stimuli introduced while acquiring the pulse data
Data stored in computer files in TXT and CSV format
Session
No.Duration
No of RR-
IntervalsSamples Posture
1, 2, 3Approx.
30-60 Sec64 30
Sitting
Relaxed35
Poincare Maps
• Poincare HRV plot is a graph in which each RRinterval is plotted against next RR interval (atype of delay map) also known as Scatter plot
36
Poincare Maps
37
Poincare Map
SD1: dispersion (standard
deviation) of points
perpendicular to the axis
of line of identity
SD2: dispersion (standard
deviation) of points along
the axis of line of identity
38
Statistical ParametersSr. No. Feature name Description Unit
1 max Maximum RR interval in the time series (ms)
2 min Minimum RR interval in the time series (ms)
3 mean Mean of RR interval time series (ms)
4 median Median of RR interval time series (ms)
5 SDNN Standard Deviation of all RR (NN) Intervals. (ms)
6 SDANNStandard deviation of the averages of RR (NN) intervals in all
segments of the entire recording.(ms)
7 NNxNumber of pairs of adjacent NN intervals deviating by more than x
ms.(count)
8 pNNx NNx count divided by the total number of all RR (NN) intervals. (%)
9 RMSSDThe square root of the mean of the sum of the squares of differences
between adjacent RR (NN) intervals.(ms)
10 SDNNi Mean of the standard deviations of all RR (NN) intervals (ms)
11 meanHR Mean of Heart Rate (bpm)
12 sdHR Standard Deviation of Heart Rate (bpm)39
Feature Selection
• Usually feature extraction/generation techniques yield number of
features which may or may not contribute in proper classification
results.
• These features at times may be redundant or non-informative.
• Presence of such features significantly degrades the classification
performance.
• Therefore selecting a proper feature subset from the original features
set becomes a vital step before any classification attempt.
• Also that less number of features used for classification can
significantly improve the computation time required for classification.
• But finding such a small and efficient subset of features is tricky and
challenging.
40
1. Statistical Dependency (SD)It estimates the statistical dependency between features and
associated class labels using a quantized feature space.
2. Mutual Information (MI)It measures arbitrary dependencies between random variables.
3. Random Subset Feature Selection (RSFS)It aims to discover a set of features that perform better than an
average feature of the available feature set.
4. Sequential Forward Selection (SFS)It initially starting from an empty set, the feature set is iteratively updated by
including the feature which results in maximal score in each step.
5. Sequential Floating Forward Selection (SFFS)It uses SFS as baseline method and further extends by iteratively finding the least
significant features and eliminates it.
Feature Selection
41
Feature Selection
SD MI RSFS SFS SFFS
1 1 73 1 11
2 2 72 2 5
3 3 65 5 36
11 11 66 4 59
4 4 70 11 2
12 12 67 3 3
55 55 68 – –
5 5 71 – –
91 91 78 – –
9 54 69 – –
Table 3: Suggested features from five algorithms in order of relevance.
42
Recognition Rate
12.40
67.90 68.30
19.80
95.00
66.67
0
10
20
30
40
50
60
70
80
90
100
Reco
gn
itio
n R
ate
(%
)
OFV PFV1 PFV2 PFV3 PFV4 PFV5
Feature Vector
OFV= Original Feature Vector, PFV1 = Feature Vector Proposed by SD,
PFV2 = Feature Vector Proposed by MI, PFV3 = Feature Vector Proposed by RSFS,
PFV4 = Feature Vector Proposed by SFS, PFV5 = Feature Vector Proposed by SFFS 43
Feature Selection
Feature
IDSD MI SFS SFFS
1 –
2
3
4 –
5
9 – – –
11
12 – –
36 – – –
54 – – –
55 – –
59 – – –
91 – –
Table 4: Common features from four PFV’s.
44
Feature Selection
Sr. No. Feature
1 max
2 min
3 SDNN
4 median
5 meanHR
6 mean
Table 5: Selected Features
45
Classification Using k-NN
Sr. No. Distance MeasureRecognition
Rate (%)
1 Euclidean 96.0
2 Cityblock 98.0
3 Chebychev 94.2
4 Squared Euclidean 96.0
5 Correlation 87.5
6 Cosine 96.0
7 Hamming 86.9
8 Jaccard 86.9
9 Mahanalobis 93.8
10 Minkowski 96.046
Classification Using k-NN
47
Eu= Euclidean, CB = Cityblock, CY = Chebychev, SEU= Squared Euclidean, CO= Co-
relation, CS= Cosine, HM = Hamming, JD= Jacard, ML= Mahanalobis, MN= Minkowski
ROC Curve
48
Sr. No k AUC
1 3 0.9877
2 5 0.9282
3 10 0.9158
4 15 0.8711
Performance Evaluation
49
TP: Genuine samples correctly identified as Genuine
TN: Imposter samples correctly rejected as Imposter
FP: Imposter samples wrongly identified as Genuine
FN: Genuine samples wrongly rejected as Imposter
ModalityGenuine
Samples
Imposter
SamplesTotal
HRV 1000 9000 10000
Table 7: Number of genuine and imposter samples used
Modality TP FP TN FN
HRV 980 20 9880 20
Table 8: Classification results showing TP, TN, FP and FN
50
Performance Evaluation
Sr. No MeasuresModality HRV
Absolute Percent
1 Accuracy 0.996 99.6
2 Misclassification Rate 0.004 0.40
3 True Positive Rate (Sensitivity) 0.980 98.0
4 False Positive Rate 0.002 0.20
5 False Negative Rate 0.020 2.0
6 True Negative Rate (specificity) 0.998 99.8
7 Positive Predictive Value (Precision) 0.980 98.0
8 Negative Predictive Value 0.998 99.8
9 Cohen’s Kappa Index Value 0.989 –
10 Area Under the Curve (AUC) 0.989 –
Cohen’s Kappa Index Value
51
Value of
Kappa (κ)
Level of
Agreement
% of Data
that are
Reliable
0.00 - 0.20 None 0 - 4
0.21 - 0.39 Minimal 4 - 15
0.40 - 0.59 Weak 15 - 35
0.60 - 0.79 Moderate 35 - 63
0.80 - 0.90 Strong 64 - 81
Above 0.90 Almost Perfect 82 - 100
Prominent Findings
52
• From ten distance measure used, cityblock performed
exceptionally well.
• Among all the five algorithms SFS performed better
feature selection.
• Fused Feature Vector (FFV) generated by fusing strong
features from wrapper algorithms resulted in a feature
set containing six features. Eventually all these six
features are from time domain technique.
• Therefore it is concluded that only six time domain
features are found to be relevant in human
identification.
ECG Based Unimodal System
Device
54
Sensor Name Handheld ECG Monitor
Make
Figure 6.2: Handheld ECG
Monitor with pair of
electrodes on both the sides.
MD100A1 (Choicemed)
Channel Single Channel
Signal bandwidth 0.5 - 75Hz
Sampling Rate 250 Hz
ModesTwo (short time &
continuous)
Short time Reading
Duration30 Seconds
Heart rate30-100bpm ±2; 101-
240bpm ±4
Dimension 136 x 84 x 21mm
Internal Memory200 ECG Strips of 30
seconds each
Connectivity USB 2.0
Figure : Handheld ECG Monitor with pair of electrodes on both the sides.
Database Specification
• 360 ECG strips of 60 healthy subjects
(37 males and 23 Females)
• 6 samples each of 30 seconds in single session.
• The ages of the subjects varied from 16 to 60
years.
• The subjects were informed of the purpose and
their consent was taken before data collection.
55
Session
No. Duration Samples Posture
1 30
Seconds 5
Sitting
Relaxed
Two Approaches For ECG Based
Biometrics
• ECG biometric methods as either fiducial-based, non
fiducial-based.
• The fiducial-based methods ex- tract temporal, amplitude,
area, angle, or dynamic (across heart- beats) features from
characteristic points on the ECG signal.
• The features include but are not limited to the amplitudes of
the P, R, and T waves, the temporal distance between wave
boundaries (onset and offset of the P, Q, R, S, and T waves),
the area of the waves, and slope information.
• The non fiducial-based methods do not use the
characteristic points as features. Instead, features like wavelet
coefficients and autocorrelation coefficients are utilized.
56
Workflow
57
ECG Signal Cleanup
58
Noise Correction & Ectopy Removal
59
For I = 1, n1If (x(i) > .25*h +(x(i-1)) and (x(i) > .25*h+(x(i+1)) then x(i) = (x(i-1) + x(i+1))/2If (x(i) < .25*h +(x(i-1)) and (x(i) < .25*h+(x(i+1)) then x(i) = (x(i-1) + x(i+1))/2End
Noise Correction/ Noise Removal
point x(i) is found such that its value is greater or less than x(i-1) and
x(i+1) by an amount equal to 1/4th of the full height, the value of
x(i) is replaced by the mean of x(i – 1) and x(i + 1). Where h = full
peak height
60
Noise Correction & Ectopy RemovalEctopy Removal/ Outlier Removal Algorithm
Removal of Baseline Wander
61
Wave Segmentation
62
Alignment
63
64
Mean Wave from a Strip
65
Template of two subjects
0
50
100
150
200
250
300
0 50 100 150 200
T ime P oint x 4 ms
Am
pli
tud
e (
mV
)
S ubject S 1
S ubject S 4
66
Rationale
Sample\
TemplateT1 T2 T3 T4 T5
A1_1 0.0002 0.0095 0.0153 0.0584 0.0255
A2_1 0.0093 0.0003 0.0036 0.0449 0.0126
A3_1 0.0153 0.0041 0.0004 0.0358 0.0060
A4_1 0.0602 0.0481 0.0393 0.0004 0.0323
A5_1 0.0259 0.0134 0.0061 0.0305 0.0003
Sample\
TemplateT1 T2 T3 T4 T5
A1_2 0.0001 0.0104 0.0165 0.0644 0.0271
A2_2 0.0100 0.0003 0.0041 0.0501 0.0134
A3_2 0.0161 0.0036 0.0003 0.0401 0.0061
A4_2 0.0633 0.0480 0.0384 0.0002 0.0324
A5_2 0.0269 0.0131 0.0063 0.0341 0.0002
Table : Matching Template with Known Sample – 1.
Table : Matching Template with Known Sample – 2.
67
Rationale
Sample\
TemplateT1 T2 T3 T4 T5
A1_4 0.0004 0.0104 0.0167 0.0687 0.0299
A2_4 0.0105 0.0006 0.0043 0.0545 0.0151
A3_4 0.0169 0.0039 0.0008 0.0436 0.0070
A4_4 0.0658 0.0511 0.0404 0.0008 0.0352
A5_4 0.0284 0.0140 0.0064 0.0366 0.0009
Sample\
TemplateT1 T2 T3 T4 T5
A1_5 0.0003 0.0099 0.0151 0.0587 0.0267
A2_5 0.0093 0.0005 0.0038 0.0452 0.0130
A3_5 0.0156 0.0037 0.0006 0.0358 0.0061
A4_5 0.0610 0.0473 0.0379 0.0005 0.0325
A5_5 0.0260 0.0129 0.0054 0.0305 0.0005
Table : Matching Template with Unknown Sample – 4
Table : Matching Template with Unknown Sample – 5
68
Sample\
TemplateT1 T2 T3 T4 T5
A1_4 0.00039 0.01038 0.01674 0.06865 0.02993
A2_4 0.01055 0.00065 0.00434 0.05453 0.01510
A3_4 0.01689 0.00390 0.00076 0.04359 0.00695
A4_4 0.06579 0.05110 0.04036 0.00080 0.03522
A5_4 0.02843 0.01396 0.00639 0.03662 0.00087
Sample\
TemplateT1 T2 T3 T4 T5
A1_2 0.0001 0.0104 0.0165 0.0644 0.0271
A2_2 0.0100 0.0003 0.0041 0.0501 0.0134
A3_2 0.0161 0.0036 0.0003 0.0401 0.0061
A4_2 0.0633 0.0480 0.0384 0.0002 0.0324
A5_2 0.0269 0.0131 0.0063 0.0341 0.0002
Table : Matching Template with Sample – 4 using tight threshold of 0.00077 exhibiting False Rejection.
Table : Matching Template with Sample – 2 exhibiting False Acceptance at threshold of 0.0040.
Rationale
Results and Discussion
69
Figure : DET plot for ECG unimodal system. Zoomed inset shows 10.5% EER at threshold of 0.0085.
70
Results and Discussion
Figure : ROC curve for ECG unimodal system with sensitivity against (1– specificity).
Performance Evaluation
71
ModalityGenuine
Samples
Imposter
SamplesTotal
ECG 60 3540 3600
Table : Number of genuine and imposter samples used.
Modality TP FP TN FN
ECG 53 367 3173 7
Table : Classification results showing TP, TN, FP and FN.
72
Performance Evaluation
SN Measures
Modality ECG
Absolute Percent (%)
1 Accuracy 0.896 89.61
2 Misclassification Rate 0.104 10.39
3 True Positive Rate (Sensitivity) 0.883 88.33
4 False Positive Rate 0.104 10.37
5 False Negative Rate 0.117 11.67
6 True Negative Rate (specificity) 0.896 89.63
7 Positive Predictive Value (Precision) 0.126 12.62
8 Negative Predictive Value 0.998 99.78
9 Cohen’s Kappa Index Value 0.197 –
10 Area Under the Curve (AUC) 0.890 –
Table : Performance Measures for ECG Unimodal Systems.
73
Value of
Kappa (κ)
Level of
Agreement
% of Data
that are
Reliable
0.00 - 0.20 None 0 - 4
0.21 - 0.39 Minimal 4 - 15
0.40 - 0.59 Weak 15 - 35
0.60 - 0.79 Moderate 35 - 63
0.80 - 0.90 Strong 64 - 81
Above 0.90 Almost Perfect 82 - 100
Performance Evaluation
HRV and ECG Based Multimodal
System
Workflow
Sr. No.Feature
NameFeatures Description
1 MinMinimum Interval duration in a particular
RR-Interval Sequence
2 MaxMaximum Interval duration in a particular
RR-Interval Sequence
3 Median Median of the RR-Interval sequence
4 Mean Mean of the whole RR-Interval sequence
5 SDNN Standard Deviation in normal RR-Interval
6 MeanHR Mean Heart Rate
Table: Time-Domain Features Obtained from RR-Interval Sequence.
HRV Template Creation
• Five feature sets • containing six features • five samples of each subject. • From these five feature sets, mean of
first three was used as template
ECG_HRV60.xls6 Columns containing Features360 Rows (60 x (5 +1)) 75
ECG Features and Template Generation
Figure 7.3: Processing, a) all the ECG wave segments from an ECG strip, b) Spatial
alignment of the segments shown in a, c) temporal alignment with spatial
alignment, d) Mean segment formed from mean of means of the three
strips.
a b
c d
76
Matching Module
• The two inputs for matcher module came fromtwo different templates i.e. HRV template andECG template.
• In the matcher module, there are two distancemeasures.
• Cityblock Distance Measure is used for HRV andCosine is used for ECG.
• As these distance measures performed well in therespective unimodal experiments.
77
Identification Process
• Score Normalization
• Obtaining the EER (From DET Plot)
• Calculating the Weights for both Modalities
• Applying the Weighted Sum Rule For Fusing
the Scores
)Xmin()Xmax(
)Xmin(x'x
78
DET Plot for EER Finding of HRV Modality
Figure : DET plot for HRV modality. Zoomed inset is showing 1.92% EER at threshold of 0.0723.79
DET Plot for EER Finding of ECG Modality
Figure : DET plot for ECG modality. Zoomed inset is showing 10.2% EER at threshold of 0.082.80
Score-Level Fusion
(Weighted Sum Rule)
81
Weights Assigned to The Two Modalities
EER and Threshold
Assigned Weights
Show fusion sheet W2.xls82
83
DET Curve for Multimodal System
ROC Curve for Multimodal System
84
Figure : ROC Curve of the multimodal system showing a large area under curve of 0.998
Performance Measure
85
Sr.
No.Modality
Genuine
Samples
Imposter
SamplesTotal
1 ECG 60 3540 3600
2 HRV 1000 9000 10000
3 Fusion 60 3540 3600
Table : Number of genuine and imposter samples used in all three systems.
Sr.
No.Modality TP FP TN FN
1 ECG 53 367 3173 7
2 HRV 980 20 9880 20
3 Fusion 60 15 3525 0
Table : Classification results showing TP, TN, FP and FN.
86
Sr. No. Measures
Modality
ECG HRV Fused System
ABS % ABS % ABS %
1 Accuracy 0.896 89.6 0.998 99.8 0.996 99.6
2 Misclassification Rate 0.104 10.4 0.002 0.2 0.004 0.4
3 True Positive Rate (Sensitivity) 0.883 88.3 1.000 100.0 1.000 100.0
4 False Positive Rate 0.104 10.4 0.002 0.2 0.004 0.4
5 False Negative Rate 0.117 11.7 0.020 2.0 0.000 0.0
6True Negative Rate
(specificity)0.896 89.6 0.998 99.8 0.996 99.6
7Positive Predictive Value
(Precision)0.126 12.6 0.980 98.0 0.800 80.0
8 Negative Predictive Value 0.998 99.8 0.998 99.8 1.000 100.0
9 Cohen’s Kappa Index Value 0.197 0.978 0.887
10 Area Under the Curve (AUC) 0.890 - 0.999 - 0.998 -
Performance Measure
87
Performance Measure
Value of
Kappa (κ)
Level of
Agreement
% of Data
that are
Reliable
0.00 - 0.20 None 0 - 4
0.21 - 0.39 Minimal 4 - 15
0.40 - 0.59 Weak 15 - 35
0.60 - 0.79 Moderate 35 - 63
0.80 - 0.90 Strong 64 - 81
Above 0.90 Almost Perfect 82 - 100
Contributions
• Contribution of a HRV data Acquisition System (Hardware,
Firmware and Computer based Data Acquisition Software.
• Contribution of two databases of
100 subjects of HRV
60 Subjects ECG
• Single sample collection timing less than any other existing
HRV 30-60 secs & ECG 30 sec strip
• Database made freely available on a dedicated website
http://www.rdatabase.net
• Person Signature in HRV data for identification purpose. In terms of
6 prominent features.
(Min, MAX, Standard deviation, Median, mean Heart Rate, Mean).
• Proposed an Efficient Multimodal system which uses Single sensor
to get two modalities. Cost effective and with reduced complexity.88
89
Conclusion
Major Issues Addressed
• Can Heart Rate Variability (HRV) Parameters
(Features) be used as Biometric Trait/ Modality? -------
--------------- Yes
• Can Features Generated from HRV Analysis be used as
an independent modality in Unimodal Environment-----
-----------------------Yes
• Can HRV Features support/improve/enhance the
biometric system in Multimodal Environment.—Yes
Conclusion
90
Biometric
Trait (s)Universality Uniqueness Permanence Collectability Performance Acceptability Circumvention
DNA H H H L H L L
Ear M M H M M H M
Face H L M H L H H
Fingerprint M H H M H M M
Gait M L L H L H M
Hand
GeometryM M M H M M M
Iris H H H M H L L
Keystroke L L L M L M M
Odor H H H L L M L
Retina H H M L H L L
Signature L L L H L H H
Voice M L L M L H H
HRV H H H H H L
Papers
• Akhter, N., Mahdi, J. F., & Manza, G. R. (2012). Microcontroller based data acquisition
system for Heart Rate Variability (HRV) measurement. International Journal of Applied
Science and Engineering Research, 1(4), 576-583. ISSN 2277-9442
• Akhter, N., Tharewal, S., Kale, V., Bhalerao, A., & Kale, K. V. (2015). Heart-Based
Biometrics and Possible Use of Heart Rate Variability in Biometric Recognition Systems.
Advanced Computing and Systems for Security, 395(1), 15-29. ISSN 2194-5357,
Springer, Berlin, Heidelberg, Germany.
• Akhter, N., Gaike, V., Shaikh, S., & Kale, K. V. (2016). Classification of Heart Rate
Variability Features for Person Identification. Journal of Medicinal Chemistry and Drug
Discovery, 1(2), 371-380. ISSN: 2347-9027
• Nazneen Akhter, Hanumant Gite, Gulam Rabbani, K. V. Kale, “Heart Rate Variability for
Biometric Authentication Using Time-Domain Features”, in Security in Computing and
Communications, Vol. 536, Abawajy, J.H., Mukherjea, S., Thampi, S.M., Ruiz-Martínez,
A. (Eds.) Springer CCIS 7899, Vol. 536 ISSN 1865-0929, Springer, Berlin, Heidelberg,
2015, pp. 168–175.
Papers
• Nazneen Akhter, Siddartha Dabhade, Nagsen Bansod, K. V. Kale, “Feature Selection for
Heart Rate Variability Based Biometric Recognition using Genetic Algorithm”, in
Intelligent Systems Technologies and Applications, Vol 384, Berretti, Stefano, Thampi,
Sabu M., Srivastava, Praveen Ranjan (Eds.), Springer Advances in Intelligent Systems
and Computing 11156, ISSN 2194-5357, Springer, Berlin, Heidelberg, Germany, 2015
• Nazneen Akhter, Sumegh Tharewal, Hanumant Gite, K. V. Kale, “Microcontroller
Based RR-Interval Measurement Using PPG Signals for Heart Rate Variability based
Biometric Application”, IEEE International Symposium on Emerging Topics in Circuits
and Systems (SET-CAS'15), August 2015, Kochi, India.
• Nazneen Akhter, Hanumant Gite, Sumegh Tharewal, K. V. Kale, “Computer Based RR-
Interval Detection System with Ectopy Correction in HRV Data”, IEEE 4th International
Conference on Advances in Computing, Communications and Informatics (ICACCI-
2015), August 2015, Kochi, India.
• Akhter, N., Gaike, V., Shaikh, S., & Kale, K. V., (2016). ECG Based Unimodal System
Performance Analysis. International Journal of Emerging Trends in Engineering and
Development. ISSN 2249-6149. UK Based RS Publications [Communicated]
• Akhter, N., Gaike, V., Shaikh, S., & Kale, K. V., (2016). HRV and ECG Based
Multimodal System: Performance and Usability. IEEE Transactions on Pattern Analysis
and Machine Intelligence. [Communicated]
Acknowledgment
I am thankful to my guide Prof. K. V. Kale Sir.
I am thankful to Principal Maulana Azad College Dr. Maqdoom
Farooqui Sir.
I am thankful to Dr. A. R. Khan Sir
I am thankful to Head of Department of Computer Science and
IT, Dr. BAMU Aurangabad. I also am thankful all faculty members.
I am thankful to all my friends in dept. and colleagues in Azad
College. Most Importantly my two best friends Shazia and Vrushali.
I am thankful to UGC for funding the Multimodal System
Development laboratory established under UGC’s SAP scheme.
I am also thankful to UGC for providing me BSR fellowship.
Thank YouFor Your Presence,
Attention and Patience
References1. AHA and ESC, “Guidelines Heart rate variability,” Eur. Heart J., pp. 354–381, 1996.
2. CDAC Mohali, News & Events Page, http://cdacmohali.in/N_ECG.aspx
3. Chang, F. C.; Chang, C. K.; Chiu, C.C.; Hsu, S. F.; Lin, Y.D., "Variations of HRV analysis in different approaches," Computers in Cardiology, 2007 , vol., no.,
pp.17,20, Sept. 30 2007-Oct. 3 2007, doi: 10.1109/CIC.2007.4745410
4. CME-cum-Workshop - Heart Rate Variability: Diagnostic and Research Tool, Workshop Souvenir, http://cmeonhrvatmgmcri.blogspot.in/
5. Nakajima K, Tamura T, Miike H, Pberg PA, Photoplethysmographic measurement of heart and respiratory rates using digital filters. Eng. in Med. and Bio. Society
Proceedings of the 15th Annual International Conference of the IEEE, 1993;1006 – 1007.
6. Task Force of the European Society of Cardiology and North American Society of Pacing and Electrophysiology (1996) Heart rate variability: standards
of measurement, physiological interpretation and clinical use. Eur Heart J 17:354–381
7. Heart rate variability: A Review, U. Rajendra Acharya , K. Paul Joseph , N. Kannathal , Choo Min Lim, Jasjit S. Suri, Med Bio Eng Comput (2006)
44:1031–1051, DOI 10.1007/s11517-006-0119-0
8. Juan Sztajzel, Heart rate variability: a noninvasive electrocardiographic method to measure the autonomic nervous system, SWISS MED WKLY 2004;134:514–
522.
9. Kleiger RE, Stein PK, Bosner MS, Rottman JN. Time domain measurements of heart rate variability. Amb Electrocardiol 1992; 10:487–98.
10. Hnatkova K, Copie X, Staunton A, Malik M. Numeric processing of Lorenz plots of R-R intervals from long-term ECGs: comparison with time-domain measures
of heart rate variability for risk stratification after myocardial infarction. J Electrocardiol 1995;28(suppl):74–80.
11. Ognian Boumbarov, Yuliyan Velchev, Krasimir Tonchev and Igor Paliy (2011). Face and ECG Based Multi-Modal Biometric Authentication, Advanced
Biometric Technologies, Dr. Girija Chetty (Ed.), ISBN: 978-953-307-487-0, InTech, Available from: http://www.intechopen.com/books/advanced-biometric-
technologies/face-andecg-based-multi-modal-biometric-authentication
12. W.-H. Lin, D. Wu, C. Li, H. Zhang, and Y.-T. Zhang, “Comparison of Heart Rate Variability from PPG with That from ECG,” in The International Conference on
Health Informatics SE - 54, vol. 42, Y.-T. Zhang, Ed. Springer International Publishing, 2014, pp. 213–215.
13. N. Selvaraj, A. Jaryal, J. Santhosh, K. K. Deepak, and S. Anand, “Assessment of heart rate variability derived from finger-tip photoplethysmography as compared
to electrocardiography,” J. Med. Eng. Technol., vol. 32, no. 6, pp. 479–484, Jan. 2008.
14. E Gil1,MOrini1,RBail´on, JVergara,LMainardi,P.Laguna,Photoplethysmography pulse rate variability as a surrogate measurement of heart rate variability during
non-stationary conditions, Physiol.Meas.31(9)(2010) 127–1290.
15. B. Park, “Psychophysiology as a Tool for HCI Research: Promises and Pitfalls,” in Human-Computer Interaction. New Trends SE - 16, vol. 5610, J. Jacko, Ed.
Springer Berlin Heidelberg, 2009, pp. 141–148.
16. Gu, Y.Y.; Zhang, Y.; Zhang, Y.T., "A novel biometric approach in human verification by photoplethysmographic signals," Information Technology Applications
in Biomedicine, 2003. 4th International IEEE EMBS Special Topic Conference on, vol., no., pp.13,14, 24-26 April 2003 doi: 10.1109/ITAB.2003.1222403
17. A. Reşit Kavsaoğlu, K. Polat, and M. Recep Bozkurt, “A novel feature ranking algorithm for biometric recognition with PPG signals,” Comput. Biol. Med., vol.
49, no. 0, pp. 1–14, Jun. 2014.
References17. C. Lin, J.-S. Wang, and P. Chung, “Mining Physiological Conditions from Heart Rate Variability Analysis,” IEEE Computational Intelligence Magazine,
vol. 5, no. 1. pp. 50–58, 2010.
18. P. Melillo, “Classification Tree for Risk Assessment in Patients Suffering From Congestive Heart Failure via Long-Term Heart Rate Variability,” IEEE J. Biomed.
Heal. Informatics, vol. 17, no. 3, pp. 727 – 733, 2013.
19. S. Nizami, J. R. Green, J. M. Eklund, and C. McGregor, “Heart disease classification through HRV analysis using parallel cascade identification and fast
orthogonal search,” in 2010 IEEE International Workshop on Medical Measurements and Applications, MeMeA 2010 - Proceedings, 2010, pp. 134–139.
20. M. Szypulska and Z. Piotrowski, “Prediction of Fatigue and Sleep Onset Using HRV Analysis,” in Mixed Design of Integrated Circuits and Systems (MIXDES),
Proceedings of the 19th International Conference, 2012, pp. 543–546.
21. Malliani A, Pagani M, Furlan R, Stefano Guzzetti, Lucini D, Montano N, Cerutti S, Mela GS. Individual recognition by heart rate variability of two different
autonomic profiles related to posture. Circulation American Heart Association. 1997;96: 4143–4145.
22. J.M. Irvine, B.K.Wiederhold, L.W. Gavshon, S.A. Israel, S.B., McGehee, R. Meyer, M.D. Wiederhold, Heart rate variability:a new biometric for human
identification, in: International Conference on Artificial Intelligence (IC-AI’2001), Las Vegas, Nevada, 2001, pp. 1106–1111.
23. S. a. Israel, J. M. Irvine, B. K. Wiederhold, and M. D. Wiederhold, “The Heartbeat: The Living Biometric,” Biometrics Theory, Methods, Appl., pp. 429–
459, 2009.
24. da Silva, H.P.; Fred, A.; Lourenco, A.; Jain, A.K., "Finger ECG signal for user authentication: Usability and performance," Biometrics: Theory,
Applications and Systems (BTAS), 2013 IEEE Sixth International Conference on , vol., no., pp.1,8, Sept. 29 2013-Oct. 2 2013. doi:
10.1109/BTAS.2013.6712689
25. Hugo Plácido da Silva, André Lourenço, Ana Fred, Nuno Raposo, Marta Aires-de-Sousa, Check Your Biosignals Here: A new dataset for off-the-person
ECG biometrics, Computer Methods and Programs in Biomedicine, Volume 113, Issue 2, February 2014, Pages 503-514
26. S. Yusuf. H, A R. Khan, and S. H. Behere, “AVR Microcontroller Based Data Acquisition System for Laboratory Experiments,” vol. 3, no. 1, pp. 208–215, 2012.
27. N. Akhter, J. F. Mahdi, G. R. Manza, and D. Registrar, “Microcontroller based data acquisition system for Heart Rate Variability (HRV) measurement,” vol. 1, no.
4, pp. 576–583, 2012.
28. C. Lin, J.-S. Wang, and P. Chung, “Mining Physiological Conditions from Heart Rate Variability Analysis,” IEEE Computational Intelligence Magazine, vol. 5,
no. 1. pp. 50–58, 2010.
29. Nazneen Akhter, Sumegh Tharewal, Vijay Kale, Ashish Bhalerao, K. V. Kale, “Heart Based Biometrics and Heart Rate Variability in Human Identification
Systems, 2nd International Doctoral Symposium on Applied Computation and Security Systems, 2015.
30. L. Biel, O. Pettersson, L. Philipson, and P. Wide, “ECG analysis: Anew approach in human identification,” IEEE Trans. Instrum. Meas.,vol. 50, no. 3, pp. 808–
812, Jun. 2001.
31. G. Wubbeler, M. Stavridis, D. Kreiseler, R.-D. Bousseljot, and C. Elster,“Verification of humans using the electrocardiogram,” PatternRecognit. Lett., vol. 28, pp.
1172–1175, 2007.[15]
32. F. Agrafioti and D. Hatzinakos, “Fusion of ECG sources for human identification,” in Proc. 3rd Int. Symp. Communications, Control and Signal Processing
(ISCCSP 2008), St. Julians, Malta, 2008.
Distance Metric Description Mathematical Expression
Euclidean
Euclidean distance is the straight-
line distance between two points in Euclidean
space.
𝑑𝑠𝑡2 = (𝑥𝑠 − 𝑦𝑡)(𝑥𝑠 − 𝑦𝑡)
′
Squared
Euclidean
The standard Euclidean distance can be squared
in order to place greater weight on objects that are
farther apart𝑑𝑠𝑡2 = (𝑥𝑠 − 𝑦𝑡)𝑉
−1(𝑥𝑠 − 𝑦𝑡)′
Mahanabolis
Mahalanobis distance normalizes based on a
covariance matrix to make the distance metric
scale-invariant𝑑𝑠𝑡2 = (𝑥𝑠 − 𝑦𝑡)𝐶
−1(𝑥𝑠 − 𝑦𝑡)′
City block
City block metric
𝑑𝑠𝑡 =
𝑗=1
𝑛
|𝑥𝑠𝑗 − 𝑦𝑡𝑗|
Minkowski
Minkowski distance is a generalization that
unifies Euclidean distance, Manhattan distance,
and Chebyshev distance 𝑑𝑠𝑡 =𝑝
𝑗=1
𝑛
|𝑥𝑠𝑗 − 𝑦𝑡𝑗|𝑝
ChebychevChebyshev distance measures distance assuming
only the most significant dimension is relevant.𝑑𝑠𝑡 = 𝑚𝑎𝑥𝑗{|𝑥𝑠𝑗 − 𝑦𝑡𝑗|}
CosineOne minus the cosine of the included angle
between points (treated as vectors 𝑑𝑠𝑡 = (1 −𝑥𝑠𝑦𝑡′
′ ′)
97
98
Performance metrics are calculated as:
Accuracy = (TN + TP) / (TN+TP+FN+FP)
Sensitivity (TPR) = TP / (TP + FN)
Specificity (TNR) = TN / (TN + FP)
Prevalence = (TP + FN) / (TN+TP+FN+FP)
Positive Predictive Value = TP / (TP + FP)
Negative Predictive Value = TN / (TN + FN)
False Positive Rate = FP / (FP + TN)
False Negative Rate = FN / (FN + TP)
Area Under ROC curve = (TPR + TNR) / 2