presentation on research work entitled heart rate

Dr. Babasaheb Ambedkar Marathwada University

Department of Computer Science and Information Technology

Presentation on Research Work Entitled

Heart Rate Variability (HRV) Based Multimodal

Biometric System for Human Authentication and

Identification

Research Student

Nazneen Akhter Shaikh

Research Guide

Prof. K. V. KaleProfessor

Department of Computer Science and Information Technology,

Dr. Babasaheb Ambedkar Marathwada University, Aurangabad

Work Outline

The entire work presented in thesis is

organized in 8 chapters.

The experimental work is presented in 4

chapters.

Every Chapter is independent and self

explanatory.

Findings and results are presented and

discussed.

2

Presentation Outline

• Introduction

• Background

• Literature Survey

• RR-Interval Measurement System

• HRV Based Unimodal System

• ECG Based Unimodal System

• ECG and HRV Based Multimodal System

• Conclusion and Future Work

• Acknowledgement

• Publications

• References3

When we think of our heart rate, we think of a number between

60 to 90 beats per minute (BPM). This number represents our

average heart rate.

In reality, our heart rate changes from beat to beat.

Heart Rate Variability (HRV) a measure of this naturally

occurring variation in the heart rate.

HRV measurements contain more information than

measurements of heart rate alone.

High HRV, a sign of good health.

Hence HRV arising as a simple, non-invasive measure and HRV

analysis the ability to assess overall cardiac health and the state

of the Autonomic Nervous System (ANS).

Introduction Heart Rate Variability (HRV)

4

Figure 1: ProminentComponents of an ECGSignal

Figure 2: R-R Intervalsin an ECG signal(showing beat to beatvariation)

The interval between two adjacent R-R peak of QRS complex known RR-

Interval Which is the only measurement required for HRV Analysis.

Introduction Heart Rate Variability (HRV)

5

Motivation for using Biosignals

May 2013. A Brazilian lady Doctor arrested for using 6 Silicone Fingers to

Hack Hospital Security System to put attendance of 6 senior colleagues

Limitations of Conventional Systems

• They are unique but not confidential and neither secret to the

individual.

• Traditional biometrics can be conveniently manipulated,

spoofed and counterfeited.

6

Alterations

Image Source Fingerprint Alterations Technical Report 2009. A. K. Jain

Motivation for using Biosignals

7

• Besides uniqueness, the biosignals areconfidential and secure to an individual.

• They are difficult to mimic and hard to be copied.

• Therefore, the identity of an individual is unlikelyto be forged, thus preserving the secrecy andprivacy of the users.

• Unlike anatomical biometric modalities that havetwo-dimensional data representation,

• Biosignals are physiologically low-frequencysignals that have one-dimensional datarepresentation.

Motivation for using Biosignals

8

Motivation for Multimodal Biometrics

Limitations of Unimodal Systems• Inefficiency of uni-modal

Biometrics Systems

• Failure to Enroll/ Failure to Acquire

• Easier to Spoof

• Inefficiencies in performance with

time, environment and sensor

variations

Nov 2013

9

Figure : Advantages of Using Multimodality for Biometric Recognition.

Motivation for Multimodal Biometrics

10

Research Problem

11

Research Problem

• Are these parameters/Features Person Specific?

• Are they capable of Identifying an individual in an

Biometric based Authentication System?

• Can HRV parameters act as independent modality in

an unimodal environment?

• Can HRV parameters act as Supportive modality or

Enhance and Improve the performance/limitation of

any traditional modality in an multimodal

environment?

12

Research Objectives1. To study the existing state of the art in relation to Heart Beat Variability (HRV)

with a perspective on using the same for Biometric purpose.

2. To design and develop instrumentation for heart beat detection, signal

conditioning and data acquisition for data transfer to implement biometrics

recognition based on HRV.

3. Experimental setups will be planned to generate an HRV database for

implementation of HRV based biometric recognition system.

4. Suitable preprocessing techniques, feature set generation techniques will be

judiciously selected and necessary feature extraction techniques will be

developed and implemented.

5. Data from other sources will be used for multimodal fusion of other modalities

with HRV for improving efficiency and robustness of traditional modalities.

6. To compile the outcome of the work.13

Background (Functioning of Heart)At any given time the chambers of the heart may be found in one of two states:

• Systole: During systole, cardiac muscle tissue is contracting to push blood out of the chamber.

• Diastole: During diastole, the cardiac muscle cells relax to allow the chamber to fill with blood.

• Blood pressure increases in the major arteries during ventricular systole and decreases during ventricular diastole.

This leads to the 2 numbers associated with blood pressure—systolic blood pressure is the higher number and diastolic blood pressure is the lower number. For example, a blood pressure of 120/80 describes the systolic pressure (120) and the diastolic pressure (80).

14

• The cardiac cycle includes all of the events thattake place during one heartbeat. There are 3phases to the cardiac cycle: atrial systole,ventricular systole, and relaxation.

• Atrial systole: During the atrial systole phase ofthe cardiac cycle, the atria contract and pushblood into the ventricles.

• Ventricular systole: During ventricular systole,the ventricles contract to push blood into the aortaand pulmonary trunk.

• Relaxation phase: During the relaxation phase,all 4 chambers of the heart are in diastole as bloodpours into the heart from the veins.

Background (Functioning of Heart)

15

• The sounds of a normal heartbeat are known as “lubb” and “dupp” and are caused by blood pushing on the valves of the heart. The “lubb” sound comes first in the heartbeat and is the longer of the two heart sounds. The “lubb” sound is produced by the closing of the AV valves at the beginning of ventricular systole.

• During a normal heartbeat, these sounds repeat in a regular pattern of lubb-dupp-pause.

Background (Heart Sounds During a Beat)

16

Background (Heart Related Measurements)

Sr.

No.Method Signal Type

1 ECG Electrical

2 Pulse Oximeter Optical

3 Blood Pressure Pressure

4 Heart Sounds Acoustic

5Laser Doppler

VibrometerySurface Displacement

6 Radar Doppler

7 Motion Imagery Skin Color Fluctuations

Table 1: Heartbeat Sensing Methods.

17

• Task Force of the European Society of Cardiology.

"Heart rate variability standards of measurement,

physiological interpretation, and clinical use."

European Heart Journal 17 (1996): 354-381.

• Acharya, U. Rajendra, et al. "Heart rate variability: a

review." Medical and biological engineering and

computing 44.12 (2006): 1031-1051.

Literature SurveyInception (Base of Idea)

18

Literature Survey

• Da Silva, H. P., Fred, A., Lourenço, A., & Jain, A. K. (2013,

September). Finger ECG signal for user authentication:

Usability and performance. In Biometrics: Theory,

Applications and Systems (BTAS), 2013 IEEE Sixth

International Conference on (pp. 1-8). IEEE.

• Silva, H., Lourenço, A., Canento, F., Fred, A. L., & Jain, A. K.

(2013, February). ECG Biometrics: Principles and

Applications. In BIOSIGNALS (pp. 215-220).

• Lourenço, A., Silva, H., Fred, A. & Jain, A. K. (2011). Unveilingthe biometric potential of finger-based ECG signals.Computational intelligence and neuroscience, 2011, 5.

Gold References for ECG Biometrics

ECG Literature Review 19

RR-Interval Measurement

System

Device Design Considerations, Testing and

Performance Analysis1. System Description

• Design Considerations• Sensors• Controlling Unit• Acquisition Software

2. System Design Testing • Hardware Ectopy Removal• Software Ectopy Removal• Device Calibrations

3. System Performance Testing• RS- Analysis• Fractal Dimension• Poincare Plot

4. System Working21

The device is made of two parts

Sensor Unit

Controlling and Processing

Unit

Heart Beat (Pulse) SensorControlling & Processing Unit

Device Design Considerations, Testing and

Performance Analysis

22

Pulse Sensors Used

The pulse sensors used in our system

works on the Photoplethysmography PPG

signals

It an electro-optic method of measuring

the cardiovascular heartbeat pulse that

found all through the human body

especially at the peripherals like fingertip

toes or ear lobes.

The pulse wave brought about by the

rhythmic throbs of blood vessels in

synchronism with the heartbeat, causing

changes in the flow of blood and the beat

detected from the changing optical density

or opacity.

Low Cost, Easily Available

Reliable and widely Accepted in Clinical

Practice.

Device Design Considerations, Testing and

Performance Analysis

23

A B

Device Design Considerations, Testing and

Performance Analysis

24

Sensor Testing

Oscilloscope TDS 2024 By Tektronix

Dept. of Physics, Dr. Babasaheb Ambedkar Marathwada University.

25

Device Calibration

USB Based Digital Oscilloscope by Hantek26

Device Calibration

27

Device Calibration

S. No

Actual Time

Interval

(ms)

Time recorded

by the device

(ms)

Difference

between

measured and

true values

(ms)

Absolute

Error (ms)

1 2000 2001 + 1 1

2 2000 1999 – 1 1

3 2000 1999 – 1 1

4 2000 2000 0 0

5 2000 1999 – 1 1

6 2000 2001 + 1 1

7 2000 2000 0 0

8 2000 2000 0 0

9 2000 2001 + 1 1

10 2000 2000 0 0

Mean Absolute

Error (ms)0.6

Table 2: Comparison of actual and recorded time interval

RR-Interval measuring system has an

accuracy of measuring of ± 0.3 ms in 2000 ms

or ± 0.15 ms in 1000 ms. 28

Data Acquisition Software

29

Working of Device and Data Samples

From Device Analysis HRV

Parameters

Features

30

HRV Based Unimodal System

Do HRV Data Have Human

Discriminative Property?

32

Do HRV Data Have Human

Discriminative Property?

33

Workflow for HRV based Unimodal

System

34

Database Specification

100 subjects (57 males and 43 Females)

3000 sequences of RR-Intervals

Each subject 30 samples,10 samples in every session.

Each sample has 64 RR-Intervals

Each sample collection duration 30-60 seconds approximately

Three different sessions

Spread over 6 months

Time interval of three months between each session

No Physiological instructions given and

No emotional stimuli introduced while acquiring the pulse data

Data stored in computer files in TXT and CSV format

Session

No.Duration

No of RR-

IntervalsSamples Posture

1, 2, 3Approx.

30-60 Sec64 30

Sitting

Relaxed35

Poincare Maps

• Poincare HRV plot is a graph in which each RRinterval is plotted against next RR interval (atype of delay map) also known as Scatter plot

36

Poincare Maps

37

Poincare Map

SD1: dispersion (standard

deviation) of points

perpendicular to the axis

of line of identity

SD2: dispersion (standard

deviation) of points along

the axis of line of identity

38

Statistical ParametersSr. No. Feature name Description Unit

1 max Maximum RR interval in the time series (ms)

2 min Minimum RR interval in the time series (ms)

3 mean Mean of RR interval time series (ms)

4 median Median of RR interval time series (ms)

5 SDNN Standard Deviation of all RR (NN) Intervals. (ms)

6 SDANNStandard deviation of the averages of RR (NN) intervals in all

segments of the entire recording.(ms)

7 NNxNumber of pairs of adjacent NN intervals deviating by more than x

ms.(count)

8 pNNx NNx count divided by the total number of all RR (NN) intervals. (%)

9 RMSSDThe square root of the mean of the sum of the squares of differences

between adjacent RR (NN) intervals.(ms)

10 SDNNi Mean of the standard deviations of all RR (NN) intervals (ms)

11 meanHR Mean of Heart Rate (bpm)

12 sdHR Standard Deviation of Heart Rate (bpm)39

Feature Selection

• Usually feature extraction/generation techniques yield number of

features which may or may not contribute in proper classification

results.

• These features at times may be redundant or non-informative.

• Presence of such features significantly degrades the classification

performance.

• Therefore selecting a proper feature subset from the original features

set becomes a vital step before any classification attempt.

• Also that less number of features used for classification can

significantly improve the computation time required for classification.

• But finding such a small and efficient subset of features is tricky and

challenging.

40

1. Statistical Dependency (SD)It estimates the statistical dependency between features and

associated class labels using a quantized feature space.

2. Mutual Information (MI)It measures arbitrary dependencies between random variables.

3. Random Subset Feature Selection (RSFS)It aims to discover a set of features that perform better than an

average feature of the available feature set.

4. Sequential Forward Selection (SFS)It initially starting from an empty set, the feature set is iteratively updated by

including the feature which results in maximal score in each step.

5. Sequential Floating Forward Selection (SFFS)It uses SFS as baseline method and further extends by iteratively finding the least

significant features and eliminates it.

Feature Selection

41

Feature Selection

SD MI RSFS SFS SFFS

1 1 73 1 11

2 2 72 2 5

3 3 65 5 36

11 11 66 4 59

4 4 70 11 2

12 12 67 3 3

55 55 68 – –

5 5 71 – –

91 91 78 – –

9 54 69 – –

Table 3: Suggested features from five algorithms in order of relevance.

42

Recognition Rate

12.40

67.90 68.30

19.80

95.00

66.67

0

10

20

30

40

50

60

70

80

90

100

Reco

gn

itio

n R

ate

(%

)

OFV PFV1 PFV2 PFV3 PFV4 PFV5

Feature Vector

OFV= Original Feature Vector, PFV1 = Feature Vector Proposed by SD,

PFV2 = Feature Vector Proposed by MI, PFV3 = Feature Vector Proposed by RSFS,

PFV4 = Feature Vector Proposed by SFS, PFV5 = Feature Vector Proposed by SFFS 43

Feature Selection

Feature

IDSD MI SFS SFFS

1 –

2

3

4 –

5

9 – – –

11

12 – –

36 – – –

54 – – –

55 – –

59 – – –

91 – –

Table 4: Common features from four PFV’s.

44

Feature Selection

Sr. No. Feature

1 max

2 min

3 SDNN

4 median

5 meanHR

6 mean

Table 5: Selected Features

45

Classification Using k-NN

Sr. No. Distance MeasureRecognition

Rate (%)

1 Euclidean 96.0

2 Cityblock 98.0

3 Chebychev 94.2

4 Squared Euclidean 96.0

5 Correlation 87.5

6 Cosine 96.0

7 Hamming 86.9

8 Jaccard 86.9

9 Mahanalobis 93.8

10 Minkowski 96.046

Classification Using k-NN

47

Eu= Euclidean, CB = Cityblock, CY = Chebychev, SEU= Squared Euclidean, CO= Co-

relation, CS= Cosine, HM = Hamming, JD= Jacard, ML= Mahanalobis, MN= Minkowski

ROC Curve

48

Sr. No k AUC

1 3 0.9877

2 5 0.9282

3 10 0.9158

4 15 0.8711

Performance Evaluation

49

TP: Genuine samples correctly identified as Genuine

TN: Imposter samples correctly rejected as Imposter

FP: Imposter samples wrongly identified as Genuine

FN: Genuine samples wrongly rejected as Imposter

ModalityGenuine

Samples

Imposter

SamplesTotal

HRV 1000 9000 10000

Table 7: Number of genuine and imposter samples used

Modality TP FP TN FN

HRV 980 20 9880 20

Table 8: Classification results showing TP, TN, FP and FN

50

Performance Evaluation

Sr. No MeasuresModality HRV

Absolute Percent

1 Accuracy 0.996 99.6

2 Misclassification Rate 0.004 0.40

3 True Positive Rate (Sensitivity) 0.980 98.0

4 False Positive Rate 0.002 0.20

5 False Negative Rate 0.020 2.0

6 True Negative Rate (specificity) 0.998 99.8

7 Positive Predictive Value (Precision) 0.980 98.0

8 Negative Predictive Value 0.998 99.8

9 Cohen’s Kappa Index Value 0.989 –

10 Area Under the Curve (AUC) 0.989 –

Cohen’s Kappa Index Value

51

Value of

Kappa (κ)

Level of

Agreement

% of Data

that are

Reliable

0.00 - 0.20 None 0 - 4

0.21 - 0.39 Minimal 4 - 15

0.40 - 0.59 Weak 15 - 35

0.60 - 0.79 Moderate 35 - 63

0.80 - 0.90 Strong 64 - 81

Above 0.90 Almost Perfect 82 - 100

Prominent Findings

52

• From ten distance measure used, cityblock performed

exceptionally well.

• Among all the five algorithms SFS performed better

feature selection.

• Fused Feature Vector (FFV) generated by fusing strong

features from wrapper algorithms resulted in a feature

set containing six features. Eventually all these six

features are from time domain technique.

• Therefore it is concluded that only six time domain

features are found to be relevant in human

identification.

ECG Based Unimodal System

Device

54

Sensor Name Handheld ECG Monitor

Make

Figure 6.2: Handheld ECG

Monitor with pair of

electrodes on both the sides.

MD100A1 (Choicemed)

Channel Single Channel

Signal bandwidth 0.5 - 75Hz

Sampling Rate 250 Hz

ModesTwo (short time &

continuous)

Short time Reading

Duration30 Seconds

Heart rate30-100bpm ±2; 101-

240bpm ±4

Dimension 136 x 84 x 21mm

Internal Memory200 ECG Strips of 30

seconds each

Connectivity USB 2.0

Figure : Handheld ECG Monitor with pair of electrodes on both the sides.

Database Specification

• 360 ECG strips of 60 healthy subjects

(37 males and 23 Females)

• 6 samples each of 30 seconds in single session.

• The ages of the subjects varied from 16 to 60

years.

• The subjects were informed of the purpose and

their consent was taken before data collection.

55

Session

No. Duration Samples Posture

1 30

Seconds 5

Sitting

Relaxed

Two Approaches For ECG Based

Biometrics

• ECG biometric methods as either fiducial-based, non

fiducial-based.

• The fiducial-based methods ex- tract temporal, amplitude,

area, angle, or dynamic (across heart- beats) features from

characteristic points on the ECG signal.

• The features include but are not limited to the amplitudes of

the P, R, and T waves, the temporal distance between wave

boundaries (onset and offset of the P, Q, R, S, and T waves),

the area of the waves, and slope information.

• The non fiducial-based methods do not use the

characteristic points as features. Instead, features like wavelet

coefficients and autocorrelation coefficients are utilized.

56

Workflow

57

ECG Signal Cleanup

58

Noise Correction & Ectopy Removal

59

For I = 1, n1If (x(i) > .25*h +(x(i-1)) and (x(i) > .25*h+(x(i+1)) then x(i) = (x(i-1) + x(i+1))/2If (x(i) < .25*h +(x(i-1)) and (x(i) < .25*h+(x(i+1)) then x(i) = (x(i-1) + x(i+1))/2End

Noise Correction/ Noise Removal

point x(i) is found such that its value is greater or less than x(i-1) and

x(i+1) by an amount equal to 1/4th of the full height, the value of

x(i) is replaced by the mean of x(i – 1) and x(i + 1). Where h = full

peak height

60

Noise Correction & Ectopy RemovalEctopy Removal/ Outlier Removal Algorithm

Removal of Baseline Wander

61

Wave Segmentation

62

Alignment

63

64

Mean Wave from a Strip

65

Template of two subjects

0

50

100

150

200

250

300

0 50 100 150 200

T ime P oint x 4 ms

Am

pli

tud

e (

mV

)

S ubject S 1

S ubject S 4

66

Rationale

Sample\

TemplateT1 T2 T3 T4 T5

A1_1 0.0002 0.0095 0.0153 0.0584 0.0255

A2_1 0.0093 0.0003 0.0036 0.0449 0.0126

A3_1 0.0153 0.0041 0.0004 0.0358 0.0060

A4_1 0.0602 0.0481 0.0393 0.0004 0.0323

A5_1 0.0259 0.0134 0.0061 0.0305 0.0003

Sample\

TemplateT1 T2 T3 T4 T5

A1_2 0.0001 0.0104 0.0165 0.0644 0.0271

A2_2 0.0100 0.0003 0.0041 0.0501 0.0134

A3_2 0.0161 0.0036 0.0003 0.0401 0.0061

A4_2 0.0633 0.0480 0.0384 0.0002 0.0324

A5_2 0.0269 0.0131 0.0063 0.0341 0.0002

Table : Matching Template with Known Sample – 1.

Table : Matching Template with Known Sample – 2.

67

Rationale

Sample\

TemplateT1 T2 T3 T4 T5

A1_4 0.0004 0.0104 0.0167 0.0687 0.0299

A2_4 0.0105 0.0006 0.0043 0.0545 0.0151

A3_4 0.0169 0.0039 0.0008 0.0436 0.0070

A4_4 0.0658 0.0511 0.0404 0.0008 0.0352

A5_4 0.0284 0.0140 0.0064 0.0366 0.0009

Sample\

TemplateT1 T2 T3 T4 T5

A1_5 0.0003 0.0099 0.0151 0.0587 0.0267

A2_5 0.0093 0.0005 0.0038 0.0452 0.0130

A3_5 0.0156 0.0037 0.0006 0.0358 0.0061

A4_5 0.0610 0.0473 0.0379 0.0005 0.0325

A5_5 0.0260 0.0129 0.0054 0.0305 0.0005

Table : Matching Template with Unknown Sample – 4

Table : Matching Template with Unknown Sample – 5

68

Sample\

TemplateT1 T2 T3 T4 T5

A1_4 0.00039 0.01038 0.01674 0.06865 0.02993

A2_4 0.01055 0.00065 0.00434 0.05453 0.01510

A3_4 0.01689 0.00390 0.00076 0.04359 0.00695

A4_4 0.06579 0.05110 0.04036 0.00080 0.03522

A5_4 0.02843 0.01396 0.00639 0.03662 0.00087

Sample\

TemplateT1 T2 T3 T4 T5

A1_2 0.0001 0.0104 0.0165 0.0644 0.0271

A2_2 0.0100 0.0003 0.0041 0.0501 0.0134

A3_2 0.0161 0.0036 0.0003 0.0401 0.0061

A4_2 0.0633 0.0480 0.0384 0.0002 0.0324

A5_2 0.0269 0.0131 0.0063 0.0341 0.0002

Table : Matching Template with Sample – 4 using tight threshold of 0.00077 exhibiting False Rejection.

Table : Matching Template with Sample – 2 exhibiting False Acceptance at threshold of 0.0040.

Rationale

Results and Discussion

69

Figure : DET plot for ECG unimodal system. Zoomed inset shows 10.5% EER at threshold of 0.0085.

70

Results and Discussion

Figure : ROC curve for ECG unimodal system with sensitivity against (1– specificity).

Performance Evaluation

71

ModalityGenuine

Samples

Imposter

SamplesTotal

ECG 60 3540 3600

Table : Number of genuine and imposter samples used.

Modality TP FP TN FN

ECG 53 367 3173 7

Table : Classification results showing TP, TN, FP and FN.

72

Performance Evaluation

SN Measures

Modality ECG

Absolute Percent (%)

1 Accuracy 0.896 89.61

2 Misclassification Rate 0.104 10.39

3 True Positive Rate (Sensitivity) 0.883 88.33

4 False Positive Rate 0.104 10.37

5 False Negative Rate 0.117 11.67

6 True Negative Rate (specificity) 0.896 89.63

7 Positive Predictive Value (Precision) 0.126 12.62

8 Negative Predictive Value 0.998 99.78

9 Cohen’s Kappa Index Value 0.197 –

10 Area Under the Curve (AUC) 0.890 –

Table : Performance Measures for ECG Unimodal Systems.

73

Value of

Kappa (κ)

Level of

Agreement

% of Data

that are

Reliable

0.00 - 0.20 None 0 - 4

0.21 - 0.39 Minimal 4 - 15

0.40 - 0.59 Weak 15 - 35

0.60 - 0.79 Moderate 35 - 63

0.80 - 0.90 Strong 64 - 81

Above 0.90 Almost Perfect 82 - 100

Performance Evaluation

HRV and ECG Based Multimodal

System

Workflow

Sr. No.Feature

NameFeatures Description

1 MinMinimum Interval duration in a particular

RR-Interval Sequence

2 MaxMaximum Interval duration in a particular

RR-Interval Sequence

3 Median Median of the RR-Interval sequence

4 Mean Mean of the whole RR-Interval sequence

5 SDNN Standard Deviation in normal RR-Interval

6 MeanHR Mean Heart Rate

Table: Time-Domain Features Obtained from RR-Interval Sequence.

HRV Template Creation

• Five feature sets • containing six features • five samples of each subject. • From these five feature sets, mean of

first three was used as template

ECG_HRV60.xls6 Columns containing Features360 Rows (60 x (5 +1)) 75

ECG Features and Template Generation

Figure 7.3: Processing, a) all the ECG wave segments from an ECG strip, b) Spatial

alignment of the segments shown in a, c) temporal alignment with spatial

alignment, d) Mean segment formed from mean of means of the three

strips.

a b

c d

76

Matching Module

• The two inputs for matcher module came fromtwo different templates i.e. HRV template andECG template.

• In the matcher module, there are two distancemeasures.

• Cityblock Distance Measure is used for HRV andCosine is used for ECG.

• As these distance measures performed well in therespective unimodal experiments.

77

Identification Process

• Score Normalization

• Obtaining the EER (From DET Plot)

• Calculating the Weights for both Modalities

• Applying the Weighted Sum Rule For Fusing

the Scores

)Xmin()Xmax(

)Xmin(x'x

78

DET Plot for EER Finding of HRV Modality

Figure : DET plot for HRV modality. Zoomed inset is showing 1.92% EER at threshold of 0.0723.79

DET Plot for EER Finding of ECG Modality

Figure : DET plot for ECG modality. Zoomed inset is showing 10.2% EER at threshold of 0.082.80

Score-Level Fusion

(Weighted Sum Rule)

81

Weights Assigned to The Two Modalities

EER and Threshold

Assigned Weights

Show fusion sheet W2.xls82

83

DET Curve for Multimodal System

ROC Curve for Multimodal System

84

Figure : ROC Curve of the multimodal system showing a large area under curve of 0.998

Performance Measure

85

Sr.

No.Modality

Genuine

Samples

Imposter

SamplesTotal

1 ECG 60 3540 3600

2 HRV 1000 9000 10000

3 Fusion 60 3540 3600

Table : Number of genuine and imposter samples used in all three systems.

Sr.

No.Modality TP FP TN FN

1 ECG 53 367 3173 7

2 HRV 980 20 9880 20

3 Fusion 60 15 3525 0

Table : Classification results showing TP, TN, FP and FN.

86

Sr. No. Measures

Modality

ECG HRV Fused System

ABS % ABS % ABS %

1 Accuracy 0.896 89.6 0.998 99.8 0.996 99.6

2 Misclassification Rate 0.104 10.4 0.002 0.2 0.004 0.4

3 True Positive Rate (Sensitivity) 0.883 88.3 1.000 100.0 1.000 100.0

4 False Positive Rate 0.104 10.4 0.002 0.2 0.004 0.4

5 False Negative Rate 0.117 11.7 0.020 2.0 0.000 0.0

6True Negative Rate

(specificity)0.896 89.6 0.998 99.8 0.996 99.6

7Positive Predictive Value

(Precision)0.126 12.6 0.980 98.0 0.800 80.0

8 Negative Predictive Value 0.998 99.8 0.998 99.8 1.000 100.0

9 Cohen’s Kappa Index Value 0.197 0.978 0.887

10 Area Under the Curve (AUC) 0.890 - 0.999 - 0.998 -

Performance Measure

87

Performance Measure

Value of

Kappa (κ)

Level of

Agreement

% of Data

that are

Reliable

0.00 - 0.20 None 0 - 4

0.21 - 0.39 Minimal 4 - 15

0.40 - 0.59 Weak 15 - 35

0.60 - 0.79 Moderate 35 - 63

0.80 - 0.90 Strong 64 - 81

Above 0.90 Almost Perfect 82 - 100

Contributions

• Contribution of a HRV data Acquisition System (Hardware,

Firmware and Computer based Data Acquisition Software.

• Contribution of two databases of

100 subjects of HRV

60 Subjects ECG

• Single sample collection timing less than any other existing

HRV 30-60 secs & ECG 30 sec strip

• Database made freely available on a dedicated website

http://www.rdatabase.net

• Person Signature in HRV data for identification purpose. In terms of

6 prominent features.

(Min, MAX, Standard deviation, Median, mean Heart Rate, Mean).

• Proposed an Efficient Multimodal system which uses Single sensor

to get two modalities. Cost effective and with reduced complexity.88

89

Conclusion

Major Issues Addressed

• Can Heart Rate Variability (HRV) Parameters

(Features) be used as Biometric Trait/ Modality? -------

--------------- Yes

• Can Features Generated from HRV Analysis be used as

an independent modality in Unimodal Environment-----

-----------------------Yes

• Can HRV Features support/improve/enhance the

biometric system in Multimodal Environment.—Yes

Conclusion

90

Biometric

Trait (s)Universality Uniqueness Permanence Collectability Performance Acceptability Circumvention

DNA H H H L H L L

Ear M M H M M H M

Face H L M H L H H

Fingerprint M H H M H M M

Gait M L L H L H M

Hand

GeometryM M M H M M M

Iris H H H M H L L

Keystroke L L L M L M M

Odor H H H L L M L

Retina H H M L H L L

Signature L L L H L H H

Voice M L L M L H H

HRV H H H H H L

Papers

• Akhter, N., Mahdi, J. F., & Manza, G. R. (2012). Microcontroller based data acquisition

system for Heart Rate Variability (HRV) measurement. International Journal of Applied

Science and Engineering Research, 1(4), 576-583. ISSN 2277-9442

• Akhter, N., Tharewal, S., Kale, V., Bhalerao, A., & Kale, K. V. (2015). Heart-Based

Biometrics and Possible Use of Heart Rate Variability in Biometric Recognition Systems.

Advanced Computing and Systems for Security, 395(1), 15-29. ISSN 2194-5357,

Springer, Berlin, Heidelberg, Germany.

• Akhter, N., Gaike, V., Shaikh, S., & Kale, K. V. (2016). Classification of Heart Rate

Variability Features for Person Identification. Journal of Medicinal Chemistry and Drug

Discovery, 1(2), 371-380. ISSN: 2347-9027

• Nazneen Akhter, Hanumant Gite, Gulam Rabbani, K. V. Kale, “Heart Rate Variability for

Biometric Authentication Using Time-Domain Features”, in Security in Computing and

Communications, Vol. 536, Abawajy, J.H., Mukherjea, S., Thampi, S.M., Ruiz-Martínez,

A. (Eds.) Springer CCIS 7899, Vol. 536 ISSN 1865-0929, Springer, Berlin, Heidelberg,

2015, pp. 168–175.

Papers

• Nazneen Akhter, Siddartha Dabhade, Nagsen Bansod, K. V. Kale, “Feature Selection for

Heart Rate Variability Based Biometric Recognition using Genetic Algorithm”, in

Intelligent Systems Technologies and Applications, Vol 384, Berretti, Stefano, Thampi,

Sabu M., Srivastava, Praveen Ranjan (Eds.), Springer Advances in Intelligent Systems

and Computing 11156, ISSN 2194-5357, Springer, Berlin, Heidelberg, Germany, 2015

• Nazneen Akhter, Sumegh Tharewal, Hanumant Gite, K. V. Kale, “Microcontroller

Based RR-Interval Measurement Using PPG Signals for Heart Rate Variability based

Biometric Application”, IEEE International Symposium on Emerging Topics in Circuits

and Systems (SET-CAS'15), August 2015, Kochi, India.

• Nazneen Akhter, Hanumant Gite, Sumegh Tharewal, K. V. Kale, “Computer Based RR-

Interval Detection System with Ectopy Correction in HRV Data”, IEEE 4th International

Conference on Advances in Computing, Communications and Informatics (ICACCI-

2015), August 2015, Kochi, India.

• Akhter, N., Gaike, V., Shaikh, S., & Kale, K. V., (2016). ECG Based Unimodal System

Performance Analysis. International Journal of Emerging Trends in Engineering and

Development. ISSN 2249-6149. UK Based RS Publications [Communicated]

• Akhter, N., Gaike, V., Shaikh, S., & Kale, K. V., (2016). HRV and ECG Based

Multimodal System: Performance and Usability. IEEE Transactions on Pattern Analysis

and Machine Intelligence. [Communicated]

Acknowledgment

I am thankful to my guide Prof. K. V. Kale Sir.

I am thankful to Principal Maulana Azad College Dr. Maqdoom

Farooqui Sir.

I am thankful to Dr. A. R. Khan Sir

I am thankful to Head of Department of Computer Science and

IT, Dr. BAMU Aurangabad. I also am thankful all faculty members.

I am thankful to all my friends in dept. and colleagues in Azad

College. Most Importantly my two best friends Shazia and Vrushali.

I am thankful to UGC for funding the Multimodal System

Development laboratory established under UGC’s SAP scheme.

I am also thankful to UGC for providing me BSR fellowship.

Thank YouFor Your Presence,

Attention and Patience

References1. AHA and ESC, “Guidelines Heart rate variability,” Eur. Heart J., pp. 354–381, 1996.

2. CDAC Mohali, News & Events Page, http://cdacmohali.in/N_ECG.aspx

3. Chang, F. C.; Chang, C. K.; Chiu, C.C.; Hsu, S. F.; Lin, Y.D., "Variations of HRV analysis in different approaches," Computers in Cardiology, 2007 , vol., no.,

pp.17,20, Sept. 30 2007-Oct. 3 2007, doi: 10.1109/CIC.2007.4745410

4. CME-cum-Workshop - Heart Rate Variability: Diagnostic and Research Tool, Workshop Souvenir, http://cmeonhrvatmgmcri.blogspot.in/

5. Nakajima K, Tamura T, Miike H, Pberg PA, Photoplethysmographic measurement of heart and respiratory rates using digital filters. Eng. in Med. and Bio. Society

Proceedings of the 15th Annual International Conference of the IEEE, 1993;1006 – 1007.

6. Task Force of the European Society of Cardiology and North American Society of Pacing and Electrophysiology (1996) Heart rate variability: standards

of measurement, physiological interpretation and clinical use. Eur Heart J 17:354–381

7. Heart rate variability: A Review, U. Rajendra Acharya , K. Paul Joseph , N. Kannathal , Choo Min Lim, Jasjit S. Suri, Med Bio Eng Comput (2006)

44:1031–1051, DOI 10.1007/s11517-006-0119-0

8. Juan Sztajzel, Heart rate variability: a noninvasive electrocardiographic method to measure the autonomic nervous system, SWISS MED WKLY 2004;134:514–

522.

9. Kleiger RE, Stein PK, Bosner MS, Rottman JN. Time domain measurements of heart rate variability. Amb Electrocardiol 1992; 10:487–98.

10. Hnatkova K, Copie X, Staunton A, Malik M. Numeric processing of Lorenz plots of R-R intervals from long-term ECGs: comparison with time-domain measures

of heart rate variability for risk stratification after myocardial infarction. J Electrocardiol 1995;28(suppl):74–80.

11. Ognian Boumbarov, Yuliyan Velchev, Krasimir Tonchev and Igor Paliy (2011). Face and ECG Based Multi-Modal Biometric Authentication, Advanced

Biometric Technologies, Dr. Girija Chetty (Ed.), ISBN: 978-953-307-487-0, InTech, Available from: http://www.intechopen.com/books/advanced-biometric-

technologies/face-andecg-based-multi-modal-biometric-authentication

12. W.-H. Lin, D. Wu, C. Li, H. Zhang, and Y.-T. Zhang, “Comparison of Heart Rate Variability from PPG with That from ECG,” in The International Conference on

Health Informatics SE - 54, vol. 42, Y.-T. Zhang, Ed. Springer International Publishing, 2014, pp. 213–215.

13. N. Selvaraj, A. Jaryal, J. Santhosh, K. K. Deepak, and S. Anand, “Assessment of heart rate variability derived from finger-tip photoplethysmography as compared

to electrocardiography,” J. Med. Eng. Technol., vol. 32, no. 6, pp. 479–484, Jan. 2008.

14. E Gil1,MOrini1,RBail´on, JVergara,LMainardi,P.Laguna,Photoplethysmography pulse rate variability as a surrogate measurement of heart rate variability during

non-stationary conditions, Physiol.Meas.31(9)(2010) 127–1290.

15. B. Park, “Psychophysiology as a Tool for HCI Research: Promises and Pitfalls,” in Human-Computer Interaction. New Trends SE - 16, vol. 5610, J. Jacko, Ed.

Springer Berlin Heidelberg, 2009, pp. 141–148.

16. Gu, Y.Y.; Zhang, Y.; Zhang, Y.T., "A novel biometric approach in human verification by photoplethysmographic signals," Information Technology Applications

in Biomedicine, 2003. 4th International IEEE EMBS Special Topic Conference on, vol., no., pp.13,14, 24-26 April 2003 doi: 10.1109/ITAB.2003.1222403

17. A. Reşit Kavsaoğlu, K. Polat, and M. Recep Bozkurt, “A novel feature ranking algorithm for biometric recognition with PPG signals,” Comput. Biol. Med., vol.

49, no. 0, pp. 1–14, Jun. 2014.

References17. C. Lin, J.-S. Wang, and P. Chung, “Mining Physiological Conditions from Heart Rate Variability Analysis,” IEEE Computational Intelligence Magazine,

vol. 5, no. 1. pp. 50–58, 2010.

18. P. Melillo, “Classification Tree for Risk Assessment in Patients Suffering From Congestive Heart Failure via Long-Term Heart Rate Variability,” IEEE J. Biomed.

Heal. Informatics, vol. 17, no. 3, pp. 727 – 733, 2013.

19. S. Nizami, J. R. Green, J. M. Eklund, and C. McGregor, “Heart disease classification through HRV analysis using parallel cascade identification and fast

orthogonal search,” in 2010 IEEE International Workshop on Medical Measurements and Applications, MeMeA 2010 - Proceedings, 2010, pp. 134–139.

20. M. Szypulska and Z. Piotrowski, “Prediction of Fatigue and Sleep Onset Using HRV Analysis,” in Mixed Design of Integrated Circuits and Systems (MIXDES),

Proceedings of the 19th International Conference, 2012, pp. 543–546.

21. Malliani A, Pagani M, Furlan R, Stefano Guzzetti, Lucini D, Montano N, Cerutti S, Mela GS. Individual recognition by heart rate variability of two different

autonomic profiles related to posture. Circulation American Heart Association. 1997;96: 4143–4145.

22. J.M. Irvine, B.K.Wiederhold, L.W. Gavshon, S.A. Israel, S.B., McGehee, R. Meyer, M.D. Wiederhold, Heart rate variability:a new biometric for human

identification, in: International Conference on Artificial Intelligence (IC-AI’2001), Las Vegas, Nevada, 2001, pp. 1106–1111.

23. S. a. Israel, J. M. Irvine, B. K. Wiederhold, and M. D. Wiederhold, “The Heartbeat: The Living Biometric,” Biometrics Theory, Methods, Appl., pp. 429–

459, 2009.

24. da Silva, H.P.; Fred, A.; Lourenco, A.; Jain, A.K., "Finger ECG signal for user authentication: Usability and performance," Biometrics: Theory,

Applications and Systems (BTAS), 2013 IEEE Sixth International Conference on , vol., no., pp.1,8, Sept. 29 2013-Oct. 2 2013. doi:

10.1109/BTAS.2013.6712689

25. Hugo Plácido da Silva, André Lourenço, Ana Fred, Nuno Raposo, Marta Aires-de-Sousa, Check Your Biosignals Here: A new dataset for off-the-person

ECG biometrics, Computer Methods and Programs in Biomedicine, Volume 113, Issue 2, February 2014, Pages 503-514

26. S. Yusuf. H, A R. Khan, and S. H. Behere, “AVR Microcontroller Based Data Acquisition System for Laboratory Experiments,” vol. 3, no. 1, pp. 208–215, 2012.

27. N. Akhter, J. F. Mahdi, G. R. Manza, and D. Registrar, “Microcontroller based data acquisition system for Heart Rate Variability (HRV) measurement,” vol. 1, no.

4, pp. 576–583, 2012.

28. C. Lin, J.-S. Wang, and P. Chung, “Mining Physiological Conditions from Heart Rate Variability Analysis,” IEEE Computational Intelligence Magazine, vol. 5,

no. 1. pp. 50–58, 2010.

29. Nazneen Akhter, Sumegh Tharewal, Vijay Kale, Ashish Bhalerao, K. V. Kale, “Heart Based Biometrics and Heart Rate Variability in Human Identification

Systems, 2nd International Doctoral Symposium on Applied Computation and Security Systems, 2015.

30. L. Biel, O. Pettersson, L. Philipson, and P. Wide, “ECG analysis: Anew approach in human identification,” IEEE Trans. Instrum. Meas.,vol. 50, no. 3, pp. 808–

812, Jun. 2001.

31. G. Wubbeler, M. Stavridis, D. Kreiseler, R.-D. Bousseljot, and C. Elster,“Verification of humans using the electrocardiogram,” PatternRecognit. Lett., vol. 28, pp.

1172–1175, 2007.[15]

32. F. Agrafioti and D. Hatzinakos, “Fusion of ECG sources for human identification,” in Proc. 3rd Int. Symp. Communications, Control and Signal Processing

(ISCCSP 2008), St. Julians, Malta, 2008.

Distance Metric Description Mathematical Expression

Euclidean

Euclidean distance is the straight-

line distance between two points in Euclidean

space.

𝑑𝑠𝑡2 = (𝑥𝑠 − 𝑦𝑡)(𝑥𝑠 − 𝑦𝑡)

′

Squared

Euclidean

The standard Euclidean distance can be squared

in order to place greater weight on objects that are

farther apart𝑑𝑠𝑡2 = (𝑥𝑠 − 𝑦𝑡)𝑉

−1(𝑥𝑠 − 𝑦𝑡)′

Mahanabolis

Mahalanobis distance normalizes based on a

covariance matrix to make the distance metric

scale-invariant𝑑𝑠𝑡2 = (𝑥𝑠 − 𝑦𝑡)𝐶

−1(𝑥𝑠 − 𝑦𝑡)′

City block

City block metric

𝑑𝑠𝑡 =

𝑗=1

𝑛

|𝑥𝑠𝑗 − 𝑦𝑡𝑗|

Minkowski

Minkowski distance is a generalization that

unifies Euclidean distance, Manhattan distance,

and Chebyshev distance 𝑑𝑠𝑡 =𝑝

𝑗=1

𝑛

|𝑥𝑠𝑗 − 𝑦𝑡𝑗|𝑝

ChebychevChebyshev distance measures distance assuming

only the most significant dimension is relevant.𝑑𝑠𝑡 = 𝑚𝑎𝑥𝑗{|𝑥𝑠𝑗 − 𝑦𝑡𝑗|}

CosineOne minus the cosine of the included angle

between points (treated as vectors 𝑑𝑠𝑡 = (1 −𝑥𝑠𝑦𝑡′

′ ′)

97

98

Performance metrics are calculated as:

Accuracy = (TN + TP) / (TN+TP+FN+FP)

Sensitivity (TPR) = TP / (TP + FN)

Specificity (TNR) = TN / (TN + FP)

Prevalence = (TP + FN) / (TN+TP+FN+FP)

Positive Predictive Value = TP / (TP + FP)

Negative Predictive Value = TN / (TN + FN)

False Positive Rate = FP / (FP + TN)

False Negative Rate = FN / (FN + TP)

Area Under ROC curve = (TPR + TNR) / 2