identifying correlation between facial expression and

P a g e | 53

Identifying correlation between facial expression and heart rate and

skin conductance with iMotions biometric platform

Jing Lei1, Johannan Sala1, Dr. Shashi Jasra1

University of Windsor

Abstract:

Emotional reactions are stimulated when humans are presented with a stimulus, triggering a series

of voluntary and involuntary responses. Human emotions can be measured from facial expressions and

physiological processes. The iMotions biometric platform can detect and analyze the responses of

different individuals, which are personalized. The iMotions software allows for the quantification of

seven basic emotions: joy, sadness, anger, fear, contempt, surprise, and disgust. Along with iMotions,

galvanic skin response (GSR) and heart rate sensors from the Shimmer Kit were used. GSR refers to the

phenomenon wherein the skin temporarily becomes a better conductor of electricity due to elevated sweat

gland activity. In this study, participants were shown videos associated with different emotions while

their facial expressions were recorded, and their heart rate/skin conductance data collected. Using

iMotions and the Shimmer kit, this project aims to identify a possible correlation between the

participants’ facial reactions and their physiological responses, namely, their heart rate and skin

conductance, when exposed to different stimuli. The results indicated that there is a slightly higher

correlation between emotion and GSR compared to emotion and heart rate. From the findings, it can be

inferred that individuals react differently to the same stimulus. The iMotions software has great potential

in forensic biometric analysis of human emotions.

Keywords: biometrics, iMotions, Shimmer Kit, facial expressions, galvanic skin response (GSR), heart rate,

emotions

1 Forensic Sciences, Faculty of Science, University of Windsor, 401 Sunset Avenue, Windsor Ontario

Communicating Author Contact: Shashi Jasra, [email protected]

http://ojs.uwindsor.ca/ojs/leddy/index.php/JEFSR/

P a g e | 54

Introduction

Biometrics concerns the automated identification of an individual based on his or her

physical and behavioural characteristics.6 The basic premise of biometrics is that everyone is

unique and can be identified by their own set of traits. Biometric identification has played a role

in the field of forensic science for quite some time now, most notably, fingerprints and DNA.

Other characteristics used for biometric recognition include iris pattern, teeth (odontology), and

the one that is relevant to this research study, facial analysis.

When people are presented with a stimulus, their body undergoes an array of reactions,

some voluntary and others involuntary. Facial expressions and physiological processes can be

used to infer an individual’s emotional state.10 Heart rate and skin conductance are well known

physiological feedbacks to stress. One study has demonstrated that the difference between

physiological signals such as heart rate, skin conductance and skin temperature is significant

amongst different emotions.7 Another study predicted heart rate responses to different facial

expressions,2 which could imply that there is a connection between heart rate and different

stimuli evoking certain emotions. A study by Kortelainen et al. measured the heart rate,

respiration frequency, as well as the facial expressions of the participants to evaluate their

emotions while being shown pictures with emotional content. The results indicated that it is

much easier to determine arousal (degree of excitation) compared to valence (positive or

negative emotion).8

Research in the field of biometrics has brought about new technology such as the

iMotions biometrics software. The iMotions software makes use of several integrated sensors,

including facial expression analysis and the Shimmer Kit’s heart rate sensor, and galvanic skin

response (GSR) sensor, all three of which were used in this research project. The iMotions

emotient FACET is a facial expression recognition and analysis software allows for the

quantification of seven basic emotions: joy, sadness, anger, surprise, fear, disgust, contempt.5

GSR, one of the most sensitive markers for emotional stimulation9 refers to the phenomenon

wherein the skin becomes a better conductor of electricity when emotionally stimulated, due to

elevated sweat gland activity.3 Both heart rate and GSR are influenced by an individual’s

emotional state. For example, emotions such as fear, and anger tend to cause an increase in heart

rate.11

Because the iMotions software is a relatively novel technology, there has been very little

research using it. A University of Windsor student has used iMotions to conduct her research

project1; however, the purpose of her project was mainly to introduce the software and its

capabilities. Thus, it was more focused on the mechanism rather than the analysis. Building off

Al Masri’s project, and placing emphasis on the analysis portion, the aim of this research study

was to determine a possible correlation between an individual’s facial expression and heart

rate/skin conductance using the iMotions software, when presented with different stimuli. In

other words, the research goal was to see how these three parameters (facial expression, GSR,

and heart rate) are related since they can all be affected by one’s emotions. With the tool of

iMotions and the Shimmer Kit together, the measurements of the three parameters can be

recorded simultaneously.

P a g e | 55

Though there are existing applications of biometrics in the forensics field, experts believe

that biometrics need to play a bigger role in forensics.4 The iMotions software has great potential

in analyzing human responses and understanding human behavior. This research project could

help in avoiding wrongful convictions and analyzing the true emotions and intentions of

witnesses and suspects. It could be useful in determining what types of stimuli elicit the greatest

response in victims and suspects during the interview process of investigations. The implications

of this research study could have several possible forensic applications and further unite the two

fields of biometrics and forensic science.

Materials

The iMotions software installed in a laptop is required along with the Emotient FACET.

For this research, version 6.2 of iMotions was used. An external webcam, Logitech HD camera

was used for better quality video recordings. Additionally, a Shimmer Kit, which includes the

heart sensor and GSR sensor is required. To connect the sensors to the laptop, Bluetooth

connection is required.

Methods

The method was divided into three phases: Setup, Data Collection, and Data Analysis.

Setup

The initial step is the setup of the room and installation of the iMotions software on a

laptop. Environmental conditions must be consistent to reduce any environmental stress on the

participants. This include having a quiet office with comfortable seating and lighting. Following

would be the connection and configuration of the hardware components to the laptop including

the external webcam, GSR sensor and heart rate sensor. After launching the software, the

connection of the devices was indicated. Next, when creating a new study, the software guide

through a series of steps to setup the experiment such as the input of stimuli and participant

information. In conducting this experiment, eight different videos were chosen as stimuli listed

as stimulus 1-8 and uploaded. When selecting the videos as stimuli for the study, each video was

emotionally loaded and carefully chosen to induce the expression of a variety of emotions. A

calibration slide was presented at the start of the experiment, which was a grey screen. The

participants were asked to provide a neutral emotion to record what is known to be their baseline

slide. The duration of the baseline stimulus was 10 seconds. It is to record and calibrate the

neutral emotion of the participant, which is used to eliminate bias. Measurements on the

subsequent stimuli will be processed using the value obtained during the baseline stimulus

(iMotions Global, n. D). It considers their resting facial expression. Along with the videos, a

black wallpaper was inserted after every stimulus for 10 seconds as a short break to allow for the

participants reading to return to their baseline.

P a g e | 56

Table 1: Description of stimuli.

Data Collection

First, a test run was performed to ensure that the software was working properly. Before

performing the experiment on the participants, they were informed about the consent ethics form

and were explained the procedure of the experiment. For this study, this experiment was

conducted on 11 participants; however, for one participant, no GSR data was collected due to

faulty connection of the sensor. Thus, this one participant was eliminated, leaving 10 participants

left, designated as respondents 1-10. Participant information was then inserted including the

participants’ gender and age. After having them sit on a comfortable seat, the GSR sensor Velcro

straps was wrapped around the participant’s index and middle finger while the heart rate sensor

Velcro strap was wrapped around the participant’s ring finger. The device strap was wrapped

around the participant’s wrist. The participants were asked to face the camera and adjusted

themselves so that their facial expressions can be recorded. A live real time reading graph was

available to ensure that the facial expressions of the participants can be read. Participants were

also informed to avoid large movements or covering of their face to avoid any discrepancies.

When the participant was ready, the record button was clicked to start the recording using the

iMotions software.

Figure 1: Shimmer Kit GSR Sensor. The GSR sensor consists of two Velcro straps that are

wrapped around the participant’s index and middle finger, connected to a small device that wraps

around the wrist. Source from © Shimmer Sensing.

Stimulus Description

Stimulus 1 Mother shaming her daughter for being obese

Stimulus 2 Pipe cannon bursting suddenly

Stimulus 3 Man punching a kangaroo

Stimulus 4 Commercial about a deaf and mute father giving up his life for his suicidal daughter

Stimulus 5 Conceited, self-righteous teenage girl

Stimulus 6 Time lapse of a rotting watermelon

Stimulus 7 A worker alone in an office takes a strangely accurate Internet quiz.

Stimulus 8 Twin baby girls showing affection to each other

P a g e | 57

Figure 2: Real-Time Graph. Frame by frame analysis of emotional responses. Live time graphs

of measured emotional responses of the participant. Source from © iMotions.

Data Analysis

Upon the completion of data collection from the participants, all raw data was exported

for further analysis. Emotient statistics, GSR statistics and Sensor Data statistics for heart rate

were all exported and opened in an excel file. In the excel file, pivot tables were created to

organize the data to create graphs for analysis. Pivot table is a feature in excel that allows users

to extract select parts of data from a large data set. Using the data obtained for the experimental

procedure, different kinds of analyses could have been done. One approach to the analysis of the

data collected is by emotion and stimulus. For each emotion, the stimulus that gave the highest

value for that specific emotion was used for analysis. Graphs showing the average fraction of the

emotion time and average GSR amplitude for all the respondents were made. Similarly, a graph

with the average emotion time by percent and heart rate for all the respondents were also created.

Another approach to the analysis is by per respondent, looking at every emotion expressed by the

participant. For each emotion, the stimulus with the highest average time for the emotion was

used for analysis, with their respectively GSR and heart rate values. Graphs showing the emotion

and GSR values for all the emotions for each respondent were made. As well, graphs showing

the emotion and heart rate for all the emotions per respondent were created. To attempt to

correlate the sets of data, the correlation coefficients were found, and linear regression analyses

were made. This was done using the graphs representing average emotion time and GSR

amplitude values for each respondent and graphs representing average emotion time and heart

rate values.

Results

Emotion and Stimulus Analysis

For each of the seven emotions, joy, anger, surprise, fear, contempt, disgust, and sadness,

the stimulus that gave the highest total average emotion time by all the respondents were taken

for analysis. The emotion times were compared with GSR and heart rate values for all the

respondents, given the same stimulus. For example, Figure 3 shows the average fraction of anger

time and average GSR amplitude for stimulus 6 for all respondents (respondents 1-10). From all

the stimuli, stimulus 6 showed the highest emotion value for anger time by all respondents

P a g e | 58

collectively, therefore, was taken for analysis for the emotion of anger since it is most relevant.

In figure 4, it represents the average percent of anger time and average heart rate for stimulus 6

for all respondents.

Figure 3: Average fraction of anger time and average GSR amplitude for stimulus 6 for all

respondents.

Figure 4: Average percent of anger time and average heart rate for stimulus 6 for all

respondents.

Similarly, another sample graph can be seen in Figure 5. It shows the average fraction of

surprise time and average GSR amplitude for stimulus 4 for all respondents. In Figure 6, it

demonstrates the average percent of surprise time and average heart rate for stimulus 4 for all

respondents. Stimulus 4 showed the highest emotion time value for surprise.

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

R1 R10 R2 R3 R4 R5 R6 R7 R8 R9

Avg

. GSR

Am

plit

ud

e(μ

S)

Avg

. Fra

ctio

n o

f A

nge

r Ti

me

Respondent

Anger Time and GSR for Stimulus 6

Avg. Fraction Anger Time Avg. GSR Amp.

0

20

40

60

80

100

120

0

10

20

30

40

50

60

70

80

90

100

R1 R10 R2 R3 R4 R5 R6 R7 R8 R9

Avg

. He

art

Rat

e (

bm

p)

Avg

. An

ger

Tim

e (

%)

Respondent

Anger Time and Heart Rate For Stimulus 6

Avg. % Anger Time Avg. HR

P a g e | 59

Figure 5: Average fraction of surprise time and average GSR amplitude for stimulus 4 for all

respondents.

Figure 6: Average percent of surprise time and average heart rate for stimulus 4 for all

respondents.

Note: Average anger time and average surprise time gave the highest grand total emotion

average time from all respondents and stimuli. See Appendix A for graphs for emotion and

stimulus analysis.

The figures show that every individual expressed different levels of emotion, GSR and heart rate

values for the same stimulus.

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

R1 R10 R2 R3 R4 R5 R6 R7 R8 R9

Avg

. GSR

Am

plit

ud

e (

μS)

Avg

. Fra

ctio

n o

f Su

rpri

se T

ime

Respondent

Surprise Time and GSR For Stimulus 4

Avg. Fraction Surprise Time Avg. GSR Amp.

0

20

40

60

80

100

120

0

10

20

30

40

50

60

70

80

90

100

R1 R10 R2 R3 R4 R5 R6 R7 R8 R9

Avg

. He

art

Rat

e (

bm

p)

Avg

. Su

rpri

se t

ime

(%

)

Respondent

Surprise Time and Heart Rate for Stimulus 4

Avg. % Surprise Time Avg. HR

P a g e | 60

Per Respondent

Graphs were created for the representation of all the seven emotions expressed by each

respondent. The values for each emotion were taken from the stimulus that gave the highest

average time for that emotion expressed by the respondent. A sample data figure, Figure 7

demonstrates the average fraction of each emotion time and average GSR amplitude, for the

same stimulus, respectively for respondent 1. In a similar graphical demonstration, Figure 8

shows the average percent of each emotion time and average heart rate, for the same stimulus,

respectively for respondent 1.

Figure 7: Average fraction of each emotion time and average GSR amplitude, for the same

stimulus, respectively for respondent 1.

Figure 8: Average percent of each emotion time and heart rate, for the same stimulus,


0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0

0.02

0.04

0.06

0.08

0.1

Joy Sadness Anger Fear Contempt Disgust Surprise

Avg

. GSR

Am

plit

ud

e (

μS)

Frac

tio

n o

f A

vg. E

mo

tio

n T

ime

Emotion

Emotion and GSR For Respondent 1

Avg. Fraction of Emotion Time Avg. GSR Amp.

0

20

40

60

80

100

120

0102030405060708090

100110


Avg

. H

ear

t R

ate

(b

pm

)

Avg

. Em

oti

on

Tim

e (

%)

Emotion

Emotion and Heart Rate For Respondent 1

Avg. % Emotion Time Avg. HR

P a g e | 61

Figure 9 and figure 10 demonstrates the same type of analysis but for respondent 8.

Figure 9: Average fraction of each emotion time and average GSR amplitude, for the same


Figure 10: Average percent of each emotion time and average heart rate for the same stimulus,


Note: Respondent 1 demonstrated the lowest average emotion values and respondent 8 with

some of the highest emotion values. See Appendix B for graphs for per respondent analysis.

Figures show that the GSR amplitude and heart rate fluctuated for the different emotional

responses.

0

0.01

0.02

0.03

0.04

0.05

0

0.2

0.4

0.6

0.8

1

1.2


Avg

. GSR

Am

plit

ud

e (

μS)

Frac

tio

n o

f A

vg. E

mo

tio

n T

ime

Emotion



0

10

20

30

40

50

60

70

80

0

20

40

60

80

100

120


Avg

. He

art

Rat

e (

bp

m)

Avg

. Em

oti

on

Tim

e (

%)

Emotion



P a g e | 62

Given the average fraction of emotion time and average GSR amplitude, the correlation

coefficient can be found using the CORREL function on excel. An alternative method, also

giving a visual representation would be creating a plot with a trend line and equation.

Figure 11: Sample linear regression graph for correlation between average GSR amplitude and

average fraction emotion time for respondent 3.

Note: Respondent 3 gave the highest correlation coefficient. See Appendix C for graphs for per

respondent GSR and emotion correlation.

Table 2: Correlation coefficient values.

Correlation coefficient between avg.

fraction of emotion time and avg. GSR

amplitude

Correlation coefficient between avg. percent

of emotion time and avg. heart rate

R1 0.281000057

-0.658557266

R10 -0.900020011

0.588952478

R2 0.695248014

0.451790842

R3 0.772030015

-0.03462079

R4 -0.311811944

-0.338298635

R5 0.480908097

0.302834575

R6 0.662398057

0.354690004

R7 -0.383311514

-0.354173805

R8 -0.425533517

0.502973069

R9 0.054788253

-0.828952249

y = 0.0747x + 0.0234R² = 0.596

0

0.02

0.04

0.06

0.08

0.1

0.12

0 0.2 0.4 0.6 0.8 1

Avg

. GSR

Am

plit

ud

e (

μS)

Avg. Fraction Emotion Time

GSR and Emotion Correlation For Respondent 3

P a g e | 63

Six participants demonstrated a positive correlation between emotion and GSR, while the

rest showed a negative correlation. Five participants showed a positive correlation between

emotion and heart rate. The highest correlation was 0.772 and the lowest correlation was -0.9

(with 1 being a perfect positive correlation and -1 being a perfect negative correlation). The

average correlation between emotion and GSR for all participants was 0.0926. The average

correlation between emotion and heart rate for all participants was 0.023.

Table 3: Total average emotion time percentage from all respondents and stimuli.

JOY SADNESS ANGER SURPRISE FEAR DISGUST CONTEMPT

Average

Emotion

Time %

28.38 22.15 35.39 36.76 18.46 25.61 11.85

Table 4: Total average GSR amplitude from all respondents.

STIMULUS 1 STIMULUS 2 STIMULUS 3 STIMULUS 4 STIMULUS 5 STIMULUS 6 STIMULUS 7 STIMULUS 8

Average GSR

Amplitude

(μS) 0.031878 0.05482 0.06287 0.05378 0.063088 0.090838 0.070318

0.057777

The highest values found in stimulus 6 with anger as the highest total average emotion time

percentage of 50.4 % and stimulus 7 with also anger at 30.1%.

Discussion & Conclusion

Per stimulus and emotion analysis showed a wide variation in emotion and GSR values

between individuals, suggesting that every individual reacts differently to the same stimulus. For

each emotion, the stimulus that gave the highest total average emotion time by all respondents

was analyzed. The values were taken from the stimulus that gave the highest values for that

emotion since it is the most relevant. Two sets of graphs were made: average fraction of emotion

time and average GSR amplitude for all respondents and average percent of emotion time and

average heart rate for all respondents. By doing the emotion and stimulus analysis, it is an

attempt to draw a general pattern between all respondents, whether there is a pattern between the

respondents. Due to the wide variation in average emotion time expressed, GSR amplitude and

heart rate values between all the respondents, it indicates that given the same stimulus, every

individual reacts differently. Moreover, the values of emotion time, GSR and heart rate are not

directly proportional. Given the same stimulus, every respondent had different levels of the three

parameters expressed. Some participants expressed higher levels in skin conductance in relation

to facial expression while others expressed higher facial expression than skin conductance.

Similarly, there was no distinct pattern when looking at the facial expression levels and heart

rate. Within a participant, there was no distinct pattern found with the levels of the three

parameters measured. For example, there was no parameter that was consistently higher than the

others found within the same participant. The different levels of the three parameters measured

were dependent on both the stimulus and emotion.

P a g e | 64

Per respondent analysis was a representation of every emotion expressed by the

participant. This analysis allows for the comparison of the different values of the three

parameters within each participant, whether there is a pattern or correlation of facial expression

and GSR values and heart rate for each respondent. Like the per stimulus and emotion analysis,

two sets of graphs were created, with one comparing the average emotion time with average

GSR amplitude and the other comparing the average emotion time with average heart rate

values. For each emotion, the values were taken from the stimulus that gave the highest average

time for that emotion being analyzed since it is the most relevant. Every stimulus induced most

or all the different emotions, however, each emotion were expressed at different intensities. The

GSR amplitudes and heart rate also fluctuated between the different emotions.

From the graphs, linear regression analysis was performed, and the correlation

coefficients were found for each respondent. The graphs were a representation of the correlation

of average GSR amplitude and average emotion time and the correlation of average heart rate

and average emotion time. A correlation coefficient is a number that quantifies a relationship

between two values. A perfect positive correlation would be indicated by a value of 1 while a

perfect negative correlation is represented by a value of -1. A positive correlation signifies that

both variables move in the same direction, that is, when one variable increases, the other

increases as well. Similarly, when one variable decreases, the other variable does so as well. A

negative correlation occurs when the two variables move in opposition to each; therefore, when

one variable increases, the other decreases, and vice versa. This data implies that there is a

slightly higher correlation between facial expression and GSR compared to facial expression and

heart rate.

The most dominant emotions displayed by the participants were anger and surprise. This

could mean that the stimuli induced anger and surprise more easily than the other emotions.

Due to the resource and time constraints as well as the limited scope of this

undergraduate research project, the exact significance of these correlations could not be

determined. Additionally, a longer break between each video could have been inserted to allow

the respondents to fully return to baseline measurements. This would have ensured that no

lingering emotions or physiological reactions from the previous stimulus would carry over to the

following stimulus. Furthermore, facial expressions, GSR, and heart rate values could not be

analyzed simultaneously because the three parameters are measured in different units of

measurement. Future research on this topic should explore potential paradigms that would allow

for the concurrent analysis of the three parameters to make an inclusive conclusion on the

correlation between facial reactions, GSR, and heart rate.

Acknowledgment

We would like to thank the Forensic Science program at University of Windsor for

providing the facilities and resources including this excellent new technology tool to conduct this

forensic research project. We would also like to thank the i-Motions team for providing support

and training, as well as sharing their expertise and insight that tremendously aided in the analysis

of the data.

P a g e | 65

References

1. Al Masri, A., Jasra, S. K. (2016) The Forensic Biometric Analysis of Emotions from Facial

Expressions and Physiological Processes from the Heart and Skin. Journal of Emerging Forensic

Sciences Research. 1(1), 61-77.

2. Critchley, H. D., Rotshtein, P., Nagai, Y., O'doherty, J., Mathias, C. J., & Dolan, R. J. (2005).

Activity in the human brain predicting differential heart rate responses to emotional facial

expressions. NeuroImage, 24(3), 751-762.

3. Gangadharbatla, H., Bradley, S., & Wise, W. (2013). Psychophysiological responses to background

brand placements in video games. Journal of Advertising, 42(2-3), 251-263.

4. Guthrie, C. (2015, June). Experts: Biometrics need to play deeper role in forensics | Planet

Biometrics News. Retrieved April 12, 2017, from http://www.planetbiometrics.com/article-

details/i/3187/desc/expert-biometrics-need-to-play-deeper-role-in-forensics/

5. iMotions Global. (n.d.). Facial Expression Analysis Solutions. Retrieved April 12, 2017, from

https://imotions.com/facial-expressions/

6. Jain, A. K., & Ross, A. (2015). Bridging the gap: from biometrics to forensics. Philosophical

Transactions of the Royal Society B: Biological Sciences, 370(1674).

7. Jang, E., Park, B., Park, M., Kim, S., & Sohn, J. (2015). Analysis of physiological signals for

recognition of boredom, pain, and surprise emotions. Journal of Physiological Anthropology, 34(1).

8. Kortelainen, J., Tiinanen, S., Huang, X., Li, X., Laukka, S., Pietikainen, M., & Seppanen, T. (2012).

Multimodal emotion recognition by combining physiological signals and facial expressions: A

preliminary study. 2012 Annual International Conference of the IEEE Engineering in Medicine and

Biology Society.

9. Mestanlk, M., Vlsnovcova, Z., & Tonhajzerova, I. (2014). The Assessment of the Autonomic

Response to Acute Stress Using Electrodermal Activity. Acta Medica Martiniana, 14(2).

10. Scherer, K. R. (2005). What are emotions? And how can they be measured? Social Science

Information, 44(4), 695-729.

11. Schwartz, G. E., Weinberger, D. A., & Singer, J. A. (1981). Cardiovascular Differentiation of

Happiness, Sadness, Anger, and Fear Following Imagery and Exercise1. Psychosomatic Medicine,

43(4), 343-364.

P a g e | 66

Appendix A: Emotion and Stimulus Analysis Graphs

Figure A1: Average fraction of joy time and average GSR amplitude for all respondents for

stimulus 3.

Figure A2: Average percent of joy time and average heart rate for stimulus 3 for all respondents.

0

0.05

0.1

0.15

0.2

0.25

0.3

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

R1 R10 R2 R3 R4 R5 R6 R7 R8 R9

Avg

. GSR

Am

plit

ud

e (μ

S)

Avg

. Fra

ctio

n o

f Jo

y Ti

me

Respondent

Joy Time and GSR For Stimulus 3

Avg. Fraction Joy Time Avg. GSR Amp.

0

20

40

60

80

100

120

0

10

20

30

40

50

60

70

80

90

100

R1 R10 R2 R3 R4 R5 R6 R7 R8 R9

Avg

. He

art

Rat

e (

bp

m)

Avg

. Em

oti

on

Tim

e (

%)

Respondent

Joy Time and Heart Rate For Stimulus 3

Avg. % Joy Time Avg. HR

P a g e | 67

Figure A3: Average fraction of fear time and average GSR amplitude for all respondents for

stimulus 3.

Figure A4: Average percent of fear time and average heart rate for stimulus 3 for all

respondents.

0

0.05

0.1

0.15

0.2

0.25

0.3

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

R1 R10 R2 R3 R4 R5 R6 R7 R8 R9

Avg

. GSR

Am

plit

ud

e (

μS)

Avg

. Fra

ctio

n o

f Fe

ar T

ime

Respondent

Fear Time and GSR For Stimulus 3

Avg. Fraction Fear Time Avg. GSR Amp.

0

20

40

60

80

100

120

0

10

20

30

40

50

60

70

80

90

100

R1 R10 R2 R3 R4 R5 R6 R7 R8 R9

Avg

. He

art

Rat

e (

bp

m)

Avg

. Fe

ar T

ime

(%

)

Respondent

Fear Time and Heart Rate for Stimulus 3

Avg. % Fear Time Avg. HR

P a g e | 68

Figure A5: Average fraction of contempt time and average GSR amplitude for all respondents

for stimulus 2.

Figure A6: Average percent of contempt time and average heart rate for stimulus 2 for all

respondents.

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

R1 R10 R2 R3 R4 R5 R6 R7 R8 R9

Avg

. GSR

Am

plit

ud

e (

μS)

Avg

. Fra

ctio

n o

f C

on

tem

pt

Tim

e

Respondent

Contempt Time and GSR For Stimulus 2

Avg. Fraction Contempt Time Avg. GSR Amp.

0

20

40

60

80

100

120

0

10

20

30

40

50

60

70

R1 R10 R2 R3 R4 R5 R6 R7 R8 R9

Acg

. He

art

Rat

e (

bp

m)

Avg

. Co

nte

mp

t Ti

me

(%

)

Respondent

Contempt Time and Heart Rate For Stimulus 2

Avg. % Contempt Time Avg. HR

P a g e | 69

Figure A7: Average fraction of disgust time and average GSR amplitude for all respondents for

stimulus 8.

Figure A8: Average percent of disgust time and average heart rate for stimulus 8 for all

respondents.

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0

0.2

0.4

0.6

0.8

1

1.2

R1 R10 R2 R3 R4 R5 R6 R7 R8 R9

Avg

. GSR

Am

plit

ud

e (

μS)

Avg

. Fra

ctio

n o

f D

isgu

st T

ime

Respondent

Disgust Time and GSR For Stimulus 8

Avg. Fraction Disgust Time Avg. GSR Amp.

0

20

40

60

80

100

120

0

20

40

60

80

100

120

R1 R2 R3 R4 R5 R6 R7 R8 R9

Avg

. He

art

Rat

e (

bp

m)

Avg

. Dis

gust

Tim

e (

%)

Respondent

Disgust Time and Heart Rate For Stimulus 8

Avg. % Disgust Time Avg. HR

P a g e | 70

Figure A9: Average fraction of sadness time and average GSR amplitude for all respondents for

stimulus 6.

Figure A10: Average percent of sadness time and average heart rate for stimulus 6 for all

respondents.

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0

0.2

0.4

0.6

0.8

1

R1 R10 R2 R3 R4 R5 R6 R7 R8 R9

Avg

. GSR

Am

plit

ud

e (

μS)

Avg

. Fra

ctio

n o

f Sa

dn

ess

Tim

e

Respondent

Sadness Time and GSR For Stimulus 6

Avg. Fraction Sadness Time Avg. GSR Amp.

0

20

40

60

80

100

120

0

20

40

60

80

100

120

R1 R10 R2 R3 R4 R5 R6 R7 R8 R9

Avg

. He

art

Rat

e (

bp

m)

Avg

. Sad

ne

ss T

ime

(%

)

Respondent

Sadness Time and Heart Rate For Stimulus 6

Avg. % Sadness Time Avg. HR

P a g e | 71

Appendix B: Per Respondent Analysis Graphs

Figure B1: Average fraction of each emotion time and average GSR amplitude, for the same


Figure B2: Average percent of each emotion time and average heart rate for the same stimulus,


0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


Avg

. GSR

Am

plit

ud

e (

μS)

Frac

tio

n o

f A

vg. E

mo

tio

n T

ime

Emotion



74

76

78

80

82

84

86

88

0

20

40

60

80

100


Avg

. H

ear

t R

ate

(b

pm

)

Avg

. Em

oti

on

Tim

e (

%)

Emotion



P a g e | 72





0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0

0.2

0.4

0.6

0.8

1

1.2


Avg

. GSR

Am

plit

ud

e (

μS)

Frac

tio

n o

f A

vg. E

mo

tio

n T

ime

Emotion



94

96

98

100

102

104

106

108

110

0

20

40

60

80

100

120


Avg

. H

ear

t R

ate

(b

pm

)

Avg

. Em

oti

on

Tim

e (

%)

Emotion



P a g e | 73





0

0.02

0.04

0.06

0.08

0.1

0.12

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


Avg

. GSR

Am

plit

ud

e (

μS)

Frac

tio

n o

f A

vg. E

mo

tio

n T

ime

Emotion



0

10

20

30

40

50

60

70

80

90

0

10

20

30

40

50

60

70

80

90

100


Avg

. H

ear

t R

ate

(b

pm

)

Avg

. Em

oti

on

Tim

e (

%)

Emotion



P a g e | 74





0.031

0.032

0.033

0.034

0.035

0.036

0.037

0.038

0.039

0.04

0.041

0

0.2

0.4

0.6

0.8

1

1.2


Avg

. GSR

Am

plit

ud

e (

μS)

Frac

tio

n o

f A

vg. E

mo

tio

n T

ime

Emotion



65

66

67

68

69

70

71

72

73

74

75

0

20

40

60

80

100

120


Avg

. H

ear

t R

ate

(b

pm

)

Avg

. Em

oti

on

Tim

e (

%)

Emotion



P a g e | 75





0

0.01

0.02

0.03

0.04

0.05

0.06

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7


Avg

. GSR

Am

plit

ud

e (

μS)

Frac

tio

n o

f A

vg. E

mo

tio

n T

ime

Emotion



0

10

20

30

40

50

60

70

80

90

0

10

20

30

40

50

60

70


Avg

. H

ear

t R

ate

(b

mp

)

Avg

. Em

oti

on

Tim

e %

Emotion



P a g e | 76





0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0.045

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8


Avg

. GSR

Am

plit

ud

e (

μS)

Frac

tio

n o

f A

vg. E

mo

tio

n T

ime

Emotion



70

72

74

76

78

80

82

0

10

20

30

40

50

60

70

80


Avg

. H

ear

t R

ate

(b

pm

)

Avg

. Em

oti

on

Tim

e (

%)

Emotion



P a g e | 77





0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0

0.2

0.4

0.6

0.8

1

1.2


Avg

. GSR

Am

plit

ud

e (

μS)

Frac

tio

n o

f A

vg. E

mo

tio

n T

ime

Emotion



75

75.5

76

76.5

77

77.5

78

78.5

79

0

20

40

60

80

100

120


Avg

. H

ear

t R

ate

(b

pm

)

Avg

. Em

oti

on

Tim

e (

%)

Emotion



P a g e | 78





0

0.005

0.01

0.015

0.02

0.025

0.03

0

0.2

0.4

0.6

0.8

1

1.2


Avg

. GSR

Am

plit

ud

e (

μS)

Frac

tio

n o

f A

vg. E

mo

tio

n T

ime

Emotion



62

64

66

68

70

72

74

0

20

40

60

80

100

120


Avg

. H

ear

t R

ate

(b

pm

)

Avg

. Em

oti

on

Tim

e (

%)

Emotion



P a g e | 79

Appendix C: Per Respondent GSR and Emotion Correlation Graphs

Figure C1: Linear regression graph for correlation between average GSR amplitude and average

fraction emotion time for respondent 10.



y = -0.0101x + 0.0311R² = 0.81

0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0 0.2 0.4 0.6 0.8 1

Avg

. GSR

Am

plit

ud

e (

μS)

Avg. Fraction of Emotion Time

Emotion and GSR Correlation For Respondent 10

y = 0.1742x + 0.0162R² = 0.079

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0 0.02 0.04 0.06 0.08 0.1

Avg

. G

SR

Am

plitu

de

(μ

S)



P a g e | 80





y = 0.1141x + 0.1284R² = 0.4834

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0 0.2 0.4 0.6 0.8 1 1.2

Avg

. GSR

Am

plit

ud

e (

μS)



y = -0.0028x + 0.0386R² = 0.0972

0.034

0.035

0.036

0.037

0.038

0.039

0.04

0.041

0 0.2 0.4 0.6 0.8 1 1.2

Avg

. GSR

Am

plit

ud

e (

μS)



P a g e | 81





y = 0.0249x + 0.0193R² = 0.2313

0

0.01

0.02

0.03

0.04

0.05

0.06

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

Avg

. GSR

Am

plit

ud

e (

μS)



y = 0.0248x + 0.0191R² = 0.4388

0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0.045

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

Avg

. GSR

Am

plit

ud

e (

μS)



P a g e | 82





y = -0.0314x + 0.0884R² = 0.1469

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0 0.2 0.4 0.6 0.8 1 1.2

Avg

. GSR

Am

plit

ud

e (

μS)



y = -0.0142x + 0.0474R² = 0.1811

0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0.045

0.05

0 0.2 0.4 0.6 0.8 1 1.2

Avg

. GSR

Am

plit

ud

e (

μS)



P a g e | 83



y = 0.0024x + 0.0231R² = 0.003

0

0.005

0.01

0.015

0.02

0.025

0.03

0 0.2 0.4 0.6 0.8 1 1.2

Avg

. GSR

Am

plit

ud

e (

μS)



identifying correlation between facial expression and

Documents