chapter 2 emotion detection architecture &...

40
Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects the first two objectives of this research, which covers design and development of a system for the computer/non-computer users to detect emotion though the psychophysiological signals (GSR/BVP/Temperature). This is done to fulfil the affective sensing requirements of a prospective affective computing system. This work deals with the instrument/system, the methodology used for the acquisition of the signals from the subjects, and the procedure that how this information is sent to the microcontroller (MSP430F2013). In this chapter, research also reflects the implementation of a new proposed model, which is one solution to the conventional models and gives an accurate analogue to digital domain conversions. This chapter discusses hardware and software aspects of the proposed model 2.1 Introduction Although the action of the autonomic nervous system cannot be controlled directly, it can be inclined in an indirect way by two mechanisms called conditioning and biofeedback.(Kandel, 2014, Lang, 2014, Grossman et al., 2013) Biofeedback is a therapeutic method in which people are trained to improve their health by using the signals from their own bodies. Physical therapists use the biofeedback method to help the stroke victims regain movement in the paralyzed muscles.(Tate and Milner, 2010) Psychologists use it to help the tense and anxious clients learn to relax. Specialists in many different fields use the biofeedback method to help their patients cope with the pain. Biofeedback is a means for relieving the ache, gaining control of our body procedures to augment relaxation, and developing a good health and more comfortable life patterns. Clinical biofeedback follows the same principle, using specialized instruments to monitor diverse physiological processes as they occur. The patterns on a computer screen and the audio tones that go up and down imitate the changes as and when they happen in the body system being monitored. (Morris and Guilak, 2009)

Upload: others

Post on 15-Aug-2020

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

Chapter 2

EMOTION DETECTION ARCHITECTURE & SYSTEM

This key chapter reflects the first two objectives of this research, which covers design and

development of a system for the computer/non-computer users to detect emotion though the

psychophysiological signals (GSR/BVP/Temperature). This is done to fulfil the affective sensing

requirements of a prospective affective computing system. This work deals with the

instrument/system, the methodology used for the acquisition of the signals from the subjects, and

the procedure that how this information is sent to the microcontroller (MSP430F2013). In this

chapter, research also reflects the implementation of a new proposed model, which is one

solution to the conventional models and gives an accurate analogue to digital domain

conversions. This chapter discusses hardware and software aspects of the proposed model

2.1 Introduction

Although the action of the autonomic nervous system cannot be controlled directly, it can

be inclined in an indirect way by two mechanisms called conditioning and

biofeedback.(Kandel, 2014, Lang, 2014, Grossman et al., 2013) Biofeedback is a

therapeutic method in which people are trained to improve their health by using the

signals from their own bodies. Physical therapists use the biofeedback method to help the

stroke victims regain movement in the paralyzed muscles.(Tate and Milner, 2010)

Psychologists use it to help the tense and anxious clients learn to relax. Specialists in

many different fields use the biofeedback method to help their patients cope with the

pain. Biofeedback is a means for relieving the ache, gaining control of our body

procedures to augment relaxation, and developing a good health and more comfortable

life patterns. Clinical biofeedback follows the same principle, using specialized

instruments to monitor diverse physiological processes as they occur. The patterns on a

computer screen and the audio tones that go up and down imitate the changes as and

when they happen in the body system being monitored. (Morris and Guilak, 2009)

Page 2: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

Example: Biofeedback provides us the data about ourselves by the means of peripheral

instruments. Using a thermometer to measure our temperature is a common example of

biofeedback.(Smalls et al., 2009)The biofeedback training publicizes us with the activity

in our diverse systems in a body so we may discover to control this activity to relieve

stress and improve health. Many stress-related illnesses (such as headaches and low back

pain) occur due to the over activation of the physiological systems in a response to the

stressful events.(Ulrich-Lai and Herman, 2009)

The biofeedback training is an educational procedure for knowledge the particular

mind/body skills. Learning to identify the physiological reactions and varying them is not

unlike knowledge how to play the piano or tennis – it requires practice. Through practice,

we become familiar with our own exclusive psychophysiological prototypes(Kreibig et

al., 2007) and responses to stress, and learn to control them rather than having them

controls us. A microcontroller-based system is designed to pick up the electrical signals,

such as pulse, GSR, and temperature, froma human body to condition it according to the

requirement and then to display the patient’s condition.

2.2 Architectural View of Conventional Model

The primary purpose of any medical instrumentation system is to measure or determine

the presence of some physical quantity that may, in some way assist the medical

personnel to make better diagnosis and treatment. Any conventional medical device

would comprise the subsequent model.(Yamashita et al., 2007)

Page 3: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

Fig. 2.1: Block Diagram of Conventional Biomedical Instrumentation System(LI et al.,

2013)

2.2.1 Subject

The Subject is the individual body, which generates a range of signals.

Research/investigation on the human body can either be interventional (trial) or

observational (test article). It incorporates both the collection and analysis of data in order

to answer the specific questions. Human subject research often involves surveys,

questionnaires, and interviews(Sawday, 2013).

2.2.2 Transducer/Sensor

A transducer converts one form of energy to another form. The main function of the

transducer is to provide a usable output in response to the subject, which may be a precise

physical quantity, property, or condition. Essentially, the sensor converts a physical

signal to an electrical signal. Depending on the transducer, the production produced can

be in the appearance of voltage, current, resistance, or capacitance. The sensor should be

minimally invasive and interfere with the living system with minimum extraction of

energy. The most important function of the transducer is to provide a usable output

signal(Wang et al., 2005).

2.2.3 Signal Conditioner

For interfacing analog signals to the microprocessor/microcontroller, a data acquisition

system is used. The function of the system is to obtain and digitize the information,

often from the hostile clinical environments, without any degradation in the resolution

or correctness of the signal. The signal conditioner converts the output of the transducer

into an electrical quantity suitable for the operation of the display or recording system.

Signal conditioning typically includes functions, such as amplification, alteration from

analog to digital, or signal transmission circuitry. The buffer amplifier helps in

increasing the sensitivity of the instruments by amplifying the original signal or its

transuded form. The A/D converter carries out the procedure of the analog to digital; the

higher the digit of bits, the higher the accuracy of conversion. Since software expenses

Page 4: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

generally far exceed the hardware costs, the analog/digital interface structure must

permit software efficient transfers of data and command the status signals to avail the

full capability of the microcontroller(Cao et al., 2006).

2.2.4 Display System

The display system provides a noticeable demonstration of the quantity. It may be on

the chart recorder, on the screen of a cathode tube, in a numeric form, or an LCD

display(Anttonen and Surakka, 2005).

2.2.5 Control System

This system controls all operations of the device. It consists of

microprocessor/microcontroller and embedded software to provide the necessary

controls. The control logic provides the necessary interface among the microprocessor

system and the elements of the attainment unit to provide the essential timing control. It

has to sample the data at correct time, make sure that the correct analog signal is

selected, initiate the A/D conversion procedure, and signal to the microcontroller or

microprocessors on completion of the conversion(Schima et al., 2006).

2.2.6 Working of conventional model

Each time you scratch an itch, clutch a snack when you are hungry, or use the bathroom

when you feel the need, you are responding to the biofeedback cues from your body

about your physiologic state. With the biofeedback training, however, you are cued by

the sensors that are attached to your body. This data is conveyed by the visual displays

or sounds. Using imagery and mental exercises, subject (human) learn to use the

feedback provided by the sensors as a measure of success and then you study to control

these functions. With practice, subject can learn to "tune in" without instrumentation

and you can control these purpose.

For example, in a biofeedback training session for annoyance, temperature sensors are

first attached to subject hands, then to his/her feet and ultimately to forehead, if needed.

The subject goal would be to increase blood flow away from the brain by raising the

Page 5: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

temperature in his/her hands or feet. Other sensors strength monitor electro-dermal or

galvanic skin response to determine how simply person sweat or get "goose bumps"

because this affects subject ability to alter his/her skin temperature. To warm up hands

and feet, subject might imagine basking in the sun on a beach while listening to a script

like "I feel warm, my hands are growing warm and heavy" or any external stimuli can

be used e.g watch movie etc. After training session, subject would be sent home with

this script on an audiotape and small thermometers to use for your everyday

practice(Martin et al., 2007).

2.3 Problems in Conventional Architecture

Although instruments based on medial has shown to do tremendous good for the

mankind, still there are some uncovered issues to be solved. The following are the

common architectural issues: (Darwish and Hassanien, 2011)

Complexity

Signal parameter support (e.g. only temperature)

High power consumption

Bulkiness

2.4 Proposed Model (Emotion Detection Model)

The architecture of this emotion detection monitoring system is a novel model and with

more portability, less complexity, low price and, more power efficiency it is a solution to

many problems with the earlier conventional models. This system includes the

mechanism of stimulation, the readings, the measurements, and finally the estimation of

the emotional state (anger, happiness, etc.) of a person. This system takes multiple inputs

from the body and can intelligently analyse those inputs for predicting the emotions.

2.4.1 System Design Process

Design and excellence are an essential part of any biofeedback product. Taking for

example the microcontroller-based system: ergonomics, aesthetics, and engineering have

been considered concurrently as part of the design process as shown in the Fig. 2.2 of the

System Design Process.

Page 6: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

Fig. 2.2: System Design Process

The product must be intended with a user-friendly control panel. Its display should seem

natural and easy to recognize. This feature can be addressed by using only a single input

connector for each of the parameters' methodically programmed and developed user-

interface with the peripherals. During the product design the subsequent design

parameters were considered:

Aesthetics: This is the outward look of a product; attention must be paid to the aesthetics

both in the form design and control panel.(Green, 2007)

Reliability: The functional reliability of the system and the electronic control can be

increased substantially by the use of an intelligent µc, well calibrated and standardized

sensors and conditioning processes.(Narayanan and Xie, 2006)

Maintainability: To ensure an easy maintenance of the system, the design must

incorporate the easy removability of different parts so that the various parts can be re-

assembled quickly for carrying the routine repairs. For an easy maintainability, the

system cards must be designed in a modular form with the standard reliable

connectors.(Kopetz, 2008)

2.4.2 Steps for Designing the Emotion Detection System

The design steps for the design of a standalone biofeedback device are shown in the Fig.

2.3. System identification explains the process and discovers the relationship between

input and output. Requirements determine the needs or conditions to be met for a new or

Product

Ergonomics

Design

Aesthetic

Design

Engineering

Design

Page 7: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

altered product, taking into account the possibly conflicting requirements of various

stakeholders, such as beneficiaries or users. Functional design specifies the sub-processes

that are required in the system.

Fig. 2.3: Product Design of the Biofeedback system

2.5 Proposed Design and How It Works

The architecture of the emotion estimation monitoring system, shown below in the Fig

2.4, has a mechanism to measure the different bio-modalities or bio-signals

(BVP/GSR/Temperature). In the designed product, the validation of subject is done at

priority. It fetch the bio-signals from the subject and then sends it to the MCU

(MSP430F2013).(Sharma and Kapoor, 2013)

Page 8: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

Fig. 2.4: Proposed acquisition system of physiological data and detect emotions.

As shown in this figure, the proposed design of the system covers different requirements:

Portability

Less Cost

Energy Efficiency

Intelligent Analysis

Multi-parameter Support

User Friendly

Home Product

Analysis of Simulations by Doctors

As shown in the Figure 2.4, the system is divided into Part A (below the red bar) and Part

B (above the red bar).

Page 9: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

PART A: Covers all the research objectives. This part has the capability to sense different

bio-signals, convert analog signal to digital, data processing, and do intelligent data

analysis (explained in the chapter 3). Final output is shown by using different coloured

LEDs. The RED LED reflects Stress, YELLOW LED reflects Calmness, and GREEN

LED reflects Joy. The PART A fulfils the research purpose. This part is not complex and

is easy to use. Any user can use the proposed device at home by following a few simple

instructions. Signal processing and control is explained in the next sections of this

chapter.

PART B: This is designed and developed for an extra functionality.

This second part, which is above the red bar, is specially designed for the doctors. By

adding this part, the doctors can check simulations on a monitor and have the detailed

readings of a patient for the records. In GSR, variable voltage according to the body

resistance is fed into the MCU MSP430 for an analog processing. The output is sent to

the 8051 microcontroller through a 2 wire designed protocol, and the final result is

further sent to the PC from a serial port using the UART Communication. Equally,the

BVP Sensor that measuresthe Blood Volume Pulse Rate is integratedin the circuit and

gives the high pulse in synchronization with the heart rate. A light is passed into the

human finger with an LED, which reflects back from the amount of blood. The

phototransistor receives the amount of light and gives the output voltage that is fed into

the MSP430 microcontroller for the analog processing. The output is sent to the 8051

microcontroller through a 2 wire designed protocol, andthe final result is sent to the PC

from a serial port using the UART Communication. In the Temp Sensor (LM-35), MSP

microcontroller has a built-in temperature sensor with which the temperature is measured

directly using the internal SD16 of MSP. The output,as a digital value, is sent to the 8051

microcontroller via a 2 wire self-designed protocol; and the final result is further sent to

the PC from a serial port using the UART Communication. To interface the two MCUs,

an isolator circuit was formed because these MCUs work on different voltages and direct

interfacing was not possible;due to this, the opto-couplers were used to send or receive

the data from either side. Another MCU(8051)(Mazidi et al., 2006) was required to

finally send the data to PC, as there was no UART Communication Protocol present in

the MSP Microcontroller.

Page 10: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

2 Wire Communication Protocol: 14 pin Microcontroller MSP430F2013 is portable but

has limitations. It has limited pins.So, to solve this, a protocol was designed,that is, two

wires Communication Protocol. This protocol can send 16-bit data in one transmission.

The communication is one way only, that is, it can send data only from MSP to 8051, not

vice versa. The data is sent using two pins named DATA and CLOCK. The data that is to

be sent is broken into 16 bits. Then one by one the bit starting from LSB is placed on the

DATA bit .A total of 16 times the clock will go low for sensing the full 16-bit ADC

sample.

From the receiver end, as soon as the clock is received, the interrupt mode stores the

present bit from the DATA pin. So it keeps on storing the bits as received and makes a

full value when 16 clocks are received and then it clears all other variables. This way, a

complete sample of 16 bits could be sent from MSP to 8051 using only two PINS.

Fig. 2.5: Wire Protocol

Optocoupler: It is used to provide the isolation between MSP and 8051, as MSP works on

3V and 8051 works on 5V. Due to this, they need isolation. The general purpose of the

optocouplers consists of a gallium arsenide infrared emitting diode driving a silicon

phototransistor in a 6-pin dual in-line package(Quinones and Joshi, 2007).

The Part B is good for analysing only the simulation. This part makes the system more

complicated and disturbs its portability. It also consumes more power and requiresthe

involvement of a doctor. So, after testing this module, the researcher has kept this part as

optional and maintained the full focus on the Part A only.

2.6 Circuit Diagram of the Proposed System

The initial move of the hardware design is to place the hierarchy of the elements. It is

rational to follow the hierarchical order when looking for the way to connect them

collectively. Once all the components are picked and the respective footprints are found

Page 11: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

in the software, the component placement and wiring can commence. This is an intuitive

part of the design, and certainly takes a few iterations before the “close-to-optimal”

solution is found.

Fig. 2.6: Circuit Diagram

The pin configuration typically can be achieved by adding several external components.

The parallel I/O ability of the MSP430 allows the configuration to control the outside

world by connecting to the external hardware. As explained previously, the PART A

fulfils the research purpose and the PART B is designed and developed for an extra

functionality. So, the circuit diagram above explains the PART A alone. The functions of

the components to the microcontroller MSP430F2013 are listed in the Table below.

Table 2.1: External Components

Pin No External Components Description

Page 12: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

2,3,5 LED

Three LEDs are attached to display the output of the

system in the form of three different emotions:

Green = Joyful

Yellow = Calm

Red = Stress

1 Vcc

1.8 V-3.6 V Supply voltage during the program

execution

4 Thermistor Temperature Sensor

6 Electrodes GSR Sensor

9

Light source (LED) and

light detector (photo

diode) BVP Sensor

2.7 SIGNAL PROCESSING

Signal acquisition is carried out within the input voltage range of the analog-digital

converter (ADC). The task of the ADC is to digitise the analog voltage with a resolution

high enough to represent the original signal. In other words, the quantisation is a process

of mapping a continuous range of values by a finite set of integer values.(Luecke, 2005)

Following are the various steps for acquiring the data from a human body:

2.7.1 Connectivity of input signal with sensors

A biofeedback system needs to deliver and receive information from the user. In order

to receive the data derived from the user's physiological signals, we must use a variety

of sensors. Each of these sensors will account for a particular physiological signal. This

system supports different parameters, and every parameter has its own sensor with a

specific sensing technique.(Ahmed et al., 2011)

2.7.1.1 Galvanic skin response (GSR)

Galvanic skin response (GSR), also known as electrodermal response (EDR),

psychogalvanic reflex (PGR), or skin conductance response (SCR), is a technique of

measuring the electrical resistance of the skin.(Villarejo et al., 2012) EDRs are the

changes in the electrical properties of a person’s skin caused by an interaction between

the environmental events and the individual’s psychological state. Various electrical

properties like conductance (SC), resistance (SR), potentials (SP), impedance (SZ), and

Page 13: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

admittance (SY) are observed. These variations can be sensed in the different parts of the

body (the palm of the hands is of utmost interest). Variations in the ionic content of the

various skin layers, depending upon the amount of sweat and hence upon the sweat

glands' activity, are accountable for these changes.

The electrical conductance of the skin is measured by the silver electrodes (GSR sensor),

which derives the variation from skin’s moisture level. The sympathetic nervous system

controls the sweat glands, thus making the skin’s conductance a good indicator of

physiological arousal.

Structure and Galvanic Skin Function

The skin is a selective barrier that serves the function of preventing the entry of any

foreign matter into the body and selectively facilitating a passage for materials from the

bloodstream to the exterior of the body. There are two forms of sweat glands present in

the human body: the apocrine and the eccrine. The latter is of primary interest to the

psychophysiologists. The primary function of the eccrine sweat glands is

thermoregulation. However, according to Edelberg(Nagai and Critchley, 2008), the sweat

glands on the palm and plantar surfaces are more responsive to the psychological

sweating than other areas. Figure 2.7 below shows the anatomy of the eccrine gland and

various layers of skin.(Milad et al., 2007)

.

Page 14: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

Fig. 2.7: Skin Anatomy(Amirlak et al., 2011)

The skin has electric properties that can change relatively quickly and are closely related

to the psychological process.(Carlson and Carlson, 2012) These changes in the skin’s

conductance and electrodermal activity (EDA) (Boucsein, 2012)are related to the

variations in the eccrine sweating. Sweat act like an electrolyte. As the sweating

increases, the skin pores start filling with the sweat making the skin more conductive.

Autonomic nervous system (ANS) has the sympathetic branch that controls the eccrine

sweating; therefore, the skin conductance reflects the rise of the sympathetic ANS, which

accompanies different psychological processes. Skin conductance and EDA have been

applied in a wide array of research, serving as indicators of such processes as awareness,

habituation, arousal, and cognitive effort in the different sub-domains of psychology and

interrelated disciplines. In judgment and decision making study, the skin conductance is

often used as an indicator of emotional arousal and affective processes.

GSR Measurement

Galvanic skin response is a non-intrusive and easy to apprehend physiological signal,

which is being explored for the emotion sensing. Human skin is a good conductor of

electricity and when a weak electrical current is delivered to the skin, changes in the

skin’s conduction of that signal can be measured. GSR is a method of regulating the

internal physical process by giving a biofeedback, which is effective in the treatment of

phobias, anxiety, and to increase the relaxation process of the subject during the

hypnosis.(Pradeep et al., 2008)

Fig. 2.8: Skin conductance measured through the sweat glands of finger tips(Mandryk et al.,

2006)

Page 15: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

The variable that is measured is either skin resistance or its reciprocal, that is, skin

conductance. GSR is measured in milli volts (mV). According to Ohm’s Law, skin

resistance (R) is equal to voltage (V) applied between the two electrodes on the skin

divided by current passed through the skin (I). The Law can be expressed as

R=V/I.(Rudenko et al., 2013) The GSR is extremely sensitive to the emotions in some

persons;anger, startle response, fear orienting response, and feelings are all among the

emotions that may produce some kind of similar GSR responses. GSR measurement is

also becoming common method in the hypnotherapy and psychotherapy practices.It can

be implemented as a method of extracting depth of hypnotic trance prior to the

commencement of the suggestion therapy. When a traumatic situation is experienced by

the client (for example, during hypnoanalysis), immediate changes in galvanic skin

response can show that the client is experiencing an emotional arousal. It is also applied

in the behaviour therapy to measure the physiological reactions, such as fear.

Range of GSR<5 Kohms indicates a high level of brain arousal and >25 Kohms indicates

a low arousal and withdrawal from mind (calm level). The GSR is measured most

conveniently at the palms of the hand, where body has the highest concentration of sweat

glands. The measurement is made using a DC current source. The Galvanic Skin

Response (GSR) is a measure of the skin's conductance between the two electrodes. The

electrodes are typically attached to the subject's fingers or toes using the electrode cuffs,

or to any other part of the body using a Silver-Chloride electrode patch. To measure the

resistance, a small voltage is applied to the skin and the skin's current conduction is

measured.(Sharma and Kapoor, 2013, Jeon et al., 2007)

The skin conductance is considered to be a function of the sweat gland activity and the

skin's pore size. An individual's baseline skin conductance will vary for many reasons,

including the gender, diet, skin type, and situation. The sweat gland activity is partly

controlled by the sympathetic nervous system. When a subject is startled or experiences

anxiety, there will be a fast increase in the skin's conductance (a period of seconds) due to

the increased activity in the sweat glands (unless the glands are saturated with sweat).

Page 16: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

GSR Sensor

Extremely pure silver electrodes (having silver with purity of 99.999%) are used to

measure the GSR. Electrodes are small plates that apply a safe and imperceptibly tiny

voltage across the skin.

There is saturation effect: when the duct of the sweat gland fills, there is no longer a

possibility of further increase in the skin conductance. The excess sweat pours out of the

duct andthe sweat gland activity increases the skin's capacity to conduct the current

passing through it. The changes in the skin conductance reflect the changes in the level of

arousal in the sympathetic nervous system. It was observed that the Analogic Digital

Converser saturates at 2.35 V. The microcontroller has a built-in ADC of 16 bits with a

resolution of:

2.35/65535=3.5v (1)

The Galvanic Skin Response oscillates between 10 kΩ and 10 MΩ (Sharma and Kapoor,

Villarejo et al., 2012), as can be seen in the existing studies about the skin conductance

obtained from the different applied voltages .As ADC has a resolution of 3.5 V and the

minimum tension is 136 mV, an operational amplifier does not have to be included. This

concept helped in achievingthe third objective, that is, energy efficiency. A person’s skin

acts as a resistance to the passage of electrical current. By placing two electrodes on the

fingers, we can calculate the GSR. To find out the value, one resistance was used, as it

can be seen in the fig 2.9, in series with the skin resistance to form a voltage divider.

Fig. 2.9: Voltage Divider.

2

2

RR

RV

S

O

(2)

Where, Rs is the resistance of the skin.

Page 17: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

It can be observed that the Vo output tension is inversely proportional to the value of the

skin resistance. The more stressed the person is, the more his/her hands will sweat, so

his/her resistance will decrease. Therefore, we can conclude that the more stress the

person is under, the higher output voltage will be.

2.7.1.2Blood Volume Pulse (BVP)

Blood Volume Pulse is the phasic variation in the blood volume with each heart rate,

heartbeat, and heart rate variability (HRV). (Chambers et al., 2005)It consists of beat-to-

beat differences in the intervals between successive heartbeats. During the systole stage,

the muscles of the ventricles contract and force the blood to flow from the ventricles

into the arteries. The rate of heart contractions over a given time period is defined as the

Heart Rate. It is usually expressed in beats per minute (bpm).(Fox et al., 2007)

Heart rate is one of the human body’s vital sign that tells the medical personnel about

the extremity of the casualties’ physiological conditions. It is one of the simplest and the

most informative cardiovascular parameters. With the observation that heart rate

fluctuation is related to various cardiovascular disorders, the analysis of the heart rate

has become a widely used tool in the assessment of the behaviour of the heart.

Page 18: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

Fig. 2.10: Structure of the heart(Borazjani et al., 2010)

Blood passes through the heart in two phases. The phase where the ventricles are filled

with blood is referred to as the "diastole" stage.(Veress et al., 2005) The pumping of the

blood out of the ventricles is referred to as the "systole" stage.(Kazui et al., 2006) During

the systole stage, the blood flows from the ventricles of the heart into the small arteries.

The difference in the size of the opening of the ventricles and the arteries causes a burst

of pressure. This pressure wave expands the arterial walls as it travels and is felt as the

"pulse".(Sutton-Tyrrell et al., 2005)

BVP Measurement

The heart rate varies between individuals. The normal human heart rate at rest is 60 to 80

bpm. At rest, an adult has an average heart rate of 72 bpm. The athletes normally have a

lower heart rate than less active people.(Poh et al., 2010)The heart rate also varies with

age. Children normally have a higher heart rate of approximately 90 bpm.

Table 2.2: Age-related ranges of heart beat(Fink et al., 2009)

Age Beats per minute

Newborn 90-100

10 years 80-90

10+ and adults 60-80

Bhattacharya, Kanjilal(Shi et al., 2009, Bhattacharya et al., 2001) stated that non-invasive

techniques can be used to determine the human body's cardiovascular condition. It was

addressed that the qualitative assessment of the overall clinical status of the

cardiovascular dysfunctions can be determined non-invasively. Various techniques and

devices have been used to measure the heart rate in humans. The pressure sensors

measure the changes in the pressure level near the heart or the vibrations produced by the

heart. The sound sensor measures the changes in the sounds near the heart, and light

sensors detect the changes in the optical property of the blood. There are various methods

to measure the heart rate, such as Mechanical method, Electrical Signal Detection,

Optical method, and Plethysmograph. The most common and accurate technique, which

is used these days is Plethysmograph.

Page 19: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

PLETHYSMOGRAPH (Allen, 2007)is a combination of the Greek word "plethysmos,"

meaning increase and "graph," meaning to write. Plethysmograph was developed in the

1960’s and 1970’s by the psycho-physiology researches.(Fleming, 1980) It is an

instrument that is used to determine the variations in the blood volume or the blood flow

in the body. These transient changes occur with each heart beat. (Lacey and Lacey, 1978)

There are several different types of plethysmograph, which vary according to the type of

transducers that is being used. The common types include: air, impedance, photoelectric,

and strain gauge plethysmograph. Each type of plethysmograph measures the change in

the blood volume in a different manner.(Shimazu et al., 1989, Cheang and Smith, 2003)

Various plethysmographs are explained in the table below:

Table 2.3:Different Types of Plethysmographs.(Terry, 2005)

Types Methodology

Air Uses an air-filled cuff. Measures the rate of change of

forearm volume, which correlates with the change in the

blood volume.

Impedance Uses low frequency alternating current applied through the

electrodes. Measures the change in the electrical

impedance, which corresponds to the change in the blood

volume.

Photoelectric Uses photodetectors. Measures the intensity of the

transmitted and reflected light, which demonstrates the

volume change in the blood perfusion.

Strain gauge Uses a rubber tube filled with mercury. Measures the

changes in limb circumference, which relates to the

changes in the blood volume.

BVP Sensor

The heart rate sensor used in this research is based on the principle of the Photoelectric

plethysmography method. This methodis also known as photoplethysmography (PPG)

and is an optical measurement technique used for detecting the blood volume changes in

the micro vascular bed of tissues. This method uses a light source (LED) to illuminate the

skin and a light detector (photo diode) to detect the changes in the optical properties due

to the change in the blood volume. This method has become very popular in the medical

Page 20: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

field, especially, in the pulse oximetry due its simple, non-invasive, and unobtrusive

monitoring.(Barreto et al., 1995)

It measures the heart rate by determining the blood volume changes in the skin periphery

(finger-tip and ear-lobe) by the photo-electric method. Compared to the other types of

plethysmograph methods mentioned in the Table 2.3, PPG technique is simple to use,

easy to set up, and low in cost.(Allen, 2007)

Dr. Nolan (Wallace et al., 2011) proposed that photoplethysmography is a non-invasive

technique that can be used to measure the variations in the heart rate.

“A PPG can prove to be quite helpful in measuring the HRV. There is some exciting

research going on in determining HRV using PPG. The analysis of signal from PPG has

great potential for enriching the interpretation of HRV.”

A plethysmograph consists of:

i. A light source, which illuminates the tissue.

ii. A light sensitive detector, which detects the amount of light transmitted from the

tissue.

Page 21: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

Fig. 2.11: Arrangement of a plethysmograph(Stojanovic and Karadaglic, 2007)

Photoplethysmography (PPG) works by placing an individual’s finger tip or ear-lobe

between two parts of a transducer consisting of a light source and a light sensitive

detector. A beam of infrared light is projected towards the detector. The blood in the

finger or ear-lobe scatters the light in the infrared range, and the amount of light reaching

the detector is inversely related to the volume of blood in the skin periphery.(Kamal et

al., 1989, Elgendi, 2012)

PPG is based upon the premise that all living tissues and blood have different light

absorbing properties. The infrared light is absorbed well in blood whereas, weakly in the

tissues.(Sundararajan, 2010)

The Figure 2.12 shows the absorption level of the infrared light in the living tissues and

blood. When the blood vessels in the finger dilate, the increased blood flow allows less

light to reach the photo-detector and when the blood vessels contract, the blood flow is

decreased and increased light reaches the photo-detector.

Fig. 2.12: Relative absorption levels of infrared light of skin

The photoplethysmograph waveform: The photoplethysmograph waveform does not

resemble the pulse seen in an electrocardiogram (which is used to record the electrical

activity of the heart). However, the periodicity of the signal is unchanged and the

Page 22: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

photoplethysmographic waveform can be effectively used to detect the changes in the heart

rate.(Peper et al., 2010)

Dicrotic Notch Anacrotic Limb

Time

Waveform (mV)

Fig. 2.13: Representation of the Photoplethysmograph waveform(Peper et al., 2010)

The upstroke, called the anacrotic limb, is abrupt due to the contraction of the ventricle

(systole). The downstroke is more gradual and corresponds to the elastic recoil of the

arterial walls. The downstroke regularly shows a fluctuation that is known as the dicrotic

notch. This is due to the vibrations set up when the aortic valves shuts. (Maffei,

2012)Each time the heart muscle contracts, blood is ejected from the ventricles and a

pulse of pressure is transmitted through the circulatory system. This pressure pulse while

travelling through the vessels causes the vessel-wall displacement, which is measurable at

various points of the periphery circulatory system.

Two methods are commonly used to measure the heart rate by the optical method. These

are:

1. Transmittance method

2. Reflectance method

Transmittance method: In the transmittance method, the light source and the light

sensitive detector are mounted in an enclosure that fits over the patient’s fingertip or ear-

lobe.(Algorri et al., 2013)

Page 23: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

Fig. 2.14: Arrangement of light source and light sensitive detector: Transmittance method

Light is transmitted through the finger tip of the patient’s finger and the output of the

light sensitive detector is determined by the amount of light reaching the detector. With

each contraction of the heart muscles, blood is forced to the extremities and the amount

of blood in the finger increases. This alters the optical density with the result that the light

transmission through the finger reduces and the resistance of the light sensitive detector

increases accordingly.

Reflectance method: This method is used in this research. The arrangement used in the

reflectance method of photoelectric plethysmography is shown in the Fig 2.15. In the

reflectance method, the light sensitive detector is placed adjacent to the light source. Part

of the light rays emitted by the light source is reflected and scattered from the skin tissues

and falls on the photodetector.(Park et al., 2013)

Fig. 2.15: Arrangement of light source and light sensitive detector: Reflectance method

(Park et al., 2013)

Page 24: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

The quantity of light that is reflected is determined by the tissue back-scattered or the

absorbed optical radiation. The output of the photodetector varies in proportion to the

volume changes of the blood vessels.

The signal from the heart rate sensor is then sent to a part of the microcontroller where all

the processing takes place for the beats per minute (BPM) value calculation. The timer is

programmed in an auto-reload mode, so that it overflows at a regular interval and

generates an interrupt at 10µsec intervals.

In order to generate the interrupts at 10µsec interval, the reload value for the timer had to

be calculated for a system clock of 22.118 MHz. The timer low byte (TL0) operates as a

16-bit timer while the timer high byte (TH0) holds the reload value. When the count in

TL0 overflows, the timer flag is set and the value in TH0 is loaded into TL0. The TH0

value was calculatedusingthe following equation:

Tsysclk = 1

Fsysclk (3)

Fsysclk is a system clock frequency of 22.118 MHz

i.e. Tsysclk = 1

22.118 X(10)6Hz =0.045µsec (4)

2.7.1. C Skin Temperature

The human skin is an organ made up of a layer of tissues that protect the underlying

muscles and organs. As skin comes in a direct contact with the surroundings, it plays a

vital role in protecting the inner body from the external threats. The skin is the largest

organ of the human body, as it covers the whole body and has the largest surface area. It

weighs more than any single organ of the body. (Kenefick et al., 2010)The skin has two

major layers: the epidermis and the dermis. These layers are made of the different types

of tissues and have different functions. The epidermis is the outer-most layer and the

dermis lies below the epidermis and contains a number of structures that are responsible

for lubrication, water-proofing, softening, and anti-bactericidal actions.

The skin temperature is an effective indicator when it comes to evaluate the human

sensations. Kataoka et al. (Shuto et al., 2011) investigated the relationship between the

stressful tasks and the skin temperature. It was found that the skin temperature falls when

Page 25: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

stress, tension, or other sensations occur; because the blood flow decreases due to the

factors lie blood vessel constriction. This was most noticeable at the extremities, such as

fingertips and nose. Similarly, according to Blessing(Ootsuka et al., 2011), the net heat

transfer between the individual and the external environment varies according to the

amount of the blood flowing through the skin, which is regulated as an intrinsic

component of the body temperature control. The non-metabolic factor influencing the

cutaneous blood flow is a sympathetically mediated vigorous vasoconstriction initiated

when the individual perceives a potentially dangerous environmental event. Yamakoshi

et al.(Yamakoshi, 2013) studied driver’s awareness level using the skin temperature. The

researchers measured the facial skin temperature, including the truncal and peripheral

site, of healthy volunteers during simulated monotonous driving. They found that the

sympathetic activity, that is, peripheral vasoconstriction was increased during the

monotonous driving situation, which resulted in a significant gradual drop in the

peripheral skin temperature.

Temperature Sensor and Measurement

Various equipments and instruments have been used in the past for the body temperature

measurement. The most common device to measure the body temperature is a

thermometer. Thermometer is a combinationof two Greek words; "thermo," which means

heat and "meter," which means measure. Therefore, a thermometer is a device that

detects the change in the heat level and converts it into a temperature value.(Boano et al.,

2011) There are different types of thermometers. The most common ones include

mercury-in-glass, infrared, gas, plastic strip, and liquid crystal thermometers. However,

the mercury-in-glass thermometers have widely been used for the clinical purposes. The

other devices that are used for measuring the temperature include thermocouples,

thermistors, resistance temperature detectors (RTD), and silicon band gap temperature

sensors. All these temperature measuring devices are designed to measure the

temperature for specific objects or environments. The temperature can be measured using

different scales. The most common temperature scales used and accepted internationally

are the Kelvin or Absolute, Centigrade or Celsius, and Fahrenheit scale. (Yin et al., 2010)

Page 26: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

Fig. 2.16: Temperature Sensor(Yu et al., 2010)

Choose R1 = –VS / 50 µA

VOUT = 1500 mV at 150°C

VOUT = 250 mV at 25°C

VOUT = –550 mV at –55°C

In this research LM35 is used as a temperature sensor. The LM35 series are precision

integrated-circuit temperature sensors with an output voltage that is linearly proportional

to the Centigrade temperature. Thus the LM35 has an advantage over the linear

temperature sensors that are calibrated in Kelvin, as the user is not required to subtract a

large constant voltage from the output to obtain a convenient Centigrade scaling. LM35

does not require any external calibration or trimming to provide the typical accuracies of

±¼°C at the room temperature and ±¾°C over a full −55°C to +150°C temperature range.

A low cost is assured by trimming and calibration at the wafer level. The low output

impedance, linear output, and precise inherent calibration of the LM35 sensor make

interfacing to readout or control circuitry especially easy.

The device is used with single power supplies, or with plus and minus supplies. As the

LM35 sensor draws only 60 μA from the supply, it has very low self-heating of less than

0.1°C in the still air. The LM35 is rated to operate over a −55°C to +150°C temperature

range, while the LM35C is rated for a −40°C to +110°C range (−10° with improved

accuracy). The LM35 series is available in the hermetic TO transistor packages, while the

LM35C, LM35CA, and LM35D are also available in the plastic TO-92 transistor

package. The LM35D is also available in an 8-lead surface-mount small outline package

and a plastic TO-220 package.

Page 27: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

There are a number of devices available for monitoring or observing the human

temperature. In this research, the aim was to go for a low-cost, compact, reliable, and

accurate temperature sensor that is capable of monitoring the skin temperature with ease

and comfort.

As stated earlier, the output from the temperature sensor is an analog voltage. This output

signal from the sensor is used as the input for the smicrocontrollerthrough the analog port

pin. The microcontroller is programmed to perform the required processing and

conversion from a voltage value into a temperature value. The relationship between the

voltage value and the temperature value is calculated by the following equation:

T(°C) = Vout−Vos

∆V/∆T (5)

Where

Vos=Ds offset, 509mv

∆V/∆T=Typical output gain,+6.45 mV/°C

2.8 Microcontroller Overview

The popular of all the electrical systems today employ some sort of microcontroller

technology. A microcontroller’s inexpensive, flexible, and self-sufficient design permits

it to command almost any modern task that employs some form of embedded systems.

From cars to refrigerators to handheld devices, microcontrollers play a dominant role in

the development of many different products for many different companies.

In this research, the Microcontroller used is MSP430F2013.The MSP430F2013 includes

a 16bit CPU, 16-bit timer,16-bit Sigma Delta Analog-to-Digital converter, brownout

detector,Watchdog timer, USI module supporting SPI and I2C serial communication

standards, and five low power modes drawing as little as 0.1µA standby current. TI’s

Ultra-Low Power microcontroller, MSP430, uses EZ430, a USB stick implementation of

a full development kit that includes power supply, I/O access, additional debugging

hardware, and few extra peripherals.

Page 28: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

Fig. 2.17: EZ430-F2013 - MSP430 16-bit microcontroller USB Stick

The MSP430 (the controller for the EZ430) employs a Reduced Instruction Set Computer

architecture (RISC) CPU. The eZ430-F2013 is a complete MSP430 development tool

including all the hardware and software to assess the 16-bit mixed signal microcontroller

MSP430F2013 and to develop a complete solution that works in a suitable USB stick

form factor. The eZ430-F2013 supports the Code Composer Studio and IAR Embedded

Workbench Integrated Development Environments to provide a full emulation with the

option of designing using a stand-alone system or detaching the removable target board

to integrate into an existing design. The USB port provides enough power to operate the

ultra-low-power MSP430, so no external power supply is required.

2.8.1 MSP430F2013 Architecture

The MSP430CPU has a 16-bit RISC architecture that is highly clear to the application.

All operations, other than the program-flow instructions, are performed as registered

operations in conjunction with seven addressing modes for source operand and four

addressing modes for destination operand. The CPU is integrated among 16 registers that

provide a reduced instruction execution time. The register-to-register operation execution

time is one cycle of the CPU clock. Four of the registers, R0 to R3, are respectively

designated as the stack pointer, constant generator, program counter, and status register.

The remaining registers are called the general-purpose registers. The peripherals are

Page 29: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

connected to the CPU using the address, data, and control buses. It can be handled with

all instructions.(Frederic et al., 2013)

Fig. 2.18: Architectural view of MSPF2013(Megalingam et al., 2011)

The MSP430 von-Neumann architecture has one address space shared with the special

function registers (SFRs), peripherals, RAM, and Flash/ROM memory. The device-

specific data sheets are available for the specific memory maps. The code accesses are

always performed on the even addresses. The data can be accessed as bytes or words. The

addressable memory space currently is 128 KB.

The CPU incorporates sixteen 16-bit registers. R0, R1, R2, and R3 have dedicated

functions. The 16-bit program-counter (PC/R0) points to the next instruction to be

executed. The stack pointer (SP/R1) is used by the CPU to store the return addresses of

the subroutine calls and interrupts. The status register (SR/R2), used as a source or

destination register, can be used in the register mode only addressed with word

instructions. The RISC instruction set of the MSP430 has only 27 instructions. The

constant generator allows the MSP430 assembler to support 24 additional and emulated

instructions. The twelve registers, R4 to R15, are general-purpose registers. All of these

registers can be used as data registers, address pointers, or index values; and can be

accessed with byte or word instructions. Seven addressing modes for the source operand

Page 30: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

and four addressing modes for the destination operand can address the complete address

space with no exceptions.

2.8.2 Modular design

The following PCB diagram shows the arrangement of the hardware on the EZ430.

Notice how the actual MSP430 attaches to the debugging and USB interfacing hardware

through a 4-pin, JTAG port.

Fig. 2.19: PCB diagram(Zantis, 2012)

Dedicated embedded emulation logic resides on the device itself and is accessed via

JTAG using no additional system resources.

The benefits of embedded emulation include:

Unobtrusive development and debugging with full-speed execution, breakpoints, and

single-steps in an application are supported.

Development is in-system subject to the same characteristics as the final application.

Mixed-signal integrity is preserved and not subject to the cabling interference

The eZ430-F2013 can be used as a stand-alone development board. Additionally, the

MSP-EZ430D target board may also be detached from the debugging interface and

integrated into another design. The plastic enclosure can be removed to expose the MSP-

EZ430U debugging interface and the MSP-EZ430D target board. The MSPEZ430D

target board can be disconnected from the debugging interface by gently pulling the two

boards apart. The target board can be used in a stand-alone design by interfacing to the

14-pins of the MSP430F2013. The holes in the MSP-EZ430D target board provide a

direct access to each pin of the MSP430F2013. The MSP-EZ430U debugging interface

may also be used as a standard Flash Emulation Tool for all devices in the MSP430F20xx

Page 31: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

family of the microcontrollers. The target boards can be designed and flashed using the

MSP-EZ430U debugging interface and other supported MSP430F20xx devices.(Gaspar

et al., 2010)

Fig. 2.20: Pin Diagram of MSP430F2013(Mainoddin and Usha, 2014)

There is one 8-bit I/O port implemented—port P1—and two bits of I/O port P2. All

individual I/O bits are independently programmable. Any combination of input, output,

and interrupt condition is possible. The edge-selectable interrupt input capability is

available for all the eight bits of port P1 and the two bits of port P2. The read/write access

to the port-control registers is supported by all instructions. Each I/O has an individually

programmable pull-up/pull-down resistor.(Mainoddin and Usha, 2014) Following is the

table that describes the details of each pin of the microcontroller that is used here:

Table 2.4: Details of each Pin

Pins Details

P1.0/TACLK/ACLK/C

A0

General-purpose digital I/O pin Timer_A, clock signal

TACLK input ACLK signal output Comparator_A+,

CA0 input

P1.1/TA0/CA1

General-purpose digital I/O pin Timer_A, capture:

CCI0A input, compare: Out0 output Comparator_A+,

CA1 input

Page 32: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

P1.2/TA1/CA2

General-purpose digital I/O pin Timer_A, capture:

CCI1A input, compare: Out1 output Comparator_A+,

CA2 input

P1.3/CAOUT/CA3 General-purpose digital I/O pin Comparator_A+, output

/ CA3 input

P1.4/SMCLK/C4/TCK

General-purpose digital I/O pin SMCLK signal output

Comparator_A+, CA4 input JTAG test clock, input

terminal for device programming and test

P1.5/TA0/CA5/TMS

General-purpose digital I/O pin Timer_A, compare:

Out0 output ADC10 analog input A5 USI: external

clock input in SPI or I2C mode; clock output in SPI

mode JTAG test mode select, input terminal for device

programming and test

P1.6/TA1/A6/SDO/SCL

/TDI/TCLK

General-purpose digital I/O pin Timer_A, capture:

CCI1B input, compare: Out1 output ADC10 analog

input A6 USI: Data output in SPI mode; I2C clock in

I2C mode JTAG test data input or test clock input

during programming and test

P1.7/A7/SDI/SDA/TDO

/TDI+

General-purpose digital I/O pin ADC10 analog input

A7 USI: Data input in SPI mode; I2C data in I2C mode

JTAG test data output terminal or test data input during

programming and test

XIN/P2.6/TA1 Input terminal of crystal oscillator General-purpose

digital I/O pin Timer_A, compare: Out1 output

XOUT/P2.7 Output terminal of crystal oscillator General-purpose

digital I/O pin

RST/NMI/SBWTDIO Reset or non maskable interrupt input Spy-Bi-Wire test

data input/output during programming and test

TEST/SBWTCK

Selects test mode for JTAG pins on Port1.The device

protection fuse is connected to TEST. Spy-Bi-Wire test

clock input during programming and test

VCC Supply voltage

VSS Ground reference

Page 33: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

2.8.3 The SD16 A Sigma-Delta ADC

An ADC takes an analog signal as an input and then converts that analog signal into a

digital stream of bits depending on its reference voltage, precision, and resolution. An n-

bit ADC (A/D converter) provides 2n discrete quantization levels corresponding to

various specified analog input signal amplitude range. There exist a number of A/D

conversion techniques varying in complexity and speed. The outputs from each sensor

are analog in nature. The output signal from the sensor is used as an input into the analog

port pin of the microcontroller. The MPS430F2013 is equipped with an analog-to-digital

(ATD) conversion system that samples an analog (continuous) signal at regular intervals

and then converts each of these analog samples into its corresponding binary value using

a sigma-delta modulation technique. As MSP430F2013 is having an in-built ADC (SD16

A Sigma-Delta), so the microcontroller is programmed to perform the required

processing and conversions.

The SD16 A is a single-converter 16-bit, analog-to-digital conversion module

implemented in the MSP430x20x3 series. It is made up of one sigma-delta analog-to-

digital converter and an internal voltage reference. It has eight fully differential

multiplexed analog input channels, of which three are internal. The operation of the

sigma-delta converters is totally different from the successive-approximation ADCs. The

idea behind them is to reduce the analog-to-digital conversion to 1 bit 1 and to take the

samples a few orders faster than the desired sample rate to compensate for its very poor

resolution. The magnitude of the analog input is then represented by the mean value of

the produced fast bit-stream. The average is then digitally processed to output the

samples at the specified rate. The Fig 2.20 shows the architecture of a sigma-delta

converter. It can be broken down into two parts: the first, with the feedback loop, is

responsible for the analog-digital conversion, whereas, the second converts the fast bit-

stream to the desired sample rate.(Zantis, 2012)

.

Subtrator

Modulator

Decimation Filter

Integrator ADC

DCA

+ - _

Low-Pass

Filter

Decimator

fm fm fs

Analogue

Input

Digital

Input

Page 34: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

Fig. 2.21: Block diagram of a sigma-delta A/D converter

Fig. 2.21:Analog-To-Digital Conversion

The analog-to-digital conversion is done by a 1-bit second-order sigma-delta modulator.

A single-bit comparator within the modulator quantizes the input signal among the

modulator frequency fM. The resulting 1-bit data stream is averaged by the digital filter

for the conversion outcome. The bit-rate of the first part is called the modulator or

oversampling frequency (fm). This is typically much faster than the sample rate (fs) at the

digital output. The decimation filter is a comb type digital filter with selectable

oversampling ratios (OSR = fm/fs) of up to 1024. The filter is also called sinc filter

because its frequency response is alike the sinc(x) = sin(x)/x function. The comb filter is

the sigma-delta converter’s characteristic feature, which has to be taken into account

through the design stage. One may think that it is a downside, however, when it comes to

anti-aliasing or notch filtering, it can be utilised by a sensible software design. The ADC

converts the ∆V = V+ − V− voltage difference among a pair of inputs, rather than the

voltage between a single input and the ground. If this feature is not required, the V−

should be tied to the ground. The sigma-delta converters often give a programmable gain

amplifier (PGA) on their inputs, which may eliminate the need for an additional external

operational-amplifier. These are the plain op-amps with the feedback resistors, and they

do not provide high input impedance. Their analog input voltage range is dependent on

the actual gain setting, which can be increased up to 32 in the SD16 A. The maximum

full-scale range for Vref = 1.2V and GAINPGA = 1 is ±VF SR, where VF SR is defined

by:

VFSR =Vref /2

GAIN =

1.2V/2

1 = ±0.6V (7)

A side effect of the averaging applied in the digital sinc3 filter is that the output does not

react promptly to the change of the input. It needs 4 periods of Ts to elapse until the

reliable value appears. This is called latency, and probably sets the most severe limitation

of sampling frequency when more than one channel is used.

Page 35: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

The F2013 SD16_A conversion system consists of an 8-channel multiplexed input anda

16-bit output sigma delta analog-to-digital converter block. Its features include a software

selectable internal/external voltage, up to a 1.1 MHz modulator input frequency, and a

selectable low-power conversion mode. The converter block is software programmable to

perform either single or continuous conversions into a 16-bit output register that is called

the SD16MEM0 register. The SD16_A module must be initialized using its two control

registers, the SD16 control and channel control (SD16CTL & SD16CCTL0) registers.

When the SD16_A module is not actively converting, it automatically shuts down to

preserve the power while putting together an accurate analog to digital domain

conversion. The following algorithm was used in this research for converting the analog

signal to a digital one.

Algorithm 2.1: Efficient Algorithm for A-D conversions

STEP 1: SD16CTL = SD16REFON + SD16SSEL_1;

// Internal Voltage Ref ON and Clock Division

STEP 2:SD16CCTL0 = SD16UNI;

// Changing SD16 to Unipolar Mode

STEP 3:SD16INCTL0 = SD16INCH_1;

// Selecting Input channel

STEP 4:SD16CTL = SD16REFON + SD16SSEL_1;

// Internal Voltage Ref ON and Clock Division

STEP 5:SD16CCTL0 = SD16UNI;

// Changing SD16 to Unipolar Mode

STEP 6:SD16INCTL0 = SD16INCH_N;

// Selecting Input channel

The SD16CTL register: The SD16_A Control Register is mainly responsible for the

selection of the clock source, the division of the clock into the sigma delta modulator, and

the enablement of the internal voltage reference.

Page 36: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

The SD16 Clock Source Select (SD16SSELx) (Bits 5 – 4): The clock source to be

divided is selected using the clock source select bits, much like the timer module.

The SD16 Reference Generator ON (SD16REFON) (Bit 2): The SD16_A module can use

an internally provided reference voltage for the modulation or it can be provided as a

user-specified voltage reference through the specified ports. The internally provided

reference voltage has a value of 1.2 V and is used when the SD16REFON bit in the

SD16CTL register is set to 1.

Table 2.5: Voltage Reference Generator Bit

SD16REFON Bit Internal Voltage Reference

0 Reference OFF

1 Reference ON

The SD16CCTL0 register: The Channel Control 0 Register is responsible for the

conversion mode, the data output settings, the oversampling ratio, and all interrupt

settings. There are two modes – Bipolar and Unipolar. In this research,only the Unipolar

mode was required. The mode is selected as follows:

Bipolar Mode, SD16UNI = 0

Unipolar Mode, SD16UNI = 1

SD16INCTL0: The analog input into the machine is configured using the Input Control

(SD16INCTL0) and Analog Input Enable (SD16AE) registers. Setting the SD16AE bits,

enable the analog circuitry for the particular differential pair of input pins and disable any

digital circuitry that might be linked to that pin.

SD16INCHx: The SD16INCTL0 Register is dependable for setting the selected input

channel and the SD16INCHx Bits (0 – 2) are responsible for selecting the analog input to

be modulated.

Key Features of MSP430F2013 Microcontroller:

Page 37: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

eZ430-F2013 development tool including a USB debugging interface and detachable

MSP430F2013 target board has the features below:

LED indicator

14 user-accessible pins

eZ430 debugging and programming interface

Supports development with all 2xx Spy Bi-Wire devices (MSP430F20xx, F21x2,

F22xx)

Supports eZ430-T2012 and eZ430-RF2500T target boards

Removable USB stick enclosure

Low Supply Voltage Range 1.8 V to 3.6 V

Ultra-Low Power Consumption

Active Mode: 220 µA at 1 MHz, 2.2 V

Standby Mode: 0.5 µA

Off Mode (RAM Retention): 0.1 µA

Five Power-Saving Modes

Ultrafast Wake-Up from the Standby Mode in less than 1 µs

16-Bit RISC Architecture with 62.5 ns Instruction Cycle Time

16-Bit Timer_Awith Two Capture/Compare Registers

On-Chip Comparator for Analog Signal Compare Function or Slope A/D

16-Bit Sigma-Delta A/D Converter With Differential PGA Inputs and Internal

Reference

Kit Contents

Page 38: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

The evaluation kit contains everything that is needed to develop and run applications

for the MSP430 microcontrollers. It includes:

One eZ430-F2013hardware set, which is housed inside a plastic enclosure that may

be opened in order to separate the MSP-EZ430D target board from the MSP-

EZ430U debugging interface

One MSP430 Development Tool CD-ROM, which contains several documents

including the following related to the eZ430-F2013:

MSP430x2xx Family User's Guide

MSP-FET430 FLASH Emulation Tool User's Guide

MSP-FET430 FLASH Emulation Tool User's Guide Errata

eZ430-F2013 User's Guide

IAR Embedded Workbench Kickstart Version

Code Composer Studio MCU Edition

Software Design

To develop the application software for the data storage tag, the IAR Embedded

Workbench is used. The IAR Embedded Workbench is a set of development tools for

building and debugging the embedded applications using assembler, C, and C++. The 16

bit MSP430 devices from Texas Instrument are supported by the IAR tool. The IAR

development tool can generate a binary file that can be downloaded on the

microcontroller. The status of all the interval registers related to the microcontroller’s

peripherals has already been discussed in the MCU architecture.There are two drivers

available to continue with the software development process. The IAR tools provide the

facility to simulate the device operation without any hardware. This feature allows the

designer to start developing the software for the application even before any hardware is

built. The second option is to debug the hardware with the emulator, that is, the USB

shaped device.

The emulator is a complete set of developing tools that provide all the hardware and

software to evaluate the MSP430-F2013 microcontroller. This USB stick shaped device

is compatible with the IAR embedded workbench integrated development environment

Page 39: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

(IDE). The IAR tool is used to compile the application software for the prototype board.

The debugging interface contains a USB port and a Spi-By Wire Interface that is

incorporated to download the binary version of the software on the microcontroller. The

primary function of the watchdog timer (WDT+) module is to perform a controlled

system restart after the software problem occurs.

Software Tools

The software for the EZ430, IAR Embedded Workbench, comes free with the purchase

of the tool. Though it is a “kickstart” version, (which meansit has a 4kB limit of code),

the standard microcontroller with which it comes is limited to 2KB of memory. The IAR

carries both a C compiler and an assembler. The code size limitations would be an issue

if the microcontroller class was taught for the development in the C programming

language, but the fact that the EZ430 is used for an introductory course in the assembly

language makes the limitations non-restrictive.

To create a new project, select Project>Create New Project. In the dialog box that

appears, choose "MSP430" in the Tool chain and "Empty project" in the Project

templates. The empty project appears in the Workspace window on the left-hand side.

Before adding any files to the project, the workspace should be saved by

File>SaveWorkspace;provide a valid file name. Choose File>Add Files to open a dialog

box in which the files can be selected; click open to add the files to the project. After the

programming, the application needs to be downloaded. However, you must first choose

Project>Rebuild All to finish the compiling and linking.

2.9 Conclusion

In this chapter, a comprehensive work on the design and development creativity was

conducted. This chapter demonstrated a research through hardware implementation.

GSR, BVP, and Temperature displayed the capable results for use in identifying and

differentiating the physiological arousal. This chapter also discussed the proposed

architecture and the design implementation in detail. This proposed architecture was

Page 40: Chapter 2 EMOTION DETECTION ARCHITECTURE & SYSTEMshodhganga.inflibnet.ac.in/bitstream/10603/45872/7/08_chapter 2.pdf · EMOTION DETECTION ARCHITECTURE & SYSTEM This key chapter reflects

designed for making the system portable, easy to use, and intelligent. This chapter has

provided a detailed explanation of the first two objectives.