chantal vilà calopa contact-freemeasurementofcardiac and

Master’s Thesis

Chantal Vilà CalopaContact-Free Measurement of Cardiacand Respiratory Activities by usingThermal Imaging

PHILIPS CHAIR FOR MEDICAL INFORMATION TECHNOLOGYUniv.-Prof. Dr.-Ing. Dr. med. Steffen LeonhardtSupervising Tutor: Carina Barbosa Pereira, M.Sc.Date: 15th April 2014

Acknowledgement

I would like to express my gratitude to MedIT, its director, Univ.-Prof. Dr.-Ing. Dr. med.Steffen Leonhardt and its academic staff for the opportunity to develop my master’s thesisin this Institute. It has been a pleasure to work here, being able to research using the mostnovel technology and being always surrounded by a friendly and comfortable atmosphere.I would like to deeply acknowledge my supervisor Carina Pereira for the useful commentsand remarks within the development of this thesis. Your guidance helped me enormously toimprove my skills and to grow as an engineer. Thank you for your motivation, enthusiasmand patience.Furthermore I would like to thank my colleagues Veronika, Matthias, Michael and Amine,and the participants of the tests, who have willingly shared their time during the validationphase.I would like to acknowledge Heristal for its affection and constant impulse.Special thanks to Juan Mosquera, for being always my support and my strength.And lovely thanks to my family, for holding me up with a lot of patience, during myengineering’s studies. Thanks for giving me the chance to come to Germany and concludesuccessfully my degree.This thesis is dedicated to you.

Statement of Original Authorship

I certify that the work contained in this thesis is my own work and has not been previouslysubmitted for a degree or diploma in any other higher education institution. To thebest of my knowledge and belief, the thesis contains no material previously published orwritten by another person except where due reference is made.

Place, Date Signature

Abstract

Motivation: Real time measurements of respiratory and heart rates are crucial inneonatal intensive care units. The actual technology is based on contact methods thatrequire close-proximity sensing and subject’s cooperation. To very sensitive skins, likethat of newborns, these sensors are harmful and provoke discomfort. This thesis presentsa non-contact method to monitor neonatal vital signals.

Methods: Two algorithms, which allow detection of respiratory rate and heart rate,have been developed. They are based on medical thermal imaging that is based in theinfrared spectrum. This technique does not use electrodes or any source of illumination.This is only a passive method, which detects the energy emitted by the baby. Algorithmsstudy the temperature change in a concrete region of interest. On one hand, the detectionof the respiratory algorithm is focused on the temperature change in nostrils. On theother hand, heart rate’s detection is focused on the carotid artery, situated in the neck.A tracking algorithm enhances the quality of recordings. The method is based on FFTand includes an Adaptive Estimation Function. In order to validate and optimize thealgorithms, it was tested on different subjects.In the validation of the respiratory rate, three distances between the camera and thesubject were analyzed. At last, a Graphical User Interface was designed to make theinteraction between the user and the software easier.

Results: It has been achieved a mean Complement of the Absolute Normalized Difference(CAND) of 89, 63± 8, 83% for the detection of the respiratory rate and CAND of 86, 29±14, 78% for the detection of the heart rate.Regarding the respiratory rate’s detection, when the distance between the camera andthe subject is 50cm, results obtained present a CAND of 89, 63 ± 8, 83%. Improving thedistance, a CAND of 89, 21± 5, 11% has been obtained in the case of 75cm, and a CANDof 82, 72 ± 12% in the case of 100cm. Taking into account the gender of the subjects,CAND obtained was 85, 33± 10, 79% in males and 93, 93± 1, 6% in females.For the heart rate algorithm, CAND was 77, 05 ± 19, 17% in male and 92, 44 ± 4, 96% infemale.

Conclusions: Both algorithms were successfully developed. A higher level of data pro-cessing was required to extract the heart rate than the one required to extract the respir-atory rate. While the subject’s gender had an important role for the detection of the heartrate, in the detection of the respiratory rate results were quite similar in males and fe-males. Since results obtained by using thermal imaging were satisfactory, further researchis encouraged to develop new applications in neonatology and medicine.

vii

Contents

Acknowledgement iii

Statement of Original Authorship v

Abstract vii

Table of Contents ix

List of Figures xi

List of Tables xv

List of Symbols xvii

1 Introduction 11.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 State of art . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Physiology of the Respiratory and Cardiovascular System 52.1 Respiratory System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.1.1 Mechanics of ventilation . . . . . . . . . . . . . . . . . . . . . . . . 62.1.2 Pulmonary Volumes and Capacity . . . . . . . . . . . . . . . . . . . 92.1.3 Respiratory Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.2 The Cardiovascular System . . . . . . . . . . . . . . . . . . . . . . . . . . 112.2.1 The Heart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.2.2 The Circulatory System . . . . . . . . . . . . . . . . . . . . . . . . 15

3 Infrared Thermography 193.1 Physical fundaments of the IRT . . . . . . . . . . . . . . . . . . . . . . . . 193.2 Heat emitted by the human body . . . . . . . . . . . . . . . . . . . . . . . 223.3 Detection of the emitted signal . . . . . . . . . . . . . . . . . . . . . . . . 233.4 Infrared cameras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.5 Medical applications of IRT . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.5.1 Detection of fever . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283.5.2 Infrared Imaging of the Breast . . . . . . . . . . . . . . . . . . . . . 293.5.3 Biometrics: Face recognition . . . . . . . . . . . . . . . . . . . . . . 29

4 Infrared data processing 314.1 Respiratory rate detection . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

4.1.1 Description of the algorithm step by step . . . . . . . . . . . . . . . 34

ix

Contents

4.2 Heart Rate detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404.2.1 Description of the algorithm step by step . . . . . . . . . . . . . . . 42

4.3 Experimental protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474.3.1 Extraction of the respiratory rate . . . . . . . . . . . . . . . . . . . 484.3.2 Extraction of heart rate . . . . . . . . . . . . . . . . . . . . . . . . 48

4.4 Comparison between IRT’s and GT’s results . . . . . . . . . . . . . . . . . 504.5 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

4.5.1 Infrared Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . 504.5.2 SOMNOlab 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

4.6 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534.6.1 Pre-processing of infrared data: IRBIS 3 Professional . . . . . . . . 534.6.2 Statistical analysis: SPSS . . . . . . . . . . . . . . . . . . . . . . . 54

5 Study of results and discussion 555.1 Graphical user interface (GUI) . . . . . . . . . . . . . . . . . . . . . . . . . 55

5.1.1 GUI - Respiratory rate detection . . . . . . . . . . . . . . . . . . . 565.1.2 GUI - Heart rate detection . . . . . . . . . . . . . . . . . . . . . . . 60

5.2 Respiratory rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 655.2.1 Experimental results . . . . . . . . . . . . . . . . . . . . . . . . . . 655.2.2 Statistics and discussion . . . . . . . . . . . . . . . . . . . . . . . . 67

5.3 Heart rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 715.3.1 Experimental results . . . . . . . . . . . . . . . . . . . . . . . . . . 725.3.2 Statistics and discussion . . . . . . . . . . . . . . . . . . . . . . . . 74

6 Conclusions and future work 796.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 796.2 Possible applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 806.3 Future perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

A Appendix 1 83A.1 Detailed tables of the results obtained in tests . . . . . . . . . . . . . . . . 83

A.1.1 Respiratory rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83A.2 Heart rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

Bibliography 89

x

List of Figures

1.1 Methodology: the three main phases. . . . . . . . . . . . . . . . . . . . . . 3

2.1 Graphical representation of the respiratory system. Adapted from [24]. . . 52.2 Representation of breathing cycle: (a) post-expiratory pause, (b) inspiration

and (c) expiration. Adapted from [24]. . . . . . . . . . . . . . . . . . . . . 72.3 Graphical representation of the varied pressure in the lungs. (a) Alveolar

and pleural pressure. (b) Variation of lung’s pressures and volume duringnormal respiration. Adapted from [15]. . . . . . . . . . . . . . . . . . . . . 8

2.4 Illustration of volume and capacity magnitudes in the lung [15]. . . . . . . 102.5 (a) Anatomy of the heart [15]. (b) The cardiac conduction system.

Adapted from [15]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.6 (a) Representation of the heart’s electrical activity. The color red represents

depolarizing and green represents repolarizing [24]. (b) Schematic repres-entation of an electrocardiogram. Adapted from [24]. . . . . . . . . . . . . 14

2.7 The Circulatory System [31]. . . . . . . . . . . . . . . . . . . . . . . . . . . 152.8 Detail of the common carotid arteries. . . . . . . . . . . . . . . . . . . . . 16

3.1 Emissivity of some metals and other materials depending on the wavelengthof the signal [13]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.2 Planck’s law for a black body [13]. . . . . . . . . . . . . . . . . . . . . . . . 213.3 Stefan Boltzmann’s law graphic [13]. . . . . . . . . . . . . . . . . . . . . . 223.4 Electromagnetic spectrum and a detailed diagram of the infrared spectrum. 223.5 Spectral emittance at temperatures similar to the temperature of the human

body [5]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233.6 Heat exchange processes of the body, which affect the skin temperature. . . 243.7 Components of the radiation received by the camera. . . . . . . . . . . . . 253.8 Spectral transmittance in the infrared spectrum [13]. . . . . . . . . . . . . 253.9 Representation of the photoelectric effect. . . . . . . . . . . . . . . . . . . 263.10 General classification of infrared cameras [13]. . . . . . . . . . . . . . . . . 273.11 (a) Beam path of a scanning camera (1-Detector, 2-Lens, 3-Horizontal de-

flector, 4-Vertical deflector, 5-Lens, 6-Object, 7- Measured spot). (b) Beampath of a FPA camera (1-Object, 2-Lens, 3-Detector) [13]. . . . . . . . . . 28

3.12 (a) Original Image. (b) Image segmented using Bayesian approach [8]. . . . 29

4.1 Typical temperature distributions for the three breathing phases: expiration(red), post-expiratory pause (green) and inspiration (blue) [21]. . . . . . . 32

4.2 (a) Infrared image recorded during inspiration phase. (b) Infrared imagesrecorded during expiration phase. . . . . . . . . . . . . . . . . . . . . . . . 32

4.3 Flowchart of the detection algorithm from the respiratory rate. . . . . . . . 334.4 Infrared image format. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

xi

List of Figures

4.5 Representation of the three ROIs: normal ROI (green), big ROI (blue), onlyone nostril (yellow). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

4.6 Basic set of 2-D Transformation [28]. . . . . . . . . . . . . . . . . . . . . . 354.7 Block diagram of the KLT tracker [2]. . . . . . . . . . . . . . . . . . . . . . 364.8 Illustration of the ROI’s selection (steps 1 and 2), its transformation to grey

(step 3a) and the computation of the mean (step 3b). . . . . . . . . . . . . 374.9 Illustration of step 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384.10 Magnitude and Phase Responses of (a) high-pass filter and (b) low-pass filter. 394.11 Frames recorded with the infrared camera taken from the left side of three

patient’s neck. The detected blood vessel is indicated by a red rectangle. . 404.12 Flowchart of the detection algorithm from the heart rate. . . . . . . . . . . 414.13 Steps 1, 2 and 3 of the detection algorithm. . . . . . . . . . . . . . . . . . 424.14 Normal parameters for a straight line [9]. . . . . . . . . . . . . . . . . . . . 434.15 (a) Sinusoidal curves resulting from the Hough transform applied in the

example image. (b) and (c) correspond to output images when computingedge() in MATLAB. The color axis scale in image (b) include a biggerrange of temperatures than in image (c). For this reason, in (b) is possibleto observe the warm mark of the carotid artery. . . . . . . . . . . . . . . . 43

4.16 Magnitude and Phase Responses of (a) high-pass filter and (b) low-pass filter. 454.17 Illustration of step 7. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454.18 Operating method of the adaptive estimation function. . . . . . . . . . . . 464.19 Equipment used during tests: (a) Infrared Camera. (b) Ground truth device

[SOMNOlab 2 (Weinmann Geraete fuer Medizin GmbH + Co, Hamburg,Germany)]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

4.20 (a) Experimental setup of infrared thermography respiration monitoring.The variable “d” defines the distance between the camera and subject’snose and θ the angle of the camera. (b) Real scenario. . . . . . . . . . . . . 48

4.21 Experimental setup of infrared thermography heart monitoring. Variable“d” defines the distance between the camera and subject’s neck and θ theangle of the camera. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

4.22 Different views of the infrared camera VarioCAM hr head. . . . . . . . . . 514.23 Setup of SOMNOlab 2 device to measure abdomen and thorax efforts, as

well as photoplethysmography. (a) represents the thorax belt and (b) theabdomen belt. They allow measuring the efforts of the thorax and abdo-men, respectively, during respiration. (c) represents the PPG sensor, whichmeasures the cardiac cycle by illuminating finger’s skin. Adapted from [30]. 52

4.24 (a) Pulse and (b) PPG signals recorded by SOMNOlab 2. . . . . . . . . . . 524.25 Main screen of IRBIS 3 Professional. . . . . . . . . . . . . . . . . . . . . . 534.26 (a) Description of the main parts of a boxplot. (b) Example of a boxplot. . 54

5.1 Screenshot of the graphical user interface (main screen). . . . . . . . . . . 555.2 (a) GUI screen. (b) Selected Region of Interest (ROI). . . . . . . . . . . . 565.3 Screenshot of the graphical user interface (screen for the selection of the

ROI in the respiratory rate algorithm). . . . . . . . . . . . . . . . . . . . . 57

xii

List of Figures

5.4 (a) Signal s(t), mean value of the pixels inside the ROI. (b) Signal u(t), afterapplying normalization to signal s(t). (c) Filtered signal p(t). (d) Selectednumber of frames to compute the FFT. . . . . . . . . . . . . . . . . . . . . 58

5.5 (a) Fast Fourier Transform of the signal x(t). (b) Temporal respiratory rateextracted every 10 seconds by using Thermal Imaging. . . . . . . . . . . . 58

5.6 GUI screen that visualizes all results, the ones obtained by infrared ther-mography (on the left) and the ones obtained as ground truth (on the right). 59

5.7 Screenshot of the graphical user interface (screen for the comparison betweenresults obtained by IRT and GT). . . . . . . . . . . . . . . . . . . . . . . . 60

5.8 Screenshot of the graphical user interface (screen for the computation of theheart rate algorithm). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

5.9 (a) Selection of the blood flow direction using Hough transformation. (b)Rotated image showing the ROI. . . . . . . . . . . . . . . . . . . . . . . . 61

5.10 (a) Temperature profile in the ROI for each y-position. (b) Temperatureprofile for a selected y-position. (c) Normalized selected signal. (d) Filteredselected signal. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

5.11 (a) Temperature profile for the selected NHR frames (NHR = 512). (b)Periodic extension of the temperature profile shown in (a). . . . . . . . . . 63

5.12 Mean power spectrum of the 512 last frames. . . . . . . . . . . . . . . . . . 635.13 (a) Heart rate extracted by using Thermal Imaging. (b) Heart rate extracted

by SOMNOlab 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 645.14 Results of the comparison. It presents (1) heart rate recorded by the IRT,

(2) heart rate recorded by GT and (3) CAND. . . . . . . . . . . . . . . . . 645.15 Infrared thermal signal of the nostrils of subject 09. . . . . . . . . . . . . . 675.16 Boxplot of CAND obtained according to the dimension of data (Gr, R, B,

G). (a) Distance of 50 cm. (b) Distance of 75 cm. (c) Distance of 100 cm. . 685.17 Boxplot of CAND obtained according to the distance between the subject

and the camera (50cm, 75cm, 100cm). (a) Normal region. (b) Big region.(c) One nostril. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

5.18 Boxplot of CAND obtained according to the region of interest (normal re-gion, big region, one nostril). (a) Distance of 50 cm. (b) Distance of 75 cm.(c) Distance of 100 cm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

5.19 Boxplot of CAND obtained according to the gender of the subjects. (a)Male. (b) Female. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

5.20 Boxplot of CAND obtained according to different values ofM and NHR. (1)M = 50, NHR = 256. (2) M = 50, NHR = 512. (3) M = 100, NHR = 256.(4) M = 100, NHR = 512. . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

5.21 Boxplot of CAND obtained according to the region of interest. . . . . . . . 765.22 Boxplot of CAND obtained according to the gender of the subject. (a) Male.

(b) Female. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

xiii

List of Tables

2.1 Approximate normal values of lung volumes and capacities [15], [18] . . . . 102.2 Normal values of the respiratory rate [18] . . . . . . . . . . . . . . . . . . . 112.3 Variation of the heart rate according to age . . . . . . . . . . . . . . . . . 14

4.1 Properties of the 2-D transformation types [28] . . . . . . . . . . . . . . . . 364.2 Main properties of both filters, elliptic filter and Butterworth filter . . . . . 394.3 Main properties of both filters, elliptic low-pass and high-pass filters . . . . 454.4 Angles in which the camera was placed during recordings . . . . . . . . . . 49

5.1 Main information about the subjects . . . . . . . . . . . . . . . . . . . . . 655.2 Summary of the results. The respiratory rate (RR) is measured in breaths

per minute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 665.3 Main information about the subjects . . . . . . . . . . . . . . . . . . . . . 715.4 Summary of the results. The respiratory rate (RR) is measured in breaths

per minute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 725.5 Summary of the results. The respiratory rate (RR) is measured in breaths

per minute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 735.6 Properties of boxplots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 745.7 Properties of boxplots of each region of interest . . . . . . . . . . . . . . . 76

A.1 Results respiratory rate with a distance of 50 cm between the subject andthe camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83



A.4 Results heart rate choosing M as 50 . . . . . . . . . . . . . . . . . . . . . . 86A.5 Results heart rate choosing M as 100 . . . . . . . . . . . . . . . . . . . . . 87

xv

List of Symbols

Acronyms

AV AtrioventricularAVI Audio Video InterleaveB Big Region Of InterestBP Blood Pressurebpm Beats per MinuteCAND Complement of the Absolute Normalized DifferenceCO Cardiac OutputCVPR Computer Vision and Pattern RecognitionDFT Discrete Fourier TransformECG ElectrocardiographyERV Expiratory Reserve VolumeFFT Fast Fourier TransformFPA Focal Plane ArrayFRC Functional Residual CapacityGFSK Gaussian Frequency-Shift KeyingGT Ground TruthGUI Graphical User InterfaceHIA Helmholtz Institute AachenHR Heart RateIC Inspiratory CapacityIEEE Institute of Electrical and Electronics EngineersIR InfraredIRT Infrared ThermographyIRV Inspiratory Reserve VolumeKLT Kanade-Lucas-TomasiLWIR Long Wave InfraredMedIT Medical Information TechnologyMWIR Middle Wave InfraredN Normal Region Of InterestNICU Neonatal Intensive Care UnitNIR Near InfraredNIR Neonatal Infrared ThermographyO One nostril (Region Of Interest)PPG Photoplethysmography

xvii

List of Symbols

RDS Respiratory Distress SyndromeRGB Red, Green, BlueROI Region Of InterestRR Respiratory RateRV Residual VolumeRVSM Radar Vital Signs MonitorRWTH Rheinisch-Westfälische Technische Hochschule AachenSA SinoatrialSARS Severe Acute Respiratory SyndromeSVR Systemic Vascular ResistanceSWIR Short Wave InfraredTLC Total Lung CapacityVC Vital CapacityVLWIR Very Long Wave InfraredWHO World Health Organization

Physical Quantities

α Absorption coefficient -ε Emission coefficient -Φ Total radiation power Wλ Wavelength mρ Density of energy W/m3

θ Angle degreeA Area m2

d Distance mFs Sampling Frequency HzI Image pixelM Frames selected to compute the adapt-

ive estimation functionframes

NHR Frames selected for the HR detection al-gorithm

frames

NRR Frames selected for the RR detection al-gorithm

frames

P Pressure mmHgp Transformation parameters -PA Alveolar pressure mmHgPL Transpulmonary pressure mmHgPpl Pleural pressure mmHg

xviii

List of Symbols

Rx Width of the image pixelRy Length of the image pixelT Temperature KTC Surrounding Temperature Kt time sV Volume lv Frequency HzVT Tidal volume lW Energy W/m2

WE Emitted Energy W/m2

WL Conductive Band W/m2

Wph Photon’s Energy W/m2

WR Received Energy W/m2

Wr Reflected Energy W/m2

WT Transmitted Energy W/m2

Wv Valence Band W/m2

Constants

h Planck’s constant 6.63 · 10−34[J · s]K Boltzmann’s constant 1.34 · 10−23[J/K]c Speed of light 3 · 108[m/s]σ Stefan-Boltzmann constant 5.669 · 10−12[W/(cm2K4)]π Pi 3.141592653589

xix

1 Introduction

The adaptation process of premature and newborn infants to the extra-uterine environmentincludes tasks that are beyond doctors and nurses’ capabilities. Infant’s body temperaturefalls immediately after delivery and can easily reach too low levels if heat is not conserved.Therefore, it is necessary to use medical devices to (a) control their vital signs, (b) assuretheir correct development and (c) promptly detect signs of illness.Premature babies, defined as infants of less than 36-week-gestational age, and some fullterm infants are not capable of maintaining their own body temperature and protect them-selves against infections and viruses. Thus, the more energy is needed for reaching optimalbody temperature, the less energy is available for other processes such as growth, braindevelopment, or lung maturation. For this reason, these babies are placed during sometime in incubators, which provide a perfect environment.During this period, one of the most important tasks for nurses is the control of babies’ vi-tal signs; hence, real time measurements of heart and breathing rates are obligatory.Nevertheless, this is sometimes harmful for the baby. The majority of the developed meas-urement devices use contact methods that require close-proximity sensing and subject’scooperation. E.g., central and peripheral temperatures are measured by using cable-boundsensors, which are placed into the rectum or attached to the skin of the infant. Further-more, adhesive connections cause mechanical stress to the very sensitive skin of the infantand lying on cables may ultimately cause ulcers.Thus, would it be possible to obtain measures of vital signs without opening the incubator?How could we avoid taking the baby out of his/her perfect environment? Could we detectheart and respiratory rates (HR and RR, respectively) without touching the baby? Canwe protect the infant’s skin from sensor’s contact?

1.1 Motivation

Statistics published by “The World Bank” show a downward trend of the neonatal mortal-ity in the last twenty-five years. Countries like Hungary, Turkey, Serbia or Saudi Arabiareduced it more than 70%, principally due to the development of new technologies, whichprovide an accurate observation of babies and facilitate their treatment.This demonstrates an important progress but is not enough. Almost three million babiesdied worldwide in their first month of life in 2012. According to the World Health Organ-ization (WHO), “the majority of all neonatal deaths (75%) occur during the first week oflife, and between 25% to 45% occur within the first 24 hours. Up to two thirds of newborndeaths could be prevented if skilled health workers perform effective health measures at birthand during the first week of life” .This thesis is aimed at the search of a technological development, which reduces these

1

1 Introduction

statistics; a small step to achieve better technology to take care of these new lives.To assess the most basic body functions, measure of vital signs is crucial. Heart rate andrespiratory rate are two important examples, since heart and breathing failures are pre-valent in the first week of life. As an illustration, monitoring of breathing function can beused to predict sudden infant death syndrome [21].Medical techniques such as electrocardiography (ECG) and photoplethysmography (PPG)allow measuring these vital signs. However, as previously mentioned, they require thecontact with the baby. Thereby, one of the aims of this thesis is to develop a contactlesstechnique for vital signs detection, in order to avoid the contact of harmful sensors. Thisreduces wounds in sensitive skins and discards any possible discomfort from the newborn.A new technology based in the infrared spectrum has been developed in the last years.Infrared thermography, also named as thermal imaging, is one of the most appropriatemethodologies to detect HR and RR.To sum up, the goal of this work is to implement and validate algorithms which permit toaccurately measure HR and RR in infrared imagery.

1.2 State of art

The first documented application of infrared imaging in medicine was in 1956 when femalebreast cancer patients were examined. Their thermograms demonstrated asymmetric hotspots and vascularity.Two decades later, in 1980 Clark and Stothers [7], who performed the first measurementsin a neonatal intensive care unit (NICU), introduced the “Neonatal Infrared Thermo-graphy” (NIRT) imaging.In 1997, Greneker introduced the first contact-free vital sign measurement method basedon active sensing [14]. The Radar Vital Signs Monitor (RVSM), as it was called, wasdeveloped to monitor the performance of Olympic athletes. Based on the Doppler Effect,it was able to measure the subject’s heartbeat and respiration rate at a distance up to30 feet without the requirement of a physical connection to the subject. Its weakness,nevertheless, was the fact that motion artifacts corrupt breath signals.In 2000, Pavlidis et al. used infrared imaging to detect facial patterns of stress fromdistance [22], and later, the same method was used to compute periorbital perfusion.With these two methods the corresponding polygraph channel (by finger contact sensingor abdominal transducing) was replaced and subjects felt as comfortable as possibleduring examination.Four years later, the same author and his co-workers developed a second-order statisticalmethod to estimate the breathing rate using thermal video sequences [21]. Taking acertain Region of Interest (ROI) around the nose, they labelled each one of the pixels inthe ROI as an inspiration or expiration pixel, depending on its statistical distribution.If the number of inspiration pixels was higher than the number of expiration pixels, theframe was labelled as inspiration. Otherwise, the frame was labelled as expiration. Oncethe full breathing cycle was detected, the breathing rate was computed and continuously

2

1.3 Structure

updated. In 2010 the present method was improved by adding an automatic tracking ofthe nasal region as well as wavelet analysis [10].In the 2005 Conference on Computer Vision and Pattern Recognition (CVPR 2005) wasproposed for the first time an FFT-based computational method to measure cardiacpulse of a major superficial vessel via Infrared Thermography [27]. It was developedduring next years with new experiments and analysis [12]. This method was based on theinformation contained in the thermal signal, emitted from a major superficial vessel andacquired through a highly sensitive thermal imaging system. To compute the frequency ofmodulation (pulse), a line-based region along the vessel was extracted, where temperatureis modulated by pulsatile blood flow. Then, applying Fast Fourier Transform (FFT)to individual points (pixels) along the line of interest the pulse propagation effect wasdeleted and the heart rate extracted [27]. This method is the one developed in this thesis.

1.3 Structure

The current master thesis is divided in six main chapters. It starts with a detailed descrip-tion of the human physiology (chapter 2), specifically from the cardiac and respiratorysystems standpoint. Forthwith (in chapter 3), infrared thermography is introduced andwith it the most important physical basics, a brief description about infrared cameras andsome actual applications. In the fourth chapter, algorithms used to obtain the two vitalsigns, heart rate and breathing frequency, are described. The methodology consists ofthree phases: (1) recording the subject with an infrared camera, (2) processing data withMATLAB and (3) extracting vital signs, as illustrated in Figure 1.1.

èd

Infrared CameraSubject under test MATLAB

Figure 1.1: Methodology: the three main phases.

Then, in chapter 5, the experimental scenario of two tests [extraction of (a) respiratory rateand (b) heart rate] is presented. Furthermore, it includes a description about the equipmentrequired during tests, as well as two examples of the application of both algorithms. Finally,chapter 6 presents and discusses the results obtained. At the end, chapter 7 formulatesthe most important conclusions and includes some ideas for future works.

3

2 Physiology of the Respiratory andCardiovascular System

Vital signs, such as heart rate, breathing rate, body temperature and blood pressure, areused to determine the physiological functioning of the human body. These physiologicalparameters vary with sex, age, weight, exercise tolerance and body conditions [1]. Thepresent thesis focuses on the detection of two of them: heart rate and breathing rate.Therefore, the next sections describe some physiological aspects regarding respiratory sys-tem and cardiovascular system.

2.1 Respiratory System

The respiratory system plays a major role in the transport of oxygen to the tissues aswell as in removing carbon dioxide from them. According to Guyton, respiration canbe branched into four main functions. The first function corresponds to pulmonaryventilation, which consists to the air exchange (inflow/inspiration and outflow/expira-tion) between the atmosphere and the lung alveoli. The second and third functions arerelated to diffusion of gases (oxygen and carbon dioxide) between alveoli and blood and,their transport in the blood to and from the body tissues, respectively. The fourth andlast function of this complex system is regulation of ventilation [15]. Therefore, structuressuch nose, pharynx, larynx, trachea, bronchi, and lungs, represented in Figure 2.1, are themain constituents the respiratory system [24].

Nasalcavity

Nostril

Trachea

Right lungLeft lung

Pharynx

Larynx

Bronchus

Figure 2.1: Graphical representation of the respiratory system. Adapted from [24].

5

2 Physiology of the Respiratory and Cardiovascular System

The development of this physiological system is characterized to be slow. During the firstsixteen weeks of gestation takes place the growth of fetal lungs. The alveoli, on the otherhand, are not completely developed at the moment of birth but at the age of 5-6 years [11].According to Polin et al., there is an average of 50 million (ranging from 20 to 70 million)alveoli in a term lung and an average of 300 million (ranging form 212 to 605 million)in an adult lung [23]. Furthermore, the alveoli radius in adults is fourfold higher than inpremature babies. At last, the chest wall finishes its development and, consecutively, thedevelopment of the present physiological system, at the age of 8 years.

2.1.1 Mechanics of ventilation

According to Silverthorn [26], the breathing cycle can be defined in three different phases:(1) inspiration, (2) expiration, and (3) post-expiratory pause. This cycle is achieved bychanging rhythmically the pressure in the thoracic cavity. The difference between theintrapulmonary pressure (pressure inside the body) and the atmospheric pressure (pressureoutside the body) produces the airflow1.Assuming a constant temperature, Boyle’s law states that the volume of the containerfrom a given quantity of gas is inversely proportional to its pressure as given by

P1 · V1 = P2 · V2, (2.1)

where P1 and V1 are pressure and volume of the gas at a certain instant of time (t1) andP2 and V2 represent pressure and volume of the same gas at an instant t2. Based on thislaw, the intrapulmonary pressure falls when the volume of the lungs increases and it riseswhen the volume of the lungs decreases.The necessary movements of expansion and contraction of lungs are produced by differentmechanisms. However, the diaphragm, an internal skeletal muscle, plays the major role.It contracts and relaxes pulling the lower surfaces of the lungs downwards (inspiration)or upwards (expiration). Therefore, when the diaphragm contracts the volume insidethe lungs increases and the intrapulmonary pressure falls, so that the pressure insidethe lungs is lower than the one outside the body - inspiration phase. On the contrary,when the diaphragm relaxes (taking a dome shape), the intrapulmonary pressure increasesand produces the expiration phase. The post-expiratory pause occurs, in turn, when thepressure inside the lungs equalizes the atmospheric pressure [26]. Figure 2.2 depicts thethree breathing phases (inspiration, expiration and post-expiratory pause) as well as themovement of the diaphragm.Unlike the contraction of the diaphragm, which requires muscular effort, its relaxation

is an energy-saving passive process. The airflow is thrown out of the lungs without usingenergy. This is due to the thin film of water inside the lung alveoli. The surface of thissmall amount of water is in contact with air, thus the water molecules on the surfaceattempt always to contract (Principle of Surface Tension). This results in a pressure thatforces the air out of the alveoli [15]. It is calculated using Laplace’s relationship:

1The atmospheric pressure (pressure outside the body) is 760 mmHg at sea level.

6


(b) Inspiration(c) Expiration

(a) At rest

760 mmHg intrapulmonary pressure

757 mmHg763 mmHg

Diaphragm

Figure 2.2: Representation of breathing cycle: (a) post-expiratory pause, (b) inspiration and(c) expiration. Adapted from [24].

Intrapulmonary pressure = 2 · Surface Tension Elastic ForceRadius of an alveolus [mm · Hg]. (2.2)

As demonstrated in Equation 2.2, the pressure generated in the alveoli is inversely relatedto the radius of the alveolus. Since the alveoli of newborn infants present small radii, highpressures are produced, increasing the tendency to lung collapse. Premature babies oftensuffer from Respiratory Distress Syndrome (RDS), an insufficiency of surfactant produc-tion that leads to lung collapse. It can be six- to eightfold greater than in an adult. Mostimportantly, if not treated on time it is a fatal disease. Many infants die of suffocationsince large portions of the lung become atelectatic. In addition, they spend a huge amountof energy in the process of breathing, reducing the energy for the development of the otherbody parts (e.g. the brain) [15].Transpulmonary pressure, PL, (depicted in Figure 2.3) plays an important role in respira-tion, since it prevents the lung from collapsing by keeping the alveoli open, i.e. the lunginflated. It corresponds to the pressure difference across the lung wall, in other words,to the difference between alveolar pressure PA (also termed intrapulmonary pressure) andpleural pressure Ppl in the lungs (PL = PA−Ppl). Therefore, e.g. at rest, alveolar pressureand pleural pressure are 0 mmHg and −5 mmHg, respectively, leading to a transpulmon-ary pressure of 5 mmHg. The lower section of Figure 2.3(b) represents graphically thevariation of all mentioned pressures.

7


A healthy lung has always a positive transpulmonary pressure. A transpulmonary pressureequal to zero indicates, on the other hand, a pneumothorax. This condition is characterizedby air or gas accumulation in the pleural space, inducing lung collapse.

Alveolarpressure

Pleuralpressure

(a)

0

+1.5

0

– 1.5

– 3

– 4.5

– 6

0.50

ExpirationInspiration

Transpulmonary pressure

Lung volume

Alveolar pressure

Pleural pressure

Pre

ssure

(

mm

Hg )

Volu

me c

hange (

lite

rs )

0.25

(b)

Figure 2.3: Graphical representation of the varied pressure in the lungs. (a) Alveolar and pleuralpressure. (b) Variation of lung’s pressures and volume during normal respiration.Adapted from [15].

Rib cage is another anatomical structure, whose movements (up and down) enhance thecycle of respiration. When it rises, the ribs, which in rest are slant downward, are projectedforward producing an increase of the diameter of the chest cavity. Indeed, the chest volumeduring inspiration is approximately 20 per cent higher than during expiration [15].Lastly, the warming of the inhaled air also produces a lung expansion. According toCharles’ law “by assuming a constant pressure, the volume of a given quantity of gas isdirectly proportional to its absolute temperature”. This is,

V1 · T1 = V2 · T2, (2.3)

where V and T stand for volume and temperature of a certain amount of gas.As shown in Figure 2.1, the air enters the nose through the nostrils and travels through twocanals called nasal cavities. Because of the narrowness of the passage, most air contacts themucous membranes on its way, fact that enables the nose to cleanse, warm and humidify it.The air warming occurs due to the large blood vessels contained in the mucosa. Normallythe temperature rises to within 1◦F of body temperature. The inhaled air is warmed beforereaching the alveoli, it expands and provokes the lungs to inflate [24].Hence, the combination of all mechanisms above introduced, induce the lung volume toincrease and decrease. In the upper section of Figure 2.3(b) the changing of this volumeduring a breathing cycle is represented.

8


2.1.2 Pulmonary Volumes and Capacity

According to Guyton [15], the study of pulmonary ventilation has developed four standardvolumes and four standard capacities to describe and compute breathing ventilation.The air volume in the lung can be divided in:

1. tidal volume (VT ) — difference of volume between inspiration and expiration atrest;

2. inspiratory reserve volume (IRV ) — possible amount of air inhaled after anormal inspiration;

3. expiratory reserve volume (ERV ) — possible amount of air exhaled after anormal expiration;

4. residual volume (RV ) — amount of air resting in the lungs after a powerful ex-piration phase.

By adding them together, the maximum expansion of the lung can be calculated. Con-sidering two or more of these volumes together, it is possible to describe the followingcapacities:

1. inspiratory capacity (IC) — total amount of air that can be taken in the deepestinspiration. It is the sum of inspiratory reserve volume and tidal volume;

2. functional residual capacity (FRC) — air in the lung after expiration. It iscomputed as the sum of expiratory reserve volume and residual volume;

3. vital capacity (V C) — after taking a deep breath and filling the lungs with themaximum amount of air, the vital capacity is the total air that can be expired. Itis computed as the sum of inspiratory reserve volume, tidal volume and expiratoryreserve volume;

4. total lung capacity (TLC) — amount of air needed to fulfill the entire lungs. Itconsists of the sum of all volumes previously introduced: TLC = VT +IRV +ERV +RV .

Figure 2.4 illustrates both pulmonary volumes and capacities and Table 2.1 highlightstheir normal values for both newborns and adults.

9


6000

1000

2000

3000

4000

Time

Inspiration

Inspiratorycapacity

Inspiratoryreservevolume

Expiratoryreserve volume

Vitalcapacity

Expiration

Total lungcapacity

Tidalvolume

Functionalresidualcapacity

Residualvolume

Lu

ng

vo

lum

e (

ml)

5000

Figure 2.4: Illustration of volume and capacity magnitudes in the lung [15].

Table 2.1: Approximate normal values of lung volumes and capacities [15], [18]Lung Volumes and Capacities Healthy Newborn Healthy AdultInspiratory reserve volume (IRV ) 60 ml 3000 mlTidal volume (VT ) 15 ml 500 mlExpiratory reserve volume (ERV ) 40 ml 1100 mlResidual volume (RV ) 40 ml 1200 mlInspiratory capacity (IC) 75 ml 3500 mlFunctional residual capacity (FRC) 80 ml 2300 mlVital capacity (V C) 115 ml 4600 mlTotal lung capacity (TLC) 155 ml 5800 ml

10

2.2 The Cardiovascular System

2.1.3 Respiratory Rate

The respiratory rate (RR) is the number of breaths taken within a specified time (normally60 seconds). The normal respiratory rates in humans, which are dependent on age, aredescribed in Table 2.2.

Table 2.2: Normal values of the respiratory rate [18]Age Group Respiratory RateNew-born 35-50 breaths per minute6 months 25-40 breaths per minute3 years 20-30 breaths per minute6 years 18-25 breaths per minute10 years 15-20 breaths per minuteAdults (>13 years) 12-20 breaths per minute

Respiratory rate can be classified according its frequency. Whereas an abnormally fast RRdefines a tachypnea, an abnormally slow RR characterizes a bradypnoea. The term apnoeais commonly applied to patient that breath spontaneously [24].Contrarily to adults, the functional residual capacity (FRC) in neonates is increased, dueto the rigidity of the chest wall and the immature elastic fibres of the premature lungs. Inturn, a flatter diaphragm provokes a limited inspiratory reserve volume (IRV ). Therefore,to maintain a necessary input and output of gases, RR is increased, leading to respiratoryfatigue. In addition, neonates suffer frequently from apnoea (less than 5 seconds) followedby tachypnoea [11].


The cardiovascular system can be divided in two main parts: the heart and thecirculatory system. The former pumps blood and the latter can be described as a networkof distributing and collecting conduits (arteries, capillaries and veins) that (1) transportoxygen, carbon dioxide, nutrients and hormones to nourish all body cells and (2) allowstabilization of the body temperature and pH.

2.2.1 The Heart

The heart is a muscular organ, located in the thoracic cavity, which pumps the bloodthroughout the blood vessels. It can be divided in two: the right heart, which distributes

11


blood to the lungs, and the left heart, which distributes blood to the rest of the body. Therhythmical contraction of the heart transmits electrical signals through the muscle andproduces the well-known heart rate [15].

Anatomy of the Heart

As previously referred, the heart can be described as two separate pumps. Both aredivided in two chambers, the atrium and the ventricle. The atrium works as a collectingchamber and as a weak pump that helps to eject blood into the ventricle. Ventricles are incharged of pumping the blood. These are characterized to be stronger than atria. Thus, asdetailed in Figure 2.5(a), the heart is composed of four chambers made of cardiac muscleor myocardium: the right atrium, the right ventricle, the left atrium and the left ventricle.A membranous septum separates the atria and a muscular septum separates the ventricles.Additionally, four valves (tricuspid, mitral, pulmonary and aortic) allow or not the bloodto flow into the different chambers.The blood flows through the heart in the following one-way circuit: Blood returning fromthe body is injected in the right atrium, which sends the blood into the ventricle throughthe tricuspid valve. When the pulmonary valve opens, the blood is pumped to the lungsthrough the pulmonary artery. In the lungs, carbon dioxide is unloaded (to be exhaled)and oxygen is picked up. The cleaned blood returns to the left side of the heart, first tothe left atrium and later to the left ventricle. The valve situated between this atrium andventricle is called mitral valve, and the valve that allows the blood to be pumped withstrong pressure to the peripheral organs, is the aortic valve [24], [18].

Trunk and lower extremity

Head and upper extremity

Aorta

Pulmonary artery

Lungs

Pulmonary vein

Left atrium

Mitral valve

Aortic valve

Left ventricle

Right ventricle

Inferior vena cava

Tricuspid valve

Pulmonary valve

Right atrium

Superior vena cava

(a)

Sinoatrialnode (pacemaker)

Internodalpathways

Purkinje fibers

Atrioventricular node

Atrioventricular bundle

Left bundle branch

Right bundle branch

(b)

Figure 2.5: (a) Anatomy of the heart [15]. (b) The cardiac conduction system.Adapted from [15].

12


Cardiac Cycle

The contraction and relaxation of the heart muscle is controlled and timed by the cardiacconduction system, which enables the synchronization between the four chambers. Itconsists of a special group of cells which either generate electrical events or conduct theseelectrical signals through a complex network [18]:

1. Sinoatrial (SA) node — situated in the right atrium. It is the pacemaker thatstarts each heartbeat.

2. Atrioventricular (AV) node — transports de SA electrical excitation to the vent-ricles.

3. Atrioventricular (AV) bundle or “Bundle of His” — exit path through whichsignals leave the AV node.

4. Right and left bundle branches — the two divisions of the AV bundle.

5. Purkinje fibers — complex network in both ventricles, which distribute the elec-trical signal to them.

A cardiac cycle consists of one complete contraction (systole) and relaxation (diastole) of allfour chambers. It starts with the excitation of the SA node. This produces the stimulationof both atria, which contract almost at the same time, and reaches the AV node. Thesignal travels through the AV bundle and Purkinje fibers, causing the contraction of thetwo ventricles. The arrows in Figure 2.5(b) indicate the path of these electrical signals [24].These electrical events can be detected by electrodes applied on the skin and can be plottedin an electrocardiogram (ECG). In the following Figure (Figure 2.6(a)) the relationshipbetween the electrical activity of the heart and the electrocardiogram is represented.As shown in Figure 2.6(b), the normal ECG is compounded of three principal structures:P wave, QRS complex and T wave. When the electric signal produced by the SA nodedepolarizes the atria, the electrodes detect the P wave. The P-Q segment occurs duringatrial contraction or atrial systole lasting approximately 160 msec. Continuously startsthe so-called QRS complex, which consists on a small downward deflection (Q), a tall peak(R) and a second downward deflection (S). It represents the excitation of the AV nodeand the depolarization of both ventricles, which are produced in different times becauseof their different sizes. Shortly after this complex and during the S-T segment starts theventricular contraction, or ventricular systole. At last, the occurrence of T wave is due torepolarization of both ventricles.

13


T

1

T

4

5

6

2

3

(a)

Atriacontract

Ventriclescontract

+1

0

–1

Millivo

lts

P wave T wave

QRS interval Q–T

R R

QS

(b)

Figure 2.6: (a) Representation of the heart’s electrical activity. The color red representsdepolarizing and green represents repolarizing [24]. (b) Schematic representation ofan electrocardiogram. Adapted from [24].

Heart Rate

As previously explained, the transmission of the action potentials throughout the heartmuscle cause a rhythmical beat [15]. Therefore, heart rate is defined as the frequency inwhich the heart pumps blood to the body. It is not a stable value, since it depends on thebody needs for oxygen and nutrients. At rest, the adult heart rate is usually around 70 to80 beats per minute (bpm). Nevertheless, depending on the age, the heart rate can vary.The common values are described in Table 2.3.

Table 2.3: Variation of the heart rate according to ageAge Group Heart RateNew-born 120 -160 beats per minuteInfant(1-12 months) 80 -140 beats per minuteToddler (1-3 years) 80 -130 beats per minutePreschooler (3-5 years) 80 -120 beats per minuteSchool Age (6-12 years) 70 -110 beats per minuteAdults (>13 years) 55 -105 beats per minute

Arrhythmia is defined as any abnormal rhythm of the heart beat [24]. The most importanttypes are: tachycardia or fast heart rate, that consist on a continuous resting heart rateof 100 bpm or higher; bradycardia or slow heart rate, that defines a continuous restingheart rate of 60 bpm or lower; and sinus arrhythmia, i.e. a high variation of the interval’s

14


duration between successive beats.As described in Table 2.3, the heart of newborn infants beats at 120 times a minute, as fastas the heart of an adult doing exercise. This is caused by the high demand of oxygen-richblood from the growing tissues. However, the HR reduces gradually in the following years,and by late teens it is around 70 beats per minute.

2.2.2 The Circulatory System

The circulatory system can be divided in two main circuits: the pulmonary circulation andthe systemic circulation, as shown in Figure 2.7. The pulmonary circulation is in chargeof carrying the blood to the lungs vessels, where the blood is cleaned while exchangingcarbon dioxide for oxygen. The systemic circulation, in turn, supplies blood rich in oxygenand nutrients to every cell of the body [24].

Figure 2.7: The Circulatory System [31].

In both circuits, the blood pumped from the heart flows through vessels with large diametercalled arteries. These are responsible to distribute the blood to smaller tubes: arteriolesand capillaries. The latters have diameters in the order of microns and are estimated tobe a billion in an adult. After exchange of nutrients and gases in the capillaries, the bloodflows to venules and veins until it reaches again the heart.As well-known, arteries and veins play different roles in the circulatory system. While theformers transport blood under high pressure and high velocity, the latters contain blood

15


under very low pressure and velocity. These different functions have an impact on theiranatomical structure. Hence, arteries have thicker walls than veins, with more elastic andmuscular tissue [24].The common carotid arteries are major blood vessels that transport the blood to the neck,face and brain. They are a pair of arteries situated in the neck: one on the left side, leftcommon carotid artery, and one on the right side, right common carotid artery. These,in turn, branch into two: the external and the internal carotid artery, as it is possible toobserve in Figure 2.8. The former, supplies blood to the neck, scalp and face; whereas thelatter, supplies blood to the brain [24]. The diameter of common carotid arteries averages6.5 mm in women and 6.1 mm in men [17]. The blood flow, on the other hand, ranges427± 106 ml/min (21− 50 years) [32].

Figure 2.8: Detail of the common carotid arteries.

The regulation of blood pressure in the body is done by baroreceptors, nerve clusterssituated in the walls of some arteries and veins. Specifically there is one at carotid’s mainbranch point, called carotid sinus, which controls and maintains normal blood pressurein the brain [18]. According to P. A. Laizzo, blood pressure (BP) is “the force appliedon arterial walls as the heart pumps blood through the circulatory system” [18]. It is acyclic value usually measured in millimeters of mercury (mmHg). The highest pressurelevels are achieved when the left ventricle pumps the blood to the body (approximately140 mmHg). The lowest values, on the other hand, are achieved during the relaxation ofthe heart (approximately 90 mmHg). This parameter can be calculated as

BP = CO · SV R, (2.4)

where CO is the cardiac output (volume of blood pumped by the heart in one minute)

16


and SV R stands for the systemic vascular resistance [resistance offered by the systemiccirculation — usually 1PRU (peripheral resistance unit)] [18]. As defined in Equation 2.4,blood pressure is related to blood flow and to blood resistance. Thus, the lower the pressuredifference (∆P ) between two points, the lower the flow; and the lower the resistance, thehigher the flow.

17

3 Infrared Thermography

Infrared thermography (IRT) is a non-invasive and non-contact measurement methodbased on passive sensing. It consists of a remote sensing system, which measures energythat is naturally available without using any source of illumination. This energy isheat emitted from the human body through the skin by evaporation, convection andradiation [16].The first recognition of this heat emitted in the infrared (IR) wave spectrum was madeby William Herschel in 1800, but only since the 1960s has been used in medicine tomeasure skin temperatures. Since then, infrared imaging has been developed enormouslyand applied in numerous medical fields such as oncology (e.g. breast and skin cancerdetection) and endocrinology (diagnose of diabetes type 2). In addition, this techniqueallows the monitoring of vital parameters such as heart rate and breathing rate [4].The current chapter provides a detailed description of the physical fundaments of this noveltechnology as well as an overview of the human heat emission, its transmission throughair and the electronic equipment used to detect it. In the last section, some applicationsare presented.

3.1 Physical fundaments of the IRT

The infrared thermography applied in medicine, also known as medical infrared thermo-graphy, consists of the measurement of energy radiated from a body due to its temperature.According to Planck’s law, any body where the temperature is higher than zero Kelvin(0◦ K), emits radiation with a certain distribution of wavelengths:

ρ(T, λ)dλ = 2hc2

λ51

e hcλKT − 1

dλ. (3.1)

Here, ρ(T, λ) (W/m3) is the density of emitted energy that depends on T (K) andλ (m). These variables correspond to the temperature of the so-called black body andthe wavelength of its radiation, respectively. In addition, h (J · s) stands for the Planck’sconstant, K (J/K) is the Boltzmann’s constant and c (m/s) represents the speed of thelight [16].A black body is defined as an ideal body, which absorbs completely the energy that re-ceives and, at the same time, totally radiates the energy related to its temperature. Thismeans α(T ) = ε(T ) = 1, where α(T ) is the absorption coefficient and ε(T ) the emissioncoefficient. Nevertheless, real bodies are not ideal. If the body emits the same portion ofenergy in all wavelengths (0 < ε(T ) = constant < 1), it is called grey body. If it emitsa different amount of energy depending on the wavelength (0 ≤ ε(λ, T ) ≤ 1) is called

19


selective radiator [16] , [6].Hence, the energy emitted depends on the spectral emissivity of the material. Some spec-tral emittances of metals and other materials are shown in Figure 3.1.

SilverGoldPlatinumRhodiumChromeGraphiteTantalumMolybdenumSeleniumAntimonium

11 2 3 4 5 6 7 8 9 1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Wavelength (ìm)

Specifi

c E

mis

sivi

ty

Figure 3.1: Emissivity of some metals and other materials depending on the wavelength of thesignal [13].

Therefore, the total radiation power of the body (Φ) (W ) depends on both temperatureand emissivity of the material. It is calculated using Stefan-Boltzmann’s law:

ΦA

= εσ(T 4 − T 4C), (3.2)

where A (m2) is the surface area of the body, σ the Stefan-Boltzmann constant (σ =2π5K4

15c2h3 = 5.669 ·10−12 Wcm−2K−4) and TC (K) stands for the temperature of the surround-ing. Moreover, Equation 3.2 states that the total radiation power is a function of thefourth power of the temperature:∫ ∞

0ε(λ, T )ρ(λ, T )dρ ∝ T 4. (3.3)

The wavelength, at which a black body emits radiation, has a maximum, which shifts withchanging temperature. This is shown by Wien’s law:

Tλmax = 2898µmK. (3.4)

Here, temperature is inversely proportional to wavelength (i.e. directly proportional tofrequency). The emitted wavelength is about 10 µm when close to room temperature.

20

3.1 Physical fundaments of the IRT

Summarizing, Figure 3.2 shows a graphical representation of Planck’s law for a black body.It describes the trend of emitted energy’s intensity in relation to temperature. Bodies withtemperatures higher than 500◦C, for example, emit radiation in both visible and infraredspectrum. For each curve, there is a maximal intensity. The higher the temperature, thesmaller is the wavelength of this maximum, as stated by Wien’s law. Moreover, the areaunder each curve provides the total energy emitted by the body, which is stated by Stefan-Boltzmann’s law [13]. For this reason, the body with the highest temperature radiateswith higher intensity, as shown in Figure 3.3.

Visualspectrum

2000 ºC

6000 ºC

500 ºC

0 ºC

-196 ºC

0.1 1 10 100

1E-06

1E-05

1E-04

1E-03

1E-02

1E-01

1E+00

1E+01

1E+02

1E+03

1E+04

1E+05

Wavelength (µm)

Speci

fic S

pect

ral E

mitt

ance

(W

/cm

²µm

)

Figure 3.2: Planck’s law for a black body [13].

The spectrum of frequencies emitted by this energy is principally in the infrared regionof the electromagnetic spectrum (0.75µm − 1000µm), i.e. between visible light andmicrowaves.As plotted in Figure 3.4, the infrared spectrum can be divided in 5 different bands: (1) nearinfrared (NIR - 0.74 µm to 1 µm), (2) short wave infrared (SWIR - 1 µm to approximately3 µm), (3) middle wave infrared (MWIR - 3 µm to 5 µm), (4) long wave infrared (LWIR- 8 µm to 14 µm), and (5) very long wave infrared (VLWIR - 14 µm to 1 mm) [6], [1].Figure 3.5 shows the spectrum of the energy emitted by a body according to its temperat-ure. There are only two bands in which the temperature of the human body emits energy,and which provide adequate thermal sensivity: MWIR and LWIR. Nevertheless, the LWIRis chosen for imaging the arterial pulse, since the emission of energy is higher within thisband than in MWIR. Moreover, the LWIR spectrum is becoming most used due to itsmore economical sensor technology [3]. Sections 3.3 (page 23) and 3.4 (page 26) providemore information about this topic.

21


0 100 200 300 4000

0.5

1.5

1

0 ... inf

(2...20) ìm

(8...14)ìm

(3...5)ìm

Temperature (ºC)

Sp

eci

fic r

ad

ian

t e

mitt

an

ce(W

/cm

²)

Figure 3.3: Stefan Boltzmann’s law graphic [13].

Gammaradiation

X - ray Ultraviolet Visible spectrum Infraredradiation

Milimetric /submilimetric

Radio waves

Near IR Short IR

Atmo

sphe

ric

Middle Wave IR (MWIR)

AbsorptionLong waveIR (LWIR)

Very long wave IR (VLWIR)

1 A 100 A 0.4 ìm 0.75 ìm 300 ìm 3 mm 100 km

0.75 ìm 0.9 ìm0.4 ìm

2.4 ìm 3 ìm 5 ìm 7 ìm 14 ìm 300 ìm

Wavelength (ë)

Figure 3.4: Electromagnetic spectrum and a detailed diagram of the infrared spectrum.

3.2 Heat emitted by the human body

Body temperature is the difference between the amount of heat produced by metabolismwithin the body and the amount of heat lost to external environment. The core tem-perature, which is the body temperature within deep tissue, ranges from 36 to 38◦C. Itis relatively constant, despite the degree of physical activity. On the other hand, skintemperature is lower, averaging 33◦C. It depends on the skin site and fluctuates due toheat flow into it. Since measuring core temperature is generally more difficult and time

22

3.3 Detection of the emitted signal

3 4 5 6 7 8 9 10 11 12 13 14 15 16 170

1E+7

2E+7

3E+7

4E+7

5E+7

1E+7

T = 0 ?C

T = 36.6 ?C

T = 50 ?C

Wavelength (ìm)

Spect

ral E

mitt

ance (

W/m

²)

Figure 3.5: Spectral emittance at temperatures similar to the temperature of the humanbody [5].

consuming, skin temperature is normally measured [8].Heat is produced by the human body through metabolism and transported to the skin byboth conduction (through the tissue) and convection (through blood). In superficial bloodvessels, this convection provokes fluctuations of the skin temperature. Consequently, theskin temperature is influenced by the blood flow.The heat in the surface of the skin is dissipated to the environment via conduction, con-vection, radiation, and evaporation. These methods of heat loss are classified as “dry”heat exchange with the environment. Apart from them, heat is also lost by evaporationof sweat and respiration. Sweat is evaporated on the skin surface, whereas respiratoryevaporation takes place in the respiratory tract. It is important to emphasize, that theheat losses due to respirations are minimal and insignificant [8].Figure 3.6, illustrates the human body heat exchange. Sooner or later, body andenvironment reach equilibrium. Only pathological processes such as inflammation, cancer,etc. can disturb the heat flux.


The energy received WR = εσT 4R (W/m2) by the detector is, principally, a sum of the

following components: the emitted radiation We = εσT 4e (W/m2), the reflected radiation

Wr = εσT 4r (W/m2) and the transmitted radiation Wt = εσT 4

t (W/m2) (see Figure 3.7).

WR = We +Wr +Wt. (3.5)

23


Basal heat productionMetabolism, exercise, etc.

Heat resistanceMuscle, tissue, etc.

Blood

Skin

CONVECTION

RADIATION

CONVECTION

CONDUCTION

CONDUCTION

EVAPORATION

Figure 3.6: Heat exchange processes of the body, which affect the skin temperature.

24


Target surface

Reflected radiationEmitted radiationTransmited radiation

Figure 3.7: Components of the radiation received by the camera.

Emitted energy (We) is the most interesting component under study, because it is theonly one that provides information about the temperature of the body. The other twocomponents are related to other sources, which are not of interest. If the camera is placedin front of the body’s surface and calibrated in an optimal way, Wr and Wt will not berepresented in the recording [6]. The level of transmittance of air is strongly dependenton the wavelength (see Figure 3.8). While some ranges provide high transmittances main-tained over longer distances, like the LWIR spectrum, others include high attenuation,such as the MWIR spectrum.

(3...5)µm (8...14)µm

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 2 4 6 8 10 12 14

Wavelength (ìm)

Spect

ral t

ransm

ittance

Figure 3.8: Spectral transmittance in the infrared spectrum [13].

The detection of the received energy is based on the photoelectric effect. Figure 3.9represents the physical mechanism of this phenomenon. Electrodes in the camera’s detectorare situated in its valence band; this is, in equilibrium (Wv). When heat radiation formthe body under study achieves these electrodes, they are excited and move to a higherenergy band, the conductive band (WL), due to the energy transported by the radiated

25


W

+ +

_

_

_WL

Wv

hí1=WL-Wv

hí2>WL-WV

Figure 3.9: Representation of the photoelectric effect.

photons. This energy is computed as

Wph = hν = hc

λ, (3.6)

where h (J · s) is the Planck’s constant, ν (Hz) stands for the frequency of the photon, c(m/s) represents the speed of light and λ (m) is the wavelength of the photon [8].When returning to equilibrium, the difference of energy between the two bands (∆W )produces an electrical signal (current) which frequency depends on ∆W and informs aboutsome characteristics of the original IR signal. Typical materials for detectors are PbS(lead (II) sulfide), PbSe (lead selenide), InSb (indium antimonide) or HgCdTe (mercurycadmium telluride) [16].

3.4 Infrared cameras

The radiation emitted by the body is impossible to visualize with the human eye, but it ispossible using special IR cameras, which convert the energy detected into a multi-colouredimage and displays it on a monitor screen. By detecting this energy in a specific regionof interest and in an interval of time, it is possible to obtain a group of images thatdetermine temperature variation. It is important to highlight that thermal imaging onlyvisualizes the skin surface temperature, its absolute value, and the distribution of thermalfields. Its accuracy is limited, due to limited knowledge of the emissivity, and is notcapable to obtain absolute measurements such as blood flow or blood pressure [6], [1].In general, infrared cameras can be classified as follows: (1) thermography systems, whichare used to measure temperature of a specific region of interest, and (2) IR-Imager,which objective is to visualize objects, commonly used in darkness. This classification isillustrated in Figure 3.10.

In both typologies are included scanning and focal plane array cameras. Scanning cameras,on one hand, are characterized by the fact that the measured object is scanned with anoptical-mechanical deflection system and uses a one-element-detector for the conversion

26

3.4 Infrared cameras

of the infrared radiation. Figure 3.11(a) illustrates the fundamental beam path in thecamera.

Infrared Cameras

Thermography systems

Scanner Focal Plane Array (FPA)

Cooled Uncooled

IR-‐Imager

IR-‐Imager Scanner Focal Plane Array (FPA)

Cooled Uncooled

Figure 3.10: General classification of infrared cameras [13].

On the other hand, the focal plane array (FPA) is a matrix of infrared detectors, whichcapture the complete signal. In this case, the camera is simpler and more compact.Figure 3.11(b) describes the beam path. These cameras can include a cooling system,necessary for the operation of the semiconductor materials. This prevents the detectorfrom being flooded by its own radiation. Nevertheless, cooled infrared cameras are veryexpensive. To reduce costs and increment the velocity of detection, uncooled cameraswere developed. They operate at ambient temperature using small temperature controlelements. Although the stabilization of sensors to an operating temperature reduces imagenoise, resolution and image quality are also diminished.

As mentioned in section 3.1, two spectrum ranges can be detected: MWIR and LWIR.The MWIR cameras generally have more pixels count, hence higher resolution for the sameprice. Nevertheless, the MWIR FPA must be cooled to temperatures as low as −196.15◦C,which increases its cost. MWIR has been considered the best choice for high-resolutionmedical thermography application, thus is quite effective in detecting small tumours [6].

27


(a) (b)

Figure 3.11: (a) Beam path of a scanning camera (1-Detector, 2-Lens, 3-Horizontal deflector,4-Vertical deflector, 5-Lens, 6-Object, 7- Measured spot). (b) Beam path of a FPAcamera (1-Object, 2-Lens, 3-Detector) [13].

3.5 Medical applications of IRT

Using IRT in the medical field has several advantages that need to be mentioned. Firstly,it is a non-contact method, which does not emit any harmful radiation (it is a passivemethod) and permits the patient to stay comfortable, since the camera can be placed at aconsiderable distance. Secondly, the emissivity of human skin (ε(T )) is nearly 1.0, whichpermits the IRT to be a highly efficient method. Third, the data obtained can be easilymanipulated and processed. Finally, it is possible to take measures during both day andnight, because there is no need for sources of illumination [8].In the following subsections, some medical applications of the infrared thermography aredescribed.

3.5.1 Detection of fever

Infrared imaging has been proposed as a system for the detection of fever. As it measuresthe skin temperature, which is lower than the core body temperature, is it possible todetect fever. It is only necessary to establish an appropriate temperature baseline at, forexample, the forehead. This simple application can help to detect and defence from thescreening of some diseases, like SARS (Severe Acute Respiratory Syndrom), thus one ofthe first symptoms of a disease is a temperature change. See [8] for more informationabout this application.

28

3.5 Medical applications of IRT

3.5.2 Infrared Imaging of the Breast

Another important application of IRT is detection of breast cancer. It is based on theresearch carried out by Lawson in 1956, which states that skin temperature over a cancerin the breast is higher than that of normal tissue due to venous convection [19]. Furtherstudies reveal that angiogenesis, this is, the recruitment of existing blood vessels and theformation of new ones, is produced by precancerous or cancerous cells. The existing vesselsmaintain a steady supply of nutrients to the growing mass, which produces their openingand also the formation of new blood vessels, which connect the tumour to existing arteriesand arterioles. This produces the increase in heat and vascular asymmetry between bothbreasts, making possible the detection of the tumour in infrared images [8].

3.5.3 Biometrics: Face recognition

Face recognition is the most extended biometric modality, thus is the natural way ofidentification among humans. Since the beginning, the research has been developed withcameras using the visible spectrum (wavelengths from 380 to 750 nm), although in thisband the light variability, provoked by the reflection of incident light, is a problem.Therefore, researchers have started using thermal infrared instead. The methodology,similar at the one used by cameras working on the visible band, consists of three phases:face detection, feature extraction, and classification.Detection is needed to lately recognize the face. It is studied from a statistical point ofview using a Bayesian approach. With it is possible to delineate the face from the rest ofthe scene. Figure 3.12 shows an example of face detection.Feature extraction reduces the face to a feature vector, which is a highly detailed mathem-atical description of the physiognomy. Due to temperature variation created by the bloodflow, the infrared camera detects major superficial vessels from the face. For this reason,a unique feature vector can be extracted by segmentation of the vascular network. Thistechnique provides an identifying feature vector that endures through superficial changesof appearance and aging.Finally, classification matches the feature vector to one of the records kept in thedatabase [8].

(a) (b)

Figure 3.12: (a) Original Image. (b) Image segmented using Bayesian approach [8].

29

4 Infrared data processing

In the present chapter, all methods and mathematical procedures required to extract bothbreathing and heart rates using infrared thermography (IRT) are described. The informa-tion written in the previous chapters is now used as fundamental basics for the design andunderstanding of the detection algorithm.Furthermore the chapter provides a detailed description of the recording scenario frommeasurements performed to validate these algorithms. In addition, equipment used aswell as the required pre-processing software is presented. Finally, a slight introduction ofthe software used to perform the statistical analysis is also included.The algorithms were based on the method developed by Garbey et.al. [12] published in2007 and implemented using MATLAB (MATLAB 2013b, 64-bit (maci64), The Math-Works Inc., Natick, MA).

4.1 Respiratory rate detection

As mentioned in the previous chapter, thermal imaging measures the instantaneous tem-perature of the body by sensing its emitted infrared energy. For the detection of therespiratory rate the selection of a region of interest, whose temperature varies at the samefrequency as the respiratory rate, was necessary. This is possible in the nostrils, sinceduring inspiration they present lower temperature than during expiration. This is due tothe warming of the inhaled air that takes place in the nasal cavity and heat exchange inthe lungs (see section 2.1). Nevertheless, temperature variation detected by the cameracontain the respiratory frequency smoothed, shifted and noisy [1]. Figure 4.1 describesthe typical temperature distribution during a respiratory cycle. As inspiration and post-expiratory pause have similar thermal signatures, are considered together as one uniquephase [21].Two thermograms are shown in Figure 4.2. Figure 4.2(a) demonstrates lower temperaturein the nostrils during inspiration. Conversely, Figure 4.2(b) illustrates high temperaturedetected during expiration.

31


24.50 24.55 24.60 24.65 24.70 24.75 24.80 24.851

2

3

4

5

6

7

8

9

Temperature (ºC)

Figure 4.1: Typical temperature distributions for the three breathing phases: expiration (red),post-expiratory pause (green) and inspiration (blue) [21].

(a) (b)

Figure 4.2: (a) Infrared image recorded during inspiration phase. (b) Infrared images recordedduring expiration phase.

Based on information given by thermograms, an algorithm was developed to detect sub-jects’ respiratory rate. It is graphically schematised in Figure 4.3 and described in thefollowing subsection.

32


Input: Data recordedby the IR camera

Step 1: Definition ofthe ROI

Step 2: Tracking

Step 3: Reducedimension (3D -> 1D)

Step 4:Normalization

Step 5: Filter

Step 6: Select framesunder study

Step 7: Fast FourierTransform

Step 8: Extraction ofthe RR

Output: Respiratoryrate in time

until the end of the video

Figure 4.3: Flowchart of the detection algorithm from the respiratory rate.

33


4.1.1 Description of the algorithm step by step

Data obtained with IR camera was pre-processed by IRBIS 3 Professional (described insection 4.6.1) and introduced in MATLAB as an AVI video. The video was recorded in aconstant frame rate [or sampling frequency (Fs)] of 50 frames per minute. IR-frames, asshown in Figure 4.4, were taken in RGB24 format and 647x441 pixels, each pixel definedby 24 bits, 8 bits each dimension. Consequently, the video was a 4D-matrix (3D-frames xtime).

RG

B

647 pixels

44

1 p

ixe

ls

Figure 4.4: Infrared image format.

Since all information is condensed in the nostrils, a smaller region around them wasdefined and only this was analysed. The analysis consisted of an initial average of thepixels inside the region of interest. Then, data was first normalized and second filtered.Afterwards, the FFT was computed and the breathing rate, defined as the highest energypeak, was extracted. The present steps are described in detail forthwith.

Step 1 - Three regions of interest (ROI) were selected: (1) “normal ROI”, pixels inside arectangle that embraces just both nostrils; (2) “big ROI”, which includes 20 more pixelsin both dimensions (height and width) than the “normal ROI”; (3) “one nostril ROI”,which only includes one of the two nostrils. These three ROIs are represented in Figure 4.5.

Step 2 - After selecting the ROI, a tracking algorithm was launched to follow accurately theregion over time. This was done by an iterative update of the transformation parametersfrom one frame to the next one,

p −→ p+ ∆p, (4.1)

where p are the parameters which define the ROI of a certain frame and ∆p their variationfrom that frame to the next one. The function of the tracking algorithm is principally todefine this variation (displacement).

34


Figure 4.5: Representation of the three ROIs: normal ROI (green), big ROI (blue), only onenostril (yellow).

In Figure 4.6 and Table 4.1, are described the most important displacement models thatcan be used.

Figure 4.6: Basic set of 2-D Transformation [28].

In the present thesis the Kanade-Lucas-Tomasi (KLT) tracking algorithm [20], [29] incombination with a feature identification method developed by Shi and Tomasi [25] wasused. The KLT algorithm is a feature-tracking algorithm which success consists on theoptimization of the computation costs. It is faster than other trackers because decideswhere to search the new position of the feature point using the so-called optical flow. Thistechnique, based on local Taylor series approximation of the frame I(x, y), consists in apre-study of the spatial intensity around the pixel under consideration.KLT detects Harris corners in the first frame and for each corner computes the motionbetween consecutive frames using local affine transformation. Continuously, it links themotion vectors in successive frames to get a track for each Harris point. The completealgorithm is schematized in Figure 4.7.

35


Table 4.1: Properties of the 2-D transformation types [28]Transformation Preserves # DoF Parameters p Warps W (x; p) Jacobian J = ∂W

∂p

Translation Orientation 2 (tx, ty) [I|t]2x3

1 00 1

Euclidean Lengths 3 (tx, ty, θ) [R|t]2x3

1 0 −x sin θ −y cos θ0 1 x cos θ −y sin θ

Similarity Angles 4 (tx, ty, a, b) [sR|t]2x3

1 0 x −y0 1 y x

Affine Parallelism 6 (tx, ty, a00, a01, a10, a11) [A]2x3

1 0 x y 0 00 1 0 0 x y

Projective Straight lines 8 (h00, h01, ..., h21) [H]3x3 See section 6.1.3 from [28]

Figure 4.7: Block diagram of the KLT tracker [2].

36


This algorithm is defined as

∆p = H−1∑x

∇I ∂W∂p

T [T (x)− I(W (x; p))], (4.2)

and

H−1 =∑x

∇I ∂W∂p

T∇I ∂W∂p

, (4.3)

where p are the parameters and ∆p their variation; H−1 defines the Harris corners; ∇Istands for the gradient of the image; ∂W

∂pdefines the Jacobian of the Wrap matrix (it

carries important information about the local behaviour of W (x; p)), and finally, T (x) isthe template of the image I(x).In conclusion, this tracker follows with accuracy the ROI and provides the position of thenostrils over time. Henceforth, only this selected region was used for the data processing.

Step 3 - As mentioned above, the IR-data had RGB24 format. For this reason, wasnecessary a reduction of its dimensions, so that the original 3D frames were reduced to asingular value. This was done by (a) transforming RGB format to grey (Equation 4.4) andby (b) computing the mean value of all pixels (Equation 4.5). Both steps are illustratedin Figure 4.8.

I(x, y, n)grey = (0.2989 · I(x, y, n)R) + (0.5870 · I(x, y, n)G) + (0.1140 · I(x, y, n)B), (4.4)

S(n) =∑x

∑y I(x, y, n)greyRxRy

. (4.5)

Here, I(x, y, n) is the nth-infrared frame with 0 ≤ x ≤ Rx, 0 ≤ y ≤ Ry and0 ≤ I(x, y, n) ≤ 1. After these steps, the value extracted from each frame was saved ina vector (S(n)), whose length correspond to the number of frames.

frames

441 pixels

647 pixels

x

y

frames

frames1

1

Input data: Thermal ImageSelection of the ROI and Tracking (Steps 1and 2)

Mean of the ROI(Step 3)

x

y

frames

Transformation togrey (Step 3)

Figure 4.8: Illustration of the ROI’s selection (steps 1 and 2), its transformation to grey (step3a) and the computation of the mean (step 3b).

37


Step 4 - In order to minimize the Gibbs phenomenon produced by any jump discon-tinuity in the original signal, the vector S(n) was normalized. It was used a low-ordertrigonometric polynomial as follows:

U(n) = Sn − (α cos t+ β), (4.6)

with α = 12

(S(0)−S(N−1)

)and β = 1

2

(S(0)+S(N−1)

). An example of the normalization

process is shown in Figure 4.9.

0 20 40 60 80 100 1200.5

0.52

0.54

0.56

0.58

0.6

0.62

0.64

Time [seconds]

Mean Value of the ROI

s (

t)

0 20 40 60 80 100 120-0.08

-0.06

-0.04

-0.02

0

0.02

0.04

Time [seconds]

Normalized signal

u (

t)

Figure 4.9: Illustration of step 4.

Step 5 - Assuming that breathing rate of all subjects ranged between 8 and 24 breaths perminute, a reasonable assumption considering that all subjects were healthy and relaxedduring the recording, all signals with lower or higher frequency were removed. For thisreason, U(t) was filtered with an elliptic low-pass filter and a Butterworth high-pass filter.The elliptic filter, also known as Cauer filter, was chosen as low-pass filter, since it providesa fast transition in gain between the passband and the stopband. Its gain is computed asfollows,

Gn(w) = 1√1 + ε2R2

n(ξ, w/wo), (4.7)

where Rn is the nth-order elliptic rational function, w0 is the cut-off frequency, ε the ripplefactor and ξ the selectivity factor.On the other hand, a Butterworth filter was chosen as high-pass filter, since it stands fora maximal flatness in its frequency response. The gain of a high-pass filter is given by,

Gn(w) = Go√1 +

(wow

)2n, (4.8)

where G0 is de DC gain, w0 represents the cut-off frequency and n stands for the order ofthe filter.The properties of both filters are detailed in Figure 4.10 and Table 4.2.

38


0 5 10 15 20

-90

-80

-70

-60

-50

-40

-30

-20

-10

0

Ma

gn

itud

e (

dB

)

-2.6818

-2.006

-1.3302

-0.6543

0.0215

Frequency (Hz)

Ph

ase

(ra

dia

ns)

(a)

0 5 10 15 20

-60

-50

-40

-30

-20

-10

0

Ma

gn

itud

e (

dB

)

0.17

0.6694

1.1688

1.6681

2.1675

2.6668

3.1662

Frequency (Hz)

Ph

ase

(ra

dia

ns)

(b)

Figure 4.10: Magnitude and Phase Responses of (a) high-pass filter and (b) low-pass filter.

Table 4.2: Main properties of both filters, elliptic filter and Butterworth filterLow-pass filter High-pass filter

Type Elliptic ButterworthOrder 2 2Frequency rate 50 50Cutt-off frequency 24

60∼= 0.4 8

60∼= 0.13

Ripple Equiripple (0.1 dB) Approx. zerodB between the stopband and thepeak value in the passband 80 -

Step 6 - To obtain a temporal signal of the respiratory rate, steps 7 and 8 were computedevery 10 seconds. For this reason, a recursive bucle was designed to select the framesunder study (NRR). On the first interaction, the bucle selects the first NRR frames fromp(t). On the second one, however, are selected the frames from NRR + 1 to NRR + 500(500frames = 10seconds · samplingfrequency); on the third one, from 2NRR + 1 to2NRR + 500; and so on.

Step 7 - After the selection of frames, it was possible to compute the Fast Fourier Trans-form (FFT) of the signal and represent its frequency components. For this reason, theclassical Cooley and Tukey 1-D base-2 Fast Fourier Transform, an efficient algorithm forthe calculation of the DFT, was computed. It is described as

X(k) =N∑j=1

x(j) · w(j−1)(k−1)N , (4.9)

where N is the length of the transformation, x(n) the signal to be transformed andwN = e(−2πi)/N corresponds to a Nth root of unity.Depending on the length of the transformation, the execution time of the FFT is shorteror longer. Defining N as a power of two, the minimum execution time is achieved. If L

39


states as the length of the signal x(n), N was always defined as the next power of twohigher than L.

Step 8 - After transformation, it was able to distinguish the most powerful (highest energy)value of the signal within the operational frequency band. Consequently, by extractingthe respective frequency from this value, the respiratory rate was found.

4.2 Heart Rate detection

The detection of the heart rate was based on the study of the temperature variation in theskin due to the pulsatile blood flowing through superficial blood vessels. For this experi-ment the external carotid arteries, situated in both sides of the neck, were analysed (seesection 2.2.2). Nevertheless, it is also possible to extract the pulse from other superficialvessels such as superficial temporal artero-venous complex or radial artero-venous complexin the wrist, among others.At every heartbeat the blood pumped by the heart flows through the carotid vessel and,due to its high temperature, produces a momentary increase of the temperature in theartery. Since carotid arteries are superficial vessels, the infrared camera, which is con-tinuously recording one side of the neck, can detect this change in temperature. Thesefluctuations, however, are not as powerful as the ones detected in the nostrils. Therefore,the detection of the heart rate needs a more complex processing algorithm.

Figure 4.11: Frames recorded with the infrared camera taken from the left side of three patient’sneck. The detected blood vessel is indicated by a red rectangle.

The thermograms shown in Figure 4.11 were recorded with the IR-camera. The red regionsrepresent warmer regions of the skin, whereas the coldest regions are depicted in blue. Hotlines can be identified. They correspond to region above the carotid artery.Based on the information given by the thermograms, an algorithm to detect heart ratewas developed. It is represented in Figure 4.12 and described in the following subsection.

40


Input: Data recordedby the IR camera

Step 1: Rotation(Hough)

Step 2: Selection ofthe ROI

Step 3: Computationof the X-mean

Step 4:Normalization

Step 5: Filter

Step 6: Select framesunder study

(M+N frames)

Select frames fromM to M+N

Select frames from(M-k) to (M+N-k)


Step 7: Periodification

Step 10: Extraction ofthe Heart Rate

Step 9b: Convolution


Step 7: Periodification

Step 9a: Historicalpower spectrum

Output: Heartrate in time

for k=0:M-1

until the end of the video

Figure 4.12: Flowchart of the detection algorithm from the heart rate.

41


4.2.1 Description of the algorithm step by step

Like in section 4.1.1, in which the breathing rate detection algorithm was described, in thissection is analysed the heart rate detection algorithm. Each step presented in Figure 4.12is now deeply explained. It starts with the selection of the ROI, which includes a noveltechnique to rotate images. Following, the vessel is defined as a line and a single valueis defined for each pixel within the line, as shown in Figure 4.13. After some processing(normalization and filtering), an adaptive estimation function is computed. This includesperiodification of the signal and FFT. At last, the heart rate, defined as the highestenergetic peak, is extracted.

timex

y y

time

Matrix (y x Number of frames)

Step 3Step 1 Step 2

Figure 4.13: Steps 1, 2 and 3 of the detection algorithm.

Step 1 - Due to thermal diffusion in the skin tissue, the modulation of skin temperatureis strongest near blood vessel. For this reason it was necessary to define a ROI. It wascrucial to select a small rectangular area so that its longest side was completely parallelto the direction of the blood flow (see Figure 4.13).The identification of the direction of the blood flow was achieved by using the Houghtransformation, an efficient procedure for detecting lines in pictures [3], [9]. Using a fixparameterization, in 1962 P.V.C. Hough developed a transformation in which an arbitrarystraight line in a digitized image was represented by a single point in the parameter space.This was done by utilizing angle-radius parameters (called “normal parameters”) ratherthan slope-intercept parameters.In this transformation a straight line, given by

ρ = x · cos θ + y · sin θ, (4.10)

must be firstly defined. Here θ is the angle of its normal and ρ corresponds to its algebraicdistance from the origin, as shown in Figure 4.14.

If θ is restricted to the interval [0, π], each line has unique parameters in the new plane,which means, each line corresponds to a concrete point (θ, ρ) in the parameter space. Inthe same way, using Equation 4.10 a single point from the original image (x1, y1) corres-ponds to a sinusoidal curve in the parameter space. An example of these curves is shownin Figure 4.15(a).

42


Figure 4.14: Normal parameters for a straight line [9].

Consequently, curves corresponding to collinear points have a common point of intersec-tion. By finding concurrent curves and extracting its match point (θ0, ρ0) is possible todefine the line passing through them. The equation of this straight line will be

y = − cos θsin θ x+ ρ

sin θ . (4.11)

Those points in the θ− ρ plane, in which more curves intersect, are the ones selected. Forthis, it is necessary a threshold value that defines the minimum number of sinusoidal curvesthat should intersect in a point. In this thesis, the maximum number of intersections in aconcrete point is defined as “k”. The threshold selected corresponds to 30% of k (0.3 · k).To reduce the computation required in Hough algorithm the Canny edge detection methodwas used. It permits to find all intensity edges existing in an image. By applying thismethod a binary image is obtained, in which only the most important edges are represented.An example can be found in Figures 4.15(b) and 4.15(c). Thereby, the computation ofHough’s algorithm is optimal and results are faster and more successful.

θ

ρ

−80 −60 −40 −20 0 20 40 60 80

−400

−300

−200

−100

0

100

200

300

400

(a) (b) (c)

Figure 4.15: (a) Sinusoidal curves resulting from the Hough transform applied in the exampleimage. (b) and (c) correspond to output images when computing edge() in MAT-LAB. The color axis scale in image (b) include a bigger range of temperaturesthan in image (c). For this reason, in (b) is possible to observe the warm mark ofthe carotid artery.

43


After extracting the most relevant lines from the original frame, the longest one wasselected. If the color axis scale has been correctly defined, this line corresponds to thedirection of the blood flow through the carotid artery and the ROI can be preciselyestablished.

Step 2 - Six different ROIs were extracted. They were centred in the artery’s verticalaxis and contain different sizes: (1) 8x8 pixels; (2) 16x16 pixels; (3) 32x32 pixels; (4)64x64 pixels; (5) 128x128; (6) 16x32 pixels 1.After defining the ROI, data was expressed as S(x, y, n). Here, 0 ≤ x ≤ Rx, 0 ≤ y ≤ Ry

are the spatial extent of the ROI and 1 < n < N the number of frames. The dimensionparallel to the blood flow was defined as the y-dimension, whereas the perpendicularone was defined as the x-dimension. Henceforth, only this selected region was used asprocessing data.

Step 3 - An average along the x dimension was computed and a two-dimension matrix wasobtained (y-position’s temperature for each frame) (see Figure 4.13). The mathematicalequation is given by:

S ′(y, n) = 1Rx

Rx∑x=0

S(x, y, n). (4.12)

With this computation, the ROI was compacted into a single line along the blood vessel.Each pixel (along y-dimension) had a temperature profile, which was slightly shifted in thetime (or frame) domain with respect to the others. This was due to the pulse propagationphenomenon, since blood flows with a certain velocity and needs a certain time to travelthrough the vessel. Therefore, each y-position was separately in the frequency domainstudied, since no shifted signal is present. With this technique noise was reduced.

Step 4 - To minimize the Gibbs phenomenon, each y-signal was normalized with thesame methodology used in the respiratory rate detection algorithm (see page 38).

Step 5 - After normalization, the signal was filtered by both elliptic low-pass and high-pass filters (see page 38). Characteristics of both filters are depicted in Figure 4.16 anddescribed in Table 4.3.

Step 7 - Following Garbey et al. [12], a subsection of NHR frames was selected. Theseframes were periodificated by first applying symmetry and then a periodic extension.Symmetry is defined as

∀n ∈ (0, NHR), Py(NHR − n) = −Py(n), (4.13)whereas periodification is given by,

∀n ∈ (0, 2NHR),∀k ∈ Z, Py(n+ k2NHR) = Py(n). (4.14)1(width x length) pixels

44


0 2 4 6 8 10 12

-100

-90

-80

-70

-60

-50

-40

-30

-20

-10

0

Ma

gn

itud

e (

dB

)

-10.9146

-8.6661

-6.4176

-4.169

-1.9205

0.328

Frequency (Hz)

Ph

ase

(ra

dia

ns)

(a)

Ma

gn

itud

e (

dB

)

Frequency (Hz)

Ph

ase

(ra

dia

ns)

0 2 4 6 8 10 12

-70

-60

-50

-40

-30

-20

-10

0

1.5781

4.8841

8.1901

11.4961

(b)

Figure 4.16: Magnitude and Phase Responses of (a) high-pass filter and (b) low-pass filter.

Table 4.3: Main properties of both filters, elliptic low-pass and high-pass filtersLow-pass filter High-pass filter

Type Elliptic EllipticOrder 8 8Frequency rate 25 25Cutt-off frequency 110

60∼= 1.9 50

60∼= 0.8

Ripple Equiripple (0.1 dB) Equiripple (0.1 dB)dB between the stopband and thepeak value in the passband 80 80

Figure 4.17 shows the periodification of a selected group of NHR frames (Py(n)). Thisprocess, which was computed for each y-position, empowered the heart frequency amongother frequencies and made easier its detection.

0 100 200 300 400 500 600-6

-4

-2

0

2

4

6

8x 10

-5

N° of sample

Py(t

)

Filtered signal

0 200 400 600 800 1000 1200-8

-6

-4

-2

0

2

4

6

8x 10

-5

N° of sample

Py(t

)

Periodic signal

Figure 4.17: Illustration of step 7.

45


Step 8 - Forthwith, the classical Cooley and Tukey 1-D base-2 Fast Fourier Transform,described in page 39, was computed for each y-position. Lastly, all power spectra wereaverage into a composite power spectrum:

X̄(k) = 1Ry

Ry∑y=0

Xy(k). (4.15)

Step 9 - Nevertheless, thermoregulatory vasodilation and noise might affect the resultingpower spectra. For this reason, the current measurement as well as past measurementswere taken into account in an estimation function. In this function the current powerspectrum X(k) was convolved with a weighted average of the power spectra computedduring the previous M frames. The spectrum resulting from the estimation function wasnamed “Historical power spectrum” (H(k)). The main objective was to filter out transientfeatures. It is defined as follows,

H(k) =∑Mi=1 P̄i(k)∑M

i=1∑Fj=1 P̄i(j)

, (4.16)

where M corresponds in this case to 50. It consists of two parts:(a) periodification and FFT computation of NHR frames. This step is done M times.(b) computation of the average of the M spectra obtained in (a).For a better understanding of this function, it is illustrated in Figure 4.18.

M+NHR = 612 frames

NHR=512 frames

y

. . ..

H (f)

P (f)

M=100 frames

AV

ER

AG

E

Final Spectrum

Periodification and FFT







Figure 4.18: Operating method of the adaptive estimation function.

Step 10 - Convolution between the current power spectrum (X̄(k)) and the historicalpower spectrum H(k). A final spectrum was obtained. The highest energy value of itshould correspond to the subject’s pulse.

46

4.3 Experimental protocol


Thermal imaging recordings were done in an optic laboratory of the Philips Chair forMedical Information Technology (MedIT) (Helmholtz-Institute for Biomedical Engineeringat RWTH Aachen University, Germany).To obtain optimal results, all data was acquired in a stable indoor environment(temperature stability of 1◦C) with minimal environmental infrared sources such as sun-light or electrical sources.Concerning the subjects, they seated in front of the infrared camera. While theirtemperature was recorded, measurements with a Ground Truth (GT) device wereperformed. It provides a reference signal, permitting, therefore, the validation of theresults obtained with infrared imagery.The recording equipment is shown in Figure 4.19. For infrared data acquisition, aVarioCAM hr head (InfraTec GmbH, Dresden, Germany) was used. For the GT data,SOMNOlab 2 (Weinmann Geraete fuer Medizin GmbH + Co, Hamburg, Germany) wasapplied.

(a) (b)

Figure 4.19: Equipment used during tests: (a) Infrared Camera. (b) Ground truth device[SOMNOlab 2 (Weinmann Geraete fuer Medizin GmbH + Co, Hamburg,Germany)].

The aim of this study was to record the temperature changes in both carotid artery andnostrils, in order to extract HR and RR, respectively. Since the ROIs are in differentparts of the body, two different recordings were necessary. Both procedures are describedforthwith.

47


4.3.1 Extraction of the respiratory rate

A total of ten subjects were asked to take the test. As mentioned before, they seated infront of the IR camera, which was focused on the nostrils. Simultaneously, respiratory ratewas measured using Weinmann’s SOMNOlab 2. Figure 4.20 depicts the scenario duringthe recording.

èd

Infrared ThermalCamera

Subject under test

Processing station

IEEE-1394 firewire data interface

SOMNOlab 2

Bluetooth

(a) (b)

Figure 4.20: (a) Experimental setup of infrared thermography respiration monitoring. Thevariable “d” defines the distance between the camera and subject’s nose and θ theangle of the camera. (b) Real scenario.

In order to study the effects of distance (distance (d) between nostrils and IR camera -see Figure 4.20(a)) in the accuracy of the results, three different distances were analysed:50 cm, 75 cm and 100 cm. For each distance, recordings of two minutes were carriedout. The experiment was established in a relatively closed space with a environmenttemperature varying between 22.5 and 23.5◦C. It is valuable to note that no direct lightsources such as sunshine or lamps were permitted.The camera was always at the same high. For this reason and due to the differentpatients’ high, it was necessary to change the angle θ of the camera. In Table 4.4 allangles are presented.

4.3.2 Extraction of heart rate

To extract heart rate five subjects were enrolled. Contrarily to the previous study, thecamera was focused on their neck. At the same time, SOMNOlab 2 measured their instantheart rate. The scenario is shown in Figure 4.21.For each patient two recordings, each of them with duration of 40 seconds, were car-

ried out. Once more, the experiment was established in a relatively closed space with aenvironment temperature varying between 22.5 and 23.5◦C and no direct light source waspermitted.

48


Table 4.4: Angles in which the camera was placed during recordingsSubject Recording 50 cm Recording 75 cm Recording 100 cm

1 −30◦ −20◦ −15◦

2 −30◦ −20◦ −10◦

3 −30◦ −25◦ No measurements were performed4 −30◦ −20◦ −15◦

5 −30◦ −20◦ −10◦

6 −30◦ −20◦ −15◦

7 −30◦ −20◦ −20◦

8 −30◦ −25◦ −20◦

9 −30◦ −25◦ −20◦

10 −30◦ −15◦ −15◦

èd

Infrared ThermalCamera

Subject under test

Processing station

IEEE-1394 firewire data interface

SOMNOlab 2

Bluetooth

Figure 4.21: Experimental setup of infrared thermography heart monitoring. Variable “d”defines the distance between the camera and subject’s neck and θ the angle ofthe camera.

49


4.4 Comparison between IRT’s and GT’s results

To compare vital signs computed by infrared imaging to that reported by the ground truthdevice, it was used a weighted indicator developed by Garbey et al. [12]. This indicator iscalled “Complement of the Absolute Normalized Difference” (CAND) and formulated asfollows:

CAND = 1− |GT − IRT |GT

, (4.17)

where GT is the respiratory rate computed by SOMNOlab 2, IRT is the one computed byan algorithm developed in previous chapters and CAND is a real value within the interval]−∞, 1].It consists on a weighted measure of the similarity between both results. If both rates areidentical, CAND is 1. Being IRT a real positive number, CAND will be 0 or negative forvalues of IRT > 2 ·GT .For the computation of this index, it was necessary that data from both technologies(IRT and SOMNOlab) had the same sampling frequency. Since IR respiratory rate wascomputed at a frame rate of 1 fps and SOMNOlab at 16 fps, it was necessary to reduce GT’ssampling frequency. For this reason, GT data obtained within a second was averaged.

4.5 Hardware

In this experiment, two main devices were used: an infrared camera [VarioCAM hr-basic(InfraTec GmbH, Dresden, Germany)] and a ground truth [SOMNOlab 2 (WeinmannGeraete fuer Medizin GmbH + Co, Hamburg, Germany)]. This section describes somespecifications of both devices.

4.5.1 Infrared Camera

Infrared thermograms were collected by using a LWIR camera, VarioCAM hr head(InfraTec GmbH, Dresden, Germany) (Figure 4.22). It has a spectral bandwidth from7.5 to 14µm, which belongs to the LWIR spectrum (see section 3.1), and an excellentthermal resolution. The latter is better than 0.03K at 30◦C or otherwise better than0.04K. Since low temperatures represent little signal for the camera, it is important thatall available signal is detected and used. For this reason, the camera incorporates F 1.0quality lenses, which provide a precise focussing of all thermal radiation.Furthermore, it has an uncooled microbolometer Focal Plane Array (FPA) detector witha geometric resolution of (384 x 288) IR pixels and resolution enhancement to (768 x 576)pixels, depending on the camera configuration. Regarding measurements’ accuracy, it is

50

4.5 Hardware

about ± 2◦C.Data transference to the PC interface is realized via IEEE-1394 Firewire at a maximalframe rate of 60 frames per second. The frame rates selected in this thesis were: (1) 50 fpsfor respiratory rate detection algorithm and (2) 25 fps for heart rate detection algorithm.Pre-processing steps and image scaling were done, in turn, by using IRBIS 3 Professionalsoftware (see section 4.6.1).

Figure 4.22: Different views of the infrared camera VarioCAM hr head.

4.5.2 SOMNOlab 2

The SOMNOlab 2, illustrated in Figure 4.19(b) is a system for mobile sleep diagnostics.In the current study, it was used as GT for measuring respiratory and heart rates and,thus allows the validation of IRT results. The device incorporates a long list of possiblemeasurements such as airflow, breathing movements, snoring, heart rate, oxygen satura-tion, body position, and therapy pressure, among others. However, only two of them areused: breathing movements and heart rate.To measure breathing movements SOMNOlab incorporates an effort package. With it ispossible to detect both abdomen and thorax efforts. It is based on piezoelectric crystaltechnology, sensors that detect respiration movements of thorax or abdomen. They areintegrated in a T-belt, which attaches both sensors to the human body (see Figure 4.23).Hence, the sensors detect expansion of the thoracic cage during inspiration and, its con-traction during expiration.In addition, SOMNOlab 2 includes a pulsoximetry sensor that records both pulse and

plethysmogram. This sensor is placed on the subjects’s finger, as can be observed in Figure4.23(b). Pulse is detected with a scanning rate of 16Hz, whereas plethysmography with ascanning rate of 50Hz. The precision of pulse measurements is from 1 bpm to 2% of thedisplayed value. Examples of both signals are shown in Figure 4.24. Finally, connectionbetween device and PC is wireless via radio signals at 2.4 GHz (Bluetooth with modulationGFSK 1 Mbps).

51


(a)

(b)

(c)

Figure 4.23: Setup of SOMNOlab 2 device to measure abdomen and thorax efforts, as well asphotoplethysmography. (a) represents the thorax belt and (b) the abdomen belt.They allow measuring the efforts of the thorax and abdomen, respectively, duringrespiration. (c) represents the PPG sensor, which measures the cardiac cycle byilluminating finger’s skin. Adapted from [30].

0 10 20 30 4060

62

64

66

68

70SOMNOlab 2 output signal

Time [seconds]

Puls

e [beats

per

min

ute

]

(a)

0 10 20 30 400

50

100

150

200

250

300SOMNOlab 2 output signal

Time [seconds]

Photo

ple

thysm

ogra

phy (

PP

G)

(b)

Figure 4.24: (a) Pulse and (b) PPG signals recorded by SOMNOlab 2.

52

4.6 Software

4.6 Software

In this experiment, two software programs were used: (1) IRBIS 3 Professional, an inter-face, which controls the infrared camera and connects it with the PC, and (2) SPSS, asoftware used for the statistical analysis of the results.

4.6.1 Pre-processing of infrared data: IRBIS 3 Professional

IRBIS 3 Professional, represented in Figure 4.25, is the software used to control IR cameraand edit thermographic images. It has been developed by InfraTec GmbH, Dresden,Germany. This software can be described in three main steps:First, it provides a visualization of thermal images with screen/printer-optimisedcolour palettes. Second, it permits to edit these images, since it contains graphicand image-editing functions as well as freely definable colour wedges and enlargementfactors. Also, a selection of the temperature range is aloud. Third, the softwareallows the automatic export of thermal images into AVI movies. Besides, it enables thedefinition of a partial area of the thermogram, which is defined by the user, for AVI export.

Figure 4.25: Main screen of IRBIS 3 Professional.

53


4.6.2 Statistical analysis: SPSS

SPSS is predictive analytics software developed by IBM (New York, USA). It includes thenecessary material for an analytical process. It consists of seven packages. In this study,just IBM SPSS Statistics was used.Specifically, it was used the function boxplot. It is a visual aid to examining key statisticalproperties. Using non-parametric statistics, it depicts groups of data through theirquartiles. Lines extending vertically from the box (named whistles) indicate the variabilityoutside the upper and lower quartiles. Furthermore, outliers are plotted as individualpoints. An example and an illustrative description are presented in Figure 4.26. Thespacing between the different parts of the box indicates dispersion and skewness.

Outside values

Lower adjacent value

Lower quartile (Q1)

Upper quartile (Q3)

Median

(a)

Distance

100 cm75 cm50 cm

Mea

n C

AN

D (

%)

100,00

80,00

60,00

40,00

20,0030

1789

28

10

Seite 1

(b)

Figure 4.26: (a) Description of the main parts of a boxplot. (b) Example of a boxplot.

54

5 Study of results and discussion

After defining the methodology used to compute both algorithms (chapter 4) in this chapterare presented and analyzed the results.It is divided in three main sections. It starts with a description of the GUI designed toextract the respiratory and heart rate. The second section is focused on the study of therespiratory rate results and the last one on the heart rate results.The two last parts are also divided in (1) an initial summary of the results, presented intables, and (2) a statistical analysis and discussion of these results.

5.1 Graphical user interface (GUI)

A graphical user interface (GUI) was developed in order to make easier the interactionbetween the user and the software. A screenshot of the main screen is shown in Figure 5.1.

Figure 5.1: Screenshot of the graphical user interface (main screen).

55


5.1.1 GUI - Respiratory rate detection

When clicking on the “RESPIRATORY RATE” button, the user enters on a new screen,in which it is asked to select the AVI video under study. An illustration of the screen isshown in Figure 5.2.

(a) (b)

Figure 5.2: (a) GUI screen. (b) Selected Region of Interest (ROI).

From now on, a real example is detailed and steps described in Figure 4.3 are going tobe followed one by one. In the first and second steps, the ROI is selected in the GUI, asshown in Figure 5.2. By clicking on the “Create the new video” button, the KLT trackingis applied to the video, and an AVI video, which only contains the selected ROI, is createdwith the name given by the user in the “Save as” field. One frame of this new video isshown in Figure 5.2(b).Continuously, is presented a new screen aimed to show the results of the respiratory ratedetection algorithm. The screen is plotted in Figure 5.3. On the left side, are going to beplotted the results obtained by computing the respiratory rate detection algorithm and,on the right side, the results obtained by the GT.When the user clicks on the button “Open”, it is asked to select a video. By selectingthe one before created, the respiratory rate detection algorithm is launched.First, the mean value of pixels in the ROI is computed (third step) and, therefore, datais reduced to a single vector s(t), plotted in Figure 5.4(a). The signal has a visibleperiodicity, which corresponds to the variation of pixels’ value. During inspiration,temperature in nostrils is lower, which corresponds to a dark pixel, near to black; this is,near to 0. On the other hand, during expiration, temperature is higher and is representedas a bright pixel, near to white; this is, near to 1.

56


Figure 5.3: Screenshot of the graphical user interface (screen for the selection of the ROI in therespiratory rate algorithm).

Afterwards, in the fourth step, the signal s(t) is normalized, so that the mean value isselected to be 0. It is represented in Figure 5.4(b). Furthermore, both filters, describedin section 4.1.1 are applied, obtaining as a result the signal p(t), plotted in Figure 5.4(c).This corresponds to the fifth step.As illustrated in Figure 4.3 and explained in page 39, the sixth step consists on the selectionof samples from p(t) that are going to be used in each bucle’s iteration. The first NRR

samples from p(t), which correspond to the initial frames of the video, are firstly selected.Only these frames are used to compute steps 7 and 8 in the first iteration. On the secondone, frames from NRR + 1 to NRR + 500 are selected; and so on. An example of a selectionof frames, named x(t), is shown in Figure 5.4(d), in which NRR is 612.The seventh step consists in the computation of the FFT from the signal x(t) (see

Figure 5.5(a)). The maximum peak is extracted and transformed to breaths per minute. Inthis example, the maximum peak found is 0.293 Hz. To obtain the value of the respiratoryrate (step 8), the peak frequency is multiplied per 60, as follows:

Respiratory Rate [breaths perminute] = Peakfrequency · 60. (5.1)

Steps 6, 7 and 8 are repeated every 10 seconds (bucle) until the end of the video. Thetemporal respiratory rate is shown in Figure 5.5(b).When the computation of the algorithm is completed, results are plotted in the GUI (seethe left side of Figure 5.5).

57


0 20 40 60 80 100 1200.5

0.52

0.54

0.56

0.58

0.6

0.62

0.64

Time [seconds]

Mean Value of the ROI

s (

t)

(a)

0 20 40 60 80 100 120-0.08

-0.06

-0.04

-0.02

0

0.02

0.04

Time [seconds]

Normalized signal

u (

t)

(b)

0 20 40 60 80 100 120-0.04

-0.03

-0.02

-0.01

0

0.01

0.02

0.03Filtered signal

Time [seconds]

p (

t)

(c)

0 5 10 15-0.03

-0.02

-0.01

0

0.01

0.02Signal selected for the actual computation

Time [seconds]

x (

t)

(d)

Figure 5.4: (a) Signal s(t), mean value of the pixels inside the ROI. (b) Signal u(t), afterapplying normalization to signal s(t). (c) Filtered signal p(t). (d) Selected numberof frames to compute the FFT.

0 0.5 1 1.50

0.005

0.01

0.015

X: 0.293Y: 0.01455

Frequency [Hz]

Fast Fourier Transform from x(t)

| X

( f )

|

(a)

0 20 40 60 80 100 1208

10

12

14

16

18

20

22

24Respiratory Rate

RR

[bre

ath

s p

er

min

ute

]

Time [seconds]

(b)

Figure 5.5: (a) Fast Fourier Transform of the signal x(t). (b) Temporal respiratory rate extrac-ted every 10 seconds by using Thermal Imaging.

58


Figure 5.6: GUI screen that visualizes all results, the ones obtained by infrared thermography(on the left) and the ones obtained as ground truth (on the right).

By introducing the name of the folder, in which SOMNOlab’s data is saved, the respiratoryrate recorded by the GT is also plotted (see the right side of Figure 5.6). Since the GT onlyprovides the effort signals (from thorax and abdomen), it is necessary to compute the FFTto extract the value of the breathing rate. As said before, NRR frames from the IR data wereused to compute the FFT every 10 seconds. Selecting NRR = 612, and having a samplingfrequency of 50 fps, this data correspond to 12.24 seconds. Since effort signals had a samplefrequency of 32 frames per second, to compute the FFT of the GT data were selected fewerframes than in IRT (32 frames/second ·12.24 seconds = 391GT frames). As a result, thecomputation of both respiratory rate vectors (IRT and GT) where temporally identical, anecessary fact to compute the CAND.When clicking on the “Compare Results” button, a new window is opened, in which bothvalues of the respiratory rate (the one extracted by IRT and the one extracted by the GT)are compared. This screen is shown in Figure 5.7. Three graphs are presented: (1) originaltemporal signals extracted by IRT and GT; (2) FFT of both signals and (3) CAND.

59


Figure 5.7: Screenshot of the graphical user interface (screen for the comparison between resultsobtained by IRT and GT).

5.1.2 GUI - Heart rate detection

When clicking on the “HEART RATE” button in the first screen (see Figure 5.1), theuser is redirected to a second screen, shown in Figure 5.8. It is divided in four parts, inwhich all steps defined in Figure 4.12 are computed: (1) selection of the infrared data (AVIvideo), (2) definition of the ROIs (steps 1 and 2), (3) computation of the algorithm (steps3 - 10), and (4) selection of the GT data.After selecting the AVI video under study, images are rotated using the Hough trans-

formation (shown in Figure 5.9) and the user is asked to select one ROI. Based on this,six regions of interest are automatically selected (all of them centered in the blood vessel).As example, is studied the ROI of 32x32 pixels, shown in Figure 5.9(b).By clicking on the “Run IRT algorithm” (see Figure 5.8), steps 3 to 10 are launched.In the previous phase a 3D-matrix 32x32x1000 is obtained. Its third dimension corres-ponds to the number of frames (40 seconds ·sampling frequency). Afterwards, an averagein the x-dimension is computed (third step), obtaining as a result a 2D-matrix. The 32signals (one for each y-position) are plotted in Figure 5.10(b). Normalization is appliedindependently to each y-signal (step 4) - see Figure 5.10(c).Subsequently each y-signal is filtered (step 5) as depicted in Figure 5.10(d).At every second of the recording video, the last 612 (M + NHR) recorded frames wereselected (sixth step): M = 100 corresponds to a margin for the adaptive estimation func-tion and NHR = 512 is used for the extraction of the heart rate. As example, the selected612 frames are plotted in Figure 5.11(a). In the seventh step periodification, which allowstrengthening the frequency of the signal, was applied, as illustrated in Figure 5.11(b).

60


Figure 5.8: Screenshot of the graphical user interface (screen for the computation of the heartrate algorithm).

(a) (b)

Figure 5.9: (a) Selection of the blood flow direction using Hough transformation. (b) Rotatedimage showing the ROI.

Thereafter, the FFT from each y-position of the signal p(t) was computed - step 8. Sincethe transformation to the frequency domain delete the phase between each y-position(phase created due to the blood flow), it is possible to compute the average of the 32spectra. The mean spectrum is shown in Figure 5.12.

61


0 10 20 30 400.22

0.225

0.23

0.235

0.24

0.245Mean Value of the 32 y-position signals

Time [seconds]

s (

t)

(a)

0 10 20 30 400.2395

0.2396

0.2397

0.2398

0.2399

0.24

0.2401Mean Value of a certain y-position signal

Time [seconds]

sy (

t)

(b)

0 10 20 30 40-2

-1

0

1

2

3

4x 10

-4

Normalized signal of a certain y-position

Time [seconds]

uy (

t)

(c)

0 10 20 30 40-1

-0.5

0

0.5

1x 10

-4

Filtered signal from a selected y-position

Time [seconds]

py (

t)

(d)

Figure 5.10: (a) Temperature profile in the ROI for each y-position. (b) Temperature profile fora selected y-position. (c) Normalized selected signal. (d) Filtered selected signal.

To take into account the past measurements, an adaptive estimation function was com-puted (step 9). This phase consists on the application of steps 6 and 7 to groups ofNHR = 512 frames (see Figure 4.18).In the same way that in the respiratory rate detection, heart rate was extracted from themaximum peak of the final spectrum - step 10:

HeartRate [beats perminute] = Peakfrequency · 60. (5.2)

Temporal heart rate extracted from the example using infrared thermography is shown inFigure 5.13(a).

By clicking on the “Select SOMNOlab data” button (see Figure 5.8), the GT data isintroduced in the workspace. An illustration of the heart rate extracted by SOMNOlab 2is shown in Figure 5.11(b). Unlike the respiratory rate, SOMNOlab 2 directly computesthe heart rate and, therefore, any post-processing is needed.Finally, by clicking “Compare”, a comparison between the results obtained by IRT andthe results obtained by the GT is plotted. It is shown in Figure 5.14.

62


0 5 10 15 20 25-1

-0.5

0

0.5

1x 10

-4 Selection of the last NHR

frames

Time [seconds]

py (

t)

(a)

0 10 20 30 40 50-1

-0.5

0

0.5

1x 10

-4 Periodic signal

Time [second]

py (

t)(b)

Figure 5.11: (a) Temperature profile for the selected NHR frames (NHR = 512). (b) Periodicextension of the temperature profile shown in (a).

0 0.5 1 1.5 2 2.50

1

2

3

4

5

6

7x 10

-5

X: 1.074Y: 6.144e-05

Frequency [Hz]

| X

(f)

|

Spectrum from NHR

frames

Figure 5.12: Mean power spectrum of the 512 last frames.

63


25 30 35 4050

60

70

80

90

100

110Heart Rate detected by Infrared Thermography

Time [seconds]

Heart

Rate

[beats

per

min

ute

]

(a)

25 30 35 4050

60

70

80

90

100

110Heart Rate detected by SOMNOlab 2

Time [seconds]

Heart

Rate

[beats

per

min

ute

]

(b)

Figure 5.13: (a) Heart rate extracted by using Thermal Imaging. (b) Heart rate extracted bySOMNOlab 2.

Figure 5.14: Results of the comparison. It presents (1) heart rate recorded by the IRT, (2)heart rate recorded by GT and (3) CAND.

64

5.2 Respiratory rate


As explained above, in this section are presented the results obtained with the respiratoryrate algorithm. Later, they are discussed based on the statistical analysis. To maintainthe anonymity of all subjects, who took the tests, they were coded as shown in Table 5.1.Characteristics such gender and age are also presented.

Table 5.1: Main information about the subjectsSubject Code Gender Age

01 Female 4102 Male 2303 Male 3004 Male 2605 Female 2506 Male 2007 Female 2708 Female 2309 Male 6310 Female 23

5.2.1 Experimental results

The complete results obtained by using the respiratory algorithm, have been attached inappendix A. They are presented in three tables, one for each distance between the subjectand the camera. Nevertheless, Table 5.2 presents a brief summary of the most importantvalues.

Only for investigation purposes, data was analyzed in four different ways. Step 3a wasmodified. The four cases are the following ones:

1. Transformation from RGB to grey, as explained in page 37.

2. Transformation from RGB to R. This is, selecting only the red dimension and deletingthe information contained in the other two dimensions.

3. Transformation from RGB to G. This is, selecting only the green dimension anddeleting the information contained in the other two dimensions.

4. Transformation from RGB to B. This is, selecting only the blue dimension anddeleting the information contained in the other two dimensions.

Consequently, the algorithm was run four times in each test. Tables in “Appendix A”present the complete results.

65


Table 5.2: Summary of the results. The respiratory rate (RR) is measured in breaths per minuteSubject Code Region Of Interest (ROI)

d=50 cm d=75 cm d=100 cmGT mean RR IR mean RR GT mean RR IR mean RR GT mean RR IR mean RR

01Normal region 15,34 15,75 15,34 19,22 14,66 14,51Big region 15,34 15,76 15,34 13,35 14,66 13,45One nostril 15,34 15,75 15,34 15,84 14,66 13,27

02Normal region 16,36 22,24 13,30 14,98 12,61 17,80One nostril 16,36 16,11 13,30 12,39 12,61 17,53Big region 16,36 16,47 13,30 13,88 12,61 13,02

03Normal region 12,27 12,02 16,88 14,95 No measures were performedBig region 12,27 12,02 16,88 12,79 No measures were performedOne nostril 12,27 12,04 16,88 19,43 No measures were performed






09Normal region 23,86 23,55 25,23 24,72 25,57 21,44Big region 23,86 21,61 25,23 24,74 25,57 17,44One nostril No measures were performed 25,23 24,74 25,57 24,38


In the following subsection these results are analyzed and discussed, but taking a fastview of them can be observed that smaller distances produce the best results than longerones. When the distance “d” is 50cm, results obtained by using IRT are more similar tothe ones obtained by the ground truth (GT) than when d=75cm or d=100cm.Another point to remark is the respiratory rate obtained from subject 09. This patient,whose age is 63 years, had an irregular respiration and performed constant movementswith the head. Figure 5.15, shows the signal recorded by the camera and it is possible toobserve the noise that incorporates. Nevertheless, results were satisfactory in all tests.Although only the average of the respiratory rate has been presented in this section,

the following statistics are based on the analysis of the CAND. This index provides thebest description of how right or wrong has been obtained the vital sign, since it is definedas a weighted value (normalized against the GT). For more information, “Appendix A”includes detailed tables including its value for each test and subject.

66


40 60 80 100 120-0.05

-0.04

-0.03

-0.02

-0.01

0

0.01

0.02

0.03Infrared signal from subject 09

Time [second]

Py(t

)

Figure 5.15: Infrared thermal signal of the nostrils of subject 09.

5.2.2 Statistics and discussion

The study of results from the respiratory rate have been focused on four variables:

1. dimension of data under study: grey (Gr), red (R), green (G) or blue (B);

2. distance between the camera and the subject, which can be 50 cm, 75 cm or 100 cm;

3. region of interest, selected as normal region (N), big region (B) or one nostril (O);

4. gender of the subject: male (M) or female (F).

67


Effects of the dimension of data under study

A first study of CAND was done by comparing results obtained when applying therespiratory rate algorithm in only one data dimension and also distinguishing the threedistances between the subject and the camera.

20

30

40

50

60

70

80

90

1 2 3 4Dimensions

(1=grey, 2=red, 3=green, 4=blue)

d= 50 cm

CA

ND

(a)

0

20

40

60

80

100

1 2 3 4Dimension


d=75 cm

CA

ND

(b)

0

20

40

60

80

100

1 2 3 4Dimension


d=100cm

CA

ND

(c)

Figure 5.16: Boxplot of CAND obtained according to the dimension of data (Gr, R, B, G). (a)Distance of 50 cm. (b) Distance of 75 cm. (c) Distance of 100 cm.

Figure 5.16 demonstrates the CAND values for the four dimensions (Gr, R, B and G). Theresults show that the red dimension provided the most accurate results for all distance.Due to this, and in order to simplify the statistical analysis, just this dimension will beused in the following sections.

68


Effects of the distance between the camera and the subject

A second study was based in the observation distance’s effect between camera and subject.It was known that the closer to the subject, the better results. Results also confirm thisaffirmation. Figure 5.17, presents a boxplot for each ROI in which the three distances arecompared. Is clearly visible that better results are achieved for a distance of 50 cm, sincemedian is higher and the three lower adjacent are higher than 85%. When distance isincreased, medians decrease slowly, and whistles get longer, which produce big instabilityin the results.

0

20

40

60

80

100

1 2 3Distance

(1=50cm, 2=75cm, 3=100cm)

Normal region

CA

ND

(a)

0

20

40

60

80

100

1 2 3Distance

(1=50cm, 2=75cm, 3=100cm)

Big region

CA

ND

(b)

0

20

40

60

80

100

1 2 3Distance

(1=50cm, 2=75cm, 3=100cm)

One nostril

CA

ND

(c)

Figure 5.17: Boxplot of CAND obtained according to the distance between the subject and thecamera (50cm, 75cm, 100cm). (a) Normal region. (b) Big region. (c) One nostril.

69


Effects of the ROI

The three ROI were compared according to the distance “d”. As shown in Figure 5.18,results do not have a remarkable pattern and it is not possible to assure whether one regionis better than the other one. Nevertheless, subjects that were moving or talking during therecording had better results when the big ROI was selected. On the other hand, subjects,who remained quiet and still, had better results when the normal region was computed.This can seem contradictory when thinking about the tracking. Although the trackingdesigned in this thesis maintains both nostrils inside the ROI, it does not take care aboutthe pixels that are included in this ROI, but are not part of the nostrils. These pixelsalso compute in the mean (step 3) of the ROI, but they include only noise. The more thesubject moves the nose, the more the mean value varies.

0

20

40

60

80

100d=50cm

1 2 3Region of interest(1=N, 2=B, 3=O)

CA

ND

(a)

0

20

40

60

80

100


d=75cm

CA

ND

(b)

0

20

40

60

80

100d=100cm


CA

ND

(c)

Figure 5.18: Boxplot of CAND obtained according to the region of interest (normal region, bigregion, one nostril). (a) Distance of 50 cm. (b) Distance of 75 cm. (c) Distance of100 cm.

70

5.3 Heart rate

Plays gender a role in IR detection?

Comparing results taking in account the gender of the subject, it can be observed that thesuccess of the algorithm was similar in both cases, male and female. In Figure 5.19 can beobserved that both boxplots are quite equal. Also observing the recordings from the IRcamera, it was clear that thermal imprints extracted from men were as clear as the onesextracted from women.

0

20

40

60

80

100

1

Male

CA

ND

(a)

0

20

40

60

80

100

1

Female

CA

ND

(b)

Figure 5.19: Boxplot of CAND obtained according to the gender of the subjects. (a) Male. (b)Female.

5.3 Heart rate

Like in the respiratory rate section, this section also provides the results obtained by thealgorithm, in this case, the heart rate detection algorithm. Afterwards, these are discussedbased on statistical analysis.To maintain the anonymity of all subjects, who took the tests, they were coded as shownin Table 5.3. Characteristics such gender and age are also presented.

Table 5.3: Main information about the subjectsSubject Code Gender Age

03 Male 3004 Male 2605 Female 2510 Female 2311 Female 23

71


5.3.1 Experimental results

Results obtained with the heart rate algorithm are presented in Tables 5.4 and 5.5. Inthis case, the attention has been settled to the region of interest, which plays the mostimportant role for the data processing. A wrong definition of the region of interest assuresincorrect results. Appendix A presents all results obtained in the tests, including also theones obtained when choosing NHR as 256 or M as 100.

Table 5.4: Summary of the results. The respiratory rate (RR) is measured in breaths per minute

Subject Code Region Of Interest (ROI)NHR = 512, M = 50

GT mean HR IRT mean HR

03

ROI 1 (8x8) 57,00 53,75ROI 2 (16x16) 57,00 55,81ROI 3 (32x32) 57,00 60,89ROI 4 (64x64) 57,00 57,53ROI 5 (128x128) 57,00 56,07ROI 6 (16x32) 57,00 60,46

03


04


04


05


72

5.3 Heart rate

Table 5.5: Summary of the results. The respiratory rate (RR) is measured in breaths per minute

Subject Code Region Of Interest (ROI)NHR = 512, M = 50

GT mean HR IRT mean HR

05


10


10


11


11


73


5.3.2 Statistics and discussion

The procedures applied for measuring the heart rate of the patients were similar thanthose used to measure the respiratory rate. However, the following differences shouldbe pointed out. Unlike respiratory data, heart data was processed without using IRBIS3 Professional. Therefore, a study of the dimensions was not necessary, since data wasnot in RGB format. Nevertheless, the region of interest had a more important role. Dueto the narrowness of the carotid and the weakness of the thermal signal, it was crucialto focus the processing on the correct data. In addition, this section also includes theselection of the best NHR and M values (see chapter 4) to achieve optimal results as wellas a comparison between results obtained in men and women.

Selection of NHR and M

The number of frames to compute the heart rate, NHR, and the number of past framestaken into account to compute the adaptive estimation function, M , were selected afterobserving the results. Based on past research and other investigations, it was decided totest each variable for two different values: NHR = 256 and NHR = 512 as well as M = 50and M = 100. Figure 5.20 shows the results obtained with the application of the heartrate algorithm for each case (NHR = 256 and NHR = 512 as well asM = 50 andM = 100).By analyzing this figure, we can conclude that NHR = 512 generated better results thanNHR = 256, since whistles were shorter and the median was higher.

Nevertheless, no significant difference was found between both Ms. Their most importantstatistical values are shown in Table 5.6. As it is possible to observe, all values are quitesimilar in both cases, which means that computing the adaptive estimation function with50 more frames does not add important information.

Table 5.6: Properties of boxplotsM 50 100

Upper adjacent 98,17 98,2475th percentile 96,13 95,57

Median 90,12 89,9025th percentile 82,97 83,62Lower adjacent 68,68 70,80

Minimum 16,12 17,71

74

5.3 Heart rate

0

20

40

60

80

100

1 2 3 4

Complete HR results

CA

ND

Figure 5.20: Boxplot of CAND obtained according to different values of M and NHR. (1)M = 50, NHR = 256. (2) M = 50, NHR = 512. (3) M = 100, NHR = 256. (4)M = 100, NHR = 512.

But an important factor that should be taken into consideration is the time-consume.Undoubtedly, results obtained withM = 50 were computed faster, since less mathematicaloperations were necessary. Therefore, M was definitely chosen to be 50.For the following statistical analysis just results with NHR = 512 and M = 50 were used.This is due to their better performance. Once more is going to be analyzed (1) the effectsof the region of interest and (2) the role of gender in heart rate detection.

Effects of the region of interest

As explained in Chapter 4, six regions of interest were selected according to theirdimensions (1) 8x8 pixels, (2) 16x16 pixels, (3) 32x32 pixels, (4) 64x64 pixels, (5) 128x128pixels and (6) 16x32 pixels1.Figure 5.21 and Table 5.7 show the most important properties of CAND for each of them.As is it shown, worst results were obtained in those ROI with a bigger area. This canbe explained by the selection and computation of pixels that were too far away from thecenter of the carotid. These pixels did not have the thermal signal of the artery and were,unfortunately, computed as noise. Similarly, the smallest ROI (8x8 pixels) did not presentalso the best performance. In contrast, ROI 16x16 and ROI 16x32 demonstrated to havethe best performances (shown in Figure 5.7 and Table A.4 in appendix). Nevertheless,based on Table 5.7 it was concluded that the best results were obtained by using ROI 16x32.

1(width x length) pixels

75


0

20

40

60

80

100

1 2 3 4 5 6Region of Interest

NHR

=512, M=50

CA

ND

Figure 5.21: Boxplot of CAND obtained according to the region of interest.

Table 5.7: Properties of boxplots of each region of interestROI 1 2 3 4 5 6

Upper adjacent 98,15 97,98 97,32 97,18 97,84 98,1775th percentile 95,78 94,37 96,59 96,17 96,71 96,09

Median 87,96 91,77 91,68 90,45 88,99 91,7725th percentile 83,66 83,29 83,03 73,91 81,48 83,8Lower adjacent 73,3 75,92 69,95 42,39 68,68 75,61

Minimum 73,3 75,92 16,12 42,39 56,33 46,26

Role of gender in heart rate detection

It is important to add a brief comparison between the results obtained in women andin men. Figure 5.22 presents boxplots of the CAND from both genders. Male boxplothas a median of 84.324, while female’s median is 93.195, a value considerably higher.Furthermore, the difference between both lower adjacent is enormous. While in men itsvalue is 42.39, for women is 73.91.This means that it was easier to detect the carotid artery in females than in males. This isdue to the difference of the diameter of common carotid arteries between women (6.5 mm)and men (6.1 mm) (see chapter 2). As women have a thicker artery, a bigger number of“warm” pixels were detected. Moreover, according to “The International Dermal Institut”,men’s skin is on average 25% thicker than women’s, fact that reduces the temperatureconduction from the vessel to the skin, and produces a low variability of the temperaturedetected by the camera.

76

5.3 Heart rate

0

20

40

60

80

100

1

Female

CA

ND

(a)

0

20

40

60

80

100

1

Male

CA

ND

(b)

Figure 5.22: Boxplot of CAND obtained according to the gender of the subject. (a) Male. (b)Female.

77

6 Conclusions and future work

This chapter resumes the main conclusions of the investigation realized in this thesis, aimedto fulfil the objectives presented in the introduction. Additionally, it provides possibleapplications in hospitals and medical clinics as well as future perspectives.

6.1 Conclusions

Thermal prints from respiration and pulse, were successfully detected by using InfraredImagery. The algorithm proposed by Garbey et al. in “Contact-free measurement ofCardiac Pulse based on the Analysis of Thermal Imagery” [12] was developed successfullyand, consequently, both vital signals, respiratory rate and heart rate, were accuratelyextracted.Comparing both thermal signals, the temperature variation due to respiration can bebetter detected than the one caused by the pulse, since it can achieve values of ±0.5◦C.For this reason, a higher level of data processing is required to extract the heart rate.Regarding respiratory rate, it was demonstrated that results obtained from both, male andfemale recordings are similarly satisfactory. Nevertheless, in the case of the pulse detectionis easier to detect it in women than in men. The principal reason was the thickness of theskin and the width of the carotid artery.Regarding the tests, the best detection for the respiratory rate was obtained when thedistance between the camera and the subject was shorter and the subject remained still.On the other hand, for the detection of the heart rate, the most important and difficultstep was the selection of the optimal region of interest. Best results were obtained usinga rectangular region of interest of 32 pixels along the direction of the vessel and 16 pixelsalong the perpendicular one.In summary, the objectives of the thesis, proposed in the introduction, were successfullyachieved. A novel method to detect both vital signs, an improvement of the quality ofnew-borns in incubators due to its contact-free characteristic, was developed. This methodwas optimized in all its variables (selection of the data-dimension under study, number offrames necessary to compute the vital sign, distance between the camera and the subject,etc.) in order to obtain the best results.

79

6 Conclusions and future work

6.2 Possible applications

As it has been introduced in the first chapter, the most important application for thealgorithm developed in this thesis is the detection and control of vital signs of new-borninfants. The distinctive and most important advantage that this method offers, incomparison with the actual ones, is the non-contact characteristic, which avoids thenecessity for electrodes. This is positive for the baby, because there is no need to useelectrodes, which are harmful for their skin and cause them stress. It also reduces thenecessity to open the incubator, fact that can provoke a reduction of the temperature orthe introduction of viruses.A second possible application would be a sleep-control for truck and auto drivers, speciallythe ones that drive during the night or for long distances. Since respiratory rate decreasesduring sleep, a constant detection of the respiratory rate with a small infrared camerawould detect the phases when the driver feels sleepy. With the help of a warning, it wouldbe possible to awake the driver before having an accident. This can be also applied in tosecurity guards, who stay in a fixed position, during night surveillance.Another possible application is the detection of pregnancy. Minute ventilation increasesabout 50% during pregnancy. This is due to the demand for oxygen to supply the fetusand the increase of the sensitivity of the respiratory chemoreceptors to carbon dioxidecaused by progesterone.

6.3 Future perspectives

The algorithm developed in this thesis is the basis for detection of respiratory and heartrates. It can be adapted and used as the kernel of any further application. There are,however, some points to expand the research and continue the development. The mostimportant ones are described as follows:

1. Nostrils and common carotid arteries were selected as the body components understudy. Nevertheless, thermal signal containing the respiratory and heart rates canbe found in other parts of the body, such as the face, especially the forehead, or bothwrists. An interesting work could be the detection of the heart rate from them.

2. Focusing on the respiratory rate, it can be developed a better tracker algorithm,which, instead of selecting a rectangular ROI, detects the curvature of the nostrilsand select only those pixels inside them.

3. Furthermore, to improve the quality of the results in longer distances between thesubject and the camera, it can be tested the response of the algorithm without usingIRBIS 3 Professional as a pre-processing step, but applying directly the methodology

80

6.3 Future perspectives

used in the detection of the heart rate.

4. Now, focusing on the heart rate, a more accurate methodology to select the ROI canbe researched, since this selection is extremely important to obtain optimal results.

5. At last, for both algorithms, can be designed a more sophisticated Graphical UserInterface.

81

A Appendix 1

A.1 Detailed tables of the results obtained in tests

A.1.1 Respiratory rate

The following tables present the complete results obtained in the detection of the respir-atory rate: in Table A.1 are detailed the results obtained when the distance d was 50 cm;in Table A.2, the results obtained when d=75cm; and lastly, in Table A.3 the results whend=100cm.The respiratory rate (*) is given in breaths per minute, while CAND (**) is given inpercentage.

Table A.1: Results respiratory rate with a distance of 50 cm between the subject and the camerad = 50 cm

Subject Code Region Of Interest (ROI) GT mean RR*Grey Red Green Blue

IR mean RR* Mean CAND** IR mean RR* Mean CAND** IR mean RR* Mean CAND** IR mean RR* Mean CAND**

01Normal region 15,34 15,75 93,37 15,75 93,37 17,27 83,05 15,67 92,81Big region 15,34 15,76 93,32 15,75 93,37 17,00 83,78 15,58 92,39One nostril 15,34 15,75 93,37 15,75 93,37 17,08 83,46 15,59 92,34

02Normal region 16,36 22,24 23,01 15,84 64,58 13,81 73,49 13,80 80,42One nostril 16,36 16,11 68,80 15,93 64,86 14,09 76,48 13,89 78,44Big region 16,36 16,47 63,06 15,75 65,38 14,25 78,59 13,72 80,84







09Normal region 23,86 23,55 95,98 23,65 95,49 23,55 95,44 23,55 95,98Big region 23,86 21,61 88,73 23,55 95,44 23,65 95,49 23,56 95,47


83

A Appendix 1

Table A.2: Results respiratory rate with a distance of 75 cm between the subject and the camerad=75 cm













Table A.3: Results respiratory rate with a distance of 100 cm between the subject and thecamera

d=100 cm












84

A.2 Heart rate

A.2 Heart rate

The following tables present the complete results obtained in the detection of the heartrate. In this case, two tables are presented. The first one corresponds to results obtainedwhen M=50, and the second table correspond to results obtained when M=100.The heart rate (*) is given in beats per minute, while CAND (**) is given in percentage.

85

A Appendix 1

Table A.4: Results heart rate choosing M as 50M=50

Subject Code Region Of Interest (ROI)N_HR = 256 N_HR=512

GT mean HR* IRT mean HR* Mean CAND** GT mean HR* IRT mean HR* Mean CAND**

03

ROI 1 (8x8) 57,00 61,39 85,50 57,00 53,75 90,06ROI 2 (16x16) 57,00 63,66 84,60 57,00 55,81 91,25ROI 3 (32x32) 57,00 62,91 88,07 57,00 60,89 93,18ROI 4 (64x64) 57,00 62,47 84,59 57,00 57,53 86,12ROI 5 (128x128) 57,00 66,90 80,68 57,00 56,07 89,29ROI 6 (16x32) 57,00 64,53 83,69 57,00 60,46 92,73

03


04


04


05


05


10


10


11


11


86

A.2 Heart rate

Table A.5: Results heart rate choosing M as 100M=100

Subject Code Region Of Interest (ROI)N_HR = 256 N_HR=512

GT mean HR* IRT mean HR* Mean CAND** GT mean HR* IRT mean HR* Mean CAND**

03


03


04


04


05


05


10


10


11


11


87

Bibliography

[1] Abbas, A. K., Heimann, K., Jergus, K., Orlikowsky, T., and Leonhardt,S. Neonatal non-contact respiratory monitoring based on real-time infrared thermo-graphy. 93.

[2] Baker, S., and Matthews, I. Lucas-kanade 20 years on: A unifying framework.221-255.

[3] Ballard, D. H. Generalizing the hough transform to detect arbitrary shapes. 111–122.

[4] Bronzino, J. D. Medical Devices and Systems. CRC Press.

[5] Chekmenev, S., Farag, A., and Essock, E. Multiresolution approach for non-contact measurements of arterial pulse using thermal imaging. In Conference onComputer Vision and Pattern Recognition Workshop, 2006. CVPRW ’06, pp. 129–129.

[6] Chen, W., Oetomo, S. B., and Feijs, L. Neonatal Monitoring Technologies:Design for Integrated Solutions. IGI Global.

[7] Clark, R. P., and Stothers, J. K. Neonatal skin temperature distribution usinginfra-red colour thermography. 323–333. PMID: 7411454.

[8] Diakides, N. A., and Bronzino, J. D. Medical Infrared Imaging. CRC Press.

[9] Duda, R. O., and Hart, P. E. Use of the hough transformation to detect linesand curves in pictures. 11–15.

[10] Fei, J., and Pavlidis, I. Thermistor at a distance: Unobtrusive measurement ofbreathing. 988–998.

[11] Ford, S., and Calvert, J. Adaptation for life: a review of neonatal physiology.93–98.

[12] Garbey, M., Sun, N., Merla, A., and Pavlidis, I. Contact-free measurementof cardiac pulse based on the analysis of thermal imagery. 1418–1426.

[13] GmbH, I. Einfuehrung in Theorie und Praxis der Infrarot-Thermografie.

89

Bibliography

[14] Greneker III, E. F. Radar sensing of heartbeat and respiration at a distance withsecurity applications. vol. 3066, pp. 22–27.

[15] Hall, J. E. Guyton and Hall Textbook of Medical Physiology: Enhanced E-book.Elsevier Health Sciences.

[16] Kramme, R. Medizintechnik Verfahren - Systeme - Informationsverarbeitung.Springer.

[17] Krejza, J., Arkuszewski, M., Kasner, S. E., Weigele, J., Ustymowicz, A.,Hurst, R. W., Cucchiara, B. L., and Messe, S. R. Carotid artery diameter inmen and women and the relation to body and neck size. 1103–1105. PMID: 16497983.

[18] Laizzo, P. Handbook of Cardiac Anatomy, Physiology, and Devices.

[19] Lawson, R. N., and Chughtai, M. S. Breast cancer and body temperature.68–70. PMID: 20327365 PMCID: PMC1920940.

[20] Lucas, B. D., and Kanade, T. An iterative image registration technique with anapplication to stereo vision. In Proceedings of the 7th International Joint Conferenceon Artificial Intelligence - Volume 2, IJCAI’81, Morgan Kaufmann Publishers Inc.,pp. 674–679.

[21] Murthy, R., Pavlidis, I., and Tsiamyrtzis, P. Touchless monitoring of breath-ing function. 1196–1199. PMID: 17271901.

[22] Pavlidis, I., Levine, J., and Baukol, P. Thermal imaging for anxiety detec-tion. In Proceedings of the IEEE Workshop on Computer Vision Beyond the VisibleSpectrum: Methods and Applications (CVBVS 2000), CVBVS ’00, IEEE ComputerSociety, p. 104.

[23] Polin, R. A. Fetal and neonatal physiology. 2 2. Saunders.

[24] Saladin, K. S. Anatomy & Physiology: The Unity of Form and Function., 3rd ed.The McGraw-Hill Companies.

[25] Shi, J., and Tomasi, C. Good features to track. In , 1994 IEEE Computer SocietyConference on Computer Vision and Pattern Recognition, 1994. Proceedings CVPR’94, pp. 593–600.

[26] Silverthorn, D. U. Human physiology: an integrated approach. Pearson/BenjaminCummings.

90

Bibliography

[27] Sun, N., Garbey, M., Merla, A., and Pavlidis, I. Imaging the cardiovascu-lar pulse. In IEEE Computer Society Conference on Computer Vision and PatternRecognition, 2005. CVPR 2005, vol. 2, pp. 416–421 vol. 2.

[28] Szeliski, R. Computer Vision: Algorithms and Applications. Springer.

[29] Tomasi, C., and Kanade, T. Detection and tracking of point features.

[30] Weinmann. SOMNOlab 2. Description of device and instructions for use.

[31] Whittemore, S. The circulatory system. Chelsea House.

[32] Yazici, B., Erdogmus, B., and Tugay, A. Cerebral blood flow measurementsof the extracranial carotid and vertebral arteries with doppler ultrasonography inhealthy adults. 195–198. PMID: 16320223.

91

chantal vilà calopa contact-freemeasurementofcardiac and

Documents