principles and design of iot systems level 11 course [20 … introduction to the platform and...

125
Principles and Design of IoT systems Level 11 course [20 credits] Lecture 1 Professor D K Arvind dka AT inf.ed.ac.uk

Upload: vandat

Post on 25-Apr-2018

220 views

Category:

Documents


4 download

TRANSCRIPT

Page 1: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

Principles and Design of IoT systems

Level 11 course [20 credits] Lecture 1

Professor D K Arvind

dka AT inf.ed.ac.uk

Page 2: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

2

Course administration

• Lecturer: Professor DK Arvind (dka AT inf.ed.ac.uk)

• Tutor: Andrew Bates (cxb AT inf.ed.ac.uk)

• Lectures: Week 1 – 5 only

» 10:00 – 10:50 Tuesdays in Room LG.11, David Hume Tower

» 10:00 – 10:50 Fridays in Room G.8, Gaddum Lecture Theatre, 1 George Square

• Weekly Tutorials: Week 1 - 11 in Room AT 3.02

Choose a slot at 11:00, 12:00, or 13:00 on Tuesdays

• Course web page: http://www.inf.ed.ac.uk/teaching/courses/pdiot/

Page 3: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

3

Assessment

• 30% marks based on written examination

– Final exam in April/May 2018

• 70% marks based on coursework

• Coursework

Issued on: 19 Sep. 2017

5-minute Presentation/Demonstration: 28 Nov. 2017 (11:00 – 14:00)

Coursework individual report due: 19 Jan. 2018 (16:00)

Feedback and marks: 03 Feb. 2018 (16:00)

Page 4: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

4

Topics covered in the lectures

• Overview of IoT: industrial, wearable, environmental,

healthcare, and digital media, illustrated with videos of case

studies; architecture of a typical IoT system and its

components; Overview of privacy and security issues;

Sensors and actuators – Introduction to commonly-used

sensors/actuators, modes of operation, and data format.

• Sensor fusion algorithms: data from homogeneous and

heterogeneous sensors; examples include quaternion

calculations with 6-DOF IMU sensor for 3-D animation.

• Sensor data analytics: calibration of sensor data against

reference sensors; use of Bland-Altman plots for sensor

data comparisons, illustrated with examples in healthcare

and environmental monitoring

.

Page 5: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

5

Topics covered in the lectures (Contd.)

• Sensor data analytics: Methods for pre-processing

and feature extraction methods in time-series

sensor data - Time warp algorithm; Principal

component analysis; feature extraction methods

illustrated with examples of IoT for wellbeing,

musical instrument tutoring.

• Sensor data analytics: Classification methods

using Machine Learning/Hidden Markov models

(Naïve Bayes, k-NN, Decision Tree, Logistic

Regression, Mulitlayer Perceptron, SVM) applied to

features in time-series sensor data, illustrated with

examples in sports, healthcare.

Page 6: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

6

Assessment

• 30% marks based on written examination

– Final exam in April/May 2018

• 70% marks based on coursework

• Coursework

Issued on: 19 Sep. 2017

5-minute Presentation/Demonstration: 28 Nov. 2017 (11:00 – 14:00)

Coursework report due: 19 Jan. 2018 (16:00)

Feedback and marks: 03 Feb. 2018 (16:00)

Page 7: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

7

Coursework

• Work in pairs to design, implement and demonstrate an IoT application

based on wireless sensors in 11 weeks

• Given : a NRF52-DK development board, MPU-9250 Motion Tracking

Board, on-line software development environment – the ARM Mbed

compiler

• Task : Implement a Step Tracker using the wearable sensor which

interfaces to an Android App over Bluetooth LE (BLE) and demonstrate a

working prototype

• Organisation:

» Room 3.11 (PDIoT Base) in Appleton Tower (with lockers provided for the safe-keeping of the boards) is reserved for the exclusive use of PDIoT students #

» Weekly tutorials will be held in Room 3.02 in Appleton Tower at 11:00, 12:00 and 13:00. Student pairs should sign up for one of the three hourly slots and are required to attend the weekly tutorials.

Page 8: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

8

Coursework Schedule

Discover -

• Week 1 First meeting of the PDIoT students; Explanation of the

assignment; Introduction to the platform and programming

environment; Demonstration of an end-to-end IoT system as an

exemplar.

Define -

• Week 2 Capture the requirements and use cases for the application;

Assignment of responsibilities; Tutorial on programming the platform;

Develop –

• Week 3 - 5 Implementation of the reference design on the Mbed

board. Development of the Step Tracker and its testing, Definition of

metrics for performance assessment and weekly review of progress

Page 9: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

9

Coursework Schedule (Contd.)

Develop (Contd.) -

• Week 6 First system integration and demonstration to course Tutor of the

Step Tracker on a level ground [Feedback to the students]

• Week 7 – 8 [Extra Credit] Integration of atmospheric pressure sensor;

refinement and testing of the Step Tracker when climbing stairs

• Week 9 - 10 Second system integration and presentation to course Tutor;

Performance analysis; Preparation of the presentation and final

demonstration [Feedback to the students]

Deliver -

• Week 11 (28 Nov. 2017) 5-minute presentation and demonstration by each

pair to the Course Lecturer [Feedback to the students]

Submission: The final individual reports are due on 19th January, 2018

Page 10: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

10

Coursework Assessment

[Technical evaluation - 60%]: Completion of the project; degree of

difficulty; quality and amount of work; justification of design decisions;

design for reusability.

[Presentation - 20%]: Quality of the oral presentation and written report,

and literature review.

[Analysis - 20%]: Critical analysis using quantitative methods and reflection

on design decisions.

Page 11: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

11

Nordic NRF52-DK board with the IMU breakout board

Page 12: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

12

IDE for IoT system development

• ARM-processor based Nordic NRF52-DK board

• InvenSense MPU-9250 Motion Tracking breakout board : 3-axis

accelerometer, gyroscope and magnetometer – these sensors can be

used together to provide full 3-D motion capture

• mbed online IDE and compiler for your firmware development to run on

the NEF52-DK board

• You will be developing an android application to communicate with the

NRF52-DK over BLE

• You are given a Reference Design which does the following:

Flashes an LED on the dev board

Sends debugging information to the PC over serial

Communicates with the IMU breakout board

Streams sensor data over BLE

Page 13: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

13

On completion of this course, the student will have:

1. An understanding of the constituent parts of a

typical IoT system, the operation of a selection of

sensors and actuators, and an appreciation of

methods employed to address the security and

privacy issues in IoT.

2. Knowledge of a selection of sensor fusion

algorithms, and data analytic methods for pre-

processing of time-series sensor data, feature

extraction and its classification, and illustrated with

case studies.

.

Page 14: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

14

On completion of this course, the student will have:

3. Gained expertise in the end-to-end design of a practical

IoT system employing the principles taught in the lectures,

and the quantitative evaluation of performance in terms of

speed, memory usage and power consumption.

4. Experience working with another team member with

complimentary skill sets, and develop skills in requirements

capture, user interface design, project management,

negotiations and verbal and written presentations.

5. Experience using tools such as compilers for IoT

development board using inertial sensors.

.

Page 15: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

15

Page 16: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

16

Personal

Social

Public

Sense – Learn -- Act

Communication

Computation “Intelligence”

Integration of Computation, Communication and Control to provide time-bounded decisions and actions

Page 17: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

17

Page 18: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

18

Page 19: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

19

Example of an IoT system for

Environmental Monitoring

Page 20: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

20

Digital Temperature Sensors

• Measures the temperature as an electrical signal

– Thermocouple contains two dissimilar metals

– Thermistor – resistance changes with temperature

Page 21: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

21

Thermocouple-based Temperature Sensors

• Two different metal alloys

• The Seebeck effect - temperature difference between two

dissimilar electrical conductors or semiconductors produces

a voltage difference between the two substances

• Large temperature range : –250°C to 300,000°C

Page 22: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

22

Resistance-based Temperature Sensors

• Thermistor resistance is dependant on the temperature

– Negative Temperature Coefficient (NTC) – resistance decreases as

temperature rises

– Positive Temperature Coefficient (PTC) – resistance increases as

temperature rises

• Thermistor uses ceramic and polymer material and useful over larger

temperature ranges

• Resistance Temperature Detectors (RTD) uses pure metal and is

accurate over a smaller temp range

Page 23: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

23

Relative Humidity Sensors (hygrometer)

• Sensor measures the moisture and temperature in the

air and reports rH as a percentage of the ratio of

moisture in the air to the max amount that can be

held in the air at that temperature.

• 3-types : Capacitive, Resistive, Thermal

• Capacitive – Thin strip of metal oxide placed between two

electrodes changes its electrical capacitance with the

atmosphere’s relative humidity

• Resistive – changes in the resistance of the electrodes on

either side of the salt medium as the humidity changes

• Thermal – two thermal sensors (encased in dry Nitrogen)

and the other exposed to the air, conduct electricity

proportional to the ambient humidity

Page 24: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

24

Comparison of Relative Humidity Sensors

Page 25: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

25

Airspeck-Stationary for environmental monitoring

• Solar panel provides power for

battery recharge and also shade

to reduce enclosure heating

• Sheet metal frame provides

structure and environmental

protection

• Inlet cover protects against rain

while allowing free air flow

• Air outlet at base of enclosure is

protected against rain spray

• PM 10/2.5, T/rH, NO2/O3, GPS

• Wireless Comms: 3/4G Modem,

Bluetooth

• SD card storage

• Key switch

Page 26: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

26

Gas sensor shroud and fan

Solar panel

(provides shade from sun)

Sheet metal frame and shroud

Alphasense OPC

GPS module

QoE PCB

Airflow outlets

Page 27: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

27

Principal components of hardware platform

Page 28: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

28

Firmware for the Airspeck-S

Page 29: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

29

Data Collection Architecture

Page 30: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

30

You’ve got the whole world in your hands …..

Page 31: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

31

Page 32: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

32

Solar cell charging: Radiation levels v/s Battery

voltages for 5 Airspecks

Page 33: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

33

3-D Motion Capture : Case Study

in Data Fusion

Page 34: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

34

Page 35: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

35

• Capture, Analyse and Understand Motion

• Fully wireless

• No infrastructure (i.e. camera(s))

• Real-Time and Interactive

• Easy to use

• Banalise the technology

• Democratise its usage

Requirements for IoT-based motion capture using

IMU sensors

Page 36: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

36

Orient-3 (2008) Orient-4 (2012)

Orient-5 (2017)

Page 37: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

37

Page 38: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

38

Page 39: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

39

• 2 radios: Bluetooth (2.5KB/s) and 2.4GHz (50KB/s)

• Gyroscope (8KHz)

• Accelerometer (4KHz)

• Quaternions (200Hz) using BT

• Quaternions (2500Hz) using 2.4 GHz

• Wireless charging

Orient-5

Page 40: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

40

Video at www.specknet.uk

Page 41: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

41

Music from Motion

•Category : Artistic

•Requirements: Wearable, unobtrusive, free

movement, 3-D motion in real-time, instantaneous

sound response (msec.)

•Users: Dancer/choreographer; the audience as part of

a theatre performance

•Sensors: 3-D Acc’meter, gyros, magnetometer

•Actuators: sound sequences mediated by movement

•Data Analysis: Calculate orientation in 3-D space from

sensor data; Sensor data: 512Hz, Quaternion angles:

256Hz

•Wireless protocol: 2.4GHz Zigbee radio: 150kbits/s;

TDMA protocol: 128Hz update rate with 12 devices;

64 Hz update rate with 24 devices.

Page 42: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

42

Video at www.specknet.uk

Page 43: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

43

Page 44: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

44

University Residential Centre of Berti

Website of venue: http://www.ceub.it/inglese/dove_siamo.htm

Page 45: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

45

Page 46: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

46

University Residential Centre of Berti

Website of venue: http://www.ceub.it/inglese/dove_siamo.htm

Page 47: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

47

Page 48: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

48

Page 49: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

49

Page 50: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

50

Page 51: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

51

Page 52: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

52

Page 53: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

53

Page 54: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

54

Page 55: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

55

Robots learning by imitation

Page 56: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

56

Arvind, Valtazanos “Speckled Tango Dancers: Real-time Motion Capture of two-body Interactions using on-body Wireless Sensor Networks“, 6th Int. Conf. on Wearable and Implantable Body Sensor Networks, Berkeley, USA, IEEE 3-5 June 2009.

Two-person interaction

Page 57: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

57

Data Fusion

Page 58: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

58

“…. combine data from multiple sensors and related

information from associated databases to achieve

improved accuracy and more specific inferences

than could be achieved with the use of a single

sensor alone.”

Hall and Llinas“An introduction to multisensor data fusion,” Proceedings of

the IEEE, vol. 85, no. 1, pp. 6–23, 1997.

Page 59: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

59

Classification based on relationship between data sources

H. F. Durrant-Whyte, “Sensor models and multisensor integration,” International Journal of

Robotics Research, vol. 7, no. 6, pp. 97–113, 1988.

Page 60: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

60

• Complimentary : Input sources represent different parts of

the problem space and when combined present a more

complete picture.

e.g., information on the same target provided by 2 cameras

with different fields of view

• Redundant : Input sources provide information about the

same target and could be fused to increase confidence.

e.g., data coming from overlapped areas in a visual sensor

network.

• Co-operative : when the sensor data is combined into new

information that is typical more complex than the original

data.

e.g., the combination of data from inertial magnetic unit to

obtain rotational angles

Page 61: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

61

Complimentary fusion: camera-based 3D motion capture

Page 62: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

62

Redundant Fusion: Ambient Temperature and Relative Humidity

from 5 co-located sensors

Page 63: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

63

Redundant Fusion: Temperature from 5 co-located temperature

sensors

Page 64: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

64

Redundant Fusion: Relative humidity from 5 co-located sensors

Page 65: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

65

Co-operative Fusion: Inertial Magnetic Unit + Barometeric

Altimeter for tracking heading and height

Page 66: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

66

Data fusion based on Input-Output data types

B. V. Dasarathy, “Sensor fusion potential exploitation-innovative architectures and

illustrative applications,” Proceedings of the IEEE, vol. 85, no. 1, pp. 24–38, 1997.

Page 67: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

67

• Data in – Data out : signal processing of raw data from

sensors

• Data in – Feature out : Shape extraction

• Feature in – Feature out : Fusion of video and audio data

• Data in – Decision out : Pattern recognition

• Feature in – Decision out : Object recognition

Page 68: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

68

Data in – Data out

• Data in – Data out : signal processing of raw data from

sensors

Page 69: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

69

Data in – Feature out : Shape extraction

• Data in : pixel values in an image

• Feature out : Labelled connected components in an image

Page 70: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

70

• Deal with sensor failure thanks to redundancy leading to

robustness

• Reduce uncertainty leading to greater confidence in the

output

• Multi-dimension data from different sensors leads to more

comprehensive description of environment or process

• Improve resolution due to multiple sensors

Advantages of using multiple sensors

Page 71: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

71

• Data in different formats:

analogue, digital, textual, audio, video

• Data in different dimensions

Different co-ordinate systems, units, frequency

• Temporal alignment

Data synchronisation

Common clock for spatially distributed sensors

Different propagation delays for data arriving at fusion point

Challenges of using multiple sensors

Page 72: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

72

Sensor Fusion: 3-D Orientation

Page 73: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

73

Page 74: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

74

Page 75: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

75

Raw sensor data from stationary Inertial-magnetic unit

Page 76: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

76

Calibration of sensor data from stationary Inertial-magnetic

unit

• Correction for scale and bias

• Accl. : X and Y axes. = 0; accl. due to gravity in Z axis

pointing downwards, hence -1g

• Gyroscope data is white noise

Page 77: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

77

Drifting rotational angle measured by integration of gyroscope

output

• Integration using the Trapezoidal integration method yields

drift - 50 degrees after 30s

• Integration accumulates noise over time manifest as drift

• Output of the gyroscope unaffected by the earth’s gravity

Page 78: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

78

Accelerometer

• Measures acceleration due to linear motion and due to gravity

• A filter separates the two but results in sluggish response

• Rotation around the x-axis (roll) and y-axis (pitch) is

calculated in radians using the equations below :

Page 79: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

79

• Accelerometer – noisy rotation angle, no drift; Gyroscope –

no noise in the rotation angle but with drift

• Neither the accelerometer nor the gyroscope provide

accurate rotational measurements on their own

• Measurement of rotation around the z-axis (yaw) requires

accelerometer to be combined with other sensors

Sensor fusion

Page 80: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

80

Sensor fusion

• Use calibrated accelerometer to obtain noisy roll* and pitch*

angles

• Combine them with a gyroscope signal to obtain clean and

drift-less roll and pitch angles

• Combine roll and pitch angles with magnetometer in “Tilt

Compensation” unit to calculate Yaw rotation

Page 81: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

81

• A recursive algorithm which estimates the state of a system at

time (t) by using the state of the system at time (t-1)

xk+1 = Axk + wk

zk = Hxk + vk

• Xk is the state vector at time at time k

• A is the state transition matrix (system changes with time) • wk is the state transiiton noise

• Zk is the measurement of x at time k

• H is the observation matrix (r’ship b/w state variable & m’ment)

• vk is the measurement noise

• ω is the angular velocity from the gyroscope

• Φ is the rotational angle calculated by the accelerometer signal

Kalman Filter

Page 82: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

82

Tilt Compensation

• Measurement of yaw (heading) angle around the Z-axis,

perpendicular to the earth’s surface.

• Drift problem with the gyroscope and need to know the initial

heading

• Earth’s magnetic field parallel to the earth’s surface. A level

tri-axis Magnetometer can measure accurately the heading

through the direction of the earth’s magnetic field

• Errors appear in the calculations of heading due to the tilt of

the magnetometer

• Tilt compensation maps the magnetometer data to the

horizontal plane using the roll and pitch angles calculated

Page 83: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

83

Page 84: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

84

Page 85: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

85

DDDDDDDDD

1. Evans R, Arvind DK, Detection of Gait Phases Using Orient Specks for Mobile Clinical Gait Analysis, in Proc. 2014 Wearable and Implantable Body Sensor Networks (BSN), 2014 11th International Conference on. IEEE Computer Society, p. 149-154.

Page 86: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

86

Page 87: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

87

“By 2017, 30% of wearable technology will be

unobtrusive to the naked eye. Consumer wearables

will blend seamlessly into their surroundings. Smart

contact lens are one and another interesting

wearable that is emerging is smart jewellery.”

Annette Zimmerman,

Research Director – Gartner,

December 2014.

Page 88: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

88

Connecting Places, People and Things

Page 89: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

89

Page 90: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

90

Page 91: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define
Page 92: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

92

Page 93: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

93

Stanford/Berkeley

Self contained – no external components.

24/60 GHz radio.

50 cm radio range.

No battery required.

Page 94: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

94

Page 95: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

95

Google/Novartis(Alcon)

Miniature sensor and antenna sandwiched between

two contact lenses.

Continuous monitoring of blood glucose level in

human tear.

Uploaded to smart phone.

Page 96: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

96

Platform for iPackaging

Page 97: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

97

Flexible Heart Sensor (UIUC)

Sensor laden sheath around the heart.

Irregular heart rhythm.

Changes in pH during restriction of blood supply.

Temperature fluctuations resulting from localized

burns.

Page 98: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

98

Issues in IoT Security and Privacy

Page 99: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

99

Page 100: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

100

Page 101: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

101

Page 102: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

102

Page 103: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

103

Page 104: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

104

Page 105: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

105

Page 106: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

106

Page 107: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

107

Bates CA, Ling M, Mann J and Arvind DK, Respiratory rate and flow waveform estimation from tri-axial accelerometer data, in Proc. 2010 Int. Conf. on Body Sensor Networks, BSN 2010, Singapore, IEEE.

Page 108: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

108

Page 109: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

115

Page 110: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

116

Page 111: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

117

Networked Critical Infrastructure

Page 112: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

118

Interdependencies, Interdependencies,…

Switches, control systems

Storage, pumps, control systems, compressors

e-commerce, IT

Pumps, lifts, control systems

Signalization, switches,control systems

e-government,IT

Medical equipment

TransportationTransportation

Oil & Natural Gas

EE

LL

EE

CC

TT

RR

II

CC

II

TT

YY

Potable & Waste WaterPotable & Waste Water

Emergency ResponseEmergency Response

Government

TT

EE

LL

EE

CC

OO

MM

Banking & FinanceBanking & Finance

Water for cooling, emissions control

Water for production, cooling, emissions control

Fire suppression

Cooling

Co

mm

un

ica

tion

s

SCADA

SCADA

Trading, transfers

SCADA

Co

mm

un

ica

tion

s

Location, EM contact

Currency (US Treasury; Currency (US Treasury; Federal Reserve )Federal Reserve )

DOE;DOE;DOTDOT

Regulations & enforcement Regulations & enforcement FERC; DOEFERC; DOE

Personnel/Equipment Personnel/Equipment (Military)(Military)

Fin

an

cin

g, re

gu

latio

ns

, & e

nfo

rce

me

nt

Fin

an

cin

g, re

gu

latio

ns

, & e

nfo

rce

me

nt

FASB; IRCFASB; IRC

FEMA; DOTFEMA; DOT

DOTDOT

EPAEPA

Detection, 1st responders, repair

Switches, control systems

Storage, pumps, control systems, compressors

e-commerce, IT

Pumps, lifts, control systems

Signalization, switches,control systems

e-government,IT

Medical equipment

Switches, control systemsSwitches, control systems

Storage, pumps, control systems, compressors

e-commerce, IT

Pumps, lifts, control systems

Signalization, switches,control systems

e-government,IT

Medical equipment

Storage, pumps, control systems, compressors

Storage, pumps, control systems, compressors

e-commerce, ITe-commerce, IT

Pumps, lifts, control systemsPumps, lifts, control systems

Signalization, switches,control systems

Signalization, switches,control systems

e-government,ITe-government,IT

Medical equipmentMedical equipment

TransportationTransportation

Oil & Natural Gas

EE

LL

EE

CC

TT

RR

II

CC

II

TT

YY

Potable & Waste WaterPotable & Waste Water

Emergency ResponseEmergency Response

Government

TT

EE

LL

EE

CC

OO

MM

Banking & FinanceBanking & Finance

Water for cooling, emissions control

Water for production, cooling, emissions control

Fire suppression

Cooling

Co

mm

un

ica

tion

s

SCADA

SCADA

Trading, transfers

SCADA

Co

mm

un

ica

tion

s

Location, EM contact

Currency (US Treasury; Currency (US Treasury; Federal Reserve )Federal Reserve )

DOE;DOE;DOTDOT

Regulations & enforcement Regulations & enforcement FERC; DOEFERC; DOE

Personnel/Equipment Personnel/Equipment (Military)(Military)

Fin

an

cin

g, re

gu

latio

ns

, & e

nfo

rce

me

nt

Fin

an

cin

g, re

gu

latio

ns

, & e

nfo

rce

me

nt

FASB; IRCFASB; IRC

FEMA; DOTFEMA; DOT

DOTDOT

EPAEPA

Detection, 1st responders, repair

TransportationTransportation

Oil & Natural Gas

EE

LL

EE

CC

TT

RR

II

CC

II

TT

YY

Potable & Waste WaterPotable & Waste Water

Emergency ResponseEmergency Response

Government

TT

EE

LL

EE

CC

OO

MM

Banking & FinanceBanking & Finance

TransportationTransportationTransportationTransportation

Oil & Natural GasOil & Natural Gas

EE

LL

EE

CC

TT

RR

II

CC

II

TT

YY

EE

LL

EE

CC

TT

RR

II

CC

II

TT

YY

Potable & Waste WaterPotable & Waste WaterPotable & Waste WaterPotable & Waste Water

Emergency ResponseEmergency ResponseEmergency ResponseEmergency Response

GovernmentGovernment

TT

EE

LL

EE

CC

OO

MM

TT

EE

LL

EE

CC

OO

MM

Banking & FinanceBanking & FinanceBanking & FinanceBanking & Finance

Water for cooling, emissions control

Water for production, cooling, emissions control

Fire suppression

Cooling

Water for cooling, emissions controlWater for cooling, emissions control

Water for production, cooling, emissions controlWater for production, cooling, emissions control

Fire suppressionFire suppression

CoolingCooling

Co

mm

un

ica

tion

s

SCADA

SCADA

Trading, transfers

SCADA

Co

mm

un

ica

tion

s

Location, EM contact

Co

mm

un

ica

tion

s

SCADA

SCADA

Trading, transfers

SCADA

Co

mm

un

ica

tion

s

Location, EM contact

Co

mm

un

ica

tion

s

SCADA

SCADA

Trading, transfers

SCADA

Co

mm

un

ica

tion

s

Location, EM contact

SCADA

SCADA

Trading, transfers

SCADA

Co

mm

un

ica

tion

s

Location, EM contact

Currency (US Treasury; Currency (US Treasury; Federal Reserve )Federal Reserve )

DOE;DOE;DOTDOT

Regulations & enforcement Regulations & enforcement FERC; DOEFERC; DOE

Personnel/Equipment Personnel/Equipment (Military)(Military)

Fin

an

cin

g, re

gu

latio

ns

, & e

nfo

rce

me

nt

Fin

an

cin

g, re

gu

latio

ns

, & e

nfo

rce

me

nt

FASB; IRCFASB; IRC

FEMA; DOTFEMA; DOT

DOTDOT

EPAEPA

Currency (US Treasury; Currency (US Treasury; Federal Reserve )Federal Reserve )

DOE;DOE;DOTDOT

Regulations & enforcement Regulations & enforcement FERC; DOEFERC; DOE

Personnel/Equipment Personnel/Equipment (Military)(Military)

Fin

an

cin

g, re

gu

latio

ns

, & e

nfo

rce

me

nt

Fin

an

cin

g, re

gu

latio

ns

, & e

nfo

rce

me

nt

FASB; IRCFASB; IRC

FEMA; DOTFEMA; DOT

DOTDOT

EPAEPA

DOE;DOE;DOTDOT

Regulations & enforcement Regulations & enforcement FERC; DOEFERC; DOE

Personnel/Equipment Personnel/Equipment (Military)(Military)

Fin

an

cin

g, re

gu

latio

ns

, & e

nfo

rce

me

nt

Fin

an

cin

g, re

gu

latio

ns

, & e

nfo

rce

me

nt

FASB; IRCFASB; IRC

FEMA; DOTFEMA; DOT

DOTDOT

EPAEPA

Detection, 1st responders, repair

Detection, 1st responders, repair

Detection, 1st responders, repair

Source: Miriam Heller

Page 113: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

119

The Plant: A Complex Environment

sec

msec

1 sec

secs

min

hours

Plant

Servers

Other

Computing

Devices

Business Management

Area Servers Plant

Network

Modules

Network

Gateway Network

Gateway

Process Management

Subnetwork Gateway

Application

Module History

Module

Personal Computer

Network Manager

Control Stations

Archive

Replay Module

Additional

CN Modules

Fiber Optics

Network

Interface

Module Other Data

Hiway Boxes

Multifunction

Controller Extended

Controller

Basic

Controller Advanced

Multifunction

Controller LocalProcessors

Smartine

Transmitters

PLC

Gateway

Other

Subsystems

PLC

Logic Manager Process

Manager

Advanced

Process

Manager

Transmitters

Control Network

Extenders

Field Management

Source: TRUST

Page 114: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

120

IoT systems affect the physical world

When compromised IoT systems may

• Cause material damage

• Jeopardise safety

• Harm the environment

• Cause serious industrial accident

Page 115: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

121

Unsecured IoT systems

Private information

• movement

• activity

• health

Infer from seemingly harmless data

• electricity consumption from smart meters

• room temperature

Compromised IoT systems

• impacts the physical world

• May cause material damage

• Jeopardise safety

• Harm the environment

• Cause serious industrial accident

Page 116: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

122

Secure interaction of applications

Mobile operators are providers of

• connectivity

• platforms

Two key functions

• identification based on credentials

• secure data transport

Many devices deployed in capillary networks and

connected to cellular networks via gateways

• device management

• secure bootstrapping

• Assertions: verifying device location or

trustworthiness of platform

• Encryption on radio interfaces

• Security against side-channel attacks

Page 117: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

123

Device security

Trade-off between device security and device cost

• Limited memory, low processing speeds, low

throughput radio, battery life makes existing

security protocol suboptimal

• Over-the-air firmware and software updates can

be difficult when not enough memory to store old

and new firmware leading to less robust and fatal

updates

• Enforcing remote firmware updates when devices

compromised by viruses block updates

Page 118: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

124

Device security

IoT devices are exposed:

• Sensitive data encrypted for secure storage

• Cryptographic verification of firmware and

software packages at boot or update times

• Enforcing remote firmware updates when devices

compromised by viruses block updates

• Sufficient memory for automatic rollback in the

event of update failure

Page 119: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

125

Application layer security

Battery-powered IoT devices sleep most of the time

and rely on intermediaries such as gateways and

proxies to cache requests and responses, reduce

response time, bandwidth and energy consumption

• Breach in security when a compromised

intermediary reads, changes or injects data

• Over-the-air firmware and software updates can

be difficult when not enough memory to store old

and new firmware leading to less robust and fatal

updates

• Enforcing remote firmware updates when devices

compromised by viruses block updates

Page 120: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

126

Relay attacks on proximity-based security

Page 121: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

127

Proximity-based security in IoT systems

• Locks, vehicles, expensive equipment

• Smart car keys, access cards, contactless payment

• Losing control worse than eavesdropping

• They verify freshness but not proximity

• Vulnerable to 2-person relay attacks

• Relay signal from a device in victim’s pocket to

Page 122: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

128

The company, Dyn, whose servers monitor and reroute internet traffic, said it began

experiencing what security experts called a distributed denial-of-service attack just

after 7 a.m. Reports that many sites were inaccessible started on the East Coast, but

spread westward in three waves as the day wore on and into the evening.

And in a troubling development, the attack appears to have relied on hundreds of

thousands of internet-connected devices like cameras, baby monitors and home

routers that have been infected — without their owners’ knowledge — with software

that allows hackers to command them to flood a target with overwhelming traffic.

Page 123: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

129

Distributed Denial of Service (DDoS) attacks

• Unsecured IoT devices are used as weapons to spray

the critical infrastructure target

• Neither the owners nor the manufacturers of the IoT

devices bear the costs of the attacks

• Critical infrastructure should withstand direct hacking,

and also resilient to DDoS and jamming

Page 124: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

130

Hardware-based isolation

• Security features isolated from applications on devices

• Trusted execution environment (TEE) isolated from the

rich execution environment (REE)

• Prevent IoT devices as a stepping stone to attack

critical infrastructure

Page 125: Principles and Design of IoT systems Level 11 course [20 … Introduction to the platform and programming environment; Demonstration of an end-to-end IoT system as an exemplar. Define

131

Hardware isolation using trusted execution environments