configuration of smart environments made simple

1|Simon Mayer

http://people.inf.ethz.ch/mayersi

Simon Mayer, Nadine Inhelder, Ruben Verborgh,

Rik Van de Walle, and Friedemann Mattern

Internet of Things 2014 Conference | Cambridge, MA | Oct. 7 2014

Configuration of Smart Environments Made SimpleCombining Visual Modeling with Semantic Metadata and Reasoning

2|Simon Mayer

http://people.inf.ethz.ch/mayersi

Embedded sensors and actuators measure and actuate

values in the real, physical world and allow for faster

and smarter decisions

Smart devices allow well-informed decisions

Umbrella lights up if forecast shows rain

Smart environments can perform complex tasks

Assembly line adjusts to current business needs

Smart Devices in a Smart World

3|Simon Mayer

http://people.inf.ethz.ch/mayersi3|

Enable human users to better

monitor and control entire

smart environments

4|Simon Mayer

http://people.inf.ethz.ch/mayersi

Internet of Things for basic network-level connectivity

between smart devices

Challenge: Isolated “Intranets of Things” on the application level

Web of Things: Application-level convergence

Fully integrate sensors and actuators with the World Wide Web

The IoT and the Web of Things

5|Simon Mayer

http://people.inf.ethz.ch/mayersi

Integrate Functionality across WoT Devices

Smart

Thermostat

Environmental

Sensor

Smart Umbrella

www

Web

Services

6|Simon Mayer

http://people.inf.ethz.ch/mayersi

Integrate Functionality across WoT Devices

www

I know that!

Ask me!

Is it going to rain

today?

What’s the

temperature?

76% chance!

Embed semantic functional service

descriptions in smart things representations

How? Describe service API

What?Describe what a

service does

8|Simon Mayer

http://people.inf.ethz.ch/mayersi8|

What?Describe what a

service does

Given a temperature value, I

can set this room to that

temperature!

Temperature Value Modify Room State

How? Describe service API

𝑃𝑟𝑒𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 → 𝑃𝑜𝑠𝑡𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛

9|Simon Mayer

http://people.inf.ethz.ch/mayersi9|

How? Describe service API

Given a temperature value, I

can set this room to that

temperature!

PUT to thermostat.comTemperature Value Modify Room State

𝑃𝑟𝑒𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 → 𝑃𝑜𝑠𝑡𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 ∧ 𝐻𝑇𝑇𝑃 𝑅𝑒𝑞𝑢𝑒𝑠𝑡

10|Simon Mayer

http://people.inf.ethz.ch/mayersi10|

Given a temperature value, I

can set this room to that

temperature!

No

tati

on

3

{

?newTemp a dbpedia:Temperature;

ex:hasValue ?celsiusValue;

ex:hasUnit "Celsius".

}

=>

{

?state :Office44 ex:hasTemperature ?newTemp.

_:request http:methodName "PUT";

http:requestURI

(<http://thermostat.com?temp=>?celsiusValue).

}.

Precondition

Postcondition

Request

11|Simon Mayer

http://people.inf.ethz.ch/mayersi11|

Given a temperature value, I

can set this room to that

temperature!

No

tati

on

3

{

?newTemp a dbpedia:Temperature;

ex:hasValue ?celsiusValue;

ex:hasUnit "Celsius".

}

=>

{

?state :Office44 ex:hasTemperature ?newTemp.

_:request http:methodName "PUT";

http:requestURI

(<http://thermostat.com?temp=>?celsiusValue).

}.

Postcondition

Request

12|Simon Mayer

http://people.inf.ethz.ch/mayersi12|

Given a temperature value, I

can set this room to that

temperature!

No

tati

on

3

{

?newTemp a dbpedia:Temperature;

ex:hasValue ?celsiusValue;

ex:hasUnit "Celsius".

}

=>

{

?state :SomePlace ex:hasTemperature ?newTemp.

_:request http:methodName "PUT";

http:requestURI

(<http://thermostat.com?temp=>?celsiusValue).

}.

Request

How can this service

be invoked?

13|Simon Mayer

http://people.inf.ethz.ch/mayersi13|

Given a temperature value, I

can set this room to that

temperature!

No

tati

on

3

{

?newTemp a dbpedia:Temperature;

ex:hasValue ?celsiusValue;

ex:hasUnit "Celsius".

}

=>

{

?state :SomePlace ex:hasTemperature ?newTemp.

_:request http:methodName "PUT";

http:requestURI

(<http://thermostat.com?temp=>?celsiusValue).

}.

14|Simon Mayer

http://people.inf.ethz.ch/mayersi14|

OK. So I know the device URL.

How can I find the functional metadata?

15|Simon Mayer

http://people.inf.ethz.ch/mayersi15|

HTTP OPTIONS is a good fit!

“This method allows the client to determine the options and/or requirements

associated with a resource, or the capabilities of a server, (…)” (RFC 2616)

Descriptions are returned using the Link Header (RFC 5988)

OPTIONS example.org/thermostat

Link: <therm.n3>; rel=“describedBy“

Semantic reasoning engines can process

these descriptions and automatically

combine services to achieve a user goal

Semantic Metadata

Aha!

This enables the goal-driven configuration

of smart environments!

Do *it*!

I can find out how

to do *it* myself!

18|Simon Mayer

http://people.inf.ethz.ch/mayersi18|

I want my environment to

have a temperature of 23°C

You should send a PUT

request to thermostat.com

that contains the number

“23” as payload!

19|Simon Mayer

http://people.inf.ethz.ch/mayersi19|

I want my environment to

have a temperature of 23°C

No way!

Only °F

inputs…

This will fail if no compatible

services are available…

:temp23 a dbpedia:Temperature;

ex:hasValue "23";

ex:hasUnit "Celsius".

?state a st:State;

log:includes {

:Office ex:hasTemperature :temp23.

}.

"PUT";

:temp23 a dbpedia:Temperature;

ex:hasValue "23";

ex:hasUnit "Celsius".

?state a st:State;

log:includes {

:Office44 ex:hasTemperature :temp23.

}.

"PUT";

"PUT";

The requests can be executed by the

reasoner or the client, or can be

parsed to mobile code….

A reasoning engine enables the dynamic

creation of service chains from the functional

semantic metadata of individual services

Semantic Metadata

25|Simon Mayer

http://people.inf.ethz.ch/mayersi

Advantages

Dynamic service composition in smart environments

Services becoming unavailable, new services emerging…

26|Simon Mayer

http://people.inf.ethz.ch/mayersi

Advantages

Dynamic service composition in smart environments

Rich vocabulary: Lots of ontologies out there

27|Simon Mayer

http://people.inf.ethz.ch/mayersi

Advantages

Dynamic service composition in smart environments

Rich vocabulary

Extremely efficient

300ms to chain

2000 services…

28|Simon Mayer

http://people.inf.ethz.ch/mayersi

Advantages

Dynamic service composition in smart environments

Rich vocabulary

Extremely efficient

Chaining 100 services with

up to 1000 options per step

29|Simon Mayer

http://people.inf.ethz.ch/mayersi

Challenges

Not straightforward in stateful smart environments

A room cannot have two different temperatures at the same time!

Approach: Helper ontology for state management

Details in the paper!

30|Simon Mayer

http://people.inf.ethz.ch/mayersi

Challenges

Not straightforward in stateful smart environments

Users need to be supported during goal formulation

Approach: Integration with visual programming language

31|Simon Mayer

http://people.inf.ethz.ch/mayersi

Challenges

Not straightforward in stateful smart environments

Users need to be supported during goal formulation

Approach: Integration with visual programming language

32|Simon Mayer

http://people.inf.ethz.ch/mayersi

Challenges

Not straightforward in stateful smart environments

Users need to be supported during goal formulation

Users require feedback on the interactions

Approach: Verbalization and Visualization

33|Simon Mayer

http://people.inf.ethz.ch/mayersi

Monitoring Interactions: Verbalization

I want my environment to

have a temperature of 23°C????Aha!

34|Simon Mayer

http://people.inf.ethz.ch/mayersi

Monitoring Interactions: Visualization

What is my smart

electricity meter up to?

[Mayer, Hassan, and Sörös, 2014]

35|Simon Mayer

http://people.inf.ethz.ch/mayersi

Monitoring Interactions: Visualization

So I can actually see

HTTP messages!

[Mayer, Hassan, and Sörös, 2014]

36|Simon Mayer

http://people.inf.ethz.ch/mayersi

Making functional semantics and reasoning

work in smart environments

Facilitation of the goal formulation step and provisioning

of methods to supervise composite services

Summary: Automatic Service Composition

37|Simon Mayer

http://people.inf.ethz.ch/mayersi

Scaling up to industrial automation scenarios

Enabling users to control device interactions

Future Work

YOU shouldn’t be

interacting…

38|Simon Mayer

http://people.inf.ethz.ch/mayersi

Thank You!

R. Verborgh, V. Haerinck, T. Steiner, D. Van Deursen, S. Van Hoecke, J. De Roo, R. Van de Walle, and J.

Gabarró Vallés: Functional Composition of Sensor Web APIs. Semantic Sensor Networks 2012

S. Mayer and G. Basler: Semantic Metadata to Support Device Interaction in Smart Environments. Web of

Things 2013

S. Mayer, N. Inhelder, R. Verborgh, and R. Van de Walle: User-friendly Configuration of Smart

Environments. PerCom 2014

S. Mayer, Y. N. Hassan, and G. Sörös: A Magic Lens for Revealing Device Interactions in Smart

Environments. SIGGRAPH Asia MGIA 2014

S. Mayer, N. Inhelder, R. Verborgh, R. Van de Walle, and F. Mattern: Configuration of Smart Environments

Made Simple. Internet of Things 2014

Slides are on www.slideshare.net/smnmyr

39|Simon Mayer

http://people.inf.ethz.ch/mayersi

http://andrewbleakley.com

http://www.webmarchand.com

http://www.patentspostgrant.com/

http://www.orkin.com

http://sweetclipart.com

http://la-matrice.org

http://learn.randolph.k12.ma.us

http://theparisreview.org

http://bestclipartblog.com

http://freepik.com

http://http://www.rugdots.com

http://www.productdose.com

http://westwoodblinds.co.uk

http://www.ambientdevices.com

http://www.pats.ua.ac.be

http://www.vesternet.com

http://www.rovingnetworks.com

http://www.qualitymechanicalservices.com

http://chantalteakettle.org

http://blog.eeye.com

http://cartographersguild.com

http://haverford.edu

http://clker.com

http://stickers-addict.fr

http://en.wikipedia.org

http://academycomicalarts.blogspot.com

Image Sources