are homes smart if they are aware?

Marco Aiello

September 10, 2012

Are Homes Smart if they are Aware?

1

Being aware of the presence

2

Being aware of the presence

2

Being aware of the presence

2

✦ Sensor data provides basic context components

✦ Architectures taking advantage of awareness

✦ User expresses goals, explicitly or implicitly

3

Some sensing today

4

Some sensing today

... and is all getting networked

4

Nest Thermostat

5

✦ So we have the basic blocks for being aware, but...

✦ How do we become smart?

✦ How does the user take advantage of this?

6

What is a smart home?

7

What is a smart home?

A home is smart when, being aware of its own state

and that of its users, is capable of controlling itself in

order to support the users wishes, thus improving

their quality of life.

7

What can it do?

8

What can it do?

It understands (user) goals and it can satisfy them.

8

9

9

10

Is controlled by thoughts

10

Reacts to dangersIs controlled by thoughts

10

Reacts to dangers

It saves your life

Is controlled by thoughts

10

Reacts to dangers

Reminds you of pillsIt saves your life

Is controlled by thoughts

10

Goal

Domain descriptio

Composition

Orchestration

Context Awareness

Rule Engine

Repository

Plan

Context

User layer

Composition Layer

Pervasive layer

Discovery

11

Context

›❯ Context reflects the state of the house, based on the information collected by the sensors

›❯ Represented as the values of all variables describing the house• E.g. temperature_in_kitchen=30 C, lamp1=ON,

location_of_user = in_bedroom

›❯ Keep up-to-date about the changing context through a publish-subscribe notification mechanism

12

Composition

›❯ Domain-independent AI planning based on constraint satisfaction

›❯ A composition is generated on-the-fly, depending on the available services

13

Composition

›❯ Input of the planner: • Domain description (from the repository): atomic

service operations seen as actions with pre-conditions and effects

• The current context (initial state)• A goal to be achieved, issued either by the user or

the house (rule engine)›❯ Output: • a sequence of actions-services that when executed

satisfy the goal

14

Two actions

15

Goal Language

›❯ Extended goal language to express complex requests

›❯ House domain and goal are encoded via constraints

›❯ The planning problem is translated to a constraint solving problem

›❯ Re-planning can be performed in case a service is out of order

16

Goals

17

Plans

18

Implementation

›❯ 1.83 Ghz computer running Debian lenny, 32 bit and Java 1.6.0_12.

›❯ The constraint solver standing at the core of the planner is the Choco v2.1.1 constraint solving library

›❯ Model a home with 5 rooms, and 10 devices providing 21 UPnP action

19

Evaluation

20

Discussion++

›❯ Automatic, run-time, user-centric synthesis instead of hard-wired process descriptions [all we need is atomic descriptions, completely loosely-coupled]

›❯ Planner supports complex goals, variables with large domains (e.g. temperature)

›❯ Dynamic constraint network: efficient continual addition-removal of constraints according to context changes

21

Discussion--

›❯ Efficiency: heuristics have to be improved for the system to scale

›❯ Incomplete pruning of irrelevant actions in some cases

›❯ React differently according to the kinds of faults

›❯ Concurrency issues (e.g. events interfering with execution)

22

Making a home sm4all

23

Goal

Domain descriptio

Composition

Orchestration

Context Awareness

Rule Engine

Repository

Plan

Context

User layer

Composition Layer

Pervasive layer

Discovery

24

25

26

Evaluation at THFL (Leeuwarden)

Evaluation with Master Students (Groningen)

Physical test home at FSL (Rome)

Evaluation

27

Evaluation at THFL (Leeuwarden)

Evaluation with Master Students (Groningen)

Physical test home at FSL (Rome)

23 Touch 8 BCI, average age > 71

30 Touch, average age > 25

with EU project officers

Evaluation

27

28

28

29

30

31

S M 4 L L

Sm4All project 1

t was in the spring of 2009 that via Tineke Hijma of the Tuiszorg Het FrieseLand I was asked wether I

would participate in a European project that can help people with limited mobility via domotics to be more independent at home. Initially, I thought that the SM4All (Smart Homes for All) project would be more in the medical area, but this later appeared to be not the case. SM4All is actually about how a computer-networked system in a home can satisfy specific user wishes that have some form of disability without the necessity for the person to physically act. If I would be living in such a smart home and someone would ring the doorbell, I could think `now open the door’ as an instruction and the computer system would capture such instruction and effectively open the door. The fact that I initially thought of a medical project was not un-logical since the interview that was performed in the following summer was mostly about my physical condition and its consequences. But the closing question was in the computer world, namely: ``what kind of automation would you like to have in your home if you could decide?’’ All the interviews have then been collected, analyzed and the conclusions discussed during a workshop in Madrid. For THFL was then also Tineke Hijma one of the participants.

First day

After the holidays I received a call from Johanna Rinsma, the Team-manager of Special Services at THFL, who wanted to follow up on the interview of 2009 and discuss the possibilities for me to take part in the second phase of the project. The time was such that what had been researched was now ready to be tested. The evaluation would take place at the premises of THFL in two sessions in different days. I was expected to go on November 18th, 2010.

From the reception area, I had a pretty dismal view of proud leftovers of the HTS (HogeTechnischeSchool, a secondary Dutch technical degree awarding institution) with its mighty steam engine where I had laid many footsteps in the past on my way to the Traffic Engineering lectures. A steam engine as the symbol of the first industrial revolution and so different from the computers chips that have brought so many changes in the second half of the last century in many technologies, including things of daily use such as cars and washing machines.

And after the internet hype around the turn of the millennium, we perceive the revelations of Wiki Leaks as the sign of the approach of a new era.The lounge of the night-care rooms of THFL was set up as a test area where many wires, monitors and laptops were ready for me and anticipated the entrance of a new virtual world. During the test trials the nurses Willy van der Brink and Dini Sieswerda were present to guide the participants, when necessary, to help with the translation of instructions from English to Dutch as the present

researchers were Italian and could only communicate in English with me.If one wants a TV to be turned on in the virtual home, for example, it is necessary that the computer ``knows’’ that one is thinking “now TV,’’ otherwise after a while nothing happens. Before the system can start, it must be first connected to my brain. This is done as in the hospital with an EEG that can register electroencephalographic modulations.

After an explanation of the test, I got the equipment on. I was sitting in front of a

display with a brown dot in the middle, an interface on my head taped over with a kind of bathing cap and with 16 electrodes that were linked to the system and a band around the chest just under the armpits. The test could begin. On the screen all letters and numbers appeared and at the top was the word 'water'. Water could be written by counting how often the letter W was in the first place while all letters and numbers were flashing on the screen quite rapidly: it was a question of good concentration to stay focused for

From Thoughts to ActionsHaje Haisma (1942-2011)

Translated by Marco Aiello, 2011

www.sm4all-project.eu February 2011

Haisma during the second day of evaluation with the brain computer interface on.

31

32

Electricity Awareness

33

User Awareness

Ubiquitous System layer

Service Composition layer

Occupant behavior

Actuation & Building Context

Information

Energy-aware MiddlewareEvent-based, Service-Oriented, Orchestrating

User interaction, control, feedback

Ubiquitous SensingUser control

Component interactions

Actuation

Context User control

Activity

User Interface

www.greenerbuildings.eu

34

Heat flow Awareness

35

Greener Buildings

✦ Combine electricity, user and heat flow awareness

✦ Control to save energy

✦ Couple with Smart Grid Dynamic pricing to save money

36

37

38

38

Energy savings over a week (about 10%)

39

Cost savings over a week (about 35%)

40

A note on the Smart Grid

✦ The implication of having aware self controlling buildings and renewable energy sources will make also the Power Grid a self organizing system

41

Network Evolutions

Forest fireKronecker model

Preferential AttachmentRandom Graph Random Graph Power Law

RMat Small World

Copying model

42

Decision Support for Planning the Smart Grid

43

Are homes smart if they are aware?

✦ Smartness requires awareness

✦ Need for integrated architectures

✦ Awareness alone won’t provide for smartness

44

45

Smartness is nothing without Awareness

45