ubiquitous computing max mühlhäuser, iryna gurevych (editors) preface: systematic approach to...

Ubiquitous Computing

Max Mühlhäuser, Iryna Gurevych (Editors)

Preface: Systematic Approach to Ubiquitous Computing and its Application to Real-Time Enterprises

Max Mühlhäuser

UbiquitousComputing

P-0: Outline - What this chapter covers

P-1 A first introduction– ‘Definition’: What is Ubiquitous Computing?– Importance: (why) is Ubiquitous Computing (UC) so important? First

glanceP-2 UC challenges – a first approach

– The challenge of humane UC– The challenge of integrated UC

P-3 The S.C.A.L.E. classification of Ubiquitous Computing– Intro: broad vs. deep: what should a treatment of UC cover – and why – Explanation of & introduction to the S.C.A.L.E classification– Relation to major Computer Science research fields

P-4 Reader’s Digest– Relation of S.C.A.L.E. classification to further chapters– Digest for Readers with different background/goals

2

UbiquitousComputing

P-1.1 Motivation & Definition: What is UC?

• Ubiquitous Computing describes the 3rd “era” of computing– Era 1: Mainframe Computing MC - many users use one computer– Era 2: Personal Computing PC - one user uses one computer– Era 3: Ubiquitous Computing UC - one user uses many computers

• Note, for Era 3:– some computers are worn (mobile phone, PDA; body sensors, life recorder…)– other computers are encountered on-the-move, more or less consciously

(cf. info kiosk / data projection versus badge-reader / surveillance cam)

• UC means “networked computers everywhere”– µC’s were embedded in VCRs, washing machines etc. since decades already!– BUT: now, with UC, the embedded computers become

• networked, i.e. part of a “web” of interacting computers surrounding us• “programmable”: changing firmware / SW is easier, less noticeable to user

(good: cf. bug fixes; bad: cf. undesired functionality, e.g. spying)

3

UbiquitousComputing P-1.1 Definition: What is UC?

• UC connects general & special purpose computers– the distinction is more a spectrum than two distinct classes– the power / resource spectrum (CPU, memory, bandwidth, …)

becomes huge!

How to ‘define’ Ubquitous Computing?A simple definition is given above: UC is the third era of

computing according to the aforementioned categorization

Our more detailed and self-sustained definition:Ubiquitous Computing is the dawning era of computing, in which individuals are surrounded by many networked, spontaneously yet tightly cooperating computers, some of them worn or carried, some of them encountered on the move, many of them serving dedicated purposes as part of physical objects, all of them used in an intuitive, hardly noticeable way with limited attention”.

4

UbiquitousComputing

P-1.1 Importance: Sample Nodes in UC networks

5

Tiny, rather general purpose computers acting, e.g., as nodes in sensor networks (cf. UC Berkeley’s motes, Particle Computer GmbH’s particles)Wearables and mobile computing

devices. cf. handhelds for ware-house picking, washable computerjackets, or companions like the “lovegetty” pictured: stores user’sprofile, beeps when a “compatible”person – with lovegetty – appears

Networked appliances, e.g., “smart vending machines” thattransmit fill status, errors etc. to head offices; press rumors about adaptive pricing caused protests

Smart labels such as RFIDs(above) and active badges(below) identify (option: locate, characterize)objects, humans, animals; benefits and privacy threatswidely discussed in press

UbiquitousComputing

P-2.3 Imporance: Exploding # of nodes

issue #1: on the networking side, the exponential growth (# of nodes connected in the Internet) continues way beyond the world’s populationconsequence: scalability becomes even more crucial

6

Appli-ances

everyday things

eMail

„Web“

wirelessInternet mobile non tech users

P2P P2M M2M People Machines

2000 time

Billions of

Zillions of

109

108

107

106

1010

time

log (capacity) Internet #nodes, wired speed

UbiquitousComputing

P-2.3 Importance: exploding # of nodes, consequence

issue #2: different doubling rates for networks, CPUs (Moore’s law) etc.; impact on: system architectures, algorithms, etc. (e.g., RPC becomes cheaper, mobile computing maybe more expensive?)

7

CPU (Moore‘s Law)

2000 time

log (capacity) Internet (# of nodes)

unpredictable: wireless-net bandwidth (bps)battery capacity

distributed computing:- disconnected-op?- mobile code?- server architectures

?? Impact on ...

wired-net bandwidth (bps) graphic cards (#polygons/sec)

UbiquitousComputing P-1.2 Importance: Conclusion

At the 30,000 ft. level, UC is “the next key technology to change our society” for 3 reasons:

1. UC describes the next era of computing– therefore, all of computer science / IT is potentially impacted– … it will be difficult to be selective in designing this course/book

(but we will see that a course / book on UC still makes a lot of sense)

2. UC has potential impact on every facet of our lives– computing is no more “what we do when we sit at the computer” +

“what is encapsulated/hidden deep inside VCRs etc.”– see the more / less comparison on next slide

3. UC is impossible and inevitable at the same time– components (“gadgets”) are developed and deployed, cf.

slide “UC sample nodes” above– more and more UC scenarios become profitable industry will push– many issues of “the whole” i.e. integrated UC systems still unresolved

8

UbiquitousComputing

P-2.1 From Importance to Challenges

A first glance (not yet structured, non exclusive) at challenges…

Anytime/anywhere presence of networked computers means:• more sensitivity less protection (cf. security)• more dependence less perfection (cf. reliability)• more obtrusion less attention (cf. HCI)• more garrulity less throughput (cf.

networking)

In other words: as computers become ubiquitous …• there is a risk that “the whole may be way less than the sum of the

parts” i.e. the desired integral functionality may lack way behind• some known problems aggravate considerably – we’ve got to care!• in short, UC is a problem of complex integrated systems, not of gadgets

compare to: new Airbus, Space Shuttle, not better rocket-fuel, headlights

9

UbiquitousComputing

P-2.1 Humane & Integrated UC - Overview

10

Top Level Challenges:A. Systems Integration

(Strength of Europe)B. Humane Computing

(Chance for Europe)

Outlook (see below): Scalability Connectivity Adaptability Liability (=Security +

+) Ease-of-Use

features of UC „gadgets“ are not the problem …

… but making them „humane“ and integratingthem into a useful whole

UbiquitousComputing P-2.2 Challenges: Humane UC

Mainstream UC Research emphasizes two issues:• Context-Aware Computing

ill defined; basically, the quest for adaptive software that …• understands the „situation in which the user currently works“• adapts to that situation by

– reducing complexity (e.g., adjusting UI to current needs & options)– automating input (e.g., retrieving data from „environment“, not user)– optimizing functionality

• Multiple Modalitiesagain, ill defined; modality is a „way of doing I/O“, often an

input and/or output „channel“ (what is that?) like graphics, voice,keyboard/mouse, gesture (note: examples from different categories!)

• UC applications on appliances (cell phone, PDA… today, more tomorrow) Multiple modalities to be supported• truly multimodal UI: flexible combination of „channels“, user‘s choice

• We will put these two issues in a larger context

11

UbiquitousComputing P-2.3 Challenges: UC Integration

12

?

Pertinent Application:Real-Time Enterprise.full coverage:production/supply chain,retail, post-sales

High-tier

Low-tier

Satellite

High Mobility Low MobilityWide Area

RegionalArea

LocalArea

Global Internet: seamlesscontext-aware (service!) network

UbiquitousComputing

P-3.1: S.C.A.L.E. Overview: Breadth vs. Depth

• Problem: UC touches almost every area of Computer Science, and more!– readers need background for understanding why what will be emphasized

later– a categorization of issues (taxonomy) is needed for organizing the topics

and for getting oriented cf. the S.C.A.L.E. taxonomy• So, is it a research/teaching subject in its own right?

– yes, compare to Distributed Systems:• there is distributed simulation, programming, algorithms, databases, AI, …

“distributed everything”!• yet, there are courses on Distributed Systems

– but: how to master breadth vs. depth?• be broad first, organize “problem space” S.C.A.L.E., see below• then emphasize pertinent problems

– either “new” aspect in UbiComp– or “much more relevant” aspect with UbiComp than without

• standard computer scientist should hardly need add’l background• … but will sometimes want to dig deeper in some subtopic … which is normal

• But then, is course/book going to be “complete”?– no, but more complete than any other book/course before– … and more well organized, due to S.C.A.L.E. scheme, see below

13

UbiquitousComputing P-3.1: S.C.A.L.E. - Rationale for Taxonomy

UC is novel subject reader needs overview of problem space

“Problem Space” approached in two ways:1. Holistic view of pertinent research fields

– S.C.A.L.E. classification scheme devised for that purpose– explained below, used throughout for structuring major “guide” through residual chapters

2. Holistic view of “the global UC network=system”– Mundo Reference Architecture devised for that purpose

• imagine as rough floorplan for an “entire” UC system• shows differences to Distributed Systems Ref-Arch’s and Definitions

– Mundo not supposed to be accepted worldwide• (might, at best, influence a future Ref.Arch. standard)• it’s purpose is to help readers in understanding & organizing UC field

14

UbiquitousComputing P-3.1 - S.C.A.L.E. Scheme

Major UC challenges, on a very high level: S – SCALABILITY

- how to support cooperation of “zillions” of components?- how to support nomadic users around the globe?

C – CONNECTIVITY- how to “easily” connect these zillions? Several levels of abstraction:

- wireless networks – a blessing and a curse (unreliable!)- most issues come above wired/wireless net (note: overlap with scalability):

- how to find/understand your peers? How to enable zero configuration- how to design networks for zillions2 of connections, without central-server bottleneck

A – ADAPTABILITY- usage during daily work, surrounded by 100s of components: need minimal interaction- major approach: context-aware computing – use it to automate tasks & reduce options- why “adaptability”? adapting-to-particular-user (user modeling) focussed beyond context-

awareness L- LIABILITY

- term indicates: we must go beyond today’s IT security solutions (not goals), since- today’s solutions do not scale (centralized components?), are not “humane” (see below), …- … & don’t flexibly consider conflicting (privacy, traceability) & related goals (dependability etc.)

E – EASE-OF-USE- adaptability permits “minimal” …, ease-of-use means “optimal” interaction (related!)- issue: optimal use & combination of modalities, advancement of specific modalities- issue: “understanding” natural input: a) xxx-to-text; b) text-to-meaning; “intelligence”?

15

UbiquitousComputing P-3.1 S.C.A.L.E - Scalability

S – SCALABILITY(is a “top priority” challenge reflected in acronym: scale) A. Mathematical Scalability (in the mathematical sense)

UC leads to (potential) cooperation of “zillions” of devices- thus, solutions need to work efficiently with zillions of components- comes to mind: complexity of the made the born (K. Kelly!) learn from the born

most relevant areas (basically, alternatives for addressing technical scalability):1. bionics i.e. bio-analog computing

- neural networks, cooperating robots: huge fields, only marginal importance for UC- ant colonies – often simulated / executed on single computer today; swarms, autonomous computing,

…; - what else? plants as examples? brain-like modeling human memory: example given in main chapter

“S”2. socionics: learning from complex societies … is still in its infancies wrt. UC3. Future communication/cooperation (pub/sub; P2P, Grid) – see further below (C –

connectivity)B. Technical-Economical Scalability (in the sense of global interoperability)

- human-attached encountered components need to cooperate globally- standards come to mind, but global standards are not sufficient & (often) not realistic:

e.g., interface definition languages (cf. remote method/procedure call etc.):- they define syntax (types/name of operations/operands exported @ interface- they usually don’t define, e.g.: valid call sequences, preconditions, cost/quality etc. (approaches exist)- they don’t define semantics: what does an operation actually do? approach??: ontologies & service discovery

most relevant areas (to be combined into an overall solution):1. WebServices and Business Processes2. Ontologies a holy grail for making services understand each other (cf. what does the op’

do) ?3. Service Discovery that should take into account SLA (service level agreements), QoS

(quality of Service – in the business sense of “get what you pay for”), and more 16

UbiquitousComputing P-3.1 S.C.A.L.E: Connectivity

Global interconnection of UC components is related to the following(and to “scalability” above):A. Scaleable Communication

1. wireless networks: (often!) a prerequisite for higher layers needbasic understanding of technology [ZigBee … WiMax] and beyond

2. event based communication: praised as the UC connectivity approach• means “push” paradigm, a prerequisite for scaleable open cooperation of

components (supersedes client/server!); note relation to “S”• maybe one day superseded by bionic/socionic approach; today: best offer• remaining problems (advertising, openness, integration of other paradigms)• plus: what else is needed in UC middleware (services? tools? which ones?)

B. Scaleable Cooperation1. Overlay Nets: special subnetworks in Internet; at least 3 classes:

a. Peer-to-Peer Networks: no centralized bottleneck, scale well?b. Opportunistic Networks: ad hoc net (node proximity) parallels global human “network”c. Grid Networks: dynamically available resources parallel fluctuating resource demandsd. Smart Item Networks: AutoID and beyond: towards scaleable networks of “machines”

2. Service Discovery: prerequisite for zero configuration• component environment, makes itself known & learns about environment• sounds like solution to “economic scalability” above, but isn’t: many

assumptionsabout components are made rather a connectivity issue

17

UbiquitousComputing P-3.1 S.C.A.L.E: Adaptability

A. Context Awareness (adaptation to “situation of use”)• sensed context: what sensors can measure (temP-, shock, location, …)• modeled context: info held in “other” software/DB: tasks & activities etc.• inferred context: built from (several?) sensed or modeled contexts, e.g.,

GPSstreet … co-located chemicals dangerous!, is always modeled• note: context “ages”, is “probabilistic”/maybe contradictory

(sensor imprecise? calender entry gives different location than GPS sensor?)

• most investigated context: location• may be absolute (“X is room 253”) or relative (“voltmeter is with Mr. X”)• absolute: “2x2” technologies to survive? outdoor GPS & cell?; indoor UWB & IR?• relative: technologies for tags (RFID, RF+, IR, cell, …) will coexist?

B. User Awareness (adapt to user(s) - and, mid term, provider?)• technology? “usual suspects”: user models, profiles&preferences, user

agents• great challenge: the huge crowd of “new” users

• unexperienced, hands/eyes free, little attention• understand their actions support them! [TK]

• for UC: ubiquitous user models? UC components contribute&use only fractions

18

UbiquitousComputing P-3.1 S.C.A.L.E: Liability

Liability :== protection of actors and those concerned by actions (peers, third parties, society) through “right” mix of protection/prosecution – all in presence of zillions of peers; liability = “security and beyond”

A. Scaleable security: old goals (privacy, authentication, …), but new aspects:• machine2machine communication, ad hoc “encounters”: PKI not viable

• cannot check certificate chain with (central!) roots zillions of times per second!• even if so: a party may change (e.g., due to virus) zilliseconds after check• maybe: not always/reliably connected! • early approaches: resurrecting duckling++, TCI, … : viable overall solution?

• there ain’t no end2end encryption (cf. https-connection vs. “frontend” password-spyware)B. Anthropomorphic liability: UC in everyday life conforms with human intuition?

1. intuitive models: “human-understandeable” models of trust, risk, recommendation etc. 2. user friendliness: security measures accepted observed by users?

• novel concepts, e.g.: minimal trusted, concealed entity (“ME”, see below)?3. “Social” compliance: conflicting goals conflicting forces (the core of liability!!)

• e.g., privacy often traded off with public interest, peer’s interest (cf. traceability, non repudiation)• required: max. flexibility, “lever” for tradeoff (different place/time/parties: change mix)

C. Collateral goalsmost important: dependability / reliability / correctness • above terms: overlapping definitions; security added “trustworthiness”• classical dependability: possible in light of zillions of bugs of MS Windows alone?1. Aspect Trusted Computing (TCI): no a priori trust in peers rely on TCI? (doubts!)

• idea: virtual machine layers “underneath my SW” (CPU, Bios, OS, VM) “frozen” via digital signature2. Aspect Accounting & Contracts

• user: I will get what I pay for; provider: I will get paid for what I provide19

UbiquitousComputing P-3.1 S.C.A.L.E: Ease-of-Use

remember: UI-adaptability may be considered a subset of ease-of-use (UI reducted to “what makes sense”)issue: optimal use & combination of modalities, advancement of specific modalities

A. Consider variety of I/O devices in UC world “multimodal interaction”simple distinction: hands&eyes- vs. mouth&ears-interaction 1. advanced hands&eyes interaction: GUIs predominant, but further developments needed

• examples: focus+context-displays, 3rd dimension (VR), 4th dimension (dynamic displays), immersion, narration2. mouth&ears interaction: voice underdeveloped today, great potential! nomadic (hands-/eyes free) operation

• but: needs speech understanding, is transient difficult3. integration of HCI and SWE: the grand challenge!

• today: HCI before & after SWE – but “incompatible”4. multimodality … : “beyond transcoding (program output XML, XSLT produces device-specific XML variant)”

• quest for “abstract interaction layer”: shields modalitity details from “core” program yet leverages off modality advantages• true multimodality: parallel use of modalities, lots of freedom “fusion” of inputs? “fission” of outputs?• integration with other advanced UIs? (“context-sensitive, adapting/proposing to user [AI techniques?])

… & federated devices: optimal use of multiple I/O devices currently surrounding user (includes “liability” issues)

B. Approach human “intuition”: new post-desktop metaphors – a cross cutting concern• post-desktop metaphors: cross cutting wrt. all aspects above

• example: today, eMail folders directories …; tomorrow: all app’s adapt MY favorite structure?• methaphors invented for graphical UIs approproate ones for voice, etc.?

• … and related to scalability: humans must understand scalable = probabilistic behaviour in UC as inherent characteristic• example mass data (cf. DigiCam recordings swamping user disks)• options today:reduce/delete or: classify or: accept chaos• tomorrow: model brain classify / rank automatically! ( prioritize/summarize/reduce; even: forget i.e. delete?)

• Approach ease of human-to-human dialoguesA. natural dialogue: natural language processing NLP intelligent user interfaces

• grammar and semantics – possible for limited domain / world knowledge? simple/full grammar? beyond phrases: understand dialog?

• formal knowledge: structured, semantically annotated documents <-> “informal” knowledge (eMails, Forums, WebSites, Blogs…)

• even worse: electronic communication / informal knowledge ruled by abbreviated expression1. understanding “natural” input

• means: handwriting, speech, gesture, mimics (emotion!, “true” meaning) --- to be combined with NLP 20

UbiquitousComputing

P-3.2 Relation to Computer Science Research

21

S.C.A.L.E.

scalab-ility

connect-ivity

adaptab-ility

liab-ility

ease-of-use

DistributedSystems

IT Security

HCI

ComputerNetworks

Software Engineering

Bionics

Socionics

context awarecomputing

usermodelling

AI (/NLP/KM)

Dependability

Quality-of-Service,SLA, Accounting

Dependability

UbiquitousComputing

P-4.1 SCALE Taxonomy As Introduced

22

S.C.A.L.E.

scaleab-ility

connect-ivity

scaleablecommun-

ication

scaleablecooperation

tech/math

econ-omic

bionics

socionics

wirelessnetworks

pub/sub,eventComm

pub/sub,eventComm

adaptab-ility

liab-ility

ease-of-use

WebSer-vices

Ontologies

Discovery

ServiceDiscovery

contextawareness

userawareness

scaleablesecurity

antropo-morphicliability

collateralgoals

multimodalinteraction

postdesktop

metaphors

humanedialogue

OverlayNetworks

locationawareness

usermodeling

Peer2Peer Grids Opportun-istic Nets

SmartItemsNets

trustedcomputing

accoutning& contracts

intuitivemodels

user friend-ly security

socialcompliance

advancedhands&eyesinteraction

mouth&earinteraction

integrationwith Soft-ware Eng.

multimodal& federatedinteraction

intelligentuser

interfaces

ambientlearning

cooperativeinteraction

under-standingnaturalinput

UbiquitousComputing P-4.1 SCALE Taxonomy vs. Chapters

23

S.C.A.L.E.

scaleab-ility

connect-ivity

scaleablecommun-

ication


tech/math

econ-omic

bionics

socionics

wirelessnetworks

pub/sub,eventComm

pub/sub,eventComm

adaptab-ility

liab-ility

ease-of-use

Ontologies

WebServices

Discovery

ServiceDiscovery

contextawareness

userawareness

scaleablesecurity


collateralgoals


postdesktop

metaphors

humanedialogue

OverlayNetworks

locationawareness

usermodeling


SmartItemsNets

trustedcomputing


intuitivemodels

socialcompliance






intelligentuser

interfaces

ambientlearning



2

3

4 5

6

7

8

9

10

11 12 13

14

15

16 17

18

19

20

21

22

23

24

25

1

26

27

28

UbiquitousComputing P-4.2 Reader’s Digest

24

S.C.A.L.E.

scaleab-ility

connect-ivity

scaleablecommun-

ication


tech/math

econ-omic

bionics

socionics

wirelessnetworks

pub/sub,eventComm

pub/sub,eventComm

adaptab-ility

liab-ility

ease-of-use

Ontologies

WebServices

Discovery

ServiceDiscovery

contextawareness

userawareness

scaleablesecurity


collateralgoals


postdesktop

metaphors

humanedialogue

OverlayNetworks

locationawareness

usermodeling


SmartItemsNets

trustedcomputing


intuitivemodels

socialcompliance






intelligentuser

interfaces

ambientlearning



2

3

4 5

6

7

8

9

10

11 12 13

14

15

16 17

18

19

20

21

22

23

24

25

1

26

27

28

IntroductoryChapters: 1,2,3,7,14,22Sufficient for anOverview of UC; expertsmight want to leave out“their” intro chapter

Readers who want to focus onDistributed Systems: read introchapters + 8-13, add 15-17 and5+6 according to interests

For Ubiquitous Security,read intro chapters andchapters 19-21

For HCI aspects,read intro chapters andchapters 23-28, plus 17,15-16, 4-6 according to interests

Practitioners: read chapters “SAP pilots” chapters at the end of the book

ubiquitous computing max mühlhäuser, iryna gurevych (editors) preface: systematic approach to...

Documents