Ubiquitous Computing

Lecture 28

Topics we shall Cover today Introduction to Ubiquitous Computing

HistoryDefinition NeedPhases

Challenges and Researches in Ubiquitous Computing.

2

The Trends in Computing Technology

1970s

1990s

Late 1990s

Now and Tomorrow ? 3

Pervasive Computing Era

4

Computing Evolution

5

The Major Trends in Computing

Mainframe (Past) 1:N one computer shared by many people

Personal Computer (Present) 1:1 one computer, one person

N:1 *Internet - Widespread Distributed Computing*

Ubiquitous Nk:1

Computing

6

Phase I - The Mainframe Era

Computers were a scarce resource run by experts behind closed doors.

7

8/53

Phase II - The PC Era

In 1984 the number of people using PCs surpassed that of people using mainframe computers.

PC Era: You have your computer, it contains your stuff, and you interact directly and deeply with it.

The PC is most analogous to the automobile.

Transition Phase - The Internet

The Internet brings together elements of the mainframe

era and the PC era.

Client = PC

Server = Mainframe

9

10/53

Phase III - The UC Era The UC era will have lots of computers

shared by each one of us. UC is fundamentally characterized by the

connection of things in the world with computation.

Frequently used related terms: Pervasive computing, Wearable computers, Intelligent environment, Things That Think (T³),Wearware, Personal Area Networking (PAN).

Ubiquitous Computing Mark Weiser, Xerox PARC 1988 “Ubiquitous computing enhances

computer use by making many computers available throughout the physical environment, but making them effectively invisible to the user.”

Source: Weiser, 1993a11

Pervasive (Ubiquitous) Computing Vision

“In the 21st century the technology revolution will move into the everyday, the small and the invisible…”

“The most profound technologies are those that disappear. They weave themselves into the fabrics of everyday life until they are indistinguishable from it.”

Mark Weiser (1952 –1999), XEROX PARC

Small, cheap, mobile processors and sensorsin almost all everyday objectson your body (“wearable computing”)embedded in environment (“ambient intelligence”) 12

Related Topics Several terms that share a common vision

Pervasive Computing Sentient computing Ubiquitous Computing Ambient Intelligence Wearable Computing Context Awareness ...

13

What is Ubiquitous Computing? Ubiquitous computing (ubicomp) integrates

computation into the environment, rather than having computers which are distinct objects.

The idea of ubicomp enable people to interact with information-processing devices more naturally and casually, and in ways that suit whatever location or context they find themselves in.

~from Wiki

14

Goals of Pervasive (Ubiquitous) Computing Ultimate goal:

Invisible technology Integration of virtual and physical worldsThroughout desks, rooms, buildings, and lifeTake the data out of environment, leaving

behind just an enhanced ability to act

15

What UC is NOT

It is not science fiction (SF),though it relies a great deal on it.

It is not impossible. It is not Virtual Reality (VR). It is not a Personal Digital Assistant (PDA). It is not a personal agent (PA).

16

Phases of Ubiquitous Computing

Pervasive Computing Phase I Phase I

Smart, ubiquitous I/O devices: tabs, pads, and boards Hundreds of computers per person, but casual, low-

intensity use Many, many “displays”: audio, visual, environmental Wireless networks Location-based, context-aware services

Using a computer should be as refreshing as a walk in the woods

18

Smart Objects

Real world objects are enriched with information processing capabilities

Embedded processors in everyday objects small, cheap, lightweight

Communication capability wired or wireless spontaneous networking

and interaction

Sensors and actuators

19

Smart Objects (cont.) Can remember pertinent events

They have a memory

Show context-sensitive behavior They may have sensors Location/situation/context

awareness

Are responsive/proactive Communicate with environment Networked with other smart objects

20

Smart Objects (cont.)

21

Pervasive Computing Enablers Moore’s Law of IC Technologies

Communication Technologies

Material Technologies

Sensors/Actuators

22

Moore’s Law

1

10

100

1000

10000

100000

1000000

10000000

100000000

1000000000

10000000000

100000000000

2 3 5 6 8 9 11 12 14 15 17 18 20 21 23 24 26 27 29 30 32 33 35 36 38 39 41 42 44 45 47 48 50 51 53 54

Years

Po

wer

Computing power (or number of transistors in an integrated circuit) doubles every 18 months

23

Moore’s Law

Computing power (or number of transistors in an integrated circuit) doubles every 18 months

0

10000000000

20000000000

30000000000

40000000000

50000000000

60000000000

70000000000

80000000000

2 5 8

11 14 17 20 23 26 29 32 35 38 41 44 47 50 53

Years

Pow

er

1965

24

Generalized Moore’s Law Most important

technology parameters double every 1–3 years:computation cyclesmemory, magnetic disksbandwidth

Consequence:scaling down

Problems:

• increasing cost

• energy

25

2nd Enabler: Communication Bandwidth of single fibers ~10 Gb/s

2002: ~20 Tb/s with wavelength multiplex Powerline coffee maker “automatically” connected to the Internet

Wireless mobile phone: GSM, GPRS, 3G wireless LAN (> 10 Mb/s) PAN (Bluetooth), BAN

26

Body Area Networks Very low current (some nA), some kb/s

through the human body Possible applications:

Car recognize driverPay when touching

the door of a busPhone configures itself

when it is touched

27

Spontaneous Networking Objects in an open, distributed, dynamic

world find each other and form a transitory communityDevices recognize that

they “belong together”

28

3rd Enabler: New Materials Important: whole eras named after materials

e.g., “Stone Age”, “Iron Age”, “Pottery Age”, etc.

Recent: semiconductors, fibers information and communication technologies

Organic semiconductors change the external appearance of computers

“Plastic” laser Flexible displays,…

29

Interactive Map Foldable and rollable

You are here!

30

Foldable Cell Phone

31

Smart Clothing Conductive textiles and inks

print electrically active patterns directly onto fabrics

Sensors based on fabric e.g., monitor pulse, blood

pressure, body temperature Invisible collar microphones Kidswear

game console on the sleeve? integrated GPS-driven locators? integrated small cameras (to

keep the parents calm)?

32

Solar Coat – Smart Clothings

33

Smart Glasses By 2009, computers will disappear. Visual

information will be written directly onto ourretinas by devices inour eyeglasses andcontact lenses-- Raymond Kurzweil

34

Google Glass

35

4th Enabler: Sensors/Actuators Miniaturized cameras, microphones,... Fingerprint sensor Radio sensors RFID Infrared Location sensors

e.g., GPS ...

36

Example: Radio Sensors No external power supply

energy from theactuation process

piezoelectric andpyroelectric materialstransform changes inpressure or temperatureinto energy

RF signal is transmitted via an antenna (20 m distance)

Applications: temperature surveillance, remote control (e.g., wireless light switch),...

37

RFIDs (“Smart Labels”) Identify objects from distance

small IC with RF-transponder Wireless energy supply

~1m magnetic field (induction)

ROM or EEPROM (writeable) ~100 Byte

Cost ~$0.1 ... $1 consumable and disposable

Flexible tags laminated with paper

38

Past, Present and Future Researches of Ubiquitous Computing

Computing with natural interfaces

Ubicomp inspires “off-the-desktop” applications Needs “off-the-desktop” means of interaction Speech, gestures, writing

More accessible Easier to use???

40

Computing with natural interfaces

Error prone interaction Permit new and numerous mistakes People do not have perfect recognition

As low as 54%; cursive handwriting 88%; printed handwriting 96.8%

Recognition accuracy == user satisfaction?? Not really: complexity of error recovery dialogues and value-

added benefit of any given efforts Entering a command vs. writing journal entries

Several research areas Error reduction (about 5-10%) Error detection Reusable toolkit for error handling

41

Context aware computing Current Systems

Generally using position and identification of objects Still do not provide a complete context Definition of context is limited

Research areas Context toolkits

Toolkit for sensing environment Explicit use of sensed information is up to program

What is context? How is context represented?

42

What is context? Who

Currently generally tailored to one user How important are others in determining our behavior

How could this be captured? What

Attempt to figure out what is currently happening Sense environment, use calendar software etc.

Where Location based information, e.g., GPS Most explored context information

When Easily obtained information -- Computer is good at remembering time

Although determining when one event stops and another begins is not easy Why

Even harder than the “what” question, biometric sensors might help (e.g., body temperature, heart rate, etc)

43

Toward context aware computing Context representation

Requires universal context schemes or toolkits with standard context representations

Context sensing and fusion How to make context-aware computing “ubiquitous”? In practice, there are few truly ubiquitous, single-source

context services E.g., GPS does not work indoors; different indoor localization

schemes have different characteristics (e.g., cost, range) Like sensor fusion, context fusion handles seamless

handling of sensing responsibility between boundaries of different context services

Combining multiple context sources can increase the accuracy of context information

44

Automated capture and access Recording information and data as it occurs

Computers are inherently good at recording, people are not People freed up to summarize and understand Most work in academic/ classroom settings

Time stamping lectures, digital whiteboards Challenges in “capture and access”

Sometime we don’t know we want to capture something until after its already happened

How could the computer know that? If it captures everything then we need a system of sorting and filtering

(access) Access is a problem because capturing of raw data can be

burdensome for sifting through; systems need to recognize important events facilitate access

45

Everyday computing Continuous interactions (i.e., no clear beginning or end)

Both fundamental activities like communication and long-term endeavors do not have predefined starts and ends; information from past can be recycled

Very different traditional HCI design which assumes “closure” with clear goals like spell checking, dialogue, etc.

Interruption is expected: People are constantly interrupted Computer systems must recognize interruption and change state

Also computers must appropriately inform users

Multiple activities operate concurrently: People multitask and rapidly switch task based on external

unpredictable environment Systems need to adapt to this opportunistic behavior and change

accordingly

46

Toward everyday computing Develop continuously present interface

No current model of continuously present interfaces, even people are not continuously present

Create an interface that doesn’t get annoying (e.g., wearable devices)

Determine what information should require my attention and what should be display peripherally

Connect events in the physical and virtual worlds (e.g., face to face vs. email, document, webs)

Modify/fuse existing HCI schemes to efficiently support everyday computing (but evaluation is challenging and laborious)

47

System evaluation challenges Hard to evaluate Ubicomp Systems

Little publish on ubicomp evaluation Systems often required to be fully connected leading to systems

that are hard to build Lack of development toolkits make system creation difficult Systems often need to be integrated into peoples lives which

using big clunky prototypes does not lead itself well too Task/Goal centric approaches don’t work in ubicomp

48

Example Projects Pervasive computing projects have emerged at major

universities and in industry: Project Aura (Carnegie Mellon University) Oxygen (Massachusetts Institute of Technology) Portalano (University of Washington) Endeavour (University of California at Berkeley) Place Lab (Intel Research Laboratory at Seattle)

For illustration let us look at Project Aura

49

Example Projects : Project Aura (1)

Aura (Carnegie Mellon University) Distraction-free (Invisible) Ubiquitous Computing.

50

Example Projects : Project Aura (2)

Moore’s Law Reigns Supreme Processor density Processor speed Memory capacity Disk capacity Memory cost ...

Glaring Exception Human Attention

Adam & Eve 2000 AD

Human Attention

51

Example Projects : Project Aura (3)

Aura Thesis: The most precious resource in computing is human attention.

Aura Goals: Reduce user distraction. Trade-off plentiful resources of Moore’s law for human attention. Achieve this scalably for mobile users in a failure-prone,

variable-resource environment.

52

Example Projects : Project Aura (4)

The Airport Scenario

Jane wants to send e-mail from the airport before her flight leaves.

She has several large enclosures She is using a wireless interface

She has many options. Simply send the e-mail

Is there enough bandwidth? Compress the data first

Will that help enough? Pay extra to get reserved bandwidth

Are reservations available? Send the “diff” relative to older file

Are the old versions around? Walk to a gate with more bandwidth

Where is there enough bandwidth?

How do we choose automatically?53

Example Projects : Project Aura (5)

The Mobile Task Scenario

Aura saves Scott’s task. Scott enters office and gets strong

authentication and secure access. Aura restores Scott’s task on desktop

machine and uses a large display. Scott controls application by voice. Bradley enters room. Bradley gets weak authentication,

Scott’s access changes to insecure. Aura denies voice access to sensitive

email application. Scott has multi-modal control of

PowerPoint application. Aura logs Scott out when he leaves the

room.

54

Other Scenarios of Ubiquitous Computing Buy drinks by Friday (1)

Take out the last can of soda

Swipe the can’s UPC label, which adds soda to your shopping list

Make a note that you need soda for the guests you are having over this weekend

55

Other Scenarios

Buy drinks by Friday (2) Approach a local supermarket

AutoPC informs you that you are near a supermarket

Opportunistic reminder: “If it is convenient, stop by to buy drinks.”

56

Other Scenarios

Buy drinks by Friday (3)

- Friday rolls around and you have not bought drinks

- Deadline-based reminder sent to your pager

57

Other Scenarios

Screen Fridge Provides:

Email Video messages Web surfing Food management TV Radio Virtual keyboard Digital cook book Surveillance camera

58

Other Scenarios

The Active Badge This harbinger of inch-scale computers contains a small

microprocessor and an infrared transmitter.

The badge broadcasts the identity of its wearer and so can trigger automatic doors, automatic telephone forwarding and computer displays customized to each person reading them.

The active badge and other networked tiny computers are called tabs.

59

Other Scenarios

The Active Badge

60

Other Scenarios

Edible computers: The pill-cam Miniature camera Diagnostic device It is swallowed

Try this with an ENIAC computer!

61

Other Scenarios

Artificial Retina Direct interface with

nervous system

Whole new computational paradigm (who’s the computer?)

62

Other Scenarios

Smart Dust Nano computers that couple:

Sensors Computing Communication

Grids of motes (“nano computers”)

63

Summary Moving our focus of interaction away from the

traditional two-dimensional graphical user interface on the desktop presents many exciting and new challenges to the field of HCI.

Weiser’s vision of ubiquitous computing was human-centered, and many years later, it still presents a grand challenge for those who wish to address this new interaction paradigm.

64