slides 071031

7/22/2019 Slides 071031

1/23

1

Locative Media

Lalya Gaye

Ubiquitous Computing course

IT-University in Gteborg

31 November 2007

Aims and scope

Overview of the field

Technology overview

Discussion of design and prototyping approaches

Design issues: focus on sustainability in locative media

IntroductionLocative Media Lecture

IntroductionLecture Content

Ubiquitous computing: recap

Ubicomp technologies

Locative Media: definition and origins

Themes, projects and related design issues

Characteristics, challenges and design opportunitiesTechnologies available to the general public

Sustainable Design?

Ubiquitous ComputingRecap

7/22/2019 Slides 071031

2/23

2

Mark Weisers vision (1991)

disappearing computer

everyday world literally used as interface

The most profound technologies are those thatdisappear. They weave themselves into the fabric of

everyday life until they are indistinguishable from it.


The computer: calculator -> information system ->interactive -> pc -> mobile, integrated, networked

Levels of interaction: electrical -> symbolic -> textual ->visual -> social, tangible

Evolution of the user interface: from immersing the userin the computers world to computing increasinglyadapting to the users world and skills.

Ubicomp = opposite of virtual reality: embedded reality.


Evolution of computer-human interaction:

more of the humans everyday world and everydayskills in computing

computers an increased part of our everyday life

requiring less specialised knowledge to operate them

relying increasingly on users everyday skills

smaller computers

from one computer for many user, to many computers


Designing ubicomp systems:

Focus on the interaction between user & technology (asopposed to form and function), on what experience theuser gets from it, on what added-value ubicomp brings tohis/her life.

Follow needs and requirements but also entice new

behaviours?


7/22/2019 Slides 071031

3/23

3

Enhance peoples activities by making computingavailable at hand, when and where needed (includingwhen the users are mobile)

Computing naturally blending into everyday settings,vanishes into the background

The physical and social world around us as digitallyaugmented and distributed interface

Manipulating digital data = manipulating entities in thephysical world

Literally build on peoples everyday use of the physicaland social world, in situation and in real time. Peripheral awareness Greenfield: information processing dissolving into

behaviour IT + everyday life as design material (f. ex. I/O Brush)


Implementing the ubicomp vision: Many interconnected computers per person Mobile devices combined with computers embedded

in the environment (e.g. post-hoc augmentation ofeveryday objects with sensors and networkedcommunication)

With awareness of physical & social context + eachother

-> Mapping the digital world to the physical one-> User interface: tangible and embedded in the real world


Implementing the ubicomp vision: Distributed interface: networking mobile devices and

embedded computers (sensors, processors, etc) ->flexible and seamless integrated whole -> e.g. anydisplay or input device can become ones own (usermobility)

Interaction in context and in real time (f.ex. trackingthings and people -> relevant information andinteraction opportunity to the right person at the righttime)


Types of systems: walk-up-pop-up wearables ambient displays intelligent work environments augmented, interconnected everyday

objects etc


Media cup, TecO

7/22/2019 Slides 071031

4/23

4

Ubiquitous ComputingTechnologies

Ubiquitous Computing (Weiser):computing interweaved in everyday life,where the action is (Dourish)

context awareness

embedded sensor networks

global positioning

wearable computing

augmented & mixed-reality ad hoc and p2p user networks


* Embedded sensor networks

Sensors:

- in everyday environments

- on people

- on artefacts Sensor fusion: combining different data and placements

to gather context


* Context-aware computing

computer-based devices [that] reach out into the realworld through sensors [Gellerson].

A system is context-aware if it uses context to providerelevant information and/or services to the user, whererelevancy depends on the users task. [Dey & Abowd,1999].


7/22/2019 Slides 071031

5/23

5


Enables computing to run into the background and adaptto changes of context in order to present appropriatebehaviour to specific situations. presentation of information and services to a user automatic execution of a service depending on

context appropriateness

or tagging of context to information for later retrieval[Dey].



Gellersen et al.



Gellersen et al.


* Tangible computing

Input, data, output and networking contained andaccessed within the same tangible artefact

Paper, cups, pens, umbrellas or specially designedartefacts

Tangible objects as active entities that respond to theenvironment, to user manipulation and peoples activitiesin general

Building on the users cognitiveabilities


7/22/2019 Slides 071031

6/23

6

* Social computing

Incorporating understandings of the social world intointeractive systems

Social traces left by people on objects or places

Mobile social networks between co-locatedacquaintances

enhancing user awareness by providing them

information about others and their activity


* Augmented reality

Superimposing a digital world upon the real one User experiences both as co-existing parts of the

same reality User is able to interact with their combination in real

time Interfaces:

3D computer graphics seen through transparenthead-mounted displays or augmented glasses

Spatialised audio cues heard through headphones


* Augmented reality

Mixed-reality:digital world not directly overlaid on thephysical one but still presented as part of the samereality, f.ex. with both realities displayed on the screen of hand-

held device)


* Wearable computing

Computing incorporated into clothing

Make use of body-related information or interactionforms to control processes :- body movements- biometrics

Embedded displays (e.g. glasses)


7/22/2019 Slides 071031

7/23

7

* Platforms:

Smart-Its

Smart Dust

Pin & Play

Tiny OS

etc


Smart-Its:

sensors: sound, light, acceleration (2d), pressure

core board: context-recognition, communicationinterface (RF)


Locative Media:Background

Typical contexts of use for ubicomp: home, office work,cafeterias, grad-students research labs, etc

Locative media = media with sense of place

New media + urban aesthetic practices + communityuses of public space + contextual art + mobile,ubiquitous and geographical technologies

City, public spaces Ubiquitous computing in public space:

Minority Report dystopia (video: 44:20) vs. currentcreative uses and appropriations of public space?

Locative MediaBackground

7/22/2019 Slides 071031

8/23

8

Urban aesthetic practices Mobility as creative act Creative use of public space

Walking: aboriginal walkabouts situationist drive, psycho-geography

Locatived MediaBackground

Urban aesthetic practices

Mobility as creative act

Creative use of public space

Graffi ti

Reclaim the Streets

Urban sports:

skateboarding parkour (video)

-> urban space as resourcefor aesthetic movements

Locative MediaBackground

Themes and Projects

Pervasive Gaming: the world as a game-board

Space annotation: media with a specific position inspace

Location awareness & GPS-enabled locative media

Mobile music & locative audio

Radio pirates

Social spaces

etc

Locative Media ProjectsThemes

7/22/2019 Slides 071031

9/23

9

Locative Media ProjectsPervasive Gaming

The world as game-board

Botfighters and Pirates!

Backseat Gaming (video)

Can You See Me Now? (video)

iPerG

...

Locative Media ProjectsPervasive Gaming

Can You See Me Know? Blast Theory + Equator

Media with a specific position in space User-authored social cues

Virtual:Geonotes (video)Urban Tapestries(animations)

Physical:Yellow Arrow (video)Grafedia

Locative Media ProjectsSpace Annotation

Grafedia, grafedia.net

Yellow Arrow, Count Media

GPS-drawing Non-linear narratives:

Hundekopf (video)

Locative Media ProjectsGPS & Positioning

Hundekopf, knifeandfork

Tracking and mapping paths Biomapping (video), Drift, Net_Derive (video)...

Locative Media ProjectsGPS & Positioning

Biomapping, Christian Nold Drift, Teri Rueb

7/22/2019 Slides 071031

10/23

10

Audio space annotation

Mobile music sharing/listening:

- distributed

- ad hoc

- sound walks

Mobile music making:- situated

- collaborative

Wearable audio

Locative Media ProjectsMobile Music and Locative Audio


Hear&There(Rozier, MIT Medialab, 1999)

Tacticle Sound Garden [TSG] (video)(Mark Shepard, Buffalo Univ. 2004-06)

Tejp / Audio tags(PLAY & FAL, 2003-04)



Audio Bombing (video)(Fleming et al., 2007)

Sonic Graffiti (video)(C-Y Lee, 2007)



[Murmur] (murmur.ca)


7/22/2019 Slides 071031

11/23

11

Sound walks

Electric walks (Christina Kubisch) Drift (Rueb) 34n118w(Knowlton, Spellman, 2005)

Craving (Garnicnig, Haider, 2007) Seven Mile Boots (Beloff et al., 2003-04) The Case at Kulturhuset

(Knifeandfork, 2004)

Riot! (Mobile Bristol, Hewlett Packard)


Distributed and located music

Location 33 (Carter & Liu, USC, 2005)


Mobile music sharing

SoundPryer (Mattias stergren, Interactive Institute, 2001)

TunA(Arianna Bassoli et al.,

Medialab Europe, 2002)


Mobile music sharing

Bass Station(Mark Argo & Ahmi Wolf, 2003)

Push!Music(Hkansson et al., 2005)


7/22/2019 Slides 071031

12/23

12

Situated music making

Sonic City (video)(Gaye et al., FAL & PLAY, 2002-04)

Sound Lens(Toshio Iwai, Tokyo Univ.)

Solarcoustics: CONNECT(Barnard, ITP/NYU, 2005)


Situated music making

Sound Mapping (video)(Mott et al., Reverberant, 1997)

Sonic Interface(Akitsugu Maebayashi, 1999)

Warbike(McCallum, 2005-06)

Skatesonic (video) (van Toder, 2006)


Collaborative mobile music making

ImprovE (video)(Wideberg & Hasan, 2006)

CosTune(Nishimoto et al., ATR, 2001)

Malleable Mobile Music(Atau Tanaka, Sony CSL, 2004)


Collaborative mobile music making

China Gates (Clay, Majoe, 2006)

Sequencer404 (Hatcher, Jimison et al., 2006)

Cellphonia (Bull et al, 2006)


7/22/2019 Slides 071031

13/23

13

Wearable audio

Nomadic Radio (Shawney, MIT Medialab, 1998)

Sonic Fabric (Alice Santaro, 2002)


Wearable audio

Personal instruments(Krzysztof Wodiczko, 1969)

(Chelle Hugues, RCA/CRD, 2000)


Wearable audio

Robotcowboy (Wilcox, 2007)

Hearing Sirens (Cathy van Eck, 2007)


Output: Headphones vs boombox vs using everydayobjects

SoundbugTM speakers & piezos

Flower Speakers (LETS corporation, Japan, 2004)


7/22/2019 Slides 071031

14/23

14

Bit Radio(Bureau of Inverse Technology)

7/11 (video)(New Beginnings, Gteborg)

Key Chain Radio Station(Rikako Sakai, Ivrea, 2004)

Locative Media ProjectsRadio Pirates

Hummingbirds

Jabberwocky (video)

MobiTip

Locative Media ProjectsSocial Spaces

Charateristics, Challenges and

Design Opportunities

Interactions happening anywhere, on the move :

taking advantageof the mobile setting: playing withsocial and geographic dynamics implied by mobility

-> outdoors everyday space, locationand socialcontextbecoming resources for interactionas youmove through space

-> spontaneous & situated collaborations with peoplearound or distributed across the city

Characteristics of Locative MediaInteraction Properties

7/22/2019 Slides 071031

15/23

15

Interactions happening anywhere, on the move becoming embedded in the physical and social context

of everyday life-> people managing interaction in heterogeneous

context-> and in simultaneity

with other activities(crossing a street...waiting for the bus...)

tunA, Bassoli et al, Medialab Europe, 2002

Characteristics of Locative MediaInteraction Properties

Usage extended over time and space Ergonomics Same application, many devices Same application, many places Access variability Ad-hoc meetings, windows of opportunity Shifting social roles and contexts

Shifting physical context Heterogeneous environment Scales of interaction Merging digital and physical realms

Characteristics of Locative MediaTechnical Opportunities & Challenges

User-authored content spread across public space:raises questions about property of information privacy & surveillance (loca) spamming?

Augmenting environments and supporting activities withembedded computation: what if it changes what makesthings what they are?

If ubicomp spreads into public space, according towhose will?Top-down corporations, government vs bottom-upcitizens, communities? Conflicts of interests?

Characteristics of Locative MediaDesign Issues

User control (Greenfield): How do you know you areinteracting with a computer if invisible? How do youprotect your privacy? avoid false commands? How doyou know where to look for interaction?

How to query/notify presence, access, place, manipulate

media?

How is the place? Who is there? What activities aregoing on there? How mobile is/are the user(s)? Whatmeaning do the place, activities, and things around haveand for whom?


7/22/2019 Slides 071031

16/23

16

Pro-active and calm computing vs engaging

Ubicomp vs pervasive computing: at hand when neededvs always on everywhere

Connect physical and virtual world: technical and HCIissue but also sociological, aesthetic, even political andenvironmental. F.ex. Yellow Arrow vs Geonotes: physical vs virtual markers

Graffiti style interaction vs screen-based


Enabling technologiesAvailable to General Public

Mobile peer-to-peer

Tracking, positioning and placement

Sensing and data-processing

Content creation and manipulation

Enabling TechnologiesAvailable to the General Public

* Server-Client

* Mobile peer-to-peer:

Bluetooth

WiFi

Infrared


7/22/2019 Slides 071031

17/23

17

* Bluetooth

Standard communication protocol for wireless personalarea network (PANs)

Connect and exchange information (commands, files)between devices

Microwave radio frequency -> non-directional

Short range (power-class-dependent: 1 -10 - 100 m)

Use: BluetunA, bluejacking, Nokias Digidress

Enabling TechnologiesMobile Peer-to-Peer

* WiFi

Wireless local area network

Radio, non-directional

Internet and VoIP phone access, network connectivity forfor consumer electronics, etc

Connect to local access points

Server-client vs ad hoc networks


* Phones vs Wifi-enabled PDAs

Connectivity: closed/open network vs operators

Cost

Range

Distributed vs ad hoc vs server-client

Compatibility Programmability: SDK, OS

Memory, speed


* Platform: Opentrek

http://www.develant.com/opentrek.php

Peer-to-peer networking platform specifically designedfor Wireless Ad Hoc Networks

Cross-platform!

Ad hoc networking -> collaborate


7/22/2019 Slides 071031

18/23

18

* Tracking, positioning and placement

Phone cells

WiFi hotspots

GPS

Virtual media

Physical markers: 2D barcodes, RFID, user ID tophone


* Global Positioning System(GPS)

30 geo-stationary satellites -> location, speed, direction,path

Shadows, accuracy

Use: CYSMN?, GPS drawing, Drift

GPS-enabled phones, PDAs

Platform: Geotracing

http://www.geotracing.com

Enabling TechnologiesTracking, Positioning and Placement

* Geotracing

http://www.geotracing.com


* Placing media: socialight.net

In-place and remote annotationwith smart-phone /PDA

social network community

sound, text, images, video

google maps + GPS


7/22/2019 Slides 071031

19/23

19

* RFID

Radio-frequency identification

Storing and remotely retrieving data

Storage & processing + antenna

Physical markers

Tagging objects

Range: 5-20cm

Passive (powered by inductivitywhen used) vs active RFID


* RFID Uses:

Passports

ransport payments

Product tracking

Automotive

Animal identification

RFID in inventory systems

Human implants RFID in libraries

Controversy: privacy issues.Shielding?


* 2D barcodes

QR (Quick Response) code, Datamatrix code, etc

Physical markers

Can store between one and 500 characters

Tag objects, places Scan with cameraphones

-> hyperlink (physical mobile interaction)

How to: Kaywa reader http://reader.kaywa.com/ +generator: http://qrcode.kaywa.com/


* Unique ID to phone

Physical markers with unique IDs

Tag objects, places

Send number to server-> store & retrieve media

Arrows available, but notID generator


7/22/2019 Slides 071031

20/23

20

* Sensing:

sensors

data processing: microcontrollers


* Micro-controllers Basic Stamp II, Basic X 24 http://www.basicx.com

Tutorial: http://www.tigoe.net/pcomp/index.shtml Arduino

open source hardware physical computing I/Oplatform

cheap (20 Euro) easy (Processing)

assemble yourself stand-alone or connect tocomputer (MAX/MSP, etc)

www.arduino.cc

Enabling TechnologiesSensor Data Processing

* Creating and manipulating content:

Mobile Processing

Python

J2ME

miniMIXA PdA (Pd on PDAs, linux)

Keyworx


* Mobile Processing

http://mobile.processing.org Open source programming environment for design and

prototyping software for mobile phones. Similar to Processing environment.

Runs on Java powered mobile devices. Bluetooth -> communication Control example: attach light sensor on screen so

sending info from phone to laptop

Enabling TechnologiesCreating and Manipulating Content

7/22/2019 Slides 071031

21/23

21

* MiniMIXACommercial DJ software for mobile phones, PDAshttp://www.youtube.com/watch?v=R6BSGy8mMsU

* KeyworxMultimedia platform (base for GeoTracing f.ex.)

http://www.keyworx.org/

* PDa (Puredata anywhere): Pd for Linux on PDAshttp://gige.xdv.org/pda/


* Python PyS60

Interactive object-oriented language Nokia S60 phones and more Record, playback, play MIDI notes, control MAX/MSP

patches... http://www.python.org/ PyS60: http://www.forum.nokia.com/python and

http://www.mobilenin.com/pys60/menu.htm

Tutorial (Jrgen Scheible - Mobilenin)


3rdparty software (Java, etc)

Hacking hardware: use camera, microphone, speakers,audio out...

Enabling TechnologiesHacking mobile phones

Sustainable Design?

7/22/2019 Slides 071031

22/23

22

Problem in particular with Ubicomp: technology spreadeverywhere

Production, use, reuse, disposal Use of energy + where to get it from? Computers get smaller but not batteries Issues with spreading technology into the wild: not as

controlled environment as homes or offices Littering: what happens to the embedded technology

after use or break-down? who is

responsible/accountable ? Physical & virtual littering? Peak oi l!

Sustainable Locative Media?Issues

Recycling? Use of existing material and sources of energy? Biodegradable material, f. ex. paper markers? The simpler the better? Wearability? When should power be on? How should the system

know when it should be on/off?

Sustainable Locative Media?Possible Approaches

* Hacking

Repurposing existing technology

Sustainable Locative Media?Design Inspirations

* Parasating?

Re-using existing features and properties of space andsources of energy in the environment: power, airflow,conductivity, etc.

paraSITE

Glitch (Tejp)


7/22/2019 Slides 071031

23/23

23

* Body-generated energy?

steps, body-heat, etc Humand-Powered Objects Workshop:

Bike4Tea, DynamoMouse...


* Ephemeral computing (Jernstrm)?

Deploying and packing up temporary and re-usableubicomp infrastructures

SiSSy (video)


Resources:http://www.cs.chalmers.se/idc/ituniv/kurser/07/uc/locmedia/

slides 071031

Documents