augmented reality presentation in icists 2011

Going Out Research in Mobile Augmented

RealityMark Billinghurst

HIT Lab NZ University of Canterbury

Augmented Reality (Azuma 97)

Combines Real and Virtual Images- Both can be seen at the same time

Interactive in real-time- The virtual content can be interacted

with Registered in 3D

- Virtual objects appear fixed in space

1999: AR Face to Face Collaboration

Siggraph 99 Demo

$5,000CPU: 300 MhzHDD; 9GBRAM: 512 mbCamera: VGA 30fpsGraphics: 500K poly/sec

1998: SGI O2 2008: Nokia N95

$500CPU: 332 MhzHDD; 8GBRAM: 128 mbCamera: VGA 30 fpsGraphics: 2m poly/sec

Evolution of Mobile AR

Wearable AR

Handheld AR Displays

Camera phone

1995 1997 2001 2003 2004

Camera phone- Self contained AR

WearableComputers

PDAs-Thin client AR

PDAs-Self contained AR

Camera phone- Thin client AR

Mobile Phone AR Mobile Phones

camera processor display

AR on Mobile Phones Simple graphics Optimized computer vision Collaborative Interaction

Collaborative AR

AR Tennis Two user game Audio + haptic

feedback Bluetooth

messaging

AR Tennis

Mobile AR

Txt message to download AR application (200K) See virtual content popping out of real paper

advert Tested May 2007 by Saatchi and Saatchi

Mobile AR: Touring Machine (1997)

University of Columbia Feiner, MacIntyre, Höllerer, Webster

Combines See through head mounted display GPS tracking Orientation sensor Backpack PC (custom) Tablet input

MARS View

Virtual tags overlaid on the real world “Information in place”

2008 - Location Aware Phones

Nokia NavigatorMotorola Droid

Real World Information Overlay Tag real world locations

GPS + Compass input Overlay graphics data on live video

Applications Travel guide, Advertising, etc

Eg: Mobilizy Wikitude Android based, Public API released

Other companies Layar, AcrossAir, Tochnidot, RobotVision,

etc

Layar – www.layar.com iPhone, Android > 2 million downloads 1500+ information layers

HIT Lab NZ Outdoor AR Platform

Cross platform Android, iPhone

3D onsite visualization Intuitive user interface

Positions content in space Camera, GPS, compass

Client/Server software architecture Targeting museum guide/outdoor site

applications

Prototype: Earthquake Reconstruction

See past, present and future building designs

Earthquake survivor stories shown on map view

Collect user comments Android platform

Benefits of Mobile AR Making the invisible visible

Geo-located services/information Ease interaction with virtual content

Tangible/physical interaction Reduce cognitive load

Intuitive spatial organization of information

Mobile AR Today Smart Phones (450+ million in 2011)

Cameras, GPS, Compass, Gyroscope Sensors

Fast CPU, GPU chips Free developer kits

Layar, Junaio, outdoor SDKs Qualcomm QCAR vision tracking SDK

User-generated content Flickr, Sketch-up, YouTube, Twitter, etc

$784 million USD in 2014

Looking to the Future

Directions for Future Research Enabling Technologies

Displays, tracking, information filtering User Experience

Remove social boundaries, increasing ease of use

Crossing Boundaries AR + other interface metaphors

Social Augmented Reality AR 2.0, AR everywhere

Tri-corder Problem Handheld AR

Social boundaries Not always available

Future Displays

Always on, unobtrusive

Contact Lens Display Babak Parviz

University Washington MEMS components

Transparent elements Micro-sensors

Challenges Miniaturization Assembly Eye-safe

AR User Experience

Crossing Boundaries

Jun Rekimoto, Sony CSL

Invisible Interfaces

Jun Rekimoto, Sony CSL

The MagicBook

Reality VirtualityAugmented Reality (AR)

Augmented Virtuality (AV)

ubiHome @ GIST

ubiHome

What/When/How

Where/When

Media services

Who/What/When/How

ubiKey

Couch SensorPDA

Tag-it

Door Sensor

ubiTrack

When/HowWhen/HowWho/What/When/How

Light service MR window

CAMAR - GIST

(CAMAR: Context-Aware Mobile Augmented Reality)

Ubiquitous AR (GIST, Korea)

How does your AR device work with other devices?

How is content delivered?

Reality Virtual Reality

Terminal

Ubiquitous

Desktop AR VR

Milgram

Weiser

UbiComp

Mobile AR

Ubi AR

Ubi VR

From: Joe Newman

Reality

VR

Ubiquitous

Terminal

Milgram

Weiser

Single User

Massive Multi User

Multi-user AR Interfaces

10 years of Collaborative AR – 2 to 8 users cf MSN Chat 29m, Skype 17m, Second Life 50K

Mobile/handheld AR scales to high number of users New applications, infrastructure, content

distribution…

19962 users

19994 users

20048 users

2018?? users

Social Augmented Reality

Public and private annotations everywhere

BASIC VIEW

PERSONAL VIEW

Augmented Reality 2.0 Infrastructure

Social Implications Privacy

Who can see my information? Reliability (Wikipedia effect)

How truthful is the information? Just in time education

Information without learning? Attention Becomes a Valuable

Commodity Hyperconnectivity = time slicing

Conclusions• Mobile AR enhances interaction with real

world• Many possible applications/commercial

possibilities• Important research problems need to be

solved–Enabling Technologies–Experience Design–Crossing Boundaries–Social Augmented Reality

More Information

• Mark Billinghurst– mark.billinghurst@hitlabnz.org

• Website– www.hitlabnz.org