software development forum emerging technology sig weds. sept 9 2009 augmented reality clark...
Post on 15-Jan-2016
217 Views
Preview:
TRANSCRIPT
Software Development ForumEmerging Technology SIG
Weds. Sept 9 2009
Augmented Reality
Clark DodsworthOsage Associates
clark@dodsworth.com
Maribeth BackFX Palo Alto Laboratory
back@fxpal.com
Theoretical foundations
Scope
My background
Brief AR history
Directions for development, revenue + evolution
Maribeth Back, FX-PAL:
“High Tech Chocolate: Exploring Mixed Reality applications for Industry”
Ribbit.com demo by Kristi Wells and Brendan Lee
Tonight’s Plan
Augmenting Reality and Revenue Connecting Mobile, Sensors, Location & Layers
Clark Dodsworthclark@dodsworth.com
Alfred North Whitehead,first augmented reality theorist:
#1: “Civilization advances by extending the number of important operations which we can perform without thinking of them.” ...effective AR doesn’t increase the cognitive load
#2: We think in generalities, but we live in detail.” ...AR tasks: managing detail
#3: “The art of progress is to preserve order amid change and to preserve change amid order.” ...contextually dynamic data management & display via AR
Scope - General
AR = subset of Ubiquitous Computing: same infrastructure and issues as UbiComp and its personalization.
AR = enhance personal awareness (as needed) of: physical & non-physical context, personal state, opportunities & threats...local & distant.
AR = deliver personalized info with less effort, more space & time precision, and fewer seams than before, without increasing distraction.
Scope - General
AR = subset of Ubiquitous Computing: same infrastructure and issuesas UbiComp and its personalization.
AR = enhance personal awareness (as needed) of: physical & non-physical context, personal state, opportunities & threats...local & distant.
AR = deliver personalized info with less effort, more space & time precision, and fewer seams than before, without increasing distraction.
Augmented Context + Information Salience = Value
Scope - Model
1st plateau of AR = augmented perception of internal, external, & distant reality. Smartphone and headset AR that’s non-intrusive.
Beyond that, the value of well-evolved AR has significant parallels to the value of consciousness:
2nd plateau of AR = augmented decision-making. Integrated analysis, evaluation, and management (informatics) of data streams, constantly and dynamically evaluating the salience of each stream in context of the user’s ever-changing current situation and the user’s historical paths through n dimensions of data.
Then displaying the most salient data...when appropriate: Contextual evaluation driving contextual augmentation; software ‘awareness’ augmenting the decision process via delivery of filtered salient aspects of the user’s situation. At least 2 generations further.
Scope - Model
The value of well-evolved AR has significant parallels to the value of consciousness, AKA:
...the Default Mode Network of the brain
• One of the hardest-working systems in the brain
• Active when the brain is aware, but not focused on a particular task
• Reduces activity when you concentrate on a specific task
• 2 linked brain areas: A) Imagination, empathy for the thoughts and intent of othersB) Personal memories, visualization of future actions, scenarios
Scope - Model
The value of well-evolved AR has significant parallels to the value of consciousness, AKA:
...the Default Mode Network of the brain
• One of the hardest-working systems in the brain
• Active when the brain is aware, but not focused on a particular task
• Reduces activity when you concentrate on a specific task
• 2 linked brain areas: A) Imagination, empathy for the thoughts and intent of othersB) Personal memories, visualization of future actions, scenarios
DFM = constant process of considering how to be & what to do next.
Getting and dealing with the data upon which to make all those decisions is the constant background task of 2nd-plateau AR apps.
Scope - Phone
AR hot again as part of the smartphone ecosystem.
Smartphone AR (SAR?) = low-hanging fruit. 2nd is games.
SAR & LBS* = features, not an industry. Startups will fail/be acquired.
Privacy / security issues will be more extreme than currently.
Acronym Fusion: AR + LBS + GIS + Gesture = POTS... “my computer”
*LBS = Location-Based Services
Scope - farther
AR hot again as part of the smartphone ecosystem.
Smartphone AR (SAR?) is the low-hanging fruit.
SAR & LBS = features, not an industry; startups will fail / be acquired.
Privacy / security issues will be more extreme.
Acronym Fusion: AR + LBS + GIS = POTS... “my computer”
After that: Contextual Personalization + More Sensors + onboard DBs + Projectors
Background
My work Product strategy, dev., design, improvement, & IP evaluation, all based on User-Centered Design. For emerging technologies and markets in software, consumer electronics, and entertainment.
My perspective “Product as Tool” Tool should adapt to (fit) the user and be its own user manual.
Scythes are an excellent model system for tool design, though we don’t have ~2000 years to iterate.
AR-related projects:
VideoCart, ‘92: Location-aware mobile retail touch-screen kiosk system with data-driven path mapping.
Live Wall, ‘98: 100-meter, rear-projection, 1:1 scale videophone.
“Ambient Intelligence” strategy,’98 for Philips Consumer Electronics.
Royal Tyrell Museum, Alberta, ‘06: Concept & spec: permanent LBS-based virtual paleontological exhibit complex across 11 remote
archaeological sites, for Reich + Petch.
Dubailand theme parks, ‘08: Concept evaluation & high-level system spec, strategic & tech partners: 24/7 multi-application smartphone AR experience.
Background
VideOcart
VideoCart: mobile location-aware touchscreen retail, ‘92
Live Wall: semi-mobile videophone design for Philips, ‘98
“Ambient Intelligence” strategy for Philips, ‘98
all photos: Philips
services across all hotel, leisure, retail, restaurant, and transit in Dubailand, and to
follow the visitors home.
Dubailand AR project, ‘08
• Concept > smartphone AR • Technology identification, assessment, & qualification • High-level system definition • Strategic partnering • Broadened scope to include
c. Lynne LaCascia, all rights reserved
Part of Dubailand model:
Dubailand AR project, ‘08
c. Lynne LaCascia, all rights reservedAll images © Chimera Design 2007-2008
AR Timeline
Sailboat tell-tales
~1900
Rear-view mirrors
1st jet fighter heads-up display
1958 1970s
Commercial aviation use
Helmet-mounted displays
“Augmentedenvironments,” sensor nets
1974-8
US DoD air-combat testing
1988
1st car HUDOlds Cutlass Supreme
Now 20??
Battery energy-density resolved
1992+ 1999
Horsecarriages
Tom Caudell &David Mizell
ARToolkitUW HITLab
Prehistory,soon after invention of the sail.
Dedicated tools begin:
“UbiComp”:Mark Weiser
2006
Useful smart- phone AR
Nokia’s MARA
Hands-up displays
Reed sails on boats in the prehistoric Middle East had inherent sail-edge tell-tales.
Example of sensor/display fusion
photo c. Luxury Peru Tours & Travel
Marmon Wasp racecar, 1911
Non-see-through HUD display: Rear-view mirror is just above line of sight
MIG21 heads-up display (installed): circa 1960s
Cintel “Buccaneer” aircraft cockpit heads-up display device, UK 1958
photo: Rochester Avionics Archives, UK
photo: Will Hise cc
HUDs
HUD: modern commercial aviation AR
HMDs Helmet-mounted displays
• High-priority military R&D investment
• Tethered to power and computer
• Obstructs part of the field of view
• Delicate to align, adjust, maintain
• Size and weight unacceptable
• Adds complexity/distraction to user situation
• Current state of the art = net loss of context awareness
c. Michael Edwards,2007
Rockwell-Collins Sim Eye SR100A, 2009
HMDs Helmet-mounted displays
• High-priority military R&D investment
• Tethered to power and computer
• Obstructs part of the field of view
• Delicate to align, adjust, maintain
• Size and weight unacceptable
• Adds complexity/distraction to user situation
• Current state of the art = net loss of context awareness
Not AR
c. Michael Edwards,2007
Rockwell-Collins Sim Eye SR100A, 2009
photo: Mobilizy GmbH
Mobilizy's original product, “Wikitude AR (Travel) Guide,” for Android
Hands-Up Displays
Some Players
Mobilizy • Wikitude World Browser, • new Wikitude Drive (Android)
Nokia • Point & Find, Image Space, MARA
Layar • Layar Reality Browser 2.1 (Android) • Enables adding layers easily • Example: Trulia.com • Working on iPhone dev now
Studierstube ES • Rich development framework • Client and middleware • 3D object tracking
Copyright Network World, Inc. All rights reserved.
photo: Mobilizy GmbH
Total Immersion • D’Fusion Studio, Pro, Mobile
Metaio • Unify Design: no programming
Tonchidot • Sekai Camera (iPhone & Android)
Augmento
....
Studierstube ES
Copyright Network World, Inc. All rights reserved.
photo: Mobilizy GmbH
Now
• GPS• Accelerometers• Light sensors• Proximity sensors• Compass• Touch (2- or 2.5-D)• Bluetooth
Need
• Near-field RFID (“touch,” “wand”)
• Constant visual awareness + recording
• Constant audio awareness + recording
• Galileo-grade location resolution
• Ultrasonic & IR spatial analysis
• Laser projector(s) with geometric awareness & beam stabilization for object augmentation
• ...
Hardware
Next---1. Constant, robust markerless 2D & 3D feature I.D. and object recognition (Nokia Point & Find, ...) as a general
feature
2. Motion analysis + evaluation linked to the object recognition
3. General-purpose indoor position sensing
4. More nuanced gestural input interpretation + integration with voice
5. Constant audio + visual awareness with flagging & interpretation
1. Allostatic control of data delivery: apps “learn the user” over time, to drive what to display (‘awareness driving awareness,’ ‘context drives the augmentation of context’)
1. Constant, opportunistic acquisition and evaluation of available sensor-net feeds.
Software
Still further out...
Rob Cook’s award talk at Siggraph ‘09:
“...the next holy grail for the computer graphics community should be realist augmented reality.”
http://media.siggraph.org/s2009podcasts/RobCook_AwardExport.mov
Software
Augmenting Revenue
Assumptions• Full OS citizenship: no tethering to a computer • More coverage & bandwidth• CPU & memory follow Moore’s Law• Something magic happens to battery energy density.
Work• Extend existing software onto the mobile platform with AR where
appropriate. Includes Augmenting Security for public locations. • Building AR into new apps as a normal part of software develop-
ment: Like the shift to web from desktop & enterprise.• Multi-modal interface differentiation:
Handheld vs. plugged into a desktop screen or large projector
Platforms and toolkits
Platforms iPhone 3.9 Android 1.5 Palm WebOS RIM BlackBerry OS Symbian S60 5th Edition Windows Mobile 6.5 (Samsung LinuxOS?)
Tools Studierstube ES Layar ARToolkit NYARToolkit FLARToolkit Tinmith ...
clark@dodsworth.com
end OA logo
AR-related conferences
Ubicomp 2009, Orlando, FL. Sept. 30-Oct. 3, 2009 www.ubicomp.org/ubicomp2009/ Int’l Symposium on Mixed and Augmented Reality (ISMAR09) - Orlando, FL, October 19-22, 2009 http://ismar09.org 8th Int’l Conference on Mobile and Ubiquitous Multimedia (MUM 2009) 22-25 Nov., Cambridge, UK www.mum2009.org8th IEEE Int’l Conf. on Pervasive Computing & Communications (PerCom) Mannheim (Mar 29-April 2, 2010)
www.percom.org7th IEEE Workshop on Ubiquitous Communications & Services: www.ubiquitous-management.org/mucs/2010/index.phpIntSymposium on Wearable Computers (ISWC'09) - Linz, Austria, September 4-7, 2009. www.iswc.net/register
IEEE VR March 20-26, 2010 in Waltham, Massachusetts http://conferences.computer.org/vr/2010/IEEE Symposium on 3D User Interfaces (3DUI), March 20-21 2010, Waltham, Mass
http://conferences.computer.org/3dui/3dui2010/Symposium on Haptic Interfaces March 25-26 2010, Waltham, Mass., www.hapticssymposium.org/next_conference.htmInt’l Conference on Mobile, Wireless and Optical Communication (MWOC 2009), Dec. 09 Korea
www.iacsit.org/mwoc/index.htm Interservice/Industry Training, Simulation and Education Conference (I/ITSEC) Nov. 30-Dec. 3 Orlando, FL www.iitsec.org/SVC Wireless Annual Conference Sep 25-26, 2009, Mountain View www.svcwireless.org/svcw/ac2009-homeLocation Intelligence Conference 2009 Oct 5-7 2009, Broomfield, CO www.locationintelligence.net/3rd Int’l Conference on Mobile Ubiquitous Computing, Systems, Services & Technologies UBICOMM October 11-16, 2009 -
in Sliema, Malta www.iaria.org/conferences2009/UBICOMM09.htmlMobiquitous 2010 www.mobiquitous.org/2nd ACM Int’l Workshop on Mobile Opportunistic Networking Feb. 22-23, 2010, Pisa, http://cnd.iit.cnr.it/mobiopp20102nd Int’l Conference on Mobile Lightweight Wireless Systems (MOBILIGHT 2010) www.mobilight.org5th Int’l Conference on Body Area Networks September (BodyNets) Sept. 10-12, 2010 Corfu, Greece www.bodynets.org/IMMERSECOM 2010 http://immerscom.org/ (2009 was at Berkeley)1st Int’l ICST Conference on User Centric Media (UCMedia 2009), 9-11 Dec. 09, Venice www.usercentricmedia.org
Credits
• Default Mode Network paper: Gusnard, DA, Raichle, ME: “Searching for a baseline: the resting human brain from the perspective of functional imaging.” Nature Reviews Neuroscience, 2001
• “iLamps: Geometrically Aware and Self-Configuring Projectors,” Rascar, et al. 2003
• Commercial aviation HUD image courtesy Todd Lapin, cc
• MARA image courtest Nokia Research
• Handheld AR.net (http://handheldar.net/stbes.php) for Studierstube ES diagram
• Fighter jet HUD device image cc www.rochesteravionicarchives.co.uk
• Photo of sail’s tell-tales courtesy Bill Gracey, cc
• Modern helmet-mounted see-through display courtesy Rockwell-Collins: "SIM EYE SR100-A”
• Photo of Ray Harroun’s Marmon Wasp racecar with mirror, cc The359
• Vuzix Wrap 920AV and Tac-Eye GMD images c. Vuzix Inc.
• Theme park illustrations © Chimera Design, 2007-2009
• Photo of Dubailand model © Lynne LaCascia, 2009
Software Development ForumEmerging Technology SIG
Augmented Reality
back@fxpal.com
Maribeth Back, Fuji-Xerox Palo Alto Labs“High Tech Chocolate:
Exploring Mixed Reality applications for Industry”
top related