challenge, drama & social engagement: designing mobile augmented reality experiences

Upload: web-20-expo

Post on 30-May-2018

225 views

Category:

Documents


1 download

TRANSCRIPT

  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    1/32

    Challenge, Drama & SocialEngagement: Designing Mobile

    Augmented Reality Experiences

    Gene Becker

    twitter @genebecker

    [email protected]

  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    2/32

    Digital media

    Mobile

    Tagging

    Cloud services

    Sensing

    Imaging

    Social media

    Displays

    2000 - 2020

    Storage

    MMO platforms

    Nano3D printingPower

    Robotics

    Gesture

    Augmented reality

    Wireless

    Bio

    a remarkable confluence of technologies

  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    3/32

    what is augmented reality?

    Augmented realityis

    a technology

    a field of research a vision of future computing an emerging commercial industry a new medium for creative expression

  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    4/32

    augmented reality is a technology

    Augmented realityis the display of computer

    graphics and media, overlaid and registered on

    real-world environments, interactively in real time.

    AR platforms exist at the intersection of several technicaldisciplines, including computer graphics, machine vision,sensing and sensor fusion, geographic information systems,mobile systems, ubiquitous computing and the web.

  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    5/32

    AR research: first HMD - Ivan Sutherland (1966)

    The Sword of Damocles

  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    6/32

    AR research: wearable computing - Steve Mann

    http://wearcam.org/steve.html

    http://wearcam.org/steve.htmlhttp://wearcam.org/steve.html
  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    7/32

    AR research: first mobile AR system

    Feiner et al, Touring Machine (1997), Columbia Universityhttp://graphics.cs.columbia.edu/projects/mars/mars.html

    http://graphics.cs.columbia.edu/projects/mars/mars.htmlhttp://graphics.cs.columbia.edu/projects/mars/mars.html
  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    8/32

    visual search on the physical world concept design by mac funamizu

    source: http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/

    http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/
  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    9/32

    querying the natural world concept design by mac funamizu

    source: http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/

    http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/
  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    10/32

    computational print media concept design by mac funamizu

    source: http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/

    http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/
  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    11/32

    seeing through space and time concept design by mac funamizu

    source: http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/

    http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/
  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    12/32

    the end of personal privacy? concept by TAT

    Augmented reality is the collapse of social nuanceand the end of personal privacy.

  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    13/32

    social augmented reality concept sketches by matthew buckland

    Source: http://www.matthewbuckland.com/?p=1041

    http://www.matthewbuckland.com/?p=1041http://www.matthewbuckland.com/?p=1041
  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    14/32

    social augmented reality concept sketches by matthew buckland

    Source: http://www.matthewbuckland.com/?p=1041

    http://www.matthewbuckland.com/?p=1041http://www.matthewbuckland.com/?p=1041
  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    15/32

    social augmented reality concept sketches by matthew buckland

    Source: http://www.matthewbuckland.com/?p=1041

    http://www.matthewbuckland.com/?p=1041http://www.matthewbuckland.com/?p=1041
  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    16/32

    and just wait til the glasses get here concept design by mac funamizu

    source: http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/

    http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/http://petitinvention.wordpress.com/2008/02/10/future-of-internet-search-mobile-version/
  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    17/32

    commercial mobile AR example: Layar

    [video]

  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    18/32

    commercial webcam AR example: Metaio

    [video]

  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    19/32

    augmented reality is a new medium

    Augmented realityis a new medium of creative

    expression that expands human experience

    AR combines the web, images, sound, movies, games, searchengines, databases, advertising, social mediaand explodes them

    across the physical world of people, places and things, revealingthe invisible stories that are everywhere around us.

    But we have a lot of work to do.

  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    20/32

    what makes a compelling experience?

    Challenge/

    Expression

    Drama/Sensation

    Dimensions

    of

    Compelling

    Experience

    Social

    Physical / MentalChallenge

    Self-Expression& Creativity

    Physical Stimulation(sights, sounds, touch)

    Mental / EmotionalStimulation, Imagination

    Bonding, Sharing &Cooperation

    Competingwith others

    see also: Flow and the psychology ofoptimal experience (Csikszentmihalyi)

    ref: Hull, Reid Experience Design for Pervasive Computing

  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    21/32

    the reality of AR experiences today

    Challenge/

    Expression

    Drama/

    Sensation

    Dimensions

    of

    Compelling

    Experience

    Social

    What works:* Physical engagement* Visually captivating

    * Emotional wow factor* Invisible stories, revealed

    What doesnt:

    * Awkward use models* No flow opportunities

    * No social dimension* Point experiences only

  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    22/32

    mobile AR issues & limitations

    Technical limitations

    Sensor accuracy (GPS, compass etc) Indoor positioning Line of sight Device constraints (display, form factor, weight, battery, processor, network) Lack of interoperable standards e.g. for location (lat/long/alt, CRS, orientation) Relative positioning Object recognition & tracking

    Graphics issues (lighting, transparency, occlusion)

    Use model issues

    Unnatural physical position Ambiguous social signals Cognitive model

    Authoring limitations

    Platform specific Low level tools, complex workflow Focus on point interactions; no support for stories or interesting interactivity

  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    23/32

    visionary example: AR gaming HP concept video

  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    24/32

    what makes a compelling AR experience?

    Challenge/

    Expression

    Drama/Sensation

    Dimensions

    of

    Compelling

    Experience

    Social

    Physical / MentalChallenge

    Self-Expression& Creativity

    Physical Stimulation(sights, sounds, touch)

    Mental / EmotionalStimulation, Imagination

    Bonding, Sharing &Cooperation

    Competingwith others

    fantasy

    ref: Hull, Reid Experience Design for Pervasive Computing

    running

    locating

    powerups

    wayfinding

    racing against time

    story line

    devising

    strategies

    physical/virtual

    confusion

    characters

    avoiding danger(in game and real)

    magic moments camaraderie

    racing against

    other players

    scoring points

    collecting

    items

    using physical

    affordances

    real & in game

    sounds

    savoring the win

    sharing the surprise

    kiss

  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    25/32

    mobile AR experience design practices

    1. Understand limitations and design around them

    2. Design for challenge, expression and flow

    3. Design for physical sensation & emotional drama

    4. Design for social bonding, sharing, competition

    5. Bring an interdisciplinary media production mindset

  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    26/32

    mobile AR experience design practices

    1. Understand current limitations and design around them

    Use augments with appropriate requirements for precision & realism

    Emphasize audio for immersion, continuity, story

    Use magic lens position sparingly

    Fail gracefully & transparently

  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    27/32

    mobile AR experience design practices

    2. Design for challenge, expression and flow

    Make the experience the focus, not the technology

    Set clear goals, expectations and rules

    Design goals to balance between ability level and challenge (neither too easy nortoo difficult).

    Encourage focus and concentration, limit distractions and gratuitous interactions

    Provide direct and immediate feedback so the user can adjust their actions.

    Give users a sense of personal control over the situation or activity.

    Make activities intrinsically rewarding, so there is an effortlessness of action.

  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    28/32

    mobile AR experience design practices

    3. Design for physical sensation & emotional drama

    Scout the location, know the environment & context deeply sights,sounds, smells, textures, weather, rhythms, moods

    Design for magic moments mental immersion, physical/virtual

    coincidence, synaesthetic confusion

    Emphasize sound design for immersion, continuity, story

    Develop a cross-media story universe

  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    29/32

  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    30/32

    mobile AR experience design practices

    5. Bring an interdisciplinary media production mindset

    Creating mobile AR experiences draws on skills from a broad range ofcreative disciplines including

    Scriptwriting Direction

    Casting Art direction Sound design Special effects Interaction design Location scouting Production management Software development Systems management Testing & QA Gamerunning Tech support

  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    31/32

    mobile AR authoring demo

    a large scale driving experience overlaid on the Bay Area primarily audioAR scriptable behaviors on geographic regions context-aware behaviors authored on PC, experienced on mobile

  • 8/9/2019 Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences

    32/32

    AR history: sub

    Gene Becker

    tw: @genebecker

    web: www.lightninglaboratories.com

    mail: [email protected]

    Lightning Laboratoriesprovides Research and

    Consulting in augmented reality, ubiquitous

    computing, social media and the Internet of

    Things

    Questions?