augmented reality with open source software

Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31

Open Source AR SoftwareBusiness Case DirectFactory

1


BeCompany – About Us

Head office in Zurich

Independent integrator

Technology evaluation, focus on new technologies

Consulting, implementation, support

Specialized in open source technologies

Swiss Post solution provider:DirectFactory and other web applications

2


DirectFactory & AR: Requirements

DirectFactory as tool for AR campaigns

Allow users to create/import 3D scenes

Use postcards as advertisement and carrier/interaction medium

Personalization (text/other possibilities)

AR platforms: Desktop, iOS, Android

3


Marker-based ARVideo

Tracking

Composition

3D Rendering& Animation

3D Model

Textures

Animation

3D Processing

Marker positionin 3D space

Camera positionfor rendering

4


Business Case DirectFactory

Step 1: Technology Evaluation

Step 2: Prototype

Step 3: Pilot

5



Technologies (open & closed source)

Technical requirements & conditions

Legal conditions (licensing)

Options & capabilities (supported content & markers, quality, reliability, personalization, …)

6



High quality showcasesCross-platform solutionsNo out-of-the-box personalization capabilityCore customization requires cooperation with vendor

Closed Source Vendors

7



Lack of high-quality showcases

Projects focusing on single platforms, driven by individual use cases, not market penetration

Small core development communities

Large supply of knowledgeable developers for individual platforms, but not across platforms

Low initial costs

Open Source Vendors

8


Open Source Marker-Based AR Platforms

Hardware Operating SystemRuntime

Environment Video Tracking 3D Rendering

DesktopWindows,Mac OS

Flash FLARToolKitAway3D


Flash FLARToolKitPapervision3D


Silverlight SLARToolKitSilverlight 5 3D


Silverlight SLARToolKitBalder

Mobile

iOSNative

(Objective C) ARToolKit for iOSOpenSceneGraph

(OpenGL)

Mobile Android Native (Java) ARToolKit/AndAR OpenGLMobile

Windows PhoneNative (C#) /

Mango /Silverlight

SLARToolKit Native (C#)

9


Flash vs. Silverlight

Flash

Silverlight

Nov '10 - Apr '11Source: statowl.com

10


Decision: Flash + FLARToolKit

Based on proven ARToolKit algorithms

Large supply of knowledgeable developers

Dual license (GPL for non-commercial use)

Supports multiple 3D engines:Papervision3D, Away3D, Sandy, Alternativa3D

Marker: simple geometric forms, asymmetric

11

http://code.google.com/p/papervision3d/

http://code.google.com/p/papervision3d/

http://away3d.com/

http://away3d.com/

http://www.flashsandy.org/

http://www.flashsandy.org/

http://alternativaplatform.com/en/alternativa3d/

http://alternativaplatform.com/en/alternativa3d/


Step 2: Prototype

Proof of concept: Integration DF & AR

Basic technical capabilities & limitations

Basic project workflow

Goals

12


Step 2: Desktop Prototype

Easy integration of DF personalization service(objects rendered independent from 3D scene)

Very fast implementation

Postcard feasible as marker carrier

No no-go criteria found

But: no anmiation yet – critical!

Lessons learned during Implementation

13


Step 3: Pilot

Reality check, verify assumptions

Actual implementation cost & complexity

3D model & anmiation workflow

Acceptance, understanding by recipients

Real-world technical limitations (webcam, …)

Goals

14


Limitations& Conclusion

15


Technical Limitations

AR solutions still in early development stage:„Peak of Inflated Expectations“ (Gartner)

No cross-platform runtime environment(except proprietary plug-ins)

Flash usability problems:Browser window size bug on Firefox Mac,Keyboard focus

16


Handling Limitations

Users not familiar with AR applications

Webcam not available in older computers

Rigid lenses (notebooks) can impede handling for some applications

Consideration: Alternative content for users without webcam

17


3D Scene Limitations

Existing standards (OpenGL) not consistently available across platforms

No universal standard for 3D scenes(3D modelling -> 3D rendering):Built in Cinema4D, problems importing in Away3D

Generic application + arbitrary pluggable 3D scenes very hard to achieve („AR scene library“)

Close cooperation between designer and AR/Flash developer necessary

18


Rendering Performance Limitations

Flash3D Rendering

GPU

zzzz…

CPU

Flash Stage3D („Molehill“)

DirectX 9,OpenGL 1.3,

OpenGL ES 2.0

„Future version of Flash Player“

19


Marker Limitations

Black & white

Simple, asymmetric shape (orientation detection)

Size approx. 10 x 10 cm

Surface not shiny (postcard varnishing!)

Rigid carrier material to ensure that the marker stays flat

20


Personalization Limitations

Personalization of 3D scene problematic(personalized objects separate from 3D scene)

Marker-based personalization not possible with ARToolKit

Alphanumeric code: cumbersome, risk of typing errors

Visual code (QR): additional visual marker could be confusing

21


Conclusion – as things are now :)

Many technical problems not related to AR itself

Multiple components with individual limitations involved, incompatibilities

Technical limitations dictate conceptual & functional possibilities – „function follows form“

Easy implementation, low initial costs

Realization in short timeframe possible

Be aware of non-technical limitations(appeal, handling, usability, guidance)

22


When to Use Closed Source

High quality requirements

Complex 3D scenes (many polygons)

Image (e.g. photo) as marker

Support for multiple platforms (desktop, mobile)

23


When to Use Open Source

Low initial costs

Low quality/complexity requirements(3D scene, marker)

Short time-to-market

Single target platform(desktop or particular mobile platform)

Customizability (personalization, integration in other services, …)

24


The EndThank you for your attention!

25

augmented reality with open source software

Technology

directfactory ar

ar applicationswebcam

ar campaigns

requirements directfactory

asymmetric andreas hartmann

pilot andreas hartmann

androidandreas hartmann

comsilverlightandreas