a step-by-step guide to building a mobile outdoor ar application
DESCRIPTION
A tutorial given at MGIA, SIGGRAPH Asia 2013 on building mobile AR application using the Outdoor AR Library.TRANSCRIPT
A Step-by-step guide to
Building a Mobile Outdoor AR
Application
Gun Lee
21 Nov. 2013
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Outdoor AR Browsers
Wikitude Layar Junaio
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
SW Dev. Tools & Frameworks
High-level content description
AR Browsers
Hard to customize app logic and UI
Low-level functional modules
Programming libraries
Lack of higher-level abstraction of AR scene
and content (BYO)
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Outdoor AR Framework High-level abstraction and low-level
programming support in a seamless way.
Low-level functional modules
- Flexible and highly customizable
Ready-to-use high level components
- Focus on building the interface, behavior and
content of the application rather than system
integration
The high-level and low-level tools share the
common scene (or POI) data structure
- Both skilled and novice developers can work
together under the same framework.
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Apps built with OAR Framework
[CityViewAR 2011] [CCDU 3D 2012]
[GeoBoids 3D 2012] [AntarcticAR 2012]
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Outdoor AR Framework
Covers full tier of ready-to-use
components, yet highly customizable
Mobile Device (Client)
Mobile OS
Mobile AR Application
Scene
data
Media
data
Mobile SW Library
Server
Server OS
Application Service
Scene
data
Media
data
Server SW Library Data
Communication
(Optional)
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Server Software and Tools
Web Client
(Browser)
Server Mobile Client
(App)
HTML Web-based
Authoring Tool
RESTful API
Scene
Database
AR View
Data
Manager
Local
cache
Javascript
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Server Software and Tools
Web-based Authoring Tool
Please sign up for closed beta – Jan 2014
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Outdoor AR Library
For mobile app (client)
Android (v2.2 or above)
Google Maps API
Scenegraph based rendering engine
OpenGL ES (1.1 and 2.0)
Free download from our website:
http://www.hitlabnz.org/mobileAR
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Outdoor AR Library Structure
Functional Component Layer
Service Component Layer
Map
Component
Scene Data Manager
AR
Component
List
Component
Tracking Data Manager
Scene Data
Structures
Tracking
Sensors
3D Graphics
Rendering UI Views
Tools &
Utilities
3D
Sound
Map View AR View List View Views
Controllers
Models
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Model-View-Component
Controller
Model Scene & Tracking
Data Managers
View Visualization &
UI Elements
Application Logic &
Data Binding
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Mobile App Software Library
Customization by extending components
Customized AR scenes
- Customize scene class data structure
Customized user interface
- Customize UI layout
- Customize UI behavior (e.g. onSceneSelected)
- Motion and touch gesture based interaction
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Step-by-step Guide
Setup the development environment
Create an Android app
Using the Outdoor AR Library in your app
Adding outdoor AR scenes with 3D models
Customizing UI
Run your app on a Google Glass
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Setup the dev environment
Prerequisite
Android Development Environment
- http://developer.android.com/sdk/index.html
- Eclipse, Android SDK
Google Play Services Library (for maps)
Download the Outdoor AR Library
http://www.hitlabnz.org/mobileAR
- Download, Tutorials, Forum
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Setup the dev environment
Import the library and sample projects
OutdoorARLibrary
SampleOAComponent
Dependency
Google (not Android) API 4.1.2 (API Level 16)
Google Play Services Library
- extras/google/google_play_services/libproject
Android Support Library 4
- extras/android/compatibility/v4/
android-support-v4.jar
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Create an Android app
Create a new Android Application project
Blank Activity
- “An activity is a single, focused thing that the user
can do.” – Android SDK API Reference
Anatomy of an Android App project
src
res & gen
assets
AndroidManifest.xml
Add a “Hello MGIA” button
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Using the OA Library in your app
Add the Outdoor AR Library to your
project’s build path
Add permissions in the manifest
Add a new AR view (activity)
Create a custom class inheriting
OAARComponentBase
Add activity to the manifest
- Full screen landscape style preferred
Wire a button to start the AR view
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Adding scenes with 3D models
Preparing 3D model
OBJ/MTL with JPEG or PNG textures
OpenGL Coordinate frame
- X-right, Y-up, Z-out from screen
- North = -Z
Placing the 3D model in the real world
Latitude, Longitude
- http://itouchmap.com/latlong.html
Optionally, elevation from the ground plane
SketchUp – 3D model on Google Earth
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Adding scenes with 3D models
Adding 3D models to the project
assets/OutdoorAR/3Dmodels(/name/model.obj)
Adding scenes to the data manager
Override onSetupScenes()
Handling the scene selection event
Override onSceneSelected()
Using a mock location
Override onCreate()
Sensor manager’s enableMockLocation()
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Loading and managing scenes
Add scenes by code – difficult to manage
Load from assets or local file storage
OADataManagerAssets
OADataManagerLocal
SQLite or XML file
SQLite Database Browser
Default path
- (assets/)OutdoorAR/scenes.db
- (assets/)OutdoorAR/scenes.xml
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Custom scene class
OAScene
Lat/Lon, 3D model, Name, Description,
Category
Custom scene classes extend OAScene
Override OADataManager’s loadScene()
to load in custom properties of your scene
Cast OAScene to your custom class in
onSceneSelected() or onTouchedScene()
Different scene classes can live together
- if(scene instanceof MyCustomScene)
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Customizing UI
Add layout resource
Override setupUILayout()
LayoutInflater - read in the layout res file
addContentView() - overlay UI layout on top
of AR view
findViewById() - get reference to the UI
elements
onSceneSelected() vs. onTouchedScene()
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Map and List views
OAMapComponentBase, OAListComponentBase
Similar class structure with OAARComponentBase
- Load scenes from a data manager
- setupXYZ() methods
- onSceneSelected() callback
Map components needs API key
Google Maps Android API v2
- OAGoogleMapsV2ComponentBase
https://developers.google.com/maps/documentation/an
droid/start
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Run your app on Google Glass
Startup with AR
Set AR view as a launcher activity in the app
manifest file
Aim to select
Use Timer to trigger touch events at the
center of the screen
Override onResume() and onPause() to
start/stop the timer
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Other tips for Google Glass
Touch gestures
onGenericMotionEvent()
Swipe forward/backward/up
Tap, double tap, tap and hold, etc.
Head motion
Add orientation listener to sensor manager
- getSensorManager().addSensorListener()
Avoid conflicts with AR scene viewing motions
- Time-outs
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Other tips for Google Glass
GPS sensor blocked
Stream location from a phone through
Bluetooth
Mock location
Turning off camera background
Application dependent
Override setupOptions()
enableCameraBackground()
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
CityViewAR for Glass
Demo
Thank you!
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
More Information
Website
http://www.hitlabnz.org/mobileAR
http://arforglass.org
Gun Lee
Mark Billinghurst