Augmented Reality

Joan Puig Sanz

9 Apr 2014

Agenda

1. What is Augmented Reality2. Existing Platforms3. BeyondAR Framework

1. What can we do with it?

4. What is the next step?

Augmented Reality

• According to WikiPedia:“Augmented reality (AR) is a live direct or indirect view of a physical, real-world environment whose elements are augmented (or supplemented) by computer-generated sensory input such as sound, video, graphics or GPS data”

Augmented Reality

• According to WikiPedia:“Augmented reality (AR) is a live direct or indirect view of a physical, real-world environment whose elements are augmented (or supplemented) by computer-generated sensory input such as sound, video, graphics or GPS data”

Augmented Reality !=

Virtual

reality

Make Augmented Reality

• Using geo localization• Image recognition• Sensors– Accelerometer– Magnetic field– Compass– Motion tracking camera– …

EXISTING TOOLS

Existing Tools

• Vuforia• Layar• Wikitude• Droidar • Mixare• Google Tango• BeyondAR• …

Vuforia

• Proprietary• Available for Android and iOS• Unity support• Big community• Collect some data form the user: related to the

scanned images• Target recognition– Device database: free < 100 images– Cloud Database: not free >100

Vuforia

• https://www.youtube.com/watch?v=UOfN1plW_Hw

Layar

• Proprietary• SDK Available for Android and iOS–Geo localization– Image recognition

Layar

• App browser (Android and iOS)– Geo Layers and image recognition

Demo time!

Wikitude

• Proprietary• SDK Available for Android, iOS, Epson

Moverio and Vuzix.–Geo localization– Image recognition

Wikitude

• Proprietary• SDK Available for Android, iOS, Epson

Moverio and Vuzix.–Geo localization– Image recognition

Wikitude

• Proprietary• SDK Available for Android, iOS, Epson

Moverio and Vuzix.–Geo localization– Image recognition

Wikitude

• Proprietary• SDK Available for Android, iOS, Epson

Moverio and Vuzix.–Geo localization– Image recognition

Wikitude

• Extensions for PhoneGap and Titanium• App browser (Android, iOS and BB10)– Geo Layers and image recognition

Droidar

• GPL3• SDK for Android• Location based• Supports 3D modeling

/bitstars/droidar

Mixare

• GPL3• SDK for Android and iOS• Location based• Canvas– Slow performance

• No code changes since 2 years ago

/mixare

Google Tango

https://www.youtube.com/watch?v=Qe10ExwzCqk

BeyondAR

• Apache 2• SDK for Android• Location based• 3D• Active development

/BeyondAR

LET’S CODE!

/BeyondAR

Example App available on Google play

BeyondARGetting started

• Import the library in your project• Update Manifest

<!-- Minimum permissions for BeyondAR --><uses-permission android:name="android.permission.CAMERA" /><uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" /><uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />

<!-- For BeyondAR this is not mandatory unless you want to load something from the network --><uses-permission android:name="android.permission.INTERNET" />

<!-- BeyondAR needs the following features--><uses-feature android:name="android.hardware.camera" /><uses-feature android:name="android.hardware.sensor.accelerometer" /><uses-feature android:name="android.hardware.sensor.compass" />

/BeyondAR

BeyondARGetting started

Create the UI

<?xml version="1.0" encoding="utf-8"?><FrameLayout xmlns:android="http://schemas.android.com/apk/res/android" android:layout_width="match_parent" android:layout_height="match_parent" android:id="@+id/parentFrameLayout" >

<fragment android:id="@+id/beyondarFragment" android:name="com.beyondar.android.fragment.BeyondarFragmentSupport" android:layout_width="match_parent" android:layout_height="match_parent" />

</FrameLayout>

/BeyondAR

BeyondARGetting started

Create the UI @Override

public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.example);

// ... mBeyondarFragment = (BeyondarFragmentSupport)

getSupportFragmentManager().findFragmentById (R.id.beyondarFragment); // ...

}

BeyondarFragment: Class that manages the camera and the OpenGL surface

/BeyondAR

BeyondARGetting started

Create the AR world and add an AR object// We create the world objectWorld myWorld = new World(this);myWorld.setGeoPosition(41.90533734214473d, 2.565848038959814d);

World: Container for all the BeyondarObjects• Try to manage all BeyondarObjects using this class• Responsible for the user location• You can add plugins

/BeyondAR

BeyondARGetting started

Create the AR world and add an AR object// We create the world objectWorld myWorld = new World(this);myWorld.setGeoPosition(41.90533734214473d, 2.565848038959814d);

// Create an objectGeoObject go1 = new GeoObject();go1.setGeoPosition(41.90523339794433d, 2.565036406654116d);go1.setImageResource(R.drawable.my_image);go1.setName(”Hello World");

// Add the objectmyWorld.addBeyondarObject(go1);

// give the world to the fragmentmBeyondarFragment.setWorld(myWorld);

/BeyondAR

BeyondARGetting started

Create the AR world and add an AR object// We create the world objectWorld myWorld = new World(this);myWorld.setGeoPosition(41.90533734214473d, 2.565848038959814d);

// Create an objectGeoObject go1 = new GeoObject();go1.setGeoPosition(41.90523339794433d, 2.565036406654116d);go1.setImageResource(R.drawable.my_image);go1.setName(”Hello World");

// Add the objectmyWorld.addBeyondarObject(go1);

// give the world to the fragmentmBeyondarFragment.setWorld(myWorld);

Demo Time!

/BeyondAR

BeyondARGetting started

Interaction with the AR Objects

/BeyondAR

BeyondARGetting started

Interaction with the AR Objects

OnTouchBeyondarViewListener OnClickBeyondarObjectListener

mBeyondarFragment.setOnTouchBeyondarViewListener(this);

mBeyondarFragment.setOnClickBeyondarObjectListener(this);

@Overridepublic void onClickBeyondarObject(ArrayList<BeyondarObject> beyondarObjects) {

if (beyondarObjects.size() > 0) {Toast.makeText(this, "Clicked on: " +

beyondarObjects.get(0).getName(), Toast.LENGTH_LONG).show();}

}

/BeyondAR

BeyondARGetting started

Interaction with the AR Objects

OnTouchBeyondarViewListener OnClickBeyondarObjectListener

mBeyondarFragment.setOnTouchBeyondarViewListener(this);

mBeyondarFragment.setOnClickBeyondarObjectListener(this);

@Overridepublic void onClickBeyondarObject(ArrayList<BeyondarObject> beyondarObjects) {

if (beyondarObjects.size() > 0) {Toast.makeText(this, "Clicked on: " +

beyondarObjects.get(0).getName(), Toast.LENGTH_LONG).show();}

}

Demo Time!

/BeyondAR

OTHER FEATURES

1. Location utils2. Views in the AR world3. Take screenshots4. Plugins5. …

/BeyondAR

BeyondARUsing location utils

• Easy way to use the location services

LocationManager locationManager = (LocationManager) getSystemService(Context.LOCATION_SERVICE);// We need to set the LocationManager to the BeyondarLocationManager.BeyondarLocationManager.setLocationManager(locationManager);

BeyondarLocationManager: Helper to use the location services.• Retrieves the best location using GPS and the network providers• Can be used with LocationListener, World or GeoObject

/BeyondAR

BeyondARUsing location utils

Create the AR world and add an AR object@Overrideprotected void onResume() {

super.onResume();// When the activity is resumed it is time to enable the// BeyondarLocationManagerBeyondarLocationManager.enable();

}

@Overrideprotected void onPause() {

super.onPause();// To avoid unnecessary battery usage disable BeyondarLocationManager// when the activity goes on pause.BeyondarLocationManager.disable();

}

BeyondarLocationManager.addWorldLocationUpdate(myWorld);

BeyondarLocationManager.addGeoObjectLocationUpdate(user);

/BeyondAR

OTHER FEATURES

1. Location utils2. Views in the AR world3. Take screenshots4. Plugins5. …

/BeyondAR

BeyondARWorking with Views

• Attach a View to a BeyondarObject Extend BeyondarViewAdapter It follows the same pattern than the ListAdapter

- Remember to recycle your Views!!

@Overridepublic View getView(BeyondarObject beyondarObject, View recycledView, ViewGroup parent) {

if (recycledView == null) recycledView = inflater.inflate(R.layout.beyondar_object_view, null);

TextView textView = (TextView) recycledView.findViewById(R.id.titleTextView);textView.setText(beyondarObject.getName());Button button = (Button) recycledView.findViewById(R.id.button);button.setOnClickListener(myClickListener);

// Once the view is ready we specify the positionsetPosition(beyondarObject.getScreenPositionTopRight());

return recycledView;}

Crea

te th

e vi

ew/BeyondAR

BeyondARWorking with Views

• Make a BeyondarObject from a View

ImageUtils.storeView(view, path, imageName);

// If there are no errors we can tell the object to use the// view that we just storedbeyondarObject.setImageUri(path + imageName);

/BeyondAR

BeyondARWorking with Views

• Make a BeyondarObject from a View

ImageUtils.storeView(view, path, imageName);

// If there are no errors we can tell the object to use the// view that we just storedbeyondarObject.setImageUri(path + imageName);

Demo Time!

/BeyondAR

OTHER FEATURES

1. Location utils2. Views in the AR world3. Take screenshots4. Plugins5. …

/BeyondAR

BeyondARScreenshoots

mBeyondarFragment.takeScreenshot(myOnScreenshotListener);

@Overridepublic void onScreenshot(Bitmap screenshot) {

ImageDialog dialog = new ImageDialog();dialog.setImage(screenshot);dialog.show(getSupportFragmentManager(), "ImageDialog");

}

When you are done with the image, remember to recycle it

/BeyondAR

BeyondARScreenshoots

mBeyondarFragment.takeScreenshot(myOnScreenshotListener);

@Overridepublic void onScreenshot(Bitmap screenshot) {

ImageDialog dialog = new ImageDialog();dialog.setImage(screenshot);

dialog.show(getSupportFragmentManager(), "ImageDialog");}

When you are done with the image, remember to recycle it

Demo Time!

/BeyondAR

OTHER FEATURES

1. Location utils2. Views in the AR world3. Take screenshots4. Plugins5. …

/BeyondAR

BeyondARPlugins

• Main goal:– Easy to use– Make your own features

/BeyondAR

BeyondARPlugins

• Example: Google Maps Plugin// As we want to use GoogleMaps, we are going to create the plugin and// attach it to the WorldmGoogleMapPlugin = new GoogleMapWorldPlugin(this);// Then we need to set the map in to the GoogleMapPluginmGoogleMapPlugin.setGoogleMap(mMap);// Now that we have the plugin created let's add it to our world.// NOTE: It is better to load the plugins before start adding object in to the world.mWorld.addPlugin(mGoogleMapPlugin);

/BeyondAR

OTHER FEATURES

1. Location utils2. Views in the AR world3. Take screenshots4. Plugins5. …

/BeyondAR

What is the next step?

• Get over the WOW effect• Choose your platform• Make apps– Education– Traveling– Gaming

46

beyondar.com

@joanpuigsanz

/Beyondar

@beyondar