augmented reality for the enterprise

www.cameronvetter.com 1

AugmentedReality

Let’s Make Some Holograms!

2

Goal of this Presentation

www.cameronvetter.com

Learn AR Terminology

Build an Application

Enterprise Use Cases

See Lots of Examples


About Me Senior Software Architect


@PoshPorcupine

About Me

HoloLens Video Collage


5

Presentation Roadmap


Holograms

Control

Real World UnderstandingSpatial MappingPlane FindingSpatial Understanding

GazeCursorGesturesVoice

BillboardingPlacement

Physics

Holographic Lens1268 x 720 Per Eye3 Waveguide Lenses Per Eye

Intel Atom CPUX5-Z8100 1.4 GHZ2GB Ram

Holographic Processing UnitCustom build Microsoft GPU

Sensors4 Environment Cameras1 Depth Camera1 Video Camera4 Microphones1 Light Sensor

What is the HoloLens?


7

Development Stack


Visual Studio 2015 Update 3

Windows 10 SDK

Unity 3D v5.4+

HoloLens Emulator

HoloToolkit / HoloToolkit-Unity

Trimble Architecture


Real World Understanding


World Coordinate System1 Unit = 1 Meter

User Start is OriginX=0, Y=0, Z=0

Hidden TrianglesCalled the Spatial Mesh

Spatial AnchorsAttach Hologram to Reference Points in the Spatial Map

Spatial Mapping


Room Spatial MeshVisualized with Solids in the Illustration

PhysicsHolograms and Real World interact with each other through Game Engine Physics

OcclusionWorks Both Ways

Plane FindingAbility to Find Planes in the Real World

What do we get?


Uses Spatial MapSpatial Map Data used to find Planes

Type of PlanesWallsFloorCeilingHorizontal SurfacesOther

Skipping this TopicWe now have Spatial Understanding

Plane Finding


World UnderstandingMore Detail than Spatial Map

Topology QueriesSimple Surface and Spatial Queries

Object Placement SolverConstraint Based Placement of Object Sets

Shape QueriesLook for Real World Shape Matching Custom Shape by the User

Spatial Understanding


Demos


Spatial Mapping

Spatial Understanding

Thyssenkrupp Elevator


Holograms


Body LockedTags Along with the Display

Display LockedPositionally Locked to the Display

Use SparinglyClinginess is Frustrating for Users

Breaks IllusionHolograms are Intended to Blend, Not Overlay

Billboarding


Relative to OriginOrigin is Starting Position, Not Current Position

Relative to Real WorldMust take into Account, Real World Objects

Spatial UnderstandingUsed to Accurately Place Objects with Object Placement Solver

Plane FindingUsed to Quickly Find Ceiling, Tables, Walls, and Other Surfaces

Placement


Tracks World StateReal World Objects and Holograms

Placement TypesOnFloor, OnWall, OnCeiling, OnShape, OnEdge, OnFloorAndCeiling, RandomInAir, InMidAir, UnderFurnitureEdge

Placement ConstraintNearPoint, NearWall, AwayFromOtherObjects, AwayFromPoint

Placement RulesAwayFromPosition, AwayFromWalls, AwayFromOtherObjects

Object Placement Solver


Unity Engine ProvidedAdd a RigidBody to Any Hologram to Make it Observe Game Engine Physics

Spatial Map ColliderEach Plane in the Spatial Mesh Receives a Unity Collider

Not the Other WayReal World Objects are not Impacted by Holograms

Real World ImpactHolograms are Impacted by Real World

Physics


Demos


Placement Solver

PGA Tournament


Control


Detects Using RaycastRaycast from User’s Eyes to Unity Collider

Primary InteractionLooking At Holograms is the Primary Way to Interact with Holograms

Real World has CollidersRemember During Spatial Mapping the Real World is given Colliders

Invisible ObjectsAny Holograms missing a Collider are not Selectable with Gaze

Gaze


Center of ViewDisplay Locked to Center of View

Requires GazeCursor Depends on Gaze, Expect Both to be in Every Project

Stick with Standard CursorUse Microsoft Provided Cursors Unless you Have a Good Reason to Customize, TIP: You Don’t

SelectionPrimarily used for Selection Confirmation in Conjunction with Gaze and Gestures

Cursor


The Cursor

BloomTask Switch / Start Menu

Air TapSimilar to Primary Mouse Button

ManipulationTypically Used to Move or Drag Items

HoldSimilar to Right Clicking

Gestures


Auxiliary ControlsThe Real Right Click

Use Windows 10 SpeechStandard Engine Built into Windows

System Level CommandsSelect == Air TapFace MeBiggerSmaller

Concise CommandsToo Long is Hard to RememberToo Short is Hard to Differentiate

Voice Input


Caterpillar Sales


29

LIVE DEMO!!!


01

02

0304

05

06

Place Holograms

Map the Room

Use A Gesture

Shadows

Voice Commands

Physics

30

In Summary


Holograms

Control

Real World UnderstandingSpatial MappingPlane FindingSpatial Understanding

GazeCursorGesturesVoice

BillboardingPlacement

Physics

Interior Design Reimagined




[email protected]

@poshporcupine

Any Questions?

augmented reality for the enterprise

Technology