Leap motion UI for EcoVR

Zena Wolba

Recap of EcoVR

Recap of EcoVR“proof of concept to explore new methods of visualising complex time-series environmental data, on the landscape.”

❖ A 2015 Techlauncher project led by Dr. Tim Brown

❖ Created using the Unreal Engine 4

❖ Compatible with Oculus Rift DK2 and Xbox/PS controller.

The Data

● 20 node wireless mesh sensor network measuring humidity, temperature, soil moisture

● Dendrometers measuring trunk diameter

● Gigapixel images, UAV fly overs

● Dual Wavelength Echidna Lidar (DWEL) and Zebedee for creating dense pointclouds

Image & info from http://www.wenfo.org/wald/wp-content/uploads/2015/02/2-Tim_Brown_Sensors.pdf

Data collection

○

The Site

And the Leap Motion?❖ Developed by Leap Motion Inc.

❖ The Leap Motion controller is a USB peripheral device

❖ Tracks hands and fingers using infrared cameras and software

Aim of this project...❖ Substitute the clunky

Keyboard/Mouse/Xbox controller with the more intuitive Leap controller

❖ Hand gestures to toggle menus, press buttons, adjust sliders and more!

Aim of this project...

Aim of this project...

Video!

Design❖ Built upon the Leap Motion

plugin in Unreal Engine 4.11

❖ Designed to be intuitive and VR user friendly.

❖ Necessary to create a 3D interface - UI that exists in the virtual world.

❖ Some aesthetic design choices inspired by Unity 3D widgets.

Innovation❖ Very limited support for 3D

interfaces in UE4.

❖ Created an all new 3D button widget for UE4, designed specifically for VR/Leap Motion.

❖ Also modified existing UE4 slider widget to be used with Leap Motion.

Interactions❖ Menus toggled with

hand gestures.

❖ All widgets designed to respond to touch.

❖ Gives freedom to use VR freely.

How does it work? - Menus❖ Uses UE4 functions which

get and manipulate vectors.

❖ Uses up and right vectors as axis.

❖ Then rotates the forward vector about these axis to decide where to place widgets.

How does it work? - Buttons❖ 3D object.

❖ Floats in a fixed location in front of the player.

❖ Physics enabled - responds to the user’s touch.

❖ 2 types: normal and toggle button.

❖ Hover text changes colour to register touch.

How does it work? - Sliders❖ Uses existing UE4 slider

template.

❖ Masked on a 2D plane.

❖ Also hovers in front of player and responds to the player’s touch.

❖ Had to implement algorithm to translate touch location to a point on the slider.

How does it work? - Sliders

CenterTouch location

x

What’s next?❖ Implement locomotion.

❖ Create a means to handle input for users without VR headsets or Leap Motion (using existing UI).

❖ This UI will replace the old EcoVR UI.

Thank you for watching.

Thank you for watching!