vr-oos system architecture - dlr portal€¦ · - bi-manual hmi, phantom omni, falcon dlr bi -...

VR-OOS System Architecture Workshop zu interaktiven VR-Technologien für On-Orbit Servicing

Robin Wolff DLR, Simulation and Software Technology

www.DLR.de • Chart 1 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012

Overview

- Motivation - Physical Mock-up - Servicing Tasks

- System Overview - Components - Requirements

- System Architecture - Distributed Simulation Framework - Simulation Loop - End-to-End Latency

- Simulation Module Implementations - Physics - Visualization

- Summary


Motivation Physical Evaluation Mock-up

- Tele-Presence System demonstrated at ILA 2010 in Berlin


Motivation Physical Evaluation Mock-up Virtual Evaluation Mock-up

- Tele-Presence System demonstrated at ILA 2010 in Berlin


Motivation Servicing Tasks

- Remove Multi-Layer Insulation (MLI)

- Loosen / Tighten Screws

- Remove / Insert Modules (e.g. using a Bayonet Handle)

- Operate Switches

- Take Measurements (e.g. using a Voltmeter)


EVA Task Sheet

Virtual Satellite Mock-up Physical Satellite Mock-up

System Overview System Components


Collision Detection & Force Computation

Physics Simulation

Visualization

VR Display

Haptic Device Virtual Service Robot Model

Virtual Target Satellite Model

Image

xuser xhd

xobject

xuser

xobject

xtracking

Fhd

Object Properties (Geometry, Material, Constraints, etc.)

System Overview Requirements


Central Control / Manager

Collision Detection, Force Computation Rate: >1kHz

Physics Simulation Rate: ~60-100Hz

Network

…

Visualization Rate: ~30-120Hz

- Interactive Real-Time Simulation

- Fast Response Times necessary

- Complexity & Accuracy Trade-off

- Dedicated Machines used for Computation

System Architecture Distributed Simulation Framework

- Distributes Simulation across dedicated Machines

- Each Simulation runs at its own Rate and only sees its own “World”

- Simulations are synchronized via Network Messages


… Server Client Client Client

Physics

Scene Simulation

Communication

Visualization

Scene Simulation

Communication

Haptics

Scene Simulation

Communication

Manager

Scene Simulation

Communication

Network

- Wraps Common Functionality

- Unified Scene Management

- Asynchronous Message Passing

- Transparent Processing Loop

System Architecture Common Simulation Loop


Process Updates

Interpret Updates

Collect Changes

Common

Extra

Scene Objects & State

Read & Modify

Recv. Send

Processing Loop

Distribute Changes Step Simulation

Scene Description (COLLADA)

Load Scene

System Architecture Distributed Simulation Framework

- Includes Optimizations for Network Throughput - Communicates via UDP/TCP/IPC Channels - Unlimited Number of Communication Channels - Configurable Queuing Schemes (e.g. FIFO, Most-Recent) - Optional Spatial and Temporal Threshold Filters

- What about Network Issues?

- Latency, Jitter, Loss, Bandwidth


… Server Client Client Client

Physics

Scene Simulation

Communication

Visualization

Scene Simulation

Communication

Haptics

Scene Simulation

Communication

Manager

Scene Simulation

Communication

Network

System Architecture End-to-End Latency


(RTT between all hosts = 0.2ms)


Physics Simulation

Visualization

VR Display



Image

xuser xhd

xobject

xuser

xobject

xtracking

Fhd


host.59, host.62

host.62 host.61



A. Time between Hand Movement and feeling the Surface of static, not moving Objects


LatencyA = 0.80 ms


Physics Simulation

Visualization

VR Display



Image

xuser xhd

xobject

xuser

xobject

xtracking

Fhd


host.59, host.62

host.62 host.61

(thap= 0.18ms)



B. Time between Hand Movement and seeing the Hand move


LatencyB = 6.09 ms


Physics Simulation

Visualization

VR Display



Image

xuser xhd

xobject

xuser

xobject

xtracking

Fhd


xhd host.59, host.62

host.62

(thap= 0.18ms)

host.61

(tvis= 3.30ms)



C. Time until User sees pushed Object moving


LatencyC = 7.15 ms


Physics Simulation

Visualization

VR Display



Image

xuser xhd

xobject

xuser

xobject

xtracking

Fhd


host.59, host.62

host.62 host.61

(thap= 0.18ms)

(tvis= 3.30ms)

(tphy= 0.02ms)



D. Time between Hand Movement and feeling the Surface of moving Object


LatencyD = 2.82 ms


Physics Simulation

Visualization

VR Display



Image

xuser xhd

xobject

xuser

xobject

xtracking

Fhd


host.59, host.62

host.62 host.61

(thap= 0.18ms)

(tvis= 3.30ms)

(tphy= 0.02ms)

Implemented Simulation Modules Haptics, Extensions

- Haptics - Collision Detection and

Force Computation A. Improved Voxmap-Point-Shell (VPS) B. OpenHaptics Toolkit

- Interfaces - Bi-manual HMI, Phantom Omni, Falcon

- Extensions - Finger Tracking - Speech Recognition - Logging - More coming …


VPS Algorithm

DLR Bi-manual HMI Phantom Omni Novint Falcon

ART Finger Tracking

Implemented Simulation Modules Physics

- Uses Bullet Physics Engine - Real-Time n-Body Simulation - Optimized for Speed,

rather than Accuracy - Support for Rigid Bodies - Simplified Collision Detection - Universal Constraints - Support for Soft Bodies


- Alternatives:

Implemented Simulation Modules Visualization

- InstantReality - Developed at

Fraunhofer IGD - Widely used at DLR RM

- ViSTA VR-Toolkit - Developed at RWTH

Aachen (+10 Years) - Now in Cooperation

between RWTH & DLR - Open Source - Based on OpenSG

(soon OpenSceneGraph) - Support for Scientific

Visualization Methods


Visualization using ViSTA VR-Toolkit



(Columbus EVA, ESA)


- OpenGL Shader Support

- Simple Shadow Mapping

- Realistic Star Background - Positions based on

Hipparchos and Tycho-2 Star Catalogues

- Realistic Earth - High-Resolution Textures - Night Lights - Clouds with Shadows - Atmospheric Scattering


OpenGL Shader (top), Realistic Stars (bottom)



Rendering of Earth with High-Res. Textures, Night Lights, Atmospheric Scattering, Clouds with Shadows.


- NVIDIA OptiX (work in progress) - Toolkit for Real-Time

Ray-Tracing - Allows accurate Simulation

of Light Conditions


Rendering Ray-Tracing Photograph

- Creates High-Quality Shadows, Reflection, Refraction

- Could also be used to generate virtual Camera Images to evaluate Image-Recognition Algorithms

Summary

- VR-OOS: a Virtual Reality Environment for On-Orbit Servicing

- Software Framework provides a generic Architecture for a distributed interactive Real-Time Simulation Environment

- Low-Latency Response Times

- Several Simulation Modules implemented and in use

- Live Demo Today! 15:15-16:30


Contact

Dr Robin Wolff

German Aerospace Center (DLR)

Simulation and Software Technology Software for Space Systems and Interactive Visualization

Lilienthalplatz 7 38108 Braunschweig Germany www.dlr.de/sc

[email protected]


vr-oos system architecture - dlr portal€¦ · - bi-manual hmi, phantom omni, falcon dlr bi -...

Documents