christoph anthesroland landertshamer developing vr applications with the invrs framework at ieee...

Christoph Anthes Roland Landertshamer

Developing VR ApplicationsDeveloping VR Applicationswith the with the inVRsinVRs Framework Framework

at IEEE VR’10at IEEE VR’10

Part III – Going ImmersivePart III – Going Immersive

20.03.10 inVRs - Tutorial II - Going Immersive 2

OverviewOverview

• TutorialTutorial– OverviewOverview

– Step 1 – Wrapping FunctionalityStep 1 – Wrapping Functionality

– Step 2 – Immersive DisplaysStep 2 – Immersive Displays

– Step 3 – Using the Input InterfaceStep 3 – Using the Input Interface

– Step 4 – Working With AvatarsStep 4 – Working With Avatars

– Step 5 – Coordinate SystemsStep 5 – Coordinate Systems

• OutlookOutlook• AcknowledgementsAcknowledgements


Tutorial StructureTutorial Structure

• As the previous part of the tutorial this section is based on many copy and As the previous part of the tutorial this section is based on many copy and paste operations in order to safe timepaste operations in order to safe time

• Four basic file classes are needed for this tutorialFour basic file classes are needed for this tutorial– GoingImmersive.cppGoingImmersive.cpp

• Contains the application parts from where the development can startContains the application parts from where the development can start

– Configuration FilesConfiguration Files

• Stored in the config subdirectoryStored in the config subdirectory

• Have to be altered through snippetsHave to be altered through snippets

– CodeFile - SnippetsCodeFile - Snippets

• Contain code snippets which should be copied into the Contain code snippets which should be copied into the GoingImmersive.cpp source fileGoingImmersive.cpp source file

• Can be found under the subdirectory snippetsCan be found under the subdirectory snippets

– ConfigFile - SnippetsConfigFile - Snippets

• Contain XML configuration which has to be copied in the actual Contain XML configuration which has to be copied in the actual configuration filesconfiguration files

• Can be found under the subdirectory snippetsCan be found under the subdirectory snippets

• Open the file Open the file GoingImmersive.cppGoingImmersive.cpp with the editor of your choice with the editor of your choice


Where wrapper classes are introduced to speed up Where wrapper classes are introduced to speed up application development.application development.

Chapter 1Chapter 1Wrapping FunctionalityWrapping Functionality


Wrapping FunctionalityWrapping Functionality

• Initially in the Initially in the Medieval Town TutorialMedieval Town Tutorial a fair bit of set-up operations have a fair bit of set-up operations have been performed in the codebeen performed in the code– They can be wrapped up by the help of so called application basesThey can be wrapped up by the help of so called application bases

– To use an application base an own class has to be derived from an To use an application base an own class has to be derived from an ApplicationBaseApplicationBase

• The developer has the possibility to derive from one of the already The developer has the possibility to derive from one of the already provided application bases in order to create an own application baseprovided application bases in order to create an own application base

• Or the developer can use the basic functionality of the already derived Or the developer can use the basic functionality of the already derived application bases (currently only implemented for OpenSG)application bases (currently only implemented for OpenSG)

• The basic class The basic class ApplicationBaseApplicationBase is an abstract class is an abstract class• It provides the following functions which have to be implemented:It provides the following functions which have to be implemented:

– bool preInit(const CommandLineArgumentWrapper& args)bool preInit(const CommandLineArgumentWrapper& args)• Called right after the constructors as a first initialization stepCalled right after the constructors as a first initialization step

– void initCoreComponentCallback(CoreComponents comp)void initCoreComponentCallback(CoreComponents comp)• Called at core component initialization, it has to be overwritten for Called at core component initialization, it has to be overwritten for

component notificationcomponent notification



– void initInputInterfaceCallback(ModuleInterface* void initInputInterfaceCallback(ModuleInterface* moduleInterface)moduleInterface)

• Called at core component initialization, it has to be overwritten for Called at core component initialization, it has to be overwritten for component notificationcomponent notification

– void initOutputInterfaceCallback(ModuleInterface* void initOutputInterfaceCallback(ModuleInterface* moduleInterface)moduleInterface)


– void initModuleCallback(ModuleInterface* module)void initModuleCallback(ModuleInterface* module)• Called at core component initialization, it has to be overwritten for Called at core component initialization, it has to be overwritten for

component notificationcomponent notification– void bool disableAutomaticModuleUpdate()void bool disableAutomaticModuleUpdate()


– void manualModuleUpdate(float dt)void manualModuleUpdate(float dt)• Called at core component initialization, it has to be overwritten for Called at core component initialization, it has to be overwritten for

component notificationcomponent notification



• Other functions have to be implemented by the application developer:Other functions have to be implemented by the application developer:– std::string getConfigFile(const std::string getConfigFile(const

CommandLineArgumentWrapper& args)CommandLineArgumentWrapper& args)• Must return url to main configuration file (e.g. Must return url to main configuration file (e.g. general.xmlgeneral.xml))

– bool init(const CommandLineArgumentWrapper& args)bool init(const CommandLineArgumentWrapper& args)• Initializes application, called after initialization of Initializes application, called after initialization of inVRsinVRs components components

– void run()void run()• Main application loopMain application loop, , ApplicationBase::globalUpdate()ApplicationBase::globalUpdate()must be must be

called every iterationcalled every iteration– void display(float dt)void display(float dt)

• Called by Called by ApplicationBase::globalUpdate()ApplicationBase::globalUpdate(), used to update , used to update application, dt contains elapsed time in ms since last method call.application, dt contains elapsed time in ms since last method call.

– void cleanup()void cleanup()• System cleanupSystem cleanup



• Other methods which have to be called by the application developer:Other methods which have to be called by the application developer:– bool start(int argc, char** argv)bool start(int argc, char** argv)

• Starts the application, should be called out of main method when Starts the application, should be called out of main method when application instance was createdapplication instance was created

– void globalUpdate()void globalUpdate()• Updates Updates inVRsinVRs components forwards update-command to display method. components forwards update-command to display method.

Must be called out of main application loopMust be called out of main application loop– void globalCleanup()void globalCleanup()

• Cleans up Cleans up inVRsinVRs components, must be called by application before components, must be called by application before finishing, forwards call to cleanup methodfinishing, forwards call to cleanup method



• Besides the provided methods several member variables can be used when Besides the provided methods several member variables can be used when inheriting from the ApplicationBase:inheriting from the ApplicationBase:– SceneGraphInterface* sceneGraphInterfaceSceneGraphInterface* sceneGraphInterface

• Points to the used SceneGraphInterface, if no SceneGraphInterface is used Points to the used SceneGraphInterface, if no SceneGraphInterface is used the pointer value is NULLthe pointer value is NULL

– ControllerManagerInterface* controllerManagerControllerManagerInterface* controllerManager• Pointer to the ControllerManager, NULL if no ControllerManager is usedPointer to the ControllerManager, NULL if no ControllerManager is used

– NetworkInterface* networkModuleNetworkInterface* networkModule• Points to the Network module, NULL if no Network module is usedPoints to the Network module, NULL if no Network module is used

– NavigationInterface* navigationModuleNavigationInterface* navigationModule• Pointer to the Navigation Module, NULL if no ControllerManager is usedPointer to the Navigation Module, NULL if no ControllerManager is used

– InteractionInterface* interactionModuleInteractionInterface* interactionModule• Points to the Interaction module, NULL f no Interaction module is loadedPoints to the Interaction module, NULL f no Interaction module is loaded

– User* localUserUser* localUser• This variable points to the User object for the local user.This variable points to the User object for the local user.

– CameraTransformation* activeCameraCameraTransformation* activeCamera• This variable is a pointer to the camera transformation object of the local This variable is a pointer to the camera transformation object of the local

camera.camera.



• Writing an Writing an inVRsinVRs application using the application using the ApplicationBaseApplicationBase– Class declaration: Inherit from Class declaration: Inherit from OpenSGApplicationBaseOpenSGApplicationBase

• Declaring class membersDeclaring class members– Define variable for url to configuration file, initialize it in constructorDefine variable for url to configuration file, initialize it in constructor

GoingImmersive.cppGoingImmersive.cpp




• Configure path to Plugins in Configure path to Plugins in general.xmlgeneral.xml

• Implement destructorImplement destructor– Call Call OpenSGApplicationBase::globalCleanup()OpenSGApplicationBase::globalCleanup() in order to free all in order to free all

memory reserved by memory reserved by inVRsinVRs components components

general.xmlgeneral.xml




• Implement Implement getConfigFile()getConfigFile()– Must return url to main Must return url to main inVRsinVRs configuration file (e.g. general.xml) configuration file (e.g. general.xml)

– Default config file can already be stored in member variableDefault config file can already be stored in member variable

– Url can be passed via command lineUrl can be passed via command line

• Use Use CommandLineArgumentWrapperCommandLineArgumentWrapper

• Implement Implement initialize()initialize()method method – Called automatically after all Called automatically after all inVRsinVRs components were initialized components were initialized

– Set root node for scene graph as well as initial user transformationSet root node for scene graph as well as initial user transformation

– Use variables Use variables sceneGraphInterfacesceneGraphInterface and and localUserlocalUser, which are inherited , which are inherited from from ApplicationBaseApplicationBase




• Implement Implement initialize()initialize() method method




• Empty display and cleanup methodEmpty display and cleanup method– Don't have to update or cleanup anythingDon't have to update or cleanup anything

• Create instance of application class in Create instance of application class in main()main()methodmethod





Zur Anzeige wird der QuickTime™ Dekompressor „“

benötigt.


Where the use of inVRs in combination with immersive Where the use of inVRs in combination with immersive displays is introduced.displays is introduced.

Chapter 2Chapter 2Immersive DisplaysImmersive Displays


Immersive DisplaysImmersive Displays

• Many types of Immersive Displays are availableMany types of Immersive Displays are available• inVRsinVRs focuses on stereoscopic multi-display installations focuses on stereoscopic multi-display installations

– CAVECAVE

– HMDHMD

– Curved ScreensCurved Screens

– PowerwallsPowerwalls

• inVRsinVRs uses the OpenSG multi-display functionality for setting up uses the OpenSG multi-display functionality for setting up immersive displaysimmersive displays– Thus in general a variety of other displays is possibleThus in general a variety of other displays is possible

• Multi-display functionality abstracted and handled by an external tool, the Multi-display functionality abstracted and handled by an external tool, the CAVE Scene ManagerCAVE Scene Manager

• Other multi-modal output not covered yetOther multi-modal output not covered yet– E.g. haptic displays, motion platforms, etc.E.g. haptic displays, motion platforms, etc.



• Setup examples from the JKUSetup examples from the JKU



• Using the CAVE Scene ManagerUsing the CAVE Scene Manager– Consists of 4 Header files:Consists of 4 Header files:– OSGCAVESceneManager.hOSGCAVESceneManager.h

• Contains main functionality and user API for scene manager, use similar Contains main functionality and user API for scene manager, use similar like SimpleSceneManagerlike SimpleSceneManager

– OSGCAVEConfig.hOSGCAVEConfig.h• Responsible for parsing, loading and setting scene manager configurationResponsible for parsing, loading and setting scene manager configuration

– OSGCAVEWall.hOSGCAVEWall.h• Deals with setup of wall displaysDeals with setup of wall displays

– appctrl.happctrl.h• Contains functionality for starting and shutting down display serversContains functionality for starting and shutting down display servers

• Configuration very similar to the CAVELib configurationConfiguration very similar to the CAVELib configuration



• Using the CAVE Scene ManagerUsing the CAVE Scene Manager– KeywordsKeywords

• WallsWalls– Used to define list of display panesUsed to define list of display panes

• WallDisplayWallDisplay– Detailed wall configuration: name, display, resolution, offsetDetailed wall configuration: name, display, resolution, offset

• ProjectionDataProjectionData– Alignment of different projection panesAlignment of different projection panes

• DisplayModeDisplayMode– Monoscopic (mono) or stereoscopic (stereo)Monoscopic (mono) or stereoscopic (stereo)

• OriginOrigin– Location of coordinate origin in physical spaceLocation of coordinate origin in physical space

• InterocularDistanceInterocularDistance– Eye seperation Eye seperation

• CAVEWidthCAVEWidth and and CAVEHeightCAVEHeight– Width and height in case of CAVE-like displaysWidth and height in case of CAVE-like displays



• Using the CAVE Scene ManagerUsing the CAVE Scene Manager– KeywordsKeywords

• UnitsUnits– Meters, centimeters or footMeters, centimeters or foot

• CAVE Scene Manager has to be enabled in the application baseCAVE Scene Manager has to be enabled in the application base

• Additional rendering servers are neededAdditional rendering servers are needed– server-monoserver-mono

• Provides monoscopic renderingProvides monoscopic rendering

– server-stereoserver-stereo

• Provides stereoscopic renderingProvides stereoscopic rendering

• Rendering servers are automatically started by CAVE Scene ManagerRendering servers are automatically started by CAVE Scene Manager– Typically one per displayTypically one per display



• Using the CAVE Scene ManagerUsing the CAVE Scene Manager– Has to be enabled in the application base by setting Has to be enabled in the application base by setting useCSMuseCSM to true to true

– Define CSM configuration file Define CSM configuration file

– Define automatic startup of render serversDefine automatic startup of render servers

– Define relation between world coordinates and units in real worldDefine relation between world coordinates and units in real world

– Set background image for control windowSet background image for control window

– Set path to CAVE Scene Manager configurationSet path to CAVE Scene Manager configuration

– Set path for loading imagesSet path for loading images

Snippet 1-1Snippet 1-1 general.xml general.xml





• Using the CAVE Scene ManagerUsing the CAVE Scene Manager– Either set path for server-mono or copy the binary into working directoryEither set path for server-mono or copy the binary into working directory

• Typically located at Typically located at inVRs/external/CAVESceneManager-1.0/buildinVRs/external/CAVESceneManager-1.0/build

– Application will show two windows:Application will show two windows:

• Window for render serverWindow for render server

• Control windowControl window

– Set background image for window color:Set background image for window color:

• Build and start applicationBuild and start application– Console output of CAVE Scene Manager should be displayedConsole output of CAVE Scene Manager should be displayed

– If render servers could be startedIf render servers could be started

Snippet 1-1Snippet 1-1


Where the implementation of the use of own input Where the implementation of the use of own input devices is explained.devices is explained.

Chapter 3Chapter 3Using the Input InterfaceUsing the Input Interface


Using the Input InterfaceUsing the Input Interface

• Vast amount of input devices is available in VRVast amount of input devices is available in VR• inVRsinVRs tries to be totally technology agnostic tries to be totally technology agnostic

– Abstraction of input devicesAbstraction of input devices

– Whole input data reduced toWhole input data reduced to

• ButtonsButtons

• AxesAxes

• SensorsSensors



• Configuring your input devicesConfiguring your input devices– Abstract Abstract ControllerController configured inside configured inside InputInterfaceInputInterface– Mapping between devices and abstract Mapping between devices and abstract ControllerController



• Accessing an Abstract Accessing an Abstract ControllerController• int getButtonValue(int idx)int getButtonValue(int idx)

– With this function call the current value of the button can be requested. On the With this function call the current value of the button can be requested. On the button additional callbacks can be registered which are trigged depending on button additional callbacks can be registered which are trigged depending on the button state.the button state.

• float getAxisValue(int idx)float getAxisValue(int idx)– Returns the value of the axis with the given index. Positive and negative Returns the value of the axis with the given index. Positive and negative

floating point values are valid.floating point values are valid.• SensorData getSensorValue(int idx)SensorData getSensorValue(int idx)

– Provides a SensorData object containing transformation data about the sensor Provides a SensorData object containing transformation data about the sensor with the given index.with the given index.

• int getNumberOfButtons()int getNumberOfButtons()– Returns the amount of registered buttons on the abstract controller.Returns the amount of registered buttons on the abstract controller.

• int getNumberOfAxes()int getNumberOfAxes()– Returns the amount of registered axes on the abstract controller.Returns the amount of registered axes on the abstract controller.

• int getNumberOfSensors()int getNumberOfSensors()– Returns the amount of registered sensors on the abstract controller.Returns the amount of registered sensors on the abstract controller.



• Writing own Devices for the Abstract Writing own Devices for the Abstract ControllerController• New device inherits from New device inherits from InputDeviceInputDevice• Methods that have to be implemented:Methods that have to be implemented:

– int getNumberOfButtons()int getNumberOfButtons()• This method must return the number of buttons which are provided by this This method must return the number of buttons which are provided by this

devicedevice– int getNumberOfAxes()int getNumberOfAxes()

• Must return the amount of available axes provided by the deviceMust return the amount of available axes provided by the device– int getNumberOfSensors()int getNumberOfSensors()

• Must return the amount of available sensors provided by the deviceMust return the amount of available sensors provided by the device– int getButtonValue(int idx)int getButtonValue(int idx)

• Must return the value of the button with the passed index. If device does Must return the value of the button with the passed index. If device does not provide button with this index it must return 0not provide button with this index it must return 0

– float getAxisValue(int idx)float getAxisValue(int idx)• Must return the value of the axis with the passed index. If the device does Must return the value of the axis with the passed index. If the device does

not provide an axis with this index it must return 0.not provide an axis with this index it must return 0.



• Writing own Devices for the Abstract Writing own Devices for the Abstract ControllerController• Methods that have to be implemented:Methods that have to be implemented:

– SensorData getSensorValue(int idx)SensorData getSensorValue(int idx)• Must return value of sensor (translation and orientation) with passed index. Must return value of sensor (translation and orientation) with passed index.

If the device does not provide this sensor, If the device does not provide this sensor, IdentitySensorDataIdentitySensorData must must be returnedbe returned

– void update()void update()• Called by Called by ControllerController class once a frame to update value of input device. class once a frame to update value of input device.

• Methods that can be called:Methods that can be called:– void acquireControllerLock()void acquireControllerLock()

• Call if values for input device are updated outside the Call if values for input device are updated outside the update()update() method method or from another thread.or from another thread.

– void releaseControllerLock()void releaseControllerLock()• Release lock after Release lock after acquireControllerLock()acquireControllerLock()

– void sendButtonChangeNotification(int buttonIndex, int void sendButtonChangeNotification(int buttonIndex, int newButtonValuenewButtonValue))

• Called when state of button changesCalled when state of button changes



• Writing own Devices for the Abstract ControllerWriting own Devices for the Abstract Controller– VrpnDeviceVrpnDevice provided (tools/libraries/VRPNDevice) provided (tools/libraries/VRPNDevice)– VrpnDeviceFactoryVrpnDeviceFactory

• Used during loading ControllerManager configurationUsed during loading ControllerManager configuration

• Creates VrpnDeviceCreates VrpnDevice

• Provides single methodProvides single method• create(std::string className , ArgumentVector* args)create(std::string className , ArgumentVector* args)

– Creates class with Creates class with classNameclassName 'VrpnDevice'VrpnDevice' and parameters ' and parameters argsargs parsed form configuration file or NULL if another parsed form configuration file or NULL if another classNameclassName is is passed.passed.



• Interconnect own device with applicationInterconnect own device with application– ControllerManagerControllerManager must be aware of new device must be aware of new device

– Factories must be registeredFactories must be registered

• Include headersInclude headers

• Ifdef statements needed to avoid use of specific VRPN or trackD datatypesIfdef statements needed to avoid use of specific VRPN or trackD datatypes

• Caution: Only insert snippet if VRPN device is availableCaution: Only insert snippet if VRPN device is available




• Interconnect own device with applicationInterconnect own device with application– Register device at Register device at ControllerManagerControllerManager inside inside

initInputInterfaceCallback()initInputInterfaceCallback()– This method is called before This method is called before ControllerManagerControllerManager is configured to allow is configured to allow

creation of device instance as soon as configuration is loadedcreation of device instance as soon as configuration is loaded

– Caution: only insert this snippet if you have a VRPN device you want to useCaution: only insert this snippet if you have a VRPN device you want to use




• Interconnect own device with applicationInterconnect own device with application– ControllerManager can now create TrackdDevices and VrpnDevicesControllerManager can now create TrackdDevices and VrpnDevices

– Configure Configure ControllerManagerControllerManager– Define Define VrpnControllerVrpnController

• Provided in Provided in VrpnController.xmlVrpnController.xml• Consists of single device of type Consists of single device of type VrpnDeviceVrpnDevice• Provides 3 buttons, 2 axes, 2 sensorsProvides 3 buttons, 2 axes, 2 sensors

– Change Change ControllerManagerControllerManager configuration file in configuration file in InputInterfaceInputInterface


– Alter Navigation configuration to use Alter Navigation configuration to use ControllerController effectively effectively


– After starting the application navigation is performed with After starting the application navigation is performed with VrpnDeviceVrpnDevice

Snippet 2-1 Snippet 2-1 inputinterface.xml inputinterface.xml

Snippet 2-2 Snippet 2-2 modules.xml modules.xml


Where animated avatars represent the user.Where animated avatars represent the user.

Chapter 4Chapter 4Working With AvatarsWorking With Avatars

Working With AvatarsWorking With Avatars

• In Medieval Town Tutorial static avatars were usedIn Medieval Town Tutorial static avatars were used– Implemented in class Implemented in class SimpleAvatarSimpleAvatar

• Other types of avatars availableOther types of avatars available– Provided by external package Provided by external package Avatara Avatara

• Developed by Helmut and Martin GarstenauerDeveloped by Helmut and Martin Garstenauer

– Make additional use of gathered tracking dataMake additional use of gathered tracking data

– Provide visual approximation of actual user poseProvide visual approximation of actual user pose

– External modeling tool needed for creating avatarExternal modeling tool needed for creating avatar

– Scripts and tools for Scripts and tools for AvataraAvatara package for exporting avatars exist for package for exporting avatars exist for

• 3D Studio Max3D Studio Max

• MAYAMAYA

• BlenderBlender

• Detailed information on modeling and using avatars is provide in the Detailed information on modeling and using avatars is provide in the Avatara manualAvatara manual

• Avatars can be used as well without inVRs with OpenGL or OpenSGAvatars can be used as well without inVRs with OpenGL or OpenSG




• Modeling AvatarsModeling Avatars– Use mesh skinningUse mesh skinning

– Geometry interconnected with bones of avatarGeometry interconnected with bones of avatar

– Possible to define animation sequencesPossible to define animation sequences

– Gives access to head bone, spine bone, hand position/orientationGives access to head bone, spine bone, hand position/orientation



• Using Using AvataraAvatara– Offers 3 different functionalities:Offers 3 different functionalities:

• Starting, stopping of animation phasesStarting, stopping of animation phases

• Moving head boneMoving head bone

• Setting hand position and orientationSetting hand position and orientation

– Configuration file for Configuration file for AvataraAvatara avatar avatar

undead.xmlundead.xml



• Integrating an Integrating an AvataraAvatara avatar avatar– WorldDatabaseWorldDatabase needs needs AvataraAvatarFactoryAvataraAvatarFactory to create to create

AvataraAvatarAvataraAvatar– Include header containing factoryInclude header containing factory

– Register factory at Register factory at WorldDatabaseWorldDatabase in in initCoreComponetCallback()initCoreComponetCallback()– Avatar configuration is referenced by Avatar configuration is referenced by UserDatabaseUserDatabase although the although the WorldWorld

DatabaseDatabase is responsible for loading the avatar is responsible for loading the avatar

– Checks for callback for the Checks for callback for the UserDatabaseUserDatabase initialization initialization

– Define configuration for avatar in Define configuration for avatar in UserDatabaseUserDatabase configuration configuration



Snippet 3-1 Snippet 3-1 userDatabase.xml userDatabase.xml



• Integrating an Integrating an AvataraAvatara avatar avatar– Update avatar transformationUpdate avatar transformation

• Define 2 variablesDefine 2 variables

– Pointer to avatarPointer to avatar

– Defining initial transformationDefining initial transformation

– Initialize these variables in constructorInitialize these variables in constructor

– Obtain pointer to avatar from Obtain pointer to avatar from locallocalUserUser in in initialize()initialize()






• Integrating Integrating AvataraAvatara avatar avatar– Update transformation of avatarUpdate transformation of avatar

– Request transformation of user relative to tracking system centerRequest transformation of user relative to tracking system center

– Default tracking system has at least one sensor for head transformation, one Default tracking system has at least one sensor for head transformation, one for handfor hand

– This does not provide correct position of user relative to tracking system centerThis does not provide correct position of user relative to tracking system center– UserTransformationModelUserTransformationModel calculates this position calculates this position

– Default: Default: HeadPositionUserTransformationModelHeadPositionUserTransformationModel• Subtracts height value from head transformationSubtracts height value from head transformation

– Update transformation once per frameUpdate transformation once per frame





• Test avatar without tracking systemTest avatar without tracking system– Use Use GlutSensorEmulatorDeviceGlutSensorEmulatorDevice– Simulates tracking systemSimulates tracking system

– ControllerManagerConfiguration: ControllerManagerConfiguration: MouseKeybSensorController.xmlMouseKeybSensorController.xml• GlutMouseDeviceGlutMouseDevice• GlutKeyboardDeviceGlutKeyboardDevice• GlutSensorEmulatorDeviceGlutSensorEmulatorDevice

– Simulates 2 sensorsSimulates 2 sensors

– Switch between normal navigation mode and sensor emulation with keyboard Switch between normal navigation mode and sensor emulation with keyboard button 1button 1

– In sensor emulation mode switch between sensors by pressing left mouse In sensor emulation mode switch between sensors by pressing left mouse buttonbutton

– Switch between manipulation of translation and rotation with middle mouse Switch between manipulation of translation and rotation with middle mouse buttonbutton

– Switch between manipulation along X and Y axis by pressing right mouse Switch between manipulation along X and Y axis by pressing right mouse buttonbutton


Where an introduction into and a visualization of Where an introduction into and a visualization of inVRsinVRs' ' coordinate systems is provided.coordinate systems is provided.

Chapter 5Chapter 5Coordinate SystemsCoordinate Systems


Coordinate SystemsCoordinate Systems

• inVRsinVRs coordinate systems are similar to OpenGL coordinate systems are similar to OpenGL– X-axis: point to the rightX-axis: point to the right

– Y-axis: points to topY-axis: points to top

– Z-axis: towards the observer out of the screenZ-axis: towards the observer out of the screen

• User CoordinatesUser Coordinates

1.) Origin of the VE1.) Origin of the VE

2.) Navigated transformation2.) Navigated transformation

3.) User transformation3.) User transformation

4.) Hand transformation4.) Hand transformation

5.) Head transformation5.) Head transformation



• User CoordinatesUser Coordinates– Navigated transformationNavigated transformation

• Result of navigation process provided by navigation moduleResult of navigation process provided by navigation module

• Avatar is placed at origin of this coordinate system if no other tracking Avatar is placed at origin of this coordinate system if no other tracking information providedinformation provided

• Considered the origin of the VEConsidered the origin of the VE

– User transformationUser transformation

• When tracking system available, tracked position added to navigated When tracking system available, tracked position added to navigated position to obtain avatar transformationposition to obtain avatar transformation

• If no tracking system head tracking data set to identity matrixIf no tracking system head tracking data set to identity matrix

• Can be represented either as:Can be represented either as:

– Tracked User TransformationTracked User Transformation

» Relative to center of tracking systemRelative to center of tracking system

– World User TransformationWorld User Transformation

» Relative to origin of VERelative to origin of VE

» worldUserTrans = navigatedTrans worldUserTrans = navigatedTrans ∗∗ trackedUserTrans trackedUserTrans



• User CoordinatesUser Coordinates– Hand transformationHand transformation

• Controller index 1Controller index 1• Relative to tracking system centerRelative to tracking system center

– Transformation is provided by hand sensorTransformation is provided by hand sensor• Relative to user transformationRelative to user transformation

– userHandTrans = trackedUserTrans ¹ ⁻userHandTrans = trackedUserTrans ¹ ⁻ ∗∗ trackedHandTrans trackedHandTrans• Relative to origin of VERelative to origin of VE

– worldHandTrans = navigatedTrans worldHandTrans = navigatedTrans ∗∗ trackedHandTrans trackedHandTrans– Head transformationHead transformation

• Controller index 0Controller index 0• Relative to tracking system centerRelative to tracking system center

– Transformation is provided by head sensorTransformation is provided by head sensor• Relative to user transformationRelative to user transformation

– userHeadTrans = trackedUserTrans ¹ ⁻userHeadTrans = trackedUserTrans ¹ ⁻ ∗∗ trackedHeadTrans trackedHeadTrans• Realtive to origin of VERealtive to origin of VE

– worldHeadTrans = navigatedTrans worldHeadTrans = navigatedTrans ∗∗ trackedHeadTrans trackedHeadTrans



• User CoordinatesUser Coordinates– Sensor transformationSensor transformation

• Transformation for any additional sensorTransformation for any additional sensor

• Relative to userRelative to user

– userSensorTrans = trackedUserTrans ¹ ⁻userSensorTrans = trackedUserTrans ¹ ⁻ ∗∗ trackedSensorTrans trackedSensorTrans

• Relative to worldRelative to world

– worldSensorTrans = navigatedTrans worldSensorTrans = navigatedTrans ∗∗ trackedSensorTrans trackedSensorTrans

– Cursor transformationCursor transformation

• Relative to origin of virtual worldRelative to origin of virtual world

• Different models provided by Different models provided by inVRsinVRs

– Camera transformationCamera transformation

• Corresponds to navigated transformation of user by defaultCorresponds to navigated transformation of user by default

• Calculated and constantly updated in Calculated and constantly updated in TransformationManagerTransformationManager



• Visualizing TransformationsVisualizing Transformations– Visualized using 3D models in form of coordinate systemVisualized using 3D models in form of coordinate system

• Tracked user transformationTracked user transformation

• Tracked sensor transformation for each sensorTracked sensor transformation for each sensor

– Number of entities to create corresponds to number of sensors provided by Number of entities to create corresponds to number of sensors provided by ControllerController

– Store number of sensorsStore number of sensors




• Visualizing TransformationsVisualizing Transformations– Obtain value of Obtain value of numberOfSensorsnumberOfSensors in in initialize()initialize()– Request controller form Request controller form ControllerManagerControllerManager– Store number of sensors of controllerStore number of sensors of controller

– Create entities for each sensor by calling Create entities for each sensor by calling WorldDatabase::createEntity()WorldDatabase::createEntity()

• First parameter ID of First parameter ID of EntityTypeEntityType, defined in , defined in entities.xmlentities.xml• Second parameter ID of Second parameter ID of EnvironmentEnvironment




• Visualizing TransformationsVisualizing Transformations– Update transformation of coordinate system entitiesUpdate transformation of coordinate system entities

– Method called Method called updateCoordinateSystems()updateCoordinateSystems()– Obtain pointer to Obtain pointer to EntityTypeEntityType from from WorldDatabaseWorldDatabase– Request all instances (coordinate system entities) from Request all instances (coordinate system entities) from EntityTypeEntityType– Set transformation of entity representing tracked user transformation to Set transformation of entity representing tracked user transformation to

transformation obtained from local transformation obtained from local UserUser and add position offset to platform and add position offset to platform centercenter

– Iterate over available sensorsIterate over available sensors

• Update tracked sensor transformationsUpdate tracked sensor transformations

• Request transformation form local Request transformation form local UserUser• Add to platform centerAdd to platform center



• Visualizing TransformationsVisualizing Transformations




• Visualizing TransformationsVisualizing Transformations– Call Call updateCoordinateSystems()updateCoordinateSystems()continuouslycontinuously

– Add call in Add call in update()update()



OutlookOutlook


OutlookOutlook

• We have learned how toWe have learned how to– Use the ApplicationBase in order to wrap functionality for faster application Use the ApplicationBase in order to wrap functionality for faster application

developmentdevelopment

– Use the configuration of the CAVE Scene Manger to ease access to Use the configuration of the CAVE Scene Manger to ease access to OpenSG multi-display functionalityOpenSG multi-display functionality

– Interconnect arbitrary (own) input devices with Interconnect arbitrary (own) input devices with inVRsinVRs

– Make use of articulated avatarsMake use of articulated avatars

– Work with the different coordinate systemsWork with the different coordinate systems


OutlookOutlook

• Variety of additional tools are available (some of the internal)Variety of additional tools are available (some of the internal)– Modules Modules

• Physics support (2D and 3D Physics)Physics support (2D and 3D Physics)

• Animation and camera pathAnimation and camera path

– External Tools External Tools

• Collaborative editorCollaborative editor

– Scene Graph Specific Enhancements (OpenSG) Scene Graph Specific Enhancements (OpenSG)

• Avatars (with export from 3D Max, MAYA, and Blender)Avatars (with export from 3D Max, MAYA, and Blender)

• CAVE Scene ManagerCAVE Scene Manager

• Particle WrapperParticle Wrapper

• SkyboxSkybox

• 3D Menu System3D Menu System

• Coming soonComing soon– inVRs port to OpenSG 2 (April - thanks to Georg Stevenson)inVRs port to OpenSG 2 (April - thanks to Georg Stevenson)

– AR input and output (Spring)AR input and output (Spring)

– Variety of example projects (Summer)Variety of example projects (Summer)


OutlookOutlook

• Novel conceptsNovel concepts– Clear distinction in the area of communication patternsClear distinction in the area of communication patterns

– Events and transformations are handled independentlyEvents and transformations are handled independently

– Transformation management allows for concurrent object manipulationTransformation management allows for concurrent object manipulation

– Composeable and clearly defined interaction and navigationComposeable and clearly defined interaction and navigation

– Automatic data distribution mechanism Automatic data distribution mechanism

– Full abstraction from graphics, input, network, etc.; the developer should Full abstraction from graphics, input, network, etc.; the developer should be able to focus on the application logicbe able to focus on the application logic

– Development and integration of networked physics module (Roland Development and integration of networked physics module (Roland Landertshamer)Landertshamer)

• Does not provide yetDoes not provide yet– Scripting supportScripting support

– Logic supportLogic support


AcknowledgementsAcknowledgements

• ContributorsContributors– Architecture and Main DevelopmentArchitecture and Main Development

Christoph Anthes, Helmut Bressler, Roland Landertshamer, Marina LengerChristoph Anthes, Helmut Bressler, Roland Landertshamer, Marina Lenger

– ToolsTools

Helmut Garstenauer, Martin Garstenauer, Adrian Haffegee, Marlene Helmut Garstenauer, Martin Garstenauer, Adrian Haffegee, Marlene Hochrieser, Roland Hopferwieser, Robert Owen, Stephan Reiter, Thomas Hochrieser, Roland Hopferwieser, Robert Owen, Stephan Reiter, Thomas Weberndorfer, Christian Wressnegger, Johannes ZarlWeberndorfer, Christian Wressnegger, Johannes Zarl

Questions ?Questions ?


BibliographyBibliography

• Christoph AnthesChristoph AnthesA Collaborative Interaction Framework for Networked Virtual Environments. A Collaborative Interaction Framework for Networked Virtual Environments. PhD PhD Thesis at Institute of Graphics and Parallel Processing at JKU Linz, Austria, August 2009.Thesis at Institute of Graphics and Parallel Processing at JKU Linz, Austria, August 2009.

• Christoph Anthes, Philipp Aumayr, Clemens Birklbauer, Roland Hackl, Marlene Hochrieser, Christoph Anthes, Philipp Aumayr, Clemens Birklbauer, Roland Hackl, Marlene Hochrieser, Roland Hopferwieser, Simon Opelt, Christoph Payrhuber, Mika Satomi, Stefan Simmer, Roland Hopferwieser, Simon Opelt, Christoph Payrhuber, Mika Satomi, Stefan Simmer, Georg Stevenson, Roland Landertshamer, Bernhard Lehner, Marina Lenger, Martin Georg Stevenson, Roland Landertshamer, Bernhard Lehner, Marina Lenger, Martin Lenzelbauer, Clemens Mock, and Alexander Wilhelm.Lenzelbauer, Clemens Mock, and Alexander Wilhelm.Space Trash – Development of a Networked Immersive Virtual Reality Installation. Space Trash – Development of a Networked Immersive Virtual Reality Installation. Technical Report at Institute of Graphics and Parallel Processing, Johannes Kepler Technical Report at Institute of Graphics and Parallel Processing, Johannes Kepler University Linz, June 2009. University Linz, June 2009.

• Christoph Anthes, Mika Satomi, Alexander Wilhelm, Christa Sommerer, and Jens Volkert. Christoph Anthes, Mika Satomi, Alexander Wilhelm, Christa Sommerer, and Jens Volkert. Space Trash – An Interactive Networked Virtual Reality Installation. Space Trash – An Interactive Networked Virtual Reality Installation. In Proceedings of In Proceedings of thethe 11th Virtual Reality International Conference (VRIC ’09) 11th Virtual Reality International Conference (VRIC ’09), pages 107-118, Laval, France, , pages 107-118, Laval, France, April 2009. April 2009.

• Christoph Anthes, Mika Satomi, Alexander Wilhelm, Christa Sommerer, and Jens Volkert. Christoph Anthes, Mika Satomi, Alexander Wilhelm, Christa Sommerer, and Jens Volkert. Design and Setup of Space Trash – A Collaborative Virtual Reality Installation using Design and Setup of Space Trash – A Collaborative Virtual Reality Installation using Physical Interfaces. IPhysical Interfaces. Installation description in Proceedings of thenstallation description in Proceedings of the 11th Virtual Reality 11th Virtual Reality International Conference (VRIC ’09) ReVolutionInternational Conference (VRIC ’09) ReVolution, pages 355-362, Laval, France, April 2009. , pages 355-362, Laval, France, April 2009.



• Christoph Anthes, Roland Landertshamer, Helmut Bressler, and Jens Volkert.Christoph Anthes, Roland Landertshamer, Helmut Bressler, and Jens Volkert.Developing VR Applications for the Grid. Developing VR Applications for the Grid. In In 1st International Workshop on Real–Time 1st International Workshop on Real–Time Online Interactive Applications (ROIA) on the GRIDOnline Interactive Applications (ROIA) on the GRID at the 14th International Euro-Par at the 14th International Euro-Par Conference European Conference on Parallel and Distributed Computing (Euro-Par ’08), Conference European Conference on Parallel and Distributed Computing (Euro-Par ’08), pages 343–351, Las Palmas de Gran Canaria, Spain, August 2008. pages 343–351, Las Palmas de Gran Canaria, Spain, August 2008.

• Eva Pajorová, Ladislav Hluchý, and Christoph Anthes.Eva Pajorová, Ladislav Hluchý, and Christoph Anthes.3D Geovisualization Service for Grid-oriented applications of Natural Disasters.3D Geovisualization Service for Grid-oriented applications of Natural Disasters.Poster in the 16th International Conference in Central Europe on Computer Graphics, Poster in the 16th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision (WSCG '08), Plzen, Czech Republic, February 2008. Visualization and Computer Vision (WSCG '08), Plzen, Czech Republic, February 2008.

• Christoph Anthes, Alexander Wilhelm, Roland Landertshamer, Helmut Bressler, and Jens Christoph Anthes, Alexander Wilhelm, Roland Landertshamer, Helmut Bressler, and Jens Volkert.Volkert.Net’O’Drom – An Example for the Development of Networked Immersive VR Net’O’Drom – An Example for the Development of Networked Immersive VR Applications. Applications. In International Conference on Computational Science (ICCS '07), pages In International Conference on Computational Science (ICCS '07), pages 752-759, Beijing, China, May 2007. 752-759, Beijing, China, May 2007.

• Christoph Anthes, Roland Landertshamer, and Jens Volkert.Christoph Anthes, Roland Landertshamer, and Jens Volkert.Physically-Based Interaction for Networked Virtual Environments. Physically-Based Interaction for Networked Virtual Environments. In International In International Conference on Computational Science (ICCS '07), pages 776-783, Beijing, China, May Conference on Computational Science (ICCS '07), pages 776-783, Beijing, China, May 2007. 2007.



• Christoph Anthes, Roland Landertshamer, Helmut Bressler, and Jens Volkert.Christoph Anthes, Roland Landertshamer, Helmut Bressler, and Jens Volkert.Managing Transformations and Events in Networked Virtual Environments. Managing Transformations and Events in Networked Virtual Environments. In ACM In ACM Multimedia Modeling (MMM '07), pages 722-729, Singapore, January 2007.Multimedia Modeling (MMM '07), pages 722-729, Singapore, January 2007.

• Helmut Bressler, Roland Landertshamer, Christoph Anthes, and Jens Volkert.Helmut Bressler, Roland Landertshamer, Christoph Anthes, and Jens Volkert.An efficient Physics Engine for Virtual Worlds.An efficient Physics Engine for Virtual Worlds. In In medi@terramedi@terra, pages 152-158, Athens, , pages 152-158, Athens, Greece, October 2006.Greece, October 2006.

• Christoph Anthes and Jens Volkert.Christoph Anthes and Jens Volkert.inVRs - A Framework for Building Interactive Networked Virtual Reality Systems. inVRs - A Framework for Building Interactive Networked Virtual Reality Systems. In In International Conference on High Performance Computing and Communications (HPCC International Conference on High Performance Computing and Communications (HPCC '06)'06), pages 894-904, Munich, Germany, September 2006. , pages 894-904, Munich, Germany, September 2006.

• Christoph Anthes, Adrian Haffegee, Paul Heinzlreiter, and Jens Volkert.Christoph Anthes, Adrian Haffegee, Paul Heinzlreiter, and Jens Volkert.A Scalable Network Architecture for Closely Coupled Collaboration. A Scalable Network Architecture for Closely Coupled Collaboration. In In Journal of Journal of Computing and InformaticsComputing and Informatics, 24(1):31-51, August 2005. , 24(1):31-51, August 2005.

• Christoph Anthes and Jens Volkert.Christoph Anthes and Jens Volkert.A Toolbox Supporting Collaboration in Networked Virtual Environments.A Toolbox Supporting Collaboration in Networked Virtual Environments. In In International Conference on Computational Science (ICCS '05)International Conference on Computational Science (ICCS '05), pages 383-390, Atlanta, , pages 383-390, Atlanta, GA, USA, May 2005. GA, USA, May 2005.



• Adrian Haffegee, Ronan Jamieson, Christoph Anthes, and Vassil N. Alexandrov.Adrian Haffegee, Ronan Jamieson, Christoph Anthes, and Vassil N. Alexandrov.Tools For Collaborative VR Application Development. Tools For Collaborative VR Application Development. In In International Conference on International Conference on Computational Science (ICCS '05)Computational Science (ICCS '05), pages 350-358, Atlanta, GA, USA, May 2005., pages 350-358, Atlanta, GA, USA, May 2005.

• Christoph Anthes, Paul Heinzlreiter, Adrian Haffegee, and Jens Volkert.Christoph Anthes, Paul Heinzlreiter, Adrian Haffegee, and Jens Volkert.Message Traffic in a Distributed Virtual Environment for Close-coupled Collaboration.Message Traffic in a Distributed Virtual Environment for Close-coupled Collaboration. In In Parallel and Distributed Computing and Systems (PDCS '04)Parallel and Distributed Computing and Systems (PDCS '04), pages 484-490, San , pages 484-490, San Francisco, CA, USA, September 2004. Francisco, CA, USA, September 2004.

• Christoph Anthes, Paul Heinzlreiter, Gerhard Kurka, and Jens Volkert.Christoph Anthes, Paul Heinzlreiter, Gerhard Kurka, and Jens Volkert.Navigation models for a flexible, multi-mode VR Navigation Framework. Navigation models for a flexible, multi-mode VR Navigation Framework. In In ACM ACM SIGGRAPH International Conference on Virtual Reality Continuum and its Applications in SIGGRAPH International Conference on Virtual Reality Continuum and its Applications in Industry (VRCAI '04)Industry (VRCAI '04), pages 476-479, Singapore, June 2004. , pages 476-479, Singapore, June 2004.

• Christoph Anthes, Paul Heinzlreiter, and Jens Volkert.Christoph Anthes, Paul Heinzlreiter, and Jens Volkert.An adaptive Network Architecture for Close-coupled Collaboration in Distributed An adaptive Network Architecture for Close-coupled Collaboration in Distributed Virtual Environments. Virtual Environments. In In ACM SIGGRAPH International Conference on Virtual Reality ACM SIGGRAPH International Conference on Virtual Reality Continuum and its Applications in Industry (VRCAI '04)Continuum and its Applications in Industry (VRCAI '04), pages 382-385, Singapore, June , pages 382-385, Singapore, June 2004. 2004.

christoph anthesroland landertshamer developing vr applications with the invrs framework at ieee...

Documents