technical overview

BPC: Art and Computation – Summer 2008BPC: Art and Computation – Summer 2008

Technical OverviewTechnical Overview

Erik [email protected]

BPC: Art and Computation – Summer 2008 2

What's in a virtual world?What's in a virtual world?


What's in a virtual world?What's in a virtual world? An environment Representations of physical objects

– Visual models– Auditory elements

Interactions with objects Representations of people - avatars Communication

– Gesturing– Telephony– Video


Student work in VR contextStudent work in VR contextProduce images (texture maps)Produce objects (3D models)Produce sounds (audio files)Produce movement (sequences)Produce behavior (logo programs)Arrange in space (config scripts)


Software usedSoftware usedCorel Painter: Image processing and

painting, 2D animationAudio: AudacityModeling: MayaProgramming: LogoVR: BU’s DAFFIE


Images into VEImages into VE

++


2D Animation into VE2D Animation into VE


Defining geometryDefining geometry

hquad, 2, 2, 4, 2, 4, 4, 2, 4, 0hquad, 2, 2, 4, 2, 4, 4, 2, 4, 0vrect, 2, 5, 4, 5, 2vrect, 2, 5, 4, 5, 2vrect, 4, 5, 4, 7, 2vrect, 4, 5, 4, 7, 2vrect, 4, 7, 2, 7, 2vrect, 4, 7, 2, 7, 2vrect, 2, 7, 2, 5, 2vrect, 2, 7, 2, 5, 2vcyl, 0, 2, 0.5, 3vcyl, 0, 2, 0.5, 3vcone, 2, -5, 1, 2vcone, 2, -5, 1, 2vrect, -1.5, -3, 0, -3, 2vrect, -1.5, -3, 0, -3, 2vrect, 0, -3, 2, -1, 2vrect, 0, -3, 2, -1, 2vrect, 2, -1, 0.5, -1, 2vrect, 2, -1, 0.5, -1, 2vrect, 0.5, -1, -1.5, -3, 2vrect, 0.5, -1, -1.5, -3, 2hquad, 2, -5, 4, -3, 2, -1, 0, -3, 2hquad, 2, -5, 4, -3, 2, -1, 0, -3, 2


Animation of 3D modelsAnimation of 3D models

Model sequence What is changing?

– Shape– Elements

• movement• Rotation• Scale

– Whole object• Movement• Rotations• Scale


Sound mappingSound mapping

++


Programming in VE: LogoProgramming in VE: Logo


3D Models and motion: Maya3D Models and motion: Maya


Scene description fileScene description file


DAFFIEDAFFIE Distributed

InteractiveVisually and aurally richInput (models, images and sounds) from

variety of available toolsSimple import methodScript drivenAccessible to non-expert usersExtensible by programmers


The DAFFIE scene graphThe DAFFIE scene graph

The world-room-group-sequence hierarchy

World

Room Room

Group GroupGroup Group

Sequence Sequence Sequence

Sequence

Sequence Sequence


arp script example: basic arp script example: basic sequencesequencearp 2.0name clusterseqgeom_path ./playspeed 10frame geom_show clustergeom0000.objframe geom_show clustergeom0001.objframe geom_show clustergeom0002.objframe geom_show clustergeom0003.objend


Example - adding soundExample - adding soundframe geom_show clustergeom0000.obj sound_xyz_stereo 180.932040 -157.771123 -103.679324 180.932040 -43.291371 -103.679324 sound_move h2o4.aif sound_trigger h2o4.aifframe geom_show clustergeom0001.obj sound_xyz_stereo 187.016266 -157.446847 -103.679324 174.847813 -43.615647 -103.679324 sound_move h2o4.aifframe geom_show clustergeom0002.obj sound_xyz_stereo 193.031556 -156.477695 -103.679324 168.832524 -44.584799 -103.679324 sound_move h2o4.aif


arp script example: move objectarp script example: move objectframe geom_show pot21.iv geom_xyz 5.0 2.1 9.3 geom_hpr 0.0 90.0 0.0 geom_show drop21.objframe geom_show pot22.iv geom_xyz 5.5 1.9 3.2 geom_hpr 0.0 91.0 0.0 geom_show drop22.obj


arp script: arpgroup filearp script: arpgroup filearpg 2.0

name gbdog

geom_size 1.0geom_path ./geom_hpr 0 90 0geom_xyz 0.0 0.1 0.0geom_rep dog0.objtrigger_method cycleseq_path ./seq_hpr 0 90 0seq_xyz 0.0 0.1 0.0seq_file gbdog.seq

end


arp scripts: arproom filearp scripts: arproom filearproom 2.0

name cart-modelsenter_xyz 3.0 4.0 4.0enter_hpr 30.0 -98.0 0.0

group_xyz 0 0 0group_hpr 0 0 0group_path ../models/purple_cube/group purple_cube.arpg

group_xyz 2 -4 0group_hpr 60 0 0group_path ../models/blue_cube/group blue_cube.arpg

group_xyz -3 -5 2group_hpr 0 0 0group_path ../models/orange_sphere/group orange_sphere.arpg

end


The DAFFIE event systemThe DAFFIE event system

Communication between agents

Navigator Viewer

Navigator

Viewer

Sound server

Event Server

App agent

App agent

App agent

Telephony Video agent

Sound server


3D Coordinate system3D Coordinate system


Navigation - user experienceNavigation - user experience

6 degrees of freedomMap onto various input devicesCommunication with viewer (old

model) - direct sending of xyz,hpr wand motion to viewer

New model – agent sending of transform matrices to viewer


Triggering objectsTriggering objects

Light stick (by viewer)Proximity (by viewer)Proximity (by agent)Other (by agent)The trigger event


World state and eventsWorld state and events

Object descriptionsTriggerTransformsSwitchesNavigationSoundVideoEtc.


Agents and eventsAgents and events

Using events to control elements of the world– Triggering sequences– Moving objects– Changing rooms– Navigation


TelephonyTelephony Telephony collector agent

– Gets audio stream from microphone– Chops into sound samples– Sends sequence of sound samples

Sound server– Receives sequence of sound samples– Reassembles audio stream from sound

samples– Sends reconstructed audio stream to speakers


Video texture mapsVideo texture maps

Video as sequence of imagesVideo generator

– Sends sequence of imagesViewer

– Receives sequence of images– Uses texture mapping on an object in

scene


Stereo displaysStereo displays

Physical issues and camera issues– Using polarizing filters– Setting up the view– What looks closer vs further– Relative position to the screen and

viewer


Tiled wallsTiled walls

Issues and differences– Head node vs render nodes– Event management / avatars / triggering– Virtual cameras– Alignment


The EndThe End

technical overview

Documents

computation summer

ve bpc

sound mapping bpc

objframe geom

aifframe geom

soundframe geom

d modelsproduce

d animationaudio