ENV 2006 16.1

Envisioning Information

Lecture 16 – Distributed and Collaborative Visualization

Ken Brodliekwb@comp.leeds.ac.uk

ENV 2006 16.2

Outline of Lecture

• From Visualization to Computational Steering

• Distributed visualization– Extending dataflow across the network– Grid-based visualization and computational steering

• Collaborative visualization– Sharing the display screen– Sharing the visualization

ENV 2006 16.3

Dataflow Visualization Systems

• Visualization represented as pipeline:

– Read in data– Construct a visualization in

terms of geometry– Render geometry as image

• Realised as modular visualization environment

– IRIS Explorer is one example– Visual programming paradigm– Extensible – add your own

modules

http://www.nag.co.uk

data visualize render

ENV 2006 16.4

• IRIS Explorer is one of a family of similar visualization systems

• First product was AVS– Still major player but no longer visual programming– http://www.avs.com

• Amira, IBM Open Visualization Data Explorer (DX), IRIS Explorer– visual programming based : plug, play, throw away– application decomposed as set of modules, configured at run-time (blur

between building and running an application)– open : user can write modules– low-cost

Visualization Software Environments

ENV 2006 16.5

IBM Open Visualization Data Explorer – now OpenDX

• Released around 1991 by IBM

• Made open source in 1999– www.opendx.org

• A major use of it has been for weather visualization

http://www.research.ibm.com/weather/

ENV 2006 16.6

Amira

• More recent product

• Increasing use for medical applications..

• .. But also engineering including CFD

• Marketed by TGS

• www.tgs.com

ENV 2006 16.7

vtk - Visualization Toolkit

• vtk is a programming - based toolkit

• Open source C++ library

• www.kitware.com

ENV 2006 16.8

.. And there are many others

http://www.kdnuggets.com/software/visualization.html

ENV 2006 16.9

IRIS Explorer - Creating Your Own Modules

• It is possible to create your own modules • The mbuilder tool creates a wrapper around your own code• See:

http://www.nag.co.uk/visual/ie/iecbb/doc/html/unix-iemwg5-0.htm

ENV 2006 16.10

• IRIS Explorer can be driven either by GUI or by command line interface

• Commands can be grouped as a ‘script’ that IRIS Explorer runs– explorer -script <file>

• This allows Explorer to be run in batch mode, or to be driven by another application

• The scripting language is called Skm (pronounced as ‘scheme’)• Can be used interactively…• … in linux

– explorer -script %

• … in Windows, – use Skm editor (view menu)

Scripting - skm

ENV 2006 16.11

• To launch a module:(start “ReadImg”)

(start “DisplayImg”)

• To connect ports:(connect “ReadImg” “Output” “DisplayImg” “Input”)

• To start a map:(start-map “cfd”)

• See chapter 6 of User Manual on Web

http://www.nag.co.uk/visual/ie/iecbb/doc/html/unix-ug-chap06.htm

Creating a Simple Script

ENV 2006 16.12

Visualization and Simulation

• Visualization is a key tool in understanding the results of numerical simulations of complex physical phenomena

• Different modes of combining simulation and visualization:

– Post-processing

– Tracking

– Steering

ENV 2006 16.13

Linking Visualization and Simulation – Post Processing

• Post-processing

– Do the simulation and store results (step 1)

– Look at the results in a separate process (step 2)

– Revise the simulation (back to step 1)

simulation

data visualize render

Step 1

Step 2

PRO: study at your own pace

CON: must finish simulation first

ENV 2006 16.14

Linking Simulation and Visualization - Tracking

• Tracking

– Exploit extensibility of the dataflow visualization environment by including the simulation in the pipeline

– Track the behaviour of simulation as it runs

simulate visualize render

PRO: can abort fruitless simulations

ENV 2006 16.15

Linking Simulation and Visualization - Steering

• Computational steering:– By including a control module in the pipeline, we can direct the simulation in

response to the visualization

simulate visualize rendercontrol

PRO: not only can we track, we can alterthe actual course of the simulation

‘Human-in-the-loop’‘Human-in-the-loop’

ENV 2006 16.16

Computational Steering Environments

• Early visualization systems all have this extensibility feature and so can be used for steering

– IRIS Explorer for example

• New systems have emerged specifically to support steering

– SCIRun from Utah

Pressure profile for EHL contact

ENV 2006 16.17

Imagine this ….

• An explosion!

• A dangerous chemical escapes!

• Where is the fugitive pollutant headed?

• Who needs to be evacuated?

ENV 2006 16.18

Understanding What Will Happen

• Model the dispersion by solving system of PDEs

• Understand solution by visualization

• What if scenarios … need to be able to steer the simulation

• For example, what if the wind changes direction?

ENV 2006 16.19

Tracking the Pollution

ENV 2006 16.20

What can be Steered?

• Steering requires the writer of the simulation code to expose parameters that can be legitimately modified in the course of a run

– frequency of output of results– values of external influences that may vary over a simulation

• Not all parameters can be changed– time step used by numerical codes to achieve stability and/or accuracy

• Notion of backtracking is important in some simulations– Often you first observe, then wish to rewind a few timesteps, then replay with

different parameter settings

ENV 2006 16.21

Our Scenario

• We shall use this scenario to illustrate:

– Distributed visualization : we need to understand where the pollutant is headed in faster than real-time … therefore we need to run the simulation on a powerful compute resource

– Collaborative visualization : there is no time to collocate the scientist, the meteorologist, the politician or whoever needs to be involved … so we need to link people in over the network to allow them to visualize collaboratively

… while still using IRIS Explorer!

ENV 2006 16.22

Harnessing Remote Compute Resources – Grid Computing

Explorer on single host

Explorer on multiple hosts

Select remote host

Automatic authentication using: •Globus certificate

•SSH Key pair

ENV 2006 16.23

Simulation Runs Remotely

ENV 2006 16.24

A Tale

ENV 2006 16.25

The Monkey Gets the Nuts – Two Heads ARE Better than One

Thanks to Accra Academy, Ghana

ENV 2006 16.26

Collaborative Working

• Radical collocation has proved highly successful in a number of areas

– Space missions

– Safety critical software development

• Productivity doubled– Teasley et al, Univ of Michigan

• But this requires:– Social disruption

– Advance planning

– … and can end in tears

• Can we gain at least part of this success using electronic collaboration?

ENV 2006 16.27

Visualizing Collaboratively

• We need to move away from seeing collaborative visualization as a group around a display screen..

• .. Towards collaboration over a network

Collaborative visualization

ENV 2006 16.28

Collaborating in the Pollution Demonstrator

• Who needs to collaborate and in what way?

• Scientists and numerical modellers– Discuss amongst each other possible scenarios– Discuss need to pull in further Grid resources perhaps

• Meteorologist– Will play an active part in controlling the simulation

• Environmental agency decision makers– Need to analyse ‘what-if’ scenarios and construct presentations for

politicians

• Politicians, local authorities– Want to see clear presentation of consequences– Probably not interested in steering

ENV 2006 16.29

Sharing the Display Screen

• A very simple model is to broadcast the display screen of an application to a set of (passive) users

– Operating system level

– Screen image is broadcast using intelligent compression

– Only active user can enter input

data visualize render

internet

User A executes application

User B receives copy of user A desktop- does not execute application

data visualize render

ENV 2006 16.30

Sharing the Display Screen

• There are a number of available technologies for screen sharing

• VNC – Virtual Network Computing– Family of open source products evolved from original VNC development

by AT&T– RealVNC : www.realvnc.com (original development team)– tightVNC : www.tightvnc.com (new compression algorithms)– Heterogeneous

• Microsoft NetMeeting (and now MSN Messenger)

ENV 2006 16.31

Sharing Display Screen

• Advantages– Very simple concept – works for any application– Good for training– Good for presentation to a group

• Disadvantages– No independent working– Performance issues when rapid screen changes

• Variations– (1) Only one master – only one can control by mouse and keyboard input– (2) Any participant can input

ENV 2006 16.32

Sharing the Visualization

• This is a more flexible approach – and specific to dataflow visualization

• Each collaborator is an active participant in the visualization process

• Multiple, interlinked applications, where each collaborator runs their own application but data and parameter settings are programmed to be shared between the different applications

ENV 2006 16.33

internet

data visualize render

Sharing the Visualization

• Extends the dataflow model to interlink pipelines across the Internet

• Collaborative server provides the link

• So one user – for example - can send geometry to another person for viewing

collaborative server

share

share

render

ENV 2006 16.34

Programming the Collaboration

• It is useful to be able to program the collaboration

– To adapt to how people want to collaborate

– To adapt to network bandwidths

• Here raw data is exchanged so a different visualization can be created

internet

collaborative server

data visualize render

share

share

visualise render

ENV 2006 16.35

COVISA in action

sharing isosurface level

sharing data

Collaborator A Collaborator B

ENV 2006 16.36

Multiple, Interlinked Applications

• COVISA part of IRIS Explorer

• Advantages– Great flexibility– Independent working

• Disadvantages– Difficult to understand what the other user is doing

ENV 2006 16.37

Bring in the Meteorologist Remotely

Scientistin lab

Initiatecollaborativesession

Link inmeteorologistremotely

ENV 2006 16.38

Conclusions

• We have studied many aspects of scientific visualization:

– Applications and history– Different techniques for scalar

and vector data– Distributed and collaborative

visualization

• The practical work is giving experience in

– Exploratory visualization (what is going on?)

– Presentational visualization (here’s what is going on!)

• Finally, this afternoon, two case studies

– Exploration using parallel coordinates

– Focus and context for volume visualization