ahm04 1 gviz: visualization and computational steering on the grid ken brodlie, jason wood –...
TRANSCRIPT
![Page 1: AHM04 1 gViz: Visualization and Computational Steering on the Grid Ken Brodlie, Jason Wood – University of Leeds David Duce, Musbah Sagar – Oxford Brookes](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f295503460f94c42be1/html5/thumbnails/1.jpg)
1AHM04
gViz: Visualization and Computational Steering on the Grid
Ken Brodlie, Jason Wood – University of Leeds
David Duce, Musbah Sagar – Oxford Brookes University
![Page 2: AHM04 1 gViz: Visualization and Computational Steering on the Grid Ken Brodlie, Jason Wood – University of Leeds David Duce, Musbah Sagar – Oxford Brookes](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f295503460f94c42be1/html5/thumbnails/2.jpg)
2AHM04
gViz – Visualization Middleware for e-Science
• gViz is an e-Science Core Programme project – just finished…
• … has made a start at understanding:
– How to evolve existing visualization systems to the Grid
– How to link visualization and simulation environments
• gViz partners:– Academic: Leeds, Oxford,
Oxford Brookes, CLRC/RAL– Industrial: NAG, IBM UK and
Streamline Computing– International: Caltech, MIT
• Leeds contribution through the White Rose Grid e-Science Centre of Excellence
e-Science Centre of Excellence
![Page 3: AHM04 1 gViz: Visualization and Computational Steering on the Grid Ken Brodlie, Jason Wood – University of Leeds David Duce, Musbah Sagar – Oxford Brookes](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f295503460f94c42be1/html5/thumbnails/3.jpg)
3AHM04
Starting Point: Dataflow Visualization Systems
• Visualization represented as pipeline:
– Read in data– Construct a visualization in
terms of geometry– Render geometry as image
• Realised as modular visualization environment
– IRIS Explorer is one example– Visual programming paradigm– Extensible – add your own
modules– Others include IBM Open
Visualization Data Explorer
data visualize render
… pipeline design done after committingto particular system… modules assumed to execute locally
BUT
![Page 4: AHM04 1 gViz: Visualization and Computational Steering on the Grid Ken Brodlie, Jason Wood – University of Leeds David Duce, Musbah Sagar – Oxford Brookes](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f295503460f94c42be1/html5/thumbnails/4.jpg)
4AHM04
Extending the Reference Model to Grid Environments
• Revisit the visualization pipeline– Start with the traditional
reference model– Progressively bind in software
and hardware resources– Three-layer reference model
• Conceptual: intent of the visualization
– Show me isosurface of constant temperature
• Logical: bind in the software system
– Use IRIS Explorer (or vtk, or whatever)
• Physical: bind in the resources to be used
– Run the isosurface extraction on particular Grid resource
data visualize render
![Page 5: AHM04 1 gViz: Visualization and Computational Steering on the Grid Ken Brodlie, Jason Wood – University of Leeds David Duce, Musbah Sagar – Oxford Brookes](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f295503460f94c42be1/html5/thumbnails/5.jpg)
5AHM04
Developing an XML Language for Conceptual Layer: skML
• First – the conceptual layer
• Dataflow consists fundamentally of:– a map – containing links – between ports – on modules– which have parameters
• This leads us to a simple XML application for visualization: called skML
• Here a data reader is linked to an isosurfacer
<?xml version="1.0"?><skml><map><link> <module name="ReadLat” out-port="Output"> <param name="Filename"> testVol.lat </param> </module> <module id=“iso”
name="IsosurfaceLat" in-port="Input"> <param name="Threshold" min="0" max="27"> 1.8</param> </module></link>…
![Page 6: AHM04 1 gViz: Visualization and Computational Steering on the Grid Ken Brodlie, Jason Wood – University of Leeds David Duce, Musbah Sagar – Oxford Brookes](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f295503460f94c42be1/html5/thumbnails/6.jpg)
6AHM04
Diagrammatic Representation using SVG
• skML gives us an XML application for visualization at the conceptual layer
• In addition to language representation, a diagrammatic representation has been created in SVG – so we can do dataflow programming in a web browser
• Transforming to the logical layer binds in the software resource
– A new IRIS Explorer module can read skML and generate corresponding map
– skML can also be turned into an IBM Open Visualization Data Explorer network
<?xml version="1.0"?><skml><map><link> <module name="ReadLat” out-port="Output"> <param name="Filename"> testVol.lat </param> </module> <module id=“iso”
name="IsosurfaceLat" in-port="Input"> <param name="Threshold" min="0" max="27"> 1.8</param> </module></link>…
![Page 7: AHM04 1 gViz: Visualization and Computational Steering on the Grid Ken Brodlie, Jason Wood – University of Leeds David Duce, Musbah Sagar – Oxford Brookes](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f295503460f94c42be1/html5/thumbnails/7.jpg)
7AHM04
Physical Layer – Secure Distributed IRIS Explorer
IRIS Explorer on multiple hosts
Select remote host
Automatic authentication using: •Globus certificate
•SSH Key pair
• Moving to the physical layer, we need to be able to execute modules on remote Grid resources
• IRIS Explorer has been extended to allow a user to place modules on specific compute resources – dataflow pipeline thus spans the Grid
• Compute-intensive modules can be placed remotely - design the dataflow for the Grid
![Page 8: AHM04 1 gViz: Visualization and Computational Steering on the Grid Ken Brodlie, Jason Wood – University of Leeds David Duce, Musbah Sagar – Oxford Brookes](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f295503460f94c42be1/html5/thumbnails/8.jpg)
8AHM04
Next Steps
• Some tangible benefits…
• … Next release of IRIS Explorer will include the distributed execution facility…
• … but much remains to be done
• Conceptual level– Visualization ontology needed
to define and organize set of canonical processes
– Useful to include resource constraints (initial steps made with RDF)
• Logical level– Visualization data exchange
between systems needs to be studied
– Initial steps made by Julian Gallop (this conference)
• Physical level– User allocation of modules to
resources needs to be replaced by a brokering service
![Page 9: AHM04 1 gViz: Visualization and Computational Steering on the Grid Ken Brodlie, Jason Wood – University of Leeds David Duce, Musbah Sagar – Oxford Brookes](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f295503460f94c42be1/html5/thumbnails/9.jpg)
9AHM04
Computational Steering
• Computational steering requires a link between a visualization environment and a simulation environment…
• … gViz library provides this glue
• Design aims:– Use with different simulation
environments and different visualization environments
– Allow connect and disconnect– Lack of intrusion and minimize
performance loss– Robustly handle different producer-
consumer rates– Support multiple simulations– Support collaboration– Support historical audit trail
control visualize
visualization environment
simulation environment
gViz library
gViz library
![Page 10: AHM04 1 gViz: Visualization and Computational Steering on the Grid Ken Brodlie, Jason Wood – University of Leeds David Duce, Musbah Sagar – Oxford Brookes](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f295503460f94c42be1/html5/thumbnails/10.jpg)
10AHM04
Environmental Application
• Demonstrator created for an environmental crisis scenario
– Dangerous chemical escapes!– Model dispersion using system
of PDEs and solve numerically over mesh
– Visualize mesh elements where concentration exceeds threshold
– What happens when the wind changes?
– ‘faster-than-real-time’
• Simulation environment– Finite volume code written in C
![Page 11: AHM04 1 gViz: Visualization and Computational Steering on the Grid Ken Brodlie, Jason Wood – University of Leeds David Duce, Musbah Sagar – Oxford Brookes](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f295503460f94c42be1/html5/thumbnails/11.jpg)
11AHM04
Pollution Simulation Using the gViz Library and IRIS Explorer
DiscoverGrid resources
Launchsimulation
Connectto simulation
Send controlparameters
Get datato visualize
Visualize
![Page 12: AHM04 1 gViz: Visualization and Computational Steering on the Grid Ken Brodlie, Jason Wood – University of Leeds David Duce, Musbah Sagar – Oxford Brookes](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f295503460f94c42be1/html5/thumbnails/12.jpg)
12AHM04
IRIS Explorer as Visualization Environment
• Distributed module execution:– Allows visualization modules to
be collocated with simulation to minimize data traffic to desktop
• Collaborative visualization:– Allows the COVISA multi-user
visualization facility to be exploited
![Page 13: AHM04 1 gViz: Visualization and Computational Steering on the Grid Ken Brodlie, Jason Wood – University of Leeds David Duce, Musbah Sagar – Oxford Brookes](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f295503460f94c42be1/html5/thumbnails/13.jpg)
13AHM04
Pollution example with other visualization environments
• Different visualization environments can be connected through gViz library to the underlying simulation
• Note that multiple users – with multiple visualization environments – can connect… allowing collaboration amongst a team
SCIRun
Matlab
vtk
![Page 14: AHM04 1 gViz: Visualization and Computational Steering on the Grid Ken Brodlie, Jason Wood – University of Leeds David Duce, Musbah Sagar – Oxford Brookes](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f295503460f94c42be1/html5/thumbnails/14.jpg)
14AHM04
Computational Biology
• In another application the gViz library provides monitoring and control of heart modelling experiments – Arun Holden & Richard Clayton
• Multiple simulations of electrical activity of the heart
![Page 15: AHM04 1 gViz: Visualization and Computational Steering on the Grid Ken Brodlie, Jason Wood – University of Leeds David Duce, Musbah Sagar – Oxford Brookes](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f295503460f94c42be1/html5/thumbnails/15.jpg)
15AHM04
gViz Anatomy
DiscoverGrid resources
Launch simulation(register with Directory Service)
Call up Directory Serviceand select simulation
Visualize Visualizemultiplesimulations
Getresults
![Page 16: AHM04 1 gViz: Visualization and Computational Steering on the Grid Ken Brodlie, Jason Wood – University of Leeds David Duce, Musbah Sagar – Oxford Brookes](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f295503460f94c42be1/html5/thumbnails/16.jpg)
16AHM04
…Or with Matlab as Visualization Environment
![Page 17: AHM04 1 gViz: Visualization and Computational Steering on the Grid Ken Brodlie, Jason Wood – University of Leeds David Duce, Musbah Sagar – Oxford Brookes](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f295503460f94c42be1/html5/thumbnails/17.jpg)
17AHM04
… Or with Grid/Web Services approach
• Grid service interface to gViz library
• Heart Modelling Grid Service uses:
– Web interface where user specifies user name and passphrase, and location of gViz directory service
– Grid service connects to simulations to allow steering parameters to be sent, and results to be retrieved, via the gViz library
– A second grid service builds images from simulation data
• Returned as a Web page
![Page 18: AHM04 1 gViz: Visualization and Computational Steering on the Grid Ken Brodlie, Jason Wood – University of Leeds David Duce, Musbah Sagar – Oxford Brookes](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f295503460f94c42be1/html5/thumbnails/18.jpg)
18AHM04
gViz meets Integrative Biology
• The application to heart modelling continues in the Integrative Biology project with David Gavaghan
• Here Matlab is the simulation environment …
• .. linked by gViz library to IRIS Explorer as the visualization environment…
• … or indeed Matlab can act as the visualization environment
• Reality Grid steering also being used in IB project, so hope is to gain convergence between the two approaches
![Page 19: AHM04 1 gViz: Visualization and Computational Steering on the Grid Ken Brodlie, Jason Wood – University of Leeds David Duce, Musbah Sagar – Oxford Brookes](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f295503460f94c42be1/html5/thumbnails/19.jpg)
19AHM04
Conclusions
• The gViz project has begun to explore the issues in evolving visualization systems to Grid environments
• Tangible benefits:– Secure distributed IRIS Explorer in next release from NAG– gViz library code will be made available as open source (LGPL)
• Raising issues:– Ontology– Visualization data exchange– Visualization brokering service
• Continuing development of gViz library within Integrative Biology – with potential convergence with RealityGrid steering library
• Demonstration: WRG Stand, Friday 10.30 – 14.30
![Page 20: AHM04 1 gViz: Visualization and Computational Steering on the Grid Ken Brodlie, Jason Wood – University of Leeds David Duce, Musbah Sagar – Oxford Brookes](https://reader036.vdocument.in/reader036/viewer/2022081603/56649f295503460f94c42be1/html5/thumbnails/20.jpg)
20AHM04
Acknowledgements
The gViz project team has involved many people:
• Leeds University: Ken Brodlie, Jason Wood, Chris Goodyer, Martin Thompson, Mark Walkley, Haoxiang Wang, Ying Li, James Handley, Arun Holden, Richard Clayton (now Sheffield)
• Oxford Brookes University: David Duce, Musbah Sagar• Oxford University: Mike Giles, David Gavaghan• CLRC/RAL: Julian Gallop• NAG: Steve Hague, Jeremy Walton• Streamline Computing: Mike Rudgyard • IBM UK: Brian Collins, Alan Knox, John Illingworth• CACR, Caltech: Jim Pool, Santiago de Lombeyda, John McCorquadale• MIT: Bob Haimes
Development environment at Leeds: White Rose Grid – e-Science Centre of Excellence