e-viz towards an adaptive framework for visualization on the grid
TRANSCRIPT
![Page 1: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/1.jpg)
e-Viz
• Towards an Adaptive Framework for Visualization on the Grid
![Page 2: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/2.jpg)
http://www.eviz.org/
e-Vizzers
UW Swansea
UW Bangor
U Manchester
U Leeds
• e-Viz is a three year joint research project funded by UK EPSRC
• Four partner universities:– University of Leeds– (Ken Brodlie, Jason Wood)
– University of Manchester – (John Brooke, Mark Riding)
– University of Wales, Bangor (Nigel John, Chris Hughes)
– University of Wales, Swansea– (Min Chen, David Chisnall, Mark
Jones, Nicolas Roard)
![Page 3: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/3.jpg)
http://www.eviz.org/
What problem are we trying to solve?
DataChoose
Visualization Algorithm
ChooseSoftware toImplementAlgorithm
ChooseHardwareto run on
Which areapplicable to
the data?
Which isthe mostsuitable?
What’sfastest?
Availability?
Domain Scientist Visualization Specialist e-Scientist
![Page 4: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/4.jpg)
http://www.eviz.org/
Why are we trying to solve this?
• Because not all potential Grid visualization users are experts in visualization and Grid technologies.
![Page 5: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/5.jpg)
http://www.eviz.org/
Why are we trying to solve this?
Visualization Experience
Grid Experience
Application Scientists
Visualization Specialists
Grid & Visualization Developers
e-Scientists
![Page 6: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/6.jpg)
http://www.eviz.org/
Putting Visualization on the Grid
• Can the Grid help? – it’s supposed to be:– heterogeneous in architecture– seamless and transparent in use– fault tolerant in operation – capable of adapting to the changing environment in
order to provide the best service.
![Page 7: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/7.jpg)
http://www.eviz.org/
What can the Grid offer?
• Heterogeneity: architectures
SMP
NUMA
Desktop
Specialised
Computation Rendering
Cluster
![Page 8: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/8.jpg)
http://www.eviz.org/
What can the Grid offer?
• Heterogeneity: operating systems
Linux Irix AIX
Solaris BSD HP/UX
Windows MacOS
![Page 9: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/9.jpg)
http://www.eviz.org/
What can the Grid offer?
• Heterogeneity: visualization software
AVS IRIS Explorer OpenDX
VTK rtrt Simian
Volumizer Matlab
![Page 10: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/10.jpg)
http://www.eviz.org/
What can the Grid offer?
• Heterogeneity: Implications
– There are a wide variety of architectures, operating systems and software applications used in visualization.
– Each has it’s own particular rewards and benefits, but we can’t expect every Grid user to be knowledgeable and proficient with each, or even to know which is the most appropriate for a given task.
– Want to be able to offer the power afforded by each application to all potential users of visualization.
![Page 11: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/11.jpg)
http://www.eviz.org/
What can the Grid offer?
• Seamlessness and transparency: Implications
– End users should only have to learn one interface, but still be able to benefit from the features offered by a wide range of applications.
– But each visualization application has it’s own user interface…
– … and every operating system looks and behaves differently
![Page 12: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/12.jpg)
http://www.eviz.org/
What can the Grid offer?
• Fault tolerance: Implications
– “A distributed system is one in which the failure of a computer you didn’t even know existed can render your own computer unusable” – Leslie Lamport
– “A Grid system is one in which the failure of a computer you didn’t even know existed goes by unnoticed”
– Users should be able to rely on the Grid
![Page 13: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/13.jpg)
http://www.eviz.org/
What can the Grid offer?
• Fault Tolerance - User Scenario… keyboard,
please
![Page 14: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/14.jpg)
http://www.eviz.org/
What can the Grid offer?
• Adaptation: Implications
– The world is dynamic and ever changing; so is the Grid.– Network loads and status– Queues on HPC machines– Runtime limits on HPC machines– CPU loading, both remotely and locally– A Grid system should adapt to cope with changes
![Page 15: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/15.jpg)
http://www.eviz.org/
How can we solve this problem?
![Page 16: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/16.jpg)
http://www.eviz.org/
What is Visualization?
VisualizationData Image
![Page 17: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/17.jpg)
http://www.eviz.org/
What is Visualization?
Filter Map RenderData Image
![Page 18: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/18.jpg)
http://www.eviz.org/
What is Visualization?
Filter Map RenderData Image
Controls
![Page 19: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/19.jpg)
http://www.eviz.org/
What is Visualization?
VisualizationData Image
Controls
![Page 20: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/20.jpg)
http://www.eviz.org/
Common Abstract Interface
VisualizationData Image
Controls
![Page 21: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/21.jpg)
http://www.eviz.org/
Common Abstract Interface
VisualizationData Image
Controls
ComputationalSteering
FrameTransport
FormatConversion
![Page 22: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/22.jpg)
http://www.eviz.org/
Common Abstract Interface
VTKData Image
Controls
ComputationalSteering
FrameTransport
FormatConversion AVS/ExpressIRIS Explorer
![Page 23: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/23.jpg)
http://www.eviz.org/
Common Abstract Interface
Data Image
Controls
ComputationalSteering
FrameTransport
FormatConversion VTK rtrtAVS Simian
![Page 24: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/24.jpg)
http://www.eviz.org/
Common Abstract Interface
• Feasibility– can we create equivalent visualizations using different
software packages?
![Page 25: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/25.jpg)
http://www.eviz.org/
Equivalent Visualizations?
![Page 26: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/26.jpg)
http://www.eviz.org/
Equivalent Visualizations?
![Page 27: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/27.jpg)
http://www.eviz.org/
Abstraction
• Need to describe the pipeline itself – need an abstract visualization description language– gViz project gives us skML
Visualization
ComputationalSteering
FrameTransport
FormatConversion
![Page 28: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/28.jpg)
http://www.eviz.org/
Abstraction
• Format Conversion– Most visualization software applications can already
read a wide range of data formats
Visualization
ComputationalSteering
FrameTransport
FormatConversion
![Page 29: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/29.jpg)
http://www.eviz.org/
Abstraction
• Computational Steering
Visualization
ComputationalSteering
FrameTransport
FormatConversion
![Page 30: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/30.jpg)
http://www.eviz.org/
Abstraction
• Computational Steering– RealityGrid or gViz APIs can be used to control a
running pipeline– Visualization applications must be instrumented to
expose their steerable pipeline parameters– Currently instrumented VTK and RTRT (real time ray
tracer)
– GUI created, which is really a specialised Computational Steering client
- Reads in a pipeline description and dynamically configures itself to show appropriate widgets
![Page 31: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/31.jpg)
http://www.eviz.org/
Abstraction
• Frame Transport
Visualization
ComputationalSteering
FrameTransport
FormatConversion
![Page 32: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/32.jpg)
http://www.eviz.org/
Abstraction
• Frame Transport– Images can be rendered locally or remotely – in either
case, they need to be displayed on the user’s own machine, as well as any collaborators
– Have created a library to compress remote images and transport them back to the client for display. Visualization applications can be modified to use the library
– Range of image compression codecs– Local rendering supported via a rendering overlay– Linux and Windows clients
![Page 33: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/33.jpg)
http://www.eviz.org/
What do we still need?
• Decision making– What visualizations are applicable for a given input data
type– Which of the available hardware and software is most
suitable to implement such a pipeline– User may not know the answer to these questions –
provide assistance
![Page 34: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/34.jpg)
http://www.eviz.org/
Architecture
Software Store
Client Broker
Data Store
HPV Machine HPV Machine HPV Machine HPV MachineHPV Machine
WSRF
Globus
gViz
e-Viz
![Page 35: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/35.jpg)
http://www.eviz.org/
Architecture (Current)
Client Broker
HPV Machine HPV Machine HPV Machine HPV MachineHPV Machine
Software StoreData Store
WSRF
Globus
gViz
e-Viz
![Page 36: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/36.jpg)
http://www.eviz.org/
Demonstration 1
• gViz Pollution Demonstrator - shows– Active simulations as a data source– Computational steering of an active visualization– Common interface to servers
![Page 37: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/37.jpg)
http://www.eviz.org/
Demonstration 1
![Page 38: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/38.jpg)
http://www.eviz.org/
Demonstration 2
• Volume Render Demonstrator – shows– Heterogeneous access to multiple servers– (Almost) seamless switching between servers– Fault tolerance through redundancy
![Page 39: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/39.jpg)
http://www.eviz.org/
Demonstration 2
![Page 40: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/40.jpg)
http://www.eviz.org/
The Knowledge Gap
• The Broker has a knowledgebase which it will use to make informed decisions on pipeline choices.
• It’s empty!
![Page 41: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/41.jpg)
http://www.eviz.org/
How to Distribute?
Filter Map RenderData Image
![Page 42: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/42.jpg)
http://www.eviz.org/
How to Distribute?
Filter Map RenderData Image
![Page 43: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/43.jpg)
http://www.eviz.org/
How to Distribute?
Filter Map Render
Data Image
![Page 44: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/44.jpg)
http://www.eviz.org/
How to Distribute?
Filter Map Render
Data Image
![Page 45: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/45.jpg)
http://www.eviz.org/
How to Distribute?
Filter Map Render
Data Image
![Page 46: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/46.jpg)
http://www.eviz.org/
When to Distribute?
• Depends on– Where the dataset resides– How big the dataset is– Network bandwidth– Available machines– Chosen algorithm– Queue status– CPU capabilities– Graphical capabilities– …..
![Page 47: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/47.jpg)
http://www.eviz.org/
Other problems
• Single point of failure in the broker
![Page 48: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/48.jpg)
http://www.eviz.org/
Conclusions
• e-Viz has created a framework for visualization on the Grid that is:– Heterogeneous in architecture– Seamless and transparent in use– Fault tolerant– Adaptive (potentially)
• But much more to be done! (project runs another 16 months)
![Page 49: E-Viz Towards an Adaptive Framework for Visualization on the Grid](https://reader034.vdocument.in/reader034/viewer/2022052522/5515ece4550346dd6f8b5231/html5/thumbnails/49.jpg)
http://www.eviz.org/
For more information
• See a live demo:– Tuesday 22nd, 14:00-16:30, White Rose Grid stand
• Project Web Site:– http://www.eviz.org
• email:– [email protected]