web services for visualization
DESCRIPTION
Web Services for Visualization. Dr. Gordon Erlebacher School Comp. Sci. Info. Tech. Florida State University. Contents. Objectives Languages Streaming video Video creation Remote control of visualization package Interactive Web Maps Conclusions. Issues. - PowerPoint PPT PresentationTRANSCRIPT
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UPervasive Knowledge Center, Indiana Universityniversity
11
Web Services for Web Services for VisualizationVisualization
Dr. Gordon ErlebacherDr. Gordon Erlebacher
School Comp. Sci. Info. Tech.School Comp. Sci. Info. Tech.
Florida State UniversityFlorida State University
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 22
ContentsContents
ObjectivesObjectives
LanguagesLanguages
Streaming videoStreaming video
Video creationVideo creation
Remote control of visualization packageRemote control of visualization package
Interactive Web MapsInteractive Web Maps
ConclusionsConclusions
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 33
IssuesIssuesScientific visualization and information Scientific visualization and information visualization are natural interfaces to visualization are natural interfaces to complex and large datasetscomplex and large datasetsThe range and size of display devices is The range and size of display devices is increasing: PDAs, Powerwalls, E-paper, theincreasing: PDAs, Powerwalls, E-paper, the retina)retina)The proliferation of interaction devices make The proliferation of interaction devices make it difficult to reliably and consistently interact it difficult to reliably and consistently interact with the information displayedwith the information displayedThe interaction devices are often expensive The interaction devices are often expensive and non-standardand non-standard
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 44
(Picture from www.amiravis.com)
•Each user should be able to query the image differently
•Augment image with information from WEB
•Store GUI on PDA
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 55
Visualization / CollaborationVisualization / Collaboration
2D/3D navigation with multiple users2D/3D navigation with multiple users
Synchronization issuesSynchronization issues
InteractivityInteractivity
Information exchangeInformation exchange
Data analysis / mining Data analysis / mining
Feature extractionFeature extraction
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 66
Rear projection16’x8’Stereo2 projectors, blended image
Powerwall at FSUPowerwall at FSU
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 77
Partial Wish ListPartial Wish List
User Interaction should be natural, easy to useUser Interaction should be natural, easy to useGUIs should combine voice and graphicsGUIs should combine voice and graphicsGUIs should minimize unnecessary informationGUIs should minimize unnecessary informationUser interfaces should be configurable by usersUser interfaces should be configurable by usersUser interface should be portable and User interface should be portable and transportabletransportableUsers should move Users should move freelyfreely with respect to the with respect to the display and themselvesdisplay and themselves
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 88
Multiple-PDA DeviceMultiple-PDA Device
•Slice through data
•Query the Internet
•Fuse data
•User-user exchange
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 99
CPU power of PDA CPU power of PDA infinity infinity Memory of PDA Memory of PDA infinity infinityWireless bandwidth Wireless bandwidth infinity infinityWireless bandwidth 1/10Wireless bandwidth 1/10thth LAN LANDisplay resolution of PDA Display resolution of PDA infinity infinitySize of PDA screen remains limitedSize of PDA screen remains limited
Trends and AssumptionsTrends and Assumptions2000-20052000-2005
≤
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 1010
ApproachApproach
Conform to standards (Corba, Java, python, XML)Conform to standards (Corba, Java, python, XML)Allow for extensionAllow for extensionMaximize generalityMaximize generalityAllow for multiple protocols, operating systems, Allow for multiple protocols, operating systems, languageslanguagesDatasource:Datasource:– SupercomputerSupercomputer– File serversFile servers– Data feeds (i.e., sattelites)Data feeds (i.e., sattelites)
802.11b standard (2001), 802.11a/g (2002-?)802.11b standard (2001), 802.11a/g (2002-?)Currently, wireless network only serves to get the Currently, wireless network only serves to get the information from and to the PDAinformation from and to the PDA
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 1111
Proposed ArchitectureProposed Architecture
PDA PDARemote PDA
Server Server Server
Application Application Application
PC
Local Remote
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 1212
Heat convection in the Planetary Mantle
Complex FlowsMining/Extraction/Analysis
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 1313
Multiple ObjectivesMultiple Objectives
Integrate visualization with Web ServicesIntegrate visualization with Web ServicesBuild tools to enhance collaboration Build tools to enhance collaboration through visualizationthrough visualizationFacilitate feature extraction and Facilitate feature extraction and information sharing between usersinformation sharing between usersDevelop tools to query datasetsDevelop tools to query datasetsUse of system should be transparent to Use of system should be transparent to users (independent of location or users (independent of location or hardware)hardware)
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 1414
Approach(es)Approach(es)
Work with standardsWork with standards– Should function on Windows/Linux/UnixShould function on Windows/Linux/Unix
Develop for ease of maintenanceDevelop for ease of maintenance– Students only stay for short periodStudents only stay for short period
Insist on web documentation of everythingInsist on web documentation of everything– Only partially successfulOnly partially successful– Success is a cascading effectSuccess is a cascading effect
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 1515
Languages: C++Languages: C++
Fast ExecutionFast ExecutionCompiledCompiledWell-supportedWell-supportedReasonably portableReasonably portableStandard Template Library (not yet fully Standard Template Library (not yet fully standardized)standardized)Operator overloading (very useful)Operator overloading (very useful)Object-orientedObject-orientedStrongly typedStrongly typedSupported on all platformsSupported on all platforms
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 1616
Languages: PythonLanguages: Pythonhttp://www.python.orghttp://www.python.org
Fast Prototyping Fast Prototyping Essentially same capabilities as PerlEssentially same capabilities as PerlRather well supportedRather well supportedNice interface to Java (JPython)Nice interface to Java (JPython)Weakly typedWeakly typedObject-OrientedObject-OrientedExtremely flexible (sometimes dangerous)Extremely flexible (sometimes dangerous)Extremely easy to useExtremely easy to useExceptionally clear code (e.g., forces indentation)Exceptionally clear code (e.g., forces indentation)Support on all platforms (including PDA)Support on all platforms (including PDA)
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 1717
Languages: ACE/TAO Languages: ACE/TAO http://http://
www.cs.wustl.edu/~schmidt/ACE.htmlwww.cs.wustl.edu/~schmidt/ACE.html ACE: Adaptive Communication EnvironmentACE: Adaptive Communication EnvironmentTAO: The ACE ORB (CORBA Support)TAO: The ACE ORB (CORBA Support)Totally based on design patternsTotally based on design patternsWraps streaming, sockets, mutexes, etc. in high Wraps streaming, sockets, mutexes, etc. in high level classeslevel classesPortable across platforms and operating Portable across platforms and operating systems (we ported it to Linux on PDA, support systems (we ported it to Linux on PDA, support for WinCE)for WinCE)Problem: large memory footprint (4-5 Mbytes on Problem: large memory footprint (4-5 Mbytes on PDA)PDA)
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 1818
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 1919
Streaming VideoStreaming Videohttp://www.csit.fsu.edu/~dongchenhttp://www.csit.fsu.edu/~dongchen
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 2020
Video StreamingVideo Streamingwith waveletswith wavelets
VisualizationServer
Frame
Wavelettransform
Encode
VisualizationIpaq
Frame
Wavelettransform
Decode
Color animations at 4 frames/sec on Ipaq (320 x 200) and 802.11b wireless network
CORBA/SOAP GUIIpaq
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 2121
Flowchart of ACEFlowchart of ACE
Courtesy D. Schmidt
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 2222
Streaming VideoStreaming VideoActive Object:
• Had its own thread(s)
• Does not block calling method
Passive Object:
• Uses parent thread
• Blocks calling method
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 2323
Streaming VideoStreaming Video
EncoderCImageBuffers *ServerDgramGrabberConvertRgbToYUV ( )waveletTransform ()informSibling ( )
GrabberCImageBuffers *grabImage ( )informSibling ( )
ServerDGramCImageBuffers *informSibling ( )
CImageBuffersStreaming buffers (2)YUV/Wavelet buffers (2)RGB image buffers (2)
A
C
D
B
EncoderCImageBuffers *ServerDgramGrabberConvertRgbToYUV ( )waveletTransform ()informSibling ( )
GrabberCImageBuffers *grabImage ( )informSibling ( )
ServerDGramCImageBuffers *informSibling ( )
CImageBuffersStreaming buffers (2)YUV/Wavelet buffers (2)RGB image buffers (2)
A
CC
DD
B
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 2424
0
5
10
15
20
25
30
35
80 120 160 200 240
video size (pixels x pixels)
frames/second
teapot random noise constant color
0
2
4
6
8
10
12
80 160 240 320 400 480 560 640video size (pixels x pixels)
User data rate Mbps
teapot random noise constant color
0
0.3
0.6
0.9
1.2
1.5
80 120 160 200 240video size (pixels x pixels)
User data rate Mbps
teapot random noise constant color
0
10
20
30
40
80 160 240 320 400 480 560 640video size (pixels x pixels)
frames/second
teapot random noise constant color
PC PDAPC PC
10 fps 5 fps
random random
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 2525
Online Video CreationOnline Video Creation
Scientists use visualization tools to Scientists use visualization tools to analyze and navigate their large datasetsanalyze and navigate their large datasets
Videos are created for dissemination and Videos are created for dissemination and archivingarchiving– Sequence of frames is stored by viz programSequence of frames is stored by viz program– Frames are transformed into appropriate Frames are transformed into appropriate
video formatvideo format– Video format is converted to one or more Video format is converted to one or more
additional formats (for easy accessibility)additional formats (for easy accessibility)
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 2626
Online Video CreationOnline Video Creation
Generation of videos is usually a manual Generation of videos is usually a manual processprocess
Videos usually created on Windows or Videos usually created on Windows or expensive Unix systemsexpensive Unix systems
Visualizations usually created on LinuxVisualizations usually created on Linux
Conclusion: Conclusion: – Need technology to create videos directly on Need technology to create videos directly on
Linux systemsLinux systems
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 2727
Image files
Movie files
Input parameters
Choice of Codec
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 2828
Input parameters
Preset parameter combinations
Codec
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 2929
Online Video CreationOnline Video Creation
Created by J. F. BoisdetCreated by J. F. Boisdet
http://vector.csit.fsu.edu:8081/~boisdet/tehttp://vector.csit.fsu.edu:8081/~boisdet/temp/one.pymp/one.py
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 3030
AmiraAmirahttp://www.amiravis.comhttp://www.amiravis.com
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 3131
AmiraAmira
Flowcharts are created interactively by the userFlowcharts are created interactively by the user
Each component has an associated user Each component has an associated user interfaceinterface
Software capitalizes on graphic hardware (SGI, Software capitalizes on graphic hardware (SGI, Onyx, Nvidia, ATI) to achieve good performanceOnyx, Nvidia, ATI) to achieve good performance
Flowcharts, called networks, can be saved for Flowcharts, called networks, can be saved for later use.later use.
Developer version allows users to create their Developer version allows users to create their own modules for specialized visualization.own modules for specialized visualization.
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 3232
AmiraAmira
Amira is a commercial packageAmira is a commercial package
I don’t necessarily recommend this I don’t necessarily recommend this packagepackage
However,However,– It has nice features, perhaps useful to the It has nice features, perhaps useful to the
visualization of static and time-dependent fluid visualization of static and time-dependent fluid structuresstructures
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 3333
AmiraAmira
Read in 3D fileRead in 3D file
Generate several planar cross-sectionsGenerate several planar cross-sections
Generate an iso-surfaceGenerate an iso-surface
Generate a volumetric plotGenerate a volumetric plot
Combine techniquesCombine techniques
Demonstrate data querying (line cut, Demonstrate data querying (line cut, pointwise, etc.)pointwise, etc.)
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 3434
Amira FeaturesAmira Features
Very InteractiveVery InteractiveManipulatorsManipulators– Interact with the dataInteract with the data
ExtensibleExtensible– Users can write own extension modulesUsers can write own extension modules– API is very sophisticatedAPI is very sophisticated
Highly advanced algorithms to do Highly advanced algorithms to do – Isosurface, volume rendering, vector Isosurface, volume rendering, vector
visualizationvisualization– Combinations of the aboveCombinations of the above
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 3535
Heat Convection between Two Heat Convection between Two Plates (Amira)Plates (Amira)
Data, courtesy David Yeun
643
subsampling
2573 dataset
Heat flow between two plates at constant temperature
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 3636
Remote Control AmiraRemote Control Amira(Yunsong Wang)(Yunsong Wang)
http://vector.csit.fsu.edu:8081/~yunsong/chttp://vector.csit.fsu.edu:8081/~yunsong/cgi-binbac/remote_amira.pygi-binbac/remote_amira.py
Creation or loading of Amira scriptsCreation or loading of Amira scripts
Automatic initiation of AmiraAutomatic initiation of Amira
Retrieve bitmap from serverRetrieve bitmap from server
Working on retrieving x,y,z coordinatesWorking on retrieving x,y,z coordinates
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 3737
Amira Tcl Scripts
Amira Output
Amira Server
Script Creation
Amira Display
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 3838
Remote Control AmiraRemote Control Amira(Yunsong Wang)(Yunsong Wang)
http://vector.csit.fsu.edu:8081/~yunsong/chttp://vector.csit.fsu.edu:8081/~yunsong/cgi-binbac/remote_amira.pygi-binbac/remote_amira.py
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 3939
Interactive Web MapsInteractive Web Mapshttp://www.csit.fsu.edu/~garbowza/WDI/http://www.csit.fsu.edu/~garbowza/WDI/
Built by Zachary Garbow Built by Zachary Garbow – (Minnesota Supercomputer Institute)(Minnesota Supercomputer Institute)
Store large datasets on a serverStore large datasets on a server
Clients operate on the datasetClients operate on the dataset– Zooming, histograms, mean/avg/stddevZooming, histograms, mean/avg/stddev
C++ on the ServerC++ on the Server
Java Applet on the ClientJava Applet on the Client
Challenges: balance between computation on Challenges: balance between computation on client and server, and networking considerationsclient and server, and networking considerations
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 4040
Interactive Web MapsInteractive Web Maps
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 4141
Temperature field: 2D grid: 3400x500
Ra = 3×107
Ra = 3×108
Ra = 109
Ra = 1010
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 4242
Two-way flow of information!!
Map of data
Histogram
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 4343
Interactive Web MapsInteractive Web Mapshttp://www.csit.fsu.edu/~garbowza/WDI/http://www.csit.fsu.edu/~garbowza/WDI/
Built by Zachary Garbow (Minnesota Built by Zachary Garbow (Minnesota Supercomputer Institute, works with D. Supercomputer Institute, works with D. Yuen)Yuen)
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 4444
Visualization UbiquityVisualization Ubiquity
Collaboration through visualizationCollaboration through visualization
Office walls become visualization displays Office walls become visualization displays (E-Ink: thin, pliable medium capable of (E-Ink: thin, pliable medium capable of electronic encoding)electronic encoding)
Exchange of visual data becomes as Exchange of visual data becomes as ubiquitous as exchange of text documents ubiquitous as exchange of text documents in 2001in 2001
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 4545
An Ideal Visualization SystemAn Ideal Visualization System
Reusable Reusable modulesmodules
FlexibleFlexible
Ease of useEase of use
Low memoryLow memory
ExtensibleExtensible
ScriptableScriptable
Good debuggingGood debugging
Portable Intelligent defaults Changeable
defaults Interpreted and
compiled modes Novice and expert
modes Mathematical text
editor
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 4646
Future trends in VisualizationFuture trends in Visualization
Use of Object-Oriented design patterns for Use of Object-Oriented design patterns for reusabilityreusability
Plugin technology on distributed systemsPlugin technology on distributed systems
Extensive use of visualization across the Extensive use of visualization across the network network
Increased intelligence in software Increased intelligence in software
Insertion of new algorithms without Insertion of new algorithms without recompilationrecompilation
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 4747
Wireless SpeedsWireless SpeedsPresent and Near FuturePresent and Near Future
Present: 802.11bPresent: 802.11b– Range: 150 m, 10 Mbit/secRange: 150 m, 10 Mbit/sec
11stst quarter 2002: 802.11a quarter 2002: 802.11a– Range: 150 m, 54 Mbit/sec Range: 150 m, 54 Mbit/sec – Not compatible with 802.11bNot compatible with 802.11b
33rdrd quarter 2002: 802.11g quarter 2002: 802.11g– Range: N/A, 54 Mbit/secRange: N/A, 54 Mbit/sec– Compatible with 802.11b!!Compatible with 802.11b!!
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 4848
OQO: true mobile computing?OQO: true mobile computing?Fall 2002Fall 2002
Up to 1 GHz Up to 1 GHz
Crusoe chipCrusoe chip
256 Mbytes memory256 Mbytes memory
10 Gbyte hard disk10 Gbyte hard disk
• Touchscreen• USB/Firewire• Windows XP• 4” screen
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 4949
Future WorkFuture Work
Integrate these packages into consistent Integrate these packages into consistent framework framework – They are currently developed independentlyThey are currently developed independently
Increase support for XMLIncrease support for XML
(hopefully) integrate some of these packages (hopefully) integrate some of these packages into frameworks developed in Pervasive Group into frameworks developed in Pervasive Group (do not reinvent the wheel)(do not reinvent the wheel)
Investigate interoperability Java/PythonInvestigate interoperability Java/Python
Integrate Streaming with Zope or Java AppletsIntegrate Streaming with Zope or Java Applets
July 10, 2002July 10, 2002 Pervasive Knowledge Center, Indiana UniversityPervasive Knowledge Center, Indiana University 5050
ConclusionsConclusions
We have developed several web-based We have developed several web-based services related to visualizationservices related to visualization
Objective is to access and manipulate Objective is to access and manipulate data from remote sitesdata from remote sites
Hope is to allow multiple users to Hope is to allow multiple users to manipulate the same data concurrentlymanipulate the same data concurrently
Difficulty: integrating multiple languages Difficulty: integrating multiple languages together without sacrificing efficiencytogether without sacrificing efficiency