visualize your large datasets!...science edutainment extract information, gain insight visual cues,...
TRANSCRIPT
![Page 1: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/1.jpg)
Peter Messmer, 3/20/2019
VISUALIZE YOUR LARGE DATASETS!
![Page 2: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/2.jpg)
2
SCIENTIFIC VIS VS. EDUTAINMENT
2
Science Edutainment
Extract information, gain insightVisual cues, interactivity enhance focus
Helps to understand data
ParaView, VisIt, Matlab, Python,…
Tell a storySupport story with visual FX
Catch viewer’s attention
Houdini, Blender, Maya, ..
![Page 3: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/3.jpg)
3
Coordinate
transformations
Feature
extraction
Thresholding
Isosurfaces,
Isovolumes
Streamlines
Field Operators
(Gradient, Curl,.. )
Clip, Slice
Binning,
Resample
Surface
Rendering
Volume
RenderingLine
Rendering
Compositing
VISUALIZATION ≠ RENDERING ** but it’s a part of it
![Page 4: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/4.jpg)
4
CHALLENGES AT LARGE SCALE
Locality Complexity Tools
![Page 5: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/5.jpg)
5
CHALLENGES AT LARGE SCALE
Locality Complexity Tools
Leave it where it is Use optimal resource Minimal intrusion
![Page 6: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/6.jpg)
6
VISUALIZATION IN THE DATACENTER
![Page 7: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/7.jpg)
7
VISUALIZATION IN THE DATACENTERBenefits of Rendering on Supercomputer
Scale with SimulationNo Need to Scale Separate Vis Cluster
Cheaper Infrastructure All Heavy Lifting Performed on the Server
Interactive High-Fidelity Rendering Improves Perception and Scientific Insight
![Page 8: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/8.jpg)
8
CHALLENGES IN THE DATACENTER
Headless rendering
Remoting
Vis Software Stack
![Page 9: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/9.jpg)
9
HEADLESS RENDERING
![Page 10: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/10.jpg)
10
HEADLESS RENDERING
OpenGL context management
Two approaches for context handling:
- X server: mgmt. by separate process
- EGL: mgmt. by driver
How to rasterize without an attached display
![Page 11: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/11.jpg)
11
X SERVER ON HEADLESS
Recommended if code modification is not an option
nvidia-xconfig –o xorg.conf --allow-empty-initial-configuration -a
-o output file
-a enables all GPUs (-–enable-all-gpus)
--allow-empty-initial-configuration start even if no attached display detected
How to rasterize without an attached display
![Page 12: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/12.jpg)
12
CONTEXT MANAGEMENT WITH EGL
Requires minor application modification of GLX context initialization
How to rasterize without an attached display
https://devblogs.nvidia.com/egl-eye-opengl-visualization-without-x-server/
// 1. Initialize EGL
EGLDisplay eglDpy = eglGetDisplay(EGL_DEFAULT_DISPLAY);
EGLint major, minor;
eglInitialize(eglDpy, &major, &minor);
// 2. Select an appropriate configuration
EGLint numConfigs; EGLConfig eglCfg;
eglChooseConfig(eglDpy, configAttribs, &eglCfg, 1, &numConfigs);
// 3. Create a surface
EGLSurface eglSurf = eglCreatePbufferSurface(eglDpy, eglCfg, pbufferAttribs);
// 4. Bind the API
eglBindAPI(EGL_OPENGL_API);
![Page 13: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/13.jpg)
13
REMOTING
![Page 14: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/14.jpg)
14
FLEXIBLE GPU ACCELERATION ARCHITECTURE
* Diagram represents support for the NVIDIA Turing GPU family
** 4:2:2 is not natively supported on HW
*** Support is codec dependent
Independent CUDA Cores & Video Engines
![Page 15: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/15.jpg)
15
VIDEO CODEC SDKAPIs For Hardware Accelerated Video Encode/Decode
What’s New with Turing GPUs and Video Codec SDK 9.0
• Up to 3x decode throughput with multiple decoders on professional cards (Quadro & Tesla)
• Higher quality encoding - H.264 & H.265
• Higher encoding efficiency (15% lower bitrate than Pascal)
• HEVC B-frames support
• HEVC 4:4:4 decoding support
NVIDIA GeForce Now is made possible by leveraging NVENC in the datacenter and streaming the result to end clients
https://developer.nvidia.com/nvidia-video-codec-sdk
![Page 16: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/16.jpg)
16
NVPIPE
Simple C API
H.264, HEVC
RGBA32, uint4, uint8, uint16
Lossy, Lossless
Host/Device memory, OpenGL textures/PBOs
https://github.com/NVIDIA/NvPipe
Issues? Suggestions? Feedback welcome!
A Lightweight Video Codec SDK Wrapper
S9490 – GPU-Enhanced Collaborative Scientific Visualization, Wed 3/20, 11:00-11:50
![Page 17: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/17.jpg)
17
EGL RENDERING + BROWSER STREAMINGPowerful combo for rendering in the cloud
Available upon request
OpenGL
Application GLU
T
NVENC
GLU
T
EG
LU
T
OpenGL
Application
Java Script Clientmp4 stream
Web
Server
![Page 18: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/18.jpg)
18
TOOL COMPLEXITY
![Page 19: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/19.jpg)
19
KITWARE PARAVIEWOpen-Source (Distributed) Visualization Package
NVIDIA IndeX PluginOpenGL
![Page 20: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/20.jpg)
20
VTK: VISUALIZATION TOOLKIT
Process data using pipelines made up of filters
Forms the foundation of ParaView, VisItand many other vis tools
OpenGL, Software raytracing
Open Source Scientific Visualization Toolbox
S9458 – VTK-m: Lessons from Building a Visualization Toolkit for Massively Threaded Architectures, Wed 3/20, 3:00-3:50
![Page 21: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/21.jpg)
21
CONTAINERS: SIMPLIFYING WORKFLOWS
Simplifies Deployments
- Eliminates complex, time-consuming builds and installs
Get started in minutes
- Simply Pull & Run the app
Portable
- Deploy across various environments, from test to production with minimal changes
WHY CONTAINERS
S9525 – Containers Democratize HPC, Tue 3/19
![Page 22: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/22.jpg)
22
NGC CONTAINERS: ACCELERATING WORKFLOWS
Simplifies Deployments
- Eliminates complex, time-consuming builds and installs
Get started in minutes
- Simply Pull & Run the app
Portable
- Deploy across various environments, from test to production with minimal changes
WHY CONTAINERS
Optimized for Performance- Monthly DL container releases offer latest features and
superior performance on NVIDIA GPUs
Scalable Performance
- Supports multi-GPU & multi-node systems for scale-up & scale-out environments
Designed for Enterprise & HPC environments
- Supports Docker & Singularity runtimes
Run Anywhere
- Pascal/Volta/Turing-powered NVIDIA DGX, PCs, workstations, servers and top cloud platforms
WHY NGC CONTAINERS
![Page 23: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/23.jpg)
23
GPU-OPTIMIZED SOFTWARE CONTAINERSOver 50 Containers on NGC
DEEP LEARNING MACHINE LEARNING
HPC VISUALIZATION
INFERENCE
GENOMICS
NAMD | GROMACS | more
RAPIDS | H2O | more TensorRT | DeepStream | more
Parabricks ParaView | IndeX | more
TensorFlow | PyTorch | more
![Page 24: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/24.jpg)
24
RENDERING: 2D
![Page 25: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/25.jpg)
25
GPU ACCELERATED VECTOR GRAPHICS
GPUs primary rendering focus on 3D
2D rendering is so much more common
Often served out via web pages
Examples
graphs, diagrams, networks, flow charts, maps, vector artwork, Flash-like animation, etc. etc.
Acceleration of 2D Graphics
![Page 26: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/26.jpg)
26
SCALABLE VECTOR GRAPHICS (SVG)
Pros:
- Wide support, efficient implementations
- Very powerful feature set
Cons:
- Slow due to client-side rendering in browser
- SVG contains data, not just pixels
GPU cloud rendering addresses both downsides
Support via NV_path_rendering OpenGL extension
![Page 27: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/27.jpg)
27
SVG RENDERING PERFORMANCEBigger benefit for more complex scenes
Quadro RTX6000i7-5820k
![Page 28: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/28.jpg)
28
RENDERING: RAYTRACING
![Page 29: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/29.jpg)
29
ANATOMY OF A RAY-TRACING APP
- Intersection of rays with geometry
- Arbitrary new rays started at arbitrary locations
- Arbitrary operations at intersection points
- Typically in 3D space
- Hierarchical spatial decomposition as acceleration structure
Interplay of Rays and Geometry
geometry
geometry
geometry
![Page 30: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/30.jpg)
30
TURING RT CORES
RT Cores perform
● Ray-BVH Traversal● Instancing: 1 Level● Ray-Triangle Intersection
Return to SM for
● Multi-level Instancing● Custom Intersection● Shading
Programming via OptiX RT frameworkLow overhead interop with CUDA
Hardware Accelerated Ray Tracing
S9768 – New Features in OptiX 6.0Wed 3/20, 1:00-1:50pm
![Page 31: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/31.jpg)
31
BETTER INSIGHT VIA RAYTRACINGIt’s not just pretty pictures
S9589 – Interactive High-Fidelity Biomolecular and Cellular Visualization with RTX Ray Tracing APIs
Wed 3/20, 3:00-3:50pm
![Page 32: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/32.jpg)
32
OPTIX AI DENOISER IN PARAVIEW
Without
Denoiser With
Denoiser
![Page 33: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/33.jpg)
33
VISRTXVisualization Framework Powered by NVIDIA RTX Technology
Progressive forward pathtracer with NEE/MIS
Hardware-acceleration through OptiX
MDL for physically-based materials
AI denoiser
Area lights, Depth of Field, Tone mapping, etc.
Open-source C++ library
Feedback welcome (issues, PRs, e-mail)!
http://github.com/NVIDIA/VisRTX
![Page 34: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/34.jpg)
34
VISRTX + PARAVIEW
VisRTX open-source on GitHub
Shipped with upcoming ParaView 5.7
• No additional steps necessary!
S9697 – Path Tracing in ParaView-OptiX: RTX for Scientific VisualizationThu 3/21, 4:00-4:50
![Page 35: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/35.jpg)
35
RAYTRACING PAYS OFF AT SCALE
ParaView Manyspheres Benchmark
![Page 36: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/36.jpg)
36
RENDERING: VOLUMES
![Page 37: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/37.jpg)
37
NVIDIA IndeX SDK
Large scale and distributed data rendering
Scene management with volume data
Transparent support for NVLink
Higher-order filtering, advanced lighting & transfer functions
https://developer.nvidia.com/index
S9692 – NVIDIA IndeX – Implementing Cloud Services for Complex Scientific Data Visualization, Tue 3/21, 4:00-4:50
![Page 38: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/38.jpg)
38
NVIDIA INDEX FOR PARAVIEW PLUGIN
- NVIDIA IndeX rendering in ParaView
- Retain ParaView workflows
- Structured and unstructured meshes
Learn more:
http://www.nvidia.com/object/index-paraview-plugin.html
![Page 39: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/39.jpg)
39
SUMMARY
Headless rendering
Accelerated video streaming
2D graphs can benefit from GPUs as well
Raytracing great to enhance vis perception
VisRTX raytracing vis tookit (in ParaView, VTK)
GPU accelerated scalable volume rendering part of open source tools
Wide Palette for Visualization and Rendering in Datacenter/Cloud
![Page 40: VISUALIZE YOUR LARGE DATASETS!...Science Edutainment Extract information, gain insight Visual cues, interactivity enhance focus Helps to understand data ParaView, VisIt, Matlab, Python,…](https://reader033.vdocument.in/reader033/viewer/2022041607/5e34dd6a83d5220913799d47/html5/thumbnails/40.jpg)