nvidia graphics technology libraries for rt rendering ......nvidia graphics technology libraries...

NVIDIA Graphics Technology libraries for real-time rendering

Agenda

Introducing NVIDIA Graphics Technology libraries

TXAA

WaveWorks

NVIDIA Graphics Technology libraries

…and more in the works

TXAA WaveWorks

HBAO+ PN-AEN

NVIDIA Graphics Technology libraries

Field Guide

Solves a substantial graphics problem

Visible and important to end-users

For real time applications…

…with an emphasis on games

WaveWorks is…

...a library for simulating wind-driven waves on large

bodies of water, in real time, using GPU acceleration

(BUT: the application still defines the look)

(DEMO)

WaveWorks’ mission space

Wide range

of weather

conditions

Range of

hardware

Range of

lengths

Beaufort 12

Beaufort 1

GPU

CPU

1km

features

1cm

features

Detour: The Beaufort scale

An empirical observation-based wind speed scale

Devised 1805 by Francis Beaufort, British Navy

Originally 0 to 12

0 = Flat

6 = Long waves begin to form. White foam crests are very frequent.

Some airborne spray is present.

12 = Huge waves. Sea is completely white with foam and spray. Air is

filled with driving spray, greatly reducing visibility.

Modern extensions to 13 and more

The length range challenge

Need both uniqueness and fine detail for a simulation to

look good

UNIQUENESS: minimize objectionable

repeating patterns over large areas

FINE DETAIL: accurately portray near-

camera short-wavelength features

The length range challenge

WaveWorks supports a length range of 64,000 : 1 -

— 64,000 represents the uniqueness dimension within which the

simulation produces non-repeating results

— 1 is the finest spacing of height samples seen near-camera

— e.g. 1.5cm detail vs. 1km uniqueness

— Handles 1km-order wavelengths

Beaufort 12 relevance

The algo

Tessendorf‟s spectral

algo, based on

Phillips spectrum

— Used in many

movies, right back

to Waterworld and

Titanic in ‟97, and

more recently Life

of Pi

Simulation step runs in <2ms on GTX680 at max setting

The algo

Per-params-change (CUDA/CPU) Per-frame (CUDA/CPU) Per-frame (PS)

Explainer: the Phillips spectrum

Predicted by Phillips in 1950s, working from first principles

of Fluid Mechanics

Subsequently validated by experimental oceanographic data

in 1960s

Models a „fully developed‟ sea state

So a great choice for „default‟ spectrum

— NB: rest of algo is spectrum-agnostic

Explainer: the Phillips spectrum

1 10 100 1000 10000

λ, m

Beaufort 6

Beaufort 7

Beaufort 8

Beaufort 9

Explainer: dispersion

Q: given some spectral wavelength, λ, how to evolve that

wavelength‟s signal over time?

— Or: what is ω(λ) ?

A: the dispersion relation –

ω2(λ) = 2πg / λ

Used to advance the phase of each wavelength at the correct rate

The algo

Per-params-change (CUDA/CPU) Per-frame (CUDA/CPU) Per-frame (PS)

What is in the distro

./bin/x86/

./bin/x64/

./demo/

./inc/

./lib/x86/

./lib/x64/

./shader/

./IntegrationNotes.txt

WaveWorks 1.2

1.2 is latest greatest release

— Multi-res simulation for 64,000 : 1 length range

— Geometry LODing

Coarse level: quad tree

Fine level, D3D9/10: geo-morphing

Fine level, D3D11: tessellation

CPU

sim

GPU

sim

D3D9

D3D10

D3D11

Supported configurations

WaveWorks1.3

ETA: post-GDC

— foam

— GPU acceleration

for evolving

spectra

— Beaufort presets

How to exploit

The obvious use-case: render a good-looking sea

Also: as prime mover for 2ry/3ry effects

— For vessel physics

— For vessel spray/foam production

And from there, audio for spray/foam

— Conceptually: a big high-quality animated noise field

API – init/teardown

// Use these calls to globally initialize/release on D3D device create/destroy.

nvsdk_result NVSDK_Water_InitD3D9(IDirect3DDevice9* pD3DDevice);

nvsdk_result NVSDK_Water_ReleaseD3D9(IDirect3DDevice9* pD3DDevice);

nvsdk_result NVSDK_Water_InitD3D10(ID3D10Device* pD3DDevice);

nvsdk_result NVSDK_Water_ReleaseD3D10(ID3D10Device* pD3DDevice);

nvsdk_result NVSDK_Water_InitD3D11(ID3D11Device* pD3DDevice);

nvsdk_result NVSDK_Water_ReleaseD3D11(ID3D11Device* pD3DDevice);

WaveWorks

App

API – configure a simulation

simulation

API – configure a simulation

struct NVSDK_Water_Simulation_Params

{

NVSDK_Water_Simulation_DetailLevel detail_level;

float fft_period;

float2 wind_dir;

float wind_speed; // <- the most important param!

float wind_dependency;

float choppy_scale;

bool readback_displacements;

...

};

NVSDK_Water_Simulation_CreateD3D11( const NVSDK_Water_Simulation_Params& params,

ID3D11Device* pD3DDevice,

NvWaterSimulationHandle* pResult);

// D3D9/10 creation is similar

NVSDK_Water_Simulation_UpdateParams( NvWaterSimulationHandle hSim,

const NVSDK_Water_Simulation_Params& params);

NVSDK_Water_Simulation_Destroy(NvWaterSimulationHandle hSim);

API – configure a simulation

struct NVSDK_Water_Simulation_Params

{

NVSDK_Water_Simulation_DetailLevel detail_level;

float fft_period;

float2 wind_dir;

float wind_speed; // <- the most important param!

float wind_dependency;

float choppy_scale;

bool readback_displacements;

...

};

NVSDK_Water_Simulation_CreateD3D11( const NVSDK_Water_Simulation_Params& params,

ID3D11Device* pD3DDevice,

NvWaterSimulationHandle* pResult);

// D3D9/10 creation is similar

NVSDK_Water_Simulation_UpdateParams( NvWaterSimulationHandle hSim,

const NVSDK_Water_Simulation_Params& params);

NVSDK_Water_Simulation_Destroy(NvWaterSimulationHandle hSim);

Simulations are instanced

API – configure a simulation

struct NVSDK_Water_Simulation_Params

{

NVSDK_Water_Simulation_DetailLevel detail_level;

float fft_period;

float2 wind_dir;

float wind_speed; // <- the most important param!

float wind_dependency;

float choppy_scale;

bool readback_displacements;

//...

};

NVSDK_Water_Simulation_CreateD3D11( const NVSDK_Water_Simulation_Params& params,

ID3D11Device* pD3DDevice,

NvWaterSimulationHandle* pResult);

// D3D9/10 creation is similar

NVSDK_Water_Simulation_UpdateParams( NvWaterSimulationHandle hSim,

const NVSDK_Water_Simulation_Params& params);

NVSDK_Water_Simulation_Destroy(NvWaterSimulationHandle hSim);

GPU-accelerated in 1.3

WaveWorks

App

API – pumping the simulation

simulation

Update()

API – pumping the simulation

// Make these calls once per frame per simulation

NVSDK_Water_Simulation_SetTime(NvWaterSimulationHandle hSim, float fAppTime);

NVSDK_Water_Simulation_UpdateTick( NvWaterSimulationHandle hSim,

IUnknown* pDC,

NvWaterSavestateHandle hSavestate);

API – pumping the simulation

// Make these calls once per frame per simulation

NVSDK_Water_Simulation_SetTime(NvWaterSimulationHandle hSim, float fAppTime);

NVSDK_Water_Simulation_UpdateTick( NvWaterSimulationHandle hSim,

IUnknown* pDC,

NvWaterSavestateHandle hSavestate);

NB: random-access in time

WaveWorks

App Shader

API – rendering the results

WaveWorks

attribute

fragments

simulation

Update()

API – rendering the results

// HLSL APIs

struct NV_WATER_VERTEX_OUTPUT

{

NV_WATER_INTERPOLATED_VERTEX_OUTPUT interp;

float3 pos_world;

float3 pos_world_undisplaced;

float3 world_displacement;

};

NV_WATER_VERTEX_OUTPUT NV_GetDisplacedWaterVertex(NV_WATER_VERTEX_INPUT In);

struct NV_WATER_SURFACE_ATTRIBUTES

{

float3 normal;

float3 eye_dir;

float fold;

};

NV_WATER_SURFACE_ATTRIBUTES NV_GetWaterSurfaceAttributes(NV_WATER_INTERPOLATED_VERTEX_OUTPUT In);

API – rendering the results

// HLSL APIs

struct NV_WATER_VERTEX_OUTPUT

{

NV_WATER_INTERPOLATED_VERTEX_OUTPUT interp;

float3 pos_world;

float3 pos_world_undisplaced;

float3 world_displacement;

};

NV_WATER_VERTEX_OUTPUT NV_GetDisplacedWaterVertex(NV_WATER_VERTEX_INPUT In);

struct NV_WATER_SURFACE_ATTRIBUTES

{

float3 normal;

float3 eye_dir;

float fold;

};

NV_WATER_SURFACE_ATTRIBUTES NV_GetWaterSurfaceAttributes(NV_WATER_INTERPOLATED_VERTEX_OUTPUT In);

Call this in VS/DS

API – rendering the results

// HLSL APIs

struct NV_WATER_VERTEX_OUTPUT

{

NV_WATER_INTERPOLATED_VERTEX_OUTPUT interp;

float3 pos_world;

float3 pos_world_undisplaced;

float3 world_displacement;

};

NV_WATER_VERTEX_OUTPUT NV_GetDisplacedWaterVertex(NV_WATER_VERTEX_INPUT In);

struct NV_WATER_SURFACE_ATTRIBUTES

{

float3 normal;

float3 eye_dir;

float fold;

};

NV_WATER_SURFACE_ATTRIBUTES NV_GetWaterSurfaceAttributes(NV_WATER_INTERPOLATED_VERTEX_OUTPUT In);

Call this in PS

WaveWorks

App Shader

API – rendering the results

WaveWorks

attribute

fragments

simulation

Update()

shader constant indices

reflection

API – rendering the results

// C APIs – enumeration of shader constants

struct NVSDK_Water_ShaderInput_Desc

{

enum InputType {

VertexShader_FloatConstant = 0,

VertexShader_ConstantBuffer,

VertexShader_Texture,

VertexShader_Sampler,

HullShader_FloatConstant,

//... Etc.

};

InputType Type;

nvsdk_cstr Name;

nvsdk_uint RegisterOffset;

};

// D3D9/10 are similar

UINT NVSDK_Water_Simulation_GetShaderInputCountD3D11();

nvsdk_result NVSDK_Water_Simulation_GetShaderInputDescD3D11( UINT inputIndex,

NVSDK_Water_ShaderInput_Desc* pDesc);

WaveWorks

App Shader

API – rendering the results

WaveWorks

attribute

fragments

simulation

Update()

SetRenderState()

shader constant indices

reflection

API – rendering the results

// C APIs – getting the simulation to set its render state

NVSDK_Water_Simulation_SetRenderState( NvWaterSimulationHandle hSim,

IUnknown* pDC,

const float4x4& matView,

const uint* pShaderInputRegisterMappings,

NvWaterSavestateHandle hSavestate);

API – rendering the results

// C APIs – getting the simulation to set its render state

NVSDK_Water_Simulation_SetRenderState( NvWaterSimulationHandle hSim,

IUnknown* pDC,

const float4x4& matView,

const uint* pShaderInputRegisterMappings,

NvWaterSavestateHandle hSavestate);

num entries =

NVSDK_Water_Simulation_GetShaderInputCount()

WaveWorks

App Shader

API – rendering the results

WaveWorks

attribute

fragments

simulation

Update()

SetRenderState()

shader constant indices

reflection

API – rendering the results

Shader hookup is likely trickiest aspect of integration

Demo app: ships as source within distro

Includes an example using -

— Named constants

— FX file format

— Shader reflection to get constant offsets from compiled FX‟s

Good way to follow the workings

WaveWorks

App Shader

API – geometry generator

WaveWorks

attribute

fragments

WaveWorks

geomgen

fragments

geomgen simulation

Draw()

Update()

SetRenderState()

shader constant indices

reflection

API – quad-tree generator

Quadtree is a “stock” generator

shipped in the lib

Does frustum culling

Does mesh LOD

Tiling is calculated in world

space

— ensures mesh is stable w.r.t

camera rotation

API – quad-tree generator

// D3D9/10 are similar

NVSDK_Water_Quadtree_CreateD3D11( const NVSDK_Water_Quadtree_Params& params,

ID3D11Device* pD3DDevice,

NvWaterQuadtreeHandle* pResult);

NVSDK_Water_Quadtree_Destroy(NvWaterQuadtreeHandle hQuadtree);

NVSDK_Water_Quadtree_UpdateParams( NvWaterQuadtreeHandle hQuadtree,

const NVSDK_Water_Quadtree_Params& params);

UINT NVSDK_Water_Quadtree_GetShaderInputCountD3D11();

NVSDK_Water_Quadtree_GetShaderInputDescD3D11( UINT inputIndex,

NVSDK_Water_ShaderInput_Desc* pDesc);

NVSDK_Water_Quadtree_Draw( NvWaterQuadtreeHandle hQuadtree,

IUnknown* pDC,

const float4x4& matView,

const float4x4& matProj,

const uint* pShaderInputRegisterMappings,

NvWaterSavestateHandle hSavestate);

API – quad-tree generator

// D3D9/10 are similar

NVSDK_Water_Quadtree_CreateD3D11( const NVSDK_Water_Quadtree_Params& params,

ID3D11Device* pD3DDevice,

NvWaterQuadtreeHandle* pResult);

NVSDK_Water_Quadtree_Destroy(NvWaterQuadtreeHandle hQuadtree);

NVSDK_Water_Quadtree_UpdateParams( NvWaterQuadtreeHandle hQuadtree,

const NVSDK_Water_Quadtree_Params& params);

UINT NVSDK_Water_Quadtree_GetShaderInputCountD3D11();

NVSDK_Water_Quadtree_GetShaderInputDescD3D11( UINT inputIndex,

NVSDK_Water_ShaderInput_Desc* pDesc);

NVSDK_Water_Quadtree_Draw( NvWaterQuadtreeHandle hQuadtree,

IUnknown* pDC,

const float4x4& matView,

const float4x4& matProj,

const uint* pShaderInputRegisterMappings,

NvWaterSavestateHandle hSavestate);

Initiates drawing

Case study: Just Cause 2

Release 0.5

— Bimodal simulation

— Used Perlin noise for

distant LOD

Integration points of interest -

— Explicit-mode for quadtree

— Save/restore of graphics state

— Readbacks for vessel dynamics

WaveWorks

App Shader

JC2 - readbacks

WaveWorks

attribute

fragments

WaveWorks

geomgen

fragments

geomgen simulation

GetDisplacements()

Draw()

Update()

SetRenderState()

shader constant indices

reflection

JC2 - readbacks

NVSDK_Water_Simulation_GetDisplacements( NvWaterSimulationHandle hSim,

IUnknown* pDC,

const float2* inSamplePoints,

float4* outDisplacements,

uint numSamples

);

IN: x-y sample locations

JC2 - readbacks

NVSDK_Water_Simulation_GetDisplacements( NvWaterSimulationHandle hSim,

IUnknown* pDC,

const float2* inSamplePoints,

float4* outDisplacements,

uint numSamples

);

OUT: displacements

JC2 - readbacks

struct NVSDK_Water_Simulation_Params

{

NVSDK_Water_Simulation_DetailLevel detail_level;

float fft_period;

float2 wind_dir;

float wind_speed; // <- the most important param!

float wind_dependency;

float choppy_scale;

bool readback_displacements;

...

};

NVSDK_Water_Simulation_CreateD3D11( const NVSDK_Water_Simulation_Params& params,

ID3D11Device* pD3DDevice,

NvWaterSimulationHandle* pResult);

// D3D9/10 creation is similar

NVSDK_Water_Simulation_UpdateParams( NvWaterSimulationHandle hSim,

const NVSDK_Water_Simulation_Params& params);

NVSDK_Water_Simulation_Destroy(NvWaterSimulationHandle hSim);

set to „true‟ to

get readbacks

JC2 – save/restore

WaveWorks

App Shader

WaveWorks

attribute

fragments

WaveWorks

geomgen

fragments

geomgen simulation

GetDisplacements()

Draw()

Update()

SetRenderState()

shader constant indices

reflection

save/restore

Restore()

JC2 – save/restore

// Create/destroy save-state object

enum NVSDK_Water_StatePreserveFlags

{

NVSDK_Water_StatePreserve_None = 0,

NVSDK_Water_StatePreserve_Shaders = 1,

NVSDK_Water_StatePreserve_ShaderConstants = 2,

NVSDK_Water_StatePreserve_Samplers = 4,

// etc...

};

NVSDK_Water_Savestate_CreateD3D11( NVSDK_Water_StatePreserveFlags PreserveFlags,

ID3D11Device* pD3DDevice,

NvWaterSavestateHandle* pResult

);

NVSDK_Water_Savestate_Destroy(NvWaterSavestateHandle hSavestate);

// Restore state after API call, and clear any saved state

NVSDK_Water_Savestate_Restore(NvWaterSavestateHandle hSavestate, IUnknown* pDC);

JC2 – save/restore

// Typical call sequence..

// Frame N

NVSDK_Water_Simulation_SetRenderState(...);

NVSDK_Water_Quadtree_Draw(...);

NVSDK_Water_Savestate_Restore(...);

// Frame N+1

NVSDK_Water_Simulation_SetRenderState(...);

NVSDK_Water_Quadtree_Draw(...);

NVSDK_Water_Savestate_Restore(...);

// Frame N+2

NVSDK_Water_Simulation_SetRenderState(...);

NVSDK_Water_Quadtree_Draw(...);

NVSDK_Water_Savestate_Restore(...);

// etc.

clears any saved state

JC2 – quadtree explicit mode

By default, quad-tree coverage is ubiquitous

For mixed-terrain games, like JC2, it is better to completely

disable inland tiles

So quadtree supports an „explicit‟ mode for this use-case

JC2 – quadtree explicit mode

struct NVSDK_Water_Quadtree_Params

{

//...

float min_patch_length;

//...

};

NVSDK_Water_Quadtree_CreateD3D11( const NVSDK_Water_Quadtree_Params& params,

ID3D11Device* pD3DDevice,

NvWaterQuadtreeHandle* pResult);

NVSDK_Water_Quadtree_Destroy(NvWaterQuadtreeHandle hQuadtree);

// Explicit-mode quadtree

NVSDK_Water_Quadtree_AllocPatch(NvWaterQuadtreeHandle hQuadtree, int x, int y, uint lod, bool enabled);

NVSDK_Water_Quadtree_FreePatch(NvWaterQuadtreeHandle hQuadtree, int x, int y, uint lod);

JC2 – quadtree explicit mode

struct NVSDK_Water_Quadtree_Params

{

//...

float min_patch_length;

//...

};

NVSDK_Water_Quadtree_CreateD3D11( const NVSDK_Water_Quadtree_Params& params,

ID3D11Device* pD3DDevice,

NvWaterQuadtreeHandle* pResult);

NVSDK_Water_Quadtree_Destroy(NvWaterQuadtreeHandle hQuadtree);

// Explicit-mode quadtree

NVSDK_Water_Quadtree_AllocPatch(NvWaterQuadtreeHandle hQuadtree, int x, int y, uint lod, bool enabled);

NVSDK_Water_Quadtree_FreePatch(NvWaterQuadtreeHandle hQuadtree, int x, int y, uint lod);

allocating patches = explicit mode

JC2 – quadtree explicit mode

struct NVSDK_Water_Quadtree_Params

{

//...

float min_patch_length;

//...

};

NVSDK_Water_Quadtree_CreateD3D11( const NVSDK_Water_Quadtree_Params& params,

ID3D11Device* pD3DDevice,

NvWaterQuadtreeHandle* pResult);

NVSDK_Water_Quadtree_Destroy(NvWaterQuadtreeHandle hQuadtree);

// Explicit-mode quadtree

NVSDK_Water_Quadtree_AllocPatch(NvWaterQuadtreeHandle hQuadtree, int x, int y, uint lod, bool enabled);

NVSDK_Water_Quadtree_FreePatch(NvWaterQuadtreeHandle hQuadtree, int x, int y, uint lod);

implied patch size =

min_patch_length * 2^lod

JC2 – quadtree explicit mode

struct NVSDK_Water_Quadtree_Params

{

//...

float min_patch_length;

//...

};

NVSDK_Water_Quadtree_CreateD3D11( const NVSDK_Water_Quadtree_Params& params,

ID3D11Device* pD3DDevice,

NvWaterQuadtreeHandle* pResult);

NVSDK_Water_Quadtree_Destroy(NvWaterQuadtreeHandle hQuadtree);

// Explicit-mode quadtree

NVSDK_Water_Quadtree_AllocPatch(NvWaterQuadtreeHandle hQuadtree, int x, int y, uint lod, bool enabled);

NVSDK_Water_Quadtree_FreePatch(NvWaterQuadtreeHandle hQuadtree, int x, int y, uint lod);

[x,y] are whole patches @ lod

Case study: 1.3 demo

Vessel dynamics

Spray generation,

including audio

Reinjecting spray

back into the

foam map

Demo – vessel dynamics

WaveWorks

App

20K hull

sensors

simulation

Depth

readings

Vessel

dynamics GetDisplacements()

Demo – vessel dynamics

Even distribution over hull area for unbiased read

Readback data is parameterized over undisplaced coords

— So search readback data to find water height at worldpos

— i.e. reverse lookups are intrinsically hard

— Roadmap plans should help here

Demo – spray gen

WaveWorks

App

20K hull

sensors

simulation

Depth

readings

Vessel

dynamics GetDisplacements()

1-frame

FIFO

Spraygen

model

Audio FX

Spray

particles

Demo – spray reinjection

WaveWorks

App

Spray

particles Spray

reinject

shader

Foam

map

Demo – spray reinjection

WaveWorks

App

simulation

Shader

WaveWorks

attribute

fragments

SetRenderState()

Neighborhood

height map

Spray

particles Spray

reinject

shader

Foam

map

WaveWorks Roadmap

Productize 1.3 demo features

— Spray generation

— Hull sensing

— GPU acceleration for reverse lookup

Server-mode for Linux

User-defined spectra

TXAA