game computing applications - home -...

Mixed Resolution Rendering

Jeremy Shopf

AMDGame Computing Applications

Outline

» Motivation

» Uniform Upsampling

» Adaptive Upsampling

» Demo

Offscreen Particles

Used in many shipping titles

Method:

Render scene without particles

Downsample depth buffer

Render particles to low-res color buffer, using downsampled depth buffer for depth testing

Composite low-res particles

Fix up gaps caused by low-res depth testing

[Nguyen04]

High-Res Version (20 fps@1280x960)

Offscreen Particles

Offscreen Particles

2x Downsampled Offscreen Version (32 fps@1280x960)

Fix-up pixels (rest are low-res)

Offscreen Particles

Offscreen Particles

Tradeoff:

+ Save on Fill Rate

- Lose resolution

Mixed Resolution Rendering

» For relatively low frequency effects, mixed resolution rendering results in barely perceivable loss in quality

» Dramatic performance improvement

Scales linearly minus small overhead for upsampling

» Allows scaling of effects to maintain framerate

Bilinear Interpolation

Used in Offscreen Particles

+ Efficient and simple

- Doesn’t respect depth discontinuities

Requires separate discontinuity detection pass

Hi-Res Texels

Low-Res Texels

Bilateral Upsampling

» Modify bilinear weights by normal and depth discontinuities

» Weight each coarse sample by:

Bilinear weight

Normal similarity weight

Depth similarity weight

[Sloan07]

float4 vBilinearWeights[4] = {

// 0 1 2 3ddddddfloat4( 9/16, 3/16, 3/16, 1/16 ), // 0float4( 3/16, 9/16, 1/16, 3/16 ), // 1float4( 3/16, 1/16, 9/16, 3/16 ), // 2float4( 1/16, 3/16, 3/16, 9/16 ) // 3

};

Computing bilinear weights:

Hi-Res Texels

Low-Res Texels

Bilateral Upsampling

0

2

1

0

3

1

2 3

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

0 0.2 0.4 0.6 0.8 1 1.2

float3 vNormalsCoarse[4] = {…}

float vNormalHiRes = ...

for(int i=0;i<4;i++)

{vNormalWeights[i] = dot( vNormalsCoarse[i],

vNormalHiRes );vNormalWeights[i] = pow(vNormalWeights[i] , 32 );

}

Computing normal weights:

Bilateral Upsampling

Hi-Res Texels

Low-Res Texels

0

2

1

0

3

1

2 3

float fDepthsCoarse[4] = {…};float fDepthHiRes = …;

for(int i=0;i<4;i++){

float fDepthDiff = fDepthHiRes – fDepthsCoarse[i];vDepthWeights[i] = 1.0/( EPSILON + abs(fDepthDiff ));

}

Computing depth weights:

Bilateral Upsampling

Hi-Res Texels

Low-Res Texels

0

2

1

0

3

1

2 3

Putting it all together:

for(int nSample=0; nSample<4; nSample++){

float fWeight = vNormalWeights[nSample] *vDepthWeights[nSample] *vBilinearWeights[nTexel][nSample];

fTotalWeight += fWeight;vUpsampled += vShadingCoarse[nSample]*fWeight;

}

vUpsampled /= fTotalWeight;

Hi-Res Texels

Low-Res Texels

Bilateral Upsampling

0

2

1

0

3

1

2 3

Degenerate Case

No valid coarse samplesEasy to detect this situationHappens infrequently

Options: - Pick one coarse sample - Average coarse samples- Set stencil and do an additional pass to calculate these at full res

Illustration of Degenerate Case

Crease Shading

Diffuse only

Bilateral Upsampling Results

High-Res Shading(67 fps@1024x1024)

Bilateral Upsampling Results

Crease Shading

2x Downsampled Shading(166 fps@1024x1024)

Bilateral Upsampling Results

Crease Shading

Bilateral Upsampling Results

Sphere AO

No AO

High-Res AO (62 fps@1280x960)

Sphere AO

Bilateral Upsampling Results

2x Downsampled AO (135 fps@1280x960)

Sphere AO

Bilateral Upsampling Results

Bilateral Upsampling

+ Small amount of overhead

+ Easily to implement

- Uniform sampling

Undersampling small features

Oversampling low resolution regions

Pull-push Upsampling

» Used to reconstruct images from sparse pixels

» Allows adaptive sampling

» Used for lots of stuff:

DoF [Krauss and Strengert 07]

Imperfect Shadow Maps [Ritschel et al. 08]

….

[Mitchell87,Grossman97]

Pull-push Upsampling

Two phase algorithm fills in unknown values hierarchically

Pull : Bottom-up averaging of valid values

Two phase algorithm fills in unknown values hierarchically

Push : Top-down interpolation of higher levels

Pull-push Upsampling

Soft Shadow Example

» “Real-time Soft Shadow Mapping by Backprojection” [Guennebaud06,07]

» Use a min/max pyramid of the shadow depth buffer to accelerate computation by culling unimportant shadowmap texels

Pull-push Upsampling

Select pixels to be shaded based on penumbra width

8 1 4 1

1 2 1 2

4 1 ∞ 1

1 2 1 2

Estimated Penumbra Width

<

Skip Computation

[Günnebaud06]

Screen Resolution Soft Shadows

Adaptive Sampling

Screen Resolution Soft Shadows

26 fps @1024x1024

Adaptive Sampling

58 fps @1024x1024

» Shading sparse points under-utilizes GPU

» Need to pack points to be shaded

» Unpack before pull-push interpolation

GPU Implementation

Discussion

» Selection function must be cheap to compute and the shading function expensive

» Shading in packed texture may be limiting Proxy-based shading (e.g. Sphere AO)

incompatible

» Discontinuities have to be handled at full resolution

MultiRes Upsampling

» Adaptive upsampling that respects discontinuities

» Compute shading at several different resolutions

Upsample and combine hierarchically

Use bilinear interpolation in regions with no discontinuities

[NicholsWyman09]

Multiresolution Indirect Illumination

» Distribute Virtual Point Lights in scene via shadow map

» Compute direct illumination from VPLs

» 400 VPLs = 400 lighting calculations per pixel

Choosing a resolution

» For each resolution (lowest to highest)

For each texel If there is no discontinuity

Compute shading

Else

Wait to compute this region at a higher level

Detecting Discontinuities

» Construct hierarchical Min/Max depth and normal buffers of scene from viewer

Like a Mip Map, but stores min and max rather than average for all resolutions

» If Max-Min > some threshold, there is a discontinuity at this resolution

UpsamplingFor each resolution (2nd lowest to highest)

Upsample previous levels

If all four coarse samples are available, use bilinear interpolation, otherwise point sample

Combine with current res

Demo

~22 fps full res~57 fps multi res

400 VPLs per pixel1024x1024 resolution

Conclusions

» Mixed resolution rendering can save you perf with a modicum of overhead

» Uniform Upsampling

Lightweight and scales well

» Adaptive Upsampling

Adaptive but with more overhead

More Details

Read my blog, level of detail

http://www.jshopf.com/blog

"Mixed Resolution Rendering"

Contact me:

jshopf@gmail.com

Thank you: Greg Nichols and Chris Wyman,

GCAG

Thanks!

References

Peter-Pike Sloan, Naga K. Govindaraju, Derek Nowrouzezahrai, John Snyder. "Image-Based Proxy Accumulation for Real-Time Soft Global Illumination". Pacific Graphics 2007.

Greg Nichols and Chris Wyman. "Multiresolution Splatting for Indirect Illumination." ACM Symposium on Interactive 3D Graphics and Games, 83-90. 2009.

D.P. Mitchell. "Generating Antialiased Images at Low Sampling Densities". In Proceedings of SIGGRAPH 1987, pages 65-72

H. Nguyen. "Fire In the Vulcan Demo." GPU Gems.

J.P. Grossman. "Point Sample Rendering". In 9th Eurographics Workshop on Rendering 1998. Pages 181-192.

Gael Guennebaud, Loic Barthe, Mathias Paulin. "High-Quality Adaptive Soft Shadow Mapping." Eurographics 2007.

Gael Guennebaud, Loic Barthe, Mathias Paulin. "Real-time Soft Shadow Mapping by Backprojection." Eurographics Symposium on Rendering 2006.

M. Krauss and M. Strengert. "Depth-of-Field Rendering using Pyramidal Image Processing". Computer Graphics Forum Volume 26, Issue 3. 2007.

T. Ritschel, T. Grosch, M. H. Kim, H.-P. Seidel, C. Dachsbacher, J. Kautz ACM Trans. on Graphics (Proceedings SIGGRAPH Asia 2008), 27(5), 2008.