Fast and Accurate Soft Shadows using Fast and Accurate Soft Shadows using a Real-Time Beam Tracera Real-Time Beam Tracer

Ravi Ramamoorthi

Columbia Vision and Graphics CenterColumbia University

http://www.cs.columbia.edu/~ravir

Intel, Mar 29, 2007

Joint work with Ryan Overbeck (Columbia), Bill Mark (UT Austin)

Beam Tracing:

Reference Quality (Our Method)

Ray Tracing:

Comparable Time 4 Shadow Rays

Ray Tracing:

Comparable Quality 256 Shadow Rays

3 Seconds 3 Seconds 99 Seconds

Note that secondary effects (soft shadows) are often hard to accelerate in modern fast ray tracers

Result: Beam-Traced Soft ShadowsResult: Beam-Traced Soft Shadows

OutlineOutline

Motivation

Beam Tracing Algorithm Beam-Triangle Intersection Beam-KD-tree Traversal

Analysis – Primary Visibility

Soft Shadows

Future Work

BackgroundBackground

Renaissance of interest in ray tracing [Whitted 80]

Real-time ray tracing has become a reality by exploiting geometric coherence of rays in the scene Ray Packets [Wald et al. 2001] Frustum Proxies [Reshetov et al. 2005 (MLRTA)] SIMD instructions (Intel SSE2) to trace multiple rays

Ray tracing can produce significantly higher quality images than standard rasterization based algorithms Potential for a revolution in interactive graphics pipeline

ChallengesChallenges

Problems with current Real-Time Ray Tracing algorithms How to use packets/frusta well for secondary visibility? More geometric coherence than current ray tracers use Per pixel operations (even simple shading) bottleneck Can’t yet afford to subsample pixels leading to aliasing

Our solution: Real-Time Beam Tracing Trace beams instead of individual rays Old technique [Heckbert Hanrahan 84] but previously

considered too slow, complicated

Beam Tracing Beam Tracing

Start with beam and scene geometry For primary visibility, the beam is the view frustum

Beam TracingBeam Tracing

The beam is intersected against the first geometry primitive

Beam TracingBeam Tracing

The beam is split against the primitive’s boundaries (triangle edges)

Beam TracingBeam Tracing

The beam is split against the primitive’s boundaries (triangle edges)

Beam TracingBeam Tracing

Each sub-beam is intersected against the other primitives and is recursively split

Beam TracingBeam Tracing

Each sub-beam is intersected against the other primitives and is recursively split

Beam TracingBeam Tracing

Each sub-beam is intersected against the other primitives and is recursively split

Beam TracingBeam Tracing

Each sub-beam is intersected against the other primitives and is recursively split

Beam TracingBeam Tracing

Each sub-beam is intersected against the other primitives and is recursively split

Beam TracingBeam Tracing

Each sub-beam is intersected against the other primitives and is recursively split

Beam TracingBeam Tracing

The result is the visible surface of the scene

Beam TracingBeam Tracing

The final hit beams often contain many samples/pixels, for large performance gains

Beam Tracing addresses ChallengesBeam Tracing addresses Challenges

How to efficiently address secondary visibility? Large coherent area samples for hard shadows Only need area coverage on light for soft shadows

More geometric coherence than current ray tracers Automatically adapts to (all) available coherence Only split beam when necessary, at triangle boundaries

Per pixel (even simple shading) bottleneck Returns area samples instead of point samples Independent of resolution, combines with GPU, software

Aliasing: Can’t yet afford to subsample pixels Area elements: can inherently be antialiased on GPU

Beam Tracing: Cons and Contrib.Beam Tracing: Cons and Contrib.

Little work since HH84 (considered slow technique) Beam-Triangle intersection is slow vs ray-triangle No effective acceleration structures, fast traversals Often used in acoustics [Funkhouser 98,99]

Contributions Fast Beam-Triangle Intersections Fast Beam-Kd-tree traversal Includes modern ray tracing accelerations, SSE Application to primary, secondary visibility

Beam Tracing: In ActionBeam Tracing: In Action

OutlineOutline

Motivation

Beam Tracing Algorithm Beam-Triangle Intersection Beam-KD-tree Traversal

Analysis – Primary Visibility

Soft Shadows

Future Work

Beam-Triangle IntersectionBeam-Triangle Intersection

Beam intersects triangle and splits That part that hits, and that part that misses

High-Level Algorithmic Decisions Mirror ray-triangle intersection, diverging only if needed Parallelize through use of SIMD SSE Instructions Operate on (3 or 4) corners of beam as if single ray

Algorithm Overview Triangle projection (into 2D image plane) Handle trivial cases (no splitting) Beam splitting

Beam—Triangle IntersectionBeam—Triangle Intersection

Key to efficient intersection is separating trivial from non-trivial interactions.

Beam Triangle

Beam—Triangle IntersectionBeam—Triangle Intersection

Beam—Triangle Interaction Trivial

Triangle Edge Separator

Non-Trivial

Beam—Triangle IntersectionBeam—Triangle Intersection

Beam—Triangle Interaction Trivial

Triangle Edge Separator Beam Edge Separator

Non-Trivial

Beam—Triangle IntersectionBeam—Triangle Intersection

Beam—Triangle Interaction Trivial

Triangle Edge Separator Beam Edge Separator Beam Inside Triangle

Non-Trivial

Beam—Triangle IntersectionBeam—Triangle Intersection

Beam—Triangle Interaction Trivial

Triangle Edge Separator Beam Edge Separator Beam Inside Triangle

Non-Trivial Everything else

SPLIT the beam

Beam—Triangle IntersectionBeam—Triangle Intersection

Beam—Triangle Interaction Trivial

Triangle Edge Separator Beam Edge Separator Beam Inside Triangle

Non-Trivial Everything else

SPLIT the beam

Beam—Triangle: Split ProcedureBeam—Triangle: Split Procedure

For each edge Split beam into 2 sub-beams

Result is 2 sets of sub-beams Possible Hit beams Miss beams

Miss Beam

Possible Hit Beam

Beam—Triangle: Split ProcedureBeam—Triangle: Split Procedure

For each edge Split each potential hit beam into 2 sets of sub-beams

Result is 2 sets of sub-beams Potential Hit beams Miss beams

Beam—Triangle: Split ProcedureBeam—Triangle: Split Procedure

For each edge Split each potential hit beam into 2 sets of sub-beams

Result is 2 sets of sub-beams Potential Hit beams Miss beams

Beam—Triangle: Split ProcedureBeam—Triangle: Split Procedure

Sometimes an extra split is required to restrict beams to having only 3 or 4 corner rays.

Beam—Triangle: Split ProcedureBeam—Triangle: Split Procedure

Final result is 2 lists of beams Hit Beams (yellow) Miss Beams (red)

Beam-Triangle – Other splitsBeam-Triangle – Other splits

Other split cases handled in same way

In the paper we describe a fast new data parallel algorithm (using SIMD SSE instructions) for performing this beam—triangle clipping.

Beam-Triangle Intersect. – PerformanceBeam-Triangle Intersect. – Performance

15-40 FPS 5-10 FPS~800 Triangles ~2000 Triangles

OutlineOutline

Motivation

Beam Tracing Algorithm Beam-Triangle Intersection Beam-KD-tree Traversal

Analysis – Primary Visibility

Soft Shadows

Future Work

Rays vs. kd-tree -> Rays vs. SlabsRays vs. kd-tree -> Rays vs. Slabs

x

y

1x 2x

1y

2y

Ray – kd-tree traversal based on slabs

Rays vs. SlabsRays vs. Slabs

x

y

1x 2x

1y

2y

The current bounding box is the intersection of slabs

X slab

Y slab

Current Bounding Box

Rays vs. SlabsRays vs. Slabs

x

y

1x 2x

1y

2y

This ray intersects the current bounding box

Rays vs. SlabsRays vs. Slabs

x

y

1x 2x

1y

2y

We calculate the ray’s entry (tmin) and exit (tmax) distance for each slab

minxt

maxxt

Rays vs. SlabsRays vs. Slabs

x

y

1x 2x

1y

2y

We calculate the ray’s entry (tmin) and exit (tmax) distance for each slab

minxt

maxxt

minyt

maxyt

Rays vs. SlabsRays vs. Slabs

x

y

1x 2x

1y

2y

If the ray enters each slab before it exits any other slab, then it hits the bounding box

minxt

maxxt

minyt

maxyt

maxminyx tt maxminxy tt

Rays vs. SlabsRays vs. Slabs

x

y

1x 2x

1y

2y

How do we know that this ray misses the bounding box?

maxxt

minyt

maxyt

Rays vs. SlabsRays vs. Slabs

x

y

1x 2x

1y

2y

Because it exits the x slab before it enters the y slab

maxxt

minyt

maxyt

maxminxy tt

Rays vs. kd-treeRays vs. kd-tree

x

y

1x 2x

1y

2y

For a ray in a kd-tree, it is enough for a ray to keep track of where the ray enters the current bounding box (tmin) and where it exits (tmax)

minmin ttx

maxmax tt y

Beams vs. kd-treeBeams vs. kd-tree

How do we trace a beam through a kd-tree?

Beams vs. kd-treeBeams vs. kd-tree

How do we trace a beam through a kd-tree?

Kd-treeCurrent Bounding Box

Beams vs. kd-treeBeams vs. kd-tree

How do we trace a beam through a kd-tree?

Beam

Beams vs. kd-treeBeams vs. kd-tree

If we just use the tmins and tmaxes from the corner rays, the tmin/tmax points define a near and far plane for the active part of the beam. This works as long as all four corners take the same

path through the tree.

tmins

tmaxes

Beams vs. kd-treeBeams vs. kd-tree

But if the next split plane causes the beam’s rays to take different paths…

Next split plane

Beams vs. kd-treeBeams vs. kd-tree

…the new tmax value lies on a different kd plane from the other tmax values…

New tmax

Beams vs. kd-treeBeams vs. kd-tree

…and the new tmaxes define a new far plane…

Beams vs. kd-treeBeams vs. kd-tree

…which doesn’t fully represent the active portion of the beam.

Under-represented area

Beams vs. kd-treeBeams vs. kd-tree

This causes problems in 3D, when the corner of a split plane passes through a face of the beam. The beam on the right will only traverse the far side of

the split plane when both sides should be visited

Beams vs. kd-treeBeams vs. kd-tree

Another option…

Beams vs. kd-treeBeams vs. kd-tree

Split the beam where the split plane passes through the far plane of the active beam.

Beam split

Beams vs. kd-treeBeams vs. kd-tree

This assures that the tmins/tmaxes of all corner rays lie on the same kd-plane.

This works, but excessive splitting leads to slower render times.

Beams vs. kd-treeBeams vs. kd-tree

Instead we keep multiple tmin/tmax values, one for each dimension.

tmaxx

tmaxy

Beams vs. kd-treeBeams vs. kd-tree

The active part of the beam is the intersection of the beams defined by these planes.

Active part of beam

Beams vs. kd-treeBeams vs. kd-tree

To traverse the tree, we much check the tmins/tmaxes on all dimensions

Example: If all of the beam’s tmax values for the x-slab are less than the distance to the current split plane, only the near side of the plane needs to be traversed.

OutlineOutline

Motivation

Beam Tracing Algorithm Beam-Triangle Intersection Beam-KD-tree Traversal

Analysis – Primary Visibility

Soft Shadows

Future Work

Beam Tracing – Primary VisibilityBeam Tracing – Primary Visibility

Trace primary beams, obtain hit beams (areas)

Hit beams as quads to GPU for rasterization Area elements exactly represent visible surface of

scene (often much more compact than all triangles)

Compact area elements enable GPU per-pixel shaders, 6x antialiasing (ray tracing slows down)

Resolution independent (same time for 1024x1024 or higher as for 512x512)

Comparison SetupComparison Setup Average numbers over multiple views (all methods)

Beams vs Rays (our optimized) vs MLRT (Intel) Our ray tracer optimized, but one at a time (useful for

comparing statistics) MLRT uses one thread, with shading, display no quad

optimization. Diffuse shading, single source at viewer

kd-trees from MLRT (tuned to their performance)

1024x1024 resolution

Beams antialiased (for free). MLRT, rays not.

3.0 GHZ Pentium 4, 1.5GB RAM, ATI 9800 card

Beam Tracing – Primary VisibilityBeam Tracing – Primary Visibility

Simple Scene Erw6 Simple Scene Erw6

Beam Tracer: 169.49 FPS

MLRT: 12.99 FPS

816 Triangles (150 visible tris ; few hit beams)

Beams Rays

Kd steps

per pixel

0.0085 13.86

Intersect.

per pixel

0.0033 6.81

Hits per

Pixel

0.0010 0.95

Frame Rate

169.49 fps

0.99 fps

Complex Scene: Soda Hall Complex Scene: Soda Hall

Beam Tracer: 21.28 FPS

MLRT: 5.56 FPS

Z-buffer (GPU): 13-15 FPS

2M Triangles (1500 visible tris)

Beams Rays

Kd steps

per pixel

0.13 42.36

Intersect.

per pixel

0.064 4.29

Hits per

Pixel

0.0097 0.98

Frame Rate

21.28 fps

0.52 fps

Armadillo (subpixel tris; worst case)Armadillo (subpixel tris; worst case)

Beam Tracer: 1.72 FPS

MLRT: 5.99 FPS

345K Triangles (26000 visible tris)

Beams Rays

Kd steps

per pixel

1.07 46.46

Intersect.

per pixel

0.60 5.47

Hits per

Pixel

0.14 0.99

Frame Rate

1.72 fps

0.44 fps

Number of Hit Beams: Beam SplittingNumber of Hit Beams: Beam Splitting

# Hit Beams ~= 6 x # Visible Triangles

Beam Tracing: Hard ShadowsBeam Tracing: Hard Shadows

Qualitative benefits over ray tracing: Large coherent regions for secondary visibility

Beam Analysis – Primary RaysBeam Analysis – Primary Rays

Qualitative benefits over ray tracing: Large coherent regions for secondary beams High quality anti-aliased primary visibility AND

secondary visibility

Beam Tracer anti-aliased

MLRT (ray tracer) without anti-aliasing

Video 1Video 1

OutlineOutline

Motivation

Beam Tracing Algorithm Beam-Triangle Intersection Beam-KD-tree Traversal

Analysis – Primary Visibility

Soft Shadows

Future Work

Soft ShadowsSoft Shadows

Connect pixel with beam to area light vertices

Only need total area coverage on light source

Suitability for Soft ShadowsSuitability for Soft Shadows

Range where beam tracing is most useful

Typically less than 10 visible tris per pixel

Observe that hit beams 2x number visible triangles (not 6x as in primary visibility)

10x – 40x faster than ray tracing

Can be much faster than soft shadow volumes [Laine et al. 2005, Lehtinen et al. 2006]

Soft Shadows – Plant (512x512)Soft Shadows – Plant (512x512)

Only 5245 triangles, but high shadow complexity

Almost every edge a silhouette edge

Ray Tracing: Comparable Time Beam Tracing: ExactRay Tracing: Comparable Quality (256 shadow rays)

5 seconds / 9 shadow rays 5 seconds 108 seconds

Soft Shadows – Soda (512x512)Soft Shadows – Soda (512x512)

Ray Tracing: Comparable Time Beam Tracing: ExactRay Tracing: Comparable Quality (256 shadow rays)

1.84 seconds / 4 shadow rays 1.78 seconds 79 seconds

Video 2Video 2

SummarySummary

New algorithms for real-time beam tracing

New method for fast soft shadows

Beam tracing has many benefits over ray tracing Can be significantly faster when required sample

density is higher than the required geometric fidelity Area Samples vs. Point Samples Antialiasing, shading Perfect soft shadows and other secondary effects

Beam tracing old technique, assumed slow No longer true, viable method for real-time, shadows

Future WorkFuture Work

Area to area beams for faster soft shadows

Fall back on point samples when geometric fidelity is too high (tris smaller than pixel)

Multiresolution representations (aka Razor)

Explore other visual effects Specular to diffuse interactions (Caustics) Glossy reflections

The FutureThe Future

What is the future rendering algorithm?

Hybrid Beam Tracing/GPU ?

Hybrid Beam Tracing/Ray Tracing?

How does it depend on visual effects needed?

On future architectures?

Research ProjectsResearch Projects

High quality real-time rendering Real-time ray tracing All-frequency interactive relighting Volumetric scattering in mist, fog, rain

Data-driven appearance acquisition, rendering

Complex lighting, materials in computer vision

Mathematical foundations of appearance

The EndThe End