Multi-Layered ImpostorsMulti-Layered Impostorsforfor

Accelerated RenderingAccelerated Rendering

Xavier Decoret, iMAGIS

This is joint work withGernot Schaufler and Julie Dorsey at MIT

and François Sillion at iMAGIS

Multi layered impostors for accelerated rendering

Complex EnvironmentsComplex Environments

Paris:

• 411537 vertices

• 137179 triangles

• 32 textures(most 256x256)

• 6.1 MB geometry

Multi layered impostors for accelerated rendering

IBR in RT-GraphicsIBR in RT-Graphics

• Image-based representations simplify the scene

• Images are unaffected by the depicted scene’s

complexity

• Images are fast to render as textured triangles

• Images themselves can be generated with hardware

• This works if a geometric model is available

Multi layered impostors for accelerated rendering

Previous WorkPrevious Work

• Pre-generated RepresentationsGrossman98, Dally96, Maciel95, Aliaga96, Chen95, McMillan95,

Xiong96, Sillion97, Darsa98, Pulli97, Laveau94, Max96,Rafferty98

• Dynamically Updated RepresentationsTorborg96, SGI97, Lengyel97, Regan94,Shade96, Schaufler96, Mark97, Mann97Aliaga99

Multi layered impostors for accelerated rendering

ImpostorsImpostors (pre-generated) (pre-generated)

SGI Performer(Billboards)

Id Software(3D Sprites)

Maciel95

Multi layered impostors for accelerated rendering

Impostors Impostors (dynamically generated)(dynamically generated)

Schaufler95

SchauflerStürzlinger‘96

Multi layered impostors for accelerated rendering

Meshed ImpostorsMeshed Impostors [Sillion97][Sillion97]

Distant Geometry Depth Map

Discontinuities Triangulation

Multi layered impostors for accelerated rendering

Artefacts in IBRArtefacts in IBR

• Deformation caused by mesh

• Resolution mismatch

Geometry

Multi layered impostors for accelerated rendering

Artefacts in IBRArtefacts in IBR

• Incomplete representation

• Rubber sheet effects

Geometry

?

Multi layered impostors for accelerated rendering

Artefacts in IBRArtefacts in IBR

• Image cracks

• and more …– Static Shading:

• no highlights

• no reflections

– No moving Objects

Geometry

Multi layered impostors for accelerated rendering

Our ContributionsOur Contributions

• Reducing Rubber Sheet Triangles

– Multi Mesh Impostors

• Reducing distortions to improve quality

– Dynamic Update

Reduce some of the identified artefacts

Multi layered impostors for accelerated rendering

The databaseThe database

• The geometry is organized into objects

• We have a set of viewcells

The street graphO

ne e

dge

Another edge

Multi layered impostors for accelerated rendering

Model segmentationModel segmentation

Complete Geometry PVS

Local model

Distant model

Multi layered impostors for accelerated rendering

Single mesh impostorSingle mesh impostor

Distant model......replaced by impostor

Combined withlocal model

Multi layered impostors for accelerated rendering

Rubber Sheet Triangles Rubber Sheet Triangles due to parallaxdue to parallax

impostor

viewpoint

View from

above

Multi layered impostors for accelerated rendering

Locating Rubber Sheet TrianglesLocating Rubber Sheet Triangles• Parallax creates rubber sheets between objects

when objects overlap in depth

View from

above

Frontview

• In urban walkthrough, parallax is mainly horizontal

Assume a 2D problem

Multi layered impostors for accelerated rendering

Multi Mesh ImpostorMulti Mesh Impostor

• The critical zone identifies overlaps between 2 objects

• When overlapping occurs in image space, one object can uncover the other one

Edge

O1

Critical zone

O2

Edge

O1

O2

• If uncovering is too much, objects must not be on the same mesh

Multi layered impostors for accelerated rendering

Multi Mesh ImpostorMulti Mesh Impostor

• Quantifying overlapping between 2 objects• Place those objects in different layers which are

too distant in depth:– construct a relation graph– partition the graph

Single mesh Several meshes

Multi layered impostors for accelerated rendering

Objects and LayersObjects and Layers

• Two objects must go into different layers if two points on their geometry can be seen under sufficiently different viewing angles min and max.

Q

P

A

B

max

edge

min

Object 1

Object 2

y

P Q

A

B

x

M1, M2 [PQ] such as

AM1B minimal

AM2B maximal

Multi layered impostors for accelerated rendering

Relation GraphRelation Graph

• Edges joining overlapping objects

• Coloriate the graph so that joined vertices have different colors

• Each color represent a layer• Non unique

• One node per object

Multi layered impostors for accelerated rendering

All Geometry

Results & ExampleResults & Example

Edge

PVS

SingleMesh

MultiMesh

Multi layered impostors for accelerated rendering

Results & ExamplesResults & Examples

Start pointFrom which the impostor

is computed

End pointNotice rubber sheet triangles

Multi layered impostors for accelerated rendering

Results & ExamplesResults & Examples

Start pointFrom which the impostor

is computed

End pointThe view is no longer blocked

Multi layered impostors for accelerated rendering

Results & ExamplesResults & ExamplesGeometry

Single Mesh MultiMesh

Multi layered impostors for accelerated rendering

Offline vs OnlineOffline vs Online

• If the user stops, the correct image should appear.

• This requires knowledge of the current viewpoint.

• Offline approaches do not have this information

• Motion hides small artifacts of stored impostors.

Multi layered impostors for accelerated rendering

Combining preprocessingCombining preprocessingwith dynamic updateswith dynamic updates

• Single mesh impostor replace too much geometry to be updated.• Layers and their contents are suitable for regeneration of part of

the distant model.• Layers are updated front to back

to improve image quality:

– silhouettes

– distortions

– resolution mismatches

Multi layered impostors for accelerated rendering

System ArchitectureSystem Architecture

Preprocessing:• Take geometry and view cells• Find visible geometry for cells• Split into near and far part• Create impostors for far part• Store as scene per view cell

Walkthrough:• Page in geometry and textures• Do dynamic updates if possible• LOD management

Minimal Scenegraph per viewcell

Visibility Extractor

Model Segmentation

MMI ExtractorOffl

ine

View Cells Scene

Dynamic Update

On

line

Rendering

Multi layered impostors for accelerated rendering

Results & Video (Paris)Results & Video (Paris)

Storage requirements :70 Mbs

Computation time : 100 edges per hour

Achieved frame rate: 50 Hz

Multi layered impostors for accelerated rendering

Future WorkFuture Work

• Smooth transitions between different representations

• Automatic generation of street graph from street mesh

• Extension of viewcells from edges to areas and volumes

• Reverse approach: construction of viewcells to optimize use of impostors.

Multi layered impostors for accelerated rendering

AcknowledgementsAcknowledgements

This is a joint project between iMAGIS, Grenoble and MIT, Cambridge.

It was supported in part by a joint collaborative research grant of NSF and INRIA (INT-9724005), an Alfred P. Sloan Foundation Research Fellowship (BR-3659), and by a grant from Intel Corporation.

The following people contributed to the presented results :

Julie Dorsey, François Sillion, Gernot Schaufler, Max Chen, Byong Oh Mok, Yann Argotti and Sami Shalabi.