synthèse d'images animées 1 - grenoble inp - chamilo · example scan (stereo reconstruction)...

28
Advanced Geometry

Upload: others

Post on 10-Jul-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Advanced Geometry

Page 2: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Topics

1. Automatic content creation

2. Level-of-detail representations

2

[Liu et al. Eurographics 2015]

Page 3: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Content creation

3

Page 4: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Traditional manual approach

• Use of 3D modeling software

• Manually define geometry and materials

• Compose model of multiple primitives

• Requires expertise and time, expensive

4

Example image of modeled fish from Wikipedia.

Page 5: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Automatic content creation

Main idea

•Generate 3D models without manual modeling

• Two options

1. Measuring real-world objects

2. Modifying existing 3D models

5

Page 6: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Measuring real-world objects

6

Page 7: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Traditional Acquisition Techniques• Small set of discrete

measurements• Used in different applications

Anthropometric measurements Surveying

• Main advantage Easy to acquire and process

measurements

• Main disadvantage Impossible to get a detailed

shape description

Page 8: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

3D Scanners• New technology

• 3D (animation) scanners

• Record 3D video

• Active research area

• Powerful tool

• Preserve artwork / historic artifacts

• Acquire populations of 3D shapes for analysis

[P. Jenke, WSI/GRIS Tübingen]

Page 9: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Types of 3D Scanners

Scanning Techniques:o Time-of-flight

Time-of-flight laser scanner

Time-of-flight depth cameras (dynamic)

o Triangulation Laser line sweep

Structured light

o Stereo / computer vision Passive stereo

Active stereo / space time stereo

Other techniques

Page 10: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Example Scan (time of flight laser scanner)

[data set: University of Hannover]

Page 11: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Example scan (structured light scanner)

color-codedstructured light

courtesy of Phil Fong,Stanford University

motion compensatedstructured light

courtesy of Sören König,TU Dresden

Page 12: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Example scan (active stereo scanner)

Page 13: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Example scan (stereo reconstruction)

multi view matching (6 cameras)(photo-consistent space carving)

[Data set: Christan Theobald, MPII, 2006]

multi view matching (8 cameras)(piecewise smooth variational surfaceon presegmented imagessolved with Bayesian belief propagation)

[Data set: Zitnick et al.,Microsoft Research, Siggraph 2004]

Page 14: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Processing the scans

To be useful, the scans need to be processed

oHole filling

oOutlier removal

o…

oMany methods and software libraries available

14

Page 15: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Allows for direct content creation

15

Page 16: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Modifying existing 3D models

16

Page 17: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Possibilities

• Common possibilities

o Extrapolation

o Interpolation

o Structure-aware modification

• Many possibilities, active area of research

17

Page 18: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Exercise

Propose a way to interpolate and extrapolate between two 3D shapes

• How do you represent the shapes?

• What interpolation / extrapolation equation to use?

• What are the advantages and disadvantages of the method?

18

Page 19: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Shape interpolation and extrapolation

• Possible in shape space using correspondence information

19

Page 20: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Statistical shape spaces possible

• Learn statistical distribution of geometry of shape

• Use this information for synthesis

20

Page 21: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Structure-aware modification

ChallengeoDirect point-to-point correspondence cannot be established

oPoint-wise or triangle-wise modification not meaningful

oActive area of research

21

Liu et al. Eurographics 2015

Page 22: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Take advantage of symmetry information

22

Kalojanov et al. SGP 2012

Page 23: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Decompose into basic building blocks

23

Liu et al. Eurographics 2015

Page 24: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Level-of-detail representations

24

Page 25: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Key idea

25

Distance from camera = less geometric detail required

Page 26: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Subdivision surfaces

• We saw them already

• AdvantagesoVery strong geometric compression (4 triangles become 1)

o Theoretical convergence properties

• DisadvantagesoNot applicable to downsample shapes that do not have this structure

26

Page 27: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Edge collapse

• Models can be downsampled by sequence of edge collapses

27

[Hoppe, Progressive Meshes, SIGGRAPH 1996]

Page 28: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan

Example result

28

[Hoppe, Progressive Meshes, SIGGRAPH 1996]