rod-tv : surface reconstruction on demand by tensor voting page 2 outline 1. introduction and...
Post on 21-Dec-2015
217 views
TRANSCRIPT
![Page 1: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/1.jpg)
![Page 2: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/2.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 2
Outline
1. Introduction and motivation2. Rod-tv framework overview3. Rod-tv modules
3a. Multi-scale data hierarchy3b. Levels-of-detail control3c. Visibility culling3d . Tensor voting
4. Results5. Conclusions and future work
![Page 3: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/3.jpg)
Outline
1. Introduction and motivation2. Rod-tv framework overview3. Rod-tv modules
3a . Multi-scale data hierarchy3b . Levels-of-detail3c. Visibility culling3d . Tensor voting
4. Results5. Conclusions and future work
![Page 4: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/4.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 4
1. Introduction and motivation
Problem statement
ỜDigital models are demanding nowadays in some industries, for instance, 3D movie industry, 3D game industry, etc.
ỜModels can be acquired by using 3D scanning devices
> Bunny dataset – 35,948 points (scanned by Stanford University)
![Page 5: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/5.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 5
1. Introduction and motivation
Problem statement
Ờ Usually, digital models are represented as a set of 3D isolated points, i.e. (xi, yi, zi)
Ờ 3D scanned models are huge and noisy- Huge : real, detailed, accurate 3D models
- Noisy : measurement errors in the scanning process
ỜOur concern: How to render these 3D models efficiently?- [Approach 1] point-rendering in computer graphics- [Approach 2] surface reconstruction in computer vision
![Page 6: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/6.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 6
1. Introduction and motivation
Approach 1: Point-rendering
Ờ A point-rendering is a graphical technique which uses simple geometrical shapes as rendering primitives
Ờ Example- QSplat [Rusinklewicz and Levoy in 2000]
![Page 7: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/7.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 7
1. Introduction and motivation
Point-rendering in QSplat
bunny dataset - 35,948 points
point-rendering – using sphere as a rendering primitive
[high-resolution] [low-resolution]
[close-up view]
![Page 8: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/8.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 8
1. Introduction and motivation
Point-rendering in QSplat
Ờ In QSplat, the input 3D points are organized as a hierarchy
ỜQSplat uses above hierarchy structure for view-dependent, levels-of-detail rendering
> A spectrum of data resolution
decreasing data resolution
![Page 9: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/9.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 9
1. Introduction and motivation
Problems in point-rendering
bunny [zoom-in] - 8,365 pts
point-rendering [zoom-in]
Ờ Problems- upper-bound by the scanning resolution- can handle noise-free 3D point input only
![Page 10: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/10.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 10
1. Introduction and motivation
Summary of point-rendering
Ờ Advantages- Fast and costless in rendering
[Fast : hierarchy for view-dependent and levels-of-detail control][Costless : simple geometrical shapes]
Ờ Limitations- Upper-bound by scanning resolution- Noise-sensitive application system
Ờ Comments- We need:
- A flat and smooth surface in any data resolution- A noise-robust application system
![Page 11: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/11.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 11
1. Introduction and motivation
Approach 2: Surface reconstructionỜ A surface reconstruction is a vision technique
which uses polygonal patches to represent a surface
Ờ Example- Tensor voting [Medioni, Mi-Suen Li and Chi-Keung Tang in 2000]
![Page 12: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/12.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 12
1. introduction and motivation
Tensor voting ~ Flat and smooth surfaceỜ Tensor voting can generate a smooth and flat
surface at any data scanning resolution
![Page 13: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/13.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 13
1. Introduction and motivation
Tensor voting ~ Noise-robustnessỜ Tensor voting can handle noisy 3D point as an
input
![Page 14: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/14.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 14
1. Introduction and motivation
Problems in tensor voting
Ờ Problems - surface reconstructing process is usually costly and can be regarded as two reasons:
1. Lack of levels-of-detail control
- Surface is reconstructed in single data resolution only without consider the viewer position
2. Lack of visibility control
- Input 3D Points are fully processed whether they are visible or not given a particular viewing volume
![Page 15: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/15.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 15
1. Introduction and motivation
Summary of tensor voting
Ờ Advantages- Flat and smooth surface is generated in any data resolution- Noise-insensitive reconstruction system
Ờ Limitations- Lack of levels-of-detail control- Lack of visibility control
Ờ Comments- We need :
- View-dependent and levels-of-detail surface reconstruction system
![Page 16: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/16.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 16
1. Introduction and motivation
Our contributions: Rod-tv
ỜOur approachsurface reconstruction on demand
by tensor voting- Abbreviate to rod-tv
- A “graphics for visions” approach
ỜGeneral idea- Defer vision surface reconstruction
[tensor voting]
- Use graphical techniques to query a better visible subset to meet the “on demand” requirement
[levels-of-detail control and visibility culling]
![Page 17: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/17.jpg)
Outline
1. Introduction and motivation2. Rod-tv framework overview3. Rod-tv modules
3a Multi-scale dataset3b Levels-of-detail3c Visibility culling3d Tensor voting
4. Results5. Conclusions and future work
![Page 18: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/18.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 18
2. Rod-tv framework overview
input: - 3D isolated points
1. generate a spectrum of data resolution
2. desire an appropriate data resolution at given particular viewing position
3. cull all the invisible parts
4. reconstruct surface on visible subset
output: on demand surface
![Page 19: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/19.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 19
Outline
1. Introduction and motivation2. Rod-tv framework overview3. Rod-tv modules
- 3a. Multi-scale data hierarchy- 3b. Levels-of-detail control- 3c. Visibility culling- 3d. Tensor voting
4. Results5. Conclusions and future work
![Page 20: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/20.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 20
3. Rod-tv modules
Module 1: Multi-scale data hierarchyỜModule overview
- Input : a 3D dataset in single-resolution- Ambition : use a single data structure to generate a spectrum of dataset resolution
![Page 21: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/21.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 21
3. Rod-tv modules
Module 1: Multi-scale data hierarchyỜModule description
- Make use of hierarchical spatial data structure~ Grid Pyramid~ Octree
- Usually it is organized as tree- Resolutions are encoded inside tree internal nodes- Different height generated different resolutions
ỜModule implementation- Component 1: Grid Pyramid- Component 2: Octree
[both of them are spatial volume hierarchies]
![Page 22: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/22.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 22
3. Rod-tv module 1 – multi-scale data hierarchy
Component 1: Grid Pyramid
Ờ A Grid Pyramid is:- A hierarchy of volume [3D array]
- Volume resolution decrease from the bottom of the pyramid to the top
A. lo-resolution – the entire scene is represented by 1 grid cell
B. mi-resolution – the entire scene is represented by 8 grid cells
C. hi-resolution – the entire scene is represented by 64 grid cells
D. the corresponding tree representation of the Grid Pyramid
![Page 23: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/23.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 23
3. rod-tv modules – multi-scale dataset
component 1: grid pyramid
Top-down approachthe construction proceedsdownwards by iteratively subdividing current cells into 8 equal cells in 3-D case and 4 equal cells in 2-D case
Bottom-up approachthe construction proceedsupwards by iteratively consolidating a group of 8 cells in 3-D case and a group of 4 cells in 2-D case
A. input 3D points (12 pts)
B. Grid Pyramid – level 0cell size = 1number of points = 11
C. Grid Pyramid – level 1cell size = 2number of points = 8
D. Grid Pyramid – level 2cell size = 4number of points = 3
E. Grid Pyramid – level 3cell size = 8number of points = 1
F. the corresponding tree representation of the Grid Pyramid
![Page 24: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/24.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 24
3. Rod-tv module 1 – multi-scale data hierarchy
Component 1: Grid Pyramid
Ờ Advantages- Multi-scale representation capability
- Grid cell elements can be accessed in constant time
Ờ Limitations- Memory is wasted on tessellating a large group of empty grid cells- Total number of resolutions are limited
![Page 25: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/25.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 25
Ờ An Octree is:- Subset of Grid Pyramid- Adaptive cell tessellation
~ large octant is used to represent low density region
~ small octant is used to represent high density region
3. Rod-tv module 1 – multi-scale data hierarchy
Component 2: Octree
A. Octree example in 3-D space
B. the corresponding tree representation of the Grid Pyramid
![Page 26: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/26.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 26
Octree building rulesSubdivision stops if1) number of points in each cell less than certain threshold2) tree height is higher than certain threshold
A. input 3D points (12 pts)
B. Octree – level 3number of cells = 1number of points = 1
C. Octree – level 2number of cells = 4 number of points = 3
D. Octree – level 1number of cells = 13number of points = 8
E. Octree – level 0Number of cells = 19number of points = 11
F. the corresponding tree representation of the Grid Pyramid
![Page 27: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/27.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 27
3. Rod-tv module 1 – multi-scale data hierarchy
Component 2: Octree
Ờ Advantages- Memory is saved using adaptive cell tessellation
Ờ Limitations- Points are too close: unlimited subdivision- Octree is not balanced
![Page 28: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/28.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 28
3. Rod-tv modules
Summary of multi-scale data hierarchyỜ Comments
- Data structure alternative tips:~ Large memory : Grid Pyramid~ Limited memory : Octree
- Multi-scale dataset to:~ Facilitate levels-of-detail [module 2]~ Facilitate visibility culling [module 3]
![Page 29: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/29.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 29
Outline
1. Introduction and motivation2. Rod-tv framework overview3. Rod-tv modules
- 3a. Multi-scale dataset- 3b. Levels-of-detail control- 3c. Visibility culling- 3d. Tensor voting
4. Results5. Conclusions and future work
![Page 30: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/30.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 30
3. Rod-tv modules
Module 2: Levels-of-detail (on demand)
ỜModule overview- Input : a multi-scale dataset and a viewing position - Ambition : select an appropriate resolution dataset
line-of-sight distance
Closer to the viewer[need more points to represent]
Farther to the viewer[need less points to represent]
![Page 31: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/31.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 31
3. Rod-tv modules description
Module 2: Levels-of-detail (on demand)
ỜModule description- Make use of the hierarchy we build in module 1- Traverse the hierarchy from coarse-to-fine resolution- In other words, we traverse the tree from root-to-leaf order- Then, selection criteria is made on each internal node and finally appropriate data resolution is returned
~ selection criteria can be made either in object-space or image-space
ỜModule implementation- Component 1: Range based [object-space]
- Component 2: Screen based [image-space]
![Page 32: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/32.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 32
3. Rod-tv module 2 – levels-of-detail (on demand)
Component 1: range-based methodỜ Levels-of-detail selection criteria:
- Separate the 3-D space into different ranges {d1,d2,d3}
- Selection is made based on the viewer-object distance
d2d3
line-of-sight distance
d1
Decision rule
If distance < d1 then LOD 0 is returned
elseif d1 < distance < d2 then LOD 1 is returned
elseif d2 < distance < d3 then LOD 2 is returned
else LOD 3 is returned
endif
[LOD 0] [LOD 1] [LOD 2] [LOD 3]
![Page 33: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/33.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 33
step 1. define rangesd1, d2, d3 and d4
step 2. compute the viewer-object distance
step 3.start traverse from the root level
step 4.check whether the criteria is met or not
If yes, then stopElse, then continue traversing next lower level and repeat step 4
d1
d2
d3 d4
viewer-object distance
line-of-sight distanceviewer
object
d1
d3
d4
LOD 3
LOD 2
LOD 1
LOD 0
LOD 0LOD 1LOD 2LOD 3
d2
![Page 34: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/34.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 34
step 1. define rangesd1, d2, d3 and d4
step 2. compute the viewer-object distance
step 3.start traverse from the root level
step 4.check whether the criteria is met or not
If yes, then stopElse, then continue traversing next lower level and repeat step 4
d1
d2
d3 d4
viewer-object distance
line-of-sight distanceviewer
object
d1
d3
d4
LOD 3
LOD 2
LOD 1
LOD 0
LOD 0LOD 1LOD 2LOD 3
d2
![Page 35: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/35.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 35
3. Rod-tv module 2 – levels-of-detail (on demand)
Component 2: screen-based methodỜ Levels-of-detail selection criteria:
- Project the cell on the image plane- Use sphere instead of octant/cube to avoid alignment- Selection is made based on the number of pixel covered on the image plane
![Page 36: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/36.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 36
step 1. Define a threshold t for pixel coverage
step 2. Start traversing from the root of the tree
step 3.Computer the cell projection s
step 4.check whether the criteria is met or not
If s < t then stopElse, then continue traversing next lower level and repeat step 3
line-of-sight distanceviewer
object
LOD 3
LOD 2
LOD 1
LOD 0
LOD 0LOD 1LOD 2LOD 3
image-plane
1
4
12
24
![Page 37: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/37.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 37
3. Rod-tv modules
Summary of levels-of-detail (on demand)
Ờ Advantages- Range-based method : easy to implement- Screen-based method : a single threshold is used
Ờ Limitations- Range-based method : hard to set a range-thresholds
Ờ Comments- Decision is made on-the-fly (demand)- An appropriate resolution dataset is generated after running this module, therefore “on demand” processing is achieved
![Page 38: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/38.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 38
Outline
1. Introduction and motivation2. Rod-tv framework overview3. Rod-tv modules
- 3a. Multi-scale dataset- 3b. Levels-of-detail control- 3c. Visibility culling- 3d. Tensor voting
4. Results5. Conclusions and future work
![Page 39: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/39.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 39
3. Rod-tv modules
Module 3: Visibility culling
ỜModule overview- Input : an appropriate resolution dataset- Ambition : cull all the invisible parts
ỜModule description- Cull all the tokens which are outside viewing frustum- Cull all the tokens which are occluded by the frontiers
ỜModule implementation- Component 1: View frustum culling - Component 2: Occlusion culling
![Page 40: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/40.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 40
3. Rod-tv module 3 – visibility culling
Component 1: View frustum culling - Find out a visible subset that is inside the frustum- A token-frustum intersection test- Usually, a frustum consists of six clipping planes, namely,left, right, top, bottom, near and far.
![Page 41: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/41.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 41
3. Rod-tv module 3 – visibility culling
Component 1: View frustum culling
- A token is inside the view frustum if d >= 0 in all clipping planes
a x+b y+c z+d=0
TTTTTTTTTTTTTT
tokend
D = T Na,b,c
- Plane equation :
token token token tokenT = x ,y ,zTTTTTTTTTTTTTT
- Token position :
- Token-plane intersection test :
D > 0 – in front of the planeD = 0 – on the planeD < 0 – behind the plane
![Page 42: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/42.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 42
- Find out a better visible subset that is inside the viewing frustum but not visible in the final result
- In other words, we cull all the tokens which are occluded by the frontier
3. Rod-tv module 3 – visibility culling
Component 2: Occlusion culling
![Page 43: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/43.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 43
3. Rod-tv module 3 – visibility culling
Component 2: Occlusion culling
- Project token on the image plane and decision is made based on the projection and the depth information
- Make use of occlusion map (o-map) and depth map (d-map)- o-map: potential occluder are recorded and corresponding covering pixels are marked- d-map: record projected token depth information
image plane
occlusion map (o-map)
depth map (d-map)
![Page 44: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/44.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 44
Depth test
If the token projection overlaps in the o-map, then we compare the token depth in the d-map. We keep the one which is closer to the viewer and cull the other one.
![Page 45: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/45.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 45
3. Rod-tv module 3
Summary of visibility culling
Ờ Advantages- No CPU time is wasted on invisible parts:
~ Outside the viewing volume~ Occluded by the frontier~ Less than certain pixel on the image plane
Ờ Limitations- Hard to produce extract occluder set
Ờ Comments- A better visible subset dataset is produced after running this module, therefore “on demand” processing, once again, is achieved
![Page 46: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/46.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 46
Outline
1. Introduction and motivation2. Rod-tv framework overview3. Rod-tv modules
- 3a. Multi-scale dataset- 3b. Levels-of-detail control- 3c. Visibility culling- 3d. Tensor voting
4. Results5. Conclusions and future work
![Page 47: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/47.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 47
3. Rod-tv modules
Module 4: Tensor voting
ỜModule overview
~ [module 2] range/screen levels-of-detail selection~ [module 3] view frustum and occlusion culling
- Input : a “on demand” visible point set- Ambition : reconstruct “on demand” surfaces
![Page 48: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/48.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 48
3. Rod-tv modules
Module 4: Tensor voting
ỜModule description:- Make use of the original tensor voting algorithm- use tensor as data representation- use voting as data communication- consists of four stages:
~ stage 1: information encoding~ stage 2: sparse tensor voting~ stage 3: dense tensor voting~ stage 4: feature extraction
![Page 49: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/49.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 49
3. Rod-tv module 4 – tensor voting
Data representation
- Tensor voting uses a 2nd order symmetric tensor as a data representation
- Symmetric tensor can be visualized as an ellipsoid- Mathematically, we can use eigensystem to represent the ellipsoid
maxmax
max mid min midmid
min min
eλ 0 0
S= e e e 0 λ 0 e
0 0 λ e
T T T
max max mid mid min minmax mid minS=λ e e λ e e λ e e
![Page 50: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/50.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 50
3. Rod-tv module 4 – tensor voting
Data representation
symmetric max mid stick mid min plate min ball
Tmax maxstick
T Tmax max mid midplate
T T Tmax max mid mid min minball
T = λ - λ T λ - λ T λ T
T e e
T e e e e
T e e e e e e
- After reorganizing the eigensystem, the ellipsoid can be decomposed into stick , plate and ball tensor
[stick tensor] [plate tensor] [ball tensor]
![Page 51: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/51.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 51
3. Rod-tv module 4 – tensor voting
Data communication
- Tensor voting uses voting algorithm as a data communication- Voter site propagates data to its neighborhood by using predefined voting kernel- Votes then are then collected and accumulated in the votee site
voter site p voter site r
votee site q
![Page 52: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/52.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 52
3. rod-tv module 4 – tensor voting
tensor voting system
![Page 53: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/53.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 53
3. Rod-tv module 4 – tensor voting
Stage 1: information encoding
![Page 54: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/54.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 54
3. Rod-tv module 4 – tensor voting
Stage 2: sparse tensor voting
![Page 55: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/55.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 55
3. Rod-tv module 4 – tensor voting
Stage 3: dense tensor voting
![Page 56: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/56.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 56
3. Rod-tv module 4 – tensor voting
Stage 4: feature extraction
![Page 57: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/57.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 57
Outline
1. Introduction and motivation2. Rod-tv framework overview3. Rod-tv modules
- 3a. Multi-scale dataset- 3b. Levels-of-detail control- 3c. Visibility culling- 3d. Tensor voting
4. Results5. Conclusions and future work
![Page 58: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/58.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 58
4. Results
Surface reconstruction (on demand) by tensor voting
![Page 59: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/59.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 59
4. Results
Surface reconstruction (on demand) by tensor voting
![Page 60: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/60.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 60
4. Results
Surface reconstruction (on demand) by tensor voting
![Page 61: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/61.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 61
4. Results
Surface reconstruction (on demand) by tensor voting
![Page 62: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/62.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 62
4. Results
Surface reconstruction (on demand) by tensor voting
![Page 63: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/63.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 63
4. Results
Surface reconstruction (on demand) by tensor voting
![Page 64: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/64.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 64
4. Results
Surface reconstruction (on demand) by tensor voting
![Page 65: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/65.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 65
5. Conclusions and future work
Conclusions
Ờ Introduce levels-of-detail surface reconstruction for dense and noise 3D points
- [module 1] exploit the Grid Pyramid or Octree data structure for multi-scale data representation
- [module 2] use range-based or screen-based method for levels-of-detail “on demand” selection
- [module 3] apply culling algorithm for finding better “on demand” visible subset
- [module 4] reconstruct surface by means of tensor voting
Ờ Implementation Issue
- nearly real time and “on demand” surface can be generated on-the-fly (Tensor voting is the bottleneck)
![Page 66: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/66.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 66
5. Conclusions and future work
Future work
Ờ System optimization- Optimized neighborhood searching for tensorial support
- Octree parallel data structure- Compact hierarchical data structure, such as wavelet- Sub-pixel technique in both occlusion map and depth map- Viewer caching for dynamic update surface
![Page 67: rod-tv : surface reconstruction on demand by tensor voting page 2 Outline 1. Introduction and motivation 2. Rod-tv framework overview 3. Rod-tv modules](https://reader036.vdocument.in/reader036/viewer/2022062516/56649d6d5503460f94a4cc81/html5/thumbnails/67.jpg)
rod-tv : surface reconstruction on demand by tensor voting
page 67
Question and Answer