verifying the “consistency” of shading patterns and 3-d structures pawan sinha & edward...
TRANSCRIPT
![Page 1: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/1.jpg)
Verifying the “Consistency” of Shading Patterns and 3-D
Structures
Pawan Sinha & Edward Adelson
![Page 2: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/2.jpg)
What is the 3D Structure of Each Image?
![Page 3: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/3.jpg)
Infinite Number of Possible Interpretations
![Page 4: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/4.jpg)
Goal 1: 3D Shape Recovery
![Page 5: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/5.jpg)
Goal 2: Check Shading Consistency
![Page 6: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/6.jpg)
Input Patterns
![Page 7: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/7.jpg)
Goal 1: Propose 3D Structure
● Questions:– What distinguishes correct structure?– How to search for it algorithmically?
![Page 8: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/8.jpg)
What Distinguishes Correct Structure?
Wrong Right
![Page 9: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/9.jpg)
What Distinguishes Correct Structure?
● Low “Complexity”– Low Angle Variance– Planarity of Faces– Overall Compactness
![Page 10: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/10.jpg)
How to Search?
● Minimize Cost Function– How to weight constraints?– Cumbersome
● Incremental Solution– Start with 2-D Line Drawing– “Pull” vertices until regularity is
maximized● i.e. Gradient descent in regularity space
![Page 11: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/11.jpg)
Incremental Solution
![Page 12: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/12.jpg)
Goal 2: Check Shading Consistency
● Given– 3D structure– 2D gray-level image
● Assume– Structure is uniformly colored
● Find– Single light source to account for shading
![Page 13: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/13.jpg)
Quantitative Approach
● Given:– Lambertian Reflectance Model– Surface Normal– Surface Brightness
● Defines:– A cone of valid light directions for each
surface
![Page 14: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/14.jpg)
Cone of valid light direction
N=surface normalE=brightnessarccos E
N
![Page 15: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/15.jpg)
Quantitative Approach
● Consider cones for all surfaces● Intersection is direction of illumination
![Page 16: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/16.jpg)
Problems
● Small changes in grey lead to no solution
Surface l3 changes brightness
Gradient SpaceInput Patterns
BeforeAfter
Intersection
![Page 17: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/17.jpg)
Qualitative Approach
● Observation– Human vision
● Good at judging relation between brightness● Bad at judging absolute brightness
● So...– Use binary relations to:
● find light source● Not commit to particular reflectance function
![Page 18: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/18.jpg)
Qualitative Approach
● Each surface now defines a hemisphere of possible light directions– Overall consistency implies finding a non-
null intersection of hemispheres
![Page 19: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/19.jpg)
Qualitative Approach
● Hemisphere ~ set of vectors t s.t
Angle between s and t is less than 90sij t 0where sij satisfies
sij ni n j 2 0sij ni n j 0sij ni 0 sij n j 0 if surfacei brighter than surface j
sij ni 0 sij n j 0 if surface j brighter than surfacei
![Page 20: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/20.jpg)
Qualitative Approach
● Hemisphere ~ set of vectors t s.t
sij is defined between a surface i and surface j, with normals ni and nj
sij t 0where sij satisfies
sij ni n j 2 0sij ni n j 0sij ni 0 sij n j 0 if surfacei brighter than surface j
sij ni 0 sij n j 0 if surface j brighter than surfacei
![Page 21: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/21.jpg)
Qualitative Approach
● Hemisphere ~ set of vectors t s.t
s is perpindicular to the average of the normals
sij t 0where sij satisfies
sij ni n j 2 0sij ni n j 0sij ni 0 sij n j 0 if surfacei brighter than surface j
sij ni 0 sij n j 0 if surface j brighter than surfacei
![Page 22: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/22.jpg)
Qualitative Approach
● Hemisphere ~ set of vectors t s.t
s is perpindicular to plane defined by the normals
sij t 0where sij satisfies
sij ni n j 2 0sij ni n j 0sij ni 0 sij n j 0 if surfacei brighter than surface j
sij ni 0 sij n j 0 if surface j brighter than surfacei
![Page 23: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/23.jpg)
Qualitative Approach
● Hemisphere ~ set of vectors t s.t
The angle between s and ni is less than 90 & the angle between s and nj is greater
sij t 0where sij satisfies
sij ni n j 2 0sij ni n j 0sij ni 0 sij n j 0 if surfacei brighter than surface j
sij ni 0 sij n j 0 if surface j brighter than surfacei
![Page 24: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/24.jpg)
Solution
● Solution to constraints lie on a convex polygon on the unit sphere.
Direction of agreement
![Page 25: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/25.jpg)
Is it consistent?
● If no polygons found that satisfy all constraints, then– Shading is not consistent
No maximum
![Page 26: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/26.jpg)
Largely Solveable
● For each polygon, count # of constraints matched– A maximum indicates most likely lighting
direction
![Page 27: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/27.jpg)
Compound Edges
● What do unsatisfied constraints represent?– Compound edge – where surface changes
color, not just shading
Compound edges
![Page 28: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/28.jpg)
Justification
● Why do shape derivation with line drawings and not brightness?– Humans can use edges
● Can humans use grey level?– It seems like it– An experiment
![Page 29: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/29.jpg)
An Experiment
● Random Height Tesselation– Can human determine 3d structure?
3D Representation
Overhead View
![Page 30: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/30.jpg)
Limitations
● Polyhedral objects only● Later research addresses contoured
objects w/ smoothly changing brightness.
![Page 31: Verifying the “Consistency” of Shading Patterns and 3-D Structures Pawan Sinha & Edward Adelson](https://reader035.vdocument.in/reader035/viewer/2022062719/56649ec95503460f94bd77b6/html5/thumbnails/31.jpg)
Conclusions
● Humans mostly use edges to determine 3D structure
● Use shading to verify this determination
● Algorithm effective for polyhedra