what does motion reveal about transparency ?
DESCRIPTION
What Does Motion Reveal About Transparency ?. Moshe Ben-Ezra and Shree K. Nayar Columbia University ICCV Conference October 2003, Nice, France. This work was supported by an NSF ITR Award IIS-00-85864. Transparency is Very Challenging. Existence of a transparent object. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/1.jpg)
What Does Motion Reveal About Transparency ?
Moshe Ben-Ezra and Shree K. NayarColumbia University
ICCV ConferenceOctober 2003, Nice, France
This work was supported by an NSF ITR Award IIS-00-85864
![Page 2: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/2.jpg)
Transparency is Very Challenging
• Existence of a transparent object.• Finding its shape and pose
![Page 3: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/3.jpg)
Real and Virtual Features
Lambertian
V1 V2
F`
V2
V1 F
Specular
F
F`
V1 V2
Transparent
![Page 4: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/4.jpg)
Environmental Matting*
* Zongker, el al. SIGGRAPH 99,
Alternatingpattern Object Camera
• Does not recover shape and pose.• Requires controlled environment.
![Page 5: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/5.jpg)
Shape from Polarization in Highlight*
* Saito et al. CVPR’99.
Object
CameraLight
RotatingPolarizer
• Limited to a single interface at the object’s surface.• Requires controlled environment.
N
![Page 6: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/6.jpg)
Shape from Refraction and Motion*
* H. Murase. PAMI, 1992
Camera
Water
• Single interface only.
Fixed Pattern
![Page 7: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/7.jpg)
Motion is Key to Transparency
![Page 8: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/8.jpg)
Transparent Shape From Motion
Given: Views
1,,222
,
n
ne
eene zyxzyxs
And a Parametric Model (such as super-ellipse)
Recover:Shape: Values of parameters (e, n)Pose: Rotation R, Translation T
General analytic solution does not exist.
![Page 9: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/9.jpg)
Transparency From Motion
Reversed rays are parallel to each other regardless ofthe complexity of their paths
Distantfeature
![Page 10: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/10.jpg)
Approach: Initialization
Image Plane
Image Plane
![Page 11: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/11.jpg)
Approach: Initial Guess
![Page 12: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/12.jpg)
Approach: Refine
![Page 13: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/13.jpg)
Error Function
(0,0,1)
r1,1 .. r1,n
r2,1 .. r2,n
j iji rzTRff
Feature Ray ,1 ,cosvar,,
• - Object’s shape parameter vector• R,T - Object’s pose
![Page 14: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/14.jpg)
Simulation SetupParallel rays
from features
Transparentobject
Cameraside rays
![Page 15: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/15.jpg)
Example (Simulation)
Single Parameter. Newton-Raphson optimization
Initial GuessSymmetric Superellipse (n=e)
![Page 16: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/16.jpg)
Evaluation (Simulation)
GT
Both init
Pos res
Sphere
Ground Truth
Initial Guess
ComputedResult
ShapeError
GT
Both Init
Both Res
Lens
GT
Both Init
Both Res
Cube
GT
Both Init
Both Res
Water Pipe
mm15013.0:t mm
8003.0:f
25.00001.0:emm
160064.0:d
![Page 17: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/17.jpg)
Real Experiment: Sphere
![Page 18: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/18.jpg)
Features
![Page 19: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/19.jpg)
Initial Guess
![Page 20: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/20.jpg)
Setup: Initial Guess
Initial Guess: Diameter: 8 inch
![Page 21: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/21.jpg)
Setup: Result
Ground Truth: Diameter: 3 inch. Computed: 3.18 inch
![Page 22: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/22.jpg)
Result
![Page 23: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/23.jpg)
Real Test: Water Filled Pipe
![Page 24: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/24.jpg)
Features
![Page 25: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/25.jpg)
Initial Guess
![Page 26: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/26.jpg)
Setup: Initial Guess
Initial guess: Diameter: 200.0mm Thickness: 20.0mm
![Page 27: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/27.jpg)
Setup: Result
Ground Truth: Diameter: 117.0mm Thickness: 3.0mmComputed: Diameter: 116.1mm Thickness: 2.3mm
![Page 28: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/28.jpg)
Result
![Page 29: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/29.jpg)
Real Test: Superquadric
![Page 30: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/30.jpg)
Features
![Page 31: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/31.jpg)
Initial Guess
![Page 32: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/32.jpg)
Result
Ground truth: e = ? Computed: e = 0.18
![Page 33: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/33.jpg)
Summary
Shape and poseparameters
Multiple interfaces
No Segmentationrequired
![Page 34: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/34.jpg)
![Page 35: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/35.jpg)
Parameterizations of Interest
• Polynomials: modeling surfaces, lenses
• CAD models: shape of industrial objects
• Dynamic models: time dependent parameters
![Page 36: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/36.jpg)
Assumptions
• Camera parameters are known.
• Features are far* and are trackable.
• A proper model and a hypothesis (an initial guess) are given.
* One possible assumption.
![Page 37: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/37.jpg)
Real Tests Setup
![Page 38: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/38.jpg)
Implementation
• Features were manually selected and tracked (9 views).
• Captured rays, a model, refraction index and a hypothesis were given as inputs.
• Shape and pose were recovered using simple gradient decent (with derivatives).
![Page 39: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/39.jpg)
The Physics of TransparencyFirst Interface:
μ1→ μ2
Second Interface: μ2 →μ1
3
11
3
N1
N2
2 2
3211 sinsin :Refraction
2
111 sin
:reflection internal Total
2
21
221
,21
R :Reflection
![Page 40: What Does Motion Reveal About Transparency ?](https://reader035.vdocument.in/reader035/viewer/2022062521/56816784550346895ddc97d8/html5/thumbnails/40.jpg)
Parametric Shape Examples
Super-Ellipse2 parameters
Spherical Harmonics8 parameters
No analytic solution