1 self-calibration and neural network implementation of photometric stereo yuji iwahori, yumi...
TRANSCRIPT
![Page 1: 1 Self-Calibration and Neural Network Implementation of Photometric Stereo Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA](https://reader035.vdocument.in/reader035/viewer/2022081520/5697bfa21a28abf838c96274/html5/thumbnails/1.jpg)
1
Self-Calibration and Neural Network Implementation of Photometric Stereo
Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA
![Page 2: 1 Self-Calibration and Neural Network Implementation of Photometric Stereo Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA](https://reader035.vdocument.in/reader035/viewer/2022081520/5697bfa21a28abf838c96274/html5/thumbnails/2.jpg)
2
Background
Neural Network approach to Shape-from-shading by Sejnowski et al. (1987)
Realtime implementation of photometric stereo using LUT (Lookup Table) by Woodham (1994)
![Page 3: 1 Self-Calibration and Neural Network Implementation of Photometric Stereo Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA](https://reader035.vdocument.in/reader035/viewer/2022081520/5697bfa21a28abf838c96274/html5/thumbnails/3.jpg)
3
Background
Neural Network Based Photometric Stereo and extensions by Iwahori et al. (since 1995)
Required Conditions– Calibration Sphere and Test Object has the sam
e reflectance property under the same imaging conditions (taken under different directions of multiple light source)
![Page 4: 1 Self-Calibration and Neural Network Implementation of Photometric Stereo Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA](https://reader035.vdocument.in/reader035/viewer/2022081520/5697bfa21a28abf838c96274/html5/thumbnails/4.jpg)
4
Proposed Approach
Neural network implementation of photometric stereo for a rotational object with non-uniform reflectance factor.
We require no separate calibration object, instead self-calibration is done using controlled rotation of the target object itself.
![Page 5: 1 Self-Calibration and Neural Network Implementation of Photometric Stereo Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA](https://reader035.vdocument.in/reader035/viewer/2022081520/5697bfa21a28abf838c96274/html5/thumbnails/5.jpg)
5
Three Light Source Photometry
),,(),(),(
),,(),(),(
),,(),(),(
32133
32122
32111
nnnRyxyxE
nnnRyxyxE
nnnRyxyxE
Eliminating the effect of the reflectance factor gives),( yx
lyxEyxE
lyxEyxE
lyxEyxE
/),(),(
/),(),(
/),(),(
33
22
11
23
22
21 EEEl where
Let be image intensity and let be reflectance map for unit surface normal vector at each point),,( 321 nnn
E R
![Page 6: 1 Self-Calibration and Neural Network Implementation of Photometric Stereo Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA](https://reader035.vdocument.in/reader035/viewer/2022081520/5697bfa21a28abf838c96274/html5/thumbnails/6.jpg)
6
Observation System
Turn Tablex
x
y
y
-z
Camera
LightSource
Object
The target object is observed through a full 360 degrees of rotation under three separate illumination conditions.
![Page 7: 1 Self-Calibration and Neural Network Implementation of Photometric Stereo Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA](https://reader035.vdocument.in/reader035/viewer/2022081520/5697bfa21a28abf838c96274/html5/thumbnails/7.jpg)
7
Occluding Boundary
representation is used at the occluding boundary.
In stereographic projection, points on an occluding boundary lie on the circle
Except at such points, surface gradient parameters
where is given by
using .
222 gf
2222 4
4
4
4
gf
gq
gf
fp
),( 3231 nnqnnp ),( qp
),( gf
),( gf
![Page 8: 1 Self-Calibration and Neural Network Implementation of Photometric Stereo Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA](https://reader035.vdocument.in/reader035/viewer/2022081520/5697bfa21a28abf838c96274/html5/thumbnails/8.jpg)
8
Extraction of Feature Points Geometric information
– At an occluding boundary, the surface normal is perpendicular to both the tangent to the occluding contour itself and to the viewing direction.
xImage Plane
Occluding Boundary
y
z
Surface Normal
Viewing Direction
![Page 9: 1 Self-Calibration and Neural Network Implementation of Photometric Stereo Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA](https://reader035.vdocument.in/reader035/viewer/2022081520/5697bfa21a28abf838c96274/html5/thumbnails/9.jpg)
9
Extraction of Feature Points
Gaussian sphere
22
2222
)cos()sin(
4,
sin
cos2),(
rrR
rRRf
rR
Rrgf
The current is determined from , and as follows:),( gf R r
: radius of unit Gaussian sphere(=1)
: horizontal distance to the rotation axis at each occluding boundary point
R
r
g
f
0
),( gf
R
r
![Page 10: 1 Self-Calibration and Neural Network Implementation of Photometric Stereo Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA](https://reader035.vdocument.in/reader035/viewer/2022081520/5697bfa21a28abf838c96274/html5/thumbnails/10.jpg)
10
Use of Photometric Constraint The image irradiance of a tracked feature point ou
ght to become gradually higher from the left occluding boundary to rotation axis and then become gradually lower towards the right occluding boundary.
-90 0 900
Rotation angle
Rotation axis
Imag
e in
tens
ity
increa
sedecrease
![Page 11: 1 Self-Calibration and Neural Network Implementation of Photometric Stereo Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA](https://reader035.vdocument.in/reader035/viewer/2022081520/5697bfa21a28abf838c96274/html5/thumbnails/11.jpg)
11
Use of Photometric Constraint
Vertical axis: image intensity Horizontal axis: rotation angle
Photometric constraint
Rotation axis
0-90 90All points on an
occluding boundary
Rotation axis
0 90-90
![Page 12: 1 Self-Calibration and Neural Network Implementation of Photometric Stereo Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA](https://reader035.vdocument.in/reader035/viewer/2022081520/5697bfa21a28abf838c96274/html5/thumbnails/12.jpg)
12
Use of Photometric Constraint
Examples of plot for points which don’t satisfy the photometric constraint.
Vertical axis: image intensity Horizontal axis: rotation angle
Rotation axis
0-90 90
Rotation axis
900-90
![Page 13: 1 Self-Calibration and Neural Network Implementation of Photometric Stereo Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA](https://reader035.vdocument.in/reader035/viewer/2022081520/5697bfa21a28abf838c96274/html5/thumbnails/13.jpg)
13
Extraction of Training Data Set Points which happen to include the same v
alue of are sorted and the median value is selected as one unique point.
f
g
cba EEE
Feature point is 0
plane
For each (f,g) value, the representative feature point is selected for the training data set for NN learning.
The relation of image irradiance
gf
g
b
![Page 14: 1 Self-Calibration and Neural Network Implementation of Photometric Stereo Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA](https://reader035.vdocument.in/reader035/viewer/2022081520/5697bfa21a28abf838c96274/html5/thumbnails/14.jpg)
14
Extraction of Training Data Set
Vertical axis: image intensity Horizontal axis: rotation angle
Photometric constraint
Rotation axis
0 90-90Unique combination
0-90 90
Rotation axis
![Page 15: 1 Self-Calibration and Neural Network Implementation of Photometric Stereo Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA](https://reader035.vdocument.in/reader035/viewer/2022081520/5697bfa21a28abf838c96274/html5/thumbnails/15.jpg)
15
Neural Network Learning
n(1,1)
n(1,2)
a(1,1) n(2,1)n1
b(1,1)
b(1,2)
a(1,2)b(2,1)
b(2,2)
n(2,2)
1 1
n(1,P)
b(1,P) b(2,3)
n(2,3)
1 1
w(3,P)
w(1,1)
n2
n3
a(1,3)
a(P,1)
a(P,3)
11
…….
…
E1’
E2’
E3’
![Page 16: 1 Self-Calibration and Neural Network Implementation of Photometric Stereo Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA](https://reader035.vdocument.in/reader035/viewer/2022081520/5697bfa21a28abf838c96274/html5/thumbnails/16.jpg)
16
What this RBF NN does. Once learning is complete, that has been
learned is represented by the weight connecting each RBF neural network.
The resulting network generalizes in that it predicts a surface normal , given any to . Thus, the resulting network can be used to estimate the surface shape of the target object.
),,( 321 nnn1E 3E
![Page 17: 1 Self-Calibration and Neural Network Implementation of Photometric Stereo Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA](https://reader035.vdocument.in/reader035/viewer/2022081520/5697bfa21a28abf838c96274/html5/thumbnails/17.jpg)
17
Experiment
90[deg] 0[deg] ±180[deg]
Test object (Light source 1)
![Page 18: 1 Self-Calibration and Neural Network Implementation of Photometric Stereo Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA](https://reader035.vdocument.in/reader035/viewer/2022081520/5697bfa21a28abf838c96274/html5/thumbnails/18.jpg)
18Obtained Data Set as Needle Diagram
![Page 19: 1 Self-Calibration and Neural Network Implementation of Photometric Stereo Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA](https://reader035.vdocument.in/reader035/viewer/2022081520/5697bfa21a28abf838c96274/html5/thumbnails/19.jpg)
19
f
g
2
2
-2
-2
Feature Points onto Space),( gf
![Page 20: 1 Self-Calibration and Neural Network Implementation of Photometric Stereo Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA](https://reader035.vdocument.in/reader035/viewer/2022081520/5697bfa21a28abf838c96274/html5/thumbnails/20.jpg)
20
Results
Aspect Slope
![Page 21: 1 Self-Calibration and Neural Network Implementation of Photometric Stereo Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA](https://reader035.vdocument.in/reader035/viewer/2022081520/5697bfa21a28abf838c96274/html5/thumbnails/21.jpg)
21Another Example of Input Images
![Page 22: 1 Self-Calibration and Neural Network Implementation of Photometric Stereo Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA](https://reader035.vdocument.in/reader035/viewer/2022081520/5697bfa21a28abf838c96274/html5/thumbnails/22.jpg)
22
Results
Aspect Slope
![Page 23: 1 Self-Calibration and Neural Network Implementation of Photometric Stereo Yuji IWAHORI, Yumi WATANABE, Robert J. WOODHAM and Akira IWATA](https://reader035.vdocument.in/reader035/viewer/2022081520/5697bfa21a28abf838c96274/html5/thumbnails/23.jpg)
23
Conclusion Neural network based photometric stereo u
sing self-calibration, was proposed. No calibration object is required to obtain s
hape of target object using geometric and photometric constraints.
Empirical implementation has been performed.
To detect and to correct for cast shadows is remained as future work.