inferring reflectance functions from wavelet noise pieter peers philip dutré pieter peers philip...
TRANSCRIPT
![Page 1: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/1.jpg)
Inferring Reflectance FunctionsInferring Reflectance Functionsfrom Wavelet Noisefrom Wavelet Noise
Pieter Peers
Philip Dutré
Pieter Peers
Philip Dutré
June 30th 2005
Department of Computer Science
![Page 2: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/2.jpg)
Image-based Relighting / Environment MattingImage-based Relighting / Environment Matting
Scene(fixed viewpoint)
![Page 3: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/3.jpg)
Image-based Relighting / Environment MattingImage-based Relighting / Environment Matting
…
Scene(fixed viewpoint)
Novel Incident Illumination
+
![Page 4: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/4.jpg)
Image-based Relighting / Environment MattingImage-based Relighting / Environment Matting
… …
Scene(fixed viewpoint)
Novel Incident Illumination Compute Relit Image
+ =
![Page 5: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/5.jpg)
Image-based Relighting / Environment MattingImage-based Relighting / Environment Matting
… …
Scene(fixed viewpoint)
Novel Incident Illumination Compute Relit Image
+ =
ReflectanceFunction
![Page 6: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/6.jpg)
Examples of Reflectance FunctionsExamples of Reflectance Functions
Diffuse BallSpecular Ball
![Page 7: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/7.jpg)
Examples of Reflectance FunctionsExamples of Reflectance Functions
Diffuse BallSpecular Ball
![Page 8: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/8.jpg)
Examples of Reflectance FunctionsExamples of Reflectance Functions
Diffuse BallSpecular Ball
Reflectance Function Reflectance Function
![Page 9: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/9.jpg)
Reflectance Functions (frequency domain)Reflectance Functions (frequency domain)
Diffuse BallSpecular Ball
Reflectance Function (frequency domain) Reflectance Function (frequency domain)
![Page 10: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/10.jpg)
Reflectance Functions (wavelet domain)Reflectance Functions (wavelet domain)
Diffuse BallSpecular Ball
Reflectance Function (wavelet domain) Reflectance Function (wavelet domain)
![Page 11: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/11.jpg)
Relight a PixelRelight a Pixel
Novel Incident IlluminationSpecular Ball
Relit pixel value?
Reflectance Function (wavelet space)
![Page 12: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/12.jpg)
Relight a PixelRelight a Pixel
Novel Incident IlluminationSpecular Ball
Reflectance Function (wavelet space) Incident Illumination (wavelet space)
![Page 13: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/13.jpg)
Relight a PixelRelight a Pixel
Novel Incident IlluminationSpecular Ball
Reflectance Function (wavelet space) Incident Illumination (wavelet space)
( )
![Page 14: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/14.jpg)
Relight a PixelRelight a Pixel
Novel Incident IlluminationSpecular Ball
Reflectance Function (wavelet space) Incident Illumination (wavelet space)
( )
Onlynon-zero
coefficients
![Page 15: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/15.jpg)
Directly Observing Reflectance FunctionsDirectly Observing Reflectance Functions
Controlled Incident IlluminationPhotograph of Specular Ball
Emit(e.g. from CRT)
![Page 16: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/16.jpg)
Directly Observing Reflectance FunctionsDirectly Observing Reflectance Functions
Controlled Incident IlluminationPhotograph of Specular Ball
ReflectanceFunction
(unknown)
Observed pixel
Controlled Incident Illumination (wavelet space)
![Page 17: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/17.jpg)
Directly Observing Reflectance FunctionsDirectly Observing Reflectance Functions
Controlled Incident IlluminationPhotograph of Specular Ball
Unknown Reflectance Function (wavelet space)
( )
Controlled Incident Illumination (wavelet space)
![Page 18: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/18.jpg)
Directly Observing Reflectance FunctionsDirectly Observing Reflectance Functions
Controlled Incident IlluminationPhotograph of Specular Ball
Controlled Incident Illumination (wavelet space)
( )
Onlynon-zero
coefficients
Observed coefficient
Unknown Reflectance Function (wavelet space)
![Page 19: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/19.jpg)
Number of ObservationsNumber of Observations
Specular Ball
Reflectance Function (wavelet space)
#Photographs=
#Illumination pixels
Incident Illumination
![Page 20: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/20.jpg)
Number of Observations ProblemNumber of Observations Problem
Specular Ball
Reflectance Function (wavelet space)
Incident Illumination
1000x
1000
1000x
1000
#Photographs=
#Illumination pixels
![Page 21: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/21.jpg)
Wavelet Noise IlluminationWavelet Noise Illumination
Wavelet Noise
•Normal distribution of wavelet coefficients
•Mean : 0.0
•Standard deviation : 1.0
•Rescale Wavelet Noise Pattern to fit into [0..1] range
Wavelet Noise Pattern
Wavelet Noise Pattern (wavelet space)
Advantages
•Arbitrary number of different patterns possible
•Any reflectance function gives a non-zero response
•Constant average luminance
![Page 22: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/22.jpg)
Estimating Wavelet Coefficients Estimating Wavelet Coefficients
(Unknown)Reflectance Function
Wavelet Noise
Assume: positions of are knownQuestion: what are the magnitudes?
( ) =Observed
Pixel Value
![Page 23: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/23.jpg)
Estimating Wavelet Coefficients Estimating Wavelet Coefficients
( ) =
Leave out zero coefficients(of the reflectance function)
Wavelet Noise (linearized)
Reflectance Function(linearized)
Observed Pixel Value
![Page 24: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/24.jpg)
Estimating Wavelet Coefficients Estimating Wavelet Coefficients
= …
Multiple observations matrix-vector multiplication
…
Wavelet NoiseReflectance
Function
Observed PixelValues
# em
itted
pat
tern
s # observations
![Page 25: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/25.jpg)
Estimating Wavelet Coefficients Estimating Wavelet Coefficients
=
Finding magnitudes : Linear Least Squares problem
… …
Wavelet NoiseReflectance
Function
Observed PixelValues
![Page 26: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/26.jpg)
Estimating Wavelet Coefficients Estimating Wavelet Coefficients
=
Estimation error when onlya part is approximated?
… …
Wavelet NoiseReflectance
Function
Observed PixelValues
![Page 27: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/27.jpg)
Partial EstimationPartial Estimation
+… … …= = …
Wavelet NoiseReflectance
Function
ObservedPixel Values
![Page 28: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/28.jpg)
Partial EstimationPartial Estimation
According to a normal distribution
+… … …= = …
Wavelet NoiseReflectance
Function
ObservedPixel Values
![Page 29: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/29.jpg)
Partial EstimationPartial Estimation
According to a normal distribution
+… … …= = …
Wavelet NoiseReflectance
Function
ObservedPixel Values
Normal Normal distributiondistribution
![Page 30: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/30.jpg)
Partial EstimationPartial Estimation
+… …= = …
Wavelet NoiseReflectance
Function
ObservedPixel Values
Finding the best approximation for : Linear Least Squares problem
NoIse
![Page 31: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/31.jpg)
Inferring Reflectance FunctionsInferring Reflectance Functions
Reflectance Function(2D wavelet space)
Priority Queueof Candidates
![Page 32: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/32.jpg)
Inferring Reflectance FunctionsInferring Reflectance Functions
Reflectance Function(2D wavelet space)
Priority Queueof Candidates
![Page 33: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/33.jpg)
Inferring Reflectance FunctionsInferring Reflectance Functions
Reflectance Function(2D wavelet space)
Priority Queueof Candidates
![Page 34: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/34.jpg)
Reflectance Function(2D wavelet space)
Inferring Reflectance FunctionsInferring Reflectance Functions
Priority Queueof Candidates
![Page 35: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/35.jpg)
Inferring Reflectance FunctionsInferring Reflectance Functions
Reflectance Function(2D wavelet space)
Priority Queueof Candidates
![Page 36: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/36.jpg)
Inferring Reflectance FunctionsInferring Reflectance Functions
Reflectance Function(2D wavelet space)
Priority Queueof Candidates
![Page 37: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/37.jpg)
Inferring Reflectance FunctionsInferring Reflectance Functions
Reflectance Function(2D wavelet space)
Priority Queueof Candidates
![Page 38: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/38.jpg)
Inferring Reflectance FunctionsInferring Reflectance Functions
Reflectance Function(2D wavelet space)
Priority Queueof Candidates
![Page 39: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/39.jpg)
OverviewOverview
Record photographs
Emit
Wavelet Noise
Predetermined number of photographs
![Page 40: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/40.jpg)
OverviewOverview
Record photographs
Infer Reflectance Functions
Reflectance Function
Progressive Algorithm
For each pixel
![Page 41: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/41.jpg)
OverviewOverview
Record photographs
Infer Reflectance Functions
Compute Relit Image
Relight
Incident Illumination
![Page 42: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/42.jpg)
ResultsResults
64 Haar Wavelet Coefficients256 Photographs
Reference Photograph
![Page 43: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/43.jpg)
ResultsResults
64 Haar Wavelet Coefficients256 Photographs
Reference Photograph
![Page 44: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/44.jpg)
ResultsResults
64 Haar Wavelet Coefficients256 Photographs
Reference Photograph
![Page 45: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/45.jpg)
ResultsResults
64 Haar Wavelet Coefficients256 Photographs
Reference Photograph
![Page 46: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/46.jpg)
ResultsResults
64 Haar Wavelet Coefficients256 Photographs
Reference Photograph
![Page 47: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/47.jpg)
ResultsResults
128 Haar Wavelet Coefficients512 Photographs
Reference Photograph
![Page 48: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/48.jpg)
Results: High Frequency IlluminationResults: High Frequency Illumination
![Page 49: Inferring Reflectance Functions from Wavelet Noise Pieter Peers Philip Dutré Pieter Peers Philip Dutré June 30 th 2005 Department of Computer Science](https://reader035.vdocument.in/reader035/viewer/2022070415/5697bfe21a28abf838cb49a1/html5/thumbnails/49.jpg)
Conclusion & Future WorkConclusion & Future Work
Inferring Reflectance Functions from Wavelet Noise– No restriction on material properties– Stochastic illumination patterns– Trade-off quality versus acquisition time
Future Work– Noise filtering– Higher-order wavelets