capture and displays - university of california, irvine
TRANSCRIPT
![Page 1: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/1.jpg)
Capture and Capture and Displays Displays
CS 211ACS 211A
![Page 2: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/2.jpg)
HDR ImageHDR Image
![Page 3: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/3.jpg)
Bilateral FilterBilateral Filter
![Page 4: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/4.jpg)
Color GamutColor Gamut
Natural Colors
Camera Gamut
![Page 5: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/5.jpg)
Traditional DisplaysTraditional Displays• LCD panels
–The gamut is the result of the filters used
• In projectors
![Page 6: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/6.jpg)
Recent high gamut displaysRecent high gamut displays• Comes from use of LEDs
–LEDs are much saturated primaries–1.5 times HDTV gamut
• Projectors (MERL)
![Page 7: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/7.jpg)
Recent high gamut displaysRecent high gamut displays• Comes from use of LEDs
–LEDs are much saturated primaries–1.5 times HDTV gamut
• LCD panels (HP Dream Color)–Use LED backlighting
• Laser multi-primary ones are coming up
![Page 8: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/8.jpg)
Color GamutColor Gamut
Natural Colors
Camera Gamut
![Page 9: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/9.jpg)
What about capture?What about capture?• No HG capture
–Not in the required spatial and temporal resolution
• Closest is hyperspectral imagery–Captures multiple narrow spectral bands – Low spatial resolution (512x512)–Low temporal resolution (10 fps)–Cost is $50,000–Used for scientific application
![Page 10: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/10.jpg)
Gap between Capture and Gap between Capture and DisplayDisplay
• Hollywood have defined a digital cinema standard gamut –What they want?
• Close to the current high gamut displays
• No capture device• Sophisticated gamut extrapolation
methods
![Page 11: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/11.jpg)
High Resolution ImageryHigh Resolution Imagery• Capture
–Panoramic Image generation
• Display–Tiled displays
![Page 12: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/12.jpg)
Panoramic ImagePanoramic Image
![Page 13: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/13.jpg)
Basic AlgorithmBasic Algorithm• Assume distant scene, locally planar• Detect features across adjacent pics• Relate two adjacent pictures by a
homography• Stitch them
![Page 14: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/14.jpg)
Problems?Problems?• What if circular?
–Can have inconsistencies
• What is more than one row?–Same issue inconsistencies
• Use some global optimization techniques–Optimize across all images together–Bundle adjustment
![Page 15: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/15.jpg)
Another problemAnother problem• Spatial intensity
fall off in cameras–Primarily due to
the lens–Vignetting
• Radial Fall off• May not be
centered
![Page 16: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/16.jpg)
![Page 17: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/17.jpg)
Inaccurate feature detectionInaccurate feature detection
![Page 18: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/18.jpg)
Radiometric Camera Radiometric Camera CalibrationCalibration
• Finding transfer function and vignetting
• Debevec’s method–Recovers transfer function–Assumes no vignetting effect–Assures by setting aperture of a narrow
FOV camera to a very low value
![Page 19: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/19.jpg)
How to calibrate the How to calibrate the camera?camera?
• Debevec–Need a few pixels to recover transfer
function–Can be the center 10x10 of a camera
where negligible vignetting–Find g
![Page 20: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/20.jpg)
Goldman et al (2005)Goldman et al (2005)• Assumes known transfer function• Takes panoramic images with very large
overlaps• Same irradiance imaged at different pixels
– Have different intensity due to different vignetting or exposure
• Basic idea– For corresponding feature f in image i and j
• g(Zi) = ln(E) + ln (Vi) + ln(ti)• g(Zj) = ln(E) +ln(Vj) +ln(tj)
• Recover both vignetting and exposure
![Page 21: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/21.jpg)
ResultsResults
![Page 22: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/22.jpg)
Radiometric camera Radiometric camera calibrationcalibration
• Underconstrained problem• Cannot get both transfer function
and vignetting effect–Transfer function upto an exponential
ambiguity
• Both upto scale factor– If transfer function is known
![Page 23: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/23.jpg)
JuangJuang and and MajumderMajumder 20072007• Use a projector in the loop• Constrain the problem by using
known inputs to the projector
![Page 24: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/24.jpg)
24
fc( )fc( )
ModelModel
Projector ITFfp
Spatial VariationDue to Projector
P(x,y)Spatial Variation
Due to ScreenS(x,y)
Exposuretj
Spatial VariationDue to Camera
C(x,y)
Camera ITFfc
tj
tj
fp( )fp( ) P(x,y)P(x,y) S(x,y)S(x,y) C(x,y)C(x,y)iZout =Zout =
![Page 25: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/25.jpg)
L(x,y)L(x,y)fc( )fc( )
Calibration AlgorithmCalibration Algorithm
tj
tj
fp( )fp( ) P(x,y)P(x,y) S(x,y)S(x,y) C(x,y)C(x,y)iZout =Zout =
![Page 26: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/26.jpg)
• Since fc( ) is monotonic, inverse exists.
fc( )fc( )L(x,y)L(x,y) tjtjfp( )fp( )iZout =Zout =fc-1( )fc-1( )
Calibration AlgorithmCalibration Algorithm
![Page 27: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/27.jpg)
L(x,y)L(x,y) tjtjfp( )fp( )iZout =Zout =fc-1( )fc-1( )
ln fc-1( )ln fc-1( )Zout =Zout = ln L(x,y)ln L(x,y) ln (tj )ln (tj )ln fp( )ln fp( )i ++ ++
Calibration AlgorithmCalibration Algorithm• Take the log of both sides
![Page 28: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/28.jpg)
Calibration AlgorithmCalibration Algorithm• Images of multiple projector inputs
at multiple camera exposures• Setup system of equations and solve
using least-squares
ln fc-1( )ln fc-1( )Zout =Zout = ln L(x,y)ln L(x,y) ln (tj )ln (tj )ln fp( )ln fp( )i ++ ++
![Page 29: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/29.jpg)
Separating ParametersSeparating Parameters• Recall model
fc( )fc( )L(x,y)L(x,y) tjtjfp( )fp( )iZout =Zout =
P(x,y)P(x,y) S(x,y)S(x,y) C(x,y)C(x,y)
Camera vignettingCamera vignettingProjector vignettingProjector vignetting
![Page 30: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/30.jpg)
Separating ParametersSeparating Parameters• Recall model
• Different camera aperture => different L
fc( )fc( )L(x,y)L(x,y) tjtjfp( )fp( )iZout =Zout =
P(x,y)P(x,y) S(x,y)S(x,y) C(x,y)C(x,y)
Camera vignettingCamera vignettingProjector vignettingProjector vignetting
![Page 31: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/31.jpg)
kk
Separating ParametersSeparating Parameters• At narrow camera apertures, camera
vignetting is negligible, almost constant
Lf/32(x,y) =Lf/32(x,y) = P(x,y)P(x,y) S(x,y)S(x,y) Cf/32(x,y)Cf/32(x,y)
![Page 32: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/32.jpg)
Separating ParametersSeparating Parameters• At wider apertures
• Normalized vignetting effect
Lf/32(x,y) =Lf/32(x,y) = P(x,y)P(x,y) S(x,y)S(x,y)La(x,y) =La(x,y) = P(x,y)P(x,y) S(x,y)S(x,y) Ca(x,y)Ca(x,y)
==kk
==Ca(x,y)Ca(x,y)
kk
![Page 33: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/33.jpg)
High Resolution DisplaysHigh Resolution Displays• Tile Projectors or LCD panels• Seamless or with seams
![Page 34: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/34.jpg)
Building Tiled DisplaysBuilding Tiled Displays
Client
Network
ServersProjectorsDisplay Screen
Viewer
![Page 35: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/35.jpg)
Geometric/Photometric Geometric/Photometric MismatchMismatch
![Page 36: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/36.jpg)
Camera Based RegistrationCamera Based Registration• Camera feedback detects
misregistration• Encoded in a mathematical function
–Both geometric and photometric
• Change the projected image digitally–Apply the inverse function– In real-time via GPU
![Page 37: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/37.jpg)
Different spacesDifferent spacesDisplay D(s, t)
![Page 38: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/38.jpg)
Simple Geometric AlignmentSimple Geometric Alignment
Projector(xi, yi)
Display(s, t)
G
![Page 39: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/39.jpg)
Simple Geometric Simple Geometric AlignmentAlignment
Projector(xi, yi)
Display(s, t)
G = F ● H
Camera(u, v)
H F
Apply G-1 for registration
![Page 40: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/40.jpg)
Planar DisplayPlanar Display
Projector(xi, yi)
Display(s, t)
G = F ● H
Camera(u, v)
H F
• Calibrated camera (no radial distortion)• F is linear (3x3 matrix called homography)
R. Raskar, Immersive Planar Display using Roughly Aligned Projectors, IEEE VR, 2000.
![Page 41: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/41.jpg)
• G = F x H• G-1 is just a matrix inversion
Perfect ProjectorsPerfect Projectors
Projector(xi, yi)
Display(s, t)
G = F ● H
Camera(u, v)
H F
R. Raskar, Immersive Planar Display using Roughly Aligned Projectors, IEEE VR, 2000.
Homography
![Page 42: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/42.jpg)
Linear Linear GG
![Page 43: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/43.jpg)
Corrected using Corrected using GG--11
![Page 44: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/44.jpg)
Intensity VariationIntensity Variation
• If the projectors are good and similar– No vignetting
• Take care of overlaps
• If not, measure accurately
1) A. Majumder, Properties of Color Variation in Multi Projector Displays, SID Eurodisplay, 2002.2) A. Majumder and R. Stevens, Color Non-Uniformity in Multi Projector Displays: Analysis and
Solutions, IEEE Transactions on Visualization and Computer Graphics, Vol. 10, No. 2, 2003.
![Page 45: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/45.jpg)
Use some edge blendingUse some edge blending
Software Blending
Before
1) Lyon Paul, Edge-blending Multiple Projection Displays On A Dome Surface To Form Continuous Wide Angle Fields-of-View, Proceedings of 7th I/ITEC, 203-209, 1985.
2) R. Raskar et al, Seamless Camera-Registered Multi-Projector Displays Over Irregular Surfaces,Proceedings of IEEE Visualization, 161-168, 1999.
3) K. Li et.al, Early experiences and challenges in building and using a scalable display wall system,IEEE Computer Graphics and Applications 20(4), 671-680, 2000.
![Page 46: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/46.jpg)
Handle all variationsHandle all variations• Measure each projector’s vignetting• Measure each projector’s transfer
function
![Page 47: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/47.jpg)
Add them upAdd them up
![Page 48: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/48.jpg)
Strict Luminance UniformityStrict Luminance Uniformity
u
L
![Page 49: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/49.jpg)
Strict Luminance UniformityStrict Luminance Uniformity
u
L
![Page 50: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/50.jpg)
Before
After Strict Luminance Uniformity
A. Majumder and R. Stevens, Color Non-Uniformity in Multi Projector Displays: Analysis and Solutions, IEEE Transactions on Visualization and Computer Graphics, Vol. 10, No. 2, 2003.
ResultsResults
![Page 51: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/51.jpg)
Strict Luminance UniformityStrict Luminance UniformityL
u
L
Suboptimal use of system resources
Significant Contrast/ Dynamic Range Compression
![Page 52: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/52.jpg)
Smooth the Luminance Smooth the Luminance functionfunction
![Page 53: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/53.jpg)
Optimization ProblemOptimization Problem
u
L
u
L
Strict luminance uniformity is a special case.
![Page 54: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/54.jpg)
Before
After Strict Luminance Uniformity
ResultsResults
![Page 55: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/55.jpg)
Before
After Luminance Smoothing
ResultsResults
1) A. Majumder, R. Stevens, Perceptual Photometric Seamlessness in Tiled Projection Based Displays, ACM Transactions on Graphics, Vol. 24, No. 1, 2005.
2) A. Majumder, Improving Contrast of Multi-Displays Using Human Contrast Sensitivity,IEEE CVPR 2005.
![Page 56: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/56.jpg)
Smooth
Original
Flat
Different smoothing parametersDifferent smoothing parameters(2x2 array of four projectors)(2x2 array of four projectors)
Smoother
![Page 57: Capture and Displays - University of California, Irvine](https://reader031.vdocument.in/reader031/viewer/2022012101/61db629ee9b7454d696f4394/html5/thumbnails/57.jpg)
Smooth
OriginalSmoother
Flat
Different smoothing parametersDifferent smoothing parameters(3x5 array of fifteen projectors)(3x5 array of fifteen projectors)