![Page 1: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/1.jpg)
Presented by : ……..Presented by : ……..
Ramesh Raskar, Greg Welch and Ramesh Raskar, Greg Welch and Henry FuchsHenry Fuchs
University of North Carolina at Chapel University of North Carolina at Chapel HillHill
Presented by : ……..Presented by : ……..
Ramesh Raskar, Greg Welch and Ramesh Raskar, Greg Welch and Henry FuchsHenry Fuchs
University of North Carolina at Chapel University of North Carolina at Chapel HillHill
Seamless Projection Overlaps using Image Warping and Intensity
Blending
![Page 2: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/2.jpg)
Acknowledgements
• Office of the Future group at UNC Chapel HillOffice of the Future group at UNC Chapel Hill
• Andrei State (sketches)Andrei State (sketches)
• SupportSupport
– NSF Science and Technology Center for Graphics NSF Science and Technology Center for Graphics and Visualization, USAand Visualization, USA
– Advanced Networks and Services,National Tele-Advanced Networks and Services,National Tele-Immersion InitiativeImmersion Initiative
– Defense Advanced Research Projects Agency, USA Defense Advanced Research Projects Agency, USA
– Intel CorporationIntel Corporation
• Office of the Future group at UNC Chapel HillOffice of the Future group at UNC Chapel Hill
• Andrei State (sketches)Andrei State (sketches)
• SupportSupport
– NSF Science and Technology Center for Graphics NSF Science and Technology Center for Graphics and Visualization, USAand Visualization, USA
– Advanced Networks and Services,National Tele-Advanced Networks and Services,National Tele-Immersion InitiativeImmersion Initiative
– Defense Advanced Research Projects Agency, USA Defense Advanced Research Projects Agency, USA
– Intel CorporationIntel Corporation
![Page 3: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/3.jpg)
Seamless Projection Overlaps
P1
P2
P3 P1
P2P3
Display Surface
Projectors
Well Defined Display Configuration
Irregular display and projector configuration
Traditional Flexible Setup
![Page 4: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/4.jpg)
Goal• High Resolution, Wide Field of View High Resolution, Wide Field of View
DisplayDisplay
• Overlapping front projection systemOverlapping front projection system
• Seamless displaySeamless display
– Geometric Registration of Geometric Registration of overlapsoverlaps
– Intensity Blending near edgesIntensity Blending near edges
• Irregular display surface Irregular display surface configurationconfiguration
• High Resolution, Wide Field of View High Resolution, Wide Field of View DisplayDisplay
• Overlapping front projection systemOverlapping front projection system
• Seamless displaySeamless display
– Geometric Registration of Geometric Registration of overlapsoverlaps
– Intensity Blending near edgesIntensity Blending near edges
• Irregular display surface Irregular display surface configurationconfiguration
P1
P2P3
![Page 5: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/5.jpg)
Motivation :Office of the Future (UNC ‘98)
• Irregular Irregular surfacessurfaces
• No specific No specific projector projector configuratioconfigurationn
• Irregular Irregular surfacessurfaces
• No specific No specific projector projector configuratioconfigurationn
![Page 6: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/6.jpg)
Other Spatially Immersive Displays
1. CAVE (UIC 1. CAVE (UIC EVL)EVL)
• Non-overlapping Non-overlapping projectionsprojections
• Flat displays wallsFlat displays walls
• Well-defined Well-defined projection projection configurationconfiguration
1. CAVE (UIC 1. CAVE (UIC EVL)EVL)
• Non-overlapping Non-overlapping projectionsprojections
• Flat displays wallsFlat displays walls
• Well-defined Well-defined projection projection configurationconfiguration
![Page 7: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/7.jpg)
Other Spatially Immersive Displays (Cont.)
2. VisionDome 2. VisionDome (Alternate Realities)(Alternate Realities)
• Single Projector with Single Projector with expensive opticsexpensive optics
• Well defined display Well defined display surfacessurfaces
2. VisionDome 2. VisionDome (Alternate Realities)(Alternate Realities)
• Single Projector with Single Projector with expensive opticsexpensive optics
• Well defined display Well defined display surfacessurfaces
![Page 8: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/8.jpg)
Other Spatially Immersive Displays (Cont.)
3. Cylindrical Screens (Trimension, 3. Cylindrical Screens (Trimension, Panoram)Panoram)
• 3 Projectors with side-by-side rectangular 3 Projectors with side-by-side rectangular overlapoverlap
• Well defined display surfaceWell defined display surface
• Precise electro-mechanical setupPrecise electro-mechanical setup
3. Cylindrical Screens (Trimension, 3. Cylindrical Screens (Trimension, Panoram)Panoram)
• 3 Projectors with side-by-side rectangular 3 Projectors with side-by-side rectangular overlapoverlap
• Well defined display surfaceWell defined display surface
• Precise electro-mechanical setupPrecise electro-mechanical setup
![Page 9: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/9.jpg)
Office of the Future (UNC ‘98)
• Irregular surfacesIrregular surfaces
• No specific projector configurationNo specific projector configuration
• Irregular surfacesIrregular surfaces
• No specific projector configurationNo specific projector configuration
![Page 10: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/10.jpg)
Office of the Future Configuration
Projectors
Single User
Irregular Surfaces
User
Top View
![Page 11: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/11.jpg)
Our Approach
• Calibration of display configuration using a Calibration of display configuration using a video cameravideo camera
• Rendering using texture mapping hardwareRendering using texture mapping hardware
• Pair-wise registration and blending of Pair-wise registration and blending of projection overlapsprojection overlaps
• Calibration of display configuration using a Calibration of display configuration using a video cameravideo camera
• Rendering using texture mapping hardwareRendering using texture mapping hardware
• Pair-wise registration and blending of Pair-wise registration and blending of projection overlapsprojection overlaps
![Page 12: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/12.jpg)
Key Features
+ Display surfaces may not be flat wallsDisplay surfaces may not be flat walls
+ Projection axis not orthogonal to displaysProjection axis not orthogonal to displays
+ Flexible projection configurationFlexible projection configuration
+ Projector’s intrinsic or extrinsic parameters Projector’s intrinsic or extrinsic parameters not needed not needed
– Generated image correct for a single ideal Generated image correct for a single ideal viewer’s locationviewer’s location
+ Display surfaces may not be flat wallsDisplay surfaces may not be flat walls
+ Projection axis not orthogonal to displaysProjection axis not orthogonal to displays
+ Flexible projection configurationFlexible projection configuration
+ Projector’s intrinsic or extrinsic parameters Projector’s intrinsic or extrinsic parameters not needed not needed
– Generated image correct for a single ideal Generated image correct for a single ideal viewer’s locationviewer’s location
![Page 13: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/13.jpg)
Calibration
• Step 1 : Geometric RegistrationStep 1 : Geometric Registration
– Pre-distort images so that when projected Pre-distort images so that when projected they appearthey appear
◊ Perspectively correctPerspectively correct
◊ Aligned with neighboring projectionAligned with neighboring projection
• Step 2 : Intensity NormalizationStep 2 : Intensity Normalization
– Blend images from overlapping projectionsBlend images from overlapping projections
• Step 1 : Geometric RegistrationStep 1 : Geometric Registration
– Pre-distort images so that when projected Pre-distort images so that when projected they appearthey appear
◊ Perspectively correctPerspectively correct
◊ Aligned with neighboring projectionAligned with neighboring projection
• Step 2 : Intensity NormalizationStep 2 : Intensity Normalization
– Blend images from overlapping projectionsBlend images from overlapping projections
![Page 14: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/14.jpg)
Calibration Step 1 : Geometric Registration
Warping Images Warping Images
• For irregular display surfacesFor irregular display surfaces
– Desired image is pre-warpedDesired image is pre-warped
– Warping function is found using a wide field Warping function is found using a wide field of view camera (WFOV)of view camera (WFOV)
• For flat display surfacesFor flat display surfaces
– Projected images related by a 3x2 Projected images related by a 3x2 transformationtransformation
Warping Images Warping Images
• For irregular display surfacesFor irregular display surfaces
– Desired image is pre-warpedDesired image is pre-warped
– Warping function is found using a wide field Warping function is found using a wide field of view camera (WFOV)of view camera (WFOV)
• For flat display surfacesFor flat display surfaces
– Projected images related by a 3x2 Projected images related by a 3x2 transformationtransformation
![Page 15: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/15.jpg)
Calibration Step 1 : Geometric Registration
Warping Images for Irregular Warping Images for Irregular Display SurfacesDisplay Surfaces
• Keep the WFOV camera at ideal viewer’s Keep the WFOV camera at ideal viewer’s locationlocation
• Project dots with the projector one by oneProject dots with the projector one by one
• Find mapping from projector image to camera Find mapping from projector image to camera imageimage
• Invert mapping to find the warping functionInvert mapping to find the warping function
Warping Images for Irregular Warping Images for Irregular Display SurfacesDisplay Surfaces
• Keep the WFOV camera at ideal viewer’s Keep the WFOV camera at ideal viewer’s locationlocation
• Project dots with the projector one by oneProject dots with the projector one by one
• Find mapping from projector image to camera Find mapping from projector image to camera imageimage
• Invert mapping to find the warping functionInvert mapping to find the warping function
![Page 16: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/16.jpg)
Camera to Projector warping function
Display Surface
WFOV Camera
Icamera(u,v)
Observing a point on display surface
Icamera(u,v) = Which projector pixel ?
![Page 17: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/17.jpg)
Camera to Projector warping function
P1
Display Surface Projector
P2
Iproj2(x,y)
Iproj1(x,y)
Projector
Which projector pixel illuminated the same point on display surface ?
Icamera(u,v) = Iproj1(x,y), Iproj2(x,y)
![Page 18: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/18.jpg)
Calibration Step 2 : Intensity Normalization
Blending Projected Images Blending Projected Images
• For irregular display surfacesFor irregular display surfaces
• Projection overlap observed in camera image Projection overlap observed in camera image spacespace
• Assign intensity weights for projector pixelsAssign intensity weights for projector pixels
Blending Projected Images Blending Projected Images
• For irregular display surfacesFor irregular display surfaces
• Projection overlap observed in camera image Projection overlap observed in camera image spacespace
• Assign intensity weights for projector pixelsAssign intensity weights for projector pixels
![Page 19: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/19.jpg)
Intensity in Projector Overlap
P1
Display Surface
Projector
P2
Iproj2(x,y)
Iproj1(x,y)
Projector
How to make intensity in overlap region same as everywhere else ?
WFOV Camera
Icamera(u,v)
![Page 20: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/20.jpg)
Intensity Normalization
Camera Scanline
Camera Scanline
Projected Intensity
P1 P2
P1 P2
Resultant Intensity
Resultant Intensity
Before Normalization
After Normalization
Projected Intensity
![Page 21: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/21.jpg)
Rendering
• Two pass rendering methodTwo pass rendering method
• First pass : Compute the desired imageFirst pass : Compute the desired image
• Second pass :Second pass :
– Warping with standard OpenGL texture Warping with standard OpenGL texture mappingmapping
– Intensity blending using Alpha channel of Intensity blending using Alpha channel of graphics hardwaregraphics hardware
• Two pass rendering methodTwo pass rendering method
• First pass : Compute the desired imageFirst pass : Compute the desired image
• Second pass :Second pass :
– Warping with standard OpenGL texture Warping with standard OpenGL texture mappingmapping
– Intensity blending using Alpha channel of Intensity blending using Alpha channel of graphics hardwaregraphics hardware
![Page 22: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/22.jpg)
Results: correct view for one user
Live Panoramic Video Image captured using a WFOV camera cluster
![Page 23: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/23.jpg)
Summary
• Robust general purpose method to create Robust general purpose method to create seamless imagesseamless images
• General configuration of projectors and display General configuration of projectors and display surfacessurfaces
• High-resolution wide-field of view display using High-resolution wide-field of view display using cluster of projectorscluster of projectors
• No expensive optics or electro-mechanical No expensive optics or electro-mechanical setupsetup
• One-time calibration procedureOne-time calibration procedure
• Robust general purpose method to create Robust general purpose method to create seamless imagesseamless images
• General configuration of projectors and display General configuration of projectors and display surfacessurfaces
• High-resolution wide-field of view display using High-resolution wide-field of view display using cluster of projectorscluster of projectors
• No expensive optics or electro-mechanical No expensive optics or electro-mechanical setupsetup
• One-time calibration procedureOne-time calibration procedure
![Page 24: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/24.jpg)
Applications
• Office Scenarios, flexible setupOffice Scenarios, flexible setup
• Dome shaped displays using cluster of Dome shaped displays using cluster of projectors instead of expensive opticsprojectors instead of expensive optics
• Ease of setup for cylindrical projection Ease of setup for cylindrical projection screens using multiple projectorsscreens using multiple projectors
• Synthetic images or Video images on large Synthetic images or Video images on large display environmentsdisplay environments
• Office Scenarios, flexible setupOffice Scenarios, flexible setup
• Dome shaped displays using cluster of Dome shaped displays using cluster of projectors instead of expensive opticsprojectors instead of expensive optics
• Ease of setup for cylindrical projection Ease of setup for cylindrical projection screens using multiple projectorsscreens using multiple projectors
• Synthetic images or Video images on large Synthetic images or Video images on large display environmentsdisplay environments
![Page 25: Seamless Projection Overlaps using Image Warping and Intensity Blending](https://reader035.vdocument.in/reader035/viewer/2022062321/56812ebc550346895d945d4a/html5/thumbnails/25.jpg)
Ten Years from now .. • Light bulbs replaced by Light bulbs replaced by
cheap projectors cheap projectors
• Sufficient number of Sufficient number of inexpensive camerasinexpensive cameras
• Widely available texture Widely available texture mapping graphics hardwaremapping graphics hardware
• Projectors, LCD panels to Projectors, LCD panels to create ‘desktop’ instead of create ‘desktop’ instead of looking at a 21’ monitorlooking at a 21’ monitor
• Light bulbs replaced by Light bulbs replaced by cheap projectors cheap projectors
• Sufficient number of Sufficient number of inexpensive camerasinexpensive cameras
• Widely available texture Widely available texture mapping graphics hardwaremapping graphics hardware
• Projectors, LCD panels to Projectors, LCD panels to create ‘desktop’ instead of create ‘desktop’ instead of looking at a 21’ monitorlooking at a 21’ monitor