statistical and numerical techniques for photorealistic image synthesis kartic subr
TRANSCRIPT
Statistical and numerical techniquesfor photorealistic image synthesis
Kartic Subr
Who am I?
• Born in India– Bangalore University (Bachelor of Engineering) 2001– Hewlett Packard, India/Singapore
• 6 years in USA– PhD, University of California Irvine, 2008– Advisor: Jim Arvo (PhD Yale University), pioneered methods in light transport
• 2 years in France– Post doctoral researcher, ARTIS, INRIA-Grenoble (2008-2010)
My goal: Generating realistic visuals
Gustave Courbet, Stone-Breakers, 1849.
Wilhelm Oswald Gustav Achenbach, Abendstimmung in der Campagna, 1850.
Realism in art
Gustave Courbet, Stone-Breakers, 1849.
Wilhelm Oswald Gustav Achenbach, Abendstimmung in der Campagna, 1850.
Realism in art
Photograph: Nicéphore Niépce, 1826
My goal: Generating realistic visuals
Notion of “realism” depends on technology
Gerhard Richter, 1983
Pedro Campos
Hyperrealism
Realistic image synthesis
Image synthesis involves light transport
Image ?
Digital models of scene (geometry + materials)
Light sources
Virtual camera
Image synthesis adds visual impact
AvatarCaptured video+ digital model
Digital model
Applications of image synthesis
Defense
Advertising
Entertainment
Virtual prototyping
Biomedical imaging
Multidisciplinary nature of the problem
• Physically based optical simulations
• Mathematical tools for analysis
• Numerical techniques for light transport solution
• Understanding biological processeseg. Perception et cognition
Reflection of light is an integration
Light transport: multi-domain integration
• Combinatorial explosion from sampling each domain
Image spaceAperture
Exposure time
Visible spectrum
Reflectance Direct illumination Indirect illumination
[Efficient sampling strategies for Monte Carlo integration (my PhD thesis)]
Talk outline
• Recent contributions– Simulating defocus– Rendering translucent materials
• Research plan– Core problems in image synthesis– Model representation and abstraction
My contributions: Fourier depth of field
Defocus blur is important in photography
Defocus is due to aperture integraion
Image Aperture
Pixel p
Lens
Defocus
Image Aperture
Pixel p
Lens Scene
Pixel p
Monte Carlo estimation of aperture integral
Image Aperture
NA primary rays per pixel
Integrateat p
Aperture integration is very costly
ImageAperture
NP pixels
NP x NA Primary rays
NA Aperture samples
64 x #primary rays of the pinhole image
Paradox: Blurry image is costlier to compute!
Observation 1: Image
Blurry regionsshould not require
dense samplingof the image
Observation 2: Lens
Regions in focusshould not requireprofuse samplingof the lens for diffuse objects
Fourier depth of field
• Fourier domain analysis of finite aperture cameras• Adaptive sampling• Speedup of around 20 over existing algorithms
[ACM Transactions on Graphics 2009. Presented at ACM SIGGRAPH 09]
Collaborators: MIT
My contributions: Translucent materials
TranslucentOpaque
Translucency: Sub-surface scattering
• Brute force Monte Carlo: prohibitively expensive• Diffusion approximations: severe constraints on geometry
Finite difference method on new domain
• Approximation: diffusion equation
• Domain: Dual graph of tetrahedralization
Diffuse flux
Rendering translucent materials
[Computer Graphics Forum 2010. To be presented at Eurographics 2010]
[Collaborators: Microsoft Research, Tsinghua University]
• Arbitrary geometry• Heterogenous materials• Dynamically deforming shapes• In real-time!
Research program
Realistic image synthesisModel representation
and abstraction
1. Realistic image synthesis
• Bandwidth driven sampling– Transport of local light field spectrum– Derive spatial / angular sampling rates– co-advising PhD student Laurent Belcour (ARTIS)
• Importance vs radiance– Tracing from eye vs tracing from light– Monte Carlo matrix chain multiplication
Short-term
Long-term
Importance vs radiance
Radiance
Importance vs radiance
Importance
Importance vs radiance
MC matrix-chain product estimator
MC matrix-chain product estimator
Related to optimal matrix chain multiplication using dynamic programming?
2. Model representation and abstraction
• Abstracting detail in geometry– First step: images (published at SIGGRAPH Asia 09)
• Alternate representation– Voxel data to represent geometry and materials
Short-term
Long-term
Detail = oscillations between extrema
Input
Local maxima
Local minima
Image multiscale decomposition
+
+
Medium
Pixels
Intensity
Input
Fine
Coarse
1D
Allows smoothing high-contrast detail
Input
Smoothed
Thank you!• Collaborators
– Established • MIT, USA• Microsoft Research• Tsinghua University, China• University of California, Irvine
– Current• Cornell University, USA• University of California, Berkeley
– Potential• Indian Institute of Information Technology
• International journal publications– Computer Graphics Forum 2010: Translucent materials. 4th author of 6– TOG 2009: Multiscale image decomposition. 1st author of 3– TOG 2009: Fourier Depth of Field. 2nd author out of 5
• Refereed international conference papers– Pacific Graphics 2007: Statistical hypotheses. 1st author of 2– Interactive raytracing 2007: Steerable importance sampling. 1st author of 2– ICIAP 2005: Contrast enhancement. 1st author of 3
Merci !• Collaborators
– Established • MIT, USA• Microsoft Research• Tsinghua University, China• University of California, Irvine• LJK Grenoble
– Current• Cornell University, USA• University of California, Berkeley
– Potential• Indian Institute of Information Technology
• International journal publications– Computer Graphics Forum 2010: Translucent materials. 4th author of 6– TOG 2009: Multiscale image decomposition. 1st author of 3– TOG 2009: Fourier Depth of Field. 2nd author out of 5
• Refereed international conference papers– Pacific Graphics 2007: Statistical hypotheses. 1st author of 2– Interactive raytracing 2007: Steerable importance sampling. 1st author of 2– ICIAP 2005: Contrast enhancement. 1st author of 3
• Teaching– Columbia University, USA (120 h)– University of California, Irvine (360 h)
• Industry– Rhythm and Hues Studios– NVIDIA Corporation– Hewlett Packard