digital photography with flash and no-flash image pairs
Embed Size (px)
DESCRIPTION
Digital Photography with Flash and No-Flash Image Pairs. By: Georg PetschniggManeesh AgrawalaHugues HoppeRichard Szeliski Michael CohenKentaro Toyama, Microsoft Corporation Presented by : Yael Amsterdamer Advanced Topics in Visual Computing, Spring 2012. Introduction. - PowerPoint PPT PresentationTRANSCRIPT
Digital Photography with Flash and No-Flash Image PairsBy: Georg PetschniggManeesh AgrawalaHugues HoppeRichard SzeliskiMichael CohenKentaro Toyama, Microsoft Corporation
Presented by: Yael AmsterdamerAdvanced Topics in Visual Computing, Spring 2012
1The Dilemma: to Flash or not to flash?
Introduction
2Digital Photography with Flash and No-Flash Image PairsNatural lightingLow signal-to-noise ratio (SNR)Loss of detailsLonger exposure motion blurHarsh, unnatural lightingHigh SNRMore detailsMay cause unwanted artifacts (red eye, shadows, specularities)
2Why not both?The idea: use the good features of each photo to create a better image Introduction3Digital Photography with Flash and No-Flash Image Pairs
3Other works on combining images Introduction4Digital Photography with Flash and No-Flash Image Pairs
Different exposuresPanoramic viewRelightingCurrent work handles 2 images with specific characteristics,Envisioned for camera firmware 4Different ApplicationsDenoisingDetail transferSide requirement: handling flash induced shadows and specularitiesWhite BalancingContinuous Flash AdjustmentRed Eye RemovalIntroduction5Digital Photography with Flash and No-Flash Image Pairs5Camera Adjustment to lightA certain amount of light is needed for a photoTaking pictures of a dark sceneIncreasing exposure timeOpening the apertureIncreasing ISO (sensor sensitivity)The problem high ISO noise affects high-frequency details
Background6Digital Photography with Flash and No-Flash Image Pairs
6The InputF the photo with flashA the no-flash (ambient) photoComposed of 3 RGB intensity values for each pixelAp one pixel in ASame camera position, focus, apertureOtherwise, registration is requiredAdjusted ISO and exposure timeUndergo non-linear tone-mapping operationsBackground7Digital Photography with Flash and No-Flash Image Pairs71st Application: DenoisingA classic problem in image processingThe target: analytically reduce SNR One type of solution: smoothing filtersBilateral Filter [Tomasi and Manduchi 1998]Fast, non-iterativeAveraging close-by pixelsLow-pass filter, combined withEdge-stopping function
Denoising8Digital Photography with Flash and No-Flash Image Pairs8Bilateral filterHowever, results still have noise or blur (or both)Denoising9Digital Photography with Flash and No-Flash Image Pairs
ambientflash
Bilateral filter
9Improvement: Joint Bilateral FilterIn the flash image there are much more detailsWhy not use F to find edges?
Denoising10Digital Photography with Flash and No-Flash Image Pairs
Bilateral filterJoint Bilateral filter
The difference10Not all edges are real May cause over- or under-blur in joint bilateral filterWe need to eliminate their effect
Denoising11Digital Photography with Flash and No-Flash Image Pairs
11Detecting shadowsObservation: the pixels in the flash shadow should be similar to the ambient imageNot identical:NoiseInter-reflected flashCompute a shadow maskTake pixel p if is manually adjustedMask is smoothed and dilated
Denoising12Digital Photography with Flash and No-Flash Image Pairs
Pixels for which we will not use F12Detecting specularitiesTake pixels where sensor input is close to maximum (very bright)Over fixed threshold Create a specularity maskAlso smoothed and dilatedM the combination of shadow and specularity masksWhere Mp=1, we use ABase. For other pixels we use ANR.
Denoising13Digital Photography with Flash and No-Flash Image Pairs
13There is so much denoising can doIt cannot add details missing in the ambient imageExist in flash image because of high SNRWe use a quotient image:
Multiply with ANR to add the detailsMasked in the same way
Detail Transfer14Digital Photography with Flash and No-Flash Image Pairs
Bilateral filteredReduces the effect of noise in F
14Unified processDenoising + detail transfer + masking shadows and specularities
Detail Transfer15Digital Photography with Flash and No-Flash Image Pairs
15
More examples Detail Transfer16Digital Photography with Flash and No-Flash Image Pairs
16The opposite of preserving the atmosphereThe goal: simulate white light in the sceneWhat color is the light in the scene?Flash image can help us:The difference between F and A is only the flash lightAssume it is proportional to the surface albedoInherent property of the surfaceAssume the surface reflects light in the same color (will not work for plastic, glass)
White Balancing17Digital Photography with Flash and No-Flash Image Pairs17Computing the ambient lightWe cancel the color of the surface
The average c of Cp(col) for pixels with certain properties is the ambient light colorThen we scale A to cancel the ambient lightWhite Balancing18Digital Photography with Flash and No-Flash Image Pairs
18Example White Balancing19Digital Photography with Flash and No-Flash Image Pairs
Estimated ambient lightComputed colorAmbient original
White-balanced19Combining flash and no-flash imagesProviding complementary details about the sceneUse for denoising and detail transferUse for white balancingFuture workMore robustnessAutomatization of configuration
Summary20Digital Photography with Flash and No-Flash Image Pairs20
Thank you! 21