Slide 1

Digital Photography with Flash and No-Flash Image Pairs By: Georg PetschniggManeesh Agrawala Hugues HoppeRichard Szeliski Michael CohenKentaro Toyama, Microsoft Corporation Presented by: Yael Amsterdamer Advanced Topics in Visual Computing, Spring 2012

Slide 2

The Dilemma: to Flash or not to flash? 2Digital Photography with Flash and No-Flash Image Pairs Natural lighting Low signal-to-noise ratio (SNR) Loss of details Longer exposure motion blur Harsh, unnatural lighting High SNR More details May cause unwanted artifacts (red eye, shadows, specularities)

Slide 3

Why not both? The idea: use the good features of each photo to create a better image 3Digital Photography with Flash and No-Flash Image Pairs

Slide 4

Other works on combining images 4Digital Photography with Flash and No-Flash Image Pairs Different exposures Panoramic view Relighting Current work handles 2 images with specific characteristics, Envisioned for camera firmware Current work handles 2 images with specific characteristics, Envisioned for camera firmware

Slide 5

Different Applications Denoising Detail transfer Side requirement: handling flash induced shadows and specularities White Balancing Continuous Flash Adjustment Red Eye Removal 5Digital Photography with Flash and No-Flash Image Pairs

Slide 6

Camera Adjustment to light A certain amount of light is needed for a photo Taking pictures of a dark scene Increasing exposure time Opening the aperture Increasing ISO (sensor sensitivity) The problem high ISO noise affects high-frequency details 6Digital Photography with Flash and No-Flash Image Pairs

Slide 7

The Input F the photo with flash A the no-flash (ambient) photo Composed of 3 RGB intensity values for each pixel A p one pixel in A Same camera position, focus, aperture Otherwise, registration is required Adjusted ISO and exposure time Undergo non-linear tone-mapping operations 7Digital Photography with Flash and No-Flash Image Pairs

Slide 8

1st Application: Denoising A classic problem in image processing The target: analytically reduce SNR One type of solution: smoothing filters Bilateral Filter [Tomasi and Manduchi 1998] Fast, non-iterative Averaging close-by pixels Low-pass filter, combined with Edge-stopping function 8Digital Photography with Flash and No-Flash Image Pairs

Slide 9

Bilateral filter However, results still have noise or blur (or both) 9Digital Photography with Flash and No-Flash Image Pairs ambient flash Bilateral filter

Slide 10

Improvement: Joint Bilateral Filter In the flash image there are much more details Why not use F to find edges? 10Digital Photography with Flash and No-Flash Image Pairs Bilateral filter Joint Bilateral filter The difference

Slide 11

Not all edges are real May cause over- or under-blur in joint bilateral filter We need to eliminate their effect 11Digital Photography with Flash and No-Flash Image Pairs

Slide 12

Detecting shadows Observation: the pixels in the flash shadow should be similar to the ambient image Not identical: Noise Inter-reflected flash Compute a shadow mask Take pixel p if is manually adjusted Mask is smoothed and dilated 12Digital Photography with Flash and No-Flash Image Pairs Pixels for which we will not use F

Slide 13

Detecting specularities Take pixels where sensor input is close to maximum (very bright) Over fixed threshold Create a specularity mask Also smoothed and dilated M the combination of shadow and specularity masks Where M p =1, we use A Base. For other pixels we use A NR. 13Digital Photography with Flash and No-Flash Image Pairs

Slide 14

There is so much denoising can do It cannot add details missing in the ambient image Exist in flash image because of high SNR We use a quotient image: Multiply with A NR to add the details Masked in the same way 14Digital Photography with Flash and No-Flash Image Pairs Bilateral filtered Reduces the effect of noise in F

Slide 15

Unified process Denoising + detail transfer + masking shadows and specularities 15Digital Photography with Flash and No-Flash Image Pairs

Slide 16

More examples 16Digital Photography with Flash and No-Flash Image Pairs

Slide 17

The opposite of preserving the atmosphere The goal: simulate white light in the scene What color is the light in the scene? Flash image can help us: The difference between F and A is only the flash light Assume it is proportional to the surface albedo Inherent property of the surface Assume the surface reflects light in the same color (will not work for plastic, glass) 17Digital Photography with Flash and No-Flash Image Pairs

Slide 18

Computing the ambient light We cancel the color of the surface The average c of C p(col) for pixels with certain properties is the ambient light color Then we scale A to cancel the ambient light 18Digital Photography with Flash and No-Flash Image Pairs

Slide 19

Example 19Digital Photography with Flash and No-Flash Image Pairs Estimated ambient light Computed color Ambient original White- balanced

Slide 20

Combining flash and no-flash images Providing complementary details about the scene Use for denoising and detail transfer Use for white balancing Future work More robustness Automatization of configuration 20Digital Photography with Flash and No-Flash Image Pairs

Slide 21

Thank you!