resolution loss without optical blur tali treibitz alex golts yoav y. schechner technion, israel 1

Resolution Loss without

Optical Blur

Tali Treibitz

Alex Golts

Yoav Y. Schechner

Technion , Israel

1

14

airlightA

( , ) ( , ) ( ) ( , )D

I x y L x y t z A x y

object

0

1

z

ze

scatteringdirect transmission

D

object radiance

Lobject

total intensity

I

A

0 z

1 ze

Schechner, Narasimhan, Nayar

Haze

object( ) ( ) ( ) ( )I l t A x x x x

airlightobject transmittance

Schechner et al., Applied Optics ‘03

15

Pointwise Degradations

object( )I lx

object

( )x t ( )x A

pointwise attenuation:

vignetting

atmosphere attenuation

additive component:

reflection

glare

path radiance

( )x n

noise

Treibitz & Schechner, Recovery Limits in Pointwise Degradations

17


object( )I lx

object

( )x t ( )x A

pointwise attenuation

( )x n

noise


reduce SNR even if known

additive (positive) bias

18

Noise: Object size matters?19

Noise: Object size matters?

0.5

20

1


1.5


depends on:

noise level

object background intensity difference

object size

quantify this dependency!

Prior art: resolution limits due to optical blur

here: no optical blur

Visibility Under Noise23

Previous criteria• Is there something there?• Is it a tank?• What type is it?

Johnson charts:

identification recognition orientation detection

7 3.5 1.2 0.75 tank

minimum line pairs for 50% success

NIIRS- National Image Interpretability Rating Scales

Identify the wing configuration (e.g., straight, swept, delta) of all large aircraft (e.g., 707, CONCORD, BEAR, BLACKJACK). ..

Detect large hangars at airfields. Detect large static radars (e.g., AN/FPS-85, COBRA DANE, PECHORA, HENHOUSE), Detect military training areas...

Detect a medium-sized port facility and/or distinguish between taxi-ways and runways at a large airfield.

Interpretability of the imagery is precluded by obscuration, degradation, or very poor resolution

Identify small light-toned ceramic insulators that connect wires of an antenna. Identify vehicle registration numbers (VRN) on trucks. Identify screws and bolts on missile components. ..

0

1

2

3

9

pattern visible


24

Where is the Cutoff?

0.1 0.5

10

0.5

pattern visible

pattern invisible calculated analytically!


Input SNR

noise

| ( ) |

S u

25

(frequency)u

0.1 0.5

10

0.5

(frequency)


u

26

Cutoff Per Success Rate

success rate 50%

cutoffu

1

Input SNR

noise

| ( ) |

S u

Noise Suppression in the HVS

Theoretical Neuroscience, Dayan & Abbott

frequency (cycles/degree)

response of receptive field

low noise

high noise

We derive: fundamental analytical model

Model: simple linear denoising

not a denoising method

28

SNR Improvement by Averaging

WH

/W

signal

noise


output

input

SNRC(u,W)=

SNR- SNR change after averaging

29

Different Sizes of Windows

too big for signal too small for noise


30

Averaging by Optimal Window


31

SNR Improvement by Averaging

outputmax input( ) SNR u SNR

depends on frequency!

u


max ( )C u

same plot for a Gaussian filter

32

Output SNR

0.1 0.5

6.5

0.32


cutoffu

output 1SNR output 0.5SNR

Input SNR

noise

| ( ) |

S u

(frequency)u

1

33


0.1 0.5

0.32

success rate 70%

success rate 40%


Input SNR

(frequency)ucutoffu6.5

noise

| ( ) |

S u

34

Vision Success is Probabilistic

SNR=2/3

SNR determines chances of visibility

visible invisible


35

Success within a Confidence Interval

( ) prob N S xSN

68%

0 22

- success rate

SNRTreibitz & Schechner, Recovery Limits in Pointwise Degradations

0.68

Object is visible if ( )N Sx

…depends on SNR and..

( )xN

36

25

Success within a Confidence Interval

- success rate

SNRTreibitz & Schechner, Recovery Limits in Pointwise Degradations

what is the probability for correct detection?

depends on SNR object pixel

background pixel

visibility is kept if edge keeps sign

background object ( ) ( ) x x prob N N S

%(sign kept) - %(wrong sign)

noisyclear

0.5

0.1 0.5

6.5

0.32

system input SNR

cutoffu

Determining Resolution Limits

cutoff for ρ=70% success

frequency


noise

| ( ) |

S u

37


object( )I lx ( )x t ( )x A

pointwise attenuation:

vignetting

atmosphere attenuation

additive component:

reflection

glare

haze

( )x n

noise


38

Noise Model

2 2 / I g

2 200 I

Nikon D100

2

photon noise dominates

39


Detector (pixel)

50% quantum efficiency

Schechner

Photon (shot) Noise9

Electrons

Photon

or

e{nothing

either

50% quantum efficiency

Schechner

10

Photons

Electrons

e{nothing

e e

either

Photon (shot) Noise

object background| | , ( )

SSNR = S l l t x

SNR per size (frequency)

object background| | ,

SSNR = S l l

objectl

backgroundl


backgroundlobjectl

41

Resolution Limits in Haze

distance [km]

limit due to pixel size

limit due to atmosphere


minimal visible object size[m]

cutoffu

reciprocal to

42

0.1 0.5

10

0.5

(frequency)


u

43


success rate 50%

cutoffu

1

Input SNR

noise

| ( ) |

S u

Haze in the Galilee


average of 50 framesraw frame

limit due to noise and not blur

44

What now?

What are the reconstruction limits?

What is the minimal detectable object size?

What camera noise properties are acceptable for detection?

…

45

Imaging in Haze

Treibitz & Schechner, Polarization- Beneficial for Visibility Enhancement?

46


Haze Through a Polarizer

best polarized image

47


single frame- used by photographers

Haze Through a Polarizer

increased exposure time


48


two frames- Schechner et al.

Dehazing using a Polarizer

post-processing 2 frames


worst polarized image

49

goal: object detection local contrast stretch- OK


Is it worth using a polarizer?

unpolarized image

best polarized

image

post-processing 2 frames

rarely!under the constraint of equal acquisition time

50

degree of polarization

( , ) 1( ) ( , )

2 2

D x y pI x, y A x ymin

0,1p

Using a Single Polarized Image

Best polarized image

I min

51


SNR ComparisonIunpolarized Ibest

52


p

A Single Saturated Frame


SNRpolarized

SNRunpolarized

maximal value in nature

53

SNR ComparisonIunpolarized Ibest

equal acquisition time

technical details

in the paper

acquisition time = exposure time X number of frames

54


Same Total Acquisition Time

p<0.4in our experiments

SNRpolarized

SNRunpolarized

maximal value in nature

p

55

Experiment

Wide field of view

bestI

average of 2 frames

unpolarizedI


same total acquisition time

56

dehazing

tbest tworst

1 p

p

1 p

p

SNR ComparisonIunpolarized

technical details

in the paper

optimal exposures

equal acquisition time

57

Advantages of Polarization

distance map

• contrast stretch in non-uniform distances

• restoring color• compensating for attenuation

58

• Freq cutoff – due to noise – without imaging blur

• Relation between cutoff and success rate

• Application: limits in pointwise degradations

Limits in Pointwise Degradations

• Case study of performance trade-offs

59

resolution loss without optical blur tali treibitz alex golts yoav y. schechner technion, israel 1

Documents