the super resolution technology 2016

The SuperResolution

Technology

Welcome to our Webconference:

16.11.2012 Dr. Martin Stratmann/Stefan Wenz/Jennifer Just

SuperResolution with

Deconvolution in Detail

What is the Problem?

How can we solve it?

3/24 Testo AG, SuperResolution - Webconference (1.0), 1000len-juj, 15.11.2012, Confidentiality 2

Basic Problem: The Pixels

• Well known Problem:

Small objects are invisible in the image.

Even if visible, measured values are not correct.

• Obvious Solutions:

getting closer, (not always possible)

use tele lens, (have one?)

more pixels, (just a matter of money)

hot object

IR-Camera

long distance


Pushing the Limit

• No Solution: Interpolation

Increasing image size with interpolation does not help.

No new information gained.

• Our Solution: SuperResolution

Use multiple images and some mathematics.

Really get more information from multi-sampling.

• Result: Increase usable resolution by 1.6

Decrease effective pixel pitch by 1.6


Signal Chain of an IR-Camera

IR-lens & Aperture

T

x

60°C

Point Spread Funcion (PSF)

Modulation Transf. Fct (MTF)

Integration and

Sampling

Object IR-detector Electronics & Display Observer

Colormapping

Radiometric Computation


Se

nsitiv

ity

x

Limitations of Detectors and Basic Superresolution

Area of one Bolometer (pixel pitch ²)

Typical pixel pitch: 25µm

cut-off frequency: 20 [lp/mm] (Nyquist)

Super-Sampling: Take two images, slightly shifted, and combine them

x

Se

nsitiv

ity

Sensitive Area (only ~75%)


Naive Superresolution by Super-Sampling

• In Theory:

Take 4 images shifted by 1/2pixel pitch in x- and y-direction (super-sampling)

Combine them in a larger image.

• Problems:

How to shift images by exactly 12.5µm? (with a hand-held camera)

What about the Point Spread Function?


Point-Spread-Function

• PSF: How the camera blurres all points

• Lens: chromatic aberration, …

• Aperture: Diffraction, (Airy-Disc)

• For t875: f/# 1.0 -> Airy-Disc = 29 µm

Pixel size of t875 is 25µm

• Energy is distributed over several pixels

T

Airy function Airy disk

Input Output


Convolution and Deconvolution

• Blurring of images: Convolution with Gaussian distribution curve.

• Convolution:

Sending an input signal through a system result in an output signal.

The system applies its transfer function: So = T * Si

The transfer function can be specified as point-spread-function

• Inverting the process of convolution:

Try to reconstruct the signal which has been sent into a system by only

knowing the output signal.

For a given So and T, find Si such that So = T * Si holds.

Deconvolution of images: De-blurring / sharpening

De-blurring: refocus energy from region

Can not reconstruct lost information.

*

T

Si

So

=


Put it all together

• Deconvolution can reduce the blur

• Super-Sampling can reconstruct details

• Solution:

Combine super-sampling with deconvolution.

• Even better:

Use natural motion and compute image shifts

• Required:

Some arbitrary motion, ← Provided by the user.

A smart algorithm. ← Provided by Testo.


Steps from Science to Customer

• Developed algorithm together with a partner

• Main issue: Computational time required for 5 images of 160x120:

Start of optimisation: 30 s

Now: 100 ms

• Super-Resolution will be available for all current testo cameras

No hardware required, just software.

User can update its own cameras.

Results

-

Is it really better then

Interpolation?


Interpolation vs. Super-Resolution

Sensor resolution: 160 x 120 (original) / 320 x 240 (superresolved)

Is it really better then Interpolation? Yes it is!


Experimental Evidence

• MTF: Standard way to specify quality of optical systems

Measured in [line pairs / millimeter]

ISO 12233: slanted edge (problematic for processed images)

Here: Variable Frequency Target

• IFOVmeas: That is what really matters for thermal imaging

Setup with black body at reference temp

Vary aperture in front

Observe maximum

Similiar to CNPP Test


Variable Frequency Bar-Target Analysis

Cubic Interpolation Super-Resolution

- Analyse Modulation Transfer Function by observing the bar target.

- Measuring contrast and frequency


Signal Modulation

Normal Image

false temperature

Superresolution

aliasing


Detectable Spatial Frequencies

Cubic Interpolation

0

5

10

15

20

25

30

0 5 10 15 20 25 30 35

Frequency from bar target [lp/mm]

Fre

qu

en

cy f

rom

im

ag

e [

lp/m

m]

Superresolution

0

5

10

15

20

25

30

0 5 10 15 20 25 30 35

Frequency from bar target [lp/mm]

Fre

qu

en

cy

fro

m im

ag

e [

lp/m

m]

no frequencies above detector cut-off

detectable

frequencies approx. 50% above detector cut-off

detectable


Modulation Transfer Function

0,00%

20,00%

40,00%

60,00%

80,00%

100,00%

0 5 10 15 20 25 Spatial Resolution [lp/mm]

Super-Resolution

Normal

Diffraction Limit

Testo 881, 160x120, pixel pitch 25µm, F0,84

x1.6


Variable Aperture

• Setup: Black Body at 60°C,

variable aperture ( d = 2-32mm), camera at 1.6m distance

60°C

160cm

d = 32mm d = 6mm

Evaluating temperature of the hottest spot (Max-on-area)


Temperature Measurement of Variable Aperture

59,4 58,857,5

56,1

52,7

40,8

60,959,9

61,960,7

62,5

43,2

30

35

40

45

50

55

60

65

32 16 8 6 4 2

aperture diameter [mm]

T [

°C]

Normal

Superresolution

320x240 pixel, 25µm pixel pitch, 32°-Lens, Distance = 160cm

False Positive

IFOVmeas/1.6 = 4.8mm IFOVmeas =7.8mm

False Negative


Image Comparison

testo 875i Normal

testo 875i Superresolution

testo 882 Normal

testo 882 Superresolution

For 640x480 Sensor:

SuperResolution: 1280x960


Close-ups

testo 875i Normal

testo 875i Superresolution

esto 882 Normal

testo 882 Superresolution


Conclusions

• Superresolution + Deconvolution:

Combine several images

Reduce blurring

Use natural motion

• Result:

Increase spatial resolution by 1.6

Decrease effective pixel pitch by 1.6

• Available as software-update for all

current Testo-IR-Cameras

Even for already sold cameras.

Testo SuperResolution:

Get the most out of Your Pixels.

Thank you for your attention!

Any questions?