mobile robotics and olfaction lab, - Örebro university robotics and olfaction lab, aass, Örebro...

Mobile Robotics and Olfaction Lab, AASS, Örebro University

# 1

Achim J. Lilienthal

Room T1227, Mo, 11-12 o'clock (please drop me an email in advance)

[email protected]

# 3 DIP'14 © A. J. Lilienthal (Dec 16, 2014)

Now the Exam!?!

0


Allowed Aids?

Exam?


First Rule

Exam?


Grading – ECTS o labs x 3

» all need to be passed » extra points for the exam possible (1/10 of the exam per lab)

o exam » counts 100% » > 50% of the points to pass » another example

• exam has 100 points, you have 63 + 15 = 77 points

Examination

<50 50 – 59 60 – 69 70 – 79 80 – 89 90 – 100 – E D C B A

<50 50 – 74 75 – 100 U G VG


Relevant Topics?

Examination


Part 1 - Introduction

1


1.

Digital Images

Course Introduction

94 100 104 119 125 136 143 153 157 158

103 104 106 98 103 119 141 155 159 160

109 136 136 123 95 78 117 149 155 160

110 130 144 149 129 78 97 151 161 158

109 137 178 167 119 78 101 185 188 161

100 143 167 134 87 85 134 216 209 172

104 123 166 161 155 160 205 229 218 181

125 131 172 179 180 208 238 237 228 200

131 148 172 175 188 228 239 238 228 206

161 169 162 163 193 228 230 237 220 199

magnification of the rat’s nose


1.

Digital Images

Human Visual Perception o the human eye

Course Introduction


1.

Digital Images

Human Visual Perception, E-M Spectrum

Course Introduction

νλ c=

νhE =


1.

Digital Images


Image Formation – Pinhole Camera Model

Course Introduction

from Per-Erik Forssén "Visual Object Recognition"


1.

Example Task

Exam?

Answer the following statements with either True or False. Note that you do not have to answer all the questions. Correct answers will be counted as +1 point and wrong answers as -1 point. Thus you should only answer those questions you are certain about. The total for this task cannot be, however, below 0. If the question ends with the sentence "explain your answer", you can get additional points by providing a more detailed answer to the corresponding question. No minus points will be assigned if the detailed answer is wrong.

a) The relation between subjective brightness and light intensity is approximately a log function. (1p) b) The visible spectrum can be specified in units of energy or in

wavelengths. Explain your answer. (3p)


1.

Digital Images



Image Formation – Illumination / Reflectivity o f(x,y) = i(x,y) r(x,y)

Course Introduction


1.

Digital Images



Image Formation – Illumination / Reflectivity

Image Sampling and Quantization

Course Introduction


1.

Digital Images





Zooming and Shrinking o bilinear interpolation vs.

nearest neighbour interpolation

Course Introduction


1.

Digital Images





Zooming and Shrinking and Rotation o bilinear interpolation vs.

nearest neighbour interpolation

Course Introduction


Part 2 – Image Enhancement in the Spatial Domain

2


2.

Image Enhancement

Spatial Domain Image Enhancement

more suitable for visual interpretation


2.

Image Enhancement

Grey Level Transformations o 1x1 neighbourhood (point processing)


)(rTs =)(rTs =


2.

Matlab Questions?

Exam?

Which of the three images on the right is image fp (see Matlab code below) if the image on the left is f?

f = imread('bubbles.tif'); fp = imadjust(f, [0.1 0.9], [1.0 0.0], 0.5); imshow(fp);

f fp? (a) fp? (b) fp? (c)

⇒


2.

Image Enhancement

Grey Level Transformations

Histogram Processing o what is a histogram o histogram equalization



2.

Image Enhancement

Grey Level Transformations

Histogram Processing o what is a histogram o histogram equalization o adaptive / localized

histogram equalization



2.

Linear Spatial Filtering o neighbourhood relations

» 4-neighborhood, 8-neighborhood

Spatial Filtering


2.

Linear Spatial Filtering o neighbourhood relations o smoothing filters

» mean filter (box filter), Gaussian filter

Spatial Filtering


2.

Linear Spatial Filtering o neighbourhood relations o smoothing filters

» mean filter (box filter), Gaussian filter

Non-Linear Spatial Filtering » median filter (order statistics filter)

Spatial Filtering


2.

Linear Spatial Filtering o neighbourhood relations o smoothing filters o sharpening filters (Sobel)

Spatial Filtering

1 0 -1

2 0 -2

1 0 -1

1 -2 -1

0 0 0

1 2 1


2.

Linear Spatial Filtering o neighbourhood relations o smoothing filters o sharpening filters (Sobel, Laplacian)

Spatial Filtering

0 -1 0

-1 5 -1

0 -1 0


2.

Linear Spatial Filtering o neighbourhood relations o smoothing filters o sharpening filters (Sobel, Laplacian, Unsharp Masking)

Spatial Filtering


Part 3 – Bilateral Filtering

3


3.

Definition of the Bilateral Filter

Bilateral Filtering

space weight

not new

range weight

I

new

normalization factor

new

( ) ( )∑∈

−−=S

IIIGGW

IBFq

qqpp

p qp ||||||1][rs σσ


3.


Basic Properties o not linear!

Bilateral Filtering


3.


Basic Properties o influence of parameters, typical values

Bilateral Filtering

σs = 2

σs = 6

σr = 0.1 σr = 0.25 σr = ∞

(Gaussian blur)


3.


Basic Properties

Iterative Bilateral Filtering

Bilateral Filtering


3.


Basic Properties


Bilateral Filtering Color Images

Bilateral Filtering

( ) ( )∑∈

−−=S

GGW

IBFq

qqpp

p CCCqp ||||||||1][rs σσ

3D vector (RGB, Lab, …)


3.


Basic Properties


Bilateral Filtering Color Images

Many Applications o general idea useful in many different contexts

Bilateral Filtering


Part 4 – Fourier Transform and Filtering in the Frequency Domain

4


4.

Discrete Fourier Transform

Filtering in the Frequency Domain

→

sampled (0, 1, …, 128-1) spectrum = |F(u)| →


4.

Discrete Fourier Transform o complex numbers!


Euler’s formula )sin()cos( φφφ ⋅+=− je j

22 )))((Im()))((Re()( uFuFuF +=

]))(Re())(Im([tan)( 1

uFuFu −=φ

spectrum

phase angle


4.

Discrete Fourier Transform, 2D


∑∑−

=

−

=

+−=1

0

1

0

)//(2),(1),(M

x

N

y

NvyMuxjeyxfMN

vuF π

1...,,1,0 −= Nv1...,,1,0 −= Mu


4.



⋅ (-1)x+y

•F

log

→

c · log(1+|F(u,v)|)


4.




4.




Input Image ⋅ (-1)x+y F(u,v)

center the transform DFT

⋅ H(u,v)

apply filter

f(x,y)

inverse DFT

Re[f(x,y)]

Output Image

⋅ (-1)x+y


4.


Filtering in the Frequency Domain o ideal low-pass, high-pass filters

» ringing



4.

Understanding Questions o why does ringing occur?

Exam?


4.



» ringing o convolution theorem


),(),(),(),( vuHvuFyxhyxf ⋅⇔∗

),(),(),(),( vuHvuFyxhyxf ∗⇔⋅


4.




» link between spatial and frequency domain

» wrap around problem importance of padding


),(),( vuHyxh ⇔


4.




» link between spatial and frequency domain o Gaussian / Laplacian filter in the frequency domain



4.



Homomorphic Filtering, Intrinsic Images o separate

illumination and reflectance o log separable


γL<1

γH >1


4.



Homomorphic Filtering

Properties of the Fourier Transform o distributive? (over addition? over multiplication?) o scaling?, rotation?, ... o separability



4.




Properties of the Fourier Transform

Correlation, Template Matching



4.






Nyquist-Shannon Sampling Theorem o Moiré pattern o reconstruction of a continuous signal

» consider band-limited signals



4.






Nyquist-Shannon Sampling Theorem o Moiré pattern o reconstruction of continuous signal

» consider band-limited signals



4.






Nyquist-Shannon Sampling Theorem

Fast Fourier Transform o divide and conquer algorithm o M log2 M operations (vs. M2)



Part 5 – Image Restoration

5


5.

Image Degradation and Restoration o model of the image degradation process

Image Restoration

Hdeg(x,y) f(x,y) +

n(x,y)

g(x,y)


5.

Image Degradation and Restoration

Noise Models o Gaussian, uniform, salt and pepper, ...

Image Restoration


5.


Noise Models

Noise Reduction o assume additive noise

» uncorrelated with the image o estimation of noise shape and noise parameters

» by observation » by experimentation » by modelling

o apply appropriate filter (mean, median, adaptive mean, ...)

Image Restoration


5.

Adaptive Mean Filter

o behaviour based on the local variance (σL average contrast) o if the noise has zero variance (σn = 0) then f(x,y)=g(x,y) o if the local variance σL is large compared to the noise variance σn then

f(x,y)≈g(x,y) → preserve edges

o if σL ≈ σn then f(x,y)≈mL (mean over the neighbourhood) → average out noise

Noise Reduction

( ) [ ]LL

n myxgyxgyxf −−= ),(),(,ˆ2

2

σσ

^

^

^


5.


Noise Models

Noise Reduction o apply appropriate filter (notch filter)

» additive, periodic (approximately sinusoidal) noise

Image Restoration

F

→


5.


Noise Models

Noise Reduction

Image Restoration, LPI Degradation Processes o Linear, Position Invariant (LPI) Degradation Processes

Image Restoration

),(),(),(),( yxnyxfyxhyxg +∗=

),(),(),(),( vuvuFvuHvuG Ν+=

[ ]),(),( deg βαδβα −−≡−− yxHyxhimpulse response / point spread function (PSF)


5.


Noise Models

Noise Reduction

Image Restoration, LPI Degradation Processes o Linear, Position Invariant (LPI) Degradation Processes o Inverse Filtering, Deconvolution

Image Restoration

),(),(),(

),(),(),(),(0),(

vuHvuGvuF

vuvuFvuHvuGvu

=⇔

Ν+==Ν


5.


Noise Models

Noise Reduction

Image Restoration, LPI Degradation Processes o Linear, Position Invariant (LPI) Degradation Processes o Inverse Filtering, Deconvolution o estimating H

» by experimentation, image observation, modelling

Image Restoration


5.


Noise Models

Noise Reduction


Wiener Filtering

Image Restoration

),(ˆ),(

),(/),(ˆ),(ˆ

),(ˆ),(ˆ

2

2

vuHvuG

vuSvuSvuH

vuHvuF

fn

+=


5.


Noise Models

Noise Reduction


Wiener Filtering

Image Restoration

),(ˆ),(

),(ˆ

),(ˆ),(ˆ

2

2

vuHvuG

KvuH

vuHvuF

+=


Part 6 – Colour Image Processing

6


6.

Colour Fundamentals

Colour Image Processing

detector rods and cones


6.

Colour Fundamentals o brightness, hue, saturation, chromaticity (hue+saturation)



6.

Colour Fundamentals

Colour Models o CIE

» colour perceived by a standard observer (gamut of human vision)

» tri-stimulus model



6.

Colour Fundamentals

Colour Models o CIE o RGB, CMYK



6.

Colour Fundamentals

Colour Models o CIE o RGB, CMYK, HSI



6.

Colour Fundamentals

Colour Models

Pseudo Color Processing



6.

Colour Fundamentals

Colour Models


Colour Transformations o per-colour-component or vector transformation? o histogram equalization

» better in HSI or RGB?


in

out

in

out

in

out


6.

Colour Fundamentals

Colour Models


Colour Transformations

Smoothing and Sharpening of Colour Images



Part 8 – Image Segmentation

8


8.

Image Segmentation o in general a very difficult problem o full?, partial? o edge-based, region-based, motion-based

» idea: search for discontinuities and/or similarities in the image

Image Segmentation


8.

Image Segmentation

Edge Detection o edge model

Image Segmentation


8.

Image Segmentation

Edge Detection o edge model o edge detectors

» Prewitt, Sobel » Laplacian of a Gaussian

Image Segmentation


8.

Image Segmentation


» Prewitt, Sobel, Laplacian of a Gaussian » Canny edge detector

Image Segmentation


8.

Image Segmentation



• noise reduction • edge enhancement • determination of the intensity gradient • non-maximum suppression • tracing edges through the image

• thresholds Thigh and Tlow

Image Segmentation


8.

Image Segmentation



• parameter selection (Gaussian filter, hysteresis thresholds)

Image Segmentation


8.

Image Segmentation



o edge detection in colour Images

Image Segmentation


8.

Image Segmentation

Edge Detection

Edge Linking and Boundary Detection o local processing (link in neighbourhood) o global processing (Hough transform)

Image Segmentation


Part 8 – Image Segmentation (Deep Questions)

8


8.

What is Noise? What is Texture? o wet sand by Jay Sekora

Deep Questions


8.

What is Noise? What is Texture? o human skin by Ken Perlin

Deep Questions


8.

What is an Edge in Human Vision? o illusory contours by G. Kanizsa (1955)

Deep Questions


8.

Does Absolute Intensity Matter? o often perceived intensity ≠ pixel values

o center strip has constant intensity

Deep Questions


8.

What is an Illumination Edge? o sometimes it isn’t a large intensity change …

Deep Questions


8.

What is a Geometric Edge in Images? o some silhouettes are

suggested by shape cues » 3D "peanut" shape

Deep Questions


8.

What Is an Edge at The Finest Scales? o scale problems:

can’t resolve every hair and fiber in fur » Albrecht Dürer (1502)

Deep Questions


8.

Very Difficult Images o edge? o noise? o regions? o texture? o silhouette? o …

Deep Questions

Mobile Robotics and Olfaction Lab, AASS, Örebro University

# 150

Achim J. Lilienthal

Room T1227, Mo, 11-12 o'clock (please drop me an email in advance)

[email protected]

mobile robotics and olfaction lab, - Örebro university robotics and olfaction lab, aass, Örebro...

Documents