image analysis - curses at dtu computecourses.compute.dtu.dk/02502/presentations/02502 -...

DTU Compute

Image Analysis

Plenty of slides adapted from Thomas Moeslunds lectures

Rasmus R. PaulsenTim B. DyrbyDTU Compute

[email protected]

http://www.compute.dtu.dk/courses/02502

mailto:[email protected]

DTU Compute

2019Image Analysis2 DTU Compute, Technical University of Denmark

Lecture 8 – Geometric Transformation

DTU Compute


What can you do after today? Compute the translation of a point Compute the rotation of a point Compute the scaling of a point Compute the shearing of a point Construct a translation, rotation, scaling, and shearing transformation

matrix Use transformation matrices to perform point transformations Describe the difference between forward and backward mapping Use bilinear interpolation to compute the interpolated value at a point Transform an image using backward mapping and bilinear interpolation Compute a line profile using bilinear interpolation Describe the homography Describe how the coefficients of a homography can be computed using

four corresponding points

DTU Compute


Geometric transformation Moving and changing the dimensions of images Why do we need it?

DTU Compute


Change detection Patient imaged before and after surgery What are the changes in the operated organ? Patient can not be placed in the exact same position in the

scanner

Bachelor project: Image Guided Surgery Planning

Before surgery After surgery

DTU Compute


Image Registration Change one of the images so it fits with the other Formally

– Template image– Reference image– Template is moved to fit the reference

Template Reference

Compute the difference

DTU Compute


Camera Calibration

2D projective transformations (homographies). Christiano Gava, Gabriele Bleser

DTU Compute


Geometric Transform The pixel intensities are not changed The “pixel values” just change positions

Same value

Different place

DTU Compute


Different transformations Translation Rotation Scaling Shearing

Homographies Advanced transformations

DTU Compute


Translation The image is shifted – both vertically and

horizontally

DTU Compute


Rotation The image is rotated around the centre or the upper left corner Remember to use degrees and radians correctly

– Matlab uses radians – Degrees easier for us humans

DTU Compute


Rotation coordinate system

y

x

DTU Compute


Rigid body transformation Translation and rotation Rigid body = stift legeme Angles and distances are kept

DTU Compute


Scaling The size of the image is changed Scale factors

– X-scale factor – Y-scale factor

Uniform scaling:

SxSy

Sx = Sy

DTU Compute


Similarity transformation Translation, rotation, and uniform scaling Angles are kept Distances change

DTU Compute


Shearing Pixel shifted horizontally or/and vertically Shearing factors

– X-shear factor – Y-shear factor

Is less used than translation, rotation, and scaling

BxBy

DTU Compute


Transformation Mathematics Transformation of positions Structure found at position (x,y)

in the input image f Now at position (x’,y’) in output

image g A mapping function is needed

f

g

(x,y)

(x’,y’)

DTU Compute


Translation mathematics The image is shifted – both vertically and

horizontally

(x,y)(x’,y’)

DTU Compute


Matrix notation Coordinates in column matrix format

DTU Compute


Translation mathematics in matrix notation The image is shifted – both vertically and

horizontally

DTU Compute


Scaling The size of the image is changed Scale factors

– X-scale factor – Y-scale factor

Uniform scaling:

SxSy

Sx = Sy𝑆𝑆𝑥𝑥 = 1.2

𝑆𝑆𝑦𝑦 = 0.9

DTU Compute


Matrix multiplication details

Is equal to:

DTU Compute


Transformation matrix

Can be written as

Where

is a transformation matrix

DTU Compute


Rotation A rotation matrix is used y

x

DTU Compute


Shearing Pixel shifted horizontally or/and vertically

New x value depends on x and y

DTU Compute


Affine transformation The collinearity relation between points, i.e., three

points which lie on a line continue to be collinear after the transformation

DTU Compute


Combining transformations Suppose you first want to

rotate by 5 degrees and then scale by 10%

Scaling

Rotation

How do we combine the transformations?

DTU Compute


Combining transformations Combination is done by matrix multiplication

Scaling

Rotation

Combined

DTU Compute


Combining transformations Compact notation

Scaling

Rotation

Combined

DTU Compute


Combining transformsA) 1B) 2C) 3D) 4E) 5

31

1 2 0 0

A B C D E

The point (x,y)=(5,6) is transformed. First with:

and then with:

The result is:

Chart1

A

B

C

D

E

31

1

2

0

0

Sheet1

A31

B1

C2

D0

E0

DTU Compute


What do we have now? We can pick a position in the input image f and find

it in the output image g

f g

We can transfer one pixel –what about the whole image?

DTU Compute


Solution 1 : Input-to-output

f

Scaling example

0 1 2 3

0

1

2

3

g

• Run through all pixel in input image

• Find position in output image and set output pixel value

0 1 2 3

0

1

2

3

x

y

x

y

Missing value!

23 216 120

DTU Compute


Input-to-Output The input to output transform is not good! It creates holes and other nasty looking stuff What do we do now?

DTU Compute


Some observations We want to fill all the pixels in the output image

– Not just the pixels that are “hit” by the pixels in the input image

Run through all pixels in the output image?– Pick the relevant pixels in the input image?

g

0 1 2 3

0

1

2

3

x

y

We need to go “backwards”

From the output to the input

DTU Compute


Inverse transformation We want to go from the output to the input

f

Scaling example

0 1 2 3

0

1

2

3

g

0 1 2 3

0

1

2

3

x

y

x

y

?

inverse

DTU Compute


Output-to-input transformationBackward mapping

f

0 1 2 3

0

1

2

3

g

0 1 2 3

0

1

2

3

x

y

x

y

• Run through all pixel in output image

• Find position in input image and get the value

What is the value here?

DTU Compute


Bilinear Interpolation The value is calculated from

4 neighbours The value is based on the

distance to the neighbours

Value?

Lots of 89!

Not so much 189

DTU Compute


Linear Interpolation

a b

V=?

1

t

𝑣𝑣 = 𝑡𝑡𝑡𝑡 + 1 − 𝑡𝑡 𝑎𝑎

DTU Compute


Bilinear interpolation

DTU Compute


Bilinearinterpolation

A) 1B) 2C) 3D) 4E) 5

5

28

0 2 0

A B C D E

Bilinear interpolation is used to create a line profile from an image. In a given point (x,y) = (173.1, 57.8), the four nearest pixels are:

What is the interpolated value in the point:

Chart1

A

B

C

D

E

5

28

0

2

0

Sheet1

A5

B28

C0

D2

E0

DTU Compute


Output-to-input transformationBackward mapping Run through all the pixel in the output image Use the inverse transformation to find the position in

the input image Use bilinear interpolation to calculate the value Put the value in the output image

DTU Compute


Inverse transformation We can calculate the inverse

transformation for the scaling

What about the others?Scaling

Inverse

DTU Compute


General inverse transformation The transformation is

expressed as a transformation matrix A

The matrix inverse of A gives the inverse transformation

Affine transformation

Inverse transformation

Where

DTU Compute


General inverse transformation

You can create an arbitrary sequence of rotation, scaling, and shearing

Scaling ScalingRotation

Inverse:

Combined:

DTU Compute


Transformation

A) 1B) 2C) 3D) 4E) 5

26

1 1 03

A B C D E

The point (x,y) = (45, 23) is transformed using:

And the result is translated with (-15, 20). The result is:

Chart1

A

B

C

D

E

26

1

1

0

3

Sheet1

A26

B1

C1

D0

E3

DTU Compute


Profile Analysis Sample the pixel values

under the profile From start point to end

point

DTU Compute


Profile Analysis Length of profile

Unit vector

Point on profile

ps

pe

DTU Compute


Profile Analysis Point on profile

Sample all points on profile using bi-linear interpolation

ps

pe

DTU Compute


Profile Analysis ps

pe ps pe

DTU Compute


Homography

DTU Compute


Homography Maps from one plane to

another plane Projective transformation

Mapping from coordinates (x,y) to (x’, y’)

DTU Compute


Virtual Billboard Example

https://www.learnopencv.com/homography-examples-using-opencv-python-c/


DTU Compute





DTU Compute




We want to estimate:

One point in each image gives:


DTU Compute




Eight unknowns:

Requires four corresponding points


DTU Compute




Equation system:

Can be solved using linear algebra (SVD):


DTU Compute




• For each pixel in the output image

• Compute the position in the billboard image using the homography

• If position is within the billboard

• Get the billboard pixel value and put it in the output image


DTU Compute


DTU Sign Challenge – exercise 8

DTU Compute


Data 34 photos with ground

truth (GT)

DTU Compute


Goal To create a function

that can locate signs Should create a label

image– Label 0: Background– Label 1: Sign 1– Label 2: Sign 2– …

DTU Compute


Supplied scripts and functions Some scripts and functions supplied MyDTUSignFinder.m

– Should be renamed into a new cool name

You should also make a commercial for your amazingmethod!– One or two powerpoint slides!

DTU Compute


How do we measure performance?

DTU Compute


DICE Scores A: area of ground truth B: area of your result 𝐴𝐴 ∩ 𝐵𝐵: area of

intersection/overlap

Residuals is the difference betweenground truth and yourresult

𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷 =2(𝐴𝐴 ∩ 𝐵𝐵)𝐴𝐴 + 𝐵𝐵

DTU Compute


DICE Scores

1 is perfect– No residuals

0 is really bad– Only residuals

𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷 =2(𝐴𝐴 ∩ 𝐵𝐵)𝐴𝐴 + 𝐵𝐵

DTU Compute


Multiple DICE scores• Average DICE score over found matches

• Penalises

• Too many sign candidates

• Too few sign candidates

DTU Compute


DTU Compute


signeTheSignFinder

DTU Compute


Hm..Nope!

Still can’t see

DTU Compute


DTU Compute


Next week – Company presentations

Visiopharm

Dalux IH Food

JLI Vision

Videometer

Presentations start at 9:30!

Image AnalysisLecture 8 – Geometric TransformationWhat can you do after today?Geometric transformationChange detectionImage RegistrationCamera CalibrationGeometric TransformDifferent transformationsTranslationRotationRotation coordinate systemRigid body transformationScalingSimilarity transformationShearingTransformation MathematicsTranslation mathematicsMatrix notationTranslation mathematics in matrix notationScalingMatrix multiplication detailsTransformation matrixRotationShearingAffine transformationCombining transformationsCombining transformationsCombining transformationsCombining transformsWhat do we have now?Solution 1 : Input-to-outputInput-to-OutputSome observationsInverse transformationOutput-to-input transformation�Backward mappingBilinear InterpolationLinear InterpolationBilinear interpolationBilinear�interpolationOutput-to-input transformation�Backward mappingInverse transformationGeneral inverse transformationGeneral inverse transformationTransformationProfile AnalysisProfile AnalysisProfile AnalysisProfile AnalysisHomographyHomographyVirtual Billboard ExampleVirtual Billboard ExampleVirtual Billboard ExampleVirtual Billboard ExampleVirtual Billboard ExampleVirtual Billboard ExampleDTU Sign Challenge – exercise 8DataGoalSupplied scripts and functionsHow do we measure performance?DICE ScoresDICE ScoresMultiple DICE scoresSlide Number 66Slide Number 67Slide Number 68Slide Number 69Slide Number 70Next week – Company presentations

image analysis - curses at dtu computecourses.compute.dtu.dk/02502/presentations/02502 -...

Documents