ACTAUNIVERSITATIS

UPSALIENSISUPPSALA

2019

Digital Comprehensive Summaries of Uppsala Dissertationsfrom the Faculty of Science and Technology 1869

Precise Image-BasedMeasurements through IrregularSampling

TEO ASPLUND

ISSN 1651-6214ISBN 978-91-513-0783-1urn:nbn:se:uu:diva-395205

Dissertation presented at Uppsala University to be publicly examined in Room 2446, ITC,Lägerhyddsvägen 2, Uppsala, Friday, 6 December 2019 at 13:00 for the degree of Doctorof Philosophy. The examination will be conducted in English. Faculty examiner: ProfessorHugues Talbot (Université Paris-Saclay).

AbstractAsplund, T. 2019. Precise Image-Based Measurements through Irregular Sampling.(Noggranna bildbaserade mätningar via irreguljär sampling). Digital ComprehensiveSummaries of Uppsala Dissertations from the Faculty of Science and Technology 1869. 63 pp.Uppsala: Acta Universitatis Upsaliensis. ISBN 978-91-513-0783-1.

Mathematical morphology is a theory that is applicable broadly in signal processing, but inthis thesis we focus mainly on image data. Fundamental concepts of morphology include thestructuring element and the four operators: dilation, erosion, closing, and opening. One wayof thinking about the role of the structuring element is as a probe, which traverses the signal(e.g. the image) systematically and inspects how well it "fits" in a certain sense that dependson the operator.

Although morphology is defined in the discrete as well as in the continuous domain, oftenonly the discrete case is considered in practice. However, commonly digital images are arepresentation of continuous reality and thus it is of interest to maintain a correspondencebetween mathematical morphology operating in the discrete and in the continuous domain.Therefore, much of this thesis investigates how to better approximate continuous morphologyin the discrete domain. We present a number of issues relating to this goal when applyingmorphology in the regular, discrete case, and show that allowing for irregularly sampled signalscan improve this approximation, since moving to irregularly sampled signals frees us fromconstraints (namely those imposed by the sampling lattice) that harm the correspondence inthe regular case. The thesis develops a framework for applying morphology in the irregularcase, using a wide range of structuring elements, including non-flat structuring elements (orstructuring functions) and adaptive morphology. This proposed framework is then shown tobetter approximate continuous morphology than its regular, discrete counterpart.

Additionally, the thesis contains work dealing with regularly sampled images using regular,discrete morphology and weighting to improve results. However, these cases can be interpretedas specific instances of irregularly sampled signals, thus naturally connecting them to theoverarching theme of irregular sampling, precise measurements, and mathematical morphology.

Keywords: image analysis, image processing, mathematical morphology, irregular sampling,adaptive morphology, missing samples, continuous morphology, path opening.

Teo Asplund, Department of Information Technology, Division of Visual Information andInteraction, Box 337, Uppsala University, SE-751 05 Uppsala, Sweden. Department ofInformation Technology, Computerized Image Analysis and Human-Computer Interaction,Box 337, Uppsala University, SE-75105 Uppsala, Sweden.

© Teo Asplund 2019

ISSN 1651-6214ISBN 978-91-513-0783-1urn:nbn:se:uu:diva-395205 (http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-395205)

Dedicated to my familyand my friends

List of papers

This thesis is based on the following papers, which are referred to in the text

by their Roman numerals.

I Asplund, T., Luengo Hendriks, C. L., Thurley, M., and Strand, R.

(2017) Mathematical Morphology on Irregularly Sampled Data in OneDimension, In Mathematical Morphology - Theory and Applications

2.1, pp. 1-24.

II Asplund, T., Luengo Hendriks, C. L., Thurley, M., and Strand, R.

(2017) Mathematical Morphology on Irregularly Sampled Signals, In

Computer Vision - ACCV 2016 Workshops. LNCS, vol 10117,

pp. 506-520

III Asplund, T., Serna, A., Marcotegui, B., Strand, R., and Luengo

Hendriks, C. L. (2019) Mathematical Morphology on IrregularlySampled Data Applied to Segmentation of 3D Point Clouds of UrbanScenes, In Mathematical Morphology and Its Applications to Signal

and Image Processing. ISMM 2019. LNCS, vol 11564, pp. 375-387

IV Asplund, T., Luengo Hendriks, C. L., Thurley, M., and Strand, R.

(2019) Adaptive Mathematical Morphology on Irregularly SampledSignals in Two Dimensions, (submitted)

V Asplund, T., Luengo Hendriks, C. L., Thurley, M., and Strand, R.

(2019) Estimating the Gradient for Images with Missing Samples UsingElliptical Structuring Elements, (submitted)

VI Asplund, T., and Luengo Hendriks, C. L. (2016) A Faster, UnbiasedPath Opening by Upper Skeletonization and Weighted AdjacencyGraphs, In IEEE Transactions on Image Processing 25.12,

pp. 5589-5600.

©2016 IEEE. Reprinted, with permission from the authors.

Reprints were made with permission from the publishers.

Related work

During the research for this thesis the author has also contributed to the

following publications.

I Asplund, T., Luengo Hendriks, C. L., Thurley, M., and Strand, R.

(2016) A New Approach to Mathematical Morphology on OneDimensional Sampled Signals, In Proceedings of 23rd International

Conference on Pattern Recognition, IEEE pp. 3904-3909.

II Asplund, T., Luengo Hendriks, C. L., Thurley, M., and Strand, R.

(2017) Approximating Continuous One-Dimensional Morphology byIrregular Sampling, In Proceedings of the Swedish Symposium on

Image Analysis (SSBA 2017)

III Asplund, T., Bengtsson Bernander, K., and Breznik, E. (2019) CNNson Graphs: A New Pooling Approach and Similarities to MathematicalMorphology, In Proceedings of the Swedish Symposium on Deep

Learning (SSDL 2019)

Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

1.3 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2 Mathematical Morphology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.1 Basic Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.1.1 Dilation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.1.2 Erosion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.1.3 Hit-and-Miss Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.1.4 Closing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.1.5 Opening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.2 Umbra and Top . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.3 Continuous Morphology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.4 Skeletonization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.4.1 Desirable Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.4.2 Digital Skeleton . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.5 Geodesic Morphology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.5.1 Reconstruction by Dilation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.6 Granulometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.7 Morphological Gradient and Laplacian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.8 Tophat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

2.9 Adaptive Morphology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.10 Path Opening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.10.1 Constrained Path Opening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.10.2 Approximating Path Opening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.11 Estimating the Gradient for Images with Missing Pixels using

Elliptical Structuring Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

2.12 Continuous and Discrete Morphology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

2.12.1 Morphological Dilations as Partial Differential

Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3 Mathematical Morphology on Irregularly Sampled Signals . . . . . . . . . . . . . . . . . . . 35

3.1 Approximating Continuous Morphology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.1.1 Morphological Operators Introduce Higher Frequency

Content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.1.2 Effect of Shifting the Sampling Grid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.1.3 Effect of the Sampling Grid on the Structuring

Element . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

3.2 Point Clouds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3.3 Irregular Morphology in 1D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3.4 Irregular Morphology in 2D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.5 Adaptive Morphology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

3.5.1 Elliptical Adaptive Structuring Elements . . . . . . . . . . . . . . . . . . . . . . . 43

3.5.2 Adapting the Elliptical Adaptive Structuring Elements

to Irregularly Sampled Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

4 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

4.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

4.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

5 Brief Summaries of the Included Papers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

6 Summary in Swedish . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

7 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

List of Abbreviations

1D One dimension, one-dimensional

2D Two dimensions, two-dimensional

nD n dimensions, n-dimensional

LST Local structure tensor

MM Mathematical morphology

NDC Normalized differential convolution

PO Path opening

PPO Parsimonious path opening

SE Structuring element

USPO Upper skeleton path opening

1. Introduction

1.1 Motivation

Mathematical morphology (MM) is a widely used approach for processing

signals (e.g. images). Linear filters are not affected by the sampling grid

according to Nyquist-Shannon sampling theory [47, 57]. However, for mor-

phological filters, which are non-linear, this is not the case. As a result, for

example, an object being imaged will yield different results depending on the

exact location of said object within the field of view of the camera. This behav-

ior is generally undesirable, since one is usually interested in properties of the

imaged object itself, not the image. Therefore we wish to address this issue by

reducing the dependence on a sampling grid. By implementing morphology

on irregularly sampled signals, we take a step in this direction. This leads to

increased precision, allowing for sub-pixel measurements.

Let us consider the connection between the discrete and the continuous

domain. Given a band-limited image, for example an image projected through

a system of lenses [29], which has then been sampled correctly (i.e. at a

sampling rate more than twice the highest frequency according to the Nyquist-

Shannon threshold [47, 57], although in practice the sampled signal has a finite

extent, which breaks the bandlimitedness), such as the image captured by the

camera in the example above, one may reconstruct the continuous signal from

the discrete one. However, applying a non-linear filter, such as a morphological

filter, will generally introduce higher frequency content into the signal, thus

breaking the correspondence between the continuous, filtered signal and the

filtered discrete signal, since the filtered signal cannot be sampled correctly on

the sampling grid. This motivates the move to irregular sampling.

Additionally, allowing for irregularly sampled signals makes it possible to

deal with a large amount of data of interest which is irregularly sampled to

begin with, for example data obtained from range cameras. Typically, such data

is dealt with by resampling onto a regular grid when applying morphology.

This resampling can result in undesirable artifacts that negatively affect results.

The work presented in this thesis applies to irregularly sampled data, so this

resampling step is avoided.

13

1.2 Objectives

The aim of this thesis is to improve the precision of image-based measurements

through irregular sampling, specifically by introducing a framework for dealing

with mathematical morphology on irregularly sampled signals as part of the

processing. In paper VI both the input and the output signals are regularly

sampled, however in an intermediate processing step a graph representation is

constructed whose nodes represent samples at certain (not regular) positions

in the original image and edges indicate neighbor relationships.

In papers I, and II, the focus is, mainly, on regularly sampled input signals,

but with irregularly sampled output (although the methods developed are also

applicable to irregularly sampled input as is shown) while paper V deals with

the special case where the input is irregularly sampled, but all samples are

taken at grid positions (similarly to paper VI). Paper IV extends the work of

paper II dealing with regularly and irregularly sampled input and allows for a

wider set of filtering approaches than the previous papers in this thesis.

Finally, paper III, mainly deals with irregularly sampled input and output,

namely point clouds of urban scenes. What follows is a list of objectives:

• Develop a framework for mathematical morphology on irregularly sam-

pled signals.

• Develop better approximations of continuous morphology in the discrete

domain via irregular sampling or weighted measurements.

• Apply the framework for MM on irregularly sampled signals to irregu-

larly sampled input data (e.g. 3D point clouds).

• Consider the special case of irregularly sampled signals where the avail-

able samples fall on grid points, but there are missing samples.

• Develop adaptive and non-flat morphology for the irregular case.

1.3 Thesis Outline

This thesis consists of seven chapters including the introduction, the summary

in Swedish, and acknowledgments. The second chapter (i.e. following the in-

troduction) introduces mathematical morphology, focusing mostly on discrete,

regularly sampled, signals (e.g. grayscale images). The third chapter takes

these concepts, generalizes them to irregularly sampled signals, and describes

why this is useful, in particular when approximating continuous morphology

in the discrete domain, but also in cases where the input signal is irregularly

sampled. The fourth chapter presents the main contributions of the thesis and

indicates some future directions of interest. The fifth chapter gives a brief

summary of each paper included in the thesis. The sixth chapter presents a

summary of the thesis in Swedish, and the final chapter, i.e. chapter seven, is

dedicated to acknowledgments.

14

2. Mathematical Morphology

Here I give a brief introduction to mathematical morphology to give a basis

for the following chapters (see, for example the book edited by Talbot and

Najman [46], for further reading). Introduced by Matheron and Serra in the

1960s [41, 56], mathematical morphology is widely used in image analysis,

often used as a pre- or postprocessing step. Morphological operators are

conceptually similar to convolutions, but unlike convolutions, they are non-

linear.

Initially applicable to binary images (seen as sets of foreground points)

using set-theoretical concepts, morphology has been extended to several other

domains, including grayscale [56, 64] and color images [12, 67, 76], graphs [73,

19, 45], and point clouds [13, 37], among others. The main structure underlying

morphology is the complete lattice [5].

Definition 1. A complete lattice is a pair (L,�), a set, L, equipped with apartial order � where every subset X ⊂ L has a supremum and an infimumin L.

Some typical examples of complete lattices include binary sets with set

inclusion ⊂ as the partial order, where the supremum and infimum are the

union and intersection, or grayscale images with pixel-wise comparison of

intensities as the partial order and the pixel-wise supremum and infimum. For

details see the papers by Heijmans and Ronse [25, 53].

In this thesis we will mostly be concerned with signals that are defined by

sets of tuples of three values (x, y, z) (or functions z(x, y)). E.g. grayscale

images, where x, y ∈ Z indicate pixel position and z ∈ R̄, where R̄ is the

real numbers equipped with −∞ and +∞, indicates graylevel, or irregularly

sampled signals with position x, y ∈ R and value/height z ∈ R̄. In the

case of grayscale images, a partial order can easily be defined by pointwise

comparison of images, using the natural ordering of pixels. However, for

irregularly sampled signals, pointwise comparisons is not generally possible,

since the positions of samples are not restricted to a grid, meaning there is no

obvious relationship between pairs of points in two signals. In paper I, this

issue is dealt with by associating a continuous signal, with infinite support,

to each sampled signal (via a kind of interpolation) and then comparing these

continuous signals.

The basic operations of mathematical morphology are dilations, erosions,closings, and openings [56]. Commonly a set B ⊂ E, called a structuring

15

element (SE), is used to probe the data, where E is some set, e.g. R2 or Z2.

In some cases a structuring function B : E → R̄ is used instead. In the first

case, the SE is called flat, since it is a 2D image without height. In the case

of a structuring function, the probe may have a height that varies, meaning

the structuring element is non-flat. Note that the flat case is a special case of

non-flat morphology, namely using the structuring function

B(x) ={0, if x ∈ B

−∞, otherwise(2.1)

The structuring element (or structuring function) is used to probe the data

at each point, yielding a new signal where the output is based, in some sense,

on how well the probe (the SE) fits. Thus, depending on structures of interest,

one may choose an appropriately shaped structuring element.

More specifically, a morphological transformation depends on the signal Xand the structuring element B, where B is a point set where the positions are

given relative to some origin (note that the origin is not necessarily contained

within the SE). Conceptually, applying the transformation to X is done by

systematically sliding B across the entire signal outputting a new value at

every position based on some relationship between X and the translated SE,

B.

In this thesis, the terms “structuring element” and “structuring function” are

used somewhat interchangeably. However, when using the term structuring

function, the context is usually non-flat morphology (but as pointed out above,

flat morphology is just a special case, see 2.1). Conversely, when using the

term structuring element, the context is normally flat morphology, although

this distinction may sometimes get muddied when discussing generalizing from

flat to non-flat morphology.

2.1 Basic Operators

There are four fundamental morphological operators called dilations, erosions,openings, and closings. In this section we briefly describe these operators.

2.1.1 Dilation

The dilation of X by B is often notated as X ⊕B and is defined as:

[X ⊕B](x) =∨b∈B

X(x− b) (2.2)

That is, conceptually, placing B at x, consider all points in X “hit” by B and

take the supremal value. In the more general case of non-flat morphology,

16

where B is a structuring function, dilation is:

[X ⊕ B](x) =∨b∈E

(X(x− b) + B(b)

)(2.3)

Meaning the supremum is weighted based on B.

2.1.2 Erosion

The (non-flat) erosion of X by B is notated X � B and is defined as:

[X � B](x) =∧b∈E

(X(x+ b)− B(b)

)(2.4)

Note the differences compared to dilation, namely, infimum instead of supre-

mum, and offset x by b instead of −b. As a result, we have the useful property

that dilation and erosion are dual:

X ⊕ B = [X� � B̆]� (2.5)

Where � denotes the complement and B̆(x) = B(−x) is the reflected structur-

ing function. In other words, dilation of the foreground is the complement of

the erosion of the background with the reflected structuring function.

2.1.3 Hit-and-Miss Transform

The hit-and-miss transform (also called hit-or-miss transform), is used to

match patterns in an image. In this case, a compound structuring element

B = (B1, B2) is used. The hit-and-miss transform of X by B, denoted

X ⊗B can be defined in terms of ⊕ and �:

X ⊗B = (X �B1) \ (X ⊕B2) (2.6)

Here B1 describes a foreground pattern and B2 a background pattern. Nor-

mally the hit-and-miss transform is applied to binary images using binary (flat)

structuring elements.

2.1.4 Closing

The closing of X by B, denoted X • B can be defined in terms of ⊕ and �:

X • B = (X ⊕ B)� B (2.7)

Applying a closing can be thought of as sliding the structuring element along

the underside of the signal interpreted as a surface, as snugly as possible, where

the value at each point in the output is the height of the SE at that point. This

can be used to break apart connections in an image or smooth out contours, for

example.

17

Figure 2.1. From left to right: Original image, erosion, dilation, opening, and closing

using a (flat) disk shaped structuring element.

2.1.5 Opening

Similarly to the relationship between dilations and erosions, closings and open-

ings are dual. The opening of X by B is denoted X ◦ B where

X ◦ B = (X � B)⊕ B (2.8)

and

X • B = [X� ◦ B̆]� (2.9)

Figure 2.1 shows examples of these basic operators (except for the hit-and-

miss transform).

2.2 Umbra and TopOne way of interpreting grayscale morphology, is by recasting the problem

as a case of binary morphology by transforming the grayscale function into a

binary one. For a given function X from R2 → R̄, its umbra [64, 52], U(X)is defined as:

U(X) = {(x, y) ∈ R2 × R̄ | y ≤ X(x)} (2.10)

Associated with every umbra V is its top T :

T (V )(x) =∨{y | (x, y) ∈ V } (2.11)

Figure 2.2 illustrates this concept. Note that T (U(X)) = X . Using the umbra

approach we have that

X ⊕ B = T [U(X)⊕ U(B)] (2.12)

X � B = T [U(X)� U(B)] (2.13)

where the dilation and erosion on the left-hand side is the gray-scale version,

while on the right they are the binary versions, also known as Minkowski

addition and subtraction [42, 22]. This interpretation of grayscale morphology

as a form of binary morphology can be a useful way of looking at morphological

operations, and helped inspire, to some extent, the approach proposed for MM

on irregularly sampled signals in papers I, II, III, and IV. There are some

technicalities that are not discussed here, particularly regarding continuous

graylevels. See the paper by Ronse for details [52].

18

(a)

(b)

Figure 2.2. (a) A function and its umbra. (b) An umbra and its top.

2.3 Continuous Morphology

Usually, morphology as applied in image processing is only considered in the

discrete domain. For practical reasons this is sensible, since one normally deals

with regularly sampled, digital images. However, there is nothing preventing us

from choosing E = R2, if we so desire. If we are interested in the continuous

signal that may be underlying the sampled signal (e.g. the signal from the

real world that has been sampled in the digital image), we should consider

the correspondence between the sampled signal and the continuous one. In

other words, if—as is usually the case—we are interested in the imaged objectitself, not the image data, we should take care that the sampled signal is a good

representation of the object.

For a properly sampled, band-limited signal, there is a correspondence

between the continuous signal and its discrete representation according to the

Nyquist-Shannon sampling theorem [47, 57]. Here, “properly sampled” means

that the sampling rate is more than twice the highest frequency.

For linear filters, such as convolutions, this correspondence is preserved

even after their application. However, in the case of morphology, the filters are

not linear, (because of the supremum/infimum). In practice this means higher

frequency content can be introduced into the transformed signal, meaning

the correspondence between discrete and continuous domain is broken (see

Fig 2.3).

19

SE

Figure 2.3. Blue: 1D, continuous sinusoid signal. Brown: Continuous dilation of the

signal using the structuring element shown in the bottom right. The sine-wave can

be represented using slightly more than two samples per period. However, the dilated

signal introduces higher frequency content into the signal (note the cusps generated at

the valleys of the sine).

Figure 2.4. Propagation of a wave from the border of X . The skeleton S(X) will be

the “y” shape that is beginning to take form in the center of the object.

2.4 Skeletonization

The purpose of skeletonization is to simplify an object resulting in a thin,

abstract representation, called a skeleton. This skeleton should preserve the

topological and geometric properties of the original object. The idea was first

introduced in 1967 by Blum [6] for binary images. Assume a point setX ⊂ R2

from the border of which we propagate a wave inwards at constant speed from

each point. The skeleton S(X) is the set of points where two or more waves

meet. See Fig. 2.4.

A more formal description of a skeletonization follows: Given a point set

X ⊂ E, let

B(x, r) = {y ∈ E | d(x, y) ≤ r} (2.14)

where d(x, y) is the distance between x and y, and x ∈ X is the center of a

ball with radius r ≥ 0.

20

Definition 2. A ball B(x, r) ⊂ X is said to be maximal, if there is no otherball B(x, r′) such that B(x, r) � B(x, r′) and B(x, r′) ⊂ X .

The skeleton by maximal balls is the set of pointsS(X) containing all centers

of maximal balls inside X . This definition, though quite intuitive, does not

guarantee that the connectivity of the original set is preserved (e.g. the skeleton

by maximal balls of two balls touching at a point is the set containing the two,

disjoint center points of the balls), which is undesirable. More precisely, the

skeleton by maximal balls is not a homotopic skeleton (i.e. does not preserve

homotopy).

Informally, two binary images X and Y are homotopic if one can be de-

formed, continuously, into the other.

2.4.1 Desirable Properties

The following four properties are desirable for a skeleton [48]:

1. Subset of the original object

2. Thin

3. Allows reconstruction of the original object

4. Topologically equivalent to the original object

In general, all desirable properties cannot simultaneously be satisfied.

2.4.2 Digital Skeleton

In addition to not being homotopic, the skeleton by maximal balls does not

guarantee a thin (i.e. one pixel wide) skeleton in the discrete case. Instead,

commonly, a sequential thinning approach is used in the discrete case. Such

an approach may guarantee thin, homotopic skeletons.

ThinningThe thinning of an image X using a composite SE B = (B1, B2), X B is

defined as

X B = X \X ⊗B (2.15)

Skeletonization by thinning is performed by iterating thinnings using a

sequence of composite SEs. Commonly used sequences are the Golay alpha-bet [21, 56] on a given raster. Iterating these sequential thinnings converges to

some set of points, which is the skeleton by thinning.

Saha et al. [54] present a good overview of different skeletonization algo-

rithms.

21

2.5 Geodesic Morphology

The geodesic distance dX(x, y) is the length of the shortest path between xand y contained within the set X (possibly +∞, if no path exists) [33]. The

geodesic ball BX(x, r) is the subset of the ball with radius r located at x which

is entirely contained within X:

BX(x, r) = {x′ ∈ X | dX(x, x′) ≤ r} (2.16)

This leads to the definition of geodesic dilation [74, 34] of size r of some set

Y , within X , δ(r)X (Y ):

δ(r)X (Y ) =

⋃y∈Y

BX(y, r) (2.17)

2.5.1 Reconstruction by Dilation

Morphological reconstruction [74] is a useful tool to recover the shape of some

set of marked objects. In the binary case, we can obtain the reconstruction by

dilation ρX(Y ) for a marker Y (i.e. a set of points that indicate the objects of

interest) and a mask image X by successive geodesic dilations of Y inside X:

ρX(Y ) = limn→∞ δ

(n)X (Y ) (2.18)

i.e., apply larger and larger geodesic dilations until idempotency is reached.

In the grayscale case, a corresponding operation can be performed by it-

eratively dilating the marker Y (which is in this case a grayscale image) and

taking the infimum of the dilated image and the mask image X (which is

also a grayscale image). That is, an elementary grayscale geodesic dilation

δ̂(1)X (Y ) = (Y ⊕B)∧X , whereB is the unit ball, is used to define the grayscale

geodesic dilation of size n:

δ̂(n)X (Y ) = δ̂

(1)X ◦ δ̂(1)X · · · ◦ δ̂(1)X (Y ) (2.19)

meaning δ̂(n)X is the composition of n elementary grayscale geodesic dilations

δ̂(1)X . Then, analogously, grayscale reconstruction is computed by successive

grayscale geodesic dilations until reaching idempotency.

The result being that peaks of X marked by Y are extracted by the recon-

struction (see Fig. 2.5).

2.6 Granulometry

Granulometry [61] is a tool of mathematical morphology, which can be used

to find size distributions of objects in an image X , by repeatedly applying

22

Figure 2.5. The reconstruction (dashed blue) of the marker (red) using a mask (green).

The marker indicates some peaks of interest that are extracted by the reconstuction by

dilation.

openings (or closings) of increasing size and summing up the pixel values for

each result. That is, for some family of SEs {Bi}i∈{0,...,N}, a corresponding

set of openings is computed:

γi(X) = X ◦Bi (2.20)

giving the granulometric function

V (X, i) = |γi(X)| (2.21)

where | · | denotes some kind of measure, e.g., number of pixels, volume, or

sum of pixel values. From this, one may compute the size distribution

P (X, i) = V (X, i)− V (X, i+ 1) (2.22)

The shape of the structuring elementsBi should be chosen based on the objects

of interest. For example, in paper VI path openings are used to estimate length

distributions of line-like structures (i.e., paths).

Figure 2.6 shows an example of applying granulometry (with a disk-shaped

SE) to an image containing coins of different sizes.

A benefit of this approach is that there is no need to segment the objects of

interest before measuring sizes.

2.7 Morphological Gradient and Laplacian

The morphological gradient [3, 51] G(X) of an image X is normally defined

as

G(X) = (X ⊕B)− (X �B) (2.23)

23

55 60 65 70Radius

0

2

4

6

8

Inte

nsity

diffe

renc

e 105 Size distribution

Figure 2.6. Left: A photo of some coins. This image is preprocessed slightly by

inverting it (so that the coins are brighter than the background) and applying a small

closing, to remove some darker details inside the coins. Right: Part of the size

distribution yielded by the granulometric function on the preprocessed coin image.

The three peaks around 56, 61, and 68 correspond to the three different coin sizes in

the image.

where B is a unit ball SE. I.e., the morphological gradient of a signal X is

the difference between its dilation and its erosion with a unit ball. There are

a number of variations on this theme, for example the external and internal

gradient:

Ge(X) = (X ⊕B)−X (external) (2.24)

Gi(X) = X − (X �B) (internal) (2.25)

These operators yield positive values at each pixel in an image X and can be

used to estimate the gradient magnitude of the signal. A notable application of

the internal and external gradients is their use in computing the morphological

Laplacian,Δm [70] which is given by the difference between the external and

internal gradient:

Δm(X) = Ge(X)−Gi(X) (2.26)

To yield directional estimates, one may use non-isotropic structuring ele-

ments that are oriented appropriately [3] as is done in Paper V for images with

missing samples.

Figure 2.7 shows examples of the morphological gradient and Laplacian.

2.8 TophatThe tophat, TB

h , is a morphological operator that enhances parts of an image

where the SE,B does not fit. This can be used to enhance objects of interest by

suppressing the background. The tophat is the difference between the original

image and its opening:

TBh (X) = X −X ◦B (2.27)

24

Figure 2.7. Left: Morphological gradient. Right: Morphological Laplacian. The SE

used is a disk of radius 2. Note the diamond shaped artifacts (e.g. the bright diamond

in the top-left or the one near the neck of the man) in both images. These are a result of

the discretization of the disk SE and is one of the issues with approximating continuous

morphology in the regular, discrete domain.

Figure 2.8 shows an example where the tophat is used in order to enhance

linear features of an image.

2.9 Adaptive Morphology

In paper IV the proposed morphology on irregularly sampled signals is gen-

eralized in a number of ways, including allowing for adaptive morphology,

meaning the SE is no longer rigid, and may instead adapt based on position or

image content, for example. In some cases any particular structuring element

is not suited for the entire image. In such cases, adaptive morphology can be

used, shaping the SE appropriately. Because of the variety of different image

attributes that could be chosen to adapt the SE, there are many papers dealing

with adaptive morphology. This section contains a brief overview. See the

surveys by Ćurić et al. [20] and Maragos and Vachier [40] for further reading.

An early example of adaptive morphology was presented by Beucher etal. [4] who used a structuring element dependent on position to deal with

perspective. Another early example is presented in a 1993 paper by Verly

and Delanoy [71], developing adaptive MM in the context of range images,

adjusting the SE based on the distance, since a large object far away may be

similar in size to a small object close to the camera. Other early examples

include the work by Cheng and Venetsanopoulos [16, 17], and Chen et al. [15].

More recently, so called morphological amoebas were introduced [35],

adapting the shape of structuring elements based on the content of the im-

25

Figure 2.8. Left: Input image (one of the Brodatz texture images [10]), middle:

opening of the image using a small disk shaped SE, right: the difference between the

original image and its opening, i.e. the tophat. Here the tophat is used to enhance the

linear features of the input image.

age, trying to keep a high homogeneity of pixels covered by the SE at any

given point, by considering the local similarity between neighboring pixels.

Another relatively recent method was presented by Landström and Thur-

ley [32]. The main idea being to shape elliptical structuring elements based

on the eigenvectors of the local structure tensor (LST) [28] which along with

the eigenvalues give information about the orientation (and its strength) of the

data. In a later paper this approach is extended by using non-flat quadratic

structuring functions [31]. A more thorough description of this approach [32]

can be found in Section 3.5.1, where an adaptation of their approach to the

case of irregularly sampled signals is presented. For further details see also

paper IV.

2.10 Path OpeningThe path opening (PO) is a morphological filter that addresses the problem of

enhancing long, thin structures [11]. This is of interest in a number of applica-

tions, such as detecting rivers or roads in remote sensing applications [66, 26],

vessel segmentation [77, 58], identifying tracks of dust devils on Mars [63],

etc. A straightforward approach to the problem of detecting such structures

would be to apply openings with line segments at several different orientations

and taking the supremum [62]. This approach is slow, however, and though it

may work well in cases where the features of interest consist of straight line

segments, if instead the features are curved, this approach gives unsatisfac-

tory results. The path opening was first proposed in 2000 by Buckley and

Talbot [11] and deals with this problem.

Conceptually the path opening uses many flexible line-like structuring ele-

ments of a given length, L to open an image, followed by taking the supremum

of these openings by all these structuring elements. The number of structuring

elements, in a naive implementation, grows exponentially with L, however

26

Figure 2.9. Adjacency graphs used by the original path opening. (a) South-north

adjacencies, (b) southwest-northeast, (c) west-east, (d) northwest-southeast. For a

given adjacency graph, an allowed path is built by starting at some node, following

one of the arrows, and repeating L times, where L is the length of the opening. (See

paper VI, © 2016 IEEE)

in practice it can be computed in linear time with respect to L [24]. The

structuring elements are defined by a set of adjacency graphs which constrain

paths to follow some general direction. Commonly, the adjacency graphs used

describe 90◦ cones whose main axis point in one of four directions namely,

south-north, southwest-northeast, west-east, or northwest-southeast. These

adjacency graphs are illustrated in Fig. 2.9.

2.10.1 Constrained Path Opening

The path opening has a problem of overestimating lengths of line-like objects

that have some thickness, stemming from the possibility for paths to “zig-

zag” inside the object. The constrained path opening [38] is a variant of the

path opening that attempts to deal with this issue by restricting the possible

paths such that this behavior is minimized, thus yielding less biased length

measurements.

2.10.2 Approximating Path Opening

In order to speed up the computation of the path opening, approximate path

openings have been proposed. The main idea is to preselect a limited number

of paths through the image followed by applying an opening only to these

preselected paths and finally reconstructing the result using the opened image

as a marker and the original image as a mask (see Sec. 2.5.1).

Parsimonious Path OpeningThe parsimonious path opening (PPO) [44] is an approximate path opening

which has a time complexity independent of path length L. This is achieved

by preselecting 1D paths from one border of the image to the opposing border.

In other words, there is one path selected per border pixel. After this, the

27

preselected paths are opened using a 1D opening, which can be done in O(1)per pixel [43], i.e., independent of the length of the opening. In this way, the

time complexity of the parsimonious path opening is independent of L and

proportional to the number of pixels in the image.

A problem with PPO is the fact that the preselection of paths leaves blind

regions. These blind regions are affected by the content of the image. The

preselection strategy pulls paths towards bright structures, which may lead to

particular structures of interest in the middle of the image being occluded by

bright structures toward an edge of the image, since the selected paths start at

the edge of the image and are constrained within a 90◦ cone.

Another issue is the fact that the length measurements of PPO are biased

(this is also the case for the regular path openings, in fact even more so), where

paths that are mostly horizontal, vertical, or diagonal are correctly measured,

however the lengths of paths that generally follow an angle that is not a multiple

of π/4 tend to be overestimated.

Proposed path opening: Faster, Unbiased Path Opening bySkeletonization and Weighted GraphsIn paper VI, a new approximate path opening algorithm is presented, the upperskeleton path opening (USPO). This paper takes inspiration from the PPO,

preselecting paths, applying an opening of length L to the preselected paths,

followed by a reconstruction. However, the proposed approach addresses the

two problems of the PPO mentioned above. Firstly, the preselected paths do

not suffer from the problem of occlusion. This is achieved by selecting paths

by skeletonizing the original image using the so called upper skeleton [72],

yielding a grayscale skeleton which is taken as the selected paths. These paths

follow the bright ridges of the image, thus selecting the most important paths,

since we are interested in bright, long, thin structures. A problem with this

approach is that it precludes us from applying the fast 1D opening [43] used

by the PPO, since the skeleton is not easily broken up into 1D paths. Instead,

we propose a way of constructing a sparse graph out of the skeleton, upon

which a graph based path opening is applied. Empirically, this yields at least

comparable speeds to the PPO.

The second main contribution of paper VI deals with the biased length

measurements of the PPO (and the regular PO for that matter). There are three

factors that contribute to the reduction in measurement bias, namely:

• The skeleton is thin. This prevents paths from zig-zagging inside elon-

gated structures, by essentially following the main axis of objects. This

benefit is shared with PPO.

• The adjacency graphs are weighted using weights that minimize the

relative error for digital lines of arbitrary orientation [50]. The regular

path opening formulation measures diagonal and horizontal/vertical steps

as equal in length (both have a weight of one), however, realistically, a

diagonal step is longer than a horizontal/vertical one. The PPO weights

28

Figure 2.10. Top: Input image, from an electron microscope, of a DNA molecule [24].

(a) Traditional path opening, (b) constrained path opening, (c) parsimonious path

opening, and (d) upper skeleton path opening, all using the same length threshold

L = 50. (See paper VI, © 2016 IEEE)

diagonal steps as√2 and horizontal/vertical are weighted as 1. This

yields accurate measurements for angles that are multiples of π/4, but

will generally overestimate lengths for other angles and do not minimize

the relative error for arbitrary orientations. The weights proposed in

paper VI greatly improve the results of length measurements.

• The USPO does not suffer from the problem of occlusions. The skele-

tonization is not adversely affected by the presence of bright structures

closer to the edges of the image, unlike the path selection approach taken

by the PPO.

Figure 2.10 shows the traditional path opening, as well as the constrained path

opening and the approximate variants described above applied to an image to

enhance a thin, sinuous structure.

2.11 Estimating the Gradient for Images with MissingPixels using Elliptical Structuring Elements

In paper V a way of estimating the gradient for image with missing pixels

is presented. This is useful in a number of applications, for example, the

gradient magnitude can be used to enhance edges, while the components of

the gradient gives information about the orientation of said edges. A common

way of estimating the gradient for an image, using mathematical morphology,

is to apply the morphological gradient [51]. However, this only yields an

29

estimate of the gradient magnitude, not the components themselves, which are

important, for example, when computing the local structure tensor [28]. One

way to estimate directionality is to use linear structuring elements to estimate

horizontal and vertical derivatives, however this does not work well in the case

of missing samples, since the linear SE is likely to miss many pixels (since it

is thin and, depending on the percentage of dropped pixels, there are only few

nearby pixels at any point).

The proposed approach instead uses half-ellipses as structuring elements.

That is, starting from a structuring element with elliptical support:

B =

{(x, y)

∣∣∣∣ x2

a2+

y2

b2≤ 1

}(2.28)

we derive several structuring elements as the intersection of ellipses and four

different half-planes:

B−h = Bh ∩H− B+

h = Bh ∩H+ (2.29)

B−v = Bv ∩ V − B+

v = Bv ∩ V + (2.30)

Where Bh is an ellipse where a = a0 and b = b0 for some a0, b0 ∈ R, Bv is

an ellipse where a = b0 and b = a0, and

H− = {(x, y) | x < −ε} H+ = {(x, y) | x > ε} (2.31)

V − = {(x, y) | y < −ε} V + = {(x, y) | y > ε} (2.32)

These SEs can then be used to estimate the horizontal and vertical component

of the gradient:

Ix = (I ⊕B−h )− (I ⊕B+

h ) Iy = (I ⊕B−v )− (I ⊕B+

v ), (2.33)

This approach turns out to give results that yield angle estimates biased

towards multiples of π/2. However, by using the same approach outlined

above, except rotating the ellipse and half-planes for a number of angles, it is

possible to estimate directional derivatives Dθ along several angles θ. I.e., the

differences estimate

∇vθI = ∇I · vθ ≈ Dθ (2.34)

where vθ is the unit vector (cos(θ), sin(θ)). Using this, a system of equations

can be set up: ⎡⎢⎢⎢⎣vθ0

vθ1...

vθn

⎤⎥⎥⎥⎦(∂I

∂x

∂I

∂y

)T

=

⎡⎢⎢⎢⎣Dθ0

Dθ1...

Dθn

⎤⎥⎥⎥⎦ (2.35)

30

-3 -2 -1 0 1 2 3Ground truth gradient direction

-3

-2

-1

0

1

2

3Horizontal and vertical SEs

Ideal estimateMorphological w = 5 l = 9

-3 -2 -1 0 1 2 3Ground truth gradient direction

-3

-2

-1

0

1

2

3Horizontal, vertical, and diagonal SEsIdeal estimateMorphological w = 5 l = 9

Figure 2.11. In these graphs, the estimated angle for a disk image (with no missing

pixels) is plotted vs the true angle. In the ideal case, the graph should therefore be a line

with slope 1 (shown in blue). When using only the horizontal and vertical estimates,

there is a bias. However, when also including the diagonal estimates this bias almost

completely disappears. Here w and l indicate the length of the minor and major axis

of the SE respectively.

Solving for(

∂I∂x

∂I∂y

)T

gives:

(∂I

∂x

∂I

∂y

)T

=

⎡⎢⎢⎢⎣vθ0

vθ1...

vθn

⎤⎥⎥⎥⎦+ ⎡⎢⎢⎢⎣Dθ0

Dθ1...

Dθn

⎤⎥⎥⎥⎦ (2.36)

where the + indicates the pseudoinverse. This approach is similar to one taken

by Hassouna and Farag [23]. Figure 2.11 shows the angle estimate versus

the true angle at the edge of a disk with a Gaussian profile when using only

horizontal and vertical estimates as well as when diagonal estimates are also

included and shows the bias being significantly reduced.

Although the approach, as presented here, is specific to regularly sampled

images with missing samples, it can easily be applied to irregularly sampled

signals, by simply replacing the regular, discrete operators with their proposed

counterparts for irregular morphology. This approach is taken in paper IV

where the gradient estimate is used to compute the local structure tensor [28]

used to shape SEs for adaptive morphology. The paper also proposes a num-

ber of additional improvements including combining the estimates yielded by

(2.33) with the estimates yielded by replacing the dilations in (2.33) with ero-

sions, as well as making use of non-flat structuring elements. Those variants

are compared against other gradient estimation methods and shown to perform

favorably under certain conditions.

31

-4 -2 0 2 40

0.2

0.4

0.6

0.8

1

-4 -2 0 2 40

0.2

0.4

0.6

0.8

1

-4 -2 0 2 40

0.2

0.4

0.6

0.8

1

Figure 2.12. A small shift of the sampling grid causes the maximal sample to either

fall close to the extremum or completely miss it.

2.12 Continuous and Discrete Morphology

In this section we specify three main issues when applying morphology in the

regular, discrete domain, if the goal is to approximate continuous morphology.

The motivation for considering the approximation of the continuous case is,

as previously stated, the fact that one is usually interested in the subject being

imaged, not the image data itself. I.e., not the sampled set of pixels.

First, recall the Nyquist-Shannon sampling theorem:

Given a function f , with a maximal frequency of B, a sampling of f witha sampling rate greater than 2B completely determines f . [47, 57]

In other words, a continuous function f can be reconstructed from its samples,

if the sampling rate is greater than twice the highest frequency of f .

There are a number of issues to consider regarding the correpondence be-

tween regular, discrete morphology and the continuous counterpart. First:

Applying a morphological operator to a signal generally introduces higher

frequency content (see Fig. 2.3). Thus, for a band-limited signal which can

be represented by some regular sampling at a given frequency, its transformed

counterpart may no longer be properly representable at the same sampling rate.

However, regular, discrete morphology does not account for this, meaning the

correspondence between the transformed discrete signal and the continuous

one is broken.

Secondly: The regular, discrete MM operators depend on the position of the

sampling grid. In other words, shifting the grid (or equivalently the signal) will

lead to different results, since the operators depend on extrema in the signal,

which may be missed if the sampling grid is shifted even a small amount

(see Fig. 2.12). However, since the original sampled signal, assuming correct

sampling, can reconstruct the band-limited continuous signal, the information

about the extrema exists in the discrete signal, and could be taken into account.

In practice, this means the result depends on the position of an object being

imaged, which is not desirable.

32

Figure 2.13. A discretization of the continuous disk.

Thirdly: The structuring element depends on the sampling grid. Since the

SE is a subset of the image domain, the sampling grid causes discretization

problems [39]. For example, if one wants to use a disk-shaped SE, it has to be

discretized first (see Fig. 2.13), meaning a small disk (relative to the sampling

grid) may end up looking like a diamond.

These issues cause the approximation of continuous morphology, and there-

fore the correspondence to the objects being imaged, to suffer. In the following

section a brief overview of related morphological approaches that deal with

morphology and the continuous domain in some capacity is presented.

2.12.1 Morphological Dilations as Partial Differential Equations

In a 1992 paper by Brockett and Maragos [8], extended in 1994 [9], continuous-

space morphological erosions, dilations, openings, and closings are modeled

as nonlinear partial differential equations whose evolution can compute con-

tinuous morphology in the discrete domain.

For some function f : Rn → R representing a continuous nD signal, and a

continuous structuring function g : B → R, the multiscale dilation of f by gat scale s ≥ 0 is defined as

δ(x, s) = f ⊕ gs(x) =∨

b∈sB

f(x− b) + sg(b/s) (2.37)

where s is a scale parameter and gs is a scaled version of g by s. The goal is

then to study the evolution equation

∂δ

∂s(x, s) = lim

r→0+

δ(x, s+ r)− δ(x, s)

r(2.38)

In particular, when B is a unit disk, the PDE for multiscale dilations becomes

∂δ

∂s=

√∣∣∣∣∂δ∂x∣∣∣∣2

+

∣∣∣∣∂δ∂y∣∣∣∣2

(2.39)

As s increases, the PDE-based operators can create discontinuities in the spatial

derivatives, so one has to take special care at these points. The authors propose

33

replacing the conventional derivatives with the morphological counterpart, i.e.

M(f)(x) = limr→0+

(∨|v|≤r f(x+ v)

)− f(x)

r(2.40)

which is essentially the external morphological gradient (see Eqn. 2.24). See

the papers [8, 9] for details, including equations for several other SE shapes.

Similar equations can also be written down for erosion, openings, and

closings.

There are several papers built on similar ideas [69, 68, 55, 7], however, in

general the result is represented on a regular grid and therefore suffers from

the problems previously described. Additionally, the PDE based approaches

have a tendency to blur edges and are slow compared to regular, discrete mor-

phology [75].

34

3. Mathematical Morphology on IrregularlySampled Signals

In this chapter, an approach to morphology for irregularly sampled signals is

developed. In his thesis [65], Thurley develops morphology for irregularly

sampled input, however the positions of samples of the transformed signal is

the same as those in the input, which means most of the problems with approx-

imating continuous morphology remain. Calderon and Boubekeur develop

binary 3D morphology on point clouds [13], however this approach is limited

to binary signals, and must explicitly estimate the underlying surface. Also

relevant to this problem is morphology on graphs. Najman and Cousty [45]

provide a good survey of current research on morphology on graphs, however

the proposed approach does not explicitly work on any graph structure, al-

though it may be possible to achieve faster computation by incorporating some

such underlying structure.

I will also show how the idea of irregular sampling can be used to improve

the approximation of the continuous case in the discrete domain. There are

two main reasons for examining morphology in the case of irregular sampling.

First: as previously described, there are a number of issues with approximating

continuous morphology using regular, discrete morphology. These issues

can be alleviated, yielding a better approximation of the continuous case, by

allowing for irregular sampling. Secondly: there is an abundance of data

that is irregularly sampled to begin with, meaning applying regular, discrete

morphology requires some preprocessing of the data to make it amenable,

often by resampling onto a regular grid, which normally causes interpolation

artifacts and problems where the original sampling had a hole in the data.

3.1 Approximating Continuous MorphologyPreviously I described three issues regarding the approximation of continuous

MM using regular, discrete morphology. Here I will motivate how these issues

are addressed by allowing for irregular sampling.

3.1.1 Morphological Operators Introduce Higher FrequencyContent

After applying a morphological operator, thereby generally introducing higher

frequency content into a signal and breaking the correspondence between the

35

-50 0 50 100 150 200 250 3000

50

100

150

200

250

300

350Regular, discrete dilationProposedOriginal Signal

Figure 3.1. Dilation of the black signal using regular, discrete morphology (blue)

and the proposed approach (red). Note the decreased number of samples on plateaus

compared to parts of the signal that fluctuate more wildly.

discrete and the continuous domain, one solution would be to sample the sig-

nal more densely to deal with this problem. However, increasing the sampling

density everywhere is expensive, and potentially unnecessary. The operators

usually generate a signal with many plateaus (see Fig. 2.3). Accurately rep-

resenting these parts of the signal should not require many samples (in the

extreme case, two samples per plateau is sufficient, i.e., one at each end point).

On the other hand, the operators also generate cusps, where the sampling den-

sity should be increased to better capture the behavior of the quickly changing

signal. This leads to the conclusion that representing the transformed signal

as a set of irregularly taken samples could benefit the approximation of the

continuous case (without exploding the number of samples) by increasing the

sampling density near “interesting” parts of the signal (e.g. the cusps), while

decreasing the density on very smooth parts (e.g. the plateaus). Figure 3.1

shows an example of the proposed approach from paper I applied to a 1D-

signal showing the described behavior of adjusting the sampling density based

on the smoothness of the signal.

3.1.2 Effect of Shifting the Sampling Grid

As shown in Figure 2.12, shifting the sampling grid may result in extrema

being missed. Thus, even though the sampling density is the same, and the

band-limited signal is correctly sampled, the result of applying a MM operator

will differ based on a small shift of the sampling grid. However, if we allow for

irregularly sampled input, we could, in principle reconstruct the extrema of the

continuous signal and insert only those points into the input before applying the

morpological operator, thus achieving a result that is not affected by the small

36

Figure 3.2. The continuous dilation (top right) of the input (top left) is 1.0 everywhere.

However, the discrete dilation of the sampled signal (bottom left) differs in the middle

(in the vicinity of (0, 0)), since the maximum at (0, 0) was missed when sampling.

By inserting a single sample at (0, 0) (thereby yielding an irregularly sampled signal),

this issue can be dealt with (bottom right). The red points indicate the location of the

samples of the discrete signal.

shift of the grid. Figure 3.2 shows an example of a continuous signal and its

dilation, as well as the dilation of the same sampled signal where the sampling

grid is deliberately placed such that an extremum is missed. The discrete

dilation therefore differs from the continuous one, because dilation depends

on maxima in the signal. However, adding in just a single point (namely that

extremum), thus yielding an irregularly sampled signal, the proposed approach

now corresponds well with the continuous case.

3.1.3 Effect of the Sampling Grid on the Structuring Element

Finally, as previously discussed, the structuring element is affected by the

sampling grid (see Fig. 2.13). However, in the proposed approach, there is no

sampling grid, and therefore no restriction on the sampling of the SE. Figure 3.3

shows an example of dilating an image using a disk shaped SE in the case of an

37

Figure 3.3. From left to right: input image, regular dilation on sampled input, irregular

dilation on sampled input, and regular dilation on non-sampled input. The red square

indicates the zoomed in area being displayed.

image that has been subsampled (after low-pass filtering). The original image

is also dilated with a disk-shaped SE of corresponding size. Because of the

subsampling, the size of the SE in relation to the grid is very small, so the

discretized disk ends up diamond shaped. However, the approach proposed in

paper III does not require a sampling grid, so the edge of the continuous SE

can be sampled with arbitrarily many samples at the exact position of the edge.

Therefore, the proposed approach yields a dilation that better approximates the

continuous case (or in this example, the dilation of the original image before

subsampling, which acts as a stand-in for the continuous case).

3.2 Point Clouds

It is worth noting that, as an additional benefit of developing MM for irregularly

sampled signals, an abundance of data that is already irregularly sampled to

begin with can be dealt with immediately, i.e., without requiring resampling.

Other work on morphology for irregularly sampled signals includes work by

Thurley [65] and more recent work on point clouds [13, 49, 1]. Also related is

the work on mathematical morphology on graphs [73, 19, 45], since a natural

structure for representing irregularly sampled data is often a graph.

In paper III a variant of the proposed approach is used to segment 3D point

clouds of urban scenes, where the height of samples is taken as their value,

as a demonstration of its applicability to processing signals that are irregularly

sampled to begin with.

3.3 Irregular Morphology in 1D

What follows is a brief description of the proposed approach to irregular

morphology in 1D. For details see paper I (an earlier paper from 2016 by

Asplund et al. may also be of interest [2]).

Let

S = P(Z× R̄), (3.1)

38

be the set of possible samples (i.e., position and value), where P denotes the

power set. Then a (possibly irregularly) sampled continuous function is an

element from the set

S = {A ∈ S | for all (xi, yi), (xj , yj) ∈ A, xi = xj ⇒ yi = yj}. (3.2)

It is possible to define a partial order � on S, such that (S,�) forms a

complete lattice. Since two irregularly sampled signals cannot, in general,

be compared pointwise, a function T : S → R̄R is defined which takes

an irregularly sampled signal as input and yields a continuous signal with

a support equal to R. This top function is used to define the partial order:

U � V ⇐⇒ T (U) ≤ T (V ), for U, V ∈ S (3.3)

where T (U) ≤ T (V ) iff T (U)(x) ≤ T (V )(x) for all x ∈ R.

The pseudocode for the algorithm to dilate a 1D, irregularly sampled signal

is shown in Alg. 1. Roughly, the algorithm performs these four steps:

1. Select a sample and translate the SE such that its origin coincides with

the sample position.

2. Make two copies of the sample.

3. Shift the copies towards the endpoints of the SE.

4. If a copy would end up under the SE placed at a sample of greater value,

stop shifting it.

There is a parameter ε > 0 that controls the minimal margin between two

neighboring output samples. This parameter cannot be 0, since this could yield

output where two samples at the same position take different values, therefore

being part of a sampling that is not an element of S. Using a self-balancing

binary search tree [27], the time complexity of the algorithm is O(N logN),where N is the number of samples in the input.

Erosions, closings, and openings are easily performed using the same algo-

rithm by making use of the duality of dilation and erosion and composition.

That is, by multiplying the signal with −1 to invert it, applying a dilation,

followed by multiplying the result by −1, one may perform an erosion.

3.4 Irregular Morphology in 2D

In papers II, III, and IV a similar approach to the 1D approach presented

above is developed, but for the case of 2D signals (e.g. grayscale images).

Paper II presents a first attempt at generalizing to 2D, which suffers from some

problems dealt with in the later papers III and IV (mainly an issue of fuzzy

edges in the output, as well as strict requirements on the shape of the SE).

39

Algorithm 1: Duplicate-and-shift dilation pseudocode.

Data: A list of input samples, NODES, which contains positions and values

for each sample. The left, and right edges, SE−, and SE+, of the SE as

offsets from the origin.

Result: A list of output samples DNODES which contains positions and

values for each sample in the dilated signal.

1 Function Dilate–Irregular (NODES, SE−, SE+)

2 let DNODES be an empty array

3 sort NODES according to y-value in descending order

4 for each node i ∈ NODES do5 let i− and i+ be duplicates of i6 let pos(i−) = pos(i) + SE−

7 let pos(i+) = pos(i) + SE+

8 let NODES− be the list of nodes that precede i9 let j− be the nearest neighbor of i in NODES−, s.t. pos(j−) < pos(i)

10 let j+ be the nearest neighbor of i in NODES−, s.t. pos(j+) > pos(i)11 //If j− does not exist, let pos(j−) = −∞12 //If j+ does not exist, let pos(j+) = +∞13 if pos(i−) ≤ pos(j−) + SE+ then14 pos(i−) = pos(j−) + SE+ + ε

15 if pos(i+) ≥ pos(j+) + SE− then16 pos(i−) = pos(j−) + SE− − ε17 if pos(i−) ≤ pos(i+) then18 if �n ∈ DNODES : pos(i−) = pos(n) then19 insert i− into DNODES

20 if �n ∈ DNODES : pos(i+) = pos(n) then21 insert i+ into DNODES

22 if pos(i−) < pos(i) < pos(i+) then23 insert i into DNODES

24 else25 drop nodes i−, i, and i+

26 return DNODES

However, paper II also deals with questions about how to sample the output

signal to avoid a dense sampling everywhere without adversely affecting the

result. Here we give an overview of the approach in paper IV which is a more

general case of the algorithm in paper III allowing for more varied structuring

elements.

The following are necessary:

• A signed scalar field f(x, y, xc, yc, v, r) that describes a structuring ele-

ment centered at position (xc, yc) that depends on some value v (usually

derived from the input signal), and whose size is determined by r. Here

f < 0 indicates the inside of a SE, f > 0 the outside, and 0 the edge.

40

}Figure 3.4. Illustration of the steps for processing one sample (the blue “x”) as viewed

from the side (therefore the flat SEs look like line segments). The blue samples are

taken on the border of a SE placed on top of the sample currently being processed. The

red samples are taken on the border of a slightly larger SE. The samples that fall in the

shadow of a SE are discarded (the blue “x” and the red and blue stars). The remaining

samples from the larger SE are dropped downward until they intersect a SE at a lower

value, or fall through to −∞ (in which case they are discarded). These steps are then

repeated until all input samples have been processed.

• A way of sampling the zero-set f = 0, which we denote ∂f(xc, yc, v, r).

For the sake of simplicity, let us consider a function f(x, y, xc, yc, r) that is

independent of v, and its zero-set ∂f(xc, yc, r). To perform a dilation of size

r on an input signal

I = {(xi, yi, zi)}i∈{1,2,...,N} (3.4)

of N samples, where xi and yi is the sample position, and zi is its value, do

the following (see Fig. 3.4):

1. Let I0δ = ∅, in subsequent iterations, we will fill out sets Iiδ with samples

from the dilation.

2. Select an unprocessed sample (xc, yc, zc) ∈ I and mark it as processed

(blue “x” In Fig. 3.4).

3. Sample ∂f(xc, yc, r). Denote this set of samples B (shown in blue in

Fig. 3.4).

4. Sample ∂f(xc, yc, r + ε), where ε > 0 is some small real number. We

call this set of samples Bε (shown in red).

5. For each sample (x′, y′) ∈ B, and each sample (xi, yi, zi) ∈ I \{xc, yc, zc},such that zi ≥ zc, compute fi = f(x′, y′, xi, yi, r+ε). If fi ≤ 0, discard

(x′, y′) (see the blue star and blue “x” in Fig. 3.4). Let B̂ denote the

remaining set of samples (blue circle). This step discards samples from

41

the smaller SE that end up in the shadow (umbra) of a SE placed at a

higher sample.

6. Repeat the procedure in 5. except replacing B with Bε. Let B̂ε denote

the remaining samples (shown as the red circle at the top of the arrow in

Fig. 3.4). Figure 3.4 shows one such sample being discarded (red star).

7. Let

H(x, y) = {zi | (xi, yi, zi) ∈ I∧zi < zc∧f(x, y, xi, yi, r) ≤ 0} (3.5)

and let

zh(x, y) = max (H(x, y) ∪ −∞) (3.6)

Then, for each sample (x′ε, y

′ε) in B̂ε, compute zh(x

′ε, y

′ε). This means,

zh is the value of a given sample of the bigger SE (B̂ε) after being

projected downward until hitting a SE at a lower value (or falling through

to −∞). This is illustrated in Fig. 3.4 by the red circle.

8. Let

B̂c = {(x, y, zc) | (x, y) ∈ B̂}, (3.7)

B̂hε = {(x, y, zh(x, y)) | (x, y) ∈ B̂ε}, and (3.8)

Ii+1δ = Iiδ ∪ B̂c ∪ B̂h

ε (3.9)

Then INδ is the dilated signal.

In essence, two structuring elements are centered on top of each input sample

(x, y, z), one slightly larger than the other, and then their borders are sampled

(steps 2-4). After this, all newly created samples are checked and those that

fall in the shadow of a SE placed at a neighboring sample at a higher position

are removed (steps 5-6). The samples from the larger SE are projected down

until they hit a SE at a lower level, or fall through (step 7). The union of all the

sampled structuring elements is the dilated signal.

Figure 3.5 shows an example of applying the procedure described above

to a point cloud to compute a tophat of a scene containing a signpost. The

opening is computed by composing an erosion and a dilation, the erosion can

be computed using the same procedure, by duality. In order to subtract the

opening from the original signal, the opening is interpolated back onto the

original sampling positions using linear interpolation.

Since the structuring elements are defined as signed scalar fields it is possible

to combine several simple scalar fields to generate more complicated ones.

Note especially that taking the supremum/infimum of a pair of scalar fields

is like taking the intersection/union of the SEs that they represent. I.e., for

a pair of structuring elements B1 and B2 represented by the scalar fields f1and f2, the scalar field f1 ∨ f2 will represent the SE B1 ∩ B2, and similarly

f1∧f2 represents the SE B1∪B2. This can be used to construct intricate SEs.

42

0-0.5

43-131

43.5

-1-131.5

44

Signpost

-132 -1.5

44.5

-132.5

45

45.5

46

0-0.5

43-131

43.5

-1-131.5

44

Eroded signpost

-132 -1.5

44.5

-132.5

45

45.5

46

0-0.5

43-131

43.5

-1-131.5

44

Open

-132 -1.5

44.5

-132.5

45

45.5

46

00 -0.5-131

0.5

-1-131.5

1

Tophat of signpost

-132

1.5

-1.5-132.5

2

2.5

3

Figure 3.5. Examples of irregular morphology applied to a simple point cloud of a

signpost. Here, the SE is a disk with radius 0.016m.

Figure 3.6 shows some examples. Moreover a SE can be defined by sampling

a function and interpolating values to generate the scalar field.

3.5 Adaptive Morphology

So far we have only looked at the case where the SE is fixed. However,

there are cases where one may want to change the shape of the SE based

on, for example, its position (location-adaptive) or some local property of the

image (input-adaptive). This concept of adaptive morphology was previously

described in the regularly sampled case in Section 2.9. In this section the

work on MM on irregularly sampled signals is extended to allow for adaptive

morphology in the irregular case.

Returning to the idea of SEs as signed scalar fields, f(x, y, xc, yc, v, r),one may perform the same eight steps presented in the previous section even

if the shape of f depends on (xc, yc) or some value v. As an example,

we adapt the work by Landström and Thurley [32], which presents adaptive

morphology using different elliptical structuring elements based on the local

structure tensor [28] for regularly sampled signals.

3.5.1 Elliptical Adaptive Structuring Elements

In this section a brief description of the work by Landström and Thurley [32]

on adaptive morphology using elliptical SEs is presented. See also the thesis

by Landström [30]. First a brief description of the local structure tensor [28]

43

Figure 3.6. The top four images show the result of taking the minimum of a p-norm

disk with four smaller translated 2-norm disks. The bottom four images show the

minimum of the four pairs of maxima of the p-norm disk and each of the four smaller

2-norm disks. Combining signed scalar functions using ∨ and ∧ (essentially working

as intersection and union) allows for a wide variety of structuring elements.

44

Figure 3.7. The elliptical SE is shaped by the eigenvectors and their associated

eigenvalues. The semi-major axis (a) is aligned with the eigenvector e2, and the semi-

minor axis (b) aligns with the eigenvector e1. The length of these axes are decided by

the eigenvalues.

Local Structure TensorConsider a function I(x) that maps tuples x = (x1, x2) ∈ R2 to R (e.g. a

grayscale image). The local structure tensor is defined as:

LST (I)(x) = Gσ ∗(∇I(x)∇T I(x)

), (3.10)

where∇ =(

∂∂x1

, ∂∂x2

)T

andGσ is a Gaussian filter with standard deviationσ.

I.e. each x is mapped to a 2× 2–matrix. This matrix contains local directional

information. Computing the eigenvalues λ1(x) ≥ λ2(x), the following cases

apply:

1. λ1 ≈ λ2 � 0: A crossing or a point.

2. λ1 � λ2 ≈ 0: A dominant direction.

3. λ1 ≈ λ2 ≈ 0: No edge.

The eigenvectors e1(x) and e2(x) associated with λ1(x) and λ2(x) point in

the direction of largest and smallest variation respectively [14].

Adaptive Morphology Using Elliptical SEsThe approach proposed by Landström and Thurley computes the LST and

shapes an elliptical SE at each position x such that the major axis depends

on e2 and the minor axis on e1, rotating the ellipse accordingly, as shown

in Fig. 3.7. The size of the ellipse is determined by the eigenvalues. This

approach can be used to preserve strong edges when filtering, because the

ellipses near edges will be thin and oriented along the edge. In a later paper,

Landström extends this approach, using non-flat morphology [31].

45

-20 0 20 40

-130-120-110-100-90-80-70-60

Input

-20 0 20 40

Angles

-20 0 20 40

Interpolated adaptive erosion

Figure 3.8. The top image shows the irregularly sampled input (a pile of rocks). The

figure on the bottom left illustrates the angles used to align the elliptical SEs, based on

the eigenvectors of the LST. The bottom right shows an example of an erosion using

the adaptive, elliptical SEs.

3.5.2 Adapting the Elliptical Adaptive Structuring Elements toIrregularly Sampled Signals

To allow for information about the LST to affect the scalar fields, it is passed as

a parameter: f(x, y, xc, yc, LST (I)(xc, yc), r). To compute the LST for the

irregularly sampled signal a morphological approach is used to estimate the

gradient. In paper V an approach for estimating the components of the gradient

in cases of regularly sampled signals with missing samples is presented. The

same approach can be applied in the case of irregularly sampled signals (in

fact, regularly sampled signals with missing pixels can be seen as a special

case of irregularly sampled signals) exchanging regular, discrete morphological

operators for their irregular counterparts. Figure 3.8 shows this approach

applied to an irregularly sampled rock pile image to adaptively erode the

image. Figure 3.9 shows an example of applying this approach, illustrating

several of the steps, to a regularly sampled signal where 50% of samples are

46

first discarded, in order to get an irregularly sampled image. This is followed

by an adaptive dilation (as described above), and finally an interpolation onto

the regular grid for visualization purposes.

47

Figu

re3.

9.T

his

figure

illu

stra

tes

seve

ral

step

sof

the

irre

gula

rap

pro

ach

for

elliptica

lad

aptive

stru

cturi

ng

elem

ents

.Fro

mle

ftto

right:

The

regula

rly

sam

ple

din

putim

age.

The

inputim

age

afte

rdis

card

ing

50%

of

pix

els

(show

nin

red),

thus

yie

ldin

gan

irre

gula

rly

sam

ple

dim

age,

and

dra

win

gso

me

of

the

elliptica

lSE

sth

atar

ead

apte

dbas

edon

the

LST.

The

resu

ltof

dilat

ing

the

irre

gula

rly

sam

ple

dim

age

usi

ng

the

adap

ted

elliptica

lSE

s.Fin

ally

,th

ela

stim

age

show

sth

edilat

ion

afte

rin

terp

ola

ting

bac

konto

the

regula

rgri

dof

the

ori

gin

alim

age.

This

step

isnot

nec

essa

ryfo

rfu

rther

morp

holo

gic

alpro

cess

ing,butai

ds

vis

ual

izat

ion.

48

4. Conclusions and Future Work

4.1 Contributions

The major contributions of this thesis are:

• Developing a framework for mathematical morphology on irregularly

sampled signals (papers I, II, III, and IV)

• Improving the approximation of continuous morphology in the discrete

domain (papers I, II, III, and IV) by allowing for irregularly sampled

signals as output for morphology in the discrete domain, thus decoupling

from the sampling grid, which enables working around problems, such

as those stemming from non-linear filters introducing higher frequency

content into the filtered signal, thereby breaking the correspondence

between a band-limited, correctly sampled signal and its continuous

counterpart after applying a filter.

• Proposing a faster, approximate path opening (paper VI) by taking inspi-

ration from the parsimonious path opening, preselecting paths of interest

and applying an opening only to the preselected paths. The speed of

the proposed algorithm is shown to be comparable to the parsimonious

path opening. However, the proposed path opening does not suffer from

problems of occlusion present in the case of the PPO. Additionally, we

propose less biased weights for path lengths and show that this translates

to less biased length measurements for the proposed path opening than

those of the regular path opening or the PPO. We also apply the pro-

posed morphology on irregularly sampled signals to find a more precise

approximate parsimonious path opening (paper I), since the preselected

paths are essentially irregularly sampled 1D signals.

• Applying the framework for morphology on irregularly sampled signals

to the problem of segmenting 3D point clouds of urban scenes (paper III).

Since the proposed framework allows for irregularly sampled signals as

input, this allows us to apply morphology on the original data without the

need for resampling onto a regular grid, which is the common approach.

• Developing a way of estimating the components of the gradient for ir-

regularly sampled images, or images with missing samples (paper V).

The foundation of the proposed approach is morphology using elliptical

structuring elements. Regularly sampled images with missing samples

can be seen as a special case of irregularly sampled signals. A common

way of dealing with this type of problem is to use normalized differen-tial convolutions. However, NDC is not as easily applied to the general

49

case of irregularly sampled signals. The proposed approach shows com-

petitive performance and is applicable in the general case of irregular

sampling, since at its core the method uses morphology, meaning the

previously developed framework for MM on irregularly sampled data is

applicable.

• Developing adaptive morphology for irregularly sampled signals with

SEs described by scalar fields, using the local structure tensor to shape

elliptical SEs (paper IV). The necessary gradient estimation is performed

using the previously proposed method.

4.2 Future Work

There are several avenues to explore in the future, extending the current work.

Firstly, the implementation of the variants of the proposed approach to morphol-

ogy on irregularly sampled signals could probably be sped up by incorporating

some additional structure to aid computation (e.g. by creating a graph span-

ning the samples of the input signal or some subset thereof in order to enable

quicker search for neighbors, which is a necessary part of the proposed algo-

rithms). Another speed improvement may be possible by performing openings

and closings in one step, not as a composition of erosions and dilations.

It should be possible to extend the work to higher dimensions. In the case of

path openings and USPO presented in paper VI, there is not much hindering a

generalization to nD. The main obstacle is the skeletonization step. However,

assuming a suitable grayscale nD-skeletonization algorithm, the rest of the

algorithm is readily modified to deal with higher dimensionalities by using

different adjacency graphs and adjusting the weights appropriately. In the 3D

case, a suitable skeletonization algorithm might be developed by combining the

ideas of Verwer et al. [72] with some 3D-skeletonziation algorithm [18, 60].

For the gradient estimation presented in paper V, the same approach, essentially,

should be applicable in higher dimensions as well, simply by exchanging the

ellipses by n-dimensional ellipsoids oriented along relevant directions. The

remaining papers dealing with the general case of morphology on irregularly

sampled signals should also be possible to extend, although there may be issues

to consider regarding what is the inside/outside of objects as the morphological

operators are applied.

There are many questions of interest regarding how to sample the trans-

formed, irregularly sampled signals. This has been treated, to some extent,

in the papers introducing the variants of the approaches to MM on irregularly

sampled signals (specifically papers I, II, III, and IV), for example by experi-

menting with adapting sampling density based on properties of the input signal

(e.g. the gradient or the Laplacian), but more extensive experiments would be

desirable. A related issue is how to best sample the structuring elements. In-

tuitively it would seem beneficial to increase the sampling density near sharp

50

features of the border of the support of the SE. Making sure to sample the exact

positions of extreme values of non-flat SEs, as well as increasing the sampling

density in areas where the height of the SE does not vary smoothly should also

be helpful.

Although the proposed approach allows for dealing with irregularly sampled

signals directly, there are sometimes cases where interpolation of the resulting

transformed, irregular signal is desired. In these cases it is not suitable to apply

some smooth interpolation everywhere, since the morphological operators

generally create cusps in the output signal, meaning it is not smooth. On

the other hand, there are generally parts of the transformed signal that behave

smoothly. Thus a hybrid approach may be suitable, in cases where interpolation

is desired, applying a spline-based interpolation at smooth parts of the signal

for example, but using linear interpolation near sharply varying parts of the

transformed signal.

Another avenue of potential interest is to use the proposed approach together

with the pixel coverage model [59, 36], which converts an image into a fuzzy

set where boundary pixels belong to the object by some fraction that indicates

how much of the pixel is covered by the object, in order to further improve

image-based measurements.

Finally, when dealing with irregularly sampled signals, commonly there are

holes in the data, in the case of range cameras due to the surface geometry.

In the different variants of the proposed approach such cases are never dealt

with explicitly, instead treating every part of the signal the same, regardless

of sampling density. However, this may not be the best approach, since the

information content near these regions is sparse. Thus, it could be useful to

explicitly handle holes by marking such parts of the signal and taking into

account these regions when applying different operations.

51

5. Brief Summaries of the Included Papers

Paper I:This paper introduces a new approach to one-dimensional morphology on

irregularly sampled signals. The paper gives three main reasons as to why

allowing for irregularly sampled signals as output is helpful if one is interested

in approximating continuous morphology in the discrete domain. Empirically,

this is also shown by applying the approach to continuous, synthetic 1D-signals

as well as their regularly sampled counterparts. The irregularly sampled output

as well as the result of applying regular, discrete morphology is compared

against the transformed continuous signal and the proposed approach is shown

to yield a better approximation.

Additionally, allowing for irregularly sampled input makes it possible to

apply morphology directly, without resampling onto a regular grid. The ben-

efit of this is illustrated on an example where paths are preselected through

an image containing a long, thin structure using the same approach as for

parsimonious path opening. These preselected paths are one dimensional sig-

nals and can be interpreted as irregularly sampled, since the distance between

horizontal/vertical neighbors is shorter than that between diagonal neighbors.

The proposed approach is applied to these irregularly sampled signals to yield

an approximate path opening and shown to give better results than regular,

discrete morphology.

Paper II:In this paper the approach to morphology on irregularly sampled signals is

generalized to two dimensions. The paper shows that the proposed approach

yields better approximations of continuous morphology in two dimensions

than regular, discrete morphology. The paper also deals with issues of how

to sample the transformed signal and shows that the sampling density can be

drastically reduced in parts of the transformed signal (namely on plateaus)

without adversely affecting the result, thus significantly decreasing the num-

ber of samples in the transformed signal, which leads to faster subsequent

computations.

Paper III:We improve the approach presented in paper II by sampling the structuring

element differently and modifying the proposed approach appropriately. This

deals with some problems with fuzzy edges that sometimes arose when ap-

plying morphological operators using the previous approach. The paper also

52

applies this improved variant to 3D point clouds (which is a natural target for

the approach, since it deals with irregularly sampled data) of urban scenes in

order to segment the scene into objects of interest.

Paper IV:This paper generalizes the results of paper III by allowing for a much wider

range of structuring elements, both flat and non-flat. The generalization also

allows for adaptive morphology. We make use of the approach presented in

paper V to estimate the gradient for irregularly sampled signals and use this

to compute the local structure tensor in order to apply adaptive morphology,

shaping the structuring element based on directional information in the un-

derlying signal, as is done by Landström and Thurley in the case of regularly

sampled images [32].

This paper also discusses, in some detail, how to sample the output signal

based on the input signal as well as the shape of the structuring element.

Paper V:Regularly sampled images with missing samples can be seen as a special

case of irregularly sampled signals (where all samples fall on grid points, but

not at regular intervals). In this paper we propose a way of using rotated

half-ellipses as structuring elements in order to estimate components of the

gradient. The proposed approach is compared against normalized differential

convolutions and Gaussian derivatives (where the input is preprocessed to

deal with missing samples) and performs favorably. An additional advantage

of the proposed approach is that it generalizes readily to the general case of

irregularly sampled input, by replacing the regular, discrete morphology with

the previously proposed irregular variants.

Paper VI:This paper proposes an approximate variant (called USPO) that is much faster

(approximately an order of magnitude for large images) than implementa-

tions of the traditional path opening. The proposed variant also makes use

of weighted adjacency graphs to achieve less biased measurements of length.

USPO takes some inspiration from another proposed variant, called the par-

simonious path opening, which preselects one-dimensional paths through the

image. However, the preselection has problems where bright structures closer

to the edge of the image may occlude structures of interest toward the center of

the image. USPO also preselects paths, but the proposed approach of USPO

does not have the same problem of occlusions. The paper demonstrates benefits

over several path opening variants and also shows experiments where the pro-

posed path opening variant is applied to the problem of detecting blood vessels

in retinal fundus images following the approach of Sigurðsson et al. [58] but

using the USPO instead of the regular path opening. In summary, the proposed

variant has two advantages: speed and less biased length measurements.

53

6. Summary in Swedish

Denna avhandling “Precise Image-Based Measurements through Irregular

Sampling,” eller “Noggranna bildbaserade mätningar via irreguljär sampling”

behandlar frågor gällande så kallad matematisk morfologi för både kontin-

uerliga signaler och deras motsvarande samplade (diskreta) signaler.

Matematisk morfologi är ett brett använt verktyg för signalbehandling. Den

grundläggande idén är att transformera en signal genom att systematiskt föra

en “sond” över signalen, ett s.k. strukturelement, och undersöka hur väl den

passar in i signalen, i någon bemärkelse. Detta strukturelement kan ses som

ett geometriskt objekt, t.ex. en cirkelskiva. Ibland associeras en funktion till

strukturelementet som anger en viss höjd för varje punkt. Om strukturele-

mentet saknar en sådan funktion (alternativt om funktionen är 0 överallt i

strukturelementet) kallas det platt.

I denna avhandling fokuseras speciellt på bilddata. Ett annat välanvänt

verktyg för (bland annat) bildbehandling är s.k. faltning med olika faltnings-

matriser. Detta angreppssätt ger upphov till linjära filter, som inte påverkar

represenationsegenskaperna för signaler samplade på ett samplingsgitter, enligt

Nyquist-Shannon samplingsteori [47, 57]. Morfologiska filter, å andra sidan,

är icke-linjära och därmed gäller ej detsamma för dessa filter. Som följd av

detta kan olika resultat nås t.ex. beroende endast på ett objekts exakta position

i en kameras synfält. Detta är normalt inte önskvärt, eftersom det vanligtvis är

det avbildade objektet som är av intresse, inte själva bilddatan i sig. Detta är en

av anledningarna till att irreguljär morfologi är av intresse. Genom att ta fram

ett tillvägagångssätt för morfologi på irreguljärt samplad data kan vi undvika,

eller åtminstone minska påverkan av problem som uppstår p.g.a. samplings-

gittret då vi är intresserade av att approximera den kontinuerliga signalen som

bakomligger den digitala signalen, d.v.s. det faktiska objektet, inte bilddatan i

sig.

Utöver de fördelar som finns med att gå ifrån reguljär sampling (såsom

pixlar i ett reguljärt gitter) gällande approximering av kontinuerlig morfologi,

så finns också en stor mängd av data som till sin natur är irreguljärt samplad

från början, t.ex. är data från olika typer av avståndskameror ofta represen-

terad som irreguljärt samplade datapunkter. Traditionellt skulle sådan data

behandlas morfologiskt genom att först interpolera sampel på ett reguljärt git-

ter och sedan applicera reguljär, diskret morfologi. Interpolationen kan leda

54

till icke önskvärda bildartefakter som påverkar resultatet negativt, speciellt om

större hål, d.v.s. områden där sampelpunkter fattas, förekommer i datat. Det

tillvägagångssätt som tagits fram i denna avhandling kan appliceras direkt på

irreguljärt samplad data (såsom i artikel III t.ex.) och kräver alltså inte detta

interpolationssteg.

Denna avhandlig har fem huvudmål, nämligen:

• Utveckling av ett ramverk för matematisk morfologi på irreguljärt sam-

plade signaler.

• Förbättrad approximation av kontinuerlig morfologi i den diskreta domä-

nen via irreguljär sampling eller viktade mått.

• Tillämpning av ramverket för morfologi på irreguljärt samplad data så-

som 3D-punktmoln.

• Särskild behandling av specialfallet av irreguljärt samplade signaler som

kan tolkas som reguljärt samplade signaler där vissa sampel fattas.

• Utveckling av adaptiv och icke-platt morfologi i fallet med irreguljärt

samplad data.

I avhandlingen ingår sex artiklar som sammanbinds av en röd tråd, nämligen

irreguljär sampling. Artikel VI behandlar s.k. path opening som är en sorts

morfologisk operation på reguljärt samplade bilder. I denna artikel föreslås

en variant som approximerar vanlig path opening genom att välja ut “vägar”

((eng.) paths) av intresse och sedan behandla endast dessa. Detta utval ger

en samling pixlar som kan tolkas som en irreguljärt samplad signal. Artikeln

innehåller bl.a. experiment som visar att den föreslagna varianten är snabb att

beräkna och ger ett mer korrekt resultat vad gäller uppskattning av längder.

Artikel V behandlar också reguljärt samplade bilder. Kopplingen till temat

här är att vi undersöker fallet då en del sampel (pixlar) saknas, vilket kan ses

som ett specialfall av irreguljärt samplade bilder. I artikeln föreslås ett sätt att

använda morfologi för att uppskatta gradienten i denna typ av bilder.

Kvarvarande artiklar behandlar det generella fallet där signaler är samplade

irreguljärt från början. Dessa artiklar utvecklar successivt mer generella verk-

tyg för behandling av irreguljärt samplade signaler från endimensionell data till

två och (i viss mån) tre dimensioner och från platta strukturelement med strikta

begränsningar på deras form till strukturelement som kan anta en stor mängd

olika former, inte nödvändigtvis platta. Slutligen generaliseras även resultaten

så att adaptiv morfologi är möjlig, d.v.s. morfologi med strukturelement som

ändrar form.

Sammanfattningsvis innehåller denna avhandling en samling av sex artiklar

gällande matematisk morfologi och irreguljärt samplade signaler och en intro-

duktion till dessa. Matematisk morfologi används ofta på diskreta, reguljärt

samplade bilder, men är också definierad i det kontinuerliga fallet. Av flera skäl

kan en bättre approximation av kontinuerlig morfologi uppnås i den diskreta

domänen, om irreguljär sampling tillåts. Detta är en motiverande faktor till

arbetet. En annan fördel med att kunna applicera morfologi i det irreguljära

55

fallet är att man då enkelt kan behandla data som är irreguljärt samplad från

början. Denna typ av data är vanligt förekommande. Ett exempel på detta är

data från avståndskameror.

56

7. Acknowledgments

My time at the Centre for Image Analysis at the Division of Visual Information

and Interaction has been a pleasure, not least because of the great company. I

would like to express my thanks to the following people:

• Robin Strand, my main supervisor for most of my PhD. Thank you for

the support and for giving me a lot of freedom to choose my own research

directions.

• Cris L. Luengo Hendriks, my main supervisor during the first part of

my studies. Thank you for the stimulating discussions. I hope you are

having a great time at your new job!

• Gunilla Borgefors, for being a supportive co-supervisor, especially dur-

ing the beginning of my PhD.

• Matthew J. Thurley, my other co-supervisor, for helpful comments

during paper writing and research.

• Elisabeth Wetzer, for helpful comments on a draft of this thesis, for your

efforts on the SSBA newsletter, and for being such a nice office mate!

• Johan Öfverstedt, for helpful comments on a draft of this thesis and for

the fun discussions on research and more.

• Eva Breznik, for being a good friend and research collaborator. Hope-

fully we can continue working together on the Graph CNNs. Also, thanks

for the vegan cakes!

• Damian Matuszewski, for the nice lunch discussions on games, TV,

swords, research, etc., and thanks for organizing the board game evenings!

• Fredrik Nysjö. I had fun teaching the graphics course together with you

(and learned a lot)!

• The MIDA research group, for being such a nice bunch of people. The

Monday group meetings always go by so quickly.

• My office mates. Thanks to everyone (Axel Andersson, Ankit Gupta,Raphaela Heil, Gabriele Partel, Nicolas Pielawski, Leslie Solorzano,Elisabeth Wetzer, Håkan Wieslander, and Johan Öfverstedt) for mak-

ing the office such a good working environment.

• All my past and present colleagues at CBA, and Vi2 in general. As I

said, it has been a pleasure.

• The group at the Centre for Mathematical Morphology in Fontaine-bleau, for welcoming me during my visit, and for the fruitful and fun

collaboration.

• My parents, Elof Asplund and Tua Borgmästars, for being supportive

and for motivating me to work harder.

57

• My brother, Sam Asplund, for being a great older brother (even if you

sometimes hogged the family computer, or jumped out from behind a

corner to scare me, when we were kids).

• My younger brother, Björn Asplund. It is always interesting to talk with

you. I hope, and think, you have had a lot of fun at Gotland’s the past

year and I am looking forward to seeing what you will do in the future.

• My nephews, Elof Birger Brøvig Asplund, and Osvald JohannesBrøvig Asplund for being welcome distractions when I see you dur-

ing the holidays, especially when needing a break from thinking about

work.

58

References

[1] J. Angulo. Morphological scale-space operators for images supported on point

clouds. In International Conference on Scale Space and Variational Methods inComputer Vision, pages 78–89. Springer, 2015.

[2] T. Asplund, C. L. Luengo Hendriks, M. J. Thurley, and R. Strand. A new

approach to mathematical morphology on one dimensional sampled signals. In

2016 23rd International Conference on Pattern Recognition (ICPR), pages

3904–3909. IEEE, 2016.

[3] S. Beucher. Segmentation d’images et morphologie mathématique. PhD thesis,

1990.

[4] S. Beucher, J. M. Blosseville, and F. Lenoir. Traffic spatial measurements using

video image processing. In Intelligent Robots and Computer Vision VI, volume

848, pages 648–655. International Society for Optics and Photonics, 1988.

[5] G. Birkhoff. Lattice theory, volume 25. American Mathematical Soc., 1940.

[6] H. Blum. A transformation for extracting new descriptors of shape. Models forthe perception of speech and visual form, 19(5):362–380, 1967.

[7] M. Breuß, B. Burgeth, and J. Weickert. Anisotropic continuous-scale

morphology. In Iberian Conference on Pattern Recognition and Image Analysis,pages 515–522. Springer, 2007.

[8] R. W. Brockett and P. Maragos. Evolution equations for continuous-scale

morphology. In Acoustics, Speech, and Signal Processing, 1992. ICASSP-92.,1992 IEEE International Conference on, volume 3, pages 125–128. IEEE, 1992.

[9] R. W. Brockett and P. Maragos. Evolution equations for continuous-scale

morphological filtering. Signal Processing, IEEE Transactions on, 42(12):

3377–3386, 1994.

[10] P. Brodatz. Textures: A Photographic Album for Artists and Designers. Dover

Publications, 1966.

[11] M. Buckley and H. Talbot. Flexible linear openings and closings. In

Mathematical Morphology and its Applications to Image and Signal Processing,

volume 18, pages 109–118. Springer, Proceedings of the International

Symposium on Mathematical Morphology 2000.

[12] B. Burgeth and A. Kleefeld. An approach to color-morphology based on Einstein

addition and Loewner order. Pattern Recognition Letters, 47:29–39, 2014.

[13] S. Calderon and T. Boubekeur. Point morphology. ACM Transactions onGraphics (TOG), 33(4):45, 2014.

[14] L. Cammoun, C. A. Castaño-Moraga, E. Muñoz-Moreno, D. Sosa-Cabrera,

B. Acar, M. Rodriguez-Florido, A. Brun, H. Knutsson, and J. Thiran. A review

of tensors and tensor signal processing. In Tensors in Image Processing andComputer Vision, pages 1–32. Springer, 2009.

[15] C.-S. Chen, J.-L. Wu, and Y.-P. Hung. Theoretical aspects of vertically invariant

gray-level morphological operators and their application on adaptive signal and

59

image filtering. IEEE transactions on signal processing, 47(4):1049–1060,

1999.

[16] F. Cheng and A. N. Venetsanopoulos. An adaptive morphological filter for

image processing. IEEE Transactions on Image Processing, 1(4):533–539,

1992.

[17] F. Cheng and A. N. Venetsanopoulos. Adaptive morphological operators, fast

algorithms and their applications. Pattern recognition, 33(6):917–933, 2000.

[18] M. Couprie and G. Bertrand. A 3D sequential thinning scheme based on critical

kernels. In International Symposium on Mathematical Morphology and ItsApplications to Signal and Image Processing, pages 549–560. Springer, 2015.

[19] J. Cousty, L. Najman, F. Dias, and J. Serra. Morphological filtering on graphs.

Computer Vision and Image Understanding, 117(4):370–385, 2013.

[20] V. Ćurić, A. Landström, M. J. Thurley, and C. L. Luengo Hendriks. Adaptive

mathematical morphology–a survey of the field. Pattern Recognition Letters,47:18–28, 2014.

[21] M. J. E. Golay. Hexagonal parallel pattern transformations. IEEE Transactionson computers, 100(8):733–740, 1969.

[22] H. Hadwiger. Minkowskische addition und subtraktion beliebiger punktmengen

und die theoreme von Erhard Schmidt. Mathematische Zeitschrift, 53(3):

210–218, 1950.

[23] M. S. Hassouna and A. A. Farag. Multistencils fast marching methods: A highly

accurate solution to the eikonal equation on cartesian domains. IEEEtransactions on pattern analysis and machine intelligence, 29(9), 2007.

[24] H. Heijmans, M. Buckley, and H. Talbot. Path openings and closings. Journal ofMathematical Imaging and Vision, 22(2-3):107–119, 2005.

[25] H. J. A. M. Heijmans and C. Ronse. The algebraic basis of mathematical

morphology I. Dilations and erosions. Computer Vision, Graphics, and ImageProcessing, 50(3):245–295, 1990.

[26] S. Klemenjak, B. Waske, S. Valero, and J. Chanussot. Automatic detection of

rivers in high-resolution SAR data. IEEE Journal of Selected Topics in AppliedEarth Observations and Remote Sensing, 5(5):1364–1372, 2012.

[27] D. E. Knuth. The art of computer programming, vol. 3: Sorting and searching

(2nd ed.). 1998.

[28] H. Knutsson. Representing local structure using tensors. In 6th ScandinavianConference on Image Analysis, Oulu, Finland, pages 244–251. Linköping

University Electronic Press, 1989.

[29] U. Köthe. What can we learn from discrete images about the continuous world?

In International Conference on Discrete Geometry for Computer Imagery, pages

4–19. Springer, 2008.

[30] A. Landström. Elliptical Adaptive Structuring Elements for MathematicalMorphology. PhD thesis, Luleå tekniska universitet, 2014.

[31] A. Landström. An approach to adaptive quadratic structuring functions based on

the local structure tensor. In International Symposium on MathematicalMorphology and Its Applications to Signal and Image Processing, pages

729–740. Springer, 2015.

60

[32] A. Landström and M. J. Thurley. Adaptive morphology using tensor-based

elliptical structuring elements. Pattern Recognition Letters, 34(12):1416–1422,

2013.

[33] C. Lantuéjoul and S. Beucher. On the use of the geodesic metric in image

analysis. Journal of Microscopy, 121(1):39–49, 1981.

[34] C. Lantuéjoul and F. Maisonneuve. Geodesic methods in quantitative image

analysis. Pattern recognition, 17(2):177–187, 1984.

[35] R. Lerallut, E. Decenciere, and F. Meyer. Image filtering using morphological

amoebas. Image and Vision Computing, 25(4):395–404, 2007.

[36] J. Lindblad and N. Sladoje. Coverage segmentation based on linear unmixing

and minimization of perimeter and boundary thickness. Pattern RecognitionLetters, 33(6):728–738, 2012.

[37] F. Lozes, A. Elmoataz, and O. Lézoray. Partial difference operators on weighted

graphs for image processing on surfaces and point clouds. IEEE Transactions onImage Processing, 23(9):3896–3909, 2014.

[38] C. L. Luengo Hendriks. Constrained and dimensionality-independent path

openings. IEEE Transactions on Image Processing, 19(6):1587–1595, 2010.

doi: 10.1109/TIP.2010.2044959.

[39] C. L. Luengo Hendriks, G. M. van Kempen, and L. J. van Vliet. Improving the

accuracy of isotropic granulometries. Pattern Recognition Letters, 28(7):

865–872, 2007.

[40] P. Maragos and C. Vachier. Overview of adaptive morphology: Trends and

perspectives. In 2009 16th IEEE International Conference on Image Processing(ICIP), pages 2241–2244. IEEE, 2009.

[41] G. Matheron. Random sets and integral geometry. 1975.

[42] H. Minkowski. Volumen und oberfläche. In Ausgewählte Arbeiten zurZahlentheorie und zur Geometrie, pages 146–192. Springer, 1989.

[43] V. Morard, P. Dokládal, and E. Decenciere. One-dimensional openings,

granulometries and component trees inO(1) per pixel. Selected Topics in SignalProcessing, IEEE journal of, 6(7):840–848, 2012.

[44] V. Morard, P. Dokládal, and E. Decencière. Parsimonious path openings and

closings. IEEE Transactions on Image Processing, 23(4):1543–1555, 2014. doi:

10.1109/TIP.2014.2303647.

[45] L. Najman and J. Cousty. A graph-based mathematical morphology reader.

Pattern Recognition Letters, 47:3–17, 2014.

[46] L. Najman and H. Talbot, editors. Mathematical morphology: from theory toapplications. John Wiley & Sons, 2013.

[47] H. Nyquist. Certain topics in telegraph transmission theory. Transactions of theAIEE, pages 617–644, 1928. [reprinted in: Proceedings of the IEEE, vol. 90, no.

2, pp. 280-305, February 2002].

[48] I. Nyström. On quantitative shape analysis of digital volume images. PhD

thesis, 1997.

[49] M. Peternell and T. Steiner. Minkowski sum boundary surfaces of 3D-objects.

Graphical Models, 69(3-4):180–190, 2007.

[50] D. Proffitt and D. Rosen. Metrication errors and coding efficiency of

chain-encoding schemes for the representation of lines and edges. Computergraphics and image processing, 10(4):318–332, 1979.

61

[51] J.-F. Rivest, P. Soille, and S. Beucher. Morphological gradients. Journal ofElectronic Imaging, 2(4):326–337, 1993.

[52] C. Ronse. Why mathematical morphology needs complete lattices. Signalprocessing, 21(2):129–154, 1990.

[53] C. Ronse and H. J. A. M. Heijmans. The algebraic basis of mathematical

morphology: II. Openings and closings. CVGIP: Image Understanding, 54(1):

74–97, 1991.

[54] P. K. Saha, G. Borgefors, and G. S. di Baja. A survey on skeletonization

algorithms and their applications. Pattern Recognition Letters, 76:3–12, 2016.

[55] G. Sapiro, R. Kimmel, D. Shaked, B. B. Kimia, and A. M. Bruckstein.

Implementing continuous-scale morphology via curve evolution. Patternrecognition, 26(9):1363–1372, 1993.

[56] J. Serra. Image analysis and mathematical morphology. Academic Press, Inc.,

1983.

[57] C. E. Shannon. Communication in the presence of noise. Proceedings of theIRE, 37(1):10–21, 1949. [reprinted in: Proceedings of the IEEE, vol. 86, no. 2,

pp. 447–457, February 1998].

[58] E. M. Sigurðsson, S. Valero, J. A. Benediktsson, J. Chanussot, H. Talbot, and

E. Stefánsson. Automatic retinal vessel extraction based on directional

mathematical morphology and fuzzy classification. Pattern Recognition Letters,47:164–171, 2014.

[59] N. Sladoje and J. Lindblad. High-precision boundary length estimation by

utilizing gray-level information. IEEE Transactions on Pattern Analysis andMachine Intelligence, 31(2):357–363, 2008.

[60] A. Sobiecki, A. Jalba, and A. Telea. Comparison of curve and surface

skeletonization methods for voxel shapes. Pattern Recognition Letters, 47:

147–156, 2014.

[61] P. Soille. Morphological Image Analysis: Principles and Applications. Springer

Science & Business Media, 2013.

[62] P. Soille, E. J. Breen, and R. Jones. Recursive implementation of erosions and

dilations along discrete lines at arbitrary angles. IEEE Transactions on PatternAnalysis & Machine Intelligence, (5):562–567, 1996.

[63] T. Statella, P. Pina, and E. A. da Silva. Image processing algorithm for the

identification of martian dust devil tracks in MOC and HiRISE images.

Planetary and Space Science, 70(1):46–58, 2012.

[64] S. R. Sternberg. Grayscale morphology. Computer vision, graphics, and imageprocessing, 35(3):333–355, 1986.

[65] M. J. Thurley. Three dimensional data analysis for the separation and sizing ofrock piles in mining (PhD thesis). Monash University, 2002.

[66] S. Valero, J. Chanussot, J. A. Benediktsson, H. Talbot, and B. Waske. Advanced

directional mathematical morphology for the detection of the road network in

very high resolution remote sensing images. Pattern Recognition Letters, 31

(10):1120–1127, 2010.

[67] J. J. van de Gronde and J. B. T. M. Roerdink. Group-invariant colour

morphology based on frames. IEEE Transactions on Image Processing, 23(3):

1276–1288, 2014.

62

[68] R. van Den Boomgaard and A. Smeulders. The morphological structure of

images: The differential equations of morphological scale-space. IEEEtransactions on pattern analysis and machine intelligence, 16(11):1101–1113,

1994.

[69] R. van den Boomgaard and A. W. M. Smeulders. The morphological structure

of images. In Proceedings., 11th IAPR International Conference on PatternRecognition. Vol. III., pages 268–271. IEEE, 1992.

[70] L. J. Van Vliet, I. T. Young, and G. L. Beckers. A nonlinear Laplace operator as

edge detector in noisy images. Computer vision, graphics, and imageprocessing, 45(2):167–195, 1989.

[71] J. G. Verly and R. L. Delanoy. Adaptive mathematical morphology for range

imagery. IEEE Transactions on Image Processing, 2(2):272–275, 1993.

[72] B. J. H. Verwer, L. J. van Vliet, and P. W. Verbeek. Binary and grey-value

skeletons: Metrics and algorithms. International Journal of Pattern Recognitionand Artificial Intelligence, 7(05):1287–1308, 1993.

[73] L. Vincent. Graphs and mathematical morphology. Signal Processing, 16(4):

365–388, 1989.

[74] L. Vincent. Morphological grayscale reconstruction in image analysis:

Applications and efficient algorithms. Transactions on Image Processing, 2(2):

176–201, 1993.

[75] J. Weickert. Anisotropic diffusion in image processing. Teubner Stuttgart, 1998.

[76] M. Welk, M. Breuß, and V. Sridhar. Matrix morphology with extremum

principle. In International Symposium on Mathematical Morphology and ItsApplications to Signal and Image Processing, pages 177–188. Springer, 2019.

[77] B. Yin, H. Li, B. Sheng, X. Hou, Y. Chen, W. Wu, P. Li, R. Shen, Y. Bao, and

W. Jia. Vessel extraction from non-fluorescein fundus images using

orientation-aware detector. Medical image analysis, 26(1):232–242, 2015.

63

Acta Universitatis UpsaliensisDigital Comprehensive Summaries of Uppsala Dissertationsfrom the Faculty of Science and Technology 1869

Editor: The Dean of the Faculty of Science and Technology

A doctoral dissertation from the Faculty of Science andTechnology, Uppsala University, is usually a summary of anumber of papers. A few copies of the complete dissertationare kept at major Swedish research libraries, while thesummary alone is distributed internationally throughthe series Digital Comprehensive Summaries of UppsalaDissertations from the Faculty of Science and Technology.(Prior to January, 2005, the series was published under thetitle “Comprehensive Summaries of Uppsala Dissertationsfrom the Faculty of Science and Technology”.)

Distribution: publications.uu.seurn:nbn:se:uu:diva-395205

ACTAUNIVERSITATIS

UPSALIENSISUPPSALA

2019