reconstruction of trees from 3d point clouds1079140/fulltext02.pdf · reconstruction of trees from...

UPTEC F 17011

Examensarbete 30 hpMars 2017

Reconstruction of trees from 3D point clouds

Martin Stålberg

Teknisk- naturvetenskaplig fakultet UTH-enheten Besöksadress: Ångströmlaboratoriet Lägerhyddsvägen 1 Hus 4, Plan 0 Postadress: Box 536 751 21 Uppsala Telefon: 018 – 471 30 03 Telefax: 018 – 471 30 00 Hemsida: http://www.teknat.uu.se/student

Abstract

Reconstruction of trees from 3D point clouds

Martin Stålberg

The geometrical structure of a tree can consist of thousands, even millions, ofbranches, twigs and leaves in complex arrangements. The structure contains a lot ofuseful information and can be used for example to assess a tree's health or calculateparameters such as total wood volume or branch size distribution. Because of thecomplexity, capturing the structure of an entire tree used to be nearly impossible, butthe increased availability and quality of particularly digital cameras and Light Detectionand Ranging (LIDAR) instruments is making it increasingly possible.

A set of digital images of a tree, or a point cloud of a tree from a LIDAR scan,contains a lot of data, but the information about the tree structure has to beextracted from this data through analysis.

This work presents a method of reconstructing 3D models of trees from pointclouds. The model is constructed from cylindrical segments which are added one byone. Bayesian inference is used to determine how to optimize the parameters ofmodel segment candidates and whether or not to accept them as part of the model.A Hough transform for finding cylinders in point clouds is presented, and used as aheuristic to guide the proposals of model segment candidates.

Previous related works have mainly focused on high density point clouds of sparsetrees, whereas the objective of this work was to analyze low resolution point cloudsof dense almond trees. The method is evaluated on artificial and real datasets andworks rather well on high quality data, but performs poorly on low resolution datawith gaps and occlusions.

ISSN: 1401-5757, UPTEC F 17011Examinator: Tomas NybergÄmnesgranskare: Thomas SchönHandledare: James Underwood

—————————————————————————————-

Popularvetenskaplig sammanfattning

Trad spelar en viktig roll i ekosystem varlden over, och de har utgjort en viktig resurs

for manniskan genom mansklighetens historia. Anvandningsomradena for trad ar manga

och olika, men en aspekt som ar viktig i manga av fallen ar tradens fysiska form.

Inom skogsbruk spelar tradens form en avgorande roll. Avverkningsbeslut baseras pa

det och sagverk kapar och klyver stockar enligt deras form. Inom jordbruk anvands trad

som producerar frukt och notter, och dessas former modifieras genom beskarning for att

oka skord och forbattra vaxtforhallanden.

Ett trads fysiska form utgors av tusentals eller miljontals stammar, grenar, kvistar

och lov arrangerade i komplexa strukturer. Denna komplexitet gor det svart att mata

och representera den exakta strukturen av ett trad. Nar siffror pa kvantitativa varden

behovs anvands i dagslaget darfor ofta uppskattningar baserade pa ogonmatt eller en-

klare matningar, sasom stamdiameter vid brosthojd.

Om en 3D-modell av ett trad av intresse fanns att tillga kunde matt fas snabbt, nog-

grannt och automatiskt fran berakningar pa modellen istallet for att mata pa det fysiska

tradet. Hur sadana 3D-modeller kan skapas med noggrannhet och effektivitet ar ett

pagaende forskningsomrade.

I detta arbete presenteras en metod for att skapa 3D-modeller av trad utifran data i form

av sa kallade punktmoln, vilka kan skapas med exempelvis LIDAR-instrument. LIDAR

star for Light Detection and Ranging, och simplifierat kan man saga att en LIDAR sveper

en laserstrale fram och tillbaka over ytan pa ett objekt for att noggrannt mata positionen

av ett stort antal punkter pa objektets yta. En mangd sadana punktpositionsmatt kallas

for ett punktmoln. For att med hog detaljniva fanga strukturen av ett helt trad kravs

ett punktmoln med flera hundra tusen eller miljontals individuella punktpositionsmatt.

Fran ett sadant punktmoln av ett visst trad aterskapar den presenterade metoden en

3D-modell av tradet, bestaende av cylindriska segment. Rekonstruktionen utgar ifran

ett enskilt segment som representerar en del av stammen vid tradets bas, och lagger

sedan till ett segment i taget pa sa satt att modellen passar battre och battre med

punktmolnet for varje segment som laggs till. Nya segment kan antingen laggas till vid

andarna av existerande segment, som fortsattningar av stammar eller grenar, eller pa

sidorna som nya grenar.

iii

iv

Exakt vilka segment modellen ska utokas med, var och med vilken storlek och orientering

ar karnan i rekonstruktionsproblemet. Problemet forenklas av att inte behandla hela

punktmolnet pa en gang utan att istallet titta pa mindre bitar av det i taget. Pa grund

av det stora antalet punkter ar det mycket tidsodande att utfora berakningar pa hela

punktmolnet. Mindre delar har farre antal punkter som ofta ocksa ar arrangerade i

mindre komplexa strukturer.

Varje nytt segment som laggs till modellen ar ett resultat av vad som kan liknas vid en

serie tarningskast. Varje tarningskast bestammer en viss storlek, position och orienter-

ing av det nya segmentet, och efter ett visst antal tarningskast valjs den basta av de

konfigurationer som genererats av tarningskasten. Om modellen ar mer lik punktmol-

net med det nya segmentet an utan det nya segmentet sa laggs det nya segmentet till

modellen, annars inte.

Huruvida modellen ar battre med eller utan ett visst segment bestams av tva faktorer.

Den forsta faktorn ar hur val modellen stammer overens med punktmolnet, hur sanno-

likt det ar att punktmolnet skulle se ut som det gor, om tradet sag ut exakt sa som

modellen gor. Den andra faktorn ar hur val modellen uppfyller forubestamda antagan-

den om hur trad ar uppbyggda. Vissa sadana antaganden ar simpla, sasom att grenar

inte kan genomkorsa varandra och att de generellt sett blir kortare nar de stracker sig

uppat och utat. Andra antaganden ar mer subtila och specifika for vissa arter eller

typer av trad, sasom antaganden om vilka vinklar som kan forvantas mellan grenar vid

forgreningspunkter.

Den presenterade metoden utvarderades pa punktmoln fran bade matningar pa verkliga

trad och simulerade matningar pa artificiella trad. Pa punktmoln med hog upplosning

och tackning fungerade metoden val. Pa punktmoln med lag upplosning och dalig

tackning, dar delar av tradet doljs bakom grenar och stammar, fungerade metoden

daligt.

Acknowledgements

Thank you James and Thomas for your help, guidance, immense patience and the op-

portunity to do this thesis at the Australian Centre for Field Robotics. It was a fantastic

experience, even though the project turned out to be even more difficult than expected.

My warmest thanks to the people of the ACFR and the Treehouse for your hospitality

and friendliness.

Uppsala, February 2017

Martin Stalberg

v

Contents

Popularvetenskaplig sammanfattning iii

Acknowledgements v

Contents vi

1 Introduction 1

1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.3 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.4 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Reconstruction algorithm 6

2.1 Premises and assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.1.1 Unstructured point cloud . . . . . . . . . . . . . . . . . . . . . . . 6

2.1.2 Color and laser return intensity . . . . . . . . . . . . . . . . . . . . 6

2.1.3 Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.1.4 Computational performance of implementation . . . . . . . . . . . 7

2.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.3 Adaptation to point clouds . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.4 Inference method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.5 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.6 The reconstruction algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.6.1 Inference conducted on subsets . . . . . . . . . . . . . . . . . . . . 10

2.6.2 Assigned and unassigned data points . . . . . . . . . . . . . . . . . 11

2.6.3 Initialization of new model segments . . . . . . . . . . . . . . . . . 13

2.6.4 Proposal selection . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.6.5 Backwards extension of model segments . . . . . . . . . . . . . . . 15

3 Proposals for model segment candidates 17

3.1 Partial proposals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

3.1.1 Radius change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3.1.2 Length change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3.2 Full proposals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3.2.1 RANSAC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3.2.2 Alignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

vi

Contents vii

3.3 Local Hough transform for cylinders . . . . . . . . . . . . . . . . . . . . . 21

3.3.1 Spherical grid through icosahedron subdivision . . . . . . . . . . . 22

3.3.2 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.3.3 Test results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

4 Evaluation of proposals 29

4.1 Likelihood . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

4.1.1 Data points and model points . . . . . . . . . . . . . . . . . . . . . 30

4.1.2 Removing occluded model points . . . . . . . . . . . . . . . . . . . 30

4.1.3 Likelihood function . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

4.1.4 Reduction of computational cost of distance searches . . . . . . . . 32

4.2 Priors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

4.2.1 Angle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4.2.2 Radius . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

4.2.3 Intersection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

4.2.4 Adjacency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

5 Experimental results 37

5.1 Generating artificial data . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

5.1.1 Arbaro and POV-Ray . . . . . . . . . . . . . . . . . . . . . . . . . 37

5.1.2 Encoding and decoding coordinates . . . . . . . . . . . . . . . . . . 38

5.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

5.2.1 Results on artificial data . . . . . . . . . . . . . . . . . . . . . . . . 40

5.2.2 Results on real data . . . . . . . . . . . . . . . . . . . . . . . . . . 42

6 Conclusion and future work 46

6.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

6.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

A Artificial data 49

A.1 Arbaro . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

A.2 POV-Ray . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

A.3 Matlab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

Bibliography 54

1. Introduction

Trees play an important role in ecosystems all over the world and they have been an

important resource for humans throughout human history. The various uses for trees

are many and diverse but one aspect of trees that is important in many cases is their

physical structure.

In forestry, tree geometry is crucial. Harvesting decisions and yield estimates are based

on it and sawmills cut logs according to it. In agriculture, trees that produce fruit

and nuts have their structures altered through the removal of branches, a process called

pruning, to improve yield.

The complexity of trees make their structures difficult to measure and represent. There-

fore, when quantitative qualities of the structure of a tree are of importance, they are

often not measured directly. Instead, they are estimated, for example by human visual

inspection or from statistical models with simple measurements as input, such as stem

diameter at breast height.

If a 3D model of a tree of interest was available measurements could be taken accurately,

quickly and automatically using the model instead of on the physical tree itself. How to

accurately and efficiently construct such models is an ongoing research topic and what

this work is concerned with.

This work was conducted at the Australian Centre for Field Robotics (ACFR) at the

University of Sydney, where there are ongoing research efforts about the use of robotics

in agriculture. One particular agricultural environment that is being investigated is

orchards. If a method for reliably and accurately reconstrucing 3D models of trees is

developed, one of the potential uses of such models is for calculating which branches to

prune to optimise the structure, growth and yield of the trees. The pruning could then

potentially be performed by robots, relieveing humans of a physically demanding task

and reducing the need for human labour in orchards.

In Section 1.1 further background information about the topic is presented. Section 1.2

provides an overview of related work. The chapter ends with a summary of the contri-

butions of this work and an outline of the rest of this document.

1

Introduction 2

1.1 Background

The geometrical structure of a tree can consist of thousands, even millions, of branches,

twigs and leaves in complex arrangements. The structure contains a lot of useful infor-

mation and can be used for example to assess a tree’s health or calculate parameters

such as total wood volume or branch size distribution. However, because of the immense

complexity, capturing the structure of an entire tree used to be nearly impossible. Early

attempts involved manual digitization, using local positioning devices placed at differ-

ent positions on the surface of the tree in succession to map one 3D point at a time.

This was very time consuming and difficult to carry out in the field [1]. The increased

availability of computational power and technologies including digital cameras and Light

Detection and Ranging (LIDAR) instruments facilitate the task of capturing tree struc-

ture. Such technologies have, in particular during the last decade, sparked an increase

in the research efforts concerning digitization of tree geometry (see e.g. [2] and [3]).

Advantages of the photography approach are that high-resolution cameras provide dense,

instant sampling including color information. However, no depth information is recorded.

Photos from different viewpoints can be used jointly to infer depth, possibly requiring

painstaking parameter tuning, but the result may turn out to be very noisy because

of the irregular, complex and self-similar tree structure. The main advantage of the

laser scanning approach is the direct capture of 3D data. Many current laser scanning

systems do not capture any color information, however, other than possibly the return

intensity of the laser. Also, producing a dense scan can require significantly more time

than taking a photo. The time requirement can cause noise in the data if wind moves the

tree during the data acquisition. Because of self occlusion (parts of the tree hiding other

parts of the tree from the sensor’s view) scans or photographs from multiple viewpoints

are generally required to thoroughly capture the tree structure. The occlusion problem

is amplified greatly by the presence of foliage, meaning that data should preferably be

acquired during the winter season when foliage is at a minimum. Of course, this only

applies to deciduous trees, those that shed their leaves during fall. Acquiring good data

on evergreen trees is likely to be equally difficult during all seasons.

Both photography and laser scanning, especially the latter, produce large amounts of

raw data which needs to be processed or analyzed to extract the structural information.

This involves identifying and characterizing branches and how they are connected. The

result can be represented as a 3D model with small triangular faces closely matching the

surface of the tree, or a collection of geometric primitives, such as cylinders and cones.

Yet another option is to use a skeleton, a small set of connected points describing the

topology of the tree.

Introduction 3

1.2 Related Work

Most previous work focuses on the reconstruction of the trunk and major branches as

data on twigs and leaves is generally too noisy to allow reconstruction with acceptable

accuracy [2]. Some of the literature on the topic focuses on the production of 3D

models for graphical purposes, e.g. for computer games or other virtual environments,

and is more concerned with the visual appeal of the models than how accurately they

correspond to the real world originals. Extrapolation or artificial synthesis is commonly

used to replace parts of the tree that were missing or excessively noisy in the acquired

data [4]. Several works use the fact that a pair of points on a tree that are close to each

other are likely to be on the same branch.

In [4] a connectivity graph is created by letting each point in the LIDAR point cloud

be represented by a vertex and connecting each vertex with the vertices whose points

are among the k nearest neighbours of the first point. The edges are weighted with the

distances between the points. A root point (the point chosen to represent the root of

the tree) is then selected and a minimum weight spanning tree is constructed from the

connectivity graph using Dijkstra’s algorithm [5]. The points are clustered based on the

quantized path length through the spanning tree to the root node. The centroids of the

clusters are connected to each other according to the neighbouring information of their

constituent points to form a skeleton of the tree.

A similar approach to [4] is taken in [2], but the spanning tree is taken directly as the

initial skeleton. The skeleton is refined by smoothing and contracting the point cloud,

then removing redundant points deemed to contribute negligibly to the overall structure

of the skeleton. The method described in [6] improves on the one in [4] by introducing a

more sophisticated clustering method. The point cloud is repeatedly randomly clustered

with a k-means algorithm and clusters that have a cylindrical enough shape are saved

and their points removed from the point cloud before the next iteration of clustering.

The resulting cylindrical clusters are hopefully segments of branches and are connected

to produce the tree skeleton.

This work is primarily based on the paper Modeling of Tree Branches by Bayesian

Network Structure Inference by Ma et al. [7], where wide baseline stereo image pairs

are used to infer tree structure, modeled as a hidden Bayesian network. The tree model

is built in stages by generating hypotheses at each stage for the on-growing of current

branches and generation of new branches, then accepting those hypotheses that best fit

the data and prior assumptions about properties such as branch smoothness and radii.

The end result is a skeleton of the tree consisting of points called skeleton nodes, or

just nodes, and connections between them. The first stage is initialized manually by

specifying the skeleton root node with a position centered in the base of the tree, with

a growing direction and a radius. In each subsequent stage child nodes for each parent

node, one at a time, are hypothesized and evaluated according to the properties of the

parent node and how well they match the stereo images. The child nodes are inferred

Introduction 4

using the Metropolis-Hastings algorithm [8], the driving proposals for which are based

on the properties of the parent node.

In [9] the point cloud is rasterized and centroids of adjacent voxels are connected with

an edge if the points contained in the respective voxels are distributed in a fashion that

indicates a connection of the underlying structure. The resulting graph is then refined,

e.g. by removing any loops, and finally taken as the skeleton of the tree.

The method in [10] produces a tree skeleton by contracting the point cloud using con-

strained Laplacian smoothing. An initial skeleton is produced by connecting each point

to its closest neighbours and the skeleton is then refined by removing loops and points

that do not contribute much to the overall skeleton structure.

In [11] a method first developed to produce artifical trees, called a space colonization

algorithm [12], is applied to the point cloud resulting from the scan of a real tree. The

algorithm grows a skeleton of the tree in a bottom-up manner from a manually selected

root node. Points function as attractors and the placement of new skeleton nodes is

determined by the points within an area of influence from the current existing skeleton

node. The method is similar to the one in [7] but less tunable.

The previous works often focus on sparse trees and rely on high density point clouds with

little noise produced through long duration scans from static platforms and combining

scans from several view points to mitigate occlusion (see e.g. [3]). In essence, they often

rely on high quality data, whereas the data that is the target of this work is low density

point clouds of dense trees, i.e. poor data.

1.3 Contributions

The ACFR has performed scans of an almond orchard with a LIDAR of the LMS5xx

family from SICK Sensor Intelligence. The main objective of this work was to develop

a method able to reconstruct accurate models of the almond trees in this data to use for

further research towards autonomous pruning and other applications. Characteristic of

this data is that the number and density of branches is high and the resolution of the

data is low.

The developed method uses Bayesian inference to reconstruct models of trees from point

clouds. The goal of producing accurate models from poor data was not achieved, but

the method has some success on high and medium quality artificial datasets.

Additionally a Hough transform for finding cylinders in point clouds was developed, and

a method for producing simulated LIDAR scans of computer generated tree models. The

latter can be used during development to compare results on simulated point clouds with

the underlying ground truth data.

Introduction 5

1.4 Outline

A method for reconstructing models of trees from point clouds is presented in Chapter 2.

The model consists of cylindrical segments which are added iteratively to the model.

Chapter 3 describes how the model segment candidates are initialized and modified prior

to being accepted or rejected as part of the model. Chapter 4 details how model segment

candidates are evaluated to determine which modifications to accept and whether to

accept them as part of the model. Results of the method on some datasets are presented

in Chapter 5. Finally conclusions are presented in Chapter 6.

2. Reconstruction algorithm

This chapter gives a high level overview of the structure of the algorithm used for

reconstruction. Many of the details are deferred to later chapters.

The chapter begins with Section 2.1, presenting some premises for this work and some

important assumptions that were made. Section 2.2 presents the motivation for bas-

ing this work on a particular related work, then Section 2.3 and Section 2.4 explain

adaptations made according to the particulars of the problem investigated in this work.

Section 2.5 presents how the tree models are structured, and finally Section 2.6 gives an

overview of the reconstruction algorithm.

2.1 Premises and assumptions

The premises for this work and some important assumptions are detailed in this section.

2.1.1 Unstructured point cloud

When dealing with structured three dimensional data, such as depth images or volume

images, the structure of the data can be leveraged for various purposes, e.g. performance

gains for certain algorithms. This work assumes no such particular structuring of the

data to enable using input data that is a composite of several scans of the same sensor

from different viewpoints or even from different sensors. An example of such data,

relevant to making this assumption, is data from a side scanning LIDAR moving past

the target tree.

2.1.2 Color and laser return intensity

This work makes no use of any color or intensity information associated with the input

point cloud. If accurate color information was available it could be used to classify the

points into classes such as leaves, flowers and bark, which could guide the reconstruction

algorithm. There are instruments that combine LIDAR and photography to co-register

range and color data. This can be achieved for example with a LIDAR and camera

6

Reconstruction algorithm 7

mounted on the same rotating sensor head which enables capture of range scans and

images from the same viewpoint with very small time discrepancy. However, the accu-

racy of such color information when capturing tree crowns is low due to the high spatial

frequency and the small size of twigs and leaves.

Many LIDARs record the return intensity of the laser pulses. Although it is not as

straightforward as with color information, intensity can also be used to distinguish be-

tween different parts of the tree. The decision was made not to make use of intensity

information, to not widen the scope of this work too much.

However, if a classifier that could use color and intensity information to classify points

was developed, it could easily be used to increase the performance of the algorithm

described in this work. It could simply be used in a preprocessing step to remove all

points not classified as part of trunk, branches or twigs, providing the reconstruction

algorithm with clearer data.

2.1.3 Initialization

The reconstruction algorithm has to be initialized with a starting point, including a

position, radius, growing direction etc. For trees with sufficiently simple geometry, and

with good scanning data, these initialization parameters could conceivably be derived

rather easily through an automatic analysis of the point cloud. Such an analysis was

not developed in this work, instead initialization parameters were acquired manually.

2.1.4 Computational performance of implementation

In many numerical methods, quality of the result is improved with the allowed runtime

or the number of iterations. This is also the case with the method presented in this

work. Critical during the development of this method has been the tradeoff between

allowing more runtime to get better results, and allowing less runtime to get quicker

feedback on the effects of changes to the method. To get more of both, computational

performance has been an important consideration in the implementation, which was

done in Matlab1. Some of the measures taken to improve computational performance

was the vectorization of loops and the reuse of calculation results, where possible.

2.2 Motivation

The approach in [7] by Ma et. al. was chosen as a basis for this work. Although it

uses stereo images rather than point clouds the framework seemed sound and the details

could be adapted to also work for point clouds. A particularly important advantage of

an approach based on Bayesian inference is the ability to incorporate existing knowledge

1More information about Matlab at https://mathworks.com/products/matlab.html


about tree morpohology and growth patterns into the prior probability functions (see

Chapter 4). In cases where the type or species of the tree subject to reconstruction is

known one could easily tune the parameters of the prior probability functions to the

particular characteristics of that species or type.

2.3 Adaptation to point clouds

As stated previously, this work is primarily based on [7], where stereo images are used as

the input for the structural reconstruction (for further details please refer to the paper

itself or a short introduction in Section 1.2). This work is concerned with point clouds,

and therefore certain adaptations and extensions to the approach of Ma et al. where

necessary. These adaptations and extensions are detailed here.

The original likelihood function is based on pixel intensities in the stereo images. The

point clouds considered here do not have any intensity information

The algorithm essentially tries to locally optimally fit cylinders to the data points.

Essential for this is good initialization. The initial proposals, and possibly also later

proposals, are generated from the data. The details are described in Chapter 3.

2.4 Inference method

Whereas Ma et. al. [7] use the Metropolis-Hastings algorithm [8] in the inferene of

the tree structure, this work choses a different approach. With 8 scalar parameters

per model segment (two 3D positions and two radii), inferring multiple model segments

simultaneously was deemed too computationally expensive. Instead model segments are

inferred one at a time, sequentially. Essentially, it was deemed easier to find several local

optima in separate lower dimensional spaces than a single global optimum in a higher

dimensional space.

2.5 Model

The tree is modeled as a set of tapering or straight cylindrical segments. An example

of such a model segment is shown in Figure 2.1, where the most important geometric

attributes of a model segment are also described. The points labeled position 1 and

position 2 are called the first and second endpoints respectively. Many other derivative

properties that are repeatedly used in calculations, such as the distance between the

endpoints, are precalculated and stored with the model segment for increased compu-

tational efficiency. The model is constructed in such a way that in any pair of a parent


segment and child segment, the second endpoint of the parent is closer to the first end-

point of the child than the first endpoint of the parent is to the second endpoint of the

child.

Apart from the geometric attributes, each model segment also keeps a reference to its

parent segment and a list of references to all its child segments. These references store

the hierarchical information of the model. The model segments can be grouped into

branches according to this hiearchy. If a model segment is the first child of its parent

model segment, it belongs to the same branch as the parent segment. Otherwise it is

the first model segment of a new branch. A depiction of the hierarchical aspect of the

model is shown in Figure 2.2.

Figure 2.1: The tree is modeled as a set of straight or tapering cylindrical segments.An example of such a segment is shown in the figure above. The legend shows the most

important attributes of the model segment.

Figure 2.2: A graph representation of a tree model consisting of model segmentsA through H. Each edge goes from a parent to a child and is labeled with the one-based child index. Branches made up of multiple model segments are represented byrectangles. Remaining nodes constitute their own single segment branches. Each child

segment with index 1 is part of the same branch as its parent.


2.6 The reconstruction algorithm

This section gives a high level overview of the reconstruction algorithm, detailing the

control flow and how different parts are used together.

The top level control flow is detailed in Algorithm 1. The input to the algorithm is a

point cloud and an initial model segment, called the root segment, and the main output

is a set of model segments. The algorithm is centered around two sets of model segments

called current stage and next stage. Current stage holds references to model segments

whose child segments are inferred during the current iteration. For any new segments

that are added to the model, references are also added to next stage. At the end of each

iteration, the contents of current stage are replaced with the contents of next stage, and

next stage is overwritten with the empty set.

For each model segment in current stage, inference of new child segments are repeated

until the inference was unable to find a child segment that improved the model. This

may happen after 0 or more child segments were created for that model segment.

This is repeated until current stage is empty, a set maximum runtime has been reached

or a maximum number of segments in the model has been reached.

A high level description of the inference of new model segments is given in Algorithm 2

and Algorithm 3, with varying level of abstraction. In Algorithm 3 a description is given

of how a new model segment is initialized. After initialization Algorithm 2 describes how

the new segment is subject to repeated modification proposals, which are accepted if

they increase the posterior probability of the model. The evaluation of proposals is

described in detail in Chapter 4. Eventually, the new model segment is returned if the

posterior probability of the model is higher with the new model segment than without

it.

To prevent the listings of Algorithm 2 and Algorithm 3 to become overly long the level

of abstraction was set high and many details omitted. These details are described in the

following subsections.

2.6.1 Inference conducted on subsets

For performance reasons the inference does not consider the whole point cloud at once,

instead it works on subsets. The inference of child segments for a model segment s

considers only points within a distance d from the second endpoint of s, called e2. The

distance d is given by d = max(dmin, 4r2) where r2 is the radius of s at e2. The factor

4 was chosen empirically, and the value for dmin was chosen empirically based on the

resolution of the point cloud to tune the behaviour of the algorithm with small model

segments. The sphere centered at e2 with radius d is called the region of inference. Like-

wise, only a subset of the model is considered at once. This subset of the model is called

the collision set, and consists of the model segment s itself and all other model segments


Algorithm 1 The reconstruction algorithm, high level

1: procedure reconstruct(p, s0)2: Sc ← {s0} . Add the root segment to the current stage3: M ← {s0} . Add the root segment to the model4: while Sc 6= ∅ do . While there are segments in the current stage5: Sn ← ∅ . Overwrite cn with empty set6: for each segment si in Sc do . For every segment in the current stage7: while s← inferChildSegment(p, si) do8: Sn ← Sn ∪ {s} . Add the new segment to next stage9: M ←M ∪ {s} . Add the new segment to model

10: end while11: end for12: Sc ← Sn . Overwrite contents of Sc with Sn13: end while14: return M15: end procedure

whose bounding spheres are contained within, or intersect, the region of inference. This

is further explained in Chapter 4.

2.6.2 Assigned and unassigned data points

To indicate which part of the data is already covered by the model, each data point is

associated with a flag, i.e. a boolean variable. If the flag for a data point p is set, i.e. its

value is true, the data point is said to be assigned. Assigned points are ignored when

computing full proposals (see Section 3.2).

Initially the flag of each data point is set to false. The flag of a data point p is set to

true when a new model segment is added to the model, and p is inside of or close to the

new model segment.

Given a model segment s with endpoints ~A and ~B with radii rA and rB respectively,

define a polar coordinate system with origin at ~A and positive z-axis passing through~B. Given a point p, its r and z coordinates in this polar coordinate system are given by

z = ~v • ~qr = |~q − z~v|

(2.1)

where ~q = ~p− ~A and ~v is a unit vector along the line from ~A to ~B given by

~v =~B− ~Al

l = | ~B − ~A|(2.2)

~p is defined to be inside of or close to s if it is true that


Algorithm 2 The reconstruction algorithm, inference of child segments

1: procedure inferChildSegment(p, s)2: Calculate the current (initial) posterior probability3: s′ ← initializeChildSegment(p, s)4:

5: for integers i in range [1, n] do6: if i > 1 then7: if the posterior increased in the previous iteration then8: Modify s′ with the alignment proposal9: else

10: Modify s′ with a randomly chosen proposal11: end if12: end if13: Move the first endpoint of s′ backwards to touch existing segments14: Calculate the posterior probability with s′

15: if the posterior decreased then16: Revert s′ to its state at the end of the last iteration17: end if18: end for19:

20: if the last posterior is greater than the initial posterior then21: Mark all points inside or close to s′ as assigned22: return s′

23: else24: Delete s′

25: return nothing26: end if27: end procedure

z ≥ −c

z ≤ l + c

r ≤ rz + c

(2.3)

where c is the fuzziness factor, which defines exactly how close ~p must be to s to be

considered to be close to s. The value given to c was 0.2rA. The value of rs, given by

(2.4), is the radius of s at z, or the radius of s at whichever of ~A and ~B is closest to ~p.

rs =

rA, if z < 0

rB, if z > l

zl (rB − rA) + rB, otherwise

(2.4)

A visual representation of this definition is provided in Figure 2.3.


Figure 2.3: For a model segment s (blue) with endpoints ~A and ~B with radii rA andrB respectively, points on or within the thick solid black line are defined to be inside

of or close to s. The value of c is 0.2rA.

2.6.3 Initialization of new model segments

As described in Algorithm 3, the initial parameters for a new model segment can be set in

a number of ways. The parameters are obtained either from RANSAC (see Section 3.2.1

for more about RANSAC) or from the best Hough transform result (see Section 3.3 for

more about the Hough transform used). The parameters to be set are the first endpoint,~A, its radius rA, the second endpoint, ~B and its radius rB.

Central to the choice is a set of radii values, R. Let the parent segment be called s, and

its radius at its second endpoint be called rs2. If rs2 is less than or equal to rmin, R

has only one element, namely rmin. Otherwise R has 15 elements, linearly spaced in the

range [rmin, 1.5rs2]. rmin is the minimum branch radius and its value is set based on the

characteristics of the tree being modeled and on the resolution of the point cloud.

If the cylinder returned by RANSAC has a radius that is less than or equal to the

maximum element of R, it is used to set the initial values for the new model segment.

The endpoints ~A and ~B are given by

~A = min(D)~V~B = max(D)~V

D = {(~q − ~C) • ~V | ~q ∈ Q}(2.5)


where Q is the inlier set, ~V is a unit vector along the axis of the fitted cylinder and ~C

is a point on the axis of the fitted cylinder. To get the correct orientation of the new

model segment, if ~A is closer to the second endpoint of s than what ~B is, ~A and ~B are

swapped.

If the radius of the cylinder fitted by RANSAC greater than max(R), the new model

segment is instead initialized using the best result from the Hough transform. The

Hough transform is computed with R at the second endpoint of s, and if s has at least

on child segment already, the Hough transform is also computed at 7 points, pH , linearly

spaced between the end points of the child segment according to

pH = { i6 ~A+ (1− i6) ~B | i ∈ Z, 0 ≤ i ≤ 6}. (2.6)

The first child segment of a model segment s is considered as the continuation of the

branch partially modeled by s. Computing the Hough transform along the continuation

of s is done to detect branches that branch off of the continuation.

A Hough transform result can be seen as a quadruplet of a number of votes, a grid point,

a center point and a radius. The hough transform with the highest number of votes is

selected and its grid point is assigned to ~A, its center point to ~B and its radius to rA

and rB.

The results from the Hough transform are kept and used for full proposals during the in-

ference, regardless of whether the new model segment was initialized with the RANSAC

result or the best Hough transform result.

Regardless of how the child segment is initialized, the Hough transform results are stored

and used for generating change proposals during the inference.

2.6.4 Proposal selection

Algorithm 2 describes how at one point a model segment candidate is modified with

a randomly chosen change proposal. The proposal options and their probabilities are

listed in Table 2.1. Most details of the proposals are described in Chapter 3. The Hough

transform results are computed during the initialization of the new child segment, and

stored for generating change proposals. The next Hough transform result is the one with

the highest number of votes that has not already been used for a proposal.

Table 2.1: Proposal options and their probabilities

Proposal Probability

Radius change 0.4Length change 0.4Next Hough transform result 0.2


Algorithm 3 The reconstruction algorithm, initialization of child segments

1: procedure initializeChildSegment(p, s)2: e2 ← secondEndPoint(s)3: p′ ← unassigned points in p within distance d from e24: if too few points in p′ then5: return nothing6: end if7:

8: R← radii lineraly spaced around radius at second endpoint of s9: Calculate Hough transform: Hough(p′, e2, R)

10: if segment s has children then11: for points qi spaced linearly between the endpoints of the first child of s do12: Calculate Hough transform: Hough(p′, qi, R)13: end for14: end if15:

16: s′ ← new segment with s as parent17:

18: C ← RANSAC(p′)19: if Radius of C less than maximum of R then20: Assign properties to s′ from C21: else22: Assign properties to s′ according to the best Hough transform result23: end if24:

25: return s′

26: end procedure

2.6.5 Backwards extension of model segments

Algorithm 2 describes how at one point a model segment candidate’s first endpoint

is moved backwards to touch existing segments. This procedure is called backwards

extension and its purpose is to prevent gaps in the model, i.e. between adjacent model

segments.

A set of points P are calculated as

P =

{~A+

−2i

n− 1( ~B −A) | i ∈ Z, 0 ≤ i ≤ n

}(2.7)

where ~A and ~B are the first and second endpoints of the model segment candidate,

respectively, and the value of n is 50. The points P are spaced linearly along the

extended axis of the model segment candidate, with the first point coinciding with ~A

and the last point being situated at a distance of 2l from ~A in the opposite direction of~B, where l is the length of the model segment candidate.


Let ~pc ∈ P be the point closest to ~A that is inside of a model segment in the collision

set. If ~pc exists, ~A is moved to ~pc, otherwise ~A is moved to the second endpoint of the

parent model segment. Note that the parent segment is part of the collision set.

3. Proposals for model segment

candidates

As described in Chapter 2 a structural model of a tree is built by an iterative procedure.

The model is built of model segments shaped as tapering or straight cylinders. The

segments each model a part of a stem or branch. As the procedure constructs the model

it creates model segment candidates which might be added to the model if they fit the

data well enough. The model segment candidates are modified by proposals which change

one or several of their attributes. A proposal is accepted if it increases the goodness of

fit between the model and the data and vice versa. Finally, a model segment candidate

is accepted as part of the model if it after a set number of proposals has increased the

goodness of fit compared to what it was in the absence of the model segment candidate.

This chapter details the proposals and Chapter 4 details how goodness of fit is evaluated.

Partial proposals change one or a few attributes of a model segment candidate whereas

full proposals change most of, or all of, the attributes. Section 3.3 describes a local

Hough transform for cylinders. This Hough transform and RANSAC (described in

Section 3.2.1) are used as heuristics for getting good initial sets of attributes which can

then be improved by the partial proposals and the alignment proposal (described in

Section 3.2.2).

3.1 Partial proposals

The partial proposals change only one or a few of the attributes of a model segment by

small random perturbations to their current values. Some of these proposals will nudge

the state of the model segment, the set of all current values of its attributes, towards a

local optimum in the posterior probability distribution.

Two types of partial proposals are used. One changes radii and the other changes length.

A third type of partial proposal that moved either of the endpoints a small distance in a

random direction was also implemented. However, it was abandoned as it was assessed

that its contribution was already covered by the full proposals in combination with the

alignment proposal.

17

Proposals for model segment candidates 18

3.1.1 Radius change

This partial proposal changes one of the radii of the model segment candidate. One of

the ends is chosen at random, with equal probability, and its radius is changed to a new

value selected randomly from a uniform distribution on the interval [0.7r0, 1.1r0), where

r0 is the current radius. Note that the interval is not centered on the current radius but

rather the proposals are biased towards smaller radii. This is to model the tendency of

branches to get progressively thinner towards their tips.

3.1.2 Length change

This partial proposal changes the length of the model segment candidate by moving

the second endpoint along the axis of the model segment. The new length is selected

randomly from a uniform distribution on the interval [0.7l0, 1.3l0), where l0 is the current

length.

3.2 Full proposals

The full proposals change most of or all of the attributes of a model segment candidate.

Unlike the partial proposals they are not stochastic but rather data driven, i.e. they are

based on an analysis of the local data.

Three types of full proposals are used. RANSAC (RANdom SAmple and Consensus) is

used to produce the initial state of a model segment candidate by attempting to find

the largest subset of the local data that is well modeled by a cylinder. A local Hough

transform for cylinders assesses how cylindrical different subsets of the local data appear

to be in different directions, information which is then used to form proposals. The

alignment proposal utilizes distance and neighbour information from the evaluation of

the previous proposal to move the model segment candidate closer to its closest local

data points.

3.2.1 RANSAC

RANSAC is an iterative method for estimating parameters of a mathematical model

from data that contains outliers. The data points that are not outliers are called inliers.

Each iteration consists of two steps. The first step is to randomly select n data points,

where n is the smallest number of data points sufficient for calculating the parameters of

the model in question. The second step is to evaluate the model with those parameters.

The underlying assumption is that if all of the points randomly selected in an iteration

are inliers, the resulting model is highly likely to fit the inlier set well. This assumes, of

course, that the selected mathematical model is suitable.


After a predetermined number of iterations the algorithm stops and the parameters

that resulted in the best fit are returned. RANSAC is a non-deterministic method

and is therefore not guaranteed to produce good results, but the probability to do so

increases with the number of iterations. Several algorithm parameters can also be tuned

to further increase the probability. More in-depth knowledge about RANSAC can be

found in [13].

In this work a cylindrical model is used. In each iteration three data points are selected

at random. In the plane which they define, the center and radius of their circumcircle

(the circle intersecting all three points) are calculated. The radius of the cylinder is

defined as the radius of the circumcircle and the axis of the cylinder is defined by the

center of the circumcircle and a normal to the plane. The error function, which evaluates

the model, computes the error, E, for each point in the point cloud according to

~Xr = ~X − ~P ,~Xt = ~Xr − ( ~Xr • ~V ) · ~V ,E = (| ~Xt| −R)2,

(3.1)

where ~X is the point itself, ~P is the circumcenter, ~V is a unit vector orthogonal to the

circumcircle andR is the radius of the circumcircle (see Figure 3.1 for illustration). Those

points whose errors are below a certain threshold are said to constitute the consensus set.

The variety of RANSAC used here is called MSAC, M-estimator SAmple and Consensus.

In MSAC the score of the model is the sum of the errors of the points in the consensus

set, as opposed to RANSAC where the score is simply the cardinality of the consensus

set. The RANSAC Toolbox 1 for Matlab was used to run RANSAC.

Figure 3.1: The parameters of a cylinder (light blue) are determined by three ran-domly selected points (magenta). The errors for other points (black) are the squares oftheir shortest distances to the cylinder (red). The axis of the cylinder (purple) is per-pendicular to, and passes through the center of, the circumcirle of the selected points

(green).

Ideally the model should be constructed such that, if disregarding noise, whenever all n

randomly selected points are part of the inlier set the optimal set of model parameters

1By M. Zuliani. Available at https://github.com/RANSAC/RANSAC-Toolbox (2014-11-12)


for modeling the inliers will result. To produce good parameters for the model used

here, the three selected points must not only be part of the inlier set but also in, or

close to, a plane perpendicular to the axis of the cylinder on whose surface they are

located. There are more sophisticated methods for calculating cylinder parameters from

a minimal set of points that would be better suited to the task (see e.g. [14]), but these

were not looked further into due to time constraints.

3.2.2 Alignment

Whenever a proposal of any kind is accepted, the next proposal is always the alignment

proposal. The alignment proposal uses distance and neighbour information from the

evaluation of the previous proposal to move the model segment candidate closer to its

closest local data points. This is done by calculating new end points for the model

segment candidate, a calculation which makes use of the model segment’s model points.

The model points are points spaced evenly in a grid pattern on the curved part of

the surface of a model segment. They are described in detail in Section 4.1.1. Each

valid model point of the model segment candidate is assigned two weights, w1 and w2,

according to its position, as illustrated by Figure 3.2. The weights are designed so that

model points close to one of the endpoints have a higher impact on its new position than

model points further away.

Figure 3.2: The weights w1 and w2 are assigned to each model point according to itslongitudinal distance along the axis of the model segment candidate, measured from

the first endpoint. The length of the model segment is denoted by L.

New positions for both of the model segment candidate’s endpoints are proposed ac-

cording to

~Pi = ~Pi0 + 2

∑Nj wij · ~Vj∑Nj wij

, i ∈ {1, 2}, (3.2)

where Pi and Pi0 are the new and current position of endpoint i, j enumerates the model

points, N is the number of model points and ~Vj is a vector from model point j to its

nearest neighbour data point. The factor 2 was found empirically to increase the rate of


convergence, i.e. decrease the number of alignment proposals required to move the model

segment candidate a certain distance towards a cylindrical point cloud. The reason this

works can be understood intuitively by considering two parallel but offset cylinders

and realizing that the closest point on the second cylinder from a point on the first

cylinder is (almost) always closer than the corresponding point on the second cylinder

(see Figure 3.3). A multiplicative factor can provide some measure of compensation for

this.

Figure 3.3: The distance from a point P on circle A to the corresponding point oncircle B (red line) is almost always greater than the shortest distance from P to anypoint on B (blue line). The only exception is when P is located at Q in which case the

two distances are equal (green line).

3.3 Local Hough transform for cylinders

A local Hough transform for cylinders was developed to provide coarse but good initial

guesses for configurations for model segment candidates. The result is used for full

proposals that set both radii and both endpoints of a model segment candidate.

The original Hough transform was developed for detecting lines in images. Extensions

and generalizations have since been employed for detecting various kinds of shapes in

various kinds of data. The central idea of the Hough transform is that each data point

casts votes for all possible configurations of a target shape that could cause that point

to exist. The more points a certain configuration explains, the more votes it receives,

and the more likely it is to be the cause of the data. To make computation feasible

only a finite number of configurations can be considered, as computational and memory

costs increase with more configurations. The configurations are described by points in

a discretized parameter space and the data points cast votes for these parameter space

points, the voting pattern depending on the shape. The main drawback of the Hough

transform is its high memory and computation costs for high-dimensional parameter

spaces. The costs increase exponentially with the dimensionality of the parameter space

making naive implementations intractable for shapes with anything more than a few

parameters, assuming a reasonably high resolution in the parameter space.

Motivated by the assumption that tree branches are locally approximately cylindrical

a Hough transform for cylinders was developed and used in this work for detecting


directions and radii of branch segments. It is designed to detect only cylinders with a

specific radius whose axes pass through one specific point, called the transform center.

This limitation cuts computation and memory costs significantly and is not detrimental

to the performance of the model construction. As the model construction is sequential

it always aims to find connecting branch segments for one model segment at a time.

As those are necessarily connected physically to the branch represented by the current

model segment their axes pass through it. The transform also detects cylinders whose

axes pass close to the transform center, although the directional result for such cylinders

is biased. Thus, by performing the transform at several locations along the axis of the

current model segment, for many radii at each location, any connecting branches can

potentially be detected.

3.3.1 Spherical grid through icosahedron subdivision

This Hough transform is designed for point clouds of the sort resulting from a LIDAR

scan. This means each point in the point cloud is, disregarding noise, located on the

surface of the scanned object. Each point votes for a set of points in a parameter

space, which is discretized by a set of grid points. These grid points are approximately

uniformly spaced on the surface of a sphere and describe cylinder axes together with the

transform center. The grid is created by an iterative process starting with a regular unit

icosahedron, a polyhedron with twenty equilaterally triangular faces and twelve vertices,

each vertex at unit distance from the origin2. The triangular faces of the icosahedron are

each subdivided into four smaller triangles and the newly created vertices are projected

outwards to unit distance from the origin. This procedure of subdivision and projection

is repeated until a sufficiently high resolution, and number of grid points, has been

reached. See Figure 3.4 for an illustration of the procedure. The vertices of the final

shape are taken as the grid points.

Figure 3.4: An approximately isotropic spherical grid can be created by a procedureof subdivision and projection starting with a regular icosahedron.

The resulting grid is approximately uniform, i.e. the spacing between any point and its

neighbours is almost the same for every point, regardless of its position on the sphere. A

perfectly uniform grid is not necessary but desirable, as the computational cost increases

with the number of grid points. A uniform grid therefore minimizes the computational

2Implementation by Kurt. Available at http://au.mathworks.com/matlabcentral/fileexchange/

28842-gridsphere (2014-11-13)


cost for a particular lowest local grid resolution. In this work a grid of 2562 points

was used, which is achieved after four iterations of the subdivision procedure. With

resolution defined as the angle between neighbouring grid points, as measured from the

sphere’s center, this gives a resolution in the interval [4.0◦, 4.7◦] with an average of 4.3◦.

As the procedure for generating the grid is rather costly it is run once at the start of

reconstruction to produce a grid of unit radius which is then stored and used in all

Hough transform computations.

3.3.2 Algorithm

Considering the case of just one single point, offset from the transform center, the task

of the algorithm is to find cylinders of a specific radius with axes passing through the

transform center, that explain this point, i.e. with their surfaces constrained to the

point. No point on the surface of any such cylinder is closer to the transform center

than one cylinder radius. Therefore the point must be at least one cylinder radius away

from the transform center. The axes of the cylinders fulfilling the criteria form a cone

with apex at the transform center. The intersection of the cone and the unit sphere form

a circle and all grid points close enough to that circle, called the voting circle, get one

vote each. See Figure 3.5 for an illustration. If the point cloud the transform is applied

to contains multiple points on the surface of a cylinder passing through the transform

center with a radius close to the radius used in the transform, the voting circles of the

points will overlap at the intersection of the unit sphere and the cylinder axes and grid

points close to that location will receive a large number of votes.

The criterion stating that grid points need to be close enough to the voting circle to

get votes is more specifically defined as within a certain threshold angle of the voting

circle. The threshold angle used is one half of the maximum angle between any two

neighbouring grid points, which with 2562 grid points is approximately 2.4◦. This value

ensures that the voted for grid points are as close to the voting circle as possible while

still forming a gapless circle.

Strictly speaking the intersection of the axes of the described cylinders and the unit

sphere form not one circle but two, on opposing sides of the sphere. It is advantageous,

however, to only use one of these circles, the one closest to the point, as this gives a

directionality to the result.

Pseudocode for the algorithm is shown in Algorithm 4. The actual Matlab implementa-

tion is entirely vectorized through the use of multidimensional arrays and executes much

faster than what would the naive but more readable description in Algorithm 4 using

for-loops. The Matlab implementation can be run for several radii simultaneously and

in that case reuses some of the intermediate variables for increased efficiency.

The grid points are all unit vectors and the voting result gives likely directions of cylin-

ders through or close to the transform center. When applying this transform locally to


Figure 3.5: Four of the possible cylinders with axes constrained to the sphere’s centerand surfaces constrained to some specific point (cyan). The intersection of the sphereand all such cylinders is a circle (red). Voted for grid points are shown in red, others

in grey.

point clouds of trees it is important that the size of the region of the point cloud supplied

to the transform is big enough for relevant branches to be represented properly but not

so big that it includes points from branches not immediately connected to the current

region of interest.

3.3.3 Test results

In this section the properties of the transform are illustrated by results on artificial test

data. Figure 3.6 shows the result on an ideal point cloud where all points are on the

surface of a cylinder of the exact same radius as the transform and with the cylinder

axes passing exactly through the transform center. As evident in the figure, the grid

point closest to the cylinder axis has received the most votes. The same result can be

seen from another angle in Figure 3.7 (F), together with results on the same point cloud

with different transform radii and transform centers. This figure serves to illustrate

that with an off-axis transform center and a mismatched transform radius the result is

slightly biased and imprecise, but still useful for determining the general direction of the

cylinder.

Figure 3.8 illustrates a situation in which the result is somewhat poor. The point cloud

in this case is a section of a point cloud from a simulated scan of an artificial tree. The

section is located where the vertical stem of the artificial tree splits into two sub-stems.

Gaussian noise with a standard deviation of one tenth of the stem radius has been added

to the data. Local maxima of grid point votes are located close to the axes of the stem


Algorithm 4 Local Hough transform for cylinders

1: procedure cylinderHough(c, r, p)2: for i← 1, ||p|| do . For every point in p3: pi ← pi − c . Translate point by subtracting transform center4: d← |pi| . Calculate point’s distance from origin5: if (d < r) then . If point too close to transform center6: continue . Skip this point7: end if8: rp ← r/d . Projected cylinder radius9: θ ← arcsin rp . Angle of vote circle

10: αl ← max(0, θ − tol) . Lower angle bound with tolerance11: αu ← min(π/2, θ + tol) . Upper angle bound with tolerance12: for j ← 1, ||gridPoints|| do . For every grid point13: α← ∠(pi, gridPointsj) . Angle between point i and grid point j14: if (α > αl) ∧ (α < αu) then . If angle between bounds15: votesj ← votesj + 1 . Increment votes of grid point16: end if17: end for18: end for19: end procedure

and the left sub-stem. As the transform center is rather far from the axis of the right

sub-stem the corresponding local maximum is similarly biased to the other side of the

axis. Several grid points around that local maximum also have similar amounts of votes,

making the local maximum less distinct.


Figure 3.6: Result on an ideal point cloud. The grid points are sized and coloredaccording to their number of votes. See color bar for comparison. The grey sphere

shows the radius used in the transform.


(a) r = 0.7r0, x = r0 (b) r = 0.7r0, x = 0.5r0 (c) r = 0.7r0, x = 0

(d) r = r0, x = r0 (e) r = r0, x = 0.5r0 (f) r = r0, x = 0

(g) r = 1.3r0, x = r0 (h) r = 1.3r0, x = 0.5r0 (i) r = 1.3r0, x = 0

Figure 3.7: Results on the same point cloud with different transform radii and trans-form centers. Grid points are sized and colored according to their number of votes. Therows show different transform radii and the columns show different transform centers.The cylinder radius is denoted by r0. The transform radius is denoted by r. The radialdisplacement of the transform center from the cylinder axis is denoted by x. The greyspheres and red crosses illustrate transform radii and transform centers respectively.


Figure 3.8: The point cloud in this case is a spherical selection from an artificial pointcloud of a branching point scanned from only one side. The point cloud is of a verticalstem splitting in two slightly below the middle of the figure. Grid points are sized andcolored according to their number of votes and grid points with few votes have been

removed for clarity.

4. Evaluation of proposals

The modeling procedure is a Bayesian inference from the observed point cloud to a 3D

model. The point cloud is denoted by Z and the 3D model by S. The equation at the

heart of this inference is the posterior probability distribution

P (S|Z) ∝ P (Z|S) · P (S) (4.1)

meaning that the probability of the model, given the point cloud, is proportional to the

product of the probability of the point cloud, given the model, and the probability of the

model. The term on the left hand side is called the posterior probability, or posterior for

short. The factors on the right hand side are called the likelihood and prior probability,

or prior for short.

In this work (4.1) is not applied to the whole point cloud and model simultaneously

but rather locally to sections of the point cloud and small sets of model segments in

what is here called regions of inference. The model construction centers on one model

segment at a time and infers its child model segments in a region of inference centered

on its second endpoint. The region of inference includes data points within a sphere of

radius 4r2, where r2 is the radius of the model segment at the second endpoint, and all

other model segments whose bounding spheres intersect or are included in this sphere.

Bounding spheres are detailed in Section 4.2.3.

Proposals, i.e. proposed changes to model segment candidates, are evaluated based on

how they change the posterior probability in the current region of inference.

Section 4.1 describes the likelihood function, with subsections detailing data points and

model points and how the likelihood function is efficiently computed. Section 4.2 de-

scribes the prior probability function with subsections detailing its various parts.

4.1 Likelihood

The likelihood, or likelihood function, measures how well the model and data match. In

this work it is implemented as a function of the minimum distances between the model

29

Evaluation of proposals 30

and the data and vice versa. It is largely an adaptation of the likelihood function in [15]

which is used for human pose tracking.

4.1.1 Data points and model points

Two sets of points called the data points and the model points are used in the likelihood

function. The data points are simply the points of the point cloud in the current region of

inference. The model points are points generated on the surfaces of the model segments

included in the region of inference. Ideally the model points for a model segment should

be those points that would have been produced by the scan if there was a part of the tree

that had the exact same geometry and position as the model segment, as if the model

segment was there in the scene during the scan that produced the point cloud. This can

be implemented with sensor models and ray-tracing but is complex and computationally

expensive, compared to the chosen method for producing the model points.

The chosen method simply places model points in a grid pattern on the surface of a

model segment. Specifically, model points are placed on the curved area but not on the

end caps. The grid is as close as possible to being a square grid and the spacing in the

longitudinal and circumferential directions are such that the grid’s average resolution

matches the resolution of the point cloud. The longitudinal spacing is uniform along the

length of a model segment but the circumferential spacing varies along the length if the

model segment has different radii at its ends.

If the grid of model points is viewed as having its rows along the length of the model

segment and its columns around the circumference, then the number of rows is given by

Nrows = max (8, dL ue) (4.2)

where L is the length of the model segment and u is the number of model points per unit

length, ideally similar to the average distance between neighboring points in the point

cloud. Notice the ceiling operator, ensuring an integer number of rows. Regardless of

the size of the model segment the grid can have no fewer than 8 rows. Similarly, the

number of columns is given by

Ncolumns = max (8, d(r1 + r2) π ue) . (4.3)

4.1.2 Removing occluded model points

The generation of model points is implemented such that, optionally, backwards facing

points can be removed. Backwards facing points are those points that are occluded by

the model segment itself as seen from an array of predefined sensor points, i.e. positions

from which the sensor, or sensors, produced the point cloud. Depending on the type of


(a) (b)

Figure 4.1: The model points (red) of a model segment (gray) with (A) and without(B) removal of backwards facing points, as seen from a sensor point below the model

segment.

sensors this feature can partially model the effects of self occlusion. Figure 4.1 illustrates

a model segment with and without removal of backwards facing points.

To model common tree geometries, such as bifurcations with small angles, model seg-

ments are allowed to intersect to some extent. Any model point inside of another model

segment is removed. This is the only feature that models the effects of occlusion between

different model segments.

4.1.3 Likelihood function

The likelihood function produces a likelihood score which is a combination of two sets of

minimum distances. The first set is the minimum distances from model points to data

points, i.e. the set of distances measured from each model point to its closest data point.

The second set is the minimum distances from data points to model points. Neither set

is sufficient on its own. Using only the first set would mean ignoring data, as only those

data points which are a nearest neighbor of some model point would factor into the

likelihood score. Using only the second set would mean that model segments that fit

the data badly would have no negative impact on the likelihood score as long as other

parts of the model are closer to the data. Using both sets ensures that all model points

and all data points are considered.

The likelihood function is defined as

P (Z|S) = exp(c2

2(1− q1)(1− q2)), (4.4)

where q1 is a score for the minimum distances from model points to data points, and

q2 is a score for the minimum distances from data points to model points. These scores

are given by (4.5) and (4.6) respectivly, where XD is the set of data points in the region

of inference and XM is the set of model points of the model segments included in the

region of inference.


q1(XD, XM ) =1

|XM |∑

xM εXM

exp

(−c1 min

xDεXD

||xD − xM ||)

(4.5)

q2(XD, XM ) =1

|XD|∑

xDεXD

exp

(−c1 min

xM εXM

||xD − xM ||)

(4.6)

The likelihood function has two parameters, c1 and c2, both of which must be real and

greater than zero. They govern the steepness of the likelihood function in two different

aspects. c1 governs the extent of impact of individual distances on the final likelihood

value, with a small c1 giving small impact to individual distances. Similarly c2 governs

how the fit of model to data is weighed together with the fit of data to model, with

a small c2 penalizing the final value less when either of the fits is bad. The chosen

parameter values are c1 = c2 = 2.

The likelihood function is adopted directly from [15], to which the reader is referred for

further details.

4.1.4 Reduction of computational cost of distance searches

The bulk of the computational cost of the model reconstruction is due to nearest neighbor

searches from data points to model points, and vice versa. These searches are performed

to find shortest distances between model points and data points, which are used in the

calculation of the likelihood function. To speed up the search as much as possible the

implementation uses the k-d tree data structure [16], on which distance searches are

efficient. This work makes use of the k -d tree in situations where searches on a single

set of points are performed enough times that the cost of constructing the k -d tree of

those points is offset by the reduction of the total cost of the searches.

Another important reduction in computational cost comes from the reuse of constant

search results. When a model segment has been added to the model its model points

are fixed, although some of them might be removed as intersecting model segments

are added. Given that the data points are constant, a fixed model point will never

get a new closest data point. Therefore, the closest data point to a given fixed model

point only needs to be found once, and the result can then be reused for the rest of

the reconstruction, avoiding a significant portion of the cost of finding the minimum

distances from model points to data points.

4.2 Priors

The priors are the second factor from the left on the right hand side of (4.1). They

incorporate prior knowledge about tree geometry and growth into the reconstruction

algorithm. The parameters of the priors should be set according to statistics on the tree


species being modeled. Finding statistically representative parameters is a topic that

was not thoroughly investigated but the algorithm was implemented in such a way that

new priors could easily be added, or existing ones modified. The final value of the prior

is the product of the values of the individual prior functions, as described by

P (S) =N∏i=1

pi(S), (4.7)

where P (S) is the prior and pi(S) are the individual prior functions, which are described

in the following sections. Three priors were implemented. Two of them address angles

and radius ratios between adjacent model segments, respectively, and the third addresses

intersections between model segments. The priors distinguish between model segments

that are determined to be the first model segment of a new branch or a continuing model

segment of an already existing branch. This distinction is based on the ratio of the radii

of the parent model segment and the model segment candidate, and the angle between

their respective directional vectors.

To illustrate, for a specific tree model used during testing (see Figure 5.4) the geometry

was such that the following parameters were used for distinction: A model segment

candidate was determined to be a continuing segment if the the radius ratio was greater

than 0.8 and the angle less than 30◦. The radius ratio is defined as rc/rp, where rc is the

radius at the first endpoint of the model segment candidate and rp is the radius at the

second endpoint of the parent model segment. How the angle is calculated is described

in Section 4.2.1.

4.2.1 Angle

The angle prior can penalize large angles between directional vectors of consecutive

model segments on the same branch and enforce specific branching angles for model

segments on new branches, i.e. branches that are not yet represented in the model.

The angle between two model segments, α, is calculated with the dot product as α =

arccos(~v1 • ~v2), where ~v1 and ~v2 are the unit directional vectors of the respective model

segments.

The angle prior is defined by

pangle = exp

(−(α− µ)2

2σ2

), (4.8)

which is a normal distribution without the normalizing constant factor, where µ is the

mean and σ is the standard deviation. In the test case mentioned previously a mean

of 0◦ and standard deviation of 20◦ was used for continuing model segment candidates

and a mean of 45◦ and standard deviation of 40◦ was used for branching model segment

candidates.


4.2.2 Radius

Consecutive segments of the same branch are likely to have similar radii whereas the

first segment of a new branch can potentially have a much smaller radius than its parent

segment.

The radius prior is defined by

pradius = exp

(−(r − µ)2

2σ2

). (4.9)

For a specific tree model used during testing, the parameters used for continuing model

segment candidates were µr = 0.99rp and σ = µr/4, where rp is the radius of the parent

segment. For branching model segment candidates the parameters were µr = 0.5rp and

σ = µ. The distribution for continuing model segment candidates has a mean biased

towards smaller values to model the tendency of branches to get thinner towards their

tips. The standard deviation in that case is relatively small, which models the fact that

branches seldom show large variations of radius across short distances. On the other

hand, the distribution for branching model segment candidates has a large standard

deviation to model the large variation of radius ratios of child and parent branches.

4.2.3 Intersection

The procedure that determines whether model segments intersect consists of multiple

steps. Ideally the degree of intersection between two model segments would be a function

of the volume enclosed in both model segments. As the intersection procedure is called

frequently it is important that it is not computationally expensive. To the author’s

knowledge there is no simple and efficient way of computing the volume of the inter-

section of two cylindrical frusta. The intersection prior is therefore based on a simpler

approximation of the true intersection volume.

The intersection procedure is a function of a model segment candidate and all other

model segments in the current region of inference. Before approximating intersection

volumes, the first step is to remove model segments that can not possibly intersect

with the model segment candidate. To do this, each model segment has an associated

bounding sphere, which is first easily tested for intersection with the bounding sphere

of the model segment candidate. In this context a bounding sphere is the smallest

circumscribing sphere of a model segment. Figure 4.2 shows a planar section along the

axis of a model segment together with its bounding sphere. The center of the bounding

sphere is derived from

| ~Q1 − ~C| = | ~Q2 − ~C|, (4.10)


with parameters described in Figure 4.2. ~C is constrained to the part of the axis of the

model segment between ~P1 and ~P2 as described by

~C = (1− k)~P1 + k ~P2, k ∈ [0, 1] (4.11)

With a coordinate system with origin in ~P1 and positive x-axis through ~P2 and y-axis

through ~Q1, (4.10) can be rewritten as

√(xQ1 − xC)2 + (yQ1 − yC)2 =

√(xQ2 − xC)2 + (yQ2 − yC)2. (4.12)

By squaring and substituting coordinates this simplifies to

k2L2 + r21 = (1− k)2L2 + r22, (4.13)

and finally by solving for k we find that

k =1

2

(1− r21 − r22

L2

). (4.14)

Depending on the attributes of the model segment, (4.14) can yield a value of k outside

the allowed interval [0, 1]. In these cases k is taken as the closest allowed value, i.e. either

0 or 1. This means that both the center and radius of the bounding sphere coincide with

that of one of the endpoints of the model segment. Otherwise the center is given by

(4.11) and the radius by R =√k2L2 + r21.

Figure 4.2: A planar section along the axis of a model segment (blue) together withits bounding sphere (grey).

Two bounding spheres with centers ~C1 and ~C2 and radii R1 and R2 respectively intersect

if the distance between their centers is less than the sum of their radii. This can be

computed quickly and efficiently with squared distances according to


|~C1 − ~C2|2 < R21 +R2

2. (4.15)

The second step of the intersection test is only performed with those model segments

whose bounding spheres intersect with the model segment candidate. For the model

segment candidate and each other model segment N points, called axis points, are in-

terpolated linearly along each of their axes between their endpoints. N was chosen

arbitrarily to be 15. For each of the other model segments in succession it is calculated

how many of its axis points are inside the model segment candidate and how many of

the axis points of the model segment candidate are inside the other model segment. For

the purpose of calculating whether or not a point is inside a model segment the segment

can be thought of as the section of either a cylinder or a cone (the former if the end

radii are equal or very similar, otherwise the latter) that is sitting between two planes

perpendicular to its axis. Calculating if a point is inside a model segment is then as

simple as calculating whether or not it is inside the cone or cylinder and then whether it

is on the right side of both planes. The calculation is done pairwise and the intersection

is determined to be when the number of axis points of either model segment that are

inside the other model segment is above a certain threshold. Two different thresholds

are used. If the model segment candidate and the model segment have the same parent

model segment (both were created as a continuation or branch of the same model seg-

ment) intersection is defined to be when 2/3 or more of the axis points of one segment is

inside the other segment. The same criterion applies if the other segment is the parent

segment of the model segment candidate. In all other cases a threshold of 1/5 is used.

The less strict threshold for closely related model segments allow for some extent of

intersection. This is advantageous for the modeling of some forms of common tree

geometries such as tight bifurcations. The specific choices of 2/3 and 1/5 were empirically

found to provide a sufficiently good trade-off between allowing desirable intersections and

disallowing undesirable intersections.

If the model segment candidate intersects with any other model segment the intersection

prior is set to 0, otherwise to 1. Allowing values between 0 and 1 for various degrees of

intersection might be beneficial but this was not tested.

4.2.4 Adjacency

A driving factor behind the growth of a tree is the competition for light and because of

that there is indirectly a competition for space, between the tree and other objects but

also between branches of the same tree [12]. Mechanisms used for the intersection prior

could be reused to implement an adjacency prior, i.e. a prior scoring model segment

candidates based on how close they are to other model segments. However, this was not

developed further.

5. Experimental results

Artificial data was used to test and evaluate the system during development. The ar-

tificial data was created by generating artificial trees with the software Arbaro and

then importing the resulting 3D models into POV-Ray where a LIDAR scan was emu-

lated. The use of artificial data for testing has several advantages over real data. Most

importantly the existence of ground truth, in the form of the generated artificial trees,

to which the results can be compared. Also important is the possibility to gradually

increase the difficulty of the problem by going from simpler to more complex tree models

and also by emulating gradually more realistic, and therefore limited in their quality,

LIDAR scans. This allows for dealing with one or a few subproblems at a time which

simplifies the development process.

This chapter describes the generation of this artificial data and gives results on artificial

and real data. Section 5.1 describes how artificial data was created using two different

softwares. Section 5.2.1 shows results on artificial data and finally Section 5.2.2 shows

results on real data.

5.1 Generating artificial data

Artificial point clouds were produced by a multi-stage process. The software Arbaro1

was used to procedurally generate trees. 3D models of these trees were imported into

the ray-tracing software POV-Ray2 which emulated LIDAR scans by ray-tracing and

encoded the result in images with a special colormap. The colormap allows for easily

extracting 3D coordinates and producing point clouds from the images.

5.1.1 Arbaro and POV-Ray

Arbaro is an open source software for procedurally generating 3D models of plants

based on [17]. Procedural generation utilizes a set of rules together with a random

number generator to create complex structures, usually built from simple repeating

elements arranged in different ways to form the larger structure. In the case of Arbaro

1More information about Arbaro at http://arbaro.sourceforge.net/2More information about POV-Ray at http://www.povray.org/

37

Experimental results 38

the rules model various aspects of plant growth and the elements are typically tapered

or straight cylinders (used for stems, branches and twigs) or leaves. The rules are

applied according to a set of parameters which can be tweaked to model different plants.

Included in the program are several sets of parameters for generating models of various

different plants and trees.

A snapshot of the user interface of Arbaro can be seen in Figure 5.1. The generated

models can be exported in different formats, one of which is the standard file format of

POV-Ray.

Figure 5.1: The graphical user interface of the software Arbaro. The artificial treedepicted, called Pinein this work, is the default black tupelo model provided with the

program.

POV-Ray is an open source software for creating images of 3D scenes using ray-tracing.

Ray-tracing has many similarities to laser range scanning and is therefore well suited for

emulating LIDAR scans.

One of the key features of POV-Ray is the Scene Description Language (SDL) used to

define the scenes, including any objects, lights and the camera.

5.1.2 Encoding and decoding coordinates

The 3D models are textured in POV-Ray using a special colormap (see Figure 5.2) that

assigns each point on the model a color that is directly proportional to its coordinates.

The ray-tracing produces an image in which each pixel represents one sampled point

on the 3D model, with the point’s coordinates encoded in the color of the pixel. The

red, green and blue channels in the image are used to store the x-, y-, and z-coordinates


respectively. However, the color black was reserved for background pixels. An interval

of coordinates to represent is specified for each coordinate axis separately. The mapping

of coordinates to colors is one-to-one only in the box defined by the given coordinate

intervals. Coordinates outside this box wrap around with the modulo of the interval

lengths. Care must therefore be taken to specify intervals that contain all coordinates

visible in the image.

Figure 5.3 show an example of such a color encoded image.

Figure 5.2: All RGB colors are represented uniquely in this cube. By scaling andtranslating the cube to fit a 3D model each point on the model is assigned a color that

directly encodes its coordinates.

Figure 5.3: A image of a 3D model of a tree where each non-black pixel represents apoint on the surface of the model, with the point’s coordinates encoded in the color of

the pixel.

Saving the image with 16 bits of information per color channel per pixel means the

resolution along a specific axis is length · 2−16, where length is the length of the interval

associated with that axis. For example, a length of 10 m gives a resolution of about 0.15


mm along the relevant axis. The color values are integers in the range [0, 216−1] and the

coordinates are retrieved by simply scaling and translating the color values, according

to the parameters of the coordinate intervals, as described by (5.1). For further details

refer to Appendix A.

xiyizi

=1

216 − 1

xmax − xmin 0 0

0 ymax − ymin 0

0 0 zmax − zmin

rigibi

+

xmin

ymin

zmin

(5.1)

An advantage of this method over depth map-based methods (see e.g. [18] about depth-

map rendering) is that no image formation model is required to extract the coordinates.

5.2 Results

This section presents and discusses some results on both artificial and real data.

5.2.1 Results on artificial data

Artificial point clouds were created from two artificial trees, here called Pype and Pine

respectively. Pype is the default tree generated by the software PypeTree (see [19]

for details about Pype and PypeTree). Pype is built from truncated cones, the same

geometric primitive as used by the reconstruction algorithm.

Pine is a default model provided with Arbaro, called black tupelo, and has different

characteristics than Pype. In some aspects Pine is more realistic than Pype. For example

it has curving branches and varying branching angles. It also exhibits a wider range of

ratios of radii between parent and child branches than Pype.

Depicted in Figure 5.4 is the result on a high resolution, zero noise point cloud generated

from Pype. Out of the 58890 points in the point cloud, the created model leaves only

887 points unmodeled. This is a near ideal case, in terms of both input and output.

Figure 5.5 shows a result on the same point cloud as in Figure 5.4, but with Gaussian

noise added. Out of the 58890 points in the point cloud, 10041 points are not assigned

to a model segment. However, the model still follows all but a few of the tree’s branches.

The algorithm shows some robustness to noise.

Figure 5.6 shows the effects of occlusion on the reconstruction algorithm. The point cloud

consists of points from a single simulated scan. This means that there are no data points

on surfaces facing away from the scanner position. It also means that there are gaps in

the point cloud where branches cross over each other, as seen from the scanner position.

Of the 49356 points in the point cloud, the model leaves 15453 points unmodeled. As


Figure 5.4: A result on a point cloud of the artificial tree Pype. The point cloudis a composite of simulated scans at 10 m distance with a resolution of 0.064◦ from 6points spaced evenly on a circle around the tree. The model segments are shown incolor, color coded according to hierarchical distance from the root segment. Red dotsindicate data points not covered by the model. The image is cropped at the bottom.

evident from the figure, the reconstruction algorithm does not handle point clouds with

missing data well. The main difficulty arises from gaps, which are highly likely to cut off

large sections of the point cloud from modeling if the reconstruction algorithm is unable

to bridge them. However, the algorithm has some success in modeling the front part of

the tree, i.e. the part visible without occlusion from the scanner.

Figure 5.7 shows a result on Pine. Pine has rather thin branches compared to the scan

resolution used, meaning that the point cloud is rather sparse on the branches. In the

lower part of the tree, branches have radii that are much smaller than that of the stem.

Due to the relationship between model segment length and model segment radius, the

model segments in the lower part of the tree also have lengths that are much greater

than the radii of the branches. For the Hough transform to provide strong indication

of a branch, it needs to be computed at a position close to the axis of the branch,

and a radius close to the radius of the branch, where ”close” is a measure relative to

the radius of the branch. Because of the large difference in size between the stem and

branches the granularity with which the Hough transform is computed may mean that

sample positions and radii are not close to that of off-shooting branches in some cases,

giving poor indication of their existence to the reconstruction algorithm. However, the


Figure 5.5: A result on the same point cloud as in Figure 5.4, but with Gaussian noiseadded. The model segments are shown in color, color coded according to hierarchicaldistance from the root segment. Red dots indicate data points not covered by the

model. The image is cropped at the bottom.

reconstruction algorithm still didn’t miss that many branches in the lower part of the

tree in Figure 5.7.

5.2.2 Results on real data

Figure 5.8 shows a typical result on a particular dataset from a scan of an almond tree

in blossom. The scan was performed with a line scanning LIDAR mounted sideways on

a robotic platform that was driven past the tree. A combination of low scan resolution

and high spatial frequency of small features such as thin branches, twigs and flowers

gives rise to a lot of noise and occlusion in the point cloud. Therefore the cylindrical

structures that the algorithm could have modeled are not identifiable.


Figure 5.6: A result on a point cloud of the artificial tree Pype. The point cloud isfrom a single simulated scan with a resolution of 0.032◦ at 10 m distance from the tree.The model segments are shown in color, color coded according to hierarchical distancefrom the root segment. Red dots indicate data points not covered by the model. Thereconstruction has missed a significant portion of the point cloud. The image is cropped

at the bottom.


(a) The whole point cloud and model. De-tails at locations B, C and D are shown in

subfigures B-D.

(b) Near the tip of the tree. The modelfollows many of the branches but notto their tips, as they get thinner thanthe set minimum branch radius and the

point cloud gets too sparse.

(c) A branch that the reconstructionalgorithm has missed completely, al-though the point cloud does not looksignificantly different than for other

branches.

(d) Overfitting. Due in part to lowpoint cloud resolution, the reconstruc-tion algorithm has created multiplebranching model segments where thereis only a single almost straight branch

in the tree.

Figure 5.7: A model reconstructed on a point cloud of the artificial tree Pine. Thepoint cloud is a composite of simulated scans at 10 m distance with a resolution of0.064◦ from two points on opposite sides of the tree. Red dots show points that are notcovered by the model. The tree itself is 5 m high, even though it’s morphology would

suggest that it was much higher.


Figure 5.8: A typical result on one particular real dataset from a scan of an almondtree in blossom. The model segments are shown in color, color coded according tohierarchical distance from the root segment. Except for the lowermost part of thetrunk, the point cloud is almost completely devoid of structures that can be identifiedas approximately cylindrical and hence the reconstruction algorithm fails to create an

accurate model.

6. Conclusion and future work

This final chapter starts with a summary of this work and a discussion about the per-

formance of the presented method in Section 6.1. Potential directions for future work

are presented in Section 6.2.

6.1 Conclusion

A method for reconstructing 3D models of trees from point cloud data has been proposed.

The method builds a model that fits the point cloud out of geometric primitives in the

form of truncated cones. The method is an adaptation to point clouds, and extension,

of the work by Ma et al. [7]. The method was expanded with a novel Hough transform

for cylinders as a heuristic for the proposal of model segment candidates. The model

is constructed iteratively, with more model segments added in each iteration, if they

increase the posterior probability of the model.

The method is based on Bayesian inference, which gives the potential advantage that

knowledge about tree morphology and growth patterns can be incorporated into the

prior probability function. However, due to time constraints and priorities, no attempts

were made to find and use real tree morphology data for this purpose.

On an extensive and dense point cloud of a tree with approximately cylindrical branches,

the reconstruction algorithm can build a model that explains most of the point cloud.

On a point cloud with gaps and missing data, the algorithm often fails to follow branches

across data gaps and therefore misses large parts of the point cloud.

Additionally, a method for generating artificial tree point cloud data was presented.

The method uses Arbaro to generate artificial trees, POV-Ray to emulate LIDAR

scans with ray-tracing and Matlab to decode the resulting color coded images to point

clouds.

6.2 Future work

If developing this work further a useful tool would be a method for quantitatively eval-

uating the performance of the method on artificial data with ground truth. With such

46

Conclusion and future work 47

an evaluation method it would be easier to determine the impact of changes to the

reconstruction algorithm, and parameter tuning could be performed automatically by

repeated runs on the same data but with different algorithm parameters.

The ability to tune the parameters of the method to the characteristics of the particular

species or type of tree being analyzed is something that has been presented as advan-

tageous but the actual impact of doing so has not been evaluated quantitatively. With

a method for quantitative evaluation of reconstruction models this is something that

could be looked further into.

Morphological characteristics may not be constant throughout a tree, and if so the hier-

archical and geometrical properties of the model could be used in the prior probability

functions to model this.

One particular characteristic of trees that seems well suited for integration into the

presented method is that which is described by the pipe model theory [20]. The pipe

model describes a tree as a set of pipes from root to leaves, all with the same constant

diameter, and each representing the same amount of leaf area. The stem is a bundle

of all of the pipes and each branching point is where a bundle is split into sub-bundles

that continue in different directions. The pipe model theory could be used in the prior

probability function to have the reconstruction method make use of the hierarchical and

geometrical information available in the partially reconstructed 3D model.

An intuitively promising idea for another approach to recontructing models of trees

is to base the process on scans that repeat periodically since the tree was a sapling.

Construction of the initial sapling model is likely to be trivial as it would not yet have

grown a very complex geometry. Whenever a new scan is performed a model that fits

the new data is constructed by using the model from the previous scan as a basis, then

making small adaptations and extensions to make it fit the new data. Of course, to

apply this method to mature trees requires either an existing history of scans, or that an

initial model is constructed by some other method. Comparing the rate of progress in

this field of research, and technology in general, with the growth rate of trees, and given

the large effort required to conduct periodic scans of all trees of interest for a prolonged

period of time, it seems unlikely that this method would be generally useful. Although

maybe it could be useful to refine models created from meticulous initial scans with

quicker and less thorough follow-up scans.

An aspect of the point cloud data that has not been fully exploited is the empty space

around the points. A point in the point cloud does not only convey information about

a specific point on the surface of an object, it also implies that the space along the ray

between the point and the scanner is empty, and that the space on the opposite side

of the point may or may not be empty. Given a point ~p observed from the origin, the

space along the line segment {k~p | k ∈ [0, 1)} is implicitly empty, and the space along

{k~p | k > 1} may be non-empty. Some attempts were made to come up with how to

utilize this information, but none made it past the prototype stage, partially due to time

constraints.

Conclusion and future work 48

The presented reconstruction algorithm builds a model out of geometric primitives,

which remain static once accepted as part of the model. The algorithm extends the model

outwards but never modify the parts that have already been set. An opportunity for

improvement is to look at how the reconstruction could be improved if model segments

can be revisited later on.

Model segments are added sequentially, one at a time. It was a deliberate choice to

forego inferring several model segments simultaneously, as is done in [7], for reasons of

computational complexity. However, it is likely that this has a negative impact on the

result quality. One likely advantageous and rather simple modification to do along these

lines would be to allow modifications to a parent segment during the inference of its

child segments.

A. Artificial data

This appendix details how to generate artificial point clouds from artificial trees using,

in order, the softwares Arbaro (version from 2013-04-23), POV-Ray (version 3.7) and

Matlab (version r2014b).

A.1 Arbaro

Instructions for Arbaro are as follows:

1. Generate a tree (several predefined models are available).

2. Note the species name, i.e. the value of the parameter Species, available in the

hierarchy on the left hand side under General→Tree shape. The species name

will be used in the POV-Ray scene description.

3. Click File→Export tree... to open the export window.

4. Next to Export format choose Povray primitives.

5. Next to Export to file choose destination file. E.g. C:\User\pov\tree.inc

6. Next to Seed optionally define a different integer seed for the pseudorandom num-

ber generator. Note the value, this will be used in the POV-Ray scene description

file.

7. Press Start.

A.2 POV-Ray

Instructions for POV-Ray, assuming the user has followed the instructions in section

A.1. First, append the following code to povray.ini, which can be accessed through

Tools→Edit master POVRAY.INI:

49

Appendix A. Artificial data 50

; Disable gamma correction for output as this distorts the information

File_Gamma=1.0

;

; Set output format to PNG

Output_File_Type=N

;

; Set bits per color to 16, the maximum value

Bits_Per_Color=16

Second, create a new file and enter the code from the block below. Edit the code

according to relevant file names and parameter values, then save the file. When defining

the bounding box and camera, note that POV-Ray uses a left-handed coordinate system

with the y-axis as the default upwards direction.

// Define path to file here

#include "C:\User\pov\tree.inc" // Example path

// Define bounding box here

#declare xmin = -3; // Example values

#declare xmax = 3;

#declare ymin = 0;

#declare ymax = 10;

#declare zmin = -3;

#declare zmax = 3;

// Enter objects to render inside this union

#declare objectsToRender = union {

// Replace SPECIESNAME and SEED with the appropriate name and value

object{SPECIESNAME_SEED_stems} // Stems, branches, twigs etc

object{SPECIESNAME_SEED_leaves} // Leaves

}

// Define camera parameters here

camera {

// Any camera type can be used, e.g. fisheye or orthographic

perspective // Camera type

location <0.0, 2.0, 10.0> // Camera position

look_at <0.0, 2.0, 0.0> // Camera target

angle 40 // Field of view in degrees

}

// Probably no need to change anything beneath this line

// Pigment for encoding x coordinates in red

#declare xpigment = pigment {

gradient x // Pigment type


color_map {

[0.0 color <0.0, 0.0, 0.0>] // Black

[1.0 color <1.0, 0.0, 0.0>] // Red

}

scale xmax-xmin

translate xmin

}

// Pigment for encoding y coordinates in green

#declare ypigment = pigment {

gradient y // Pigment type

color_map {

[0.0 color <0.0, 0.0, 0.0>] // Black

[1.0 color <0.0, 1.0, 0.0>] // Green

}

scale ymax-ymin

translate ymin

}

// Pigment for encoding z coordinates in blue

#declare zpigment = pigment {

gradient z // Pigment type

color_map {

[0.0 color <0.0, 0.0, 0.0>] // Black

[1.0 color <0.0, 0.0, 1.0>] // Blue

}

scale zmax-zmin

translate zmin

}

union {

objectsToRender

pigment {

average // Coordinate pigments mixed using average, reducing color range

pigment_map {

[1.0 xpigment]

[1.0 ypigment]

[1.0 zpigment]

}

}

// "Ambient 3" multiplies colors by three,

// restoring full color range:


finish {ambient 3 diffuse 0}

}

Third, choose a resolution setting with no anti-aliasing, e.g. 1024x768, No AA, then

press run. Rendering with anti-aliasing distorts the coordinate information. If the tree

doesn’t fit in the resulting image, redefine the camera parameters. If there are sharp

transitions between any two non-black colors anywhere on a smooth surface this means

that the bounding box does not fit the tree. In that case redefine the bounding box.

By default, POV-Ray will save the image in the same folder as you saved the scene file

and it will also have the same name as the scene file except for the file extension.

A.3 Matlab

Assuming the instructions in appendix A.1 and appendix A.2 have been followed, the

below Matlab script extracts the coordinates from the POV-Ray output image.

% Define parameters

nbits = 16; % Number of bits per channel

file = ’C:\User\pov\tree.png’; % Image produced by POV-Ray

% Bounding box

xmin = -3;

xmax = 3;

ymin = 0;

ymax = 10;

zmin = -3;

zmax = 3;

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% Read image

I = imread(file);

% Flatten image to Nx3 array of coordinates

p = reshape(I,[],3,1);

k = all(I == 0, 2); % Indices of background points

p(k,:) = []; % Remove background points

p = double(p); % Convert to double

%% Scale and translate coordinates

s = [xmax-xmin, ymax-ymin, zmax-zmin]/(2^nbits-1); % Scale factors

t = [xmin, ymin, zmin]; % Translation terms


p = bsxfun(@times,p,s); % Scale

p = bsxfun(@plus,p,t); % Translate

%% Switch to Matlab standard axes configuration

p = p(:,[1,3,2]);

%% Reverse x-axis and y-axis

p(:,1) = -p(:,1);

p(:,2) = -p(:,2);

%% Extract vectors of coordinates

x = p(:,1);

y = p(:,2);

z = p(:,3);

Bibliography

[1] Peter Room, Jim Hanan, and Przemyslaw Prusinkiewicz. Virtual plants: new

perspectives for ecologists, pathologists and agricultural scientists. Trends in Plant

Science, 1(1):33–38, 1996.

[2] Yotam Livny, Feilong Yan, Matt Olson, Baoquan Chen, Hao Zhang, and Jihad El-

Sana. Automatic reconstruction of tree skeletal structures from point clouds. ACM

Transactions on Graphics (TOG), 29(6):151, 2010.

[3] Pasi Raumonen, Mikko Kaasalainen, Markku Akerblom, Sanna Kaasalainen, Harri

Kaartinen, Mikko Vastaranta, Markus Holopainen, Mathias Disney, and Philip

Lewis. Fast automatic precision tree models from terrestrial laser scanner data.

Remote Sensing, 5(2):491–520, 2013.

[4] Hui Xu, Nathan Gossett, and Baoquan Chen. Knowledge and heuristic-based mod-

eling of laser-scanned trees. ACM Transactions on Graphics (TOG), 26(4), 2007.

[5] Edsger W Dijkstra. A note on two problems in connexion with graphs. Numerische

mathematik, 1(1):269–271, 1959.

[6] Dong-Ming Yan, Julien Wintz, Bernard Mourrain, Wenping Wang, Frederic

Boudon, and Christophe Godin. Efficient and robust reconstruction of botanical

branching structure from laser scanned points. In 11th IEEE International Con-

ference on Computer-Aided Design and Computer Graphics, pages 572–575. IEEE,

2009.

[7] Wei Ma, Yizhou Wang, Hongbin Zha, and Wen Gao. Modeling of tree branches by

bayesian network structure inference. Trees, 10(8), 2009.

[8] Siddhartha Chib and Edward Greenberg. Understanding the metropolis-hastings

algorithm. The American Statistician, 49(4):327–335, 1995.

[9] Alexander Bucksch, Roderik C. Lindenbergh, and M. Menenti. SkelTre - Fast Skele-

tonisation for Imperfect Point Cloud Data of Botanic Trees. In Michela Spagnuolo,

Ioannis Pratikakis, Remco Veltkamp, and Theoharis Theoharis, editors, Eurograph-

ics 2009 Workshop on 3D Object Retrieval. The Eurographics Association, 2009.

[10] Zhixun Su, Yuandi Zhao, Chunjiang Zhao, Xinyu Guo, and Zhiyang Li. Skeleton

extraction for tree models. Mathematical and Computer Modelling, 54(3):1115–

1120, 2011.

54

Bibliography 55

[11] Chakkrit Preuksakarn, Frederic Boudon, Pascal Ferraro, Jean-Baptiste Durand,

Ekko Nikinmaa, Christophe Godin, et al. Reconstructing plant architecture from 3d

laser scanner data. In Proceedings of the 6th International Workshop on Functional-

Structural Plant Models, pages 12–17, 2010.

[12] Adam Runions, Brendan Lane, and Przemyslaw Prusinkiewicz. Modeling trees with

a space colonization algorithm. In Proceedings of the Third Eurographics Conference

on Natural Phenomena, NPH’07, pages 63–70, Aire-la-Ville, Switzerland, 2007.

Eurographics Association.

[13] Marco Zuliani. Ransac for dummies. With examples using the RANSAC toolbox for

Matlab and more, 2009.

[14] Christian Beder and Wolfgang Forstner. Direct solutions for computing cylinders

from minimal sets of 3d points. In Computer Vision–ECCV 2006, pages 135–146.

Springer, 2006.

[15] Nicolas Lehment, Moritz Kaiser, and Gerhard Rigoll. Using segmented 3d point

clouds for accurate likelihood approximation in human pose tracking. International

journal of computer vision, 101(3):482–497, 2013.

[16] Jon Louis Bentley. Multidimensional binary search trees used for associative search-

ing. Communications of the ACM, 18(9):509–517, 1975.

[17] Jason Weber and Joseph Penn. Creation and rendering of realistic trees. In Proceed-

ings of the 22nd annual conference on Computer graphics and interactive techniques,

pages 119–128. ACM, 1995.

[18] Andras Bodis-Szomoru and Tamas Daboczi. High-precision ground-truth data for

evaluating dense stereo and optical flow algorithms. In Proceedings of the 8th Con-

ference of the Hungarian Association for Image Processing and Pattern Recognition

(KEPAF’2011), pages 258–273, 2011.

[19] Sylvain Delagrange, Christian Jauvin, and Pascal Rochon. Pypetree: A tool for

reconstructing tree perennial tissues from point clouds. Sensors, 14(3):4271–4289,

2014.

[20] Kichiro Shinozaki, Kyoji Yoda, Kazuo Hozumi, and Tatuo Kira. A quantitative

analysis of plant form-the pipe model theory: I. basic analyses. Japanese Journal

of Ecology, 14(3):97–105, 1964.

reconstruction of trees from 3d point clouds1079140/fulltext02.pdf · reconstruction of trees from...

Documents