http:// 1 clustering features to recognize structure for purposes of 3d reconstruction from lidar...

http://www.nshorter.com 1

Clustering Features to Recognize Structure for Purposes of 3D Reconstruction from LiDAR

and other sources

Nicholas Shorter

Machine Learning Research Group Seminar Series

Email: [email protected]

Website: http://www.nshorter.com

mailto:[email protected]

http://www.nshorter.com/


Presentation Layout

1. LIDAR Overview

2. Problem Statement

3. Applications

4. Existing Approaches

5. Data Features

6. Masters Research

7. Masters Research Results


LIDAR Overview• Data Collection

– Plane Equipped with GPS, INS & LIDAR – LIDAR – Light Detection and Ranging (active sensor)

• Collection of 3D points• Laser sent out from Emitter, reflects off of Terrain, Returns to

Receiver– Receiver measures back scattered electromagnetic radiation

(laser intensity)• Time Difference Determines Range to Target

http://www.toposys.com/


LIDAR Captured Characteristics

• Range to Target (elevation from INS & LiDAR)

• Longitude and Latitude (GPS)• First and Last Return Pulses

– First – shrubbery, vegetation, power lines, birds and buildings

– Last – buildings (unless vegetation is really dense, then vegetation too)

• Returned Laser Intensity


LiDAR Noise• Essential to understand forms of noise corrupting

data in order to realize how to deal with the noise• Geolocation results from LiDAR, GPS and INS

sensor systems– Accuracy Limitations – Offset and Drift in both GPS and INS– Misalignment between INS and LiDAR

• Atmosphere – Intensity and Path Distortion• Shadowing Effect from Tall buildings• Artifacts from non uniform sampling from

multiple strips


Problem Statement

• Input– LIDAR (Light Detection and Ranging) Data

• Collection of Irregularly Distributed 3-Dimensional Points

– Aerial Photograph

• Output – Semi to complete Automatic (minimal user

intervention) development of 3-Dimensional Virtual Model


Applications for 3D Reconstruction

• Military Applications– Automatic Target Recognition– Reconstructed Models of Opponent Terrain

(UAV?)

• Tourism/Entertainment– Virtual Walkthrough of Theme Park

• Commercial– Change Detection (Natural Disasters)– Network Planning for Mobile Communication– Noise Nuisance (Universal Studios, 408

Expressway - walls)– Urban Planning


Existing 3D Reconstruction Research

• Data Sources– LIDAR – Aerial Imagery– GIS Ground Plans

• Model Based Reconstruction– Pre-defined models with parameters– Minimize error between models and data

• Data Driven– Group Coplanar Pts– Identify Break Lines– Derive Model to Minimize Error


Triangulation Based Methods

• Triangular Irregular Network (TIN)– Series of Non-Overlapping Triangles Modeling

given Surface

• TIN 3D Reconstruction Methods– Clustering approach

• Spherical Normal Vectors of Triangles

– TIN region growing approach• Merge Triangles to Same Region if Normal Vectors

within Threshold


Triangulated Irregular Networks (TIN)

• Advantages of TIN– Generation Fully Automatic– Space Uniquely Defined– Cells Indexed

• Spatial Searches, Triangle Elevation Differences

• Disadvantages of TIN– Noise Causes Triangle Normal Vectors to deviate from Ideal


Existing 3D Reconstruction Methods

• Most still under development• Most Methods Use ‘Grided’ (Interpolated)

LIDAR Data – Advantages

• Less Computationally Complex

• DTM & DSM Thresholding to distinguish Building from Non Building

• Use of additional conventional methods

– Disadvantages• Decrease in Accuracy

• Uncertainty from Building and Ground Interpolation


Building Detection Features• Investigate Returned Laser Intensity• Vegetation Points

– Significant first and last return elevation diff.– Corresponding green aerial image color– Nearest points have diff. elev. or adjacent triangles

have significantly diff. norm vectors• Common Building Points

– Spatially close in terms of long. and lat.– Bounded by aerial img. edges and exterior wall tri.– Bounded building does not contain terrain pts – or - – Triangulation of all points, building points connected


Building Reconstruct Cluster Features

• Surface change - normal vector orientation difference between adjacent triangles

• (X,Y,Z) - Longitude, Latitude, Elevation• Edge - Aerial imagery edge detection• Color - Aerial imagery corresponding color

(building surface differs from clutter)• Triangle planar coef, pt. height diff., same normal

vector, or planar equation• Feedback – difference between reconstruction and

raw LiDAR and Aerial Image Edges


Masters Research

• Greedy Insertion Triangulation – Implemented Noise Filtering Technique

• Proposed FSART normal vector clustering

• Proposed Planar Regression to combat Category Proliferation

• Realized simple planar reconstruction algorithm (MSEE Thesis)


Fuzzy SART Clustering

• Fuzzy Simplified Adaptive Resonance Theory (SART) Advantages – – Not as Sensitive to Input Order (CG FA)

– No “Mandatory” Preprocessing Techniques

– Activation Function Forms Spherical Arcs

– Activation Function is Measure not Estimate

– Long Term Weights have Intuitive Meanings

– Only 2 User Defined Parameters with Clear Meanings• Tau = Necessary Time to Learn Pattern

• VDMT = Vigilance Parameter


Fuzzy SART Activation Function

• Activation Function

• Module Degree of Match

• Angle Degree of Match

• Pattern Encoding Shape


Fuzzy SART Preprocessing

• Scaled Each Dimension to Same Max Value– Each Dimension Receives Equal Weight

• Translated Input Dimension Space to Center at Origin


Planar Regression

• Equation of a Plane:–

• Solving for Z:–

• Substitutions for Coefficients:–

• Use Method of Least Squares and Choose Estimates for Coefficients to minimize SSE:–


Planar Regression Continued…

• Derivative with Respect to Each Coefficient– – –

• System of Equations:

–

–

–


Planar Regression Pictorial

• VDMT biased against absorbing points into an existing cluster (VDMT = 0.7)

• Planar Regression implemented to join like clusters below a perpendicular threshold


Data Set Specifications

• Donated by Simone Clode & Franz Rottensteiner• Collected by AAMHatch• LIDAR - Approximately 1 point per 1.3m2 point

density (need at least 1 pt. per 1.5m2 for 3D rec.)– First & Last Return Pulses & Laser Intensity Recorded

• Aerial Photography – 15 cm pixel resolution• Data depicts urban and residential area of

Fairfield, Australia


Results

• 4 Actual Building Test Cases Presented– Aerial Photographs Included for Visual

Comparison– Outputs for Various Stages of Algorithm

presented with different levels of optimization


Aerial Photographs


3D Scatter Plots


Result Set 1

• No Scaling, Shifting, Filtering nor Planar Regression


Result Set 2

• Scaling, Shifting, No Filtering Nor Planar Regression


Result Set 3

• Scaling, Shifting, Filtering, No Planar Regression


Result Set 4 – Roof Segmentation

• Scaling, Shifting, Filtering, Planar Regression


Result Set 5 - Final


Conclusions – Fuzzy SART Preprocessing

• Fuzzy SART Preprocessing Techniques Proposed– Scaling Dimensions to same Maximum giving

each dimension equal weight– Translating Input Space to Further Separate

Data– Above Measures Improved Clustering


Conclusions – Heuristic Processing

• Triangle Differences for Determining Roof Triangles

• Planar Regression for Triangulation Clustering Proposed– Results Show Improvement Using Method


Conclusions

• ART TIN Based Clustering Showed Promising Results for Commercial Buildings

• Like Other Methods, Had Difficulties with House Roof Segmentation– Houses fractions of a size of Buildings– Houses had as much as 5 times Roof Planes as

Buildings


Acknowledgements• Data Contributors

– Dr. Simone Clode, Dr. Franz Rottensteiner, AAMHatch

– Mr. John Ellis, AeroMap– Mr. Steffen Firchau, TopoSys– Mr. Paul Mrstik, Terra Point

• Advisor (PhD Committee Chair)– Dr. Takis Kasparis

• PhD Committee Members– Dr. Michael Georgiopoulos , Dr. Georgios

Anagnostopoulos, Dr. Andy Lee, and Dr. Wasfy Mikhael

http:// 1 clustering features to recognize structure for purposes of 3d reconstruction from lidar...

Documents