IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

Reference algorithm implementations in OTB:Textbook Cases

Julien Michel1, Jordi Inglada2

1COMMUNICATIONS & SYSTÈMES2CENTRE NATIONAL D’ÉTUDES SPATIALES

IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

Introduction

OTB provides bleeding-edge algorithms to end-users.But what about:

I Correctness, performances, complexity?I Parameters?I Scalability to real market data?

This talk:

I Feedback from OTB development teamI Emphasizes the need for open source reference

implementationI Shows that OTB is a perfect framework for this

IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

Outline of the presentation

SIFT: known to be good, hard to code

Mean Shift: One step forward

Bayesian Fusion: Almost happy

Road Extraction: Reproducible extraction

From reference implementations to operational processingchains

IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

Outline

SIFT: known to be good, hard to code

Mean Shift: One step forward

Bayesian Fusion: Almost happy

Road Extraction: Reproducible extraction

From reference implementations to operational processingchains

IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

The Scale Invariant Feature Transform

A robust keypoints detector

I Keypoints: local extrema of difference of Gaussian inscale space

I Descriptors: histogram of local orientation

Applications

I RegistrationI Object recognitionI Panorama stitching . . .

IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

Algorithm main steps

1. Build the DoG pyramid2. Detect extrema within 8 neighbors3. Refine location by quadric fitting4. Discard low contrast points5. Discard edge response6. Assign one or more orientations per point7. Compute the descriptor

IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

Implementation

Main difficultiesI No reference implementation availableI Initialization differs between papersI Steps 3, 4 and 5 did not appear in some papersI Parameters for some steps are not detailed (kernel radius,

thresholds . . . )

Consequences

I Implementation fails to achieve expected performancesI OTB Sift based on the wrapping of SiftFast open source

library

IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

Results

Figure: SIFT detection (5 scales)

IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

Outline

SIFT: known to be good, hard to code

Mean Shift: One step forward

Bayesian Fusion: Almost happy

Road Extraction: Reproducible extraction

From reference implementations to operational processingchains

IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

The Mean-Shift algorithm

Overall schemeI Iterative non-parametric feature space analysis techniqueI Applied to the joint spatial-range domain

Smoothing

1. Iterate the mean-shift procedure until convergence2. Output spectral value is the spectral part of the local mode

Clustering

1. Filter the whole image to get the local modes for each pixel,2. Cluster pixels with close local modes in the joint domain

IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

Implementation

Based on EdisonI A small open source libraryI Developed by the authorsI Performing smoothing and clustering

Embedded in OTBI Clean OTB compliant wrapping classesI Supporting multi-threading (smoothing part) and streamingI At the expense of tolerance on results

IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

Results

Figure: Mean Shift clustering (spatial radius: 10, range radius: 40)

IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

Outline

SIFT: known to be good, hard to code

Mean Shift: One step forward

Bayesian Fusion: Almost happy

Road Extraction: Reproducible extraction

From reference implementations to operational processingchains

IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

The Bayesian fusion algorithm

Description

I Pan-sharpening techniqueI Based on statistical relationships between P and XSI Allows to weight P and XS informationI Promising for multi-modalities fusion

Authors wanted toI Provide reproducible evidencesI Advertise for a reference implementation in their papers

They proposed to contribute their algorithm to OTB

IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

Implementation

Inputs from the authors

I Matlab source code of the algorithmI Input data, output data and parameters for validation

Work done by the OTB team

I Transpose the Matlab code to OTB compliant C++ codeI Validate the implementation performances and correctnessI Feed the input/output data to the daily testing framework

Was available in OTB release when published

IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

Results

Figure: Using the Orfeo ToolBox Bayesian Fusion implementationon Ikonos data c©European Space Imaging)

IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

Outline

SIFT: known to be good, hard to code

Mean Shift: One step forward

Bayesian Fusion: Almost happy

Road Extraction: Reproducible extraction

From reference implementations to operational processingchains

IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

The road extraction algorithm

Description

I Path extraction from the gradient of a likelihood mapI Path filtering according to a priori knowledgeI Likelihood map = spectral angle wrt a road pixel

Advantages

I Light supervised coarse extraction of the networkI Fast algorithm (can be further refined)I Likelihood map can be changed

IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

Implementation

Research using OTB

I Existing filters were usedI Missing one were developed

Light integration cost

I Code reviewsI Testing and validationI Parts of the processing chain can be used individually

OTB: an efficient tool for fast prototyping during research

IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

Results

Figure: Using the Orfeo ToolBox Road Extraction framework

IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

Outline

SIFT: known to be good, hard to code

Mean Shift: One step forward

Bayesian Fusion: Almost happy

Road Extraction: Reproducible extraction

From reference implementations to operational processingchains

IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

Trends for operational processing chains

Efficiency

I Achieve best possible performances and correctnessI Scale to real market data: time and memory consumption

Interoperability

I Similar techniques should be switchableI Consecutive techniques should be pluggableI Standard interfaces are mandatoryI But also: progress reporting, error management, common

I/O . . .

IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

Streaming and multi-threading

I HR imagery means huge data volumesI Streaming and multi-threading:

I These are techniques allowing to overcome computationtime and memory capability limitations

I They need to split the data in order to process themI Algorithms which can not process data by chunks are not

scalableI Scalability of algorithms is crucial for end-user applications

IGARSS, July 12-17, 2009

Introduction SIFT MeanShift Bayesian Fusion Road Extraction Operational chains Conclusion

Conclusion

We sawI Usefulness of open source reference implementationsI Advantages of interoperable implementationsI From toy reproducible research to operational applications:

scalability

OTB is perfect for the job

I Fulfills the requirements aboveI You can help us improve the libraryI Any contribution is useful: algorithm description +

input/output data, Matlab/IDL/... code, OTB program, etc.