Manifold Learning– from Brain Visualization to Advanced Image Processing

Anders Brun, PhD studentMedical Informatics, Dept. of Biomed. Eng. (IMT)Center for Medical Imaging and Visualization (CMIV)Linköpings universitet, Sweden

IMTIMT

GeodesicsGeodesics

TensorsTensors

ManifoldsManifolds

Diffusion Weighted MRIDiffusion Weighted MRI

VisualizationVisualization

Intrinsic and Extrinsic meansIntrinsic and Extrinsic means

Manifold LearningManifold Learning

Machine LearningMachine Learning

Image ProcessingImage Processing

2

Thesis Advisors Hans Knutsson

Prof.Medical InformaticsDepartment of Biomedical EngineeringLinköpings universitet

Carl-Fredrik WestinAssociate Prof. of Radiology Laboratory of Mathematics in ImagingHarvard Medical School / Brigham and Women´s Hospital

Magnus HerberthsonAssistant Prof.Department of MathematicsLinköpings universitet

3

Abstract

In this talk I discuss the need to standardize image processing on manifolds. Two tools are presented, a framework for storage of geometric tensor arrays and a method for manifold learning, related to the Riemannian exponential and logaritmic map. Finally some future perspectives are presented, in particular the fusion of manifold learning and image processing.

4

Processing Tensor fields on Manifolds

Storage, processing and visualization of functions M ! V ­ V ... ­ V ­­V* ­­V* ­ ... ­ V*

Intrinsic formulations, Charts, Atlases, Coordinate Free Methods ...

Extrinsic methods can be problematic and there are more complicated cases than 2D surfaces embedded in 3D!

RP2

also SO(3), ...

5

A collaboration by:

Anders Brun, Burak Acar, Emma Munos, Leila Cammoun, Andreas Sigfridsson,Dario Sosa, Marcos Martin Fernandez, Björn Svensson, Magnus HerberthsonHans Knutsson

The Similar Tensor Array Core

The Similar Tensor Array Conventions

”Simple conventions for storing tensor array data ...”

6

What problem are we solving?

”The Ideal Customer”A fresh MSc or PhD student starting to work

with tensor data

Wanting to:Apply new algorithms to old (known) test dataApply old algorithms to new datsetsExchange tensor data between algorithms

and/or people

7

Core and Extensions

Mathematics

Processing paradigms

Data storage

Physics

Worlds

STAC

STAE A lot of tensor

processing can be done in the core

Extensions should include conventions for: DTI Elastography Image Processing Geoscience ...

Existing standards NRRD, VTK, Analyze, ...

Applications

8

Scalar data

array_dimensionality = 2 array_size = [4, 3] tensor_order = 0 tensor_index_types = {} data = [0.1 0.3 0.7 0.5 0.7 0.5 ...

0.5 0.6 0.8 0.1 0.5 0.3]

(1,1) (1,2) (1,3)

(2,1) (2,2) (2,3)

(3,1) (3,2) (3,3)

(4,1) (4,2) (4,3)

(4 * 3)

9

Vector data

array_dimensionality = 2 array_size = [4, 3] tensor_order = 1 tensor_index_types = {contravariant} data = [1.2 1.4 1.2 0.4 0.5 0.4]

... but in what basis???

(1,1) (1,2) (1,3)

(2,1) (2,2) (2,3)

(3,1) (3,2) (3,3)

(4,1) (4,2) (4,3)

(4 * 3 * 2)

10

The basis describing tensors

Indices are coordinates:(c1, c2) or generally ci

Basis vectors given by:

or generally

(1,1) (1,2) (1,3)

(2,1) (2,2) (2,3)

(3,1) (3,2) (3,3)

(4,1) (4,2) (4,3)

11

Dual vectors

array_dimensionality = 2 array_size = [4, 3] tensor_order = 1 tensor_index_types = {covariant} data = [0.2 2.4 3.1 0.3 1.2 0.7]

(1,1) (1,2) (1,3)

(2,1) (2,2) (2,3)

(3,1) (3,2) (3,3)

(4,1) (4,2) (4,3)

(4 * 3 * 2)

12

A metric instead of voxelsize

array_metric_tensor = gij

(1,1) (1,2) (1,3)

(2,1) (2,2) (2,3)

(3,1) (3,2) (3,3)

(4,1) (4,2) (4,3)

(1,1) (1,2) (1,3)

(2,1) (2,2) (2,3)

(3,1) (3,2) (3,3)

(4,1) (4,2) (4,3)

13

A metric instead of voxelsize

array_metric_tensor = gij

(1,1) (1,2) (1,3)

(2,1) (2,2) (2,3)

(3,1) (3,2) (3,3)

(4,1) (4,2) (4,3)

(1,1) (1,2) (1,3)

(2,1) (2,2) (2,3)

(3,1) (3,2) (3,3)

(4,1) (4,2) (4,3)

14

Additional core fields

[tensor_name] = D[tensor_description] = ”Diffusion”[tensor_index_names], e.g. {a, b}[array_index_names], e.g. {i, j, k}

15

Image Processing is often invariant ...

... to how the image is displayed!

16

Tensor Array Processing should be invariant too.

Remember, in some applications there is no extrinsic space!

17

Even without an extrinsic space you can do ...

Geodesic tracking (given a metric field gij)) Distances (given a metric field gij) Balls (given a metric field gij) Fiber tracking (given a metric tensor) Image processing on curved manifolds (given a metric field gij) Tilted volumes with non-isotropic voxelsizes (given a metric tensor) Correct visualizations, up to actions of the orthogonal group O(n)

(with a metric) or general linear group GL(n) (without a metric) Solve PDEs Interpolate the field

18

Manifolds in Image Processing

– It´s a Klein bottle!

19

Manifolds in Biology

Adapted from Visual Cortex: Looking into a Klein Bottle by Nicholas Swindale,Current Biology, Vol. 6 No. 7:776–779, 1996

Optical recording from the visual cortex of a macaque monkey.

“[...] reasoning based on topological arguments may be increasingly helpful in attempting to understand how the world, Klein bottles and all, is unfolded onto the surface of the cortex.” – N. Swindale

20

Dimension reduction / Manifold Learning

Dimension Reduction Principal Components Analysis, PCA (Pearson, 1901; Hotelling, 1933) Multidimensional Scaling, MDS (Torgerson, 1952; cf. Young and Householder, 1938) Projection Pursuit (Friedman and Tukey, 1974) Independent Components Analysis, ICA (Herault and Jutten, 1986)

Non-linear Dimension Reduction Self Organizing Maps, SOM (Kohonen, 1982) Principal Curves ([Hastie and Stuetzle, 1989) Kernel PCA (Schölkopf et al, 1998) Isomap (Tenenbaum et al, 2000) Locally Linear Embedding, LLE (Roweis and Saul, 2000) Laplacian Eigenmaps, LE (Belkin & Niyogi, 2002) Hessian Eigenmaps, H-LLE (Donoho & Grimes, 2003) Diffusion Maps (Coifman, S. Lafon, 2004) Local Tangent Space Alignment, LTSA (Zhang and Zha, 2004) Sample Logmaps (Brun et al 2005)

21

Isomap

Isomap (Tenenbaum et al, Science, 2000)

22

Geodesics, Exponential maps and Logarithmic maps

Lagrangian style (Exponential maps)

Eulerian ...

23

Why are the Exp and Log functions Important in Signal Processing?

Analogies of operations in vector spaces and in manifolds, reproduced from (Pennec et al.,2005).

24

An Eulerian approach to Logp(x) ...

... easy if we can calculate distances on M!

25

Example: The Swiss Roll

No distortion for a “flat” metric!

LOGMAP

26

LOGMAP – manifold hunting

From LOGMAP: A Method for estimating Riemannian normal coordinates for a set of datapoints sampled from a manifold byA. Brun, C.-F. Westin, M. Herberthson and H. Knutsson, submitted to ICIP 2005.

Logmap

Data(high.dim.)

– It´s a Klein bottle!

27

28

Example: 2D shape (S-Logmap)

29

Example: 2D shape (PCA)

30

Manifolds of Images and Image Processing

x y

time

M

L2(Rn)

– What manifold is this?Find your favourite application for R3 £ P2 ...

31

Problems in Manifold Learning

Calculation of distances!What is the topologyWhat is a suitable metric?Large samples sizesSmall sample sizes

32

Problems in Tensor Processing

How to move tensors on a manifold to perform filtering in a point?

The structure tensors in Hyper Image Manifolds?

Choosing and managing charts in an Atlas

33

34

35

36

37

38

39

40

41

Learning

Encyclopedia Britannica:“Process of acquiring modifications in existing knowledge, skills, habits, or tendencies through experience, practice, or exercise.”

Oxford Advanced Learner’s Dictionary:“gain knowledge or skill by study, experience or being taught.”