ilmenau university of technology communications research laboratory 1 tensor-based signal processing...
TRANSCRIPT
Ilmenau University of TechnologyCommunications Research Laboratory
1
Tensor-Based Signal Tensor-Based Signal Processing with Applications in Processing with Applications in
Biomedical Signal AnalysisBiomedical Signal Analysis
Technische Universität IlmenauFachgebiete Nachrichtentechnik & Biosignalverarbeitung
Ilmenau University of TechnologyCommunications Research Laboratory
2
Outline
Motivation: Applications of multi-linear signal processing Introduction to multi-linear algebra Tensor decompositions
Multilinear extensions of the SVD• HOSVD• PARAFAC/CANDECOMP
Other decompositions PARAFAC via Joint Diagonalization 3-way PARAFAC for EEG data
Methodology and current status Open issues and questions
Discussion Status of the project proposals
Ilmenau University of TechnologyCommunications Research Laboratory
3
Outline
Motivation: Applications of multi-linear signal processing Introduction to multi-linear algebra Tensor decompositions
Multilinear extensions of the SVD• HOSVD• PARAFAC/CANDECOMP
Other decompositions PARAFAC via Joint Diagonalization 3-way PARAFAC for EEG data
Methodology and current status Open issues and questions
Discussion Status of the project proposals
Ilmenau University of TechnologyCommunications Research Laboratory
4
Why tensors?Why tensors?
Well, why even matrices? Matrix equations are usually more compact insights, manipulations
Example: DFT
Not a different data model but a more compact representation More than two dimensions: tensors even more compact new insights
dn = ?
dn = ??
Ilmenau University of TechnologyCommunications Research Laboratory
5
““Classical” Communications ResearchClassical” Communications Research
Description of the Mobile Radio Channel
resolve, characterize individual propagation paths of the mobile radio channel
Ilmenau University of TechnologyCommunications Research Laboratory
6
Biomedical engineeringBiomedical engineering
For example: EEG data
diagnostics (neurology, ophthalmology) human-machine interface
Ilmenau University of TechnologyCommunications Research Laboratory
7
Automotive engineeringAutomotive engineering
Wind tunnel analysis
find sources of disturbance to optimize aerodynamic behavior
Audio SourceLocalization
Ilmenau University of TechnologyCommunications Research Laboratory
8
MotivationMotivation
More applications
Signal Processing (sensor arrays, blind multi-user detection, source separation, CDMA, SONAR and seismo-acoustic signal processing)
Computer vision (Face and facial expression recognition, handwritten text recognition)
Data mining (weblink analysis, personalized web search, cross-language information retrieval)
Neuroscience (Multisubject fMRI anlaysis, concurrent EEG/fMRI)
Chemical engineering (food industry, NIR spectroscopy)
Geophysics (moment tensor inversion)
Data compression (image coding, video coding)
Ilmenau University of TechnologyCommunications Research Laboratory
9
Outline
Motivation: Applications of multi-linear signal processing Introduction to multi-linear algebra Tensor decompositions
Multilinear extensions of the SVD• HOSVD• PARAFAC/CANDECOMP
Other decompositions PARAFAC via Joint Diagonalization 3-way PARAFAC for EEG data
Methodology and current status Open issues and questions
Discussion Status of the project proposals
Ilmenau University of TechnologyCommunications Research Laboratory
10
The term “tensor”The term “tensor”
• Here: Intuitive definition: Here: Intuitive definition: ””A tensor of order p is a collection of elements referenced by p incides“A tensor of order p is a collection of elements referenced by p incides“ multi-way array multi-way array
• Mathematics: Mathematics: 1846: W. Voigt1846: W. Voigt
• Physics: Physics: 1915: M. Grossmann and A. Einstein1915: M. Grossmann and A. Einstein
very abstract definitionvery abstract definition
describe physical quantitiesdescribe physical quantities
ScalarsScalars VectorsVectors MatricesMatrices Order-3-tensorsOrder-3-tensors Order-4-tensorsOrder-4-tensors
??
……
Ilmenau University of TechnologyCommunications Research Laboratory
11
NotationNotation Symbols
Matrix operations
Ilmenau University of TechnologyCommunications Research Laboratory
12
Matrix unfoldingsMatrix unfoldings
n-mode matrix unfoldings vary the n-th along rows, the others along columns e.g., R = 3:
n-rank of . In general, 1-, 2-, and 3-rank can differ.
M1
M2
M3
“1-mode vectors”
“2-mode vectors”
“3-mode vectors”
Ilmenau University of TechnologyCommunications Research Laboratory
13
nn-mode products-mode products
i.e., all the n-mode vectors multiplied from the left-hand-side by
n-mode product between a tensor and a matrix
11 22
outer product between two tensors: all pair-wise products between elements
Ilmenau University of TechnologyCommunications Research Laboratory
14
The tensor rankThe tensor rank
Definition of the tensor rank
A tensor is rank one, iff
A tensor is rank r iff it can be decomposed into a sum of r and not less
than r rank one tensors
(Only) connection to the n-ranks:
The rank of a tensor can exceed its size (which is a good thing and a bad
thing)2 3 4 5 6 7 8
2 3 3 4 4 4 4 4
3 3 4 5 5 6 6 6
4 4 5 6 6 7 7 8
5 4 5 6 7 8 8 9
6 4 6 7 8 9 9 10
7 4 6 7 8 9 10 11
8 4 6 8 9 10 11 12
2 x
(maximum rank, cf. [Kolda08])
Ilmenau University of TechnologyCommunications Research Laboratory
15
Outline
Motivation: Applications of multi-linear signal processing Introduction to multi-linear algebra Tensor decompositions
Multilinear extensions of the SVD• HOSVD• PARAFAC/CANDECOMP
Other decompositions PARAFAC via Joint Diagonalization 3-way PARAFAC for EEG data
Methodology and current status Open issues and questions
Discussion Status of the project proposals
Ilmenau University of TechnologyCommunications Research Laboratory
16
The Higher-Order SVD (HOSVD)The Higher-Order SVD (HOSVD)
Singular Value Decomposition Higher-Order SVD (Tucker3)
“Full HOSVD”“Full SVD”
[Tucker: 1966][de Lathauwer: 2000]
Ilmenau University of TechnologyCommunications Research Laboratory
17
Computing the HOSVDComputing the HOSVD
Computing the HOSVD
The core tensor not necessarily any zero elements (only if n-rank-deficient)
all-orthogonality condition:
also three sets of singular values: n-mode singular values
4
35
Ilmenau University of TechnologyCommunications Research Laboratory
18
The Higher-Order SVD (HOSVD)The Higher-Order SVD (HOSVD)
Singular Value Decomposition Higher-Order SVD (Tucker3)
“Full HOSVD”
“Economy size HOSVD”
Low-rank approximation (truncated HOSVD)
“Full SVD”
“Economy size SVD”
Low-rank approximation
Ilmenau University of TechnologyCommunications Research Laboratory
19
Summary HOSVDSummary HOSVD
The HOSVD …
is an extension of the SVD to tensors.
generalizes the concept of row-space and column-space to r-spaces.
• the r-mode singular vectors are an orthonormal basis for the r-space of the tensor.
is very easy to compute (Matrix-SVD of the unfoldings).
the remaining core tensor is not diagonal, it may be full of non-zero elements (same size as original data)
• not a decomposition into rank-one components
Ilmenau University of TechnologyCommunications Research Laboratory
20
Outline
Motivation: Applications of multi-linear signal processing Introduction to multi-linear algebra Tensor decompositions
Multilinear extensions of the SVD• HOSVD• PARAFAC/CANDECOMP
Other decompositions PARAFAC via Joint Diagonalization 3-way PARAFAC for EEG data
Methodology and current status Open issues and questions
Discussion Status of the project proposals
Ilmenau University of TechnologyCommunications Research Laboratory
21
Another way to look at the SVD
decomposition into a sum of rank one matrices also referred to as principal components (PCA)
Tensor case:
PARAFAC: MotivationPARAFAC: Motivation
+ +=
+ +=
Canonical Decomposition (CANDECOMP)Parallel Factor Analysis (PARAFAC)
[Carroll, Chang 1970][Harshman 1970]
Ilmenau University of TechnologyCommunications Research Laboratory
22
PARAFAC expressionsPARAFAC expressions
Many equations to express the same model:
HOSVD:
Ilmenau University of TechnologyCommunications Research Laboratory
23
HOSVD vs. PARAFACHOSVD vs. PARAFAC
Example:
HOSVD
PARAFAC “Core tensor” diagonal Not easy to find the factors Factors may be flat (underdetermined) Reveals the tensor rank Often used for analyzing data
Core tensor not necessarily diagonal, can be full. Direct, easy computation via matrix-SVDs “Factors” always tall or square Reveals the n-ranks Often used for “compressing” data2211
11 22
Ilmenau University of TechnologyCommunications Research Laboratory
24
UniquenessUniqueness
When is the PARAFAC decomposition of X into A,B,C unique?
Let be the Kruskal-rank of A.
Then, given that … … the PARAFAC decomposition is unique up to scaling and permutation.
[Kruskal, 1966]
scaling and permutation can be removed if additional constraints are imposed or prior knowledge is used
scaling:
permutation:
Ilmenau University of TechnologyCommunications Research Laboratory
25
Finding the parallel factorsFinding the parallel factors Since we only have the noisy tensor, we restate the goal:
“Plain vanilla” approach: ALS
Many years of research to improve convergence speed smart initializations smart updates: Enhanced line search … “PARAFAC”, “COMFAC”
works, but
• very slow convergence
• requires good initial solution
“Least Squares Fit”
Ilmenau University of TechnologyCommunications Research Laboratory
26
Finding the parallel factorsFinding the parallel factors
Closed-form solutions? GRAM (generalized rank annihilation method): An exact closed-form
solution if either of M1, M2 or M3 = 2 (only two slices). Can be used as initialization for other methods.
DTLD (direct trilinear decomposition): A suboptimal approximation, mostly as initialization to PARAFAC. Based on Tucker3 (HOSVD) and GRAM. Very fast though.
Ours: Reduced the problem onto joint diagonalization of matrices (which by itself is a very well studied problem).
Ilmenau University of TechnologyCommunications Research Laboratory
27
Outline
Motivation: Applications of multi-linear signal processing Introduction to multi-linear algebra Tensor decompositions
Multilinear extensions of the SVD• HOSVD• PARAFAC/CANDECOMP
Other decompositions PARAFAC via Joint Diagonalization 3-way PARAFAC for EEG data
Methodology and current status Open issues and questions
Discussion Status of the project proposals
Ilmenau University of TechnologyCommunications Research Laboratory
28
Ilmenau University of TechnologyCommunications Research Laboratory
29
Outline
Motivation: Applications of multi-linear signal processing Introduction to multi-linear algebra Tensor decompositions
Multilinear extensions of the SVD• HOSVD• PARAFAC/CANDECOMP
Other decompositions PARAFAC via Joint Diagonalization 3-way PARAFAC for EEG data
Methodology and current status Open issues and questions
Discussion Status of the project proposals
Ilmenau University of TechnologyCommunications Research Laboratory
30
PARAFAC via Joint DiagonalizationPARAFAC via Joint Diagonalization
First, consider the case where
The transform matrices diagonalize the core tensor:
We can estimate the transform matrices via joint diagonalization
the fundamental link between the HOSVD and PARAFAC
Ilmenau University of TechnologyCommunications Research Laboratory
31
PARAFAC via Joint DiagonalizationPARAFAC via Joint Diagonalization
One slide on the six diagonalization problems
Ilmenau University of TechnologyCommunications Research Laboratory
32
One slide on the R-D extension
Ilmenau University of TechnologyCommunications Research Laboratory
33
Outline
Motivation: Applications of multi-linear signal processing Introduction to multi-linear algebra Tensor decompositions
Multilinear extensions of the SVD• HOSVD• PARAFAC/CANDECOMP
Other decompositions PARAFAC via Joint Diagonalization 3-way PARAFAC for EEG data
Methodology and current status Open issues and questions
Discussion Status of the project proposals
Ilmenau University of TechnologyCommunications Research Laboratory
34
Processing ChainProcessing Chain
Time-Frequency-Analysis
Time
Fre
quen
cy
Chann
el
Time
Fre
quen
cy
Chann
el
Time
Cha
nnel
Wavelet-basiert Wigner-basiert
Biomedicalprocess
MeasurementTime-Frequency-
AnalysisComponent
Analysis
Ilmenau University of TechnologyCommunications Research Laboratory
35
Component AnalysisComponent Analysis Given a three-way tensor (time, frequency, channel), we decompose it into a
predefined number of components for each component: time-, frequency-, and spatial characteristics
Zeit
Fre
quen
z
Raum≈ + +
Ilmenau University of TechnologyCommunications Research Laboratory
36
Outline
Motivation: Applications of multi-linear signal processing Introduction to multi-linear algebra Tensor decompositions
Multilinear extensions of the SVD• HOSVD• PARAFAC/CANDECOMP
Other decompositions PARAFAC via Joint Diagonalization 3-way PARAFAC for EEG data
Methodology and current status Open issues and questions
Discussion Status of the project proposals
Ilmenau University of TechnologyCommunications Research Laboratory
37
Outline
Motivation: Applications of multi-linear signal processing Introduction to multi-linear algebra Tensor decompositions
Multilinear extensions of the SVD• HOSVD• PARAFAC/CANDECOMP
Other decompositions PARAFAC via Joint Diagonalization 3-way PARAFAC for EEG data
Methodology and current status Open issues and questions
Discussion Status of the project proposals
Ilmenau University of TechnologyCommunications Research Laboratory
38
Geplante Folgeprojekte (1)Geplante Folgeprojekte (1)
BMBF - Innovationswettbewerb Medizintechnik Modul I „Innovationswettbewerb - BASIS“ Schlüsselexperiment zum Nachweis der Machbarkeit:
„Tensor-basierte Analyse von polygraphischen Biosignalen zur Anfallsvorhersage bei Epilepsie“
Projektpartner• TU Ilmenau: FG BSV & FG NT • GJB Datentechnik GmbH, Langewiesen• Zentralklinik Bad Berka GmbH, Klinik für Neurologie
Status• Projektskizze eingereicht• Hauptantrag vorzubereiten im Sommer/Herbst 2008
Modul II „Innovationswettbewerb – Transfer“ F&E-Vorhaben „Neue Methoden der Tensor-basierten Analyse von polygraphischen Biosignalen“ Projektpartner
• TU Ilmenau: FG BSV & FG NT • GJB Datentechnik GmbH, Langewiesen• Zentralklinik Bad Berka GmbH, Klinik für Neurologie• Psychotherapeutische und Neurologische Praxen
Status• Projektskizze vorzubereiten im Herbst 2008
Ilmenau University of TechnologyCommunications Research Laboratory
39
Geplante Folgeprojekte (2)Geplante Folgeprojekte (2)
BMBF - Innovationswettbewerb Medizintechnik „Frühdiagnostik und Intervention von Essanfällen mittels Polygraphie bei Patienten mit
Bulimia Nervosa“ (BuPoly) Projektpartner
• TU Ilmenau: FG BSV & FG NT • NeuroConn GmbH, Ilmenau• Praxis Dr. Braun, Gotha
Status• Projektskizze eingereicht• Hauptantrag vorzubereiten im Sommer/Herbst 2008
„Zeitvariable Niederfeldmagnetstimulation in der Therapie depressiver Erkrankungen und deren Wirkung auf die Herzratenvariabilität“ (DeNFMagS) Projektpartner
• TU Ilmenau: FG BSV & FG NT • Neurologische Praxis Henkel/Müller, Ilmenau
Status• Projektskizze eingereicht• Hauptantrag vorzubereiten im Sommer/Herbst 2008
Ilmenau University of TechnologyCommunications Research Laboratory
40
Geplante Folgeprojekte (3)Geplante Folgeprojekte (3)
BMBF - Ernährungsforschung „Polygraphiebasierte methodische und experimentelle Untersuchung der
Reizreaktion auf lebensmittelbezogene visuelle Stimulationen bei Personen mit und ohne psychogene Essstörungen“ Projektpartner
• TU Ilmenau: FG BSV & FG NT • Praxis Dr. Braun, Gotha• Neurologische Praxis Henkel/Müller, Ilmenau
Status• Projektskizze eingereicht• Hauptantrag vorzubereiten im Sommer 2008
Ilmenau University of TechnologyCommunications Research Laboratory
41
Geplante Folgeprojekte (4)Geplante Folgeprojekte (4)LUBOM – Thüringen „Zeitvariable Niederfeldmagnetstimulation in der Therapie depressiver
Erkrankungen und deren Wirkung auf die Herzratenvariabilität“ Projektpartner
• in der ersten Phase TU Ilmenau: FG BSV & FG NT • in der nächsten Phase zusätzlich neurologische und
psychotherapeutische Praxen Status
• Projektbeginn bei Bewilligung Anfang 2009 „Die Wirkungsweise von Eye Movement Desensitization and
Reprocessing analysiert anhand polygraphischer Untersuchungen multimodaler Biosignale“ Projektpartner
• in der ersten Phase TU Ilmenau: FG BSV & FG NT • in der nächsten Phase zusätzlich neurologische und
psychotherapeutische Praxen Status
• Projektbeginn bei Bewilligung Anfang 2009
Ilmenau University of TechnologyCommunications Research Laboratory
42
Geplante Folgeprojekte (5)Geplante Folgeprojekte (5)
LUBOM – Thüringen „EKG-Analyse zur Bestimmung der anaeroben Schwelle anhand der
Absenkung des ST-Komplexes“ Projektpartner
• TU Ilmenau: FG BSV & FG NT Status
• Projektbeginn bei Bewilligung Herbst 2008TAB – Thüringen „Erkennung von psychischen Verarbeitungsprozessen angstgestörter
Patienten zur Unterstützung der lnterventionstherapie mittels mobiler onIinefähiger Biofeedbackgeräte“ Projektpartner
• TU Ilmenau: FG BSV & FG NT • GJB Datentechnik GmbH, Langewiesen• Psychotherapie Dr. Wilms, Erfurt
Status• Projektbeginn bei Bewilligung Herbst 2008
Ilmenau University of TechnologyCommunications Research Laboratory
43
Geplante Folgeprojekte (6)Geplante Folgeprojekte (6)
European Research Council: Advanced Investigators Grants im FP 7
„Mikrosensoren zur Erfassung wichtiger Lebensfunktionen“
Projektpartner
• TU Ilmenau: FG BSV & FG NT & IMN & FG NIKR
• IDMT Fraunhofer, Ilmenau
Status
• Projektskizze in Vorbereitung
• einzureichen im Winter 2008
Ilmenau University of TechnologyCommunications Research Laboratory
44
Geplante Folgeprojekte (7)Geplante Folgeprojekte (7)
DFG Forschungsprojekt zur dynamischen tensorbasierten
Analyse von nichtlinearen zeitvariablen Prozessen Projektpartner
• TU Ilmenau: FG BSV & FG NT • Zentralklinik Bad Berka GmbH, Klinik für Neurologie• GJB Datentechnik GmbH, Langewiesen• Psychotherapeutische und Neurologische Praxen
Status• Projektskizze vorzubereiten im Herbst 2008