DFG-Schwerpunktprogramm 1324

”Extraktion quantifizierbarer Information aus komplexen Systemen”

Tree Adaptive Approximation in the HierarchicalTensor Format

Jonas Ballani, Lars Grasedyck

Preprint 141

Edited by

AG Numerik/OptimierungFachbereich 12 - Mathematik und InformatikPhilipps-Universität MarburgHans-Meerwein-Str.35032 Marburg

DFG-Schwerpunktprogramm 1324

”Extraktion quantifizierbarer Information aus komplexen Systemen”

Tree Adaptive Approximation in the HierarchicalTensor Format

Jonas Ballani, Lars Grasedyck

Preprint 141

The consecutive numbering of the publications is determined by theirchronological order.

The aim of this preprint series is to make new research rapidly availablefor scientific discussion. Therefore, the responsibility for the contents issolely due to the authors. The publications will be distributed by theauthors.

Tree Adaptive Approximation in the

Hierarchical Tensor Format

Jonas Ballani ∗, Lars Grasedyck

June 24, 2013

The hierarchical tensor format allows for the low-parametric representationof tensors even in high dimensions d. The efficiency of this representationstrongly relies on an appropriate hierarchical splitting of the different direc-tions 1, . . . , d such that the associated ranks remain sufficiently small. Thissplitting can be represented by a binary tree which is usually assumed tobe given. In this paper, we address the question of finding an appropriatetree from a subset of tensor entries without any a priori knowledge on thetree structure. We propose an agglomerative strategy that can be combinedwith rank-adaptive cross approximation techniques such that tensors can beapproximated in the hierarchical format in an entirely black box way. Nu-merical examples illustrate the potential and the limitations of our approach.

1 Introduction

High-dimensional problems are encountered in many areas of practical interest, as e.g.in stochastics, quantum chemistry, or optimization. In many cases, the solution to ahigh-dimensional problem can be represented or approximated by a tensor

A ∈ Rn1×···×nd

of order (or dimension) d ∈ N with n1, . . . , nd ∈ N. As soon as the order d is largeenough, the explicit representation of A in terms of all its entries A(i1,...,id), iµ = 1, . . . , nµ,µ = 1, . . . , d, becomes prohibitively expensive. This has motivated the development ofdata-sparse tensor representations that can be applied even in high dimensions d. For adetailed introduction to tensor representations we refer the reader to [9, 7, 6].A quite general framework for the low-parametric representation of tensors has been

introduced in [8] which we further analyzed in [4]. In the so-called hierarchical tensor(or hierarchical Tucker) format, a tensor is represented by a number of parameters

∗Financial support from the DFG SPP-1324 under grant GR 3179/2-2 gratefully acknowledged.

1

that scales only linearly in the dimension d. As a key ingredient, this format relies on anappropriate hierarchy of subspaces which can be related to specific matrix representationsof a given tensor. Based on this strong connection to matrices, powerful tools have beendeveloped that allow for (approximate) arithmetic operations with tensors even in highdimensions d.The efficiency of the hierarchical tensor format crucially depends on an appropriate

splitting of the different directions 1, . . . , d in a hierarchical way. At the top level t =D := {1, . . . , d}, the index set t is subdivided into disjoint subsets t1, t2 ⊂ D witht = t1 ∪ t2. Afterwards, the subdivision process is recursively continued with t1 and t2until the bottom level of singletons t = {µ}, µ ∈ D, has been reached. The splitting canbe represented by a binary tree TD which is usually assumed to be given.Once a tree TD has been chosen, each node t ∈ TD with t ⊂ D is associated to a rank

kt ∈ N that results from a specific matrix representation of A depending on t. It turnsout that the storage complexity for the representation of A in the hierarchical tensorformat lies in O(dk3+ dkn) where k := maxt∈TD kt, n := maxµ∈D nµ. Since the ranks ktstrongly depend on the index sets t ⊂ D, the storage complexity of a given tensor maybe very sensitive to the choice of the tree TD, cf. [5].In some cases, the complexity for the representation of tensors in the hierarchical

format does not depend on the choice of the tree TD. In particular, any tensor A givenin CP format (canonical polyadic, CANDECOMP/PARAFAC) with

A =r

�

j=1

u1,j ⊗ . . .⊗ ud,j , uµ,j ∈ Rnµ ,

can be represented in the hierarchical format where all ranks kt are bounded by r for anytree, cf. [8]. Nevertheless, it might be advantageous to represent A in the hierarchicalformat as one can have kt ≪ r for most t ∈ TD, cf. [8]. Moreover, the hierarchicalrepresentation can be exploited in order to perform efficient (approximate) arithmeticswith tensors that do not suffer from the structural weakness of the CP format (non-closedness of the representation, cf. [3]).A special instance of the hierarchical tensor format is the so-called TT format (tensor

train, matrix product states) from [12, 10] where the tree TD is restricted to a linearstructure. This means that one only needs to find a suitable permutation of the in-dices 1, . . . , d which facilitates the construction of an appropriate dimension tree. Note,however, that one can construct examples for which there exists a representation in thehierarchical format where all ranks are bounded by kt ≤ k whereas the ranks in the TTformat can only be bounded by klog2(d)/2−1 for any permutation, cf. [5].In this paper, we aim at finding an appropriate tree TD for a given tensor A without

any a priori knowledge on the tree structure. The tensor A is then approximated withrespect to TD up to some (heuristic) target accuracy ε by a black box strategy which weintroduced in [1]. Black box approximation strategies tailored to the TT format havealso been developed in [11].Since the number of possible trees scales exponentially in the dimension d, a global

optimization of the storage cost over all TD is mostly too expensive. Instead, we propose

2

to construct a tree in a bottom up way by a successive agglomeration of disjoint subsetsof D. Although this strategy is known from hierarchical clustering algorithms, we cannotdirectly apply them to our problem since we are not aware of a suitable distance functionmeasuring the closeness of arbitrary subsets t1, t2 ⊂ D. As an alternative, we introducea cluster criterion that completely relies on the ranks kt which depend on A and thesubsets t ⊂ D.The rest of this paper is organized as follows. In Section 2, we recall the main ingre-

dients of the hierarchical tensor format. In Section 3, we introduce our new clusteringstrategy based on a successive agglomeration of disjoint subsets t ⊂ D. Since one mostlydoes not know the ranks kt in advance, Section 4 is devoted to rank estimation tech-niques. In Section 5, we shortly analyze the complexity of our strategy in terms of thenumber of required tensor evaluations. Finally, we study the prospects and limitationsof our approach in Section 6 by a number of numerical examples.

2 Hierarchical Tensor Format

Given d ∈ N and n1, . . . , nd ∈ N, let Iµ := {1, . . . , nµ} for µ = 1, . . . , d and define

I := I1 × · · · × Id.

We first introduce a matrix representation of a tensor A ∈ RI .

Definition 1 (matricization). Let D := {1, . . . , d}. Given a subset t ⊂ D with comple-ment s := D \ t, the matricization

Mt : RI → RIt ⊗ RIs , It :=×

µ∈t

Iµ, Is :=×µ∈s

Iµ,

of a tensor A ∈ RI is defined by its entries

Mt(A)(iµ)µ∈t,(iµ)µ∈s := A(i1,...,id), iµ ∈ Iµ, µ ∈ D.

In order to allow for a structured and data-sparse representation of tensors, subsetst ⊂ D which can be organized as a binary tree are of special interest.

Definition 2 (partition tree, dimension tree). Let J be an arbitrary index set with#J < ∞ and denote its power set by P(J). A tree TJ ⊂ P(J) is called a partition tree(for J) if the following three conditions hold:

(a) the index set J is the root of the tree TJ ,

(b) all vertices t ∈ TJ are non-empty subsets t ⊂ J ,

(c) every vertex t ∈ TJ with #t ≥ 2 has two sons t1, t2 ∈ TJ with the property

t = t1 ∪ t2, t1 ∩ t2 = ∅.

3

The set of leaves of TJ is defined by L(TJ) := {t ∈ TJ : #t = 1}. For all t ∈ TJ \ L(TJ),we denote the set of sons of t by sons(t). A partition tree TD for the particular indexset J = D = {1, . . . , d} is called a dimension tree.

In the next example, we introduce two important special cases of dimension trees.

Example 3. (a) In a balanced binary tree TD, each node t ∈ TD \ L(TD) with t ={µ1, . . . , µq} ⊂ D, q > 1, has two sons t1, t2 ∈ TD of the form

t1 = {µ1, . . . , µr}, t2 = {µr+1, . . . , µq}, r := ⌈q/2⌉.

An example for d = 7 is depicted in Figure 1. The balanced tree is of minimal depth⌈log2 d⌉.(b) In the so-called TT format introduced in [12, 10], the dimension tree is a simplelinear tree, where all nodes t ∈ TD are of the form

t = {q} or t = {q, . . . , d}, q = 1, . . . , d.

An example for d = 7 is depicted in Figure 1. The TT tree is of maximal depth d− 1.

{1, 2, 3, 4, 5, 6, 7}

{1, 2, 3, 4} {5, 6, 7}

{1, 2} {3, 4} {5, 6} {7}

{1} {2} {3} {4} {5} {6}

{1, 2, 3, 4, 5, 6, 7}

{2, 3, 4, 5, 6, 7}

{3, 4, 5, 6, 7}

{4, 5, 6, 7}

{5, 6, 7}

{6, 7}

{1}

{2}

{3}

{4}

{5}

{6} {7}

Figure 1: Left: Balanced binary tree. Right: Linear TT tree.

Based on the concept of the matricization of tensors and the definition of a dimensiontree, we can now introduce the hierarchical tensor format.

Definition 4 (hierarchical rank, hierarchical format). Let TD be a dimension tree. Thehierarchical rank k := (kt)t∈TD of a tensor A ∈ R

I is defined by

kt := rank(Mt(A)), t ∈ TD.

For a given hierarchical rank k := (kt)t∈TD , the hierarchical format H(TD,k) is definedby

H(TD,k) := {A ∈ RI : rank(Mt(A)) ≤ kt, t ∈ TD}.

4

Given a tensor A ∈ H(TD,k), the subspaces Ut := image(Mt(A)) ⊂ RIt, t ∈ TD, fulfil

the so-called nestedness property

Ut ⊂ Ut1 ⊗ Ut2 , t ∈ TD \ L(TD), sons(t) = {t1, t2}. (1)

This allows for a recursive representation of A by the relation

(Ut)·,j =

kt1�

j1=1

kt2�

j2=1

(Bt)j,j1,j2(Ut1)·,j1 ⊗ (Ut2)·,j2 , j = 1, . . . , kt,

for all t ∈ TD \ L(TD) with sons(t) = {t1, t2} where Bt ∈ Rkt×kt1×kt2 and Ut ∈ R

It×kt

such that A = (UD)·,1.As a consequence, one only needs to store the matrices Ut ∈ R

Iµ×kt in the leavest = {µ} ∈ L(TD) and the transfer tensors Bt ∈ R

kt×kt1×kt2 for all inner nodes t ∈TD \ L(TD) in order to represent a tensor in H(TD,k). The storage complexity for thisrepresentation then sums up to

Nstorage(H(TD,k)) :=�

t∈TD\L(TD)

sons(t)={t1,t2}

ktkt1kt2 +

d�

µ=1

k{µ}nµ. (2)

In the special case nµ = n for all µ = 1, . . . , d and kt ≤ k for all t ∈ TD, the storagecomplexity lies in

O�

dk3 + dnk�

.

Remark 5. In the TT format, for each node t ∈ TD \L(TD) with sons(t) = {t1, t2} oneeither has t1 ∈ L(TD) or t2 ∈ L(TD). Hence, either kt1 ≤ n or kt2 ≤ n which leads tothe bound

O�

dnk2�

.

Note that the last term in (2) does not depend on the choice of the particular tree TD.This results from the fact that for a given tensor A ∈ RI the rank tuple (kµ)µ∈D with

kµ := rank(M{µ}(A)), µ ∈ D,

represents the Tucker rank of A which does not depend on the splitting of the directions1, . . . , d. In order to represent a tensor in the hierarchical format with minimal storagecost one therefore has to solve the following minimization problem.

Problem 6. Let A ∈ RI and let kt := rank(Mt(A)) for all t ⊂ D. Among all possibledimension trees TD find a minimizer of

�

t∈TD\L(TD)

sons(t)={t1,t2}

ktkt1kt2 . (3)

Unfortunately, the determination of the global optimum of Problem 6 results to beprohibitively expensive since the number of possible dimension trees grows exponentiallyin the dimension d. We therefore suggest to apply an agglomerative strategy known fromhierarchical clustering in order to construct a suitable dimension tree in a bottom upway.

5

3 Agglomerative Clustering

Hierarchical clustering strategies are used to subdivide a set of data points into nestedsubsets of similar points (the clusters) by means of an appropriately chosen similaritymeasure. In a top down approach, one starts with the full set of data points which issuccessively refined until only atomic sets — possibly consisting of single data points— remain. In a bottom up approach, one starts with a partition into clusters of singledata points which are successively joined until a single cluster containing the whole setof data points is reached. Typically, the similarity of clusters is measured by a givendistance function which is assumed to fulfil the triangle inequality.For the construction of a dimension tree, it is natural to consider the directions 1, . . . , d

as data points that need to be clustered in a hierarchical way. This directly leads tothe question of either using a top down or a bottom up approach and to the choiceof an appropriate similarity measure for clusters t ⊂ D := {1, . . . , d}. Since we aim atminimizing the sum (3), we need to take the ranks kt into account. Note that this meansthat we cannot simply use strategies known from hierarchical clustering as the clusterst ⊂ D do not naturally possess properties that fulfil the triangle inequality.Let A ∈ RI with kt := rank(Mt(A)) for all t ⊂ D. What we know in advance is that

whenever kt is small, a cluster t ⊂ D can be well separated from D \ t. This means thatwe can regard two disjoint clusters t1, t2 ⊂ D as closely related whenever the associatedrank kt for t := t1 ∪ t2 is small. Conversely, we can split a given cluster t ⊂ D into twoloosely related clusters t1, t2 with t = t1 ∪ t2, t1 ∩ t2 = ∅, whenever both kt1 and kt2 aresmall.The first observation leads to a bottom up strategy. Starting with initial clusters

{1}, . . . , {d}, we successively could try to join a subset of clusters to a new clustert ⊂ D such that the associated rank kt remains small. The second observation wouldlead to a top down approach. Starting with the single cluster D containing all possibledirections, we could try to split a cluster into subclusters such that the associated ranksfor all subclusters remain small. However, the top down approach suffers from a severedifficulty. Already for the initial set D, one obtains a number of possible splittings intotwo disjoint subsets that scales exponentially in the dimension d. We therefore prefer touse an agglomerative bottom up strategy that avoids this exponential complexity.

Pairwise Clustering

Assume that we are given a partion P = {t1, . . . , tr} of D with clusters tν ⊂ D. In orderto keep the sum (3) as small as possible, we would like to combine two clusters s1, s2 ∈ Pto a new cluster s := s1 ∪ s2 such that the associated rank ks is as small as possible.Another strategy is motivated by the nestedness property (1). Since ks1 , ks2 are thedimensions of the subspaces Us1 ,Us2 from (1), we could measure the quality of the newsubspace Us by the ratio ks/(ks1ks2). We demonstrate the effect of both strategies bythe following examples.

Example 7. Let D := {1, . . . , 8} and let TD be a balanced dimension tree as definedin Example 3. Moreover, let k = (kt)t∈TD be a rank tuple defined by kt := 2 for all

6

t ∈ TD \D, and kD := 1. Assume that A ∈ H(TD,k) ⊂ RI , I := {1, 2}8, is a random

tensor fulfilling kt = rank(Mt(A)) for all t ∈ TD. The dimension tree is visualized inFigure 2 left where we use subscripts in order to indicate the rank kt at a cluster t.For all subsets t ⊂ D with t /∈ TD, we generically assume that the associated rankskt = rank(Mt(A)) are maximal under the condition that A ∈ H(TD,k).We now apply our agglomerative strategy to A, ignoring our knowledge on the struc-

ture of the tree TD. Starting with the initial partition

P0 := {{1}, . . . , {8}},

the lowest rank kt = 2 is obtained by the union t := t1 ∪ t2 of one of the pairs t1 ={2µ− 1}, t2 = {2µ} for µ ∈ {1, . . . , 4}. By agglomeration, we find a new partition of theform, e.g.,

P1 := {{1, 2}, {3}, . . . , {8}}.

As the combination of the cluster {1, 2} with one of the clusters {3}, . . . , {8} correspondsto a rank of 4, afterwards we would again combine one of the pairs t1 = {2µ − 1}, t2 ={2µ} for µ ∈ {2, . . . , 4}. Two possible partitions are therefore given by

P ′2 := {{1, 2}, {3, 4}, {5}, {6}, {7}, {8}}, P′′2 := {{1, 2}, {3}, {4}, {5, 6}, {7}, {8}}.

One easily checks that also in the following steps only clusters are joined that correspondto the original structure of the dimension tree from Figure 2 left. However, since theagglomeration order is not fixed in advance, one may also end up with a tree of theshape shown in Figure 2 right.

{1, . . . , 8}1

{1, . . . , 4}2 {5, . . . , 8}2

{1, 2}2 {3, 4}2 {5, 6}2 {7, 8}2

{1}2{2}2{3}2{4}2{5}2{6}2{7}2{8}2

{1, . . . , 8}1

{1, 2}2 {3, . . . , 8}2

{1}2{2}2 {3, 4}2 {5, . . . , 8}2

{3}2{4}2 {5, 6}2 {7, 8}2

{5}2{6}2{7}2{8}2

Figure 2: Left: original dimension tree for d = 8 with subscripts kt at clusters t. Right:possible dimension tree after pairwise clustering.

Note that in this example the result of the agglomerative cluster strategy would havebeen the same if we had used the minimum of the ratios kt/(kt1kt2) ∈ {1/2, 1} as adecision criterion. In the next example, we study what can go wrong for this simplepairwise strategy.

7

Example 8. Let D := {1, . . . , 16} and let TD be a balanced dimension tree as definedin Example 3. Moreover, let k = (kt)t∈TD be a rank tuple defined by

kt :=

2, t = {µ}, µ = 1, . . . , 16,

4, t = {2µ − 1, 2µ}, µ = 1, . . . , 8,

4, t = {4µ − 3, . . . , 4µ}, µ = 1, . . . , 4,

4, t = {8µ − 7, . . . , 8µ}, µ = 1, 2,

1, t = D.

Assume that A ∈ H(TD,k) ⊂ RI , I := {1, 2}16, is a random tensor fulfilling kt =

rank(Mt(A)) for all t ∈ TD. The dimension tree and the ranks kt are visualized inFigure 3. As before, we assume for all subsets t ⊂ D with t /∈ TD that the associatedranks kt = rank(Mt(A)) are maximal under the condition that A ∈ H(TD,k).

{1, . . . , 16}1

{1, . . . , 8}4 {9, . . . , 16}4

{1, 2, 3, 4}4 {5, 6, 7, 8}4 {9, 10, 11, 12}4 {13, 14, 15, 16}4

{1, 2}4 {3, 4}4 {5, 6}4 {7, 8}4 {9, 10}4 {11, 12}4 {13, 14}4 {15, 16}4

{1}2 {2}2 {3}2 {4}2 {5}2 {6}2 {7}2 {8}2 {9}2 {10}2{11}2{12}2{13}2{14}2{15}2{16}2

Figure 3: Original dimension tree for d = 16 with subscripts kt.

We now apply our agglomerative strategy to A, ignoring our knowledge on the struc-ture of the tree TD. Starting with the initial partition

P0 := {{1}, . . . , {16}},

all pairs of disjoint clusters t1, t2 ∈ P0 lead to the minimal rank kt = 4, t := t1 ∪ t2.Hence, after eight agglomerative steps, we may have reached a partition of the form

P1 = {{µ, µ + 8} : µ = 1, . . . , 8}.

Now, all combinations of disjoint clusters t1, t2 ∈ P1 lead to the minimal rank kt = 16,t := t1 ∪ t2. After four agglomerative steps, we may have reached a partition of the form

P2 = {{µ, µ + 4, µ + 8, µ + 12} : µ = 1, . . . , 4}.

Now, all combinations of disjoint clusters t1, t2 ∈ P2 lead to the minimal rank kt = 256,t := t1 ∪ t2. This results in a possible partition

P3 = {{µ, µ + 2, µ + 4, µ + 6, µ + 8, µ + 10, µ + 12, µ + 14} : µ = 1, 2}

8

{1, . . . , 16}1

{1, 3, 5, 7, 9, 13, 15}256 {2, 4, 6, 8, 10, 12, 14, 16}256

{1, 5, 9, 13}16 {3, 7, 11, 15}16 {2, 6, 10, 14}16 {4, 8, 12, 16}16

{1, 9}4 {5, 13}4 {3, 11}4 {7, 15}4 {2, 10}4 {6, 14}4 {4, 12}4 {8, 16}4

{1}2 {9}2 {5}2 {13}2 {3}2 {11}2 {7}2 {15}2 {2}2 {10}2 {6}2 {14}2 {4}2 {12}2 {8}2 {16}2

Figure 4: Dimension tree for d = 16 and pairwise clustering with subscripts kt.

for which we eventually obtain P4 = {D}. The final dimension tree is depicted in Figure4. Note that the result would have been the same if we had used the ratios kt/(kt1kt2) = 1in each step.

Example 8 shows that both strategies did not lead to an appropriate dimension tree.One reason for this is that the optimal splitting of the directions 1, 2, 3, 4 from 5, 6, 7, 8and 9, 10, 11, 12 from 13, 14, 15, 16 could not be detected by the combination of pairsof clusters. Therefore, a possible generalization is to allow the combination of a largernumber of clusters in each agglomerative step.

Generalized Clustering

Let P = {t1, . . . , tr} be again a partition of D with clusters tν ⊂ D. For p ≥ 2, the set

Cp(P ) := {C ∈ P(P ) : 2 ≤ #C ≤ p}

contains the collection of possible combinations of at most p clusters from P . Among allC ∈ Cp(P ), we can still try to minimize the associated rank ks, s :=

�

t∈C t. Analogously,we could measure the ratio of the rank ks corresponding to the new cluster s :=

�

t∈C

with respect to the given ranks kt, t ∈ C. The following remark illustrates what we needto take into account when we want to compare ratios for different cardinalities of setsC ∈ Cp(P ).

Remark 9. Let t1, t2, u1, u2 ∈ P be pairwise disjoint clusters with t := t1 ∪ t2, u :=u1∪u2, and s := t∪u. Assume that the associated ranks are given by kt1 = kt2 = ku1 =ku2 = kt = ku = 2 and ks = 4. If we allow a combination of up to p = 4 clusters, weencounter that

kskt1kt2ku1ku2

=4

24=

1

4<

1

2=

ktkt1kt2

.

This means that the union of the clusters t and u would be rated better than the naturalcombination of t1 and t2. However, we find that ks/(ktku) = 1 which does not indicateany preference of joining t and u.

9

To overcome this problem, we suggest to use the following normalized ratio as adecision criterion.

Definition 10 (rank ratio). Let P be a partition of D and let C ⊂ P . The ratio

̺(C) :=�

ks/�

t∈C

kt

�1/(#C−1)

is called the rank ratio with respect to C where s :=�

t∈C t.

In Remark 9, we find that ̺({s}) = (1/4)1/3 > 1/2 = ̺({t}) such that the combinationof t and u is not given any preference. We now come back to Example 8 and compareour agglomerative clustering strategy based on the absolute values of the ranks to thestrategy using the rank ratios with p = 4.

Example 11. Assume first that in each agglomerative step the clusters contained inC ∈ Cp(P ) are joined such the rank ks, s :=

�

t∈C t, is minimal. In Example 8, theminimum ks over all C ∈ C4(P0) is 4. Therefore, one cannot distinguish if it is betterto join two arbitrary clusters from P0 or, e.g., the clusters {1}, {2}, {3}, {4}. As aconsequence, one may successively arrive at the partition P1. Afterwards, the minimumks over all C ∈ C4(P1) is again 4 and one may end up at the partition P2 as before. Oncethe partition P2 is obtained, the rank 256 is unavoidable for some cluster in the finaldimension tree.Secondly, we use the rank ratio in order to decide which clusters we would like to join.

The minimal ratio for all C ∈ C4(P0) is obtained by, e.g., C = {{1}, {2}, {3}, {4}} forwhich ̺(C) = (4/24)1/3 = (1/4)1/3 < 1. Note that for all C ∈ C3(P0) we have ̺(C) = 1.In the next step, the minimal ratio is also obtained by C = {{1, 2, 3, 4}, {5}, {6}, {7}}since ks = 8, s :=

�

t∈C t, such that ̺(C) = (8/(4 · 23))1/3 = (1/4)1/3. Afterwards, the

minimal ratio is obtained for C = {{1, . . . , 7}, {8}} (only) since ̺(C) = 4/(8 ·2) = 1/4 <(1/4)1/3. Therefore, the splitting of the directions 1, . . . , 8 from 9, . . . , 16 is detectedeven in the worst case scenario. The maximal rank appearing at all clusters in the finaldimension tree is 8.

The preceding example clearly indicates that it can be advantageous to use a relativecluster criterion like the rank ratio from Definition 10 instead of an absolute one. Letus now assume that for a given partition P of D an optimal set C ′ ∈ Cp(P ) has beenidentified by either of the two strategies. We then need to update P by a new partition

P ′ := (P \ C ′) ∪�

∪t∈C′t�

. (4)

The clusters t ∈ C ′ still need to be organized as a binary tree. Due to Problem 6, wehave to solve the following (local) minimization problem.

Problem 12. Let P be a partition of D and let C ⊂ P . For all τ ∈ P(C) let kτ := ks,s :=

�

t∈τ t. Among all possible partition trees TC find a minimizer of�

τ∈TC\L(TC )

sons(τ)={τ1,τ2}

kτkτ1kτ2 .

10

As long as the cardinality of C ′ is small, one can try out all possibilities for TC′ in orderto solve Problem 12. Once an optimal tree TC′ has been determined, we need to updatethe set of nodes T ⊂ P(D) from the previous step. The elements of C ′ = {t1, . . . , tr},tν ⊂ D, are already contained in T such that we only need to add subsets s ⊂ D toT for which there exists an inner node τ ∈ TC′ \ L(TC′) with s =

�

t∈τ t. The updateprocedure is visualized for C ′ = {t1, . . . , t4} in Figure 5.

t1 t2 t3 t4

∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗

T

{t1, . . . , t4}

{t1, t2} {t3, t4}

{t1} {t2} {t3} {t4}

TC′

t1 ∪ . . . ∪ t4

t1 ∪ t2 t3 ∪ t4

t1 t2 t3 t4

∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗

T ′

Figure 5: Agglomeration of nodes from T to T ′ by means of a local tree TC′ .

We have summarized our strategy in Algorithm 1.

Algorithm 1 T = BuildTree(d, p)

Define P := T := {{1}, . . . , {d}}while #P > 1 doChoose C ′ ∈ Cp(P ) with C

′ := argminC∈Cp(P ) ̺(C)Find an optimal partition tree TC′ according to Problem 12Update P := (P \ C ′) ∪ {

�

t∈C′ t}Update T := T ∪ {

�

t∈τ t : τ ∈ TC′ \ L(TC′)}end while

return T

4 Rank Estimation

In the previous section, we have assumed an exact knowledge of the ranks kt for allt ⊂ D. If the ranks are not known a priori, one has to determine them numerically. Aslong as #I is comparatively small, we can assume that a tensor A ∈ RI is availablein its full representation such that we can perform standard rank revealing techniquesfor matrices applied to the matricizations Mt(A). For large #I , we have to base arank estimate on a subset of tensor entries of A. This in turn means that in general— even with exact numerical computations — all rank estimation techniques for thematricizations of A need to remain heuristic.For matrices M ∈ RI×J , the adaptive cross approximation [2] is a well-known tech-

11

nique for the construction of low-rank approximations of the form

M ≈ Xj := US−1V ⊤, (5)

with matricesU =M |I×Q, S =M |P×Q, V =M |J×P ,

and sets of pivots P ⊂ I, Q ⊂ J with #P = #Q = j. The approximations Xj can befound in an incremental way by adding rank one updates to Xj−1 which are constructedfrom the remainder Rj := M − Xj−1, cf. [2]. In each approximation step, the pivotsets P and Q are updated by a row and column index for which the modulus of thecorresponding entry in Rj is as large as possible. The number of entries required fromM then lies inO((#I+#J)k) where k corresponds to the rank of the final approximationXk.Let t ⊂ D, s := D \ t, and M := Mt(A) ∈ R

It×Is . As we have pointed out in [1],the cross approximation technique can be used to find low-rank approximations to Mby two modifications:

1. The matrices U and V from (5) are never formed explicitly. The low-rank repre-sentation is used to compute entries of the remainder Rj directly.

2. Pivot elements are not sought in full rows or columns but only on so-called fibersA(i1,...,iµ−1,·,iµ+1,...,id) ∈ R

Iµ .

It can be shown that the construction of the pivot sets P and Q requires the evaluationof O(dnk2) entries from A, n := maxµ∈D nµ, where the rank k of M is determined withrespect to some (heuristic) target accuracy ε with �M −Xk�2 � ε�M�2.An alternative approach is to first determine a random subset of row indices I ⊂ It

and a random subset of column indices J ⊂ Is such that #I and #J are small. We canthen apply rank-revealing strategies to the submatrixM :=Mt(A)|I×J . Since we mostlydo not know an appropriate size of M in advance, we suggest to adaptively enlarge thesize of M by the following simple strategy.For ℓ = 0, ..., ℓmax, we construct a sequence of random index sets Iℓ ⊂ It, Jℓ ⊂ Is

of size sℓ := #Iℓ = #Jℓ until the matrix Mℓ := Mt(A)|Iℓ×Jℓ has reached some level ofredundancy that indicates the (heuristic) size of the rank ofMt(A). First, we start withsmall initial random pivot sets I0 ⊂ It, J0 ⊂ Is, of size s0 := #I0 = #J0 (say s0 = 10).We can then estimate the rank k0 of M0 := Mt(A)|I0×J0 by means of the standardadaptive cross approximation. If at step ℓ = 0, ..., ℓmax we have that kℓ < 2sℓ, we stopthe iteration and return kℓ as a good rank estimate. Otherwise, we set sℓ+1 := 3kℓ andproceed to the next step. By means of this strategy we aim at approaching the true rankof Mt(A) from the bottom. The number of entries required from A then lies in O(sk)where s and k correspond to the size and the rank of the final submatrixM , respectively.

5 Complexity

In a short complexity analysis, we focus on a comparison of the computational cost forthe proposed adaptive setup of a dimension tree with the cost for the final black box

12

approximation of a tensor by, e.g., [1]. In particular, we would like to compare thenumber of tensor evaluations arising from a combination of the two approaches.In order to estimate the complexity of Algorithm 1, first note that each computation

of ̺(C), C ∈ Cp(P ), corresponds to the determination of a rank ks, s :=�

t∈C t, forwhich we assume a still abstract complexity of Nrank tensor evaluations. For the initialpartition P = {{1}, . . . , {d}}, we have

#Cp(P ) =

�

d

2

�

+ . . .+

�

d

p

�

= O(dp).

The computation of ̺(C) for all C ∈ Cp(P ) for the first time therefore requiresO(dpNrank)

tensor evaluations. After an agglomerative step, the partition P is updated to P ′ ac-cording to (4) such that the values ̺(C) need only be computed for all C ∈ P ′ for whichs ∈ C, s :=

�

t∈C′ t. This requires at most O(dp−1Nrank) tensor evaluations. As there

are at most d agglomerative steps, we again arrive at a complexity of O(dpNrank).The local optimization from Problem 12 does not require any tensor evaluations since

all necessary ranks are already available. The combinatorial complexity is determinedby the number of labeled full binary trees with p leaves which is given by p!Cp−1 whereCp denotes the p-th Catalan number. As long as p ≤ d/2, we can hence estimate

p!Cp−1 =p!(2p − 2)!

p!(p− 1)!=

p−2�

j=0

(p+ j) < dp−1.

This means that in each agglomerative step the combinatorial complexity for the localminimization is also bounded by O(dp−1). This leads to an overall bound of O(dp) whichdoes not exceed the number of O(dpNrank) tensor evaluations.Assume now that Nrank = O(k

2) where k is the maximal rank of all matricizationsconsidered in the agglomerative strategy. This bound can, e.g., be achieved by applyingthe rank estimation technique based on random submatrices as introduced in the lastsection. We then have to evaluate O(dpk2) tensor entries to construct a dimension treeby our adaptive strategy. This needs to be compared with the black box strategy from[1] which requires the evaluation of O(dk3 + d2k2n) tensor entries. We can easily seethat for p = 2, 3 both strategies may lead to comparable costs whereas for p ≥ 4 the costfor the construction of the tree will dominate. This effect was also observed (though notrigorously analyzed) in the numerical examples presented in the next section.

6 Numerical Examples

In the numerical part, we are interested in the following questions:

1. How well can we recover a known tree structure by our clustering strategy?

2. How much do we have to invest for the estimation of the ranks kt in order to reacha given target accuracy?

13

3. How large is the impact of the choice of the maximal cluster size p on the qualityof the final tree?

Throughout this section, we restrict ourselves to tensors A ∈ RI with #Iµ := n forall µ = 1, . . . , d. In order to compare the storage complexity of tensors A ∈ H(TD,k)with respect to different trees TD, we introduce the effective rank keff as the positive realsolution of

(d− 1)k3eff + dnkeff = Nstorage(H(TD,k))

where Nstorage was defined in (2). A low value of keff then corresponds to a low storagecomplexity of a tensor with respect to TD.As discussed in Section 4, we heuristically estimate the rank of the matricization of

a tensor by the rank of an appropriately chosen random submatrix for a prescribedaccuracy εest. Once a tree TD has been determined by our (deterministic) clusteringstrategy, we approximate a given tensor with respect to TD by the black box strategyfrom [1] with a prescribed (heuristic) target accuracy εfinal. Since the determinationof the ranks involves a random component, we perform 100 runs for each example andreport the resulting average effective rank kavgeff and the worst effective rank k

worsteff of the

final approximation over all runs. Our first example is motivated by Example 8.

Example 13. Given d ∈ N, let TD be a balanced dimension tree as defined in Example3(a). For n = 2, let A ∈ H(TD,k) ⊂ R

I be a random tensor with ranks

kt :=

1, t = D,

2, t ∈ L(TD),

4, otherwise,

such that the singular values of the matricizations Mt(A) at each node t ∈ TD decayapproximately like 2−j , j = 1, . . . , kt. We now permute the indices of A such that theoriginal structure of the tree TD becomes invisible. We then apply our algorithm witha prescribed accuracy of εest = εfinal = 10

−4 in dimension d = 8, 16, 32 for differentvalues of the maximal cluster size p = 2, 3, 4. For all runs, the accuracy of the finalapproximation was close to the presribed error tolerance.The average and best effective rank for the adaptive strategy can be compared in

Table 1 with the original effective rank korigeff which we had obtained taking into accountthe knowledge of the hidden tree structure. Moreover, we report the effective ranks kbaleffand klineff which we obtain for a fixed (unpermuted) balanced and linear dimension tree,respectively, without taking into account our adaptive strategy.We can see that the ranks kbaleff and k

lineff rapidly increase with increasing d. For d = 32,

we could not even meet the given target accuracy assuming a maximal number of 3000pivot elements in (5). When applying our adaptive strategy, the original tensor structuretends to be the better resolved the more we increase the maximal cluster size p for alld = 8, 16, 32. However, with increasing d it gets more and more difficult to recover theoriginal tree structure for all p = 2, 3, 4.

14

d korigeff kbaleff k

lineff p = 2 p = 3 p = 4

8 3.0 4.8 4.5 4.3 / 4.3 3.1 / 3.2 3.0 / 3.016 3.2 19.5 19.2 8.8 / 13.0 4.1 / 4.9 4.0 / 4.032 3.3 > 200 > 200 26.5 / 35.1 13.7 / 34.0 5.2 / 8.3

Table 1: Random tensors with n = 2 with original effective ranks korigeff . Ranks kbaleff and

klineff for fixed balanced and linear dimension trees. Average and worst effectiveranks kavgeff / k

worsteff for different values of the maximal cluster size p.

This example indicates that an appropriate splitting of the directions 1, . . . , d maynot be discovered on the level of pairwise combinations such that we need to apply ouralgorithm with p > 2. In the next example, we study the behaviour of our methodwith respect to different accuracies for the construction of the tree and for the finalapproximation.

Example 14. Given d = 16, let TD be a balanced dimension tree as defined in Example3(a). For n = 20, let A ∈ H(TD,k) ⊂ R

I be a random tensor with ranks kt = 20for all t ∈ TD \ D such that the singular values of the matricizations Mt(A) at eachnode t ∈ TD decay approximately like 2

−j , j = 1, . . . , kt. Again, we permute theindices of A such that the original structure of the tree TD becomes invisible. We nowapply our algorithm with a maximal cluster size of p = 2 for different accuracies ofεest, εfinal ∈ {10

−3, 10−4, 10−5, 10−6}, εfinal ≤ εest. For all runs, the accuracy of the finalapproximation was close to the presribed error tolerance.The average and best effective rank for the adaptive strategy can be compared in

Table 2 with the original effective rank korigeff which we had obtained taking into accountthe knowledge of the hidden tree structure. Moreover, we report the effective ranks kbaleffand klineff which we obtain for a fixed (unpermuted) balanced and linear dimension tree,respectively, without taking into account our adaptive strategy.

εfinal korigeff k

baleff k

lineff εest = 1e-03 εest = 1e-04 εest = 1e-05 εest = 1e-06

1e-03 2.9 7.1 7.2 3.7 / 5.9 – – –1e-04 5.4 33.3 35.2 10.7 / 21.9 5.6 / 6.9 – –1e-05 7.9 – – 22.9 / 44.6 8.3 / 11.7 7.9 / 7.9 –1e-06 10.7 – – 42.5 / 68.8 11.8 / 21.6 10.7 / 10.7 10.7 / 10.8

Table 2: Random tensors in d = 16 with n = 20 and original effective ranks korigeff . Rankskbaleff and k

lineff for fixed balanced and linear dimension trees. Average and worst

effective ranks kavgeff / kworsteff for different values for the accuracy εest controlling

the generation of the tree and for the final approximation accuracy εfinal.

Given a fixed tree, the ranks kbaleff and klineff rapidly increase with the accuracy εfinal.

For a too crude choice of the accuracy εest, the original tree structure cannot be resolvedup to the final target accuracy εfinal. If however εest ≈ εfinal, the original structure canbe identified.

15

In a third example, we study the behaviour of our method for a tensor generated bya smooth function.

Example 15. Given d = 16, let Q ∈ Rd×d and define

f(x) := exp(−12x⊤Qx), x ∈ Ω := [−2, 2]d.

In order to control the separability of the function f with respect to the variablesx1, . . . , xd, we first introduce standard Givens rotations G(i, j, ϕ) ∈ R

d×d, ϕ ∈ R, by

G(i, j, ϕ)k,ℓ :=

cosϕ, k = i, ℓ = i, or k = j, ℓ = j,

sinϕ, k = i, ℓ = j,

− sinϕ, k = j, ℓ = i,

1, ℓ = k �= i, j,

0, otherwise.

Let now

Q :=

3�

i=0

3�

j=1

G(4i + 1, 4i + j + 1, ϕ)

3�

j=1

G(1, 4j + 1, ϕ).

Note that for ϕ = 0, we have Q = Id such that the function f is separable with respectto x1, . . . , xd. With increasing ϕ, we expect a stronger coupling between the directionslinked through the Givens rotations.For n = 32, let A ∈ RI be the discretization of f on an equidistant tensor grid on Ω

of size n in each direction. As in the preceding examples, we permute the indices of Aand apply our algorithm with maximal cluster sizes of p = 2, 3, 4 for different accuraciesεfinal = εest. In Table 3 (top), we compare the results to the effective rank k

origeff obtained

by using a balanced binary tree for an unpermuted tensor with ϕ = π/16. We reportthe results of the same experiment with ϕ = π/8 in Table 3 (bottom).As expected, a larger value of ϕ leads to a stronger coupling of f with respect to

x1, . . . , xd which is reflected by higher effective ranks in all examples. Moreover theoriginal tensor structure tends to be better resolved if we increase the maximal clustersize up to p = 4. Note however that already for p = 2 moderate effective ranks could beobtained in all examples.

In the last example, we apply our algorithm to the discretization of a two-dimensionalfunction using a tensorization approach.

Example 16. Let α ∈ R and define a function g : R× R→ R by

g(x, y) := (|y − αx|+ 1)−1.

Given d ∈ N, let N := 2d and define an equidistant grid on [−1, 1) of mesh size h := 2/Nby

ξℓ = ℓh− 1, ℓ = 0, . . . , N − 1.

16

εfinal korigeff k

baleff k

lineff p = 2 p = 3 p = 4

1e-03 2.3 2.8 2.5 2.4 / 2.8 2.4 / 2.6 2.4 / 2.71e-04 3.9 7.5 6.5 5.1 / 6.6 4.7 / 6.7 4.7 / 6.51e-05 4.7 11.8 10.0 5.6 / 7.4 5.1 / 6.6 4.6 / 5.51e-06 6.9 22.3 17.7 8.7 / 12.2 7.3 / 10.3 6.8 / 9.41e-07 9.1 – – 11.7 / 14.1 9.2 / 11.5 8.2 / 9.71e-08 11.0 – – 15.3 / 24.8 13.5 / 17.2 10.8 / 13.7

εfinal korigeff k

baleff k

lineff p = 2 p = 3 p = 4

1e-03 4.5 10.6 8.8 6.1 / 8.7 5.7 / 8.5 5.6 / 7.81e-04 7.8 25.4 19.7 10.0 / 14.1 8.6 / 13.8 7.8 / 11.71e-05 11.0 49.8 38.0 14.5 / 19.7 12.1 / 19.8 10.2 / 13.71e-06 14.7 71.4 46.7 20.0 / 30.7 16.7 / 26.5 13.8 / 19.51e-07 18.9 – – 30.4 / 47.4 25.3 / 38.8 18.8 / 27.81e-08 22.9 – – 35.7 / 48.4 28.6 / 42.0 21.5 / 32.3

Table 3: Function generated tensor from Example 15 for ϕ = π/16 (top) and ϕ = π/8(bottom). Ranks kbaleff and k

lineff for fixed balanced and linear dimension trees.

Average and worst effective ranks kavgeff / kworsteff for different values of the accu-

racy εfinal = εest and of the maximal cluster size p.

The points ξℓ may equally be indexed by a tuple (i1, . . . , id) ∈ {0, 1}d, i.e.,

ξ(i1,...,id) := ξℓ, ℓ =d

�

µ=1

iµ2µ−1.

We now consider a tensor A ∈ RI , I := {0, 1}2d, defined by

A(i1,...,i2d) := g(ξ(i1,...,id), ξ(id+1,...,i2d)), iµ ∈ {0, 1}, µ = 1, . . . , 2d.

For d = 8, we can store all entries of A and perform standard singular value decompo-sitions of the matricizations of A in order to determine their ranks up to an accuracy ofεest = εfinal = 10

−8. We now perform our algorithm with p = 2 for different values of αand compare the resulting effective ranks to the approximation with respect to a fixedbalanced and fixed linear tree in Table 4. A visualization of the corresponding treesproduced by our algorithm is depicted in Figure 6.For non-trivial values of α, the adaptive strategy allows for a much better resolution

of the tensor structure than a fixed choice of the dimension tree. As Figure 6 illus-trates, the tensor from the given example tends to be optimally represented by a linearstructure. The corresponding (non-trivial) permutations of the directions 1, . . . , 2d couldbe identified by our algorithm. They indicate a typical pattern of the coupling of thecomponents of the variables x and y in dependence on α.

17

α kbaleff klineff k

adapeff

0 2.5 2.4 2.41/4 11.6 9.8 3.81/2 16.7 14.9 3.83/4 20.4 19.0 4.71 23.6 22.6 3.9

Table 4: Tensorization of a two-dimensional function with 256× 256 points. SVD-basedrank estimation with accuracy εfinal = 10

−8. Ranks kbaleff and klineff for fixed

balanced and linear dimension trees. Ranks kadapeff with adaptively generateddimension trees for different values of α.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

α = 0

1 2 3 9 4 10 5 11 6 12 7 13 16 15 8 14

α = 14

3 10 2 9

1 4 11 5 12 6 13 16 8 15 7 14

α = 12

1 2 9 3 10 4 11 5 12 6 13 7 15 8 14 16

α = 34

2 10 1 9

3 11 4 12 5 13 6 14 8 16 7 15

α = 1

Figure 6: Tensorization of a two-dimensional function. Dimension trees for differentvalues of α with components in x numbered from 1 to 8 (blue) and componentsin y numbered from 9 to 16 (green).

7 Conclusion

In this paper, we have introduced a strategy for the detection of an appriopriate di-mension tree for tensors that can be represented in the hierarchical tensor format. Our

18

algorithm is based on an agglomeration of clusters in a bottom up way controlled by therelative reduction of the corresponding ranks in each step. By means of the numericalexperiments, we verified that in many cases a reasonable tree structure could alreadybe revealed by the combination of pairs of clusters. However, we also pointed out thatone can find examples for which the combination of a higher number of clusters needsto be considered. On the one hand, this leads to higher computational costs for theconstruction of the tree in a setup step. On the other hand, one can benefit from theimproved structure of the final representation in follow up computations.

References

[1] J. Ballani, L. Grasedyck, and M. Kluge. Black box approximation of tensors inhierarchical Tucker format. Linear Algebra Appl., 438(2):639–657, 2013.

[2] M. Bebendorf. Approximation of boundary element matrices. Numer. Math.,86(4):565–589, 2000.

[3] V. De Silva and L.-H. Lim. Tensor rank and the ill-posedness of the best low-rankapproximation problem. SIAM J. Matrix Anal. Appl., 30:1084–1127, 2008.

[4] L. Grasedyck. Hierarchical singular value decomposition of tensors. SIAM J. MatrixAnal. Appl., 31:2029–2054, 2010.

[5] L. Grasedyck and W. Hackbusch. An introduction to hierarchical (H-) rank andTT-rank of tensors with examples. Comput. Methods Appl. Math., 11(3):291–304,2011.

[6] L. Grasedyck, D. Kressner, and C. Tobler. A literature survey of low-rank tensorapproximation techniques. Preprint, ANCHP, MATHICSE, EPF Lausanne, 2013.

[7] W. Hackbusch. Tensor Spaces and Numerical Tensor Calculus. Springer, Berlin,2012.

[8] W. Hackbusch and S. Kühn. A new scheme for the tensor representation. J. FourierAnal. Appl., 15(5):706–722, 2009.

[9] T. G. Kolda and B. W. Bader. Tensor decompositions and applications. SIAMReview, 51(3):455–500, 2009.

[10] I. V. Oseledets. Tensor-train decomposition. SIAM J. Sci. Comput., 33(5):2295–2317, 2011.

[11] I. V. Oseledets and E. E. Tyrtyshnikov. TT-cross approximation for multidimen-sional arrays. Linear Algebra Appl., 432:70–88, 2010.

[12] G. Vidal. Efficient classical simulation of slightly entangled quantum computations.Phys. Rev. Lett., 91(14), 2003.

19

Preprint Series DFG-SPP 1324

http://www.dfg-spp1324.de

Reports

[1] R. Ramlau, G. Teschke, and M. Zhariy. A Compressive Landweber Iteration forSolving Ill-Posed Inverse Problems. Preprint 1, DFG-SPP 1324, September 2008.

[2] G. Plonka. The Easy Path Wavelet Transform: A New Adaptive Wavelet Trans-form for Sparse Representation of Two-dimensional Data. Preprint 2, DFG-SPP1324, September 2008.

[3] E. Novak and H. Woźniakowski. Optimal Order of Convergence and (In-)Tractability of Multivariate Approximation of Smooth Functions. Preprint 3,DFG-SPP 1324, October 2008.

[4] M. Espig, L. Grasedyck, and W. Hackbusch. Black Box Low Tensor Rank Ap-proximation Using Fibre-Crosses. Preprint 4, DFG-SPP 1324, October 2008.

[5] T. Bonesky, S. Dahlke, P. Maass, and T. Raasch. Adaptive Wavelet Methodsand Sparsity Reconstruction for Inverse Heat Conduction Problems. Preprint 5,DFG-SPP 1324, January 2009.

[6] E. Novak and H. Woźniakowski. Approximation of Infinitely Differentiable Mul-tivariate Functions Is Intractable. Preprint 6, DFG-SPP 1324, January 2009.

[7] J. Ma and G. Plonka. A Review of Curvelets and Recent Applications. Preprint 7,DFG-SPP 1324, February 2009.

[8] L. Denis, D. A. Lorenz, and D. Trede. Greedy Solution of Ill-Posed Problems:Error Bounds and Exact Inversion. Preprint 8, DFG-SPP 1324, April 2009.

[9] U. Friedrich. A Two Parameter Generalization of Lions’ Nonoverlapping DomainDecomposition Method for Linear Elliptic PDEs. Preprint 9, DFG-SPP 1324,April 2009.

[10] K. Bredies and D. A. Lorenz. Minimization of Non-smooth, Non-convex Func-tionals by Iterative Thresholding. Preprint 10, DFG-SPP 1324, April 2009.

[11] K. Bredies and D. A. Lorenz. Regularization with Non-convex Separable Con-straints. Preprint 11, DFG-SPP 1324, April 2009.

[12] M. Döhler, S. Kunis, and D. Potts. Nonequispaced Hyperbolic Cross Fast FourierTransform. Preprint 12, DFG-SPP 1324, April 2009.

[13] C. Bender. Dual Pricing of Multi-Exercise Options under Volume Constraints.Preprint 13, DFG-SPP 1324, April 2009.

[14] T. Müller-Gronbach and K. Ritter. Variable Subspace Sampling and Multi-levelAlgorithms. Preprint 14, DFG-SPP 1324, May 2009.

[15] G. Plonka, S. Tenorth, and A. Iske. Optimally Sparse Image Representation bythe Easy Path Wavelet Transform. Preprint 15, DFG-SPP 1324, May 2009.

[16] S. Dahlke, E. Novak, and W. Sickel. Optimal Approximation of Elliptic Problemsby Linear and Nonlinear Mappings IV: Errors in L2 and Other Norms. Preprint 16,DFG-SPP 1324, June 2009.

[17] B. Jin, T. Khan, P. Maass, and M. Pidcock. Function Spaces and Optimal Currentsin Impedance Tomography. Preprint 17, DFG-SPP 1324, June 2009.

[18] G. Plonka and J. Ma. Curvelet-Wavelet Regularized Split Bregman Iteration forCompressed Sensing. Preprint 18, DFG-SPP 1324, June 2009.

[19] G. Teschke and C. Borries. Accelerated Projected Steepest Descent Method forNonlinear Inverse Problems with Sparsity Constraints. Preprint 19, DFG-SPP1324, July 2009.

[20] L. Grasedyck. Hierarchical Singular Value Decomposition of Tensors. Preprint 20,DFG-SPP 1324, July 2009.

[21] D. Rudolf. Error Bounds for Computing the Expectation by Markov Chain MonteCarlo. Preprint 21, DFG-SPP 1324, July 2009.

[22] M. Hansen and W. Sickel. Best m-term Approximation and Lizorkin-TriebelSpaces. Preprint 22, DFG-SPP 1324, August 2009.

[23] F.J. Hickernell, T. Müller-Gronbach, B. Niu, and K. Ritter. Multi-level MonteCarlo Algorithms for Infinite-dimensional Integration on RN. Preprint 23, DFG-SPP 1324, August 2009.

[24] S. Dereich and F. Heidenreich. A Multilevel Monte Carlo Algorithm for LévyDriven Stochastic Differential Equations. Preprint 24, DFG-SPP 1324, August2009.

[25] S. Dahlke, M. Fornasier, and T. Raasch. Multilevel Preconditioning for AdaptiveSparse Optimization. Preprint 25, DFG-SPP 1324, August 2009.

[26] S. Dereich. Multilevel Monte Carlo Algorithms for Lévy-driven SDEs with Gaus-sian Correction. Preprint 26, DFG-SPP 1324, August 2009.

[27] G. Plonka, S. Tenorth, and D. Roşca. A New Hybrid Method for Image Approx-imation using the Easy Path Wavelet Transform. Preprint 27, DFG-SPP 1324,October 2009.

[28] O. Koch and C. Lubich. Dynamical Low-rank Approximation of Tensors.Preprint 28, DFG-SPP 1324, November 2009.

[29] E. Faou, V. Gradinaru, and C. Lubich. Computing Semi-classical Quantum Dy-namics with Hagedorn Wavepackets. Preprint 29, DFG-SPP 1324, November2009.

[30] D. Conte and C. Lubich. An Error Analysis of the Multi-configuration Time-dependent Hartree Method of Quantum Dynamics. Preprint 30, DFG-SPP 1324,November 2009.

[31] C. E. Powell and E. Ullmann. Preconditioning Stochastic Galerkin Saddle PointProblems. Preprint 31, DFG-SPP 1324, November 2009.

[32] O. G. Ernst and E. Ullmann. Stochastic Galerkin Matrices. Preprint 32, DFG-SPP1324, November 2009.

[33] F. Lindner and R. L. Schilling. Weak Order for the Discretization of the StochasticHeat Equation Driven by Impulsive Noise. Preprint 33, DFG-SPP 1324, November2009.

[34] L. Kämmerer and S. Kunis. On the Stability of the Hyperbolic Cross DiscreteFourier Transform. Preprint 34, DFG-SPP 1324, December 2009.

[35] P. Cerejeiras, M. Ferreira, U. Kähler, and G. Teschke. Inversion of the noisy Radontransform on SO(3) by Gabor frames and sparse recovery principles. Preprint 35,DFG-SPP 1324, January 2010.

[36] T. Jahnke and T. Udrescu. Solving Chemical Master Equations by AdaptiveWavelet Compression. Preprint 36, DFG-SPP 1324, January 2010.

[37] P. Kittipoom, G. Kutyniok, and W.-Q Lim. Irregular Shearlet Frames: Geometryand Approximation Properties. Preprint 37, DFG-SPP 1324, February 2010.

[38] G. Kutyniok and W.-Q Lim. Compactly Supported Shearlets are OptimallySparse. Preprint 38, DFG-SPP 1324, February 2010.

[39] M. Hansen and W. Sickel. Best m-Term Approximation and Tensor Products ofSobolev and Besov Spaces – the Case of Non-compact Embeddings. Preprint 39,DFG-SPP 1324, March 2010.

[40] B. Niu, F.J. Hickernell, T. Müller-Gronbach, and K. Ritter. Deterministic Multi-level Algorithms for Infinite-dimensional Integration on RN. Preprint 40, DFG-SPP 1324, March 2010.

[41] P. Kittipoom, G. Kutyniok, and W.-Q Lim. Construction of Compactly SupportedShearlet Frames. Preprint 41, DFG-SPP 1324, March 2010.

[42] C. Bender and J. Steiner. Error Criteria for Numerical Solutions ofBackward SDEs. Preprint 42, DFG-SPP 1324, April 2010.

[43] L. Grasedyck. Polynomial Approximation in Hierarchical Tucker Format byVector-Tensorization. Preprint 43, DFG-SPP 1324, April 2010.

[44] M. Hansen und W. Sickel. Best m-Term Approximation and Sobolev-Besov Spacesof Dominating Mixed Smoothness - the Case of Compact Embeddings. Preprint 44,DFG-SPP 1324, April 2010.

[45] P. Binev, W. Dahmen, and P. Lamby. Fast High-Dimensional Approximation withSparse Occupancy Trees. Preprint 45, DFG-SPP 1324, May 2010.

[46] J. Ballani and L. Grasedyck. A Projection Method to Solve Linear Systems inTensor Format. Preprint 46, DFG-SPP 1324, May 2010.

[47] P. Binev, A. Cohen, W. Dahmen, R. DeVore, G. Petrova, and P. Wojtaszczyk.Convergence Rates for Greedy Algorithms in Reduced Basis Methods. Preprint 47,DFG-SPP 1324, May 2010.

[48] S. Kestler and K. Urban. Adaptive Wavelet Methods on Unbounded Domains.Preprint 48, DFG-SPP 1324, June 2010.

[49] H. Yserentant. The Mixed Regularity of Electronic Wave Functions Multiplied byExplicit Correlation Factors. Preprint 49, DFG-SPP 1324, June 2010.

[50] H. Yserentant. On the Complexity of the Electronic Schrödinger Equation.Preprint 50, DFG-SPP 1324, June 2010.

[51] M. Guillemard and A. Iske. Curvature Analysis of Frequency Modulated Manifoldsin Dimensionality Reduction. Preprint 51, DFG-SPP 1324, June 2010.

[52] E. Herrholz and G. Teschke. Compressive Sensing Principles and Iterative SparseRecovery for Inverse and Ill-Posed Problems. Preprint 52, DFG-SPP 1324, July2010.

[53] L. Kämmerer, S. Kunis, and D. Potts. Interpolation Lattices for Hyperbolic CrossTrigonometric Polynomials. Preprint 53, DFG-SPP 1324, July 2010.

[54] G. Kutyniok and W.-Q Lim. Shearlets on Bounded Domains. Preprint 54, DFG-SPP 1324, July 2010.

[55] A. Zeiser. Wavelet Approximation in Weighted Sobolev Spaces of Mixed Orderwith Applications to the Electronic Schrödinger Equation. Preprint 55, DFG-SPP1324, July 2010.

[56] G. Kutyniok, J. Lemvig, and W.-Q Lim. Compactly Supported Shearlets.Preprint 56, DFG-SPP 1324, July 2010.

[57] A. Zeiser. On the Optimality of the Inexact Inverse Iteration Coupled with Adap-tive Finite Element Methods. Preprint 57, DFG-SPP 1324, July 2010.

[58] S. Jokar. Sparse Recovery and Kronecker Products. Preprint 58, DFG-SPP 1324,August 2010.

[59] T. Aboiyar, E. H. Georgoulis, and A. Iske. Adaptive ADER Methods Using Kernel-Based Polyharmonic Spline WENO Reconstruction. Preprint 59, DFG-SPP 1324,August 2010.

[60] O. G. Ernst, A. Mugler, H.-J. Starkloff, and E. Ullmann. On the Convergence ofGeneralized Polynomial Chaos Expansions. Preprint 60, DFG-SPP 1324, August2010.

[61] S. Holtz, T. Rohwedder, and R. Schneider. On Manifolds of Tensors of FixedTT-Rank. Preprint 61, DFG-SPP 1324, September 2010.

[62] J. Ballani, L. Grasedyck, and M. Kluge. Black Box Approximation of Tensors inHierarchical Tucker Format. Preprint 62, DFG-SPP 1324, October 2010.

[63] M. Hansen. On Tensor Products of Quasi-Banach Spaces. Preprint 63, DFG-SPP1324, October 2010.

[64] S. Dahlke, G. Steidl, and G. Teschke. Shearlet Coorbit Spaces: Compactly Sup-ported Analyzing Shearlets, Traces and Embeddings. Preprint 64, DFG-SPP 1324,October 2010.

[65] W. Hackbusch. Tensorisation of Vectors and their Efficient Convolution.Preprint 65, DFG-SPP 1324, November 2010.

[66] P. A. Cioica, S. Dahlke, S. Kinzel, F. Lindner, T. Raasch, K. Ritter, and R. L.Schilling. Spatial Besov Regularity for Stochastic Partial Differential Equationson Lipschitz Domains. Preprint 66, DFG-SPP 1324, November 2010.

[67] E. Novak and H. Woźniakowski. On the Power of Function Values for the Ap-proximation Problem in Various Settings. Preprint 67, DFG-SPP 1324, November2010.

[68] A. Hinrichs, E. Novak, and H. Woźniakowski. The Curse of Dimensionality forMonotone and Convex Functions of Many Variables. Preprint 68, DFG-SPP 1324,November 2010.

[69] G. Kutyniok and W.-Q Lim. Image Separation Using Shearlets. Preprint 69,DFG-SPP 1324, November 2010.

[70] B. Jin and P. Maass. An Analysis of Electrical Impedance Tomography withApplications to Tikhonov Regularization. Preprint 70, DFG-SPP 1324, December2010.

[71] S. Holtz, T. Rohwedder, and R. Schneider. The Alternating Linear Scheme forTensor Optimisation in the TT Format. Preprint 71, DFG-SPP 1324, December2010.

[72] T. Müller-Gronbach and K. Ritter. A Local Refinement Strategy for ConstructiveQuantization of Scalar SDEs. Preprint 72, DFG-SPP 1324, December 2010.

[73] T. Rohwedder and R. Schneider. An Analysis for the DIIS Acceleration Methodused in Quantum Chemistry Calculations. Preprint 73, DFG-SPP 1324, December2010.

[74] C. Bender and J. Steiner. Least-Squares Monte Carlo for Backward SDEs.Preprint 74, DFG-SPP 1324, December 2010.

[75] C. Bender. Primal and Dual Pricing of Multiple Exercise Options in ContinuousTime. Preprint 75, DFG-SPP 1324, December 2010.

[76] H. Harbrecht, M. Peters, and R. Schneider. On the Low-rank Approximation bythe Pivoted Cholesky Decomposition. Preprint 76, DFG-SPP 1324, December2010.

[77] P. A. Cioica, S. Dahlke, N. Döhring, S. Kinzel, F. Lindner, T. Raasch, K. Ritter,and R. L. Schilling. Adaptive Wavelet Methods for Elliptic Stochastic PartialDifferential Equations. Preprint 77, DFG-SPP 1324, January 2011.

[78] G. Plonka, S. Tenorth, and A. Iske. Optimal Representation of Piecewise HölderSmooth Bivariate Functions by the Easy Path Wavelet Transform. Preprint 78,DFG-SPP 1324, January 2011.

[79] A. Mugler and H.-J. Starkloff. On Elliptic Partial Differential Equations withRandom Coefficients. Preprint 79, DFG-SPP 1324, January 2011.

[80] T. Müller-Gronbach, K. Ritter, and L. Yaroslavtseva. A Derandomization of theEuler Scheme for Scalar Stochastic Differential Equations. Preprint 80, DFG-SPP1324, January 2011.

[81] W. Dahmen, C. Huang, C. Schwab, and G. Welper. Adaptive Petrov-Galerkinmethods for first order transport equations. Preprint 81, DFG-SPP 1324, January2011.

[82] K. Grella and C. Schwab. Sparse Tensor Spherical Harmonics Approximation inRadiative Transfer. Preprint 82, DFG-SPP 1324, January 2011.

[83] D.A. Lorenz, S. Schiffler, and D. Trede. Beyond Convergence Rates: Exact In-version With Tikhonov Regularization With Sparsity Constraints. Preprint 83,DFG-SPP 1324, January 2011.

[84] S. Dereich, M. Scheutzow, and R. Schottstedt. Constructive quantization: Ap-proximation by empirical measures. Preprint 84, DFG-SPP 1324, January 2011.

[85] S. Dahlke and W. Sickel. On Besov Regularity of Solutions to Nonlinear EllipticPartial Differential Equations. Preprint 85, DFG-SPP 1324, January 2011.

[86] S. Dahlke, U. Friedrich, P. Maass, T. Raasch, and R.A. Ressel. An adaptivewavelet method for parameter identification problems in parabolic partial differ-ential equations. Preprint 86, DFG-SPP 1324, January 2011.

[87] A. Cohen, W. Dahmen, and G. Welper. Adaptivity and Variational Stabilizationfor Convection-Diffusion Equations. Preprint 87, DFG-SPP 1324, January 2011.

[88] T. Jahnke. On Reduced Models for the Chemical Master Equation. Preprint 88,DFG-SPP 1324, January 2011.

[89] P. Binev, W. Dahmen, R. DeVore, P. Lamby, D. Savu, and R. Sharpley. Com-pressed Sensing and Electron Microscopy. Preprint 89, DFG-SPP 1324, March2011.

[90] P. Binev, F. Blanco-Silva, D. Blom, W. Dahmen, P. Lamby, R. Sharpley, andT. Vogt. High Quality Image Formation by Nonlocal Means Applied to High-Angle Annular Dark Field Scanning Transmission Electron Microscopy (HAADF-STEM). Preprint 90, DFG-SPP 1324, March 2011.

[91] R. A. Ressel. A Parameter Identification Problem for a Nonlinear Parabolic Dif-ferential Equation. Preprint 91, DFG-SPP 1324, May 2011.

[92] G. Kutyniok. Data Separation by Sparse Representations. Preprint 92, DFG-SPP1324, May 2011.

[93] M. A. Davenport, M. F. Duarte, Y. C. Eldar, and G. Kutyniok. Introduction toCompressed Sensing. Preprint 93, DFG-SPP 1324, May 2011.

[94] H.-C. Kreusler and H. Yserentant. The Mixed Regularity of Electronic WaveFunctions in Fractional Order and Weighted Sobolev Spaces. Preprint 94, DFG-SPP 1324, June 2011.

[95] E. Ullmann, H. C. Elman, and O. G. Ernst. Efficient Iterative Solvers forStochastic Galerkin Discretizations of Log-Transformed Random Diffusion Prob-lems. Preprint 95, DFG-SPP 1324, June 2011.

[96] S. Kunis and I. Melzer. On the Butterfly Sparse Fourier Transform. Preprint 96,DFG-SPP 1324, June 2011.

[97] T. Rohwedder. The Continuous Coupled Cluster Formulation for the ElectronicSchrödinger Equation. Preprint 97, DFG-SPP 1324, June 2011.

[98] T. Rohwedder and R. Schneider. Error Estimates for the Coupled Cluster Method.Preprint 98, DFG-SPP 1324, June 2011.

[99] P. A. Cioica and S. Dahlke. Spatial Besov Regularity for Semilinear StochasticPartial Differential Equations on Bounded Lipschitz Domains. Preprint 99, DFG-SPP 1324, July 2011.

[100] L. Grasedyck and W. Hackbusch. An Introduction to Hierarchical (H-) Rank andTT-Rank of Tensors with Examples. Preprint 100, DFG-SPP 1324, August 2011.

[101] N. Chegini, S. Dahlke, U. Friedrich, and R. Stevenson. Piecewise Tensor ProductWavelet Bases by Extensions and Approximation Rates. Preprint 101, DFG-SPP1324, September 2011.

[102] S. Dahlke, P. Oswald, and T. Raasch. A Note on Quarkonial Systems and Multi-level Partition of Unity Methods. Preprint 102, DFG-SPP 1324, September 2011.

[103] A. Uschmajew. Local Convergence of the Alternating Least Squares AlgorithmFor Canonical Tensor Approximation. Preprint 103, DFG-SPP 1324, September2011.

[104] S. Kvaal. Multiconfigurational time-dependent Hartree method for describing par-ticle loss due to absorbing boundary conditions. Preprint 104, DFG-SPP 1324,September 2011.

[105] M. Guillemard and A. Iske. On Groupoid C*-Algebras, Persistent Homology andTime-Frequency Analysis. Preprint 105, DFG-SPP 1324, September 2011.

[106] A. Hinrichs, E. Novak, and H. Woźniakowski. Discontinuous information in theworst case and randomized settings. Preprint 106, DFG-SPP 1324, September2011.

[107] M. Espig, W. Hackbusch, A. Litvinenko, H. Matthies, and E. Zander. EfficientAnalysis of High Dimensional Data in Tensor Formats. Preprint 107, DFG-SPP1324, September 2011.

[108] M. Espig, W. Hackbusch, S. Handschuh, and R. Schneider. Optimization Problemsin Contracted Tensor Networks. Preprint 108, DFG-SPP 1324, October 2011.

[109] S. Dereich, T. Müller-Gronbach, and K. Ritter. On the Complexity of ComputingQuadrature Formulas for SDEs. Preprint 109, DFG-SPP 1324, October 2011.

[110] D. Belomestny. Solving optimal stopping problems by empirical dual optimizationand penalization. Preprint 110, DFG-SPP 1324, November 2011.

[111] D. Belomestny and J. Schoenmakers. Multilevel dual approach for pricing Amer-ican style derivatives. Preprint 111, DFG-SPP 1324, November 2011.

[112] T. Rohwedder and A. Uschmajew. Local convergence of alternating schemes foroptimization of convex problems in the TT format. Preprint 112, DFG-SPP 1324,December 2011.

[113] T. Görner, R. Hielscher, and S. Kunis. Efficient and accurate computation ofspherical mean values at scattered center points. Preprint 113, DFG-SPP 1324,December 2011.

[114] Y. Dong, T. Görner, and S. Kunis. An iterative reconstruction scheme for pho-toacoustic imaging. Preprint 114, DFG-SPP 1324, December 2011.

[115] L. Kämmerer. Reconstructing hyperbolic cross trigonometric polynomials by sam-pling along generated sets. Preprint 115, DFG-SPP 1324, February 2012.

[116] H. Chen and R. Schneider. Numerical analysis of augmented plane waves methodsfor full-potential electronic structure calculations. Preprint 116, DFG-SPP 1324,February 2012.

[117] J. Ma, G. Plonka, and M.Y. Hussaini. Compressive Video Sampling with Ap-proximate Message Passing Decoding. Preprint 117, DFG-SPP 1324, February2012.

[118] D. Heinen and G. Plonka. Wavelet shrinkage on paths for scattered data denoising.Preprint 118, DFG-SPP 1324, February 2012.

[119] T. Jahnke and M. Kreim. Error bound for piecewise deterministic processes mod-eling stochastic reaction systems. Preprint 119, DFG-SPP 1324, March 2012.

[120] C. Bender and J. Steiner. A-posteriori estimates for backward SDEs. Preprint120, DFG-SPP 1324, April 2012.

[121] M. Espig, W. Hackbusch, A. Litvinenkoy, H.G. Matthiesy, and P. Wähnert. Eff-cient low-rank approximation of the stochastic Galerkin matrix in tensor formats.Preprint 121, DFG-SPP 1324, May 2012.

[122] O. Bokanowski, J. Garcke, M. Griebel, and I. Klompmaker. An adaptive sparsegrid semi-Lagrangian scheme for first order Hamilton-Jacobi Bellman equations.Preprint 122, DFG-SPP 1324, June 2012.

[123] A. Mugler and H.-J. Starkloff. On the convergence of the stochastic Galerkinmethod for random elliptic partial differential equations. Preprint 123, DFG-SPP1324, June 2012.

[124] P.A. Cioica, S. Dahlke, N. Döhring, U. Friedrich, S. Kinzel, F. Lindner, T. Raasch,K. Ritter, and R.L. Schilling. On the convergence analysis of Rothe’s method.Preprint 124, DFG-SPP 1324, July 2012.

[125] P. Binev, A. Cohen, W. Dahmen, and R. DeVore. Classification Algorithms usingAdaptive Partitioning. Preprint 125, DFG-SPP 1324, July 2012.

[126] C. Lubich, T. Rohwedder, R. Schneider, and B. Vandereycken. Dynamical approx-imation of hierarchical Tucker and Tensor-Train tensors. Preprint 126, DFG-SPP1324, July 2012.

[127] M. Kovács, S. Larsson, and K. Urban. On Wavelet-Galerkin methods for semilinearparabolic equations with additive noise. Preprint 127, DFG-SPP 1324, August2012.

[128] M. Bachmayr, H. Chen, and R. Schneider. Numerical analysis of Gaussian ap-proximations in quantum chemistry. Preprint 128, DFG-SPP 1324, August 2012.

[129] D. Rudolf. Explicit error bounds for Markov chain Monte Carlo. Preprint 129,DFG-SPP 1324, August 2012.

[130] P.A. Cioica, K.-H. Kim, K. Lee, and F. Lindner. On the Lq(Lp)-regularity andBesov smoothness of stochastic parabolic equations on bounded Lipschitz domains.Preprint 130, DFG-SPP 1324, December 2012.

[131] M. Hansen. n−term Approximation Rates and Besov Regularity for Elliptic PDEson Polyhedral Domains. Preprint 131, DFG-SPP 1324, December 2012.

[132] R. E. Bank and H. Yserentant. On the H1 -stability of the L2 -projection ontofinite element spaces. Preprint 132, DFG-SPP 1324, December 2012.

[133] M. Gnewuch, S. Mayer, and K. Ritter. On Weighted Hilbert Spaces and Inte-gration of Functions of Infinitely Many Variables. Preprint 133, DFG-SPP 1324,December 2012.

[134] D. Crisan, J. Diehl, P.K. Friz, and H. Oberhauser. Robust Filtering: CorrelatedNoise and Multidimensional Observation. Preprint 134, DFG-SPP 1324, January2013.

[135] Wolfgang Dahmen, Christian Plesken, and Gerrit Welper. Double Greedy Algo-rithms: Reduced Basis Methods for Transport Dominated Problems. Preprint135, DFG-SPP 1324, February 2013.

[136] Aicke Hinrichs, Erich Novak, Mario Ullrich, and Henryk Wozniakowski. The Curseof Dimensionality for Numerical Integration of Smooth Functions. Preprint 136,DFG-SPP 1324, February 2013.

[137] Markus Bachmayr, Wolfgang Dahmen, Ronald DeVore, and Lars Grasedyck. Ap-proximation of High-Dimensional Rank One Tensors. Preprint 137, DFG-SPP1324, March 2013.

[138] Markus Bachmayr and Wolfgang Dahmen. Adaptive Near-Optimal Rank TensorApproximation for High-Dimensional Operator Equations. Preprint 138, DFG-SPP 1324, April 2013.

[139] Felix Lindner. Singular Behavior of the Solution to the Stochastic Heat Equationon a Polygonal Domain. Preprint 139, DFG-SPP 1324, May 2013.

[140] Stephan Dahlke, Dominik Lellek, Shiu Hong Lui, and Rob Stevenson. AdaptiveWavelet Schwarz Methods for the Navier-Stokes Equation. Preprint 140, DFG-SPP 1324, May 2013.

[141] Jonas Ballani and Lars Grasedyck. Tree Adaptive Approximation in the Hierar-chical Tensor Format. Preprint 141, DFG-SPP 1324, June 2013.