Complex moment-based method with nonlineartransformation for computing large and sparse

interior singular triplets

Akira Imakura1,* and Tetsuya Sakurai1

1University of Tsukuba, Japan

∗Email : imakura@cs.tsukuba.ac.jp

Abstract

This paper considers computing interior singular triplets corresponding to the singu-lar values in some interval. Based on the concept of the complex moment-based paralleleigensolvers, in this paper, we propose a novel complex moment-based method with anonlinear transformation. We also analyse the error bounds of the proposed method andprovide some practical techniques. Numerical experiments indicate that the proposedcomplex moment-based method with the nonlinear transformation computes accuratesingular triplets for both exterior and interior problems within small computation timecompared with existing methods.

1 IntroductionGiven a rectangular matrix A ∈ Rm×n (m ≥ n), let

A = UΣV T =n∑i=1

σiuivTi

be a singular value decomposition of A, where σi are singular values and ui and vi arethe corresponding left and right singular vectors, respectively, and U = [u1,u2, . . . ,un],V = [v1,v2, . . . ,vn] and Σ = diag(σ1, σ2, . . . , σn). To compute partial singular tripletsspecifically corresponding to the larger part of singular values, there are several projection-type methods such as Golub-Kahan-Lanczos method [6], Jacobi-Davidson type method [8]and randomized SVD algorithm [22].

This paper considers computing interior singular triplets corresponding to the singularvalues in some interval,

(σi,ui,vi), σi ∈ Ω := [a, b], (1)

where 0 ≤ a < b. One of the simplest ideas to compute (1) is to apply some eigensolver forsolving the corresponding symmetric eigenvalue problems,

ATAv = σ2v or AATu = σ2u, σ ∈ Ω = [a, b]. (2)

One possible choice for solving interior eigenvalue problem (2) is complex moment-basedparallel eigensolvers first proposed in [26] that are one of the hottest parallel methods forsolving interior eigenvalue problems. However, as shown in Section 4, this simple strategydoes not work well in some situation due to the numerical instability.

arX

iv:2

109.

1365

5v1

[m

ath.

NA

] 2

8 Se

p 20

21

Based on the concept of the complex moment-based eigensolvers, in this paper, we pro-pose a novel complex moment-based method to compute interior singular triplets (1). Fromthe analysis of error bounds, we also show that the accuracy of the proposed method canbe improved via a nonlinear problem although the target problem is a linear singular valueproblem. Some practical techniques are also provided.

The remainder of this paper is organized as follows. In Section 2, we briefly introducethe complex moment-based parallel eigensolvers. In Section 3, we propose a novel complexmoment-based method for computing interior singular triplets and analyse its error bound.Here, we also propose an improvement technique using a nonlinear transformation. Numeri-cal results are reported in Section 4. Section 5 concludes the paper.

Throughout the paper, the following notations are used. We define the range space ofthe matrix V = [v1,v2, . . . ,vL] by R(V ) := spanv1,v2, . . . ,vL. We also use MATLABnotations.

2 Complex moment-based parallel eigensolversThe proposed method in this paper is based on the concept of the complex moment-basedparallel eigensolvers first proposed in [26] by Sakurai and Sugiura. Therefore, here, webriefly introduce the basic concepts of the complex moment-based eigensolvers for solvinginterior generalized eigenvalue problems of the form:

Axi = λiBxi, A,B ∈ Cn×n, xi ∈ Cn \ 0, λi ∈ Ω ⊂ C,

where zB − A is non-singular on a boundary Γ of the target region Ω.The complex moment-based eigensolvers construct a special subspace using contour in-

tegral:

SΩ = R(S), S = [S0, S1, . . . , SM−1], Sk :=1

2πi

∮Γ

zk(zB − A)−1BVindz, (3)

where L,M ∈ N+ are the input parameters and Vin ∈ Cn×L is an input matrix. For thissubspace SΩ, we have the following theorem; see e.g., [13].

Theorem 1. The complex moment-based subspace SΩ is equivalent to an invariant subspacewith respect to the eigenvectors corresponding to the eigenvalues in a given region Ω ⊂ C,that is,

SΩ = spanxi|λi ∈ Ω,

if and only if rank(S) = d, where d is the number of target eigenvalues.

Based on this theorem, complex moment-based eigensolvers are mathematically designedon projection methods [13]. Practical algorithms are derived by approximating the contourintegral (3) using the numerical integration rule:

Sk :=N∑j=1

ωjzkj (zjB − A)−1BV, (4)

where zj is a quadrature point and ωj is its corresponding weight.

2

The most time-consuming part of using complex moment-based eigensolvers involvessolving linear systems (4) at each quadrature point. Since these linear systems can be inde-pendently solved, the complex moment-based eigensolvers have a good scalability that wasdemonstrated in previous research [18, 19].

Thanks to the high parallel efficiency, complex moment-based eigensolvers have attractedconsiderable attention. Currently, there are several methods including direct extensions ofSakurai and Sugiura’s approach [9–11,13,14,16,27], the FEAST eigensolver [23] developedby Polizzi and its improvements [7, 19, 20, 29]. High-performance parallel software basedon the complex moment-based eigensolvers have been developed [5, 33]. Complex moment-based machine learning algorithms have also been developed [15, 31]. For details of thesemethods, refer to the study by [25] and the references therein.

3 Complex moment-based method for computing interiorsingular triplets

Herein, we propose a complex moment-based method for computing interior singular triplets(1) and analyse its error bound. Based on the analysis, we propose an improvement techniqueusing a nonlinear transformation to improve accuracy of the proposed method. Some practicaltechniques are also provided.

3.1 Derivation of the proposed methodBased on the concept of the complex moment-based parallel eigensolvers, now, we have thefollowing theorem.

Theorem 2. Let L,M ∈ N+ be the input parameters and Vin ∈ Rn×L be an input matrix. Wedefine S ∈ Rn×LM and Sk ∈ Rn×L as follows:

S := [S0, S1, . . . , SM−1], Sk :=1

2πi

∮Γ

zk(zI − ATA)−1Vindz, (5)

where Γ is a positively oriented Jordan curve around [a2, b2]. Then, the subspaces R(AS)and R(S) are equivalent to subspaces with respect to the left and right singular vectorscorresponding to the singular values in a given interval Ω = [a, b], i.e.,

R(AS) = spanui|σi ∈ Ω = [a, b], and R(S) = spanvi|σi ∈ Ω = [a, b],

if and only if rank(S) = t, where t is the number of target singular values.

Proof. Using the singular value decomposition of A, A = UΣV T, and Cauchy’s integralformula, the matrix Sk can be decomposed as

Sk =1

2πi

∮Γ

n∑i=1

zk

z − σ2i

vivTi Vindz =

∑σi∈Ω

(σ2i )kviv

Ti Vin,

that proves Theorem 2.

3

This theorem denotes that the target singular triplets (1) can be obtained by some pro-jection method with R(AS) and/or R(S) constructed by contour integral (5). In practice,the contour integral (5) is approximated by a numerical integration rule such as the N -pointtrapezoidal rule, as follows:

S := [S0, S1, . . . , SM−1], Sk :=N∑j=1

ωjzkj (zjI − ATA)−1Vin,

where (zj, ωj), j = 1, 2, . . . , N , are the quadrature points and the corresponding weights,respectively. Then, the approximate singular triplets are computed by a projection method.Here, we consider using two-sided projection method with subspaces R(AS) and R(S).Note that one can also consider one-sided projection method.

Let U and V be the orthogonal matrices whose columns are orthonormal basis ofR(AS)

andR(S), respectively. From the definition of the subspaceR(AS), the matrix U is obtainedby a QR factorization of AV ,

AV = UB. (6)

In a two-sided projection method, singular triplets are approximated as

(σi,ui,vi) ≈ (σi, ui, vi) = (φi, Upi, V qi),

and set P = [p1,p2, . . . ,pLM ], Q = [q1, q2, . . . , qLM ] and Φ = diag(φ1, φ2, . . . , φLM).Based on the Galerkin condition, the residual R = A − (UP )Φ(V Q)T is orthogonalized tothe subspaces R(AS) and R(S), that is UTRV = O. From (6), we have UTAV = B, thenthe target singular triplets can be approximated by using a singular value decomposition ofthe matrix B,

B = PΦQ =∑σi∈Ω

φipiqTi . (7)

To improve the accuracy, we can use an iteration technique. The basic concept is that thematrix S(`−1)

0 is iteratively calculated, from the initial matrix S(0)0 = Vin as follows:

S(ν)0 :=

N∑j=1

ωj(zjI − ATA)−1S(ν−1)0 , ν = 1, 2, . . . , `− 1. (8)

Then, S(`) is constructed from S(`−1)0 by

S(`) := [S(`)0 , S

(`)1 , . . . , S

(`)M−1], S

(`)k :=

N∑j=1

ωjzkj (zjI − ATA)−1S

(`−1)0 . (9)

Additionally, for improving the numerical stability, we use a low-rank approximation with athreshold δ based on the singular value decomposition of S(`):

S(`) = [US1, US2]

[ΣS1 OO ΣS2

] [WTS1

WTS2

]≈ US1ΣS1W

TS1, (10)

where ΣS1 is a diagonal matrix whose diagonal entries are the larger part of the singularvalues, and the columns of US1,WS1 are the corresponding singular vectors. Then, US1 isused for projection method instead of S.

4

Algorithm 1 A block SS–SVD method

Input: L,M,N, ` ∈ N+, δ ∈ R, Vin ∈ Rn×L, (zj, ωj) for j = 1, 2, . . . , N/2Output: Approximate singular triplet (σi, ui, vi) for i = 1, 2, . . . , t

1: Compute S(ν)0 = 2

∑N/2j=1 Re

(ωj(zjI − ATA)−1S

(ν−1)0

)for ν = 1, 2, . . . , `− 1

2: Compute S(`)k = 2

∑N/2j=1 Re

(ωjz

kj (zjI − ATA)−1S

(`−1)0

)for k = 0, 1, . . . ,M − 1, and

set S(`) = [S(`)0 , S

(`)1 , . . . , S

(`)M−1]

3: Compute low-rank approx. of S(`) using the threshold δ:S(`) = [US1, US2][ΣS1, O;O,ΣS2][WS1,WS2]T ≈ US1ΣS1W

TS1, and set V = US1

4: Compute QR factorization of AUS1: [U , B] = qr(AV )

5: Compute singular triplets (φi,pi, qi) of the matrixB and set (σi, ui, vi) = (φi, Upi, V qi)

Because of the symmetric property of ATA, if quadrature points and the correspondingweights are symmetric about the real axis,

(zj, ωj) = (zj+N/2, ωj+N/2), j = 1, 2, . . . , N/2,

we can reduce the number of linear systems as

Sk = 2

N/2∑j=1

Re(ωjz

kj (zjI − ATA)−1Vin

).

The practical algorithm of the proposed method is shown in Algorithm 1. One of the mosttime-consuming part of the complex moment-based method involves solving linear systemsat each quadrature point in (8) and (9). However, as these linear systems can be independentlysolved, the proposed method is expected to exhibit good scalability in the same manner asthe complex moment-based parallel eigensolvers.

3.2 Error analysis of the proposed methodAssume that (zj, ωj) satisfy

N∑j=1

ωjzkj

= 0, k = 0, 1, . . . , N − 26= 0, k = −1

. (11)

Here, we have the following proposition for S; see, e.g., [12].

Proposition 1. Let (zj, ωj) satisfy (11), then we have the following relationship,

Sk =N∑j=1

n∑i=1

ωjzkj

zj − σ2i

vivTi Vin =

n∑i=1

σ2ki

(N∑j=1

ωjzj − σ2

i

)viv

Ti Vin = (ATA)kS0,

for k = 0, 1, . . . , N − 1.

Let f(σi) be a filter function

f(σi) :=N∑j=1

ωjzj − σ2

i

,

5

commonly used in the analyses of some eigensolvers [7,12,13,28,29]. Then, the matrix S(`)k

can be rewritten as

S(`)k =

n∑i=1

(N∑j=1

ωjzkj

zj − σ2i

)(N∑j=1

ωjzj − σ2

i

)`−1

vivTi Vin

=n∑i=1

(σ2i )k

(N∑j=1

ωjzj − σ2

i

)`

vivTi Vin

=n∑i=1

σ2ki (f(σi))

`vivTi Vin

= V Σ2kV TV (f(Σ))`V TVin,

where f(Σ) = diag(f(σ1), f(σ2), . . . , f(σn)) that privides

S(`) = F `K (12)

withF = V f(Σ2)V T, K = [Vin, (A

TA)Vin, . . . , (ATA)M−1Vin].

Using (12), we have the following theorem for the error bound of the proposed method inthe same manner as an error analysis of the subspace iteration method, see, e.g., Lemma 6.2.1of [4] and Theorem 5.2 of [24].

Theorem 3. Let (σi,ui,vi) be exact singular triplets of A. Assume that f(σi) are orderedby decreasing magnitude |f(σi)| ≥ |f(σi+1)|. Define P(`)

U and P(`)V as orthogonal projectors

onto the subspaces R(AS(`)) and R(S(`)), respectively. We also define PLM as the spectralprojector with an invariant subspace spanv1,v2, . . . ,vLM. Assume that the matrix PLMKis full rank. Then, for each right singular vector vi, i = 1, 2, . . . , LM , there exists a uniquevector si ∈ R(K) such that PLMsi = vi. Here, we have

‖(I − P(`)U )ui‖2 ≤ αiβi

∣∣∣∣f(σLM+1)

f(σi)

∣∣∣∣` , i = 1, 2, . . . , LM,

and

‖(I − P(`)V )vi‖2 ≤ βi

∣∣∣∣f(σLM+1)

f(σi)

∣∣∣∣` , i = 1, 2, . . . , LM,

where αi = maxj≥LM+1 σj/σi and βi = ‖vi − si‖2.

Proof. Since PLMK is full rank, there exists a unique vector si ∈ R(K) as

vi =LM∑j=1

αjPLMkj = PLM

(LM∑j=1

αjkj

)=: PLMsi,

where K = [k1,k2, . . . ,kLM ]. Then, using wi = (I − PLM)si, we have

si = PLMsi + (I − PLM)si = vi + wi. (13)

6

Let yi = (1/σi)A(1/f(σi))`F `si ∈ R(AS(`)). Then, multiplying the matrix (1/σi)A(1/f(σi))

`F `

to (13) from the left-side and considering ui = (1/σi)A(1/f(σi))`F `vi and wi = (I −

PLM)si = P⊥LMsi, we have

yi − ui =1

σiA

(1

f(σi)F

)`wi =

1

σiAP⊥LM

(1

f(σi)FP⊥LM

)`wi,

that provides

‖yi − ui‖2 ≤1

σi‖AP⊥LM‖2

∣∣∣∣∣∣∣∣∣∣(

1

f(σi)FP⊥LM

)`∣∣∣∣∣∣∣∣∣∣2

‖wi‖2.

Here, using the relationship

minyi∈R(AS(`))

‖yi − ui‖2 = ‖(I − P(`)U )ui‖2,

we thus have

‖(I − P(`)U )ui‖2 ≤ ‖yi − ui‖2 ≤ αiβi

∣∣∣∣f(σLM+1)

f(σi)

∣∣∣∣` .In the same manner, letting zi = (1/f(σi))

`F `si ∈ R(S(`)) and considering vi =(1/f(σi))

`F `vi, we have

‖(I − P(`)V )vi‖2 ≤ ‖zi − vi‖2 ≤ βi

∣∣∣∣f(σLM+1)

f(σi)

∣∣∣∣` ,that proves Theorem 3.

Theorem 3 indicates that the accuracy of the proposed method depends on the subspacedimension LM . Given a sufficiently large subspace, i.e.,

|f(σLM+1)/f(σi)|` ≈ 0,

the target singular triplets can be obtained accurately, even if some singular values existoutside but near the region.

3.3 An improvement technique using a nonlinear transformationTheorem 3 also indicates that if there is a cluster of singular values outside but near the region,then, to obtain accurate singular triplets, we have to use huge LM that takes into account thenumber of the clustered singular values even though these are not the target. This becomeshuge computational costs. Such a situation happens in the case that the singular values areuniformly distributed on the logarithmic scale.

To overcome this difficulty, in this paper, inspired by complex moment-based nonlineareigensolvers [1–3, 17, 30, 32], we consider introducing a nonlinear transformation, z = g(t),with an analytic monotonic increasing function g. Then, we reset

Sk =1

2πi

∮Γt

tk(g(t)I − ATA)−1Vindt,

7

where Γt is a Jordan curve around [g−1(a2), g−1(b2)]. Note that this also holds Theorem 2.The matrix Sk is approximated by using contour integral as

Sk =N∑j=1

ψjtkj (g(tj)I − ATA)−1Vin,

where (tj, ψj), j = 1, 2, . . . , N are the quadrature points and the corresponding weights,respectively.

Although Proposition 1 usually does not hold in the case using the nonlinear transform,since we have

Sk =1

2πi

∮Γ

n∑i=1

tk

g(t)− σ2i

vivTi Vindt

=∑σi∈Ω

(g−1(σ2i ))

kvivTi Vin

= (g−1(ATA))kS0,

we expect

Sk =N∑j=1

n∑i=1

ψjtkj

g(tj)− σ2i

vivTi Vin

≈n∑i=1

(g−1(σi))2k

(N∑j=1

ψjg(tj)− σ2

i

)viv

Ti Vin

= (g−1(ATA))kS0. (14)

Therefore, letting fg(σ) be a filter function

fg(σi) :=N∑j=1

ψjg(tj)− σ2

i

,

then S(`)k can be rewritten as

S(`)k ≈ V (g−1(Σ))2kV TV (fg(Σ))`V TVin,

that providesS(`) ≈ F `

gKg

withFg = V fg(Σ)V T, Kg = [Vin, g

−1(ATA)Vin, . . . , (g−1(ATA))M−1Vin].

Thus, we have approximately the same error bounds as Theorem 3 with fg(σ) instead of f(σ)as the filter function.

To set the function g as |fg(σLM+1)/fg(σi)| |f(σLM+1)/f(σi)|, the obtained accuracyis expected to be improved. For example, if the singular values are uniformly distributed onthe logarithmic scale, g(t) = exp(t) is a good choice to achieve small |fg(σLM+1)/fg(σi)|.The algorithm of the proposed method with a nonlinear transform is shown in Algorithm 2.

8

Algorithm 2 A block SS–SVD method with nonlinear transformation

Input: L,M,N, ` ∈ N+, δ ∈ R, Vin ∈ Rn×L, (tj, ψj) for j = 1, 2, . . . , N/2Output: Approximate singular triplet (σi, ui, vi) for i = 1, 2, . . . , t

1: Compute S(ν)0 = 2

∑N/2j=1 Re

(ψj(g(tj)I − ATA)−1S

(ν−1)0

)for ν = 1, 2, . . . , `− 1

2: Compute S(`)k = 2

∑N/2j=1 Re

(ψjt

kj (g(tj)I − ATA)−1S

(`−1)0

)for k = 0, 1, . . . ,M − 1, and set S(`) = [S

(`)0 , S

(`)1 , . . . , S

(`)M−1]

3: Compute low-rank approx. of S(`) using the threshold δ:S(`) = [US1, US2][ΣS1, O;O,ΣS2][WS1,WS2]T ≈ US1ΣS1W

TS1, and set V = US1

4: Compute QR factorization of AUS1: [U , B] = qr(AV )

5: Compute singular triplets (φi,pi, qi) of the matrixB and set (σi, ui, vi) = (φi, Upi, V qi)

3.4 Practical techniquesHere, we introduce two practical techniques: an efficient residual norm computation and aspurious singular value detection.

3.4.1 Efficient residual norm computation

Computation of the residual norms for obtained singular triplets is computationally costlywhen the problem size is large. Here, we consider an efficient computation of the residual2-norm ‖ri‖2 = ‖ATui − σivi‖2.

Since the proposed method based on the Galerkin-type two-sided projection providesB = PΦQT (7) and AV = UB (6), we have AV Q = UPΦ, i.e., we have Avi = σiui for alli. Therefore, the residual 2-norm ‖ri‖2 can be replaced as

‖ri‖2 =

∣∣∣∣∣∣∣∣ 1

σiATAvi − σivi

∣∣∣∣∣∣∣∣2

=1

σi‖ATAvi − σ2

i vi‖2.

Let S(`)+ = [S

(`)1 , S

(`)2 , . . . , S

(`)M ]. For the case of Algorithm 1, from Proposition 1, we have

S(`)+ = (ATA)S(`). Therefore, the residual 2-norm ‖ri‖ is rewritten as

‖ri‖2 =1

σi‖ATAUS1qi − σ2

iUS1qi‖2

=1

σi

∥∥∥ATAS(`)VS1Σ−1S1qi − σ

2iUS1qi

∥∥∥2

=1

σi

∥∥∥S(`)+ VS1Σ−1

S1qi − σ2iUS1qi

∥∥∥2, (15)

that achieves an efficient residual 2-norm computation without a matrix product for A, sincethe matrix S(`)

+ is efficiently obtained by contour integral as well as (8) and (9).Next, we consider the case of using the nonlinear transformation. Assuming that the

relative residual of symmetric eigenvalue problem is almost invariant to nonlinear transform,

‖ATAvi − σ2i vi‖2

‖ATA‖2

≈ ‖g−1(ATA)vi − g−1(σ2

i )vi‖2

‖g−1(ATA)‖2

,

9

we have

‖ri‖2 ≈1

σi‖g−1(ATA)vi − g−1(σ2

i )vi‖2‖ATA‖2

‖g−1(ATA)‖2

. (16)

Here, from (14), we have S(`)+ ≈ g−1(ATA)S(`). Then, the 1st part of (16) can be approxi-

mated as

1

σi‖g−1(ATA)vi − g−1(σ2

i )vi‖2

=1

σi‖g−1(ATA)US1qi − g−1(σ2

i )US1qi‖2

=1

σi

∥∥∥g−1(ATA)S(`)VS1Σ−1S1qi − g

−1(σ2i )US1qi

∥∥∥2

≈ 1

σi

∥∥∥S(`)+ VS1Σ−1

S1qi − g−1(σ2

i )US1qi

∥∥∥2

=: ‖ri‖2.

Since the 2nd part of (16) is constant for i, we have

‖ATA‖2

‖g−1(ATA)‖2

≈ ‖ri′‖2

‖ri′‖2

, i′ ∈ 1, 2, . . . , t.

As a result, we can efficiently approximate all ‖ri‖2 by

‖ri‖2 ≈ µ‖ri‖2, µ =‖ri′‖2

‖ri′‖2

, (17)

with only one exact residual 2-norm ‖ri′‖2 = ‖ATui′ − σi′vi′‖2.

3.4.2 Spurious singular value detection

Similarly to other methods, the proposed method may compute spurious singular values in thetarget region. In the proposed method, both subspacesR(AUS1) andR(US1) are constructedfrom the matrix US1 obtained by a low-rank approximation (10) of S. Here, we focus on thislow-rank approximation and introduce a technique for spurious singular value detection.

Let US1 be split as US1 = [U(1)S1 , U

(2)S1 ], where U (1)

S1 and U(2)S1 correspond to large and

small singular values in ΣS1, respectively. Then, since V = US1, the approximation of rightsingular vectors vi = V qi is replaced as

vi = US1qi = [U(1)S1 , U

(2)S1 ]

[q

(1)i

q(2)i

].

From Theorem 2, we expect spanvi|σi ∈ Ω ≈ R(U(1)S1 ). Therefore, the magnitude of the

elements of q(1)i is expected to be much larger than that of q(2)

i .Based on this concept, we detect spurious singular values using the following index:

τi :=‖qi‖2

2

‖qi‖2Σ−1

S1

=qTi qi

qTi Σ−1

S1qi.

If τi is smaller than some constant ε, we treat the eigenpair as a spurious singular values.

10

10-1810-1610-1410-1210-1010-810-610-410-2100

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4

|f(σ

)|, |f

g(σ)

|

σ

f(σ)fg(σ)

target region

(a) [a, b] = [0.8, 1.2].

10-1810-1610-1410-1210-1010-810-610-410-2100

10-7 10-6 10-5 10-4 10-3 10-2 10-1 100

|f(σ

)|, |f

g(σ)

|

σ

f(σ)fg(σ)

target region

(b) [a, b] = [10−3, 10−1].

Figure 1: The values of |f(σ)| and |fg(σ)|.

4 Numerical experimentsHere, we evaluate the performance of the proposed method (Algorithms 1 and 2). For Algo-rithm 2, we set g as z = g(t) = exp(t).

Let the quadrature points zj and tj be on an ellipse with center γ, major axis ρ and aspectratio α, i.e.,

zj = tj = γ + ρ (cos(θj) + αi sin(θj)) , θj =2π

N

(j − 1

2

), j = 1, 2, . . . , N.

The corresponding weights are set as

ωj =ρ

N(α cos(θj) + i sin(θj)) , ψj =

ρ

Nexp(tj) (α cos(θj) + i sin(θj)) .

We set (γ, ρ, α) = ((a2 + b2)/2, (b2 − a2)/2, 0.1) for Algorithm 1 and (γ, ρ, α) = (log(a) +log(b), log(b)− log(a), 0.1) for Algorithm 2.

In these numerical experiments, the algorithms were implemented in MATLAB R2019a.The input matrix Vin was set as a random matrix generated by the Mersenne Twister in MAT-LAB and each linear system was solved using the MATLAB command “\”.

4.1 Experiment I: filter functionHere, we compare the filter functions f(σ) and fg(σ) for two cases. Figure 1 shows that thevalues of the filter functions |f(σ)| and |fg(σ)| for [a, b] = [0.8, 1.2] and [10−3, 10−1].

As shown in Figure 1(a), in the case that the target region is around 1, there is no largedifference between the filter functions f(σ) and fg(σ). On the other hand, as shown in Fig-ure 1(b), in the case that the target region is cross to 0, there is large difference. For the regionof σ > b, both |f(σ)| and |fg(σ)| decrease with increasing σ, specifically |f(σ)| shows rapiddecreasing. For the region of 0 < σ < a, |f(σ)| shows large value |f(σ)| ≈ 0.5. Instead,|fg(σ)| drastically decrease. This result indicates that, a cluster of singular value in the region[0, 10−4] causes a negative effect on the accuracy of the SS-SVD based on f(σ), but not theaccuracy of the SS-SVD with nonlinear transform based on fg(σ).

4.2 Experiment II: model problemIn this subsection, we compare the accuracy of four methods:

• Naive: The block SS–CAA method [16] via the symmetric eigenvalue problem (2);

11

10-1810-1610-1410-1210-1010-810-610-410-2100

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4

|σ*

- σ|

/ σ

*

σ

f(σ)fg(σ)

NaiveNaive with NT

SS-SVDSS-SVD with NT

10-1810-1610-1410-1210-1010-810-610-410-2100

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4

rela

tive

resi

dual

2-n

orm

σ

f(σ)fg(σ)

NaiveNaive with NT

SS-SVDSS-SVD with NT

(a) The model problem 1 (18).

10-1810-1610-1410-1210-1010-810-610-410-2100

10-8 10-7 10-6 10-5 10-4 10-3 10-2 10-1 100

|σ*

- σ|

/ σ

*

σ

f(σ)fg(σ)

NaiveNaive with NT

SS-SVDSS-SVD with NT

10-1810-1610-1410-1210-1010-810-610-410-2100

10-8 10-7 10-6 10-5 10-4 10-3 10-2 10-1 100

rela

tive

resi

dual

2-n

orm

σ

f(σ)fg(σ)

NaiveNaive with NT

SS-SVDSS-SVD with NT

(b) The model problem 2 (19).

Figure 2: Relative error |σ∗i − σi|/σ∗i of singular values and residual 2-norm ‖ri‖2.

10-1810-1610-1410-1210-1010-810-610-410-2100

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4

rela

tive

resi

dual

2-n

orm

σ

f(σ)fg(σ)

SS-SVDApprox. (15)

SS-SVD with NTApprox. (17)

(a) The model problem 1 (18).

10-1810-1610-1410-1210-1010-810-610-410-2100

10-8 10-7 10-6 10-5 10-4 10-3 10-2 10-1 100

rela

tive

resi

dual

2-n

orm

σ

f(σ)fg(σ)

SS-SVDApprox. (15)

SS-SVD with NTApprox. (17)

(b) The model problem 2 (19).

Figure 3: Residual norm and its approximation.

• Naive with NT: Naive with the nonlinear trasnform;

• SS–SVD: The proposed method (Algorithm 1);

• SS–SVD with NT: The proposed method with the nonlinear trainsform (Algorithm 2),

using two small model problems of size m = 1000, n = 200. For the model problem 1, weset

Σ = diag(0.005, 0.015, . . . , 1.995) ∈ R200×200 (18)

such that singular values are uniformly distributed, and for the model problem 2, we set

Σ = diag(10−10.0, 10−9.95, . . . , 10−0.05) ∈ R200×200 (19)

such that singular values are uniformly distributed on the logarithmic scale. Then, the matrixA is constructed as A = UΣV T with orthogonal matrices U and V constructed from randommatrices.

For the model problem 1, we compute 40 singular triplets such that σi ∈ [0.8, 1.2] and forthe model problem 2, we compute 40 singular triplets such that σi ∈ [10−3, 10−1]. We set theparameters as (L,M,N, `, δ) = (20, 4, 32, 1, 10−20).

12

Table 1: The target region [a, b], the number t of the target singular triplets and parametersetting for the proposed method.

Exterior Interior

[a, b] t L M [a, b] t L M

[0.120, 1.01] 21 15 4 [0.060, 0.08] 17 15 4[0.080, 1.01] 40 30 4 [0.045, 0.08] 39 30 4[0.045, 1.01] 79 60 4 [0.030, 0.08] 85 60 4[0.025, 1.01] 153 120 4 [0.020, 0.08] 158 120 4

0

5

10

15

20

25

0 20 40 60 80 100 120 140 160

Tim

e [s

ec.]

The number t of the target singular triplets

svdssvd

SS-SVD for exteriorSS-SVD for interior

Figure 4: The computation time v.s. the number of the target singular triplets.

In Figure 2, we show relative error of singular value |σ∗i − σi|/σ∗i , where σ∗i is the ex-act singular value, and residual 2-norm ‖ri‖2 = ‖ATui − σivi‖2. In the case of the modelproblem 1 (18) whose singular values are uniformly distributed, as shown in Figure 2(a), allmethods show almost the same accuracy both for the relative error and residual 2-norm. Onthe other hand, in the case of the model problem 2 (19) whose singular values are uniformlydistributed on the logarithmic scale, as shown in Figure 2(b), the nonlinear transformationshows drastically improves the accuracy. Specifically, the SS-SVD with the nonlinear trans-formation shows the best results both for relative error and residual 2-norm.

In Figure 3, we show the residual 2-norm and its estimations (15) and (17) for the pro-posed methods. We observe that the both approximations well estimated the exact residual2-norms for both problems.

4.3 Experiment III: real-world problemWe evaluate the computation time of the proposed method compared with that of MATLABfunctions svd for computing all the singular triplets and svds for computing partial sin-gular triplets. We used a 60000 × 784 sparse matrix obtained from a 10-class classificationof handwritten digits (MNIST) [21]. The average nonzero elements per row is 149.9. Wenormalized the matrix as the largest singular value is 1. For svds, we computed the largest10, 20, . . . , 160 singular triplets. For the proposed method, the target interval, the number tof the target singular triplets and parameters L,M are shown in Table 1. Here, we fix M = 4and set L as LM ≈ 3t. We also set (N, `, δ) = (32, 1, 10−20).

The computation time of all methods are shown in Figure 4. The breakdown of the com-putation time and accuracy of the proposed method are also shown in Table 2. As shown in

13

Table 2: Breakdown of the computation time and accuracy of the proposed method.

Exterior

region Computation time [sec.] accuracy

[a, b] Steps 1–2 3 4 5 Total error residual

[0.120, 1.01] 0.809 0.002 0.144 0.006 0.962 1.67E-15 1.57E-13[0.080, 1.01] 0.842 0.005 0.299 0.017 1.146 1.70E-15 1.90E-14[0.045, 1.01] 0.886 0.015 0.792 0.049 1.693 2.94E-15 1.03E-14[0.025, 1.01] 0.997 0.054 1.847 0.178 2.898 2.48E-15 3.90E-15

Interior

region Computation time [sec.] accuracy

[a, b] Steps 1–2 3 4 5 Total error residual

[0.060, 0.08] 0.830 0.002 0.150 0.006 0.983 1.09E-15 8.26E-14[0.045, 0.08] 0.858 0.005 0.286 0.017 1.148 2.62E-15 8.99E-14[0.030, 0.08] 0.886 0.015 0.795 0.047 1.695 2.18E-15 5.02E-13[0.020, 0.08] 1.024 0.056 1.762 0.184 2.842 2.48E-15 3.99E-16

Figure 4, the computation time of svds increases with increasing the number t of the targetsingular values. Then, in this experiment, svd is faster than svds when t > 30. Instead,the proposed method is much faster than svds and svd for both exterior and interior cases,although the computation time, specifically Step 4, of the proposed method increases withincreasing t and L.

5 ConclusionsBased on the concept of the complex moment-based eigensolvers, in this paper, we proposeda novel complex moment-based method to compute interior singular triplets (1). We alsoanalysed error bounds of the proposed method and proposed an improvement technique us-ing a nonlinear transformation to improve accuracy of the proposed method. The proposedmethod has high parallel efficiency as the complex moment-based parallel eigensolvers. Fromthe numerical experiments, the proposed complex moment-based method with the nonlineartransformation can compute accurate singular triplets for both exterior and interior problems.The computation time of the proposed method is much faster than svds and svd.

In the future, we will evaluate the parallel performance of the proposed method for morelarge real-world problems.

References[1] J. Asakura, T. Sakurai, H. Tadano, T. Ikegami, K. Kimura, A numerical method for

nonlinear eigenvalue problems using contour integrals, JSIAM Letters 1 (2009) 52–55.

14

[2] J. Asakura, T. Sakurai, H. Tadano, T. Ikegami, K. Kimura, A numerical method forpolynomial eigenvalue problems using contour integral, Japan Journal of Industrial andApplied Mathematics 27 (1) (2010) 73–90.

[3] W.-J. Beyn, An integral method for solving nonlinear eigenvalue problems, Linear Al-gebra and its Applications 436 (10) (2012) 3839–3863.

[4] F. Chatelin, Eigenvalues of Matrices, Wiley, Chichester, 1993.

[5] FEAST Eigenvalue Solver, http://www.ecs.umass.edu/˜polizzi/feast/.

[6] G. H. Golub, C. F. Van Loan, Matrix Computations, 3rd ed., The Johns Hopkins Uni-versity Press, 1996.

[7] S. Guttel, E. Polizzi, P. T. P. Tang, G. Viaud, Zolotarev quadrature rules and load balanc-ing for the FEAST eigensolver, SIAM Journal on Scientific Computing 37 (4) (2015)A2100–A2122.

[8] M. E. Hochstenbach, A Jacobi–Davidson type SVD method, SIAM J. Sci. Comput. 23(2001) 606–628.

[9] T. Ikegami, T. Sakurai, Contour integral eigensolver for non-Hermitian systems: aRayleigh-Ritz-type approach, Taiwanese Journal of Mathematics (2010) 825–837.

[10] T. Ikegami, T. Sakurai, U. Nagashima, A filter diagonalization for generalized eigen-value problems based on the Sakurai–Sugiura projection method, Journal of Computa-tional and Applied Mathematics 233 (8) (2010) 1927–1936.

[11] A. Imakura, L. Du, T. Sakurai, A block Arnoldi-type contour integral spectral projectionmethod for solving generalized eigenvalue problems, Applied Mathematics Letters 32(2014) 22–27.

[12] A. Imakura, L. Du, T. Sakurai, Error bounds of Rayleigh–Ritz type contour integral-based eigensolver for solving generalized eigenvalue problems, Numer. Alg. 71 (2016)103–120.

[13] A. Imakura, L. Du, T. Sakurai, Relationships among contour integral-based methodsfor solving generalized eigenvalue problems, Japan Journal of Industrial and AppliedMathematics 33 (3) (2016) 721–750.

[14] A. Imakura, Y. Futamura, T. Sakurai, Structure-preserving technique in the block SS–Hankel method for solving Hermitian generalized eigenvalue problems, in: Interna-tional Conference on Parallel Processing and Applied Mathematics, Springer, 2017.

[15] A. Imakura, M. Matsuda, X. Ye, T. Sakurai, Complex moment-based supervised eigen-map for dimensionality reduction, in: Proceedings of the AAAI Conference on ArtificialIntelligence, vol. 33, 2019.

[16] A. Imakura, T. Sakurai, Block Krylov-type complex moment-based eigensolvers forsolving generalized eigenvalue problems, Numerical Algorithms 75 (2) (2017) 413–433.

15

[17] A. Imakura, T. Sakurai, Block SS-CAA: A complex moment-based parallel nonlineareigensolver using the block communication-avoiding Arnoldi procedure, Parallel Com-puting (2018) 34–48.

[18] S. Iwase, Y. Futamura, A. Imakura, T. Sakurai, T. Ono, Efficient and scalable calcula-tion of complex band structure using Sakurai-Sugiura method, in: Proceedings of theInternational Conference for High Performance Computing, Networking, Storage andAnalysis, ACM, 2017.

[19] J. Kestyn, V. Kalantzis, E. Polizzi, Y. Saad, PFEAST: a high performance sparse eigen-value solver using distributed-memory linear solvers, in: High Performance Computing,Networking, Storage and Analysis, SC16: International Conference for, IEEE, 2016.

[20] J. Kestyn, E. Polizzi, P. T. Peter Tang, FEAST eigensolver for non-hermitian problems,SIAM Journal on Scientific Computing 38 (5) (2016) S772–S799.

[21] Y. LeCun, The MNIST database of handwritten digits, http://yann. lecun.com/exdb/mnist/.

[22] N.Halko, P.G.Martinsson, J. A.Tropp, Finding structure with randomness: probabilisticalgorithms for constructing approximate matrix decompositions, SIAM Rev. 53 (2011)217–288.

[23] E. Polizzi, A density matrix-based algorithm for solving eigenvalue problems, Phys.Rev. B 79 (2009) 115112.

[24] Y. Saad, Numerical Methods for Large Eigenvalue Problems, 2nd ed., Manchester Uni-versity Press, 2011.

[25] T. Sakurai, Y. Futamura, A. Imakura, T. Imamura, Scalable eigen-analysis engine forlarge-scale eigenvalue problems, in: Sato M. (eds) Advanced Software Technologiesfor Post-Peta Scale Computing, Springer, Singapore, 2019.

[26] T. Sakurai, H. Sugiura, A projection method for generalized eigenvalue problems usingnumerical integration, Journal of computational and applied mathematics 159 (1) (2003)119–128.

[27] T. Sakurai, H. Tadano, CIRR: a Rayleigh-Ritz type method with counter integral forgeneralized eigenvalue problems, Hokkaido Math. J. 36 (2007) 745–757.

[28] G. Schofield, J. R. Chelikowsky, Y. Saad, A spectrum slicing method for the kohn-shamproblem, Comput. Phys. Commun. 183 (2012) 497–505.

[29] P. T. P. Tang, E. Polizzi, FEAST as a subspace iteration eigensolver accelerated byapproximate spectral projection, SIAM Journal on Matrix Analysis and Applications35 (2) (2014) 354–390.

[30] M. Van Barel, P. Kravanja, Nonlinear eigenvalue problems and contour integrals, Jour-nal of Computational and Applied Mathematics 292 (2016) 526–540.

16

[31] T. Yano, Y. Futamura, A. Imakura, T. Sakurai, Efficient implementation of a dimension-ality reduction method using a complex moment-based subspace, in: The InternationalConference on High Performance Computing in Asia-Pacific Region, HPC Asia 2021,2021.

[32] S. Yokota, T. Sakurai, A projection method for nonlinear eigenvalue problems usingcontour integrals, JSIAM Letters 5 (2013) 41–44.

[33] z-Pares: Parallel Eigenvalue Solver, http://zpares.cs.tsukuba.ac.jp/.

17