matlab assignments – lecture 1, fall 2016 › ... › assignments › all-matlab.pdf ·...

September 13, 2016

Matlab Assignments – Lecture 1, Fall 2016

A first step in the calculation of the eigenvalues of an n × n matrix A is to bring thematrix into a simpler shape via a similarity transformation

B = S−1AS

with S the transformation matrix (i.e., S is of the same size as A and non-singular).

Assignment 1.1. Prove that the matrices A and B have the same eigenvalues.

We first choose as transformation matrix

S = [v,Av,A2v, . . . ,An−1v ],

in which we take the vector v that has ones as its entries. The space spanned by thevectors v,Av,A2v, · · · ,An−1v is called the Krylov subspace generated by the matrixA and vector v. As you will see in the remainder of this course, this subspace playsan important role in most modern iterative methods for solving eigenvalue problems orlinear systems. Here, we assume that A is such that S is non-singular.

Assignment 1.2. Use the test matrix ’lehmer’ that can be obtained with the Mat-

lab command A = gallery(’lehmer’,n); , in which n is the size of the matrix.

(a) Investigate by printing A on your screen the properties of A. Take n = 5. Alsocompute the eigenvalues of A using the command eig.

(b) Generate S for n = 5, and compute the transformed matrix B. Inspect the prop-erties of B by printing the matrix on your screen. Compute the eigenvalues of B. Arethey the same as of A?

(c) Repeat the above assignments for n = 9 and n = 10. Explain your results.

We are now going to use for S the matrix that has an orthonormal basis for theKrylov subspace as its columns. This can be done using the following (modifiedGram–Schmidt) algorithm:

v1 = v/‖v‖2for j = 1, . . . , n− 1

t = Avj

for i = 1, . . . , j

t = t− vi(t,vi)

end

vj+1 = t/‖t‖2end

S = [v1, . . . ,vn ]

Assignment 1.3.

(a) Generate S for n = 5, and compute the transformed matrix B. Inspect the prop-erties of B by printing the matrix on your screen. Compute the eigenvalues of B. Arethey the same as of A?

(b) Repeat the above assignments for n = 9 and n = 10. Explain your results.

1


In these assignments we will investigate how the performance of Matlab’s LU-de-composition can be improved by applying a suitable reordering scheme. We will alsoinvestigate the effect of pivoting on the final accuracy.

Download the files ocean.m, topo.mat and ocean.mat from the course webpage.The file ocean.mat contains the matrix A, the right-hand side vector b and a matrixP. These files define an ocean circulation model on a grid with a resolution of 30. Pmaps every entry in the solution vector x onto a corresponding gridpoint on the mapof the world.

Assignment 2.1. In this assignment you will device a renumbering scheme to min-imise the fill in of the matrix of the ocean problem.

(a) Run the script ocean. You will see the nonzero structure of the matrix A. De-termine the percentage of nonzero elements in A. The matrix A is banded, exceptfor a few arrows. These arrows are related to circulation conditions around islandsand continents, hence each arrow in A corresponds to a continent. Notice that A isstored using Matlab’s sparse matrix storage. Only the nonzero elements of A and thecorresponding indices are stored.

(b) Make an LU-decomposition of A using the commandsthresh = 0; LU = lu(A,thresh);.

The parameter thresh determines a pivoting threshold (help lu). Since in the firstassignment we only study reordering of the equations to minimise fill in, we suppressreordering of the equation to enhance numerical stability by putting thresh = 0;.Check the nonzero pattern in the LU-decomposition of A. Determine the percentageof nonzero elements.

(c) Clearly, the ‘arrows’ in the matrix cause a lot of fill in. A simple idea to overcomethis problem is to change the direction of the arrows. This comes down to eliminatingthe unknowns in reverse order.Reorder the matrix in this way. To reorder you can define an index vector I with entriesI(i)=j in which i is the original number of the unknown and j the new numberof the unknown after reordering. The statement B = A(I,:) reorders the rows ofthe matrix, the statement B = A(:,I), and B = A(I,I) reorders both the rows andthe columns and preserves the symmetric structure of the matrix. Note that this ismuch cheaper than defining a permutation matrix, and multiplying with it! Check thesparsity pattern of the permuted matrix. Compute its LU-decomposition and checkthe resulting nonzero pattern. What is the percentage of the nonzeros in the L and Ufactors?

(d) Row with many zeros cause a lot of fill in (why?). It is therefore a good ideato permute the rows with the most nonzeros to the bottom of the matrix. Make analgorithm that constructs an index vector I that permutes the rows with the mostnonzero elements to the bottom of the matrix. Apply this permutation symmetricallyto preserve the symmetric structure in the matrix. Check the sparsity pattern of thepermuted matrix. Compute its LU-decomposition and check the resulting nonzeropattern. What is the percentage of the nonzeros in the L and U factors?

(e) The Reverse Cuthill–McKee algorithm aims to minimise the bandwidth of thematrix. A Reverse Cuthill–McKee ordering can be computed with the command I =

2

symrcm(A). Apply this permutation, check the resulting sparsity pattern of the per-muted matrix and of the LU-factorisation of this matrix. What is the percentage ofthe nonzeros in the L and U factors?

(f) The minimum degree ordering aims to minimise the fill in. This ordering can becomputed with the command I = realmmd(A) (see also Matlab’s command amd andsymamd). Apply this permutation, check the resulting sparsity pattern of the permutedmatrix and of the LU-factorisation of this matrix. What is the percentage of thenonzeros in the L and U factors?

(g) You have used different permutation strategies. Which one worked best for ourproblem?

Assignment 2.2. In this assignment you will investigate the effect of pivoting on thefill in and on the accuracy of the solution.

(a) Use the Reverse Cuthill-McKee algorithm. Make an LU-decomposition while yousuppress pivoting thresh = 0. Calculate the solution of the system using this LU-decomposition. Check the term |L| |U|. Solve the system using this decomposition.Permute your solution back to the original ordering and compare it with the solutionthat you get with Matlab’s backslash command \. Repeat this assignment, but nowperform partial pivoting by setting thresh = 1. Does partial pivoting cause consider-able additional fill in in this case?

(b) Same assignment, but now use the minimum degree ordering.

3


In these assignments, we investigate the stability of several variants of the Gram-Schmidt orthonormalisation process. These assignments extend the ones of Lecture 1.

Assignment 3.1. Let a1, . . . ,ak be n-vectors. We use the Gram–Schmidt process toconstruct an orthonormal sequence q1, . . . ,qk such that

span(Aj) = span(Qj) for all j = 1, 2, . . . , k. (3.1)

Here, Aj is the n × j matrix with columns a1, . . . ,aj . Similarly Qj ≡ [q1, . . . ,qj ].With span(Aj) we refer to the space spanned by the columns of the matrix Aj.

There are several variants of Gram–Schmidt that are mathematically equivalent.In this assignment we try to get some insight in their stability properties. All variantsfollow an recursive construction.

Gram–Schmidt

classical

for j = 1, . . . , k do

v = aj

rj = Q∗

j−1v

v← v −Qj−1 rj

rjj = ‖v‖2qj = v/rjj

repeated

for j = 1, . . . , k do

v = aj, rj = 0

repeat twice

h = Q∗

j−1v

v← v −Qj−1 h

rj ← rj + h


modified

for j = 1, . . . , k do

v = aj

for i = 1, . . . , j − 1 do

rij = q∗

i v

v← v − qirij


Here, we take the conventions that Q0 = [ ] (in Matlab, take for Q0 an n× 0 matrix:Q=zeros(n,0);), Q∗

0a1 = [ ], etc., the loop i = 1 : 0 is empty and is to be skipped. rj

is a (j − 1)-vector with ith entry equal to rij (of modified Gram–Schmidt).

Assemble the vectors rj and scalars rjj in a matrix Rk: the ij-entry of Rk isthe ith coordinate rij of rj if i < j,0 if i > j, andrjj if i = j:

rj is the jth column of Rk appended with rjj and zeros.

(a) Show that Rk is k × k upper triangular and

Ak = Qk Rk

for the three variants: the variants are mathematically equivalent.

The equivalence is demonstrated under the assumption that no rounding errors areinvolved.

4

(b) Write Matlab function subroutines for each of these variants

(input A, output Q and R; function [Q,R]=clasGS(A);

function [Q,R]=repeatedGS(A);

function [Q,R]=modGS(A);).1

(c) To assess the orthogonality (or loss of orthogonality) plot the log10 of the

νj ≡ ‖Ij −Q∗

jQj‖2as function of j. Here Ij is the j × j identity matrix. Note that in exact arithmeticνj = 0.

(d) Compare the results for, for instance, the matrices (larger n, n = 200?) fromAssignment 1.2. Try also the square n × n matrix A (n = 200) obtained by replacingall diagonal entries of a random n × n matrix by α (say, α = 0.1). Relate the loss oforthogonality to the condition number C2(Aj) (cond(A,2);) of Aj .

(e) In a situation where repeating the orthogonalisation is helpful, investigate (experi-mentally) the effect of increasing the number of repetitions in repeated Gram–Schmidt.

(f) For repeated Gram–Schmidt, the following repetition criterion is popular. If thetangents between vj = v and span(Qj−1) is small, smaller than κ (say less than κ =0.5) then repeat. The tangents can be computed as ‖h‖2/‖v‖2. Why? Include thisrepetition criterion (DGKS [Daniel-Gragg-Kaufman-Stewart] criterion) in your codeand investigate its effect (on the orthogonality, and on the computational costs).

Note. A robust Matlab routine should check whether divison by rjj is allowed,i.e., whether ‖vj‖2 6= 0: do not expand Qj−1 if ‖vj‖2 = 0. Unfortunately, evenif in exact arithmetic ‖vj‖2 = 0, in rounded arithmetic the computed value will benon-zero but can be as big as un

√k ‖aj‖2. Here, u is the relative machine precision

(0.5*eps in Matlab). Therefore, since ‖aj‖22 = ‖rj‖22 + ‖vj‖22, do not expand if‖vj‖2 < un

√k ‖rj‖2. Here, we neglected terms of order O(u2) and we replaced ‖aj‖2

by ‖rj‖2 since the computation of the norm of aj would require an additional innerproduct with an n-vector, while the computation of ‖rj‖2 involves a j-vector only.

Assignment 3.2. Now, suppose A is a square n×n matrix. The Arnoldi algorithmis obtained from the Gram–Schmidt process by replacing the first line vj = aj byvj = Avj−1 except for j = 1, where v1 = a1. This leads to the Arnoldi factorisation

AQk = Qk+1Hk,

where Hk is the (k+1)×k right block of Rk+1, the upper-triangular matrix as producedby Gram–Schmidt after k + 1 steps: Hk is upper Hessenberg.

1Some hints for Matlab programming: Matlab skips empty loops;if A is an n × k matrix (n × k array), then

v=A(:,j);

sets v equal to the jth column of A, whileA(:,j)=w;

sets the jth column of A equal to the vector w, with an error message of the sizes do not match;v=v-Q*r;

replaces the ‘old’ vector v by its updated version;r=Q’*v;

computes Q∗v: Matlab’s prime takes the complex conjugate transpose;if Q is an n × (j − 1) array, then

Q=[Q,v/r(j,j)];

expands the array Q with the column vector v/r(j,j) to an n × j array.

5

(a) Write a Matlab function subroutines to perform the Arnoldi process for the Gram–Schmidt variants and investigate the stability behaviour ((loss of) orthogonality). Asan example take the test matrix ’lehmer’ (as in Assignment 1.2).

6


In these assignments we will implement the SOR method for solving Ax = b for x andinvestigate its convergence.

Download the file sor.m from the course webpage. This file contains code to gen-erate a 2D-Poisson matrix A on a square equidistant n × n grid; A is n2 × n2. Theright-hand-side vector b is such that the solution x equals one. The code splits thematrix A into a strictly lower triangular part L, a main diagonal D, and a strictlyupper triangular part U.

Assignment 4.1. Implement the SOR method to solve this system. Use the mostefficient variant of the algorithm. Use the relative residual norm to test for convergence.Test your algorithm with n = 10, maximum number of iterations 1000, tolerance 10−6,and omega = 1.

Assignment 4.2. Determine numerically a near-optimal value for ω (Hint: ω shouldbe between 1 and 2).

Assignment 4.3. Investigate how the number of iterations depends on the gridsizen. Do this by determining the number of iterations for n = 4, n = 8, n = 16 andn = 32. Take ω = 1. Repeat this assignment but now for near optimal values of ω.

Assignment 4.4. The calculation of the residual is an expensive operation. Wewould therefore like to use a cheaper termination criterion, preferably one on basis ofthe true error instead of on the residual. In this assignment we derive such a criterion.

(a) Show that the spectral radius of G ≡M−1R approximately satisfies

ρ(G) ≈ ‖xk+1 − xk‖2‖xk − xk−1‖2

.

(b) Show that if ρ(M−1R) is known, an estimate for the error is given by

‖x− xk‖2 ≤ρ(G)

1− ρ(G)‖xk − xk−1‖2.

(c) Implement this criterion in your code, and test it for n = 10.

7

Exploiting sparsity; An explanation of the code for Lecture 5 and 6

In practise, high dimensional matrices are usually sparse, that is, the number of nonzero entries in each row is small. Iterative solvers require preconditioning to be fast:rather then solving

Ax = b, (5.1)

the preconditioned system

M−1Ax = M−1b (5.2)

is solved. Here, M approximates A in some (weak) sense and systems Mu = r canefficiently be solved. Preconditioners M are often of the form M = LU with L and Usparse non-singular triangular matrices, L lower and U upper triangular. Via Lu′ = rand Uu = u′, the system Mu = r can efficiently be solved for u.

Now suppose we have a (file with the) Matlab subroutine, say

function [x,hist,t]=gcr(A,b,x0,kmax,tol);,

that solves (5.1) iteratively. Here,at the input side• A is the matrix A,• b is the vector b,• x0 is the initial guess x0 (often x0 = 0, x0=0*b in Matlab),• kmax is the maximum number of iteration steps that is allowed and• tol is the required reduction of the residual norm,

that is, stop the iteration if ‖rk‖2/‖r0‖2 <tol.At the output site• x is the solution as computed by the subroutine at termination

(i.e., x= xk with k=kmax or ‖rk‖2/‖r0‖2 <tol),• hist is the convergence history,

that is, the sequence (‖rj‖2/‖r0‖2)kj=0of intermediate residual norm reductions,

• t is the time the routine needed.To use this routine we have to form the vector M−1b, which is not a problem. But,

we also either have1) to form the matrix M−1A or2) to adapt the code gcr.m to compute the vector c = M−1Ar from r.

Discussion.1) To focus the discussion, assume that M = LD−1U with D diagonal, and L and Usparse triangular matrices that are explicitly available. Generally, the matrices L−1,U−1 will be full (check that L−1 is a full matrix if, for instance, L has all ones on themain diagonal and on the first lower co-diagonal and zeros elsewhere). Therefore, thematrix M−1A will generally be full as well. Forming the matrix M−1A explicitly isextremely expensive (in, both computational costs as in memory and is often even notpossible). And even if the matrix A ≡ M−1A would be available, the computationof c = Ar from r by multiplication by A (at 2n2 flop) is much more expensive thanobtaining c by performing the following four steps

c′ = Ar, solve Lc′′ = c′ for c′′, c′′′ = Dc′′, solve Uc = c′′′ for c, (5.3)

(at 2 kn flop with k the sum of the maximum number of non-zeros in each row of L, Uand A plus 1. If the matrices are sparse k will be ≪ n).

8

2) Adapting the code gcr.m to perform the steps in (5.3) is not attractive for severalreasons:a) the code has to be adapted at several places (to be explicit: (i) in the input list ofgcr, ‘A’ has to be extended to ‘A,L,D,U’, and (ii) whenever the command c=A*u is usedin gcr, it has to be replaced by a coding of the steps in (5.3)),b) other types of preconditioning would require similar but new adaptations, andc) if you want to exploit other solvers (as GMRES), the code for these solvers has tobe adapted as well.Of course, each adaptation caries the danger of introducing errors in the code.

Defining the action of a matrix by means of a function

An efficient way out is to call a function subroutine, say MyPreMV, in the gcr codewhenever a matrix-vector multiplication is required. For instance,

function [x,hist,t]=gcr(MV,b,x0,kmax,tol),

where now MV is a handle of this function, as MV=@MyPreMV,2 and, at each occurrence, inthe gcr code the command c=A*u; is to be replaced by c=feval(MV,u);. For instance,MyPreMV.m could be (a file of) the function subroutine

function c=MyPreMV(u)

c1=A*u; c2=L\c1; c3=D*c2; c=U\c3;return

(5.4)

We can make the routine MyPreMV as efficient as possible without fiddling with theroutine gcr (for instance, we can allow old ‘c-values’ to be replaced by new ones as


c=A*u; c=L\c; c=D*c; c=U\c;return

(5.5)

thus saving memory). Now, we can call gcr in the command window, by first executingthe command MV=@MyPreMV;, followed by

x=gcr(MV,b,0*b,500,1.e-6); (5.6)

If another type of preconditioning is required, we can make another function sub-routine, say MyPreMV2, to handle this new matrix vector product. Then, redefining MV

to MV=@MyPreMV2; allows us to use the command (5.6) again.

This subroutine approach for incorporating the MV is also useful if, for instance,c is the solution at time T , c = Y(T ), of a high dimensional time dependent lineardifferential equation Y′(t) = HY(t) with initial condition Y(0) = u: in this case, cdepends linearly on u. Then a subroutine that solves the differential equation definesthe action of the matrix (A = exp(TH)) while the matrix itself is not available.

2For information, search Matlab’s documentation for ‘function handle’. Matlab allows calling afunction inside another function by handle as well as by name. To be more specific, we could alsotake MV=’MyPreMV’. Here, we focus on function handles for passing functions, rather than on strings offunction names. The approach with function handles is more elegant in Matlab. The approach withfunction names will require the use of global variables as we will explain in the subsection below onglobal variables.

9

Incorporating the action of the matrix in a function

Note that, in order to be able to use the subroutine MyPreMV of (5.4), we have to getthe matrices A, L and U ‘known’ to this subroutine. We do not want to do that byexplicitly passing these quantities, that is, via an input argument of the form

function c=MyPreMV(u,A,L,D,U). (5.7)

Because, we then have to include these quantities also in the input list of gcr, whichwe were trying to avoid. The alternative of defining these quantities inside the sub-routine MyPreMV is even more undesirable, since then the quantities would be definedagain and again in each iterative step of gcr in which the routine MyPreMV is called(by c=feval(MV,u);, here MV=@MyPreMV). This is where function handles can be con-veniently exploited.3 In a function, say, DefineMatrixAction, we can define the quan-tities as L, D and U, depending on a given matrix A, as well as the function MyPreMV.By means of a function handle we can make this function MyPreMV available outsideDefineMatrixAction. In the following, as an example, we take L to be the lower tri-angular part of A, U the upper triangular part, both including the diagonal of A, andD the diagonal of A.

function [MV,bt]=DefineMatrixAction(A,b)

L=tril(A); U=triu(A); D=diag(diag(A));


c=A*u; c=L\c; c=D*c; c=U\c;end

MV=@MyPreMV;

bt=L\b; bt=D*bt; bt=U\bt;end

(5.8)

This function DefineMatrixAction defines the preconditioned matrix-vector multipli-cation MyPreMV. Via the list of ouput arguments, the function handle in MV=@MyPreMV;,allows to make the preconditioned matrix-vector multiplication MyPreMV available out-side DefineMatrixAction. Note that DefineMatrixAction also computes the precon-ditioned right-hand side vector b ≡ U−1DL−1b.

Since the function MyPreMV is defined inside the function DefineMatrixAction, thequantities as L and U that are ‘known’ inside DefineMatrixAction are also ‘known’ toMyPreMV. In particular, the undesirable explicit passing of quantities as L to MyPreMV

(as in (5.7)) is not required now.4

Before executing the command (5.6) in Matlab’s command window, execute acommand as DefineMatrixAction:

[MV,bt]=DefineMatrixAction(A,b);

x=gcr(MV,bt,0*b,500,1.e-6);

3For an alternative approach, using global variables, see the next subsection.4If you are familiar to C++ programming, then the observation that the function handle @MyPreMV

actually is a pointer to the function MyPreMV might be illuminating.

10

Matrix actions using global variables

In this subsection, we suggest the use of global variables as an alternative to the useof function handles. If you are comfortable with function handles, you can skip thissubsection.

If the matrix vector multiplications c = Au have been defined in gcr as c=feval(MV,u),then, as an alternative to function handles, as MV=@MyPreMV, the name of a functioncan be used: with MyPreMV as in (5.4), command (5.6) with MV=@MyPreMV or withMV=’MyPreMV’ leads to the same result.

However, calling a function by name rather than by handle, makes it more trickyto get the quantities A, L, . . . ‘known’ to MyPreMV without explicitly passing them (seethe discussion in the preceding subsection).

The problem is that, if MyPreMV has been defined as in (5.8), then MyPreMV is a localfunction and the string ’MyPreMV’ is recognised as the name of a function only inside thefunction DefineMatrixAction. Defining MV=’MyPreMV’ inside DefineMatrixAction

(instead of MV=@MyPreMV as in (5.8)) with MV in the output list turns MV outsideDefineMatrixAction into the string ’MyPreMV’ but the string has now further mean-ing (it will not be recognised as the name of a function). In summary, in contrast tofunction handles, strings as ‘MyPreMV’ are recognised as the name of a function onlyif the function MyPreMV is not defined inside another function (i.e., the file MyPreMV.m

should only consist of the function MyPreMV): MyPreMV should be a global function.

If MyPreMV is a global function, global variables can be used to get the quantities A,L, . . . ‘known’ to MyPreMV without explicitly passing them:


global A L D U

c1=A*u; c2=L\c1; c3=D*c2; c=U\c3;return

(5.9)

Matlab (help global): “If several functions, all declare a particular name as GLOBAL,then they all share a single copy of that variable. Any assignment to that variable, inany function, is available to all the other functions declaring it GLOBAL”. Global vari-ables are ‘known’ also in other function subroutines in which they have been declaredglobal, whereas the other type of variables, so-called local variables, are known onlyinside the function subroutine in which they are used. Before executing the command(5.6) in the commandwindowexecute a command like MakeMatrix,

MakeMatrix(A);

MV=’MyPreMV’;

x=gcr(MV,b,0*b,500,1.e-6);

where MakeMatrix is a function subroutine in which A, L, D, and U are defined anddeclared to be global. For instance,

function MakeMatrix(A)

global A L D U

L=tril(A); U=triu(A); D=diag(diag(A));

return

(5.10)

11

Note. To make sure that global variables do not accidentally get mixed up with localones, global variables are usually given long complicated names, so rather MyMATRIX

than A.

Note. The ‘declaring’ subroutine MakeMatrix and the ‘matrix-vector subroutine’MyPreMV go together: if you want another matrix-vector multiplication (MV) (or an-other preconditioner), then you have to write a new declaring subroutine, MakeMatrix2say, and a new MV subroutine, MyPreMV2 say. In order to keep things together that gotogether, I placed these two related subroutines in the same file as MyPReMV.m. How-ever, a second routine in a file can only be called from routines in the file and notfrom routines outside this file nor from the command window. I solved this obstacle byletting MakeMatrix be executed whenever MyPreMV was called by an empty argument:

MyPreMV([]);

x=gcr(’MyPreMV’,b,0*b,500,1.e-6);

,

where the file MyPreMV.m contains two function subroutines and looks like


global MyMATRIX MyLOWER MyDIAGONAL MyUPPER

if isempty(u), MakeMatrix(); c=size(MyMATRIX,1); return, end

c=MyMATRIX*u; c=MyLOWER\c; c=MyDIAGONAL*c; c=MyUPPER\c;return

function MakeMatrix()

global MyMATRIX MyLOWER MyDIAGONAL MyUPPER

MyMATRIX=...; MyLOWER=tril(MyMATRIX);

MyDIAGONAL=diag(diag(MyMATRIX); MyUPPER=triu(MyMATRIX);

return

(5.11)If u is empty (u=[]), then the global variables MyMATRIX, . . . , are created. If u is anon-empty vector (as it will be when called by gcr.m), then the ‘MakeMatrix line’ isskipped in MyPreMV and the the preconditioned MV is performed.

Note. If the matrix is not available, but only its action via a subroutine as MyPreMV,then Matlab’s command size can not be applied to A to find the dimension. Withn=MyPreMV([]);, tha above routine returns the dimension.

Note. If you want to have a global variables as MyMATRIX also available in the commandwindow or the workspace, then you have to declare it global in the command window(or in an M-file, not being a function subroutine, as main.m) as well.

Note. The routines for iterative solvers of the Matlab company (as gmres.m andbicgstab.m) take a matrix as input as well as a function subroutine (either known byits name or by its handle) that performes the MV.

The folder structure (Code with Lecture 6).

The folder Problems contains the files, as problem0.m, that define the matrix A, say,and the function routine that performs the matrix-vector multiplication (MV), c = Au,

12

(of the type MyPreMV). This MV is passed as first argument in the output list by afunction handle as

function [MV,b]=problem0(n).

The second argument in the output list contains the right-hand side vector b. The n

in the input of this example function problem0 specifies the required dimension of thematrix. Some problems functions can provide a third output argument Lambda. Thisis a vector of all eigenvalues of the matrix.

problem1b.m relies on the use of global variables, as explained in the precedingsubsection. The other routines use function handles.

The folder Solvers contains the files with codes of iterative solver as LMR, Richard-son (polynomialsolver.m), GCR (gcr.m), etc.

The folder Subroutines contains miscellanuous subroutines (for orthogonalisation—Gram-Schmidt variants—, etc.).

Before running main, run install. This subroutine sets the path for Matlab (tellsMatlab also to ‘look’ in the folders Problems, Solvers, Subroutines for subroutines).

Example. In the first assignment of Lecture 5 you are asked to explore the significantsof exploiting sparsity. You are asked to experiment with a diagonal matrix in severalformats. You can use the routine problem1.m as provided in Lecture 5’s Matlab

package. You can also write your own brand. For instance, you can create a fileMakeDiagonal.m with a routine like

function [MV,b]=MakeDiagonal()

n=10000;

MyDIAGONAL=sqrt(1:n)’;

function c=MyDiagMV(u)

c=MyDIAGONAL.*u;

end

MV=@MyDiagMV;

xe=ones(n,1); b=MyDiagMV(xe);

end

and in the command window you can call these routines like

[MV,b]=MakeDiagonal();

tic, x=polynomialsolver(MV,b,0*b,500,1.e-6,(1+n)/2); toc

The routine polynomialsolver.m multiplies the residual in each step by (I− 1

µA) with

A as, in this case, defined by MyDiagMV.m and µ =(1+n)/2 (Note that this puts µ inthe center of the spectrum). It stops the iteration when the residual is reduced by1.e-6 or when more than 500 steps were required. x is the approximate solution attermination.The choice b=MyDiagMV(xe)makes sure that the exact solution is known, which may beconvenient in an experimental stage, since it gives access to the error (norm(x-xe,2)).With x=polynomialsolver(’MyDiagMV’,b,0*b,500,1.e-6); α = 1/µ is computed ineach step, to minimize the norm of the new residual (then polynomialsolver is LMR).

13

The tic toc commands give the time that Matlab spent in between the tic-toc. clockand etime can do this as well. With

profile on, x=polynomialsolver(. . .); profile viewer

Matlab gives a very detailed report on timings.

14


Assignment 5.1. This assignment uses the codes main.m, matrix1.m and gcr.m

(and modifications)

(a) The file matrix1.m defines the matrix vector multiplication (MV). The matrix isa diagonal” A = diag(Lambda), where Lambda is a column vector and diag(Lambda) isthe diagonal matrix with (i, i) entry equal to Lambda(i, 1). The MV can be performedas c = A ∗ u, but also as c = Lambda. ∗ u. Can you observe a difference? Explanation?The command spdiags can also be used to define the matrix. Discuss the effectivenessof defining the matrix with this command.

The matrix in the above experiments is diagonal. Does it effect the number ofiterations steps if we replace the diagonal matrix Λ = diag(Lambda) by the matrixA ≡ Q∗ΛQ, with Q unitary (and we replace b by Q∗b)? The file matrix2.m isprepared to do this transformation by means of a Householder reflection. It is notadvisable to form the matrix A explicitly. Why not? Fill in the details in this filematrix2.m and answer the question on the number of iteration steps? Explain theresult?Do you come to the same conclusion if you replace Λ and b by T−1ΛT and T−1b,respectively? Take for T the bi-diagonal matrix I−αS with α ∈ (0, 1) en S the matrixwith all zeros except on the first co-diagonal, where it has the values 1 (Si+1,i = 1).How do you code the application of T−1 to a vector?

Conclusion: the convergence history for symmetric problems depends on the eigen-values and not on the eigenvectors.

(b) What method do you prefer, Richardson with optimal α, or Minimal Residuals incase A is a diagonal with eigenvalues in [λ−, λ+] ⊂ (0,∞). Does it make a differencewhether the n eigenvalues are uniformly distributed between [1, 400] or [1, 2]?

(c) Does the ordering of the parameters in Chebyshev iteration with a fixed polynomial(of degree ℓ) affect the convergence?

(d) Let A be the n × n tridiagonal matrix with 2 on all the diagonal entries and −1on all the co-diagonal entries. Use eig (for a number of (small) n) to find an interval[λ−, λ+] ⊂ (0,∞) that contains the eigenvalue. How sensitive does Chebyshev iterationdepend on the estimates of λ−? (Use γλ− in the estimates instead of λ− for γ between0.5 and 2.

Investigate how effective Chebyshev iteration is on a problem Ax = b with spectrumin the complex plane close to an interval [λ−, λ+] ⊂ (0,∞). (Replace Λ and b by TΛTand T−1b with T as in (a)).

(e) Write a code for three terms Chebyshev iteration, i.e., the degree of the Chebyshevpolynomial equals the iteration step.

(f) Compare Chebyshev iteration and GCR (check the strong and weak points of thesemethods as listed on the transparencies).

15


Assignment 6.1. Conjugate Residuals. If A is Hermitian (A∗ = A), then theorthogonalization loop in GCR can be reduced to a single update. The resulting al-gorithm is called CR (Conjugate Residuals). Write a function subroutine CR.m for CR.Compare the convergence (in number of iteration steps and in computational time) ofGCR and CR for a symmetric matrix. (Diagonal matrices can be used for experiments(why?). Take A = diag(Λ) with Λ = 1 : 400 and Λ =

√1 : 400. Consider also shifted

versions A← A− σI with σ between 1 en 10.)How does CR perform on a (weakly) non Hermitian problem (Take a matrix of the

form TDT with D diagonal and T = I− γS with γ small and S the Shift matrix, thatis, all entries are equal to 0 except for Si+1,i = 1 all i)?

Assignment 6.2. Restart and truncation. The computational costs as well asstorage requirements of both GCR and GMRES increase with increasing step number.To limit the computational costs, the methods can be restarted after ℓ steps, i.e.,every ℓth steps, solve Ax = b with ℓ steps of GCR or GMRES with initial guessthe approximate xjℓ from the preceding sweep of ℓ steps. Show that this strategy canconveniently be incorporated in GCR by replacing the line ‘for j = 0, 1, . . . , k− 1 do’by ‘for j = π(k), π(k) + 1, . . . , k − 1’, with π(k) ≡ mℓ if k ≥ mℓ (m ∈ N0 maximal).(Note that the loop in the Matlab code starts with 1 rather than 0: Matlab doesnot allow 0 index such as u(:, 0).) Write a restarted version for GCR and GMRES.Investigate experimentally the effect of the restart on the convergence.

Another strategy to limit the computational costs in GCR is to limit the orthog-onalization loop to the ℓ most recent vectors cj , that is, take π(k) = max(0, k − ℓ).Incorporate this truncation strategy in GCR (adapt the code also to minimise storage).What is the effect of truncation on the convergence? Try also symmetric matrices andweakly non-symmetric ones.

Can truncation also be incorporated in GMRES?

Assignment 6.3. Stability GMRES. GCR and GMRES are mathematically equiv-alent, that is, started with the same initial guess and assuming exact arithmetic, thekth GMRES residual is equal to the kth GCR residual. The number of MVs (matrixvector multiplications) are the same. With respect to the other computational costs(and memory requirements), GCR is approximately twice as expensive as GMRES.

Is this visible in the computational time?To compare stability, select a step number k0 (say, k0 = 2 or k0 = 20) and perturb

the k0 generating vector (with w = rk0, uk0

= w + ∆ in GCR, with w = Avk−1,orthogonalize w + ∆ in GMRES. Take ∆ a random vector of size δ‖w‖2 (δ = 10−5,δ = 10−2). How does this affect convergence?

16


In this assignment we will investigate the superlinear convergence of CG.

Assignment 7.1.

(a) Implement the CG algorithm. Start with x0 = 0. Your algorithm should be calledas follows:

[x,res] = cg(A,b,m iter,eps); .The input parameters are:

A: the system matrix,

b: the right-hand side,

m iter: maximum number of iterations,

eps: residual tolerance.The output parameters are:

x: iterative solution (as produced by CG at termination),

res: convergence history, i.e., the row (‖r0‖2, ‖r1‖2, . . .) of

residual norms ‖rk‖2 in every iteration step,

(b) Define a sparse diagonal matrix A and right-hand-side vector b of dimension 1000

A =

1

2. . .

1000

, b =

1

1...

1

.

Use the system Ax = b to check if your CG code is correct. Plot the residual normas function of the iteration number. Determine the “rate of convergence” (reduction ofthe residual norm per iteration) around the 30th iteration. How does this compare tothe theoretical rate of convergence? What is the condition number of A?

(c) With b again all 1s, now define the diagonal matrix A, also of dimension 1000,

A =

1. . .

1

11. . .

1000

.

Solve the new system Ax = b. Plot the residual norm as function of the iterationnumber, and determine the rate of convergence around the 30th iteration. How doesthis compare to the theoretical rate of convergence? What is the condition number ofA? Explain the difference with the previous assignment.

(d) Extend your code with the possibility to compute the Ritz values. Determine forthe above examples the convergence to the eigenvalue 1. Use this information to explainthe superlinear CG convergence of the above examples.

17


Assignment 8.1. CG has been designed for symmetric definite systems. MINRESand SYMMLQ can handle symmetric non-definite systems. Nevertheless we can tryto apply CG also to non-definite systems. In this assignment, we investigate how thisworks in practice.

Consider the diagonal matrix D with eigenvalues 1 : 1000 shifted by σ.

(a) Solve the system Ax = b with A = D and b random and normalised with CG,MINRES and SYMMLQ. Select the shift close to a small eigenvalue.

What is the effect of rotating the matrix, i.e., replace A by (I−2VV∗)D(I−2VV∗)with V a low dimensional (dimension 1?) orthonormal matrix?

What is the effect of replacing b by a vector of all ones and then normalised?What is the effect of selecting x randomly and setting b equal to Ax?

(b) Select the shift close to a small Ritz value (if you did not code the extraction ofRitz values from CG, then you can use minres0.m to compute the Ritz values).

Assignment 8.2. The expansion vector for the search subspace in GCR at step k isselected to be equal rk: uk = rk, whereas the expansion vector in GMRES is equal tovk.

(a) What is the effect of perturbing the expansion vector at, say step k = 10, by arelatively small perturbation, i.e., use uk = rk + δ∆ in GCR and vk ← vk + δ∆k inGMRES with ‖∆‖ ≈ ‖rk‖, and ‖∆‖ ≈ ‖vk‖ respectively, and δ = 10−3. Also takeδ = 1.

(b) Perturb the first step.Explain why GCR seems to be insensitive to perturbations.

(c) Consider the system Ax = b + δ∆ and use v0 = b/‖b‖2 to generate the Krylovbasis used in GMRES. Explain why GMRES is sensitive to perturbations.

Conclusion. GCR is a subspace method: any vector can be used to effectively expandthe search subspace. As a consequence any additional information available on thesolution (singularities, . . . ) cab be exploited in GCR.

GMRES is Krylov subspace method: the algorithm heavily exploits the Krylov struc-ture (Hessenberg matrix, etc.) to have an efficient algorithm (efficient as compared toGCR). However, perturbations spoil the Krylov structure and, consequently, the effec-tivity of the method.

(d) GCR may stagnate (if c∗krk = r∗kArk = 0) and breakdown in the next step (why?).The choice uk = ck−1 overcomes this breakdown (see Exercise 8.5).

Write an ORTHODIR code (i.e., use the choice uk = ck−1) and check that OR-THODIR and GCR have the same residuals if GCR does not break down. How sensitiveis ORTHODIR to perturbations on the vectors that generate the search subspace.

18


In these assignments we will investigate regularisation effect of iterative methods.

Download the file nolet.tar from the course webpage. This archive containsthe files nolet.m, soluti.m, plotsol.m, matrix.dat and solution.dat. The scriptsoluti.m computes the (regularised) solution using the SVD. The script plotsol.m

plots the solution.

Assignment 9.1.

(a) Run the script nolet.m. It will show you the model solution to the problem andthe singular values of the system matrix A. Next it will ask you how big the noise inthe right-hand side should be (relative error in the right-hand side). Take 0. The scriptwill ask you how many singular values (vectors) should be included in the computationof the solution. Predict this number on basis of the distribution of the singular values,and validate this experimentally. The script will give you the solution and the error(difference between computed solution and model solution) after every calculation.

(b) Repeat the assignment with a relative error in the right-hand side of 0.01.

(c) Repeat the assignment with a relative error in the right-hand side of 0.1.

Assignment 9.2.

(a) Implement the CGLS algorithm (as efficient as possible). Start with x0 = 0. Youralgorithm should be called as follows:

[x it res err] = cgls( A, b, m iter, eps, x mod ); .The input parameters are:

• A: the system matrix,

• b: the right-hand side,

• m iter: maximum number of iterations,

• eps: error tolerance.

• x mod: the model solution.

The output parameters are:

• x it: Iterative solution,

• res: residual norm ‖ATb−ATAxk‖2 in every iteration.

• err: norm of difference vector between iterative solution and model solution‖xit − xmod‖2 in every iteration.

(b) Use your algorithm to solve the Nolet problem. Plot after every iteration the ap-proximate solution by including the call plotsol(x it) after you update your solution.Also plot the norm of the residual and the error as function of the iteration number.First compute the solution without adding noise to the right-hand side. How manyiterations do you need to minimise the difference between iterative solution and modelsolution?

(c) Repeat the assignment with a relative error in the right-hand side of 0.01.

(d) Repeat the assignment with a relative error in the right-hand side of 0.1.

19


In these assignments we will investigate how we can parallelise ILU and SSOR.

Download the file precon.m from the course webpage. The matrix A in thiscode is from the Poisson equation discretised on a uniform n × n grid (type help

private/poisson in Matlab for more information).

Assignment 10.1.

(a) Add to the script precon.m code to create an SSOR preconditioner (without re-laxation parameter. Your preconditioner must have the form M = LLT. Use thispreconditioner to solve the system with the Matlab routine pcg. Make sure your im-plementation is efficient. (How does Matlab handle the preconditioner: left-explicit,right-explicit, . . . , implicit? Note that the Matlab documentation explains that ef-fectively left-explicit preconditioning is used but does not explain what is meant by“effectively”. Inspect the code. What are the advantages of ‘Matlab’s approach’?Matlab offers the possibility to include one preconditioner M in pcg, but it is alsopossible to include two preconditioners M1 and M2. What is the advantage of this sec-ond possibility?) Investigate how the number of iterations to solve the system Ax = bdepends on n, the number of gridpoints in each direction.

(b) Same assignment, but now use the incomplete Cholesky preconditioner without fill-in. Use the Matlab routine ichol (replaces cholinc) to compute the preconditioner.

(c) The forming of the preconditioners costs time as well. How does that affect theperformance? (Is it important to have sparse A and L?)

Assignment 10.2.

(a) Make a red-black ordering (or checker board ordering) of the unknowns, that is,of the gridpoints. This ordering splits the unknowns into two groups such that noneof the unknowns within a group has a direct neighbour in the group. The resultingmatrix has the block form

A =

[A11 A12

A21 A22

]

in which A11 and A22 are diagonal matrices.

(b) Make an SSOR preconditioner on basis of the reordered matrix. Compare itsperformance with the SSOR preconditioner that you used in the first assignment. Whyis this preconditioner better suited for parallel computing than the one you used inassignment 1?

(c) Same questions, now for the incomplete Cholesky preconditioner. Can you relateincomplete Cholesky to SSOR both for the checker board ordering?

20


In these assignments you will investigate the iterative solution of the real linear systemAx = b, in which A is a block matrix of the following form

A =

[F B

BT −C

]. (11.1)

Here, F is symmetric and positive definite, and C is symmetric positive semi-definite.C may be zero. Such block matrices arise in many applications. Systems with a blockmatrix (11.1) are known as KKT systems or saddle-point systems.

In this assignment both F and C are diagonal matrices.

Saddle-point systems are notoriously difficult to solve by iterative methods. Oneapproach is to precondition them using a preconditioner based on the block LU-factorization [

F B

BT −C

]=

[I O

BTF−1 I

][F B

OT −MS

].

Here, MS ≡ BF−1BT + C and −MS is the Schur complement. Take

P =

[F B

O −MS

](11.2)

as preconditioner. This leads to the right-preconditioned matrix

AP−1 =

[I O

BTF−1 I

].

Explain why this implies that GMRES applied to a right-preconditioned systemsaddle point system with (11.2) as preconditioner must find the exact solution in atmost two iterations (see Exercise 11.9).

The preconditioner that we have defined above is nonsymmetric while the systemmatrix (11.1) is symmetric. This may seem somewhat unnatural. A symmetric precon-ditioner for saddle point systems is the block diagonal matrix

P =

[F O

OT MS

]. (11.3)

Note that this preconditioner is also positive definite. Therefore, (implicitly pre-conditioned) MINRES is applicable.

The preconditioned matrix P−1A has three distinct non-zero eigenvalues in caseC = 0 (see Exercise 11.10) and MINRES find the exact solution then in at most threeiterations.

In the next assignments you have to perform several numerical experiments onsaddle-point systems from the test-set Schenk IBMNA, which is part of the Tim Davis’matrix collection at the University of Florida(http://www.cise.ufl.edu/research/sparse/matrices/). In all the assignments you shoulduse the Matlab-script kkt.m that you can download form the course web page. It isnot necessary to program yourself. In the experiments you also need the script idrs.mwhich you can download from http://ta.twi.tudelft.nl/nw/users/gijzen/IDR.html.

21

Assignment 11.1. Download, in Matlab format, the problem c-18 from the TimDavis’ collection (the Matlab command load c-18, loads the problem Ax = b into astruct named ‘Problem’ that contains the matrix [problem.A] as well as the right-hand-side vector [Problem.b]. With the Matlab command Problem, the other quantitiesin the struct are displayed as well). Check that the matrix c-18 has the above blockstructure. What is the size of the matrix? Explain why MINRES is a natural choice forsolving the system. Try to solve this system with MINRES without preconditioning.Conclusion?

Now we are going to combine MINRES with preconditioning.

Assignment 11.2. Solve the problem with preconditioned MINRES, taking (11.3)as the preconditioner. Discuss the convergence. (Warning: Matlab uses left precon-ditioning in MINRES, but gives the residual norms for the unpreconditioned system.Therefore, residual norms may go up in your convergence curve.)

For large systems it is not possible to compute the Schur-complement matrix. Asimple idea is to approximate the Schur complement by its main diagonal. The resultingpreconditioning matrix then becomes

P =

[F O

OT diag(MS)

], (11.4)

which is a diagonal matrix.

Assignment 11.3. Solve the problem with preconditioned MINRES, taking (11.4) asthe preconditioner. Also solve the system with the following methods (preconditionedwith (11.4)): Bi-CGSTAB, IDR(1) and IDR(4). Plot the convergence curves of the fourmethods in one figure. Put on the x-axis the number of matrix-vector multiplicationsand on the y-axis the residual norm divided by the norm of the right-hand side vector.Which method is fastest? Is this what you would expect?

Following the same reasoning as above we can also use

P =

[F B

OT −diag(MS)

](11.5)

as preconditioner. This matrix is nonsymmetric, but a Krylov solver for nonsymmet-ric systems combined with this preconditioner might be more efficient than MINREScombined with the symmetric preconditioner. We will investigate this in the nextassignment.

Assignment 11.4. Solve the problem preconditioned with (11.5) with the followingmethods: Bi-CGSTAB, IDR(1), IDR(4), Bi-CG and QMR. Plot the convergence curvesof the five methods in one figure. Put on the x-axis the number of matrix-vectormultiplications and on the y-axis the residual norm divided by the norm of the right-hand side vector. Note that Bi-CG and QMR need two matrix-vector multiplicationsper iteration. Discuss the convergence. Which method is fastest?

Assignment 11.5. Select at least two other test problems from the same test set.Repeat Assignment 11.3 and Assignment 11.4. Discuss and draw conclusions.

22


Assignment 12.1. Pseudo-spectrum. Let A be a (complex) n× n matrix.For a discussion of the notion ‘pseudo-spectrum’, see Theorem 12.17 and preceedingparagraph and subsequent two paragraphs in the set of exercises.

(a) Consider the matrix A = (Aij) with ones on the +1 and −k diagonal for k = 2,(i.e., Ai,j = 1 if j − i ∈ {1,−k}) and zeros elsewhere.Plot the spectrum of A for n ∈ {100, 200, 300}. Discuss the results.Is there a difference between the (computed) spectrum of A and A∗? Is there a differ-ence between the results of Matlab’s eig(A) and eig(A,’nobalance’)?5

Plot also the eigenvalues of A + ∆, where ∆ is random n × n matrix with ‖∆‖2 ≤ ε(ε = 10j for j = −2,−4,−6,−8).

(b) Download the Matlab tool EigTool from Oxford university: follow the appropri-ate link on the course homepage. This tool allows you to compute the pseudo-spectrumof matrices (of modest dimension).Run the command eigtool(gallery(’grcar’,32)) in Matlab to plot the pseudoeigenvalues of the grcar matrix of dimension 32.Plot the pseudo-spectrum (with eigtool) of the matrix A from from part (a).Compare the results with the plot of the eigenvalues A + ∆ as computed in part (a).

(c) Is the sensitivity of the eigenvalues to perturbations reflected in the angle betweenright and left eigenvector: plot the condition number of the eigenvalues of the matrixA from from part (a) (eigenvalue λ for λ ∈ (0,∞) versus Cλ(A))? Recall that

Cλ(A) =‖x‖2 ‖y‖2

y∗x

if λ is a simple eigenvalue with right eigenvector x and left eigenvector y, i.e., Ax = λx,y∗A = λy∗ and both x and y are non-zero n-vectors; see also Matlab’s routinecondeig.

(d) Compute the conditioning of the (right) eigenvectors as well.

(e) Derive an analytic expression for the eigenvalues of the matrix A as defined inpart (a).

Assignment 12.2. Let A be a Hermitian n× n matrix (real symmetric if you wish)with a few negative eigenvalues.

(a) Extend the Lanczos code for computing Ritz values to compute harmonic Ritzvalues. Make sure your code is efficient. (The results in Exercise 6.4(f) can be used.Note that in this case Hk is Hermitian. Unfortunately, the matrix in 4) nor the one in5) is Hermitian. There is also an Hermitian version that allows more stability).

(b) Plot Ritz values and harmonic Ritz values of A (around 0).

(c) Do harmonic Ritz values lead to faster convergence?

5What is purpose of ‘balancing’ the matrix A, i.e., of eig(A,’balance’)?

23


Assignment 13.1. Let A be an n × n matrix. We are interested in the stabilityof the Arnoldi process to compute approximate eigenvalues (and of GMRES to solvelinear systems). We try to obtain some insight from experiments.

(a) We consider an alternative way of expanding the Krylov subspaces Kk(A,v1) gen-erated by A and some non-zero vector v1.Let (u, ϑ) be a Ritz pair in Kk(A,v1) (that is, u ∈ Kk(A,v1) and r ≡ Au − ϑu ⊥Kk(A,v1)). Show that r is a multiple of vk+1, where vk+1 is the next Arnoldi vector(i.e., an orthonormalized version of Avk).

This offers two alternatives for expanding the Arnoldi basis:1) Arnoldi expansion: expand by Avk

2) residual expansion: expand by r.We are interested on the effect of perturbations on the quality of the search subspace.

Therefore, compute Hk as V∗

k AVk.

(b) Perturb the k0 expansion vector with a random vector. What is the effect on theconvergence? Does it depend on the expansion strategy? Does it depend on the sizeof the perturbation? When approximating the eigenvalue with, say, largest real part,the eigenvalues with second largest real part also tend to converge. What is the effectof the perturbation (in relation to the expansion strategy) on the convergence to the“second” eigenvalue?

(c) In the next experiment, we follow the Arnoldi expansion and use the Hessenbergmatrix as obtained as side-product from the Gram–Schmidt orthogonalization.

We perturb each matrix vector multiplication, that is, we work with A(v) = Av+ frather than with Av, where f is random such that ‖f‖ ≤ η ‖A‖2 ‖v‖2

Select η = ε = 10−8. What is the effect of the perturbation?Try also η = ε/‖r‖2 (the value of η is step dependent here!).

24


In this assignment you will develop a simple multigrid code for solving the 1-dimensionalPoisson equation.

Assignment 14.1. Generate a 1D Poisson system using the following commands:

level = input(’Level = ’)

n = 2∧level-1h = 1/(n+1);

e = ones(n,1);

A = (1/h∧2)*spdiags([-e 2*e -e], -1:1, n, n);

b = ones(n,1);

The parameter Level determines the size of the system. Take Level = 10. Writea code that performs k steps of Gauss–Seidel iteration on the system, starting with arandom initial guess. Plot the residual and the error after every iteration (until k = 10)and verify that the Gauss–Seidel iterations also smooth the residual. Why is the errorsmoother than the corresponding residual?

Assignment 14.2. Write subroutines for the prolongation and the restriction oper-ation. The restriction operation is such that a vector xc on the courser level takes asvalues in the gridpoints

xc(i) = 0.25xf (2i − 1) + 0.5xf (2i) + 0.25xf (2i + 1),

where xf is the vector on the finer grid. The prolongation operation is such that

xf (2i) = xc(i)

andxf (2i + 1) = 0.5 (xc(i) + xc(i + 1)).

Note that x(0) = x(n + 1) = 0.Test your subroutines, for example on the solution of the system.

Assignment 14.3. Write a two grid method.A cycle must consist of the following steps:• Perform k steps of Gauss-Seidel iteration on the approximate solution xf (pre-

smoothing);• Compute the residual rf (stop if the norm of the residual is small enough).• Transfer the residual to the courser grid (one level courser),

using your restriction routine.• Solve the system Acuc = rc, where all vectors are defined at the courser level.• Prolong uc to the finer level, add the resulting uf to xf .• Perform one Gauss–Seidel iteration (post smoothing).• Repeat the above steps until convergence.

Test your program for k = 1 and different problem sizes. How does the number ofiterations depends on the problem size? What is the effect of increasing k?

Assignment 14.4. Make a recursive version of your program such that your programperforms a complete V-cycle. Take level = 2 the lowest, on which you solve the systemwith a direct solver. How does the number of iterations depend on the problem size?How does the (maximal) residual reduction factor per cycle depend on the grid size?

25

matlab assignments – lecture 1, fall 2016 › ... › assignments › all-matlab.pdf ·...

Documents