performance comparisons of preconditioned iterative...

IT 15 023

Examensarbete 30 hpMarch 2015

Performance comparisons of preconditioned iterative methods for problems arising in PDE-constrained optimization

Shiraz Farouq

Masterprogram i tillämpad beräkningsvetenskapMaster Programme in Computational Science

Teknisk- naturvetenskaplig fakultet UTH-enheten Besöksadress: Ångströmlaboratoriet Lägerhyddsvägen 1 Hus 4, Plan 0 Postadress: Box 536 751 21 Uppsala Telefon: 018 – 471 30 03 Telefax: 018 – 471 30 00 Hemsida: http://www.teknat.uu.se/student

Abstract

Performance comparisons of preconditioned iterativemethods for problems arising in PDE-constrainedoptimizationShiraz Farouq

The governing dynamics of simple and complex processes, whether physical,biological, social, economic, engineering, or even rather a mere figment of imagination,can be studied via numerical simulations of mathematical models. These models inmany cases can be thought to consist of one, or frequently, several coupled partialdifferential equations (PDEs). In many applications, the aim of such simulations is notonly to study the behavior of the underlying processes, but also to optimize orcontrol those in some optimal way. These are referred to as optimal controlproblems constrained by PDEs and are stated in the form of a constrainedminimization problem. The general framework under which such problems arestudied is referred to as PDE-constrained optimization.

In this thesis, we aim to solve three benchmark optimal control problems, namely, theoptimal control of the Poisson equation, the optimal control of theconvection-diffusion equation and the optimal control of the Stokes system. Numerically tackling these problems lead to a large optimality system with a saddlepoint structure. Systems with a saddle point structure are indefinite and in general,ill-conditioned, thus posing great challenges for iterative solvers seeking to find theirsolution. Preconditioning the optimality system is a possible strategy to deal with theissue. The main focus of the thesis is therefore to solve the resulting optimalitysystems with various preconditioners available in literature and compare theirefficiency. Moreover, additional challenges arise when dealing withconvection-diffusion control problems which we effectively deal by employing thelocal projection stabilization (LPS) scheme. Furthermore, Axelsson and Neytcheva in[40] proposed a preconditioner for efficiently solving large nonlinear coupledmulti-physics problems. We successfully apply this preconditioner to the first twobenchmark problems with promising results.

Tryckt av: Reprocentralen ITCIT 15 023Examinator: Jarmo RantakokkoÄmnesgranskare: Sverker HolmgrenHandledare: Maya Neytcheva

perhaps my readers will say that "I had perhapssomething to say, but did not know how to express it."

— Fyodor Dostoyevsky, The Idiot

1

Contents

1 Introduction 5

1.1 Major goal of the study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.2 Problem setting and available tools . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.3 Layout of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2 Mathematical framework for PDE-constrained optimization problems 10

2.1 The saddle point theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.1.1 The discretized saddle point system . . . . . . . . . . . . . . . . . . . . . . 11

2.2 Distributed optimal control of the Poisson equation . . . . . . . . . . . . . . . . . . 122.2.1 The weak formulation of the state equation . . . . . . . . . . . . . . . . . . 122.2.2 The finite element discretization of the state equation . . . . . . . . . . . . 132.2.3 The finite element discretization of the minimization problem . . . . . . . . 142.2.4 First order optimality conditions . . . . . . . . . . . . . . . . . . . . . . . . . 152.2.5 The reduced optimality system . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.3 Distributed optimal control of the convection-diffusion equation . . . . . . . . . . . 162.3.1 Weak formulation of the state equation . . . . . . . . . . . . . . . . . . . . . 172.3.2 Stabilization Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.3.2.1 The Petrov-Galerkin formulation: Streamline Upwind Petrov-Galerkin(SUPG) Stabilization . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.3.2.2 Local Projection Stabilization (LPS) scheme . . . . . . . . . . . . 192.3.3 Finite element discretization of the state equation . . . . . . . . . . . . . . . 212.3.4 The finite element discretization of the minimization problem . . . . . . . . 222.3.5 First order optimality conditions . . . . . . . . . . . . . . . . . . . . . . . . . 222.3.6 The reduced optimality system . . . . . . . . . . . . . . . . . . . . . . . . . 222.3.7 Solving the convection-diffusion optimality system using the LPS scheme . 23

2.4 Distributed optimal control of Stokes equations . . . . . . . . . . . . . . . . . . . . 232.4.1 Weak formulation of the state equation . . . . . . . . . . . . . . . . . . . . . 252.4.2 The finite element discretization of the state equation . . . . . . . . . . . . 262.4.3 Solvability of the Stokes system . . . . . . . . . . . . . . . . . . . . . . . . 262.4.4 The finite element discretization of the minimization problem . . . . . . . . 272.4.5 First order optimality conditions . . . . . . . . . . . . . . . . . . . . . . . . . 272.4.6 The reduced optimality system . . . . . . . . . . . . . . . . . . . . . . . . . 28

2.5 Error estimates for homogeneous distributed optimal control problems . . . . . . . 28

3 An overview on iterative methods 29

3.1 Iterative methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293.1.1 Relaxation scheme: Richardson iteration . . . . . . . . . . . . . . . . . . . 303.1.2 Chebyshev semi-iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.2 Projection Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303.2.1 General framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.3 Krylov subspace iterative methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 323.3.1 The Generalized Minimal Residual (GMRES) Method . . . . . . . . . . . . 323.3.2 The Minimal Residual (MINRES) Method . . . . . . . . . . . . . . . . . . . 34

2

4 Preconditioned Krylov subspace iterative methods and multigrid (MG) precondition-

ers 36

4.1 The idea of preconditioned Krylov subspace iterative methods . . . . . . . . . . . 374.2 Multigrid methods as a preconditioner . . . . . . . . . . . . . . . . . . . . . . . . . 38

5 Preconditioners for saddle point systems arising in PDE-constrained optimization

problems 39

5.1 Preconditioner construction techniques for saddle point systems . . . . . . . . . . 395.1.1 (Negative) Schur complement approximation . . . . . . . . . . . . . . . . . 395.1.2 Inexact Schur complement approximation . . . . . . . . . . . . . . . . . . . 405.1.3 Operator preconditioning with norms . . . . . . . . . . . . . . . . . . . . . . 415.1.4 Preconditioners based on interpolation theory . . . . . . . . . . . . . . . . . 425.1.5 Schur complement approximation using the commutator argument . . . . . 435.1.6 Operator factorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

5.1.6.1 Efficient solution for PF . . . . . . . . . . . . . . . . . . . . . . . . 445.2 Preconditioners for PDE-constrained optimization problems . . . . . . . . . . . . . 46

5.2.1 Preconditioners for the distributed optimal control problem constrained bythe Poisson equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465.2.1.1 Operator preconditioning with standard norms . . . . . . . . . . . 465.2.1.2 Operator preconditioning with non-standard norms . . . . . . . . . 475.2.1.3 Operator preconditioning with interpolation operator . . . . . . . . 475.2.1.4 Schur complement approximation . . . . . . . . . . . . . . . . . . 485.2.1.5 Operator factorization . . . . . . . . . . . . . . . . . . . . . . . . . 48

5.2.2 Preconditioners for the distributed optimal control problem constrained bythe convection-diffusion equation . . . . . . . . . . . . . . . . . . . . . . . . 495.2.2.1 (Negative) Schur complement approximation . . . . . . . . . . . . 505.2.2.2 Operator preconditioning with non-standard norms . . . . . . . . . 515.2.2.3 Operator factorization . . . . . . . . . . . . . . . . . . . . . . . . . 51

5.2.3 Preconditioners for distributed optimal control problem constrained by theStokes system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 525.2.3.1 Operator preconditioning with standard norms . . . . . . . . . . . 525.2.3.2 Operator preconditioning with non-standard norms . . . . . . . . . 535.2.3.3 Inexact Schur complement approximation . . . . . . . . . . . . . . 535.2.3.4 Mesh independent Schur complement approximation . . . . . . . 545.2.3.5 Parameter independent Schur complement approximation . . . . 555.2.3.6 Schur complement approximation using the commutator argument 56

5.3 Overview of spectrally equivalent numerical approximation of discrete differentialoperators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575.3.1 Approximation of the discrete differential operators arising in distributed op-

timal control of the Poisson equation . . . . . . . . . . . . . . . . . . . . . . 585.3.2 Approximation of the discrete differential operators arising in distributed op-

timal control of the convection-diffusion equation . . . . . . . . . . . . . . . 585.3.3 Approximation of the discrete differential operators arising in distributed op-

timal control of the Stokes system . . . . . . . . . . . . . . . . . . . . . . . 59

6 Numerical results 60

6.1 Distributed optimal control problem constrained by the Poisson equation . . . . . . 606.1.1 Block diagonal preconditioner pson Pbd1 , Pearson and Wathen . . . . . . . . 616.1.2 Block lower triangular preconditioner pson Pblt2 , Pearson and Wathen . . . . 626.1.3 Block diagonal preconditioner pson Pbd2 , Pearson and Wathen . . . . . . . . 626.1.4 Block diagonal preconditioner pson Pnsn , W. Zulehner . . . . . . . . . . . . . 636.1.5 Block factorization preconditioner pson PF, Axelsson and Neytcheva . . . . . 636.1.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

3

6.2 Distributed optimal control problem constrained by the convection-diffusion equation 676.2.1 Block diagonal preconditioner cd Pbd1 , Pearson and Wathen . . . . . . . . . 686.2.2 Block lower triangular preconditioner cd Pblt , Pearson and Wathen . . . . . 696.2.3 Block diagonal preconditioner cd Pbd2 , Pearson and Wathen . . . . . . . . . 696.2.4 Block diagonal preconditioner cd Pnsn , W. Zulehner . . . . . . . . . . . . . . 706.2.5 Block factorization preconditioner cd PF, Axelsson and Neytcheva . . . . . . 706.2.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

6.3 Distributed optimal control problem constrained by the Stokes system . . . . . . . 746.3.1 Block diagonal preconditioner stk Pbd1 , Rees and Wathen . . . . . . . . . . . 756.3.2 Block diagonal preconditioner stk Pcomm, Pearson . . . . . . . . . . . . . . . 766.3.3 Block diagonal preconditioner pson Pnsn , W. Zulehner . . . . . . . . . . . . . 776.3.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

7 Summary and conclusion 82

A Results for distributed convection-diffusion control problem for = 1/1500 88

B Local projection stabilization (LPS) 91

B.1 The double-glazing problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

4

Mad Hatter: Why is a raven like awriting-desk?

Lewis Carroll, Alice in Wonderland

1Introduction

PDE-constrained optimization problems appear is a variety of social, scientific, industrial, medicaland engineering applications including optimal control, optimal design and parameter identifica-tion. The general formulation of a PDE-constrained optimization problem is given by

miny,u

J (y, u)

s.t. C(y, u) = 0,(1.1)

where J represents the cost functional, C is a PDE-constraint, y is the state variable, and u isthe decision variable. The PDE-constraint, which basically models the underlying process thatneeds to be controlled, is referred to as the state equation. Depending on the nature of theobjective function J and decision variable u, the PDE-constrained optimization problem (1.1)assumes the nature of an optimal control, optimal design or parameter identification problem.Correspondingly, u is then referred to as a control, design or parameter identification variable. Foran overview on a wide range of PDE-constrained optimization problems, the reader is referred to[19, 20, 21].

In this thesis, we are concerned with the optimal control problems constrained by PDEs. As abrief introduction to the topic, consider a heating source u applied to the surface of a domain Ωto control its temperature y. Let us assume that the temperature distribution y in Ω is satisfied bythe following partial differential equation:

− ∆y = u in Ω. (1.2)

Now suppose that we want the domain Ω to acquire a target (desired) temperature distribution yfor some forcing term u. In other words, we want to control the heating source u, such that thedomain under consideration acquires a temperature y that is as close to the target temperature yas possible. This target tracking must be defined over some norm, which in our case is the L2(Ω)norm. The process can now be described in the form of a constrained minimization problem.Hence we write:

miny

J (y) =12y − y2

L2(Ω)

s.t. − ∆y = u,(1.3)

5

where u now represents the control variable.

Let us further generalize the problem by including boundary conditions. Let ∂Ω denote the bound-ary of Ω. Then

y = gD on ∂ΩD,∂y∂n

= gN on ∂ΩN ,

where ∂ΩD represents the Dirichlet boundary and ∂ΩN represents the Neumann boundary.∂y∂n

is the directional derivative in the outer normal direction of Ω. gD and gN are the Dirichlet andNeumann boundary data, respectively. Further,

∂Ω = ∂ΩD ∪ ∂ΩN , ∂ΩD ∩ ∂ΩN = ∅.

The above problem, in general, is ill-posed. Theory suggests adding the norm of the controlu along with the so called Tikhonov regularization parameter (also called the cost parameter)β > 0 to the cost functional. This makes the solution to the problem, well-defined. So we nowwrite:

miny,u

J (y, u) =12y − y2

L2(Ω) +12

βu|2L2(Ω)

s.t. − ∆y = u,


= gN on ∂ΩN ,

∂Ω = ∂ΩD ∪ ∂ΩN , ∂ΩD ∩ ∂ΩN = ∅.

(1.4)

One way to deal with this minimization problem and find its optimal solution is through the first

order optimality conditions, also known as the Karush-Kuhn-Tucker (KKT) conditions. This resultsin the involvement of another function, the Lagrange multiplier λ (also referred in literature as thedual or adjoint state variable). The solutions to the minimization problem for the state functiony and control function u are called the optimal state and the optimal control respectively. Theexistence and the uniqueness of the optimal solution is not discussed here and the reader isreferred to [21].

Numerically dealing with the PDE-constrained optimization problems require two steps, namely,discretization and optimization. Which step is taken first depends on which one of following twoapproaches is pursued:

• The discretize-then-optimize approach.

• The optimize-then-discretize approach.

In the first approach, we discretize the objective function and the state equation(s), formulate itsLagrangian and the corresponding first order optimality conditions, and then build an optimalitysystem to numerically solve the problem. In the second approach, we first formulate the La-grangian and its corresponding first order optimality conditions, discretize them, and then finallysolve the problem numerically. Many PDE-constrained optimization problems, especially wherethe PDE is self-adjoint, both approaches lead to the same discrete optimality system. However,certain PDE’s are not self adjoint, for example, the convection-diffusion equation. Heinkenschlosset al. in [23] showed that when streamline upwind Petrov-Galerkin (SUPG) stabilization1 is ap-plied to the optimal control of the convection-diffusion equation, the optimize-then-discretize and

1If a pure Galerkin approach is used, both approaches will result in a similar algebraic system. However, the solutionwill have unwanted oscillations.

6

discretize-then-optimize approaches can lead to different discrete algebraic2 systems, resultingin different solution outcomes. This also poses challenge in the context of designing efficientpreconditioners for such problems.

Whichever approach is taken, the resulting optimality system acquires a saddle point structure,given by:

Ax = b, (1.5)

where we solve for x withA =

A BT

1B2 −C

∈ Rn×n

andb ∈ Rn. (1.6)

The problem that we have discussed so far is called a distributed optimal control problem wherethe control u is distributed throughout the domain Ω. Another related problem is referred to asthe boundary control problem where the control only acts on the boundary so that u ∈ L2(∂Ω).For example, a typical Dirichlet boundary control problem is of the form

miny,u

12

Ω(y − y)2dx +

12

β

∂Ωu2ds

s.t. − ∆y = g in Ωy = u on ∂Ω.

(1.7)

Similarly, a Neumann boundary control problem is given by

miny,u

12

Ω(y − y)2dx +

12

β

∂Ωu2ds

s.t. − ∆y = g in Ω∂y∂n

= u on ∂Ω.

(1.8)

We consider only the distributed optimal control problems. Moreover, we have limited ourselvesto distributed optimal control problems without constraints on the control and/or the state. Suchproblems are considered in [52, 53, 54]. Furthermore, time-dependent optimal control problemsare also not considered here, and the interested reader is referred to [56, 57].

Remark (The parameter identification problem) In many other applications, reconstruction ofthe state of a dynamical system from measured data is required. These are referred to as the pa-rameter identification or the inverse problem. Examples include weather forecasting, oil reservoirmodeling and various models of cognitive motor control. Such problems can also be stated as aPDE-constrained optimization problem. While these problems are not the focus of this thesis, wehowever give a brief sketch of it.

Consider a thin membrane3 whose deflection y under some force u is defined by the Poissonequation

−∇ · σ∇y = u in Ω,

where σ(Ω) represents the unobservable material property (spatially varying) of the system, i.e.,the coefficient that we want to recover. Suppose we have some observable measurements y for

2We use the term algebraic here, for in the case of optimize-then-discretize, where the convection-diffusion equationis discretized using the SUPG, there is no finite dimensional problem for which the resulting system can be consideredas an optimality system, cf. [23].

3This example is taken from [44].

7

displacements caused under the influence of certain forces u. The task is to find σ for which theoutput state y of the system best matches the observations y. This task takes the form of anoptimization problem:

miny,σ

J (y, σ) =12y − y2 + βR(σ)

s.t. −∇ · σ∇y = u in Ω,(1.9)

where R(σ) is the regularization operator introduced to make the solution well-behaved.

We assumed that the dimension of solution space y and the observation space y are matched.In most situations, however, only few observations are available, which requires introducing anoperator Ψ that maps the solution space to the observation space. In this case, we write:

miny,σ

J (y, σ) =12Ψy − y2 + βR(σ)

s.t. −∇ · σ∇y = u in Ω.(1.10)

Interested readers can refer to, for instance, [44, 45, 46].

1.1 Major goal of the study

Due to their robustness, direct solvers are methods of choice when it comes to solving discretesaddle point systems. However, as the mesh size h gets smaller, the discrete saddle point systembecomes large. This results in increased computational requirements which direct solvers, ingeneral, find difficult to cope with. In these circumstances, due to their modest requirements forcomputer resources, iterative solution methods become the feasible method of choice. Thesemethods are usually based on projection processes onto Krylov subspaces. Examples includethe conjugate gradient (CG) method, the minimal residual (MINRES) method, and the generalizedminimal residual (GMRES) method. However, due to indefiniteness and, in general, unfavorablespectral properties of a large discrete saddle point system, iterative methods suffer from slow rateof convergence and in some cases a lack of it. In the context of PDE-constrained optimizationproblems, where some regularization parameter, say β approaches zero, the issue is furtherexacerbated. Therefore, efficient design of iterative solvers becomes necessary. One way toachieve efficiency in iterative methods is to use a suitable preconditioning strategy to improvethe spectral properties of the underlying saddle point system, see for instance, Axelsson andNeytcheva [1]. Further, saddle point systems possess a certain block structure, which has beenutilized for designing efficient preconditioners in various studies such as [17, 24, 26, 35].

We will discuss and implement various preconditioning strategies available in literature for thesaddle point systems arising in PDE-constrained optimization problems. Three benchmark prob-lems are considered: the distributed optimal control of the Poisson equation, the distributedoptimal control of the convection-diffusion equation and the distributed optimal control of theStokes system. We aim to summarize the proposed preconditioning techniques and test the mostpromising ones on the defined benchmark problems to provide a fair comparison of efficiency ofthe solvers both in terms of number of iterations as well as in required computing time.

1.2 Problem setting and available tools

We limit ourselves to two space dimensions. To discretize and create the matrices, the opensource FEM library deal.ii [58] is used. The package provides tools for constructing the dis-cretization meshes as well as for assembling the necessary FEM matrices. In addition, deal.ii

8

provides a number of preconditioners and iterative solvers, but also interface to other well estab-lished numerical algebra packages, such as Trilinos [59], Hypre [60] and PETSc [61].

1.3 Layout of the Thesis

This thesis is organized in the following manner:

• In Chapter 2, using the discretize-then-optimize approach, we describe the mathematicalmodel for our three benchmark distributed optimal control problems. We describe the pro-cess of discretization of the objective function and its constraints. Finally we describe theformulation of the first order optimality conditions, concluding that the process leads to asaddle point system.

• In Chapter 3, we discuss the basic theory of different iterative methods.

• In Chapter 4, we discuss the basic idea and motivation behind preconditioning iterativemethods based on Krylov subspaces. The topic of multigrid methods is also touched uponin this chapter.

• In Chapter 5, we give the literature overview of various techniques for constructing precondi-tioners for saddle point systems arising in different PDE-constrained optimization problems.

• In Chapter 6, we present and discuss the results of our numerical experiments.

• In Chapter 7, we summarize the results and draw conclusions.

9

"Curiouser and curiouser!" cried Al-ice


2Mathematical framework for PDE-constrained

optimization problems

The benchmark PDE-constrained optimization problems that we consider in our study are:

1. The distributed optimal control of the Poisson equation.

2. The distributed optimal control of the convection-diffusion equation.

3. The distributed optimal control of the Stokes system.

We start with the continuous formulation of the minimization problem. Each problem is then tack-led using the discretize-then-optimize approach. A starting point is to state the weak formulationfor the state equations and then use an appropriate finite element method for discretization1.The continuous minimization problems is then replaced by its discrete counterpart. Introducingthe Lagrange multiplier and solving for the optimality conditions yields an optimality system witha saddle point structure. In many cases, this optimality system can be reduced, more specif-ically, by eliminating the control u. The resulting system is then called the reduced optimalitysystem.

2.1 The saddle point theory

We follow the introductory discussion in [35, 5]. Let V and Λ be the Hilbert spaces with innerproducts (·, ·)V and (·, ·)Λ, associated with the norms · V and · Λ respectively. The mixedvariational formulation reads:

Find y ∈ V and λ ∈ Λ

a(y, v) + b(v, λ) = f (v) ∀v ∈ V,b(y, µ)− c(λ, µ) = g(µ) ∀µ ∈ Λ,

(2.1)

1Although we take the discretize-then-optimize approach here, note that for both the Poisson control problem as well asthe Stokes control problem, using the second approach, i.e., optimize-then-discretize, yield the same discrete optimalitysystem. For the case of convection-diffusion control problem, this is not the case, see Heinkenschloss et al. in [23]. Wewill further elaborate on the issue in section 2.3.

10

where a, b, c are bounded bilinear forms on V × V → R, V × Λ → R, and Λ × Λ → R respec-tively, whereas f ∈ V∗ and g ∈ Λ∗ are the linear forms; with V∗

h and Λ∗h being the dual space of

Vh and Λh respectively.

Let us consider the following assumptions:

1. a and c are symmetric, i.e.,

a(w, v) = a(v, w) ∀v, w ∈ V, c(r, µ) = c(µ, r) ∀µ, r ∈ Λ. (2.2)

2. a and c are nonnegative, i.e.,

a(v, v) ≥ 0 ∀v ∈ V, c(µ, µ) ≥ 0 ∀µ ∈ Λ. (2.3)

If condition (1) holds, then the functional

S(v, µ) =12

a(v, v) + b(v, µ)−12

c(µ, µ)− f (v)− g(µ) (2.4)

is a convex function of v ∈ V and a concave function of µ ∈ Λ; more on this can be found in[6]. Such a functional S is referred to as a saddle point function. If additionally, condition (2) alsoholds, then (y, µ) is the solution to (2.1) if and only if (y, µ) is a saddle point of S , i.e.,

S(y, µ) ≤ S(y, λ) ≤ S(v, λ). (2.5)

Let us now reformulate (2.1) as a (non-mixed) variational problem: Let X be the product spaceV × Λ equipped with inner product ((y, λ), (v, µ))X = (y, v)V + (p, µ)Λ with norm (y, λ)X =((y, λ), (y, λ))X. Further, let X∗ be the dual space of X. Then the variational formulation

reads:

Find x1 = (y, λ) ∈ X such that

B(x1, x2) = F (x2), ∀x2(v, µ) ∈ X, (2.6)

whereB(z, x2) = a(w, v) + b(v, r) + b(w, µ)− c(r, µ), F (x2) = f (v) + g(µ), (2.7)

for x2 = (v, µ), z = (w, r).

Associating a linear operator A ∈ L(X, X∗) to B, we write

Ax1, x2X∗ ,X = B(x1, x2). (2.8)

Finally, using the operator notation we rewrite (2.6), which now reads:

Ax = F , in X∗. (2.9)

The existence and uniqueness of a solution for (2.9) and (2.6) follows from the book by Babushkaand Aziz [2]. In the special case of c(·, ·) = 0, existence and uniqueness follows the classicalresult by Brezzi [3].

2.1.1 The discretized saddle point system

The discretized version of (2.9) leads to a large saddle point system represented by

Ax = F , 2 (2.10)2Instead of writing Ahxh = Fh, we consider it convenient to reuse the notation of the continuous case (2.9).

11

or A BT

1B2 −C

x1x2

=

fg

, (2.11)

where f ∈ Rn1 , g ∈ Rm1 , A ∈ Rn1×n1 , B1, B2 ∈ Rm1×n1 , C ∈ Rm1×m1 , m1 ≤ n1.

Let us consider the following conditions:

1. A is symmetric, i.e., A = AT.

2. The symmetric part of A, i.e., H ≡12(A + AT) is positive semidefinite.

3. B1 = B2 = B or B is symmetric, i.e., B1 = BT2 .

4. C is symmetric, i.e., C = CT and positive semidefinite.

If all the above conditions are satisfied then A is symmetric and indefinite. Indefiniteness impliesA has both positive and negative eigenvalues. If either A = AT(condition 1) or B1 = B2(condition3), or both are not satisfied, then A is non-symmetric, in which case A is referred to as the gen-eralized saddle point system3. Moreover, the discretized version of X introduces an additionalparameter into the picture, i.e., the mesh size h. As h decreases, the saddle point system be-comes large, while its spectral properties deteriorate. In practice, larger saddle point systemsare typically ill conditioned. Further discussion on saddle point problems can be found with moredetail in [5].

2.2 Distributed optimal control of the Poisson equation

Consider the distributed optimal control problem constrained by the Poisson equation:

miny,u

J (y, u) =12y − y2

L2(Ω) +12

βu2L2(Ω),

−∆y = u in Ω,


= gN on ∂ΩN ,

∂Ω = ∂ΩD ∪ ∂ΩN , ∂ΩD ∩ ∂ΩN = ∅.

(2.12)

where y is the state of the system, y is the desired state of the system to be achieved and β > 0is a regularization parameter. Further, ∂Ω represents the boundary of the domain Ω.

2.2.1 The weak formulation of the state equation

The starting point for finite element discretization is to find the variational or weak formulation ofthe equations involved. Let us first start with the state equation, which in this case is the Poissonequation. For a weak formulation, we need an appropriate set of test functions v. Further, weneed a natural functional space, where v and the weak solution y can naturally exist. So ifΩ ⊂ R2, then this homely space can be defined as a Sobolev space, i.e.,

H1(Ω) =

y : Ω → R| y,

∂y∂x1

,∂y∂x2

∈ L2(Ω)

,

3Linearized Navier-Stokes equations is an example.

12

with the corresponding solution and test spaces given by

H1E = y ∈ H

1(Ω)|y = gD on ΩD,

H1E0

= v ∈ H1(Ω)|v = 0 on ΩD.

(2.13)

Note that while the Dirichlet condition is part of the solution space H1E, it vanishes in the test

space H1E0

, i.e., the test functions v takes zero value on the Dirichlet boundary. The Neumannboundary condition, in contrast, is not restricted in the solution or the test spaces. The weakformulation now reads:

Find y ∈ H1E such that

a(y, v) = l(v) ∀v ∈ H1E0

, (2.14)

where the bilinear and linear forms a(·, ·) and l(·) are given by

a(y, v) =

Ω∇y.∇v

l(v) =

Ωvu +

∂ΩNvgN .

2.2.2 The finite element discretization of the state equation

We use the Galerkin approach for finite element discretization. Let Yh0 ⊂ H1

E0be the n-dimensional

vector space of test functions spanned by φ1, ..., φn. Let YhE ⊂ H1

E be the vector space for trialfunctions having basis φ1, ..., φn. We extend the trial basis by φn+1, ..., φn+n∂

to satisfy theDirichlet boundary data on ΩD. The finite element approximation yh ∈ Yh

E is uniquely determinedby the vector of coefficients y = y1, ..., yn. Thus

yh =n

∑i=1

yiφi +n+n∂

∑i=n+1

yiφi. (2.15)

Note that the space YhE is constructed such that the trial functions (shape functions) in (2.15)

coincides the appropriate test functions in YhE0

. This is referred to as the Galerkin method. Usingthis formulation for our state equation, the finite element discretization reads:

Find yh ∈ YhE such that

a(yh, vh) = l(vh) ∀v ∈ YhE0

(2.16)

where the bilinear and linear forms a(·, ·) and l(·) are given by

a(yh, vh) =

Ω∇yh.∇vh,

l(vh) =

Ωvhuh +

∂ΩNvhgN .

We discretize the control u using the trial space. Let ψi be the set of basis functions such thatspanψ1, ..., ψm = Uh ⊂ L2(Ω). We note that ψi may be different than φi. The Galerkin finiteelement approximation of the control u is

uh =m

∑i=1

uiψi, (2.17)

where the discretized control uh is uniquely determined by the vector of coefficients u = u1, ..., un.

13

The Galerkin finite element approximation of the state equation can also be written as a systemof algebraic equations, i.e.,

Ky = Qu + d. (2.18)

with

K = ki,j ∈ Rn×n, ki,j =

Ω∇φj.∇φi,

Q = qi,j ∈ Rn×m, qi,j =

Ωφiψj,

d = dii=1,...n, di =

∂ΩNφigN −

n+n∂

∑j=n+j

yj

Ω∇φi.∇φj,

where K is the stiffness matrix, Q is the mass matrix and d is a vector containing the boundaryterms.

2.2.3 The finite element discretization of the minimization problem

We replace the continuous minimization problem (2.12) with its discrete counterpart, i.e.,

minyh ,uh

J (yh, uh) =12yh − y2

L2(Ω) +12

βuh2L2(Ω) (2.19)

Ω∇yh.∇vh =

Ωvhuh ∀v ∈ Yh

E0. (2.20)

Now,

yh − y2 =

Ω(yh − y)2

= ∑i

∑j

yiyj

Ωφiφj − 2 ∑

jyj

Ωφj y +

Ωy2

= yT Myy − 2yTb + C,

whereMy = my

i,j ∈ Rn×n, myi,j =

Ωφiφj

is the state mass matrix,b = bi,j ∈ Rn, bi =

Ωyφi,

and C is some constant.

Moreover,uh

22 = uT Muu,

whereMu = mu

i,j ∈ Rm×m, mui,j =

Ωψiψj

is the control mass matrix.

Further, we know that (2.20) can we written in the form (2.18), i.e.,

Ky = Qu + d. (2.21)

14

Thus, we have the minimization problem

miny,u

J (y, u) =12

yT Myy − yTb +12

βuT Muu

s.t. Ky = Qu + d.(2.22)

which is an equivalent representation of minimization problem (2.19).

Finally, we introduce the Lagrange multiplier λ. The finite element discretization of λ is doneusing the functional space of the state y. So we write 4

λh =n

∑i=1

λiφi +n+n∂

∑i=n+1

λiφi. (2.23)

The discretized Lagrange multiplier λh is uniquely determined by the vector of coefficients λ =λ1, ..., λn. λ is also called the adjoint state variable since it satisfies

KTλ = b − Myy. (2.24)

2.2.4 First order optimality conditions

We write the discrete Lagrangian function:

L(y, u, λ) =12

yT Myy − yTb +β

2uT Muu − λT(Ky − Qu − d). (2.25)

The first order optimality conditions or Karush-Kuhn-Tucker (KKT) conditions are obtained bydifferentiating L with respect to the state y, control u and the Lagrange multiplier λ resulting inthe so called adjoint, gradient and state equations, i.e.,

∇yL(y∗, u∗, λ∗) = Myy∗− b + KTλ∗ = 0 (2.26)

∇uL(y∗, u∗, λ∗) = βMuu∗− QTλ∗ = 0 (2.27)

∇λL(y∗, u∗, λ∗) = Ky∗− Qu∗

− d = 0. (2.28)

The resulting optimality system is by

AF

yuλ

≡

My 0 KT

0 βMu −QK −Q 0

yuλ

=

b0d

. (2.29)

If the state y and the control u are discretized using the same basis, then M = My = Mu = Qand the stiffness matrix K is symmetric, i.e., K = KT. We can then write

AF

yuλ

≡

M 0 KT

0 βM −MK −M 0

yuλ

=

b0d

. (2.30)

Clearly (2.30) has a saddle point structure, i.e., it has the form

AF =

A BT

B −C

4Note however that ∑n+n∂i=n+1 λiφi = 0 follows from [55].

15

whereA =

M 00 βM

, B =

K −M

and C = 0.

Remark Under the assumption that the test and trial spaces are the same, the stiffness matrix Kis symmetric, i.e., K = KT. Then as discussed in (Chapter 4, [18]), for any vector v in Rn,

vKv ≥ 0, (2.31)

implying K is at least semi positive-definite. Moreover, if ΩD = ∅ then K is positive definite. Incase of pure Neumann boundary conditions, K is only positive semi-definite, and for the solutionof (2.18) to exist, the relation

ui =

Ωφiu +

∂ΩφigN = 0 (2.32)

must hold.

Further, for any vector v ∈ Rn, there holds

vMv ≥ 0, (2.33)

i.e., the mass matrix M is symmetric positive definite.

2.2.5 The reduced optimality system

If the state y, control u and the adjoint λ are discretized using the same finite element spaces,then it is possible to eliminate the gradient equation using the relation

u =1β

λ.

This gives us

AR

yλ

≡

M KT

K −1β

M

yλ

=

bd

. (2.34)

Note that AR also has the saddle point structure, i.e., it has the form

AR =

A BT

B −C

whereA = M, B = K and C =

1β

M.

2.3 Distributed optimal control of the convection-diffusion equa-

tion

Convection-diffusion processes help describe physical processes involving both diffusion andtransport. One such example is the spread and transport of a pollutant with wind. The process isdescribed through the following equation:

−ε∆y + (w ·∇)y = u in Ω,


= gN on ∂ΩN ,

∂Ω = ∂ΩD ∪ ∂ΩN , ∂ΩD ∩ ∂ΩN = ∅,

(2.35)

16

where w is some vector field (for instance, direction of the wind) and gD and gN are the givenboundary data. Further, we assume that w is divergence free, i.e., ∇ · w = 0. The scalarquantity ε (for instance, viscosity) satisfies 0 < ε 1 and the smaller the value it takes, the moreconvection-dominated the problem becomes.

Note that when w = 0, the problem is not self adjoint. This simply means that if we define theconvection-diffusion operator by

L = −∇2 + w∇ (2.36)

then

Ω(Lu)v =

Ωu(Lv). (2.37)

The consequence to this is that the coefficient matrix derived through discretization is no longersymmetric. Further, when it comes to solving convection-diffusion control problems, discretize-

then-optimize and optimize-then-discretize, yield different outcomes. One method to mitigatethe issue is through employing the subgrid modeling and local projection stabilization (LPS)schemes, as discussed in [30, 29, 28].

In a convection-diffusion control problem, the aim is to find a control u such that the state yattains a desired state y. The problem can be defined as a minimization of the cost functionalwith a convection-diffusion PDE stated as a constraint. The problem can be formulated as:

miny,u

J (y, u) =12y − y2

L2(Ω) +12

βu2L2(Ω)

s.t. − ε∆y + (w ·∇)y + cy = u in Ω


= gN on ∂ΩN ,

∂Ω = ∂ΩD ∪ ∂ΩN , ∂ΩD ∩ ∂ΩN = ∅.

(2.38)

2.3.1 Weak formulation of the state equation

The state equation in this case is the convection-diffusion equation. Before we define the weakformulation, let us first define the space where trial and test functions will live. So if Ω ⊂ R2, thenthis space can be defined by the Sobolev space

H1(Ω) =

y : Ω → R| y,

∂y∂x1

,∂y∂x2

∈ L2(Ω)

,

whereas its residents, i.e., the test and trial spaces are

H1E = y ∈ H

1(Ω)|y = gD on ΩD

H1E0

= v ∈ H1(Ω)|v = 0 on ΩD.

The weak formulation of the convection-diffusion equation reads:

Find y ∈ H1E such that

a(y, v) = l(v) ∀v ∈ H1E0

. (2.39)

The bilinear form a(· , · ) on the left side is given by

a(y, v) =

Ω∇y.∇u +

Ω(w.∇y)v. (2.40)

Similarly, the linear form l(· ) on the right side is given by

l(v) =

Ωuv +

∂ΩNgNv. (2.41)

17

2.3.2 Stabilization Techniques

The finite element discretization of convection-diffusion equation is not straightforward. As dis-cussed in [18], when the convection term dominates, the discrete solution to the problem isinaccurate due to the presence of non-physical oscillations. To further understand the issue, wequantify the effect of convection and diffusion terms by the Peclet number. Let the velocity w berepresented by

w = C0w∗, (2.42)

where C0 is a positive constant and w∗ is normalized to have a value of one in some norm. Thenthe Peclet number is given by

P =C0L

, (2.43)

where L is the characteristic length scale of the problem.

So if P ≤ 1, the problem is diffusion dominated and the solution may be within acceptable rangeof accuracy, nonetheless, it is best to have P → 0.

If P 1, the solution can have a steep gradient requiring a very fine mesh to correctly resolveit. If the mesh is not fine, the standard Galerkin finite element discretization method will not workdesirably.

Different approaches have been proposed to captivate the demon of unwarranted oscillations inprocesses governed by a convection-diffusion equation. One such method is the streamline up-wind Petrov-Galerkin (SUPG) method. However, using SUPG in the context of optimal control ofthe convection-diffusion equation, optimize-then-discretize and discretize-then-optimize result instructurally different algebraic systems, leading to different solution outcomes and precondition-ing strategies. In order to overcomes this issue, Becker and Vexler in [29] proposed a method re-ferred to as the local projection stabilization (LPS) scheme, which borrows some ideas from sub-grid modeling method discussed in [30, 31]. Pearson and Wathen in [28] used the LPS schemeresulting in an optimality system that has same structure whether optimize-then-discretize ordiscretize-then-optimize is used.

2.3.2.1 The Petrov-Galerkin formulation: Streamline Upwind Petrov-Galerkin (SUPG) Sta-

bilization

In this formulation, the trial and test spaces are different, i.e., the trial space is given by YhE ⊂ HE

1 ,and the test space is given by the vectors of the form

vh + δw.∇vh, (2.44)

where vh ∈ VhE0

and δ is a constant that needs to be calculated apriori. Using this formulation, thebilinear form apg(·, ·) is given by

apg(yh, vh) =

Ω∇yh.∇uh +

Ω(w.∇yh)vh + δ

Ω(w.∇yh)(w.∇vh)− δ

Ω(∇2yh)(w.∇vh).

(2.45)In case of linear and bilinear elements, the higher order term is zero. Thus

apg(yh, vh) =

Ω∇yh.∇uh +

Ω(w.∇yh)vh + δ

Ω(w.∇yh)(w.∇vh). (2.46)

Similarly, the linear form lpg(·) is given by

lpg(vh) =

Ωuvh +

∂ΩNgNvh + δ

Ωu(w.∇vh). (2.47)

18

The Petrov-Galerkin finite element method, also called Streamline Upwind Petrov-Galerkin (SUPG)

scheme, reads:

Find yh ∈ VhE such that

apg(yhvh) = lpg(vh) ∀vh ∈ VhE0

. (2.48)

To find the value of δ, we let δk to be defined locally on individual elements. As suggested in[18, 24], we have

δk =

hk2w2

1 −

1Pk

h

if Pk

h > 1

0 otherwise(2.49)

whereP

kh =

w2hk2

and hk is measured in the direction of the wind. For example, as described in [24], if we have arectangular domain with sides given by hx and hy and w = [cosθ, sinθ], then

hk = min

hx|cosθ|

,hy

|sinθ|

.

2.3.2.2 Local Projection Stabilization (LPS) scheme

We follow with the discussion on LPS scheme in [29]. Consider a two-dimensional mesh THconsisting of open cells which in the present case consist of quadrilaterals.

Figure 2.1: TH with 4 cells.

Using uniform refinement on mesh TH gives us a refined mesh Th. For instance, if TH consistsof 4 cells, a uniform refinement of it will create 16 cells on Th. Each refined cell on TH creates 4cells in Th, referred to as a patch P; so we will have 4 patches in this case.

19

Figure 2.2: Refined mesh Th with 16 cells and 4 patches (each patch portrayed by a differentcolor) with representing the support on the patches.

LPS uses standard finite element discretization with stabilization based on local projections. Nowthe sequel requires us to define an L2(P) orthogonal projection operator5

Ph : L2(Ω) → VconstH , (2.50)

on the patches of the domain, with VconstH being a cell-wise6 constant function on the patches.

Further, this operator satisfies the following approximation and stability properties on the patches,cf. [29]:

PhvL2(P) ≤ cvL2(P) ∀v ∈ L2(P). (2.51)

v − PhvL2(P) ≤ ch∇vL2(P) ∀v ∈ H1(P). (2.52)

We now introduce a positive stabilization parameter δ associated with a bilinear symmetric stabi-lization form τδ

h : Vh × Vh → R given by:

τδh (uh, vh) = δ(w ·∇uh − Ph(w ·∇uh)× (w ·∇vh − Ph(w ·∇vh))). (2.53)

The stabilization parameter δ is defined locally on individual elements and depends on the Pecletnumber

Pkh =

hkw

.

Following from Becker and Vexler in [29], we have

δk =

hkw

, if Pkh ≥ 1,

0, otherwise.(2.54)

An Algorithm to solve the convection-diffusion equation based on the LPS scheme is discussedin Appendix B.

5Note that orthogonal projection operator gives a good average approximation of a function, compared to an interpola-tion operator. However, both these operators have difficulty in approximating highly oscillatory or discontinuous functions.

6There are four cells in a patch in our considered example.

20

Remark The L2(P) orthogonal projection is computed on the gradient of the approximated solu-tion uh on each patch P ∈ TH given by

Ph(w ·∇uh) → MPh ξ = (w ·∇uH , vH), (2.55)

where MPh is the standard mass matrix assembled on TH and ξ ∈ VconstH is a vector of cell-wise

constant functions on patches. The idea here is to capture sharp oscillations in the gradient ofapproximated solution uH on the patches. These captured oscillations ξ are then interpolated tothe fine mesh Th and added to the computed solution uh.

2.3.3 Finite element discretization of the state equation

Let Vh0 ⊂ H1

E0be the n-dimensional vector space of test functions spanned by φ1, ..., φn. Let

YhE ⊂ H1

E be the vector space for trial functions spanned by φ1, ..., φn and extended further byφn+1, ..., φn+n∂

to satisfy the Dirichlet boundary data on ΩD. The finite element approximationyh ∈ Yh

E is uniquely determined by the vector of coefficients y = y1, ..., yn. So we write

yh =n

∑i=1

Yiφi +n+n∂

∑i=n+1

Yiφi. (2.56)

We can discretize the control u using the trial space. Let φi be the set of basis functions such thatspanφ1, ..., φn := Uh ⊂ L2(Ω). Then the finite element approximation of the control u is givenby

uh =m

∑i=1

uiψi, (2.57)

where uh is uniquely determined by the vector of coefficients u = u1, ..., un.

The finite element approximation of the state equation can be written as a system of algebraicequations given by

Fy = Mu + d, (2.58)

where F = K + N + T. Further,

M = miji,j=1,...,n, mij =

Ωφiφj,

K = kiji,j=1,...,n, kij =

Ω∇φi.∇φj,

N = niji,j=1,...,n, nij =

Ω(w∇φj).φi,

T = τδh,i,ji,j=1,...,n, τδ

h,i,j = δ

Ω(w ·∇φi − Ph(w ·∇φi)× (w ·∇φj − Ph(w ·∇φj))),

d = di=1,...n, di = −

n+n∂

∑j=n+j

yj

Ω∇φi.∇φj.

Here, M is the mass matrix, K is the stiffness matrix, N contains the convection term (skew-symmetric), d contains the boundary terms (assuming gN = 0) and T is the LPS stabilizationterm.

21


We replace the continuous minimization problem (2.38) with its discrete counterpart:

miny,u

12

yT My − yTb +12

βuT Mu

s.t. Fy = Mu + d,(2.59)

where b is given byb = bii=1,...n,

ΩyφidΩ.

We now introduce the Lagrange multiplier λ, discretizing it using the functional space of state y.So we obtain

λh =n

∑i=1

λiφi +n+n∂

∑i=n+1

λiφi. (2.60)

The discretized Lagrange multiplier (adjoint state) λh is uniquely determined by the vector ofcoefficients λ = λ1, ..., λn.


We formulate the discrete Lagrangian function given by

L(y, u, λ) =12

yT My − yTb + βuT Mu − λT(Fy − Mu + d). (2.61)

By definition, the optimal point of L is obtained when its partial derivatives with respect to y, u,and λ are set to zero. This leads to the following optimality system, i.e.,

AF

yuλ

≡

M 0 FT

0 βM −MF −M 0

yuλ

=

b0d

. (2.62)

Note the difference between the systems (2.30) and (2.62). Whereas K is a discrete Laplacianand thus symmetric positive definite (spd), F is non-symmetric as it contains the discrete coun-terparts of convection (skew-symmetric) and stabilization terms.

We can see that (2.62) has a saddle point structure, i.e., it has the form

AF =

A BT

B 0

,

whereA =

M 00 βM

, B =

F − M

and C = 0.


If the state y, control u and the adjoint λ are discretized using the same finite element spaces,then it is possible to eliminate the gradient equation in (2.62) using the relation

u =1β

λ,

22

which gives us the reduced optimality system

AR

yλ

≡

M FT

F −1β

M

yλ

=

bd

. (2.63)

Note that AR also has a saddle point structure, i.e., it has the form

AR =

A BT

B −C

,

whereA = M, B = F, and C =

1β

M.

2.3.7 Solving the convection-diffusion optimality system using the LPS

scheme

An Algorithm to solve for the convection-diffusion equation using the LPS scheme is describedin Appendix B. We now use it to sketch the solution method for the reduced optimality system7

(2.63). In the first step, we solve for the system

AR

yλ

≡

M (K + N + Υ)T

(K + N + Υ) −1β

M

yλ

=

bd

, (2.64)

for y and λ, using fixed8 number of iterations. Here,

Υ = υδh,i,ji,j=1,...,n, υδ

h,i,j = δ

Ω(w ·∇φi)(w ·∇φj). (2.65)

In the next step, we solve for

AR

yλ

≡

M (K + N + Υ)T

(K + N + Υ) −1β

M

yλ

=

b + δξλd + δξy

. (2.66)

where the vectors ξy and ξλ are computed with

Mξy = (w.∇y, φi), Mξλ = (w.∇λ, φi) (2.67)

respectively.

2.4 Distributed optimal control of Stokes equations

Consider the motion of an incompressible fluid satisfying the Stokes system in some domainΩ. Let y be the vector field of the velocity and p be a scalar field representing the pressure.Let u be the forcing field that arises through the applications of some external force such as anelectromagnetic field. Thus we have the Stokes equations given by

−∆y +∇p = u in Ω (2.68)7The steps for the full optimality system are similar.8In our numerical experiments, we used one iteration.

23

∇ ·y = 0 in Ω (2.69)

where (2.69) is referred to as the incompressibility constraint. Introducing the Dirichlet and Neu-mann boundary conditions we have


−np = gN on ∂ΩN ,

where n is the outer normal to the boundary and∂y∂n

is the directional derivative in the normaldirection of Ω. Further,

∂Ω = ∂ΩD ∪ ∂ΩN , ∂ΩD ∩ ∂ΩN = ∅.

In order for a unique velocity solution to exist, we must have a nontrivial boundary data associatedwith ΩD, i.e.,

∂ΩDds = 0. (2.70)

If the velocity field is specified on the whole boundary, i.e., ∂ΩD = ∂Ω and ∂ΩN = 0, then thepressure p is unique only up to a constant (referred to as the hydrostatic pressure level).

Following the discussion in [18] we note that for a general flow, the Dirichlet boundary can besubdivided into segments according to their relation with the normal components of the imposedvelocity field, i.e.,

∂Ω+ = x ∈ ∂Ω | gD.n > 0, the outflow boundary,∂Ω0 = x ∈ ∂Ω | gD.n = 0, the characteristic boundary,

∂Ω− = x ∈ ∂Ω | gD.n < 0, the inflow boundary.(2.71)

Then the following compatibility condition must hold

∂Ω−

gD.n =

∂Ω+

gD.n = 0. (2.72)

The above condition is the law of conservation of mass flow, i.e., the volume of fluid entering thedomain must be equal to the volume of fluid leaving the domain. This condition must be satisfiedfor the solution of Stokes equations to exist. Further, this condition is also associated with thenon-uniqueness of pressure, as we note that the incompressibility constraint does not containany pressure term.

Here we only discuss the case when gD.n = 0 is specified everywhere on ∂Ω. This type of flowis called an enclosed flow and it automatically satisfies (2.72). When

∂Ω−

gD.n = 0, the problemis referred to as inflow/outflow type. For technical details on such problems, the interested readerin referred to [18].

We now solve the optimization problem concerning the attainment of desired states y and p, i.e.,we find u such that the velocity y and pressure p are close to the desired states. The problemcan be defined as a minimization of the cost functional with Stokes equations as the constraint.This is stated as follows:

miny,p,u

12y − y2

L2(Ω) +12

αp − p2L2(Ω) +

12

βu2L2(Ω)

s.t. − ∆y +∇p = u in Ω∇ ·y = 0 in Ω

y = gD on ∂Ω.

(2.73)

24

where α is the pressure regularization parameter. Pressure regularization, however, is a difficultproblem and some of the reasons for that has been discussed in [16]. In most of the literature wefind distributed Stokes control problem without a pressure regularization, i.e.,

miny,u

12y − y2

L2(Ω) +12

βu2L2(Ω)

s.t. − ∆y +∇p = u in Ω∇ ·y = 0 in Ω

y = gD on ∂Ω.

(2.74)

2.4.1 Weak formulation of the state equation

Given Ω ∈ R2, the solution and test spaces for the velocity field are given by

H1E = y ∈ H1(Ω)|y = gD on ΩD

H1E0

= v ∈ H1(Ω)|v = 0 on ΩD(2.75)

whereH

1(Ω) =

y : Ω → R| y,

∂y∂x1

,∂y∂x2

∈ L2(Ω)

.

The appropriate solution space for p is L2(Ω).

The weak formulation for the Stokes equation reads as follows:

Find y ∈ H1E and p ∈ L2(Ω) such that

Ω∇y : ∇v −

Ωp.∇v =

Ωu.v ∀v ∈ H1

E0(2.76)

Ωq∇y = 0 ∀q ∈ L2(Ω), (2.77)

where∇y : ∇v = ∑

i,j

∂yi∂xj

∂vi∂xj

(2.78)

is the component-wise scalar product. Further, a suitably chosen trial space q ∈ L2(Ω) ensuresthat (2.77) is finite.

However, it is required to show that the above weak solutions are uniquely defined. While it iseasier to show this for velocity, in case of pressure it is required to show

Ωp∇y = 0 ∀y ∈ H1

E0=⇒

p = constant, i f ∂Ω = ∂ΩDp = 0, otherwise

. (2.79)

For enclosed flow, the following inf-sup condition due to Brezzi [3] must hold for the case ofp = constant

infq =constant

supv =0

|(q,∇.v)|v1,Ωq0,Ω

≥ γ > 0. (2.80)

This condition is a sufficient condition to show that the pressure is unique up to a constant. Moredetails on this can be found in [18].

25

2.4.2 The finite element discretization of the state equation

Let φ : j = 1, ..., ny + n∂ be the vector valued basis function belonging to the finite dimensionalvelocity subspace, i.e., Yh

E = spanφj ⊂ H1E0

. Similarly, let ψk : k = 1, ..., np be the finiteelement basis function for the pressure space, i.e., Qh

E = spanψk ⊂ L2(Ω). Further, wediscretize the control u using the velocity space. Then the Galerkin finite element discretizationof y, p and u can be written as:

yh =ny+n∂

∑i=1

yiφi, ph =np

∑k=1

pkψk, uh =ny

∑i=1

uiφi. (2.81)

This yields the following linear system

Ky BT

B 0

yp

=

My0

u +

fg

, (2.82)

where

Ky = ki.j ∈ Rny×ny , ki.j =

Ω∇φi : ∇φj

My = myi.j ∈ Rny×ny , my

i.j =

Ωφi.φj

B = bk,j ∈ Rnp×ny , bk,j = −

Ωψk∇.φj

f = fi ∈ Rny , fi = −

ny+n∂

∑j=ny+1

yj

Ω∇φi : ∇φj

g = gk ∈ Rnp , gk =ny+n∂

∑j=ny+1

yj

Ωψi∇.φj.

The subscript y for M and K represent the standard notation for Gram matrices resulting fromvector valued basis functions. Note that ∇φi : ∇φj represents the component wise scalar product.The coefficients yjj=nv+1,...,nv+n∂

interpolates the boundary data gD.

2.4.3 Solvability of the Stokes system

Given that the incompressibility constraint does not involve the pressure term, the finite elementapproximation of Stokes equations is not straightforward. Analyzing (2.82) we have

Kyy + BTp = 0

By = 0.

Multiplying the first equation by yT and the second by pT gives

yTKyy + yT BTp = 0

pT By = 0.

This implies that yTKyy = 0, and since Ky is positive definite, y=0, and the system (2.82) isuniquely solvable with respect to velocity. Now substituting y = 0 in (2.83) we have

BTp = 0.

26

This implies that the pressure solution is unique up to the null space of the matrix BT. Therefore,for unique solvability, (2.82) must satisfy the discrete analogue of (2.80) referred to as the LBB(Ladyzhenskaya-Babuska-Brezzi) condition [4]. Thus for every pk ∈ p, there must be a yi ∈ y,such that9

yT BTp ≥ γ

yTKyy

pT Mpp, γ > 0, (2.83)

where Mp is the pressure mass matrix and γ is a constant independent of mesh size h.

The main point here is that the finite element space should be chosen such that null(BT) = 1in case of enclosed flow and null(BT) = 0 otherwise. One way to satisfy the condition is to usebi-quadratic approximation for the velocity space and a quadratic approximation for the pressurespace. This is called the Q2-Q1 or the Taylor-Hood approximation method. Since the spacesfor velocity and pressure are approximated independent of each other, the discretization of thesystem is referred to as the mixed approximation.


Replacing the continuous minimization problem with its discrete counterpart leads to

miny,p,u

12yT Myy −vTb +

12

βuT Myu, (2.84)

whereb = bi ∈ Rny , bi =

Ωyφi

andd = di ∈ Rnp , di =

Ωpφi.

Finally, we introduce the Lagrange multipliers λ and µ. We discretize λ using the functional spaceof the state y and µ using the functional space of pressure. So we have

λh =ny

∑i=1

λiφi, µh =np

∑k=1

µiψk. (2.85)

The discretized Lagrange multipliers λh and µh are then uniquely determined by the vector ofcoefficients λ = λ1, ..., λny and µ = µ1, ..., µnp.


Solving for the critical point leads to the discrete optimality system

AF

ypuλµ

≡

My 0 0 Ky BT

0 0 0 B 00 0 βMy −MT

y 0Ky BT −My 0 0B 0 0 0 0

ypuλµ

=

b00fg

. (2.86)

Again, we have a saddle point structure with the form

AF =

A BT

1B1 0

,

9http://ocw.mit.edu/courses/mathematics/18-086-mathematical-methods-for-engineers-ii-spring-2006/readings/am65.pdf

27

http://ocw.mit.edu/courses/mathematics/18-086-mathematical-methods-for-engineers-ii-spring-2006/readings/am65.pdf

http://ocw.mit.edu/courses/mathematics/18-086-mathematical-methods-for-engineers-ii-spring-2006/readings/am65.pdf

where

A =

My 0 00 0 00 0 βMy

, B1 =

Ky BT −MyB 0 0

and C = 0.


Again, if the state y, the control u and the adjoint λ are discretized using the same finite elementspaces, then it is possible to eliminate the gradient equation using the relation

u =1β

λ.

This gives the reduced optimality system

AR

yλµp

≡

My Ky 0 BT

Ky −1β

My BT 0

0 B 0 0B 0 0 0

yλµp

=

b0fg

. (2.87)

We clearly see that the reduced optimality system also has a saddle point structure, i.e., it hasthe form

AR =

A BT

1B1 −C

,

where

A =

My Ky

Ky −1β

Mp

, B1 =

0 BB 0

and C =

0 00 0

.

2.5 Error estimates for homogeneous distributed optimal con-

trol problems

We follow briefly the discussion on error estimates in (Chapter 2, [24]). If u is the unique solu-tion of the optimal control problem and uh is the corresponding discrete solution, then the errorsatisfies the relation

u − uh2 ≤ C1h2

βy2, (2.88)

where C1 is a constant independent of the mesh size h and the regularization parameter β.Clearly, the error estimate deteriorates as β < h2, in particular, when β h2.

28

The Queen: My dear, here we mustrun as fast as we can, just to stayin place. And if you wish to go any-where you must run twice as fast asthat.


3An overview on iterative methods

3.1 Iterative methods

Consider the large sparse systemAx = b, (3.1)

where A ∈ Rn×n is a non-singular matrix.

Let us solve (3.1) with an iterative method. The idea goes by splitting A into P and A1 suchthat

A = P −A1 (3.2)

for some non-singular choice of P . Then for some initial guess x0, we can write an iterationscheme

Pxm = A1xm−1 + b, m = 1, 2, ... (3.3)

We can also write the above iteration scheme as

xm+1 = (I − P−1

A)xm + P−1b, (3.4)

where (I −P−1A) is called the iteration matrix. The spectral radius of the iteration matrix is givenby

ρ(I − P−1

A) = max|λi|, (3.5)

where λi are the eigenvalues of the iteration matrix.

If (I −P−1A) is a normal matrix1, then the necessary and sufficient condition for convergence isgiven by

ρ(I − P−1

A) < 1. (3.6)

Note that if P = diag(A), the above iteration is the well known Jacobi iteration. If P is the lowertriangular part of A, then the above iteration is the Gauss-Seidel method.

The convergence of these simple iterative methods can be accelerated using some relaxationschemes or combining them with Krylov subspace methods. Another way is to use the Cheby-shev semi-iteration method. Note that relaxation schemes and Chebyshev semi-iteration method

1(I − P−1A)(I − P−1A)T = (I − P−1A)T(I − P−1A).

29

require some information on the eigenvalues of the system while Krylov subspace methods donot require such information.

3.1.1 Relaxation scheme: Richardson iteration

The residual for the m-th iteration is given by

rm = P−1(b −Axm). (3.7)

Further, we can rewrite (3.4) asrm+1 = rm − τP−1xm, (3.8)

where τ > 0 is called the relaxation parameter. The value of τ depends on the norm being usedto minimize the error and the maximum and minimum eigenvalues of A. Note that when P = I,the above iteration is known as the Richardson iteration.

Since computing the eigenvalues for a general matrix can be expensive, allowing τ to vary ateach step may accelerate the process. Thus, we obtain the following generalized Richardsoniteration:

rm+1 = rm − τmxm. (3.9)

The error at the m + 1-th iteration is

em+1 = em − τmrm. (3.10)

We now want to minimize this error in Am-norm at each step. Let m = 1 and if A is symmetricpositive definite, then the value of τm at each iteration can be expressed as

τm =< rk, rk >< rk, rk >A

. (3.11)

3.1.2 Chebyshev semi-iteration

The idea of this method is to accelerate the Richardson Iteration (also Jacobi) using the Cheby-shev iterates ym satisfying the relation

x − ym = pm(I − P−1

A)(x − x0), (3.12)

where pm(z) = ∑mj=0 αjzj is a polynomial of degree m satisfying pm(0) = 1, given the con-

straintm

∑j=0

αj = 1. (3.13)

Note that pm does not depend on the initial guess and the right hand side b of the system (3.1).If we can explicitly find the eigenvalues of the iteration matrix, then computing pm is a fairlyreasonable task. The advantage of using Chebyshev semi-iteration as a linear operator whenused as preconditioner for a linear system, or as a nested part of a preconditioner, is discussedin [32].

3.2 Projection Methods

Projection processes are one of the most general techniques employed by iterative methods tosolve systems of the form (3.1). We define m-dimensional real subspaces K and L, where m ismuch smaller than the dimension of A. K is the subspace of the approximants and is called the

30

search subspace. L is the subspace belonging to the residual vector r = b −Ax and is calledthe constraint subspace or the left subspace. In order to find an approximate solution of (3.1) bya correction in subspace K, we impose m orthogonality constraints on the residual r.

Projection methods fall into two categories; orthogonal and oblique. In an orthogonal projection,the subspace K and L are the same, whereas in case of oblique projection, they are differentand can potentially be unrelated to each other.

3.2.1 General framework

Let K and L be two m-dimensional subspaces in Rn. The projection technique onto the subspaceK and orthogonal to L is a process of finding the approximate solution x of (3.1) by imposing theconditions

Find x ∈ K,such that b −Ax ⊥ L.

(3.14)

Since we start with an initial guess x0, the approximation is sought in the affine space x0 +K andnot simply in the homogeneous vector space K. This changes the above defined formulation ofthe projection method, which now reads

Find x ∈ x0 +K,such that b − Ax ⊥ L.

(3.15)

Iterative methods use successive projections as defined above. Each new projection may take anew pair from the subspaces L and K and the initial guess x0 takes on the value of approximationobtained in the last projection step.

So withx = x0 + δ, (3.16)

the initial residual vector r0 can be written as

r0 = b −Ax0. (3.17)

This results in the formulationb −A(x0 + δ) ⊥ L, (3.18)

orr0 −Aδ ⊥ L. (3.19)

Thus, the general form of one projection step is

x = x0 + δ, δ ∈ K, (3.20)(r0,Aδ, w) = 0, for all w ∈ L. (3.21)

In summary, the idea of a projection method is to find an approximate solution xm of (3.1) froman affine subspace x0 +Km by imposing the Petrov-Galerkin condition

b −Axm⊥Lm,

where m represents the dimension of K and L and x0 is the starting point towards the approximatesolution xm. Note that when K = L, the Petrov-Galerkin conditions are referred as the Galerkinconditions.

31

3.3 Krylov subspace iterative methods

Krylov subspaces are subspaces of the form

Km(A, v) = spanv,Av,A2v, ...,Am−1v, (3.22)

for some vector v such that x = p(A)v. Here, p is a polynomial of degree not exceeding m − 1.Note that the dimension of Km increases by one with each subsequent iteration.

Let v = r0 = b −Ax0 for some initial guess x0. Then we can write:

Km(A, r0) = spanr0,Ar0,A2r0, ...,Am−1r0. (3.23)

A Krylov subspace iterative method computes the iterates

xm = x0 + pm−1(A)r0, (3.24)

where pm−1 is a polynomial of degree not exceeding m − 1.

These methods may exhibit superlinear convergence as they are able to pick the optimal polyno-mial p without any prior knowledge of the spectral properties of the system. This is in contrast tothe simple iterative methods where such information is often necessary.

Note that different choices of Lm yield different versions of Krylov subspace iterative methods.Two important choices in literature are Lm = Km and its minimum residual variation Lm = AKm.Another choice is to define Lm to be a Krylov subspace over AT, i.e., Lm = Km(AT , r0). For ourpurpose we consider only the first choice.

We now present a brief overview on some of the very classical Krylov subspace iterative methodsavailable in literature. The conjugate gradient (CG) method first developed by Hestenes andSteifel [13] works only when the algebraic matrix system is symmetric positive definite. Theminimal residual (MINRES) method by Paige and Saunders [14] is used when the algebraicmatrix system is symmetric and indefinite. For non-symmetric algebraic systems, some of thechoices available are generalized minimal residual (GMRES) method by Saad and Schultz [7],the generalized conjugate residual (GCR) method by Axelsson [9], and the CG method with non-standard inner product known as Bramble-Pasciak CG method [10]. More details on these andother methods can be found in [11, 15]. Brief description on the basics of GMRES and theMINRES method follows next.

Remark A conjugate gradient method can be viewed as a Galerkin process to derive a general-ization for non-symmetric algebraic matrix systems. For more to this, see the Axelsson-Galerkinmethod [8], also known as the generalized conjugate gradient (GCG) method.

3.3.1 The Generalized Minimal Residual (GMRES) Method

Large sparse systems (3.1) are usually not symmetric which requires making further relaxationson the assumptions to construct iterative methods. The GMRES algorithm is based on the projec-tion method where K = Km and L = AKm, with K being the m-th dimensional Krylov subspace.This technique minimizes the residual norm over all the vectors in x0 +Km.

Let Vm be an n × m matrix with column vectors [v1, ..., vm] with v1 = r0/r02. Let Hm be them × m Hessenberg matrix whose non-zero entries hij are defined by the basic Arnoldi algorithm[15]. The last operation of the Arnoldi loop gives

wm = vm+1 ∗ hm+1,m.

32

Let em define the unit vector where the m-th entry is the only non-zero. Then we have the followingrelation, cf. [15],

AVm = VmHm + wm[0, ..., em]

= VmHm + hm+1,m[0, ..., vm+1 em]T

= Vm+1Hm.

(3.25)

HereHm =

Hm

hm+1,mem

(3.26)

is an (m + 1)× m Hessenberg matrix.

In the CG method, we search for a vector xm that minimizes the error emA over the vectors inthe affine subspace x0 +Km(A, r0). In GMRES, we search for a vector xm in the affine subspacex0 +Km(A, r0) which minimizes rm2. Thus, we have

rm2 = b −Axm2

= b −A(x0 + Vmym)2

= r0 −AVmym)2

= r0 −AVm+1Hmym)2

= (v1r02)−AVm+1Hmym2

= Vm+1(r02 e1− Hmym)2.

(3.27)

Using the fact that Vm+1 has orthonormal columns, we have

rm2 = r02 e1− Hmym2. (3.28)

The GMRES approximation results in the following linear least squares problem, i.e.,

min rm2 = min r02 e1− Hmym2, (3.29)

where ym minimizes rm2. The minimizer ym requires the solution of (m + 1)× m least squaresproblem. The above ideas lead to the following algorithm, cf. [15].

Algorithm 1 GMRES

1: Compute r0 = b − Ax0, β := r02, and v1 := r0/β2: For j = 1, 2, ..., m, Do:3: Compute wj := AM−1vj4: For i = 1, ..., j, Do:5: hij := (w, vi)6: w := w − hijvi7: EndDo8: hj+1,j = w2. If hj+1,j = 0 set m := j and go to 119: vj+1 = wj/hj+1,j

10: EndDo11: Define the (m + 1)× m Hessenberg Matrix Hm = hij1≤i≤m+1,1≤j≤m12: Compute ym, the minimizer of βe1 − Hmy2 and xm = x0 + Vmym

We point out that for GMRES, as m increases, then for a matrix of order n × n, the memory re-quirement increases by O(mn) and computational cost increases by O(m2n). This is becausethe method relies on the Hessenberg matrix H, which requires that to generate vm+1, every sub-sequent vector vjj=1,...,m must be stored. There are three possible ways to overcome this issue.

33

The first is to restart the algorithm periodically, say after k iterations, such that the last iterationxk becomes the first iteration x0 in the next cycle. This is called restarted GMRES or GMRES(k).One problem with the restarted version of GMRES is that it may stagnate. The second is to usetruncation in the Arnoldi orthogonalization leading to the so called Truncated GMRES versionsknown as Quasi-GMRES [15]. Yet another way to mitigate computational and memory costs is touse preconditioning techniques; this we shall discuss in the next chapter.

3.3.2 The Minimal Residual (MINRES) Method

When A is symmetric, the Hessenberg matrix Hm becomes symmetric tridiagonal and leads tothree term recurrence in the Arnoldi process given by

γm+1vm+1 = Avm − δmvm − γjvm−1, (3.30)

where δm =< Avm, vm > and γm+1 is chosen to normalize vm−1.

MINRES is a Krylov subspace based method derived from the Lanzcos algorithm [12] and worksif A is symmetric and indefinite. The idea of Lanczos method is to find an orthogonal basis forKm(A, r0) based on the recurrence relation as defined by (3.30).

We now define a tridiagonal symmetric matrix

Tm = tridiag(γm, δm, γm+1). (3.31)

Let Vm be an n × m matrix with column vectors [v1, ..., vm] with v1 = r0/r02. Let em define theunit vector where the m-th entry is the only non-zero. Then we can write the recurrence relation(3.30) as

AVm = VmTm + γm+1[0, ..., vm+1 em]T

= Vm+1Tm.(3.32)

HereTm =

Tm

γm+1 em

∈ Rm+1,m. (3.33)

As in GMRES, we search for a vector xm in the affine subspace x0 +Km(A, r0) which minimizesrm2, i.e.,

min rm2 = min r02 e1− Tmym2, (3.34)

but with Hm replaced by Tm. The algorithm resulting from this minimization problem is calledMINRES. The MINRES steps are shown in Algorithm 2, cf. [18].

34

Algorithm 2 MINRES1: v0 := 0, w0 := 0, w1 := 02: Choose x0, Compute v1 := b −Ax0, set γ1 := v1 = 13: Set η = γ1, s0 = s1 = 0, c0 = c1 = 14: For j = 1, 2, ..., m, Do :5: vj := vj/γj6: δj :=< Avj, vj >7: vj+1 := Avj − δjvj − γjvj−18: γj+1 := vj+19: α0 := cjδj − cj−1sjγj

10: α1 :=

α20 + γ2

j+111: α2 := sjδj − cj−1cjγj12: α3 := sj−1γj13: cj+1 := α0/α1; sj+1 := γj+1/α114: wj+1 := (vj − α3wj−1 − α2wj)/α115: xj = xj−1 + cj+1ηwj+116: η := −sj+1η17: Test for convergence18: EndDo

35

Alice: How long is forever?White Rabbit: Sometimes, just onesecond.


4Preconditioned Krylov subspace iterative

methods and multigrid (MG) preconditioners

The presence of underlying parameters, especially when they take on critical values can causethe algebraic matrix system to have bad spectral properties, for instance, a bad condition number.The condition number of a general matrix A is defined by

κ(A) = A−1

A.

A is said to be ill-conditioned if κ(A) 1. Further, the effect of the mesh size h on κ is wellunderstood in the numerical analysis literature: κ(A) depends on h through the relation

κ(A) = O(h−γ), γ ≥ 1.

So for instance, if γ = 1 and we intend to improve the quality of discretization by refining (halving)the mesh once, the condition number deteriorates by a factor of two.

While direct solution methods are robust in terms of underlying parameters such as the meshsize h, or the regularization parameter β in case of optimal control problems, they pose highcomputational demands in terms of memory size and computational time. Iterative methods,although they scale linearly with respect to the size of A, are not necessarily robust with respectto underlying parameters. Moreover, the convergence rate of the iterative methods depends onproperties such as the clustering of the eigenvalues and the condition number κ(A).

Preconditioning is one way to overcome robustness and efficiency issues of iterative solvers.The idea of preconditioning is to alleviate the spectral properties of A so as to improve the con-vergence rate of the iterative methods. An optimal preconditioner P satisfies the following twoconditions:

1. The condition number of the preconditioned system is of order 1, i.e., κ(P−1A) = O(1).

2. The evaluation of P−1v for a vector v is inexpensive i.e., of the order O(n).

Generally, in constructing an optimal preconditioner, there is always a compromise between con-dition (1) and condition (2).

While h is the primary parameter that effects the spectral properties, other model parameters suchas the regularization parameter β can cause further deterioration in them. Therefore, another

36

desirable property of a good preconditioner is that it should be parameter robust, i.e., the conditionnumber of the preconditioned system can be bounded above by a constant independent of modelparameters.

We quote an interesting general comment on the construction of preconditioners from [15] whichreads, "Finding a good preconditioner to solve a given sparse linear system is often viewed as a

combination of science and art. Theoretical results are rare and some methods work surprisingly

well, often despite expectations".

There are several possible ways in which A can be preconditioned. The preconditioner can beapplied from the left leading to the following preconditioned system, i.e.,

P−1

Ax = P−1b. (4.1)

Another way is to apply the preconditioner from the right, i.e.,

AP−1u = b, x ≡ P

−1u. (4.2)

The third way is to have P factored into, typically, triangular matrices PL and PR, i.e.,P = PLPR.This leads to,

P−1L AP

−1R u = P

−1L b, x ≡ P−1

R u. (4.3)

To any symmetric or non-symmetric iteration method, we can apply left, right and split precondi-tioning techniques. Further, if P approximates A−1, then we need only to multiply with it. Finally,P does not have to be explicitly computable. It can be a procedure (algorithm) that implementsthe action of P−1 on a vector.

4.1 The idea of preconditioned Krylov subspace iterative meth-

ods

Following the discussion on iterative methods, let us recall equation (3.8) from the last chapter,i.e.,

rm+1 = rm − τP−1xm, (4.4)

where P in the current context is the preconditioner of A.

Using induction on the above equation we can rewrite

rm ∈ spanr0,P−1Ar0, (P−1

A)2r0, ..., (P−1A)mr0. (4.5)

As noted earlier, a Krylov subspace iterative method aims to find the best choice of xm in the affinespace xm ∈ x0 +Km, where Km in the m-th Krylov subspace, and in the context of preconditioningis defined as

Km(P−1

A, r0) = spanr0,P−1Ar0, (P−1

A)2r0, ..., (P−1A)mr0. (4.6)

Different preconditioning techniques (right, left, split) lead to different versions of a precondi-tioned Krylov subspace iterative methods. For instance, in case of GMRES, we have the rightpreconditioned GMRES and the left preconditioned GMRES. The convergence behavior of leftand right preconditioned GMRES methods are comparable, however, the comparison may not beapplicable in case when P is ill conditioned.

Further, the right preconditioned GMRES results in a variant of GMRES, called the flexible GM-RES or FGMRES. This version is very useful in cases where the preconditioner changes at eachstep. However, for FGMRES, convergence results are not easy to prove since the subspace ofapproximants is not the standard Krylov subspace, i.e., the application of AP−1

m on vector v of

37

Krylov subspace is not in the span of Vm+1. The algorithm for different preconditioned versions ofKrylov subspace iterative methods can be found in [15, 18]. For other methods such as precondi-tioned generalized conjugate residual (GCR) and preconditioned generalized conjugate gradient(GCG), the reader is referred to [11, 15].

4.2 Multigrid methods as a preconditioner

Preconditioned Krylov subspace based iterative methods are general purpose solvers. However,in general, their performance, both in terms of convergence and efficiency, deteriorates as themesh size h decreases. Multigrid1 methods belong to the class of optimal methods that have opti-mal or nearly optimal computational complexity and an optimal convergence rate, independent ofthe mesh size h and other underlying parameters. These methods can be successfully employedto solve a variety of linear and non linear PDE problems. Let us briefly discuss the idea of theclass of multigrid2 methods referred to as the Geometric Multigrid (GMG) by using a well studiedcase of the Laplace equation

∆y = u ∈ Ω. (4.7)

In a GMG, we utilize a sequence of nested geometric meshes. An analysis of an iterative schemesuch as Jacobi iteration to solve the above problem tells us that as the mesh size gets finer,high frequency error components damp down quickly, while those with low frequency will takelonger to converge. This increases the total number of iterations to solve the problem. Thebasic idea of a multigrid method is to project the low frequency components from finer mesh,say Ωh, to a coarser mesh, say ΩH, thus relying on the geometric information of the underlyingmesh. A multigrid process requires the construction of restriction and prolongation operators.The prolongation operator maps the coarse grid space to the fine grid space, while the restriction

operator maps the fine grid space to the coarse grid space. Since it is inexpensive to iterate overa coarse grid, iteration over low frequency error components, termed as smoothing in the MGliterature, is then used to achieve the error reduction originally present on the fine grid.

In the class of MG methods referred to as the Algebraic Multigrid (AMG), geometric meshes arereplaced with algebraically constructed subspaces, and therefore no knowledge of the underlyinggeometry is needed. Thus, instead of the mesh size h, there are levels. So the fine mesh Ωhis now interpreted as a subspace Xh ∈ Rn at some fine level. Similarly, the coarse mesh ΩH isreplaced by a subspace XH ∈ Rn for some coarse level. Finally, the prolongation and restrictionoperators are also defined algebraically.

Further technical details on multigrid methods can be found in [15]. Other optimal or nearlyoptimal preconditioners are the Algebraic Multilevel Iteration (AMLI) method [47, 48, 49, 50] andthe domain Decomposition (DD) methods [51].

1These methods can also be used as solvers, but mostly they are used as a preconditioner.2The idea is extendable to other class of multigrid algorithms such as the algebraic multigrid (AMG).

38

Mad Hatter: You would have to behalf mad to dream me up.


5Preconditioners for saddle point systemsarising in PDE-constrained optimization

problems

In this chapter, we briefly describe different approaches to construct preconditioners for saddlepoint systems. Based on these techniques we then discuss different preconditioners available inliterature for our three benchmark PDE-constrained optimization problems.

5.1 Preconditioner construction techniques for saddle point

systems

5.1.1 (Negative) Schur complement approximation

Recall the saddle point system:

A =

A BT

B −C

. (5.1)

An established idea of constructing block diagonal and block triangular preconditioners for a sad-dle point system (5.1) is based on constructing a good approximation to the Schur complement.This classical approach is primarily algebraic, and relies on the knowledge of the diagonal blockA and/or C and the Schur complement of (5.1).

The (negative) Schur complements of A can be written as:

T = A + BTC−1B (5.2)

andS = C + BA−1BT (5.3)

39

Assuming that A and C are symmetric positive definite, the Schur complements S and T are thenalso symmetric positive definite. Thus we have the following block diagonal ideal precondition-ers

P0 =

−T 00 C

(5.4)

andP1 =

A 00 −S

. (5.5)

In cases where neither A nor C are positive definite, a possible strategy is the construction ofinexact Schur complement [34].

Another class of preconditioner, called non-symmetric block lower triangular ideal preconditionertakes the form

P2 =

A 0B −S

. (5.6)

So if P−10 A, P−1

1 A and P−12 A are non singular, their spectra are given by

P−10 A =

1 −

√5

2, 1,

1 +√

52

, P−1

1 A =

1 −

√5

2, 1,

1 +√

52

, P−1

2 A = 1 .

P−10 A, P−1

1 A are diagonalizable while P−12 A is not. Thus if A is preconditioned with P0 or P1

with an appropriate Krylov subspace iterative method, then the process will terminate in threeiterations. If A is preconditioned with P2, two iterations will be required. Further, the followingbound holds on the condition numbers, i.e.,

κ(P−10 A) ≤

√5 + 1

√5 − 1

≈ 2.62, κ(P−11 A) ≤

√5 + 1

√5 − 1

≈ 2.62.

However preconditioning A, say for instance with P1, with an appropriate Krylov subspace iter-ative method would require the exact construction of S. Since S can be dense, even if A and Bare sparse, an iterative solver can still be very expensive. The alternative to this is to approxi-mate A and S by A and S such that while they keep their nice spectral properties, yet they canbe efficiently applied to the vectors from the left. The same line of thought is applicable whenusing P0 as a preconditioner. Therefore, the key component here is a good Schur complementapproximation.

5.1.2 Inexact Schur complement approximation

Recall the saddle point system (5.1) with C = 0, i.e.,

A =

A BT

B 0

,

where A is symmetric and indefinite and B is of full rank. Moreover, A is non-singular satisfyingthe strictly positive inf-sup constant γ inline with Brezzi’s theory [3]. Then by lemma 1 in [34], wehave a transformation matrix H giving us an ideal block diagonal preconditioner

Pixs =

P 00 R

, (5.7)

where P =12(HT A + A) and R = BP−1BT. Here, R is referred to as the inexact Schur comple-

ment, cf. [34].

40

5.1.3 Operator preconditioning with norms

The idea of operator preconditioning was collected from previous such attempts by Hiptmair whodiscussed the issue in [33]. As noted in [33], the idea of operator preconditioning is that givena continuous bijective linear operator A : X → X∗ on function spaces X and X∗ and anotherisomorphism P : X∗ → X, we can then have PA to be an endomorphism (a mapping on itself)of X. Since such endoporphisms after discretization yield well conditioned matrices, it becomespossible to use P as a preconditioner of A. Zulehner [35] and Kollmann [26] applied the approachto construct symmetric positive definite block diagonal preconditioners using both the standardand non-standard norms.

In our considered benchmark problems, after eliminating the control u we are left with the statevariable y and the adjoint variable λ resulting in a reduced optimality system. These problemscan then be written in a mixed variational formulation. Let Vh and Λh be the Hilbert spaces withinner products (·, ·)Vh

and (·, ·)Λhand associated with the norms · Vh and · Λh . Then we

have the following mixed variational formulation:

Find yh ∈ Vh and λh ∈ Λh such that

a(yh, vh) + b(vh, λh) = f (vh) ∀vh ∈ Vh,b(yh, µh)− c(λh, µh) = g(µh) ∀µh ∈ Λh,

(5.8)

where a, b , c are bounded bilinear forms on Vh × Vh → Rn, Vh × Λh → Rn, and Λh × Λh → Rn

respectively, whereas f ∈ V∗h and g ∈ Λ∗

h are the linear forms; with V∗h and Λ∗

h being the dualspace of Vh and Λh respectively. Further, the following assumptions hold:

1. a and c are symmetric, i.e.,

a(wh, vh) = a(vh, wh) ∀vh, wh ∈ Vh, c(ξh, µh) = c(µh, ξh) ∀µh, µh ∈ Λh.(5.9)

2. a and c are nonnegative, i.e.,

a(vh, vh) ≥ 0 ∀vh ∈ Vh, c(µh, µh) ≥ 0 ∀µh ∈ Λh. (5.10)

which makes (5.8) a symmetric indefinite problem.

We can write the mixed variational formulation (5.8) as a variational problem:

Find xh = (yh, λh) ∈ Xh : Vh × Λh such that

B(xh, sh) = F (sh), ∀sh ∈ Xh, (5.11)

whereB(zh, sh) = a(wh, vh) + b(vh, rh) + b(wh, µh)− c(rh, µh), (5.12)

for s = (v, µ), z = (w, r).

Let A ∈ L(xh, X∗h), with X∗

h being the the dual space of X, be the linear operator associated withthe bilinear form B given by

Axh, sh = B(xh, sh). (5.13)

Thus using the operator notation, we have

Axh = F in X∗h . (5.14)

The problem (5.14) is well posed if the following inf-sup condition by Babushka and Aziz [2] aresatisfied for the symmetric case for some constants c1, c2 > 0, i.e.,

c1zhXh ≤ AzhX∗h≤ c2zXh ∀zh ∈ Xh. (5.15)

41

Now let φi : 1, ..., n be the set of basis functions belonging to the space Vh and let φi : 1, ..., n bethe set of basis functions belonging to the space Λh. Then using the representation

wh =n

∑i=0

Wiφi, rh =m

∑i=0

Riψi (5.16)

we can rewrite (5.15) as

c1zhPXh≤ AzhP−1

Xh≤ c2zhPXh

∀zh ∈ Rn+m, (5.17)

withzh =

(Wi)

ni=1

(Ri)mi=1

, (5.18)

and the symmetric and positive definite block diagonal matrix

PXh=

((φi, φj)Vh)

ni,j=1 0

0 ((ψi, ψj)Λh)mi,j=1

. (5.19)

The resulting inner product PXh in norm Xh then satisfies the following bounds, i.e.,

κPXh(P−1

XhA) ≤

c1c2

, (5.20)

where κPXhrepresents the condition number.

Hence, the norm for Hilbert space Xh for satisfying the well-posedness condition (5.15) yields apreconditioner PXh for the operator A.

Note that if A depends on the underlying parameters, such as mesh size h and the regulariza-tion parameter β, then if the constants in (5.15) are independent of these parameters, then thecorresponding preconditioner is called parameter-robust, cf. [26].

5.1.4 Preconditioners based on interpolation theory

If the blocks A and C of the saddle point system (5.1) are positive definite, then one can eitheruse P0 or P1 as a preconditioner. Using interpolation theory, Zulehner in [34] suggests thatinterpolating between these two preconditioners yields a family of preconditioners, Pθ for θ ∈

(0, 1), given byPθ = P

1/20 (P1/2

0 P1P1/20 )θ

P1/20 , (5.21)

where Pθ represents the inner product of the interpolation space with index θ associated with thetwo spaces whose inner product is represented by P0 and P1. Another form to (5.21) is by usingthe so called K-representation, cf. [34], given by

Pθ = [P0,P2]θ θ ∈ (0, 1). (5.22)

The idea is that within the family of preconditioners, one may be able to find a particular θ suchthe interpolation can be computed efficiently, resulting in a preconditioner that best fits certaingiven criteria. Since Pθ represents an inner norm, the technique can be interpreted as opera-tor preconditioning in which the idea is to find the norm in the finite dimensional Hilbert spaceXh.

42

5.1.5 Schur complement approximation using the commutator argument

The commutator argument has been introduced in (Chapter 8, [18]) as one of the methods to ap-proximate the Schur complement matrix BF−1

y BT as it occurs in the Oseen problem. The motivat-ing argument here is that the Oseen system contains a discrete convection-diffusion component,i.e.,

L = −∇2 + wh∇. (5.23)

defined on the velocity space. Disregarding the fact that pressure is defined to be in L2(Ω) andassuming that its differential form "makes sense", we may have a discrete convection-diffusioncomponent defined on the pressure space, i.e.,

Lp = −∇2 + wh∇. (5.24)

Finally, introducing the "critical" assumption that convection-diffusion operators commutate in thevelocity and the pressure spaces with the gradient operator, gives

Eh = (−∇2 + wh∇)v∇−∇(−∇2 + wh∇)p, (5.25)

where Eh is considered to be small in some sense.

In the velocity space, the matrix representation of the discrete gradient operator is M−1y BT and

the discrete matrix representation of the (negative) divergence operator is M−1p B. The purpose of

the mass matrix My is to provide correct scaling. In a similar way, these operators can be definedon the pressure space.

Thus the approximation for BF−1y BT, see (Chapter 8, [18]), is obtained using the discretized ver-

sion of the commutator in terms of finite element matrices, i.e.,

Eh = (M−1y Fv)(M−1

y BT)− (M−1y BT)(M−1

p Fp) (5.26)

leading toBF−1

y BT≈ BM−1

y BT Fp M−1p , (5.27)

cf. [18].

5.1.6 Operator factorization

Axelsson and Neytcheva in [40] introduced an operator splitting technique for solving nonlinearcoupled multi-physics problems in the context of an interface problem. Cahn-Hilliard equationis one of the methods to model moving interfaces and the discretization of this equation usingstandard finite elements leads to a matrix representation of the form

A0 =

M −J − βKαK M + ∆tW

, (5.28)

where α, β are constants, J is the Jacobian, ∆t is some small time step1 and ∆tW is the convectionterm. For the 2-D case2, it is shown in [40, 41], that under certain relations between the time andspatial discretization steps, ∆t and h, neglecting the convection and nonlinear terms δtW and Jcan be justified, resulting in a high quality approximation ART of A0, given by

ART =

M −βKαK M

. (5.29)

1We avoid time index for simplicity.2A 3-D result is also available.

43

It is further shown in [40, 41], that ART can be efficiently preconditioned by

PF =

M −βKαK M + 2

αβK

. (5.30)

The preconditioner PF is demonstrated for efficiently solving two phase flow problem involvingCahn-Hilliard equation in [42], and also for efficiently solving complex-valued algebraic systemsin [43]. We show that this preconditioner can be effectively applied to saddle point systems arisingin PDE-constrained optimization problems.

5.1.6.1 Efficient solution for PF

Let us consider the solution of the system

PF

yλ

=

M −βKαK M + 2

αβK

yλ

=

bd

. (5.31)

One way to solve the problem is to use the exact block LU (block lower triangular, block uppertriangular) factorization as discussed in [40]. This leads to

PF =

M 0αK M +

αβK

I −βM−1K0 M−1(M +

αβK)

(5.32)

The above system can be solved with the following algorithm:

Algorithm 3 Algorithm for Block LU solve1: Solve Mf = b2: Solve (M +

αβK)g = d − αKf

3: Solve (M +

αβK)λ = Mg

4: Solve y = f −

β√

α(g − λ)

We now discuss a more efficient algorithm to solve for the solution of PF based on block LDU(block lower triangular, block diagonal, block upper triangular) factorization technique, cf. [40, 42].So we write

PF =

I 0

αKM−1 I +

αβKM−1

M 00 M

I −βM−1K0 I +

αβKM−1

. (5.33)

Then the inverse P−1F can be obtained as

P−1F =

I βM−1K(I +

αβM−1K)−1

0 (I +

αβM−1K)−1

M−1 0

0 M−1

×

I 0

−αKM−1(I +

αβKM−1)−1 (I +

αβKM−1)−1

=

M−1

β

√α(I − (I +

αβM−1K)−1)M−10 (M +

αβK)−1

×

I 0

−

√αβ(I − (I +

αβKM−1)−1) M(M +

αβK)−1

=

(M +

αβK)−1(I − M(M +

αβK)−1) + (M +

αβK)−1

β

√α(I − M(M +

αβK)−1))(M +

αβK)−1

−

√αβ(M +

αβK)−1(I − M(M +

αβK)−1) (M +

αβK)−1 M(M +

αβK)−1

(5.34)

Let H = (M +

αβK)−1. Then

44

P−1F =

H(2I − MH)

β

√α(I − HM)H

−

√αβ

H(I − MH) HMH

. (5.35)

Multiplying the vector

bd

with P

−1F we have

yλ

= P−1

F

bd

=

H

(2b − MHb) +

β

√α(d − MHd)

−H

β

√α(b − MHb)− MHd

. (5.36)

Transforming the above further leads to

yλ

=

β

√α

H

√αβ(2b − MHb) + (d − MHd)

−H

√αβ(b − MHb)− MHd

=

β

√α

H

√αβ

b + d

+

β

√α

H

√αβ

b − MH

√αβ

b + d

−H

√αβ

b − MH

√αβ

b + d

.

(5.37)

Let s1 = H

√αβ

b + d

and s2 = H

√αβ

b − Ms1

. Then

yλ

=

β

√α(s1 + s2)

−s2

. (5.38)

And finally, the following algorithm can be used to solve (5.38), cf. [40].

Algorithm 4 Solving the factorized operator PF

1: Compute f =√

αβ

b + d

2: Solve (M +

αβK)s1 = f

3: Compute g = Ms1 −

√αβ

b

4: Solve (M +

αβK)λ = g

5: Compute y =

β

√α(s1 − λ)

Clearly, we can observe in Algorithm 4 that the LDU approach has a lower computational cost ascompared to the LU approach. In the LU case, see Algorithm 3, three solves are required; onefor M and two for M +

αβK, whereas in case of LDU, two solves are needed with M +

αβK.

45

It is further shown in [40] that the eigenvalues λ of P−1F ART are contained in the interval [

12

, 1].Thus the condition number is given by

κ(P−1F ART) =

10.5

= 2. (5.39)

5.2 Preconditioners for PDE-constrained optimization prob-

lems

As noted earlier, numerically dealing with distributed optimal control problems constrained byPDEs may yield an optimality system representing a saddle point structure of the form (5.1).Here we discuss various preconditioners available in literature to solve for our three benchmarkPDE-constrained optimization problems.

5.2.1 Preconditioners for the distributed optimal control problem constrained

by the Poisson equation

Recall that numerically tackling the distributed Poisson control problem leads to an optimalitysystem

AF =

M 0 KT

0 βM −MK −M 0

(5.40)

with its reduced form given by

AR =

M KT

K −1β

M

. (5.41)

Extending the work of Rees [24], Pearson in [27] derives a Schur complement approximationthat is independent of mesh size h and the regularization parameter β. This Schur complementapproximation is then used for constructing the block diagonal and the block lower triangularpreconditioner for preconditioning the saddle point system (5.40). The reduced version of thispreconditioner can be applied to the reduced optimality system (5.41)

Operator preconditioning technique based on standard and non-standard norms, see Zulehner[35] and Kollmann [26], operator preconditioning using interpolation operator, see Zulehner [34]and Kollmann [26], are other methods to solve the reduced optimality system (5.41). We note thatoperator preconditioning with non-standard norms and operator preconditioning with interpolationoperator lead to equivalent preconditioners that are both independent of mesh size h and theregularization parameter β.

A factorization based preconditioner proposed by Axelsson and Neytcheva in [40] can be effec-tively applied to the saddle point system (5.41) by transforming it into (5.29).

We now present a basic overview on the preconditioners developed using the approaches dis-cussed above.

5.2.1.1 Operator preconditioning with standard norms

Operator preconditioning with standard norms results in a symmetric positive definite block diag-onal preconditioner

pson Psn =

K 00 K

. (5.42)

46

for the reduced optimality system (5.41), cf. [26, 34]. Further, this preconditioner is robust withrespect to mesh size h but not with respect to the regularization parameter β.

5.2.1.2 Operator preconditioning with non-standard norms

Operator preconditioning with non-standard norms results in a symmetric positive definite blockdiagonal preconditioner

pson Pnsn =

M +

βK 0

01β(M +

βK)

(5.43)

for the reduced optimality system (5.41).

The bound on the condition number is given by

κ(pson P−1nsnAR) ≤

11/

√2≈

√2,

cf. [35]. Further, this preconditioner is robust to the underlying parameters, i.e., h and β.

5.2.1.3 Operator preconditioning with interpolation operator

Scaling the reduced optimality system (5.41), we have

AR

yλ

≡

M

βK

βK −M

y

1β

λ

=

b

1β

d

. (5.44)

The ideal preconditioners for the above system are

P0 =

M 00 M + βKM−1K

(5.45)

andP1 =

M + βKM−1K 0

0 M

. (5.46)

Choosing θ =12

, a preconditioner based on the interpolation theory is defined by

P1/2 = [P0,P1]1/2

=

[M + βKM−1K, M]1/2 0

0 [M + βKM−1K, M]1/2

.

(5.47)

Using the relation (5.21), it can be observed that

[βKM−1K, M]1/2 =

βK. (5.48)

This immediately leads to the following preconditioner

pson Pθ =

M +

βK 0

0 M +

βK

, (5.49)

cf. [26, 34]. This preconditioner is equivalent to the preconditioner (5.43) obtained using non-standard norms.

47

5.2.1.4 Schur complement approximation

Rees in [24] and Pearson in [27] proposed a block diagonal preconditioner

pson Pbd1 =

M 0 00 β M 00 0 S

(5.50)

and a block lower triangular preconditioner

pson Pblt =

M 0 00 β M 0K −M −S

(5.51)

for the saddle point system (5.40), where M is an approximation3 of the mass matrix M and S isthe Schur complement approximation.

A Schur complement approximation proposed by Rees in [24] is

S := KM−1K = S1. (5.52)

The eigenvalues of S−11 S are shown to be bounded by

λ(S−11 S) ∈

1β

¯ch4 + 1,1β

C + 1

(5.53)

where c and C are real positive constants independent of the mesh size h. Clearly, this approxima-tion suffers from convergence rate deterioration as β beta becomes smaller. In order to remedythe issue, Pearson in [27] proposed an improvement to the approximation of S, i.e.,

S ≈ (K +1

βM)M−1(K +

1β

M) := S2. (5.54)

The approximation has been proved to be both h and β independent with the following bound,i.e.,

λ(S−12 S) ∈

12

, 1

. (5.55)

For the reduced optimality system (5.41) with S approximated by S24, we have the block diagonal

preconditioner

pson Pbd2 =

M 00 (K + 1√

βM)M−1(K + 1√

βM)

. (5.56)

5.2.1.5 Operator factorization

As noted earlier, an efficient preconditioner for operators of the form

ART

yλ

≡

M −βKT

αK M

yλ

=

bd

(5.57)

isPF ≡

M −βKT

αK M +

αβK

, (5.58)

320 Chebyshev iterations were used to approximate M in [27].4One can also use the approximation S ≈ S1 = KM−1K

48

cf. [40].

We clearly see that the operator AR in (5.41) is parametrically different from the operator ART in(5.57). Setting α = 1 and making a substitution

λ = −βλ (5.59)

to the reduced optimality system (5.41) gives a transformed version

ART

yλ

≡

M −βKT

K M

yλ

=

bd

. (5.60)

Setting α = 1 in (5.58) gives us

pson PF ≡

M −βKT

K M +

βK

, (5.61)

which is expected to result in a good preconditioner for the transformed reduced optimality system(5.60). We are further convinced that this preconditioner is independent of both the mesh size hand the regularization parameter β.

5.2.2 Preconditioners for the distributed optimal control problem constrained

by the convection-diffusion equation

Rees in (Chapter 6, [24]) used both the discretize-then-optimize as well as the optimize-then-

discretize approaches to tackle the distributed convection-diffusion control problem. The pre-conditioners proposed therein, however, are not β independent. Further, the discretized systemusing the discretize-then-optimize approach results in block matrices of the form:

(Qsd)i,j =

Ωφi.φj + δ

Ω(w∇φj).

Since no spectrally equivalent operator approximation is available for this block, numerical eval-uation relied on a direct method resulting in loss of linear time scaling. Similarly, using optimize-

then-discretize approach results in block matrices of the form

(Rsd)i,j =

Ωφi.φj − δ

Ω(w∇φj).

Again, since no spectrally equivalent operator approximation is available for the block, numericalevaluation relied on a direct method resulting in loss of linear time scaling.

As noted earlier, the optimize-then-discretize and discretize-then-optimize approaches differ inthe presence of stabilization terms, such as resulting from streamline upwind Petrov-Galerkin(SUPG) stabilization. The optimize-then-discretize approach is strongly consistent, i.e., if wereplace yh, uh, λh by their optimal solutions, then all the three equations of the optimality systemare satisfied. Whereas in the case of discretize-then-optimize only the last equation, i.e., thestate equation of the optimality system is strongly consistent. The numerical results done in[24] gives credence to the point that the strong consistency property of optimize-then-discretize

optimality system makes is a better method, thus corroborating the results of Heinkenschloss etal. [23].

Moreover, it is also noted in [23], that the errors between optimize-then-discretize and discretize-

then-optimize approaches are small when piecewise linear polynomials are used for the dis-cretization of the state y, the control u and the adjoint λ; however there can be significant differ-ences in errors in the two approaches if higher-order finite elements are used for the state y and

49

the adjoint λ. Another observation in [23] is that applying SUPG to optimal control problems willalways give no better then first order accuracy in the presence of boundary layers.

Given the problems associated with the SUPG method in convection-diffusion equations, Geur-mond in [30] used the idea of scale separation or subgrid modeling. Based on this idea, Beckerand Vexler in [29] used the local projection stabilization (LPS) scheme in the context of opti-mal control of a process governed by the convection-diffusion-reaction equation. Since the LPSscheme is adjoint-consistent, the optimize and discretize steps commute. In simple words, itmeans that with the LPS scheme, the optimality systems obtained with optimize-then-discretize

and discretize-then-optimize are the same.

Pearson and Wathen in [28] used the LPS scheme resulting in optimality systems which havethe same structure whether optimize-then-discretize or discretize-then-optimize is used. This isgiven by

A

yuλ

≡

M 0 FT

0 βM −MF −M 0

yuλ

=

b0d

. (5.62)

The reduced optimality system for the problem is

AR

yλ

≡

M FT

F −1β

M

yλ

=

bd

. (5.63)

We now present an overview of different preconditioners available in literature for solving thesaddle point systems (5.62) and (5.63).

5.2.2.1 (Negative) Schur complement approximation

Pearson and Wathen in [28] derive a block diagonal preconditioner

cd Pbd1 =

M 0 00 β M 00 0 S

(5.64)


cd Pblt =

M 0 00 β M 0F −M −S

(5.65)

to solve for the saddle point system (5.62), where M is the approximation of the mass matrixand

S = (F +1

βM)M−1(F +

1β

M) (5.66)

is the Schur complement approximation. The approximation S is based on the extension of resultsproved in [27] with the spectral bound given by

λ(S−1S) ∈

12

, 1

. (5.67)

Clearly, S is independent with respect to both h and β. Finally, for the reduced optimality system(5.63), we have the block diagonal preconditioner

cd Pbd2 =

M 00 (F + 1√

βM)M−1(F + 1√

βM)

, (5.68)

50

cf. [28].


As noted earlier, Zulehner in [35] using non-standard norms derived the symmetric positive defi-nite block diagonal preconditioner

pson Pnsn =

M +

βK 0

01β(M +

βK)

for the reduced optimality system (5.41) arising in the distributed optimal control of the Poissonequation. Further, this preconditioner is robust with respect to the underlying parameters, i.e.,h and β. We expect that by replacing K with F in the above preconditioner will give us a goodpreconditioner for the saddle point system (5.63). So we write

cd Pnsn =

M +

βF 0

01β(M +

βF)

. (5.69)

5.2.2.3 Operator factorization

As noted earlier, an efficient preconditioner for operators of the form

ART

yλ

≡

M −βKT

αK M

yλ

=

bd

(5.70)

isPF ≡

M −βKT

αK M + 2

αβK

, (5.71)

cf. [40].

We clearly see that the operator AR in (5.63) is parametrically as well as structurally different fromthe operator ART in (5.70). Replacing K with F, setting α = 1 and making a substitution

λ = −βλ (5.72)

to the reduced optimality system (5.63) gives a transformed version

ART

yλ

≡

M −βFT

F M

yλ

=

bd

. (5.73)

Further, replacing K with F and setting α = 1 in (5.71) gives

cd PF ≡

M −βFT

F M +

βF

, (5.74)

which we expect to be a good preconditioner for the transformed saddle point system (5.73).This preconditioner is expected to be independent of mesh size h and regularization parameterβ.

51

5.2.3 Preconditioners for distributed optimal control problem constrained

by the Stokes system

As noted earlier, numerically tackling the distributed Stokes control problem leads to an optimalitysystem

AF =

My 0 0 Ky BT

0 0 0 B 00 0 βMy −MT

y 0Ky BT −My 0 0B 0 0 0 0

, (5.75)


AR =

My Ky 0 BT

Ky −1β

My BT 0

0 B 0 0B 0 0 0

. (5.76)

Rees in [17] derives a block diagonal and a block lower triangular preconditioner for the saddlepoint system (5.75) using the classical Schur complement approximation technique. This precon-ditioner is mesh size h independent, but not regularization parameter β independent.

Operator preconditioning based on standard and non-standard norms, see Zulehner [35] andKollmann [26], and inexact Schur approximation, see Zulehner [34], are some techniques forconstructing preconditioners for the reduced optimality system (5.76). Operator preconditioningwith non-standard norms and inexact Schur approximation lead to equivalent preconditioners thatare independent with respect to the mesh size h and the regularization parameter β.

Pearson in [16] using the Cahouet-Chabard [22] approximation and the commutator argument toapproximate the Schur block S, derived four different preconditioners for the reduced optimalitysystem (5.76).

Basic overview on preconditioners developed using the approaches discussed above followsnext.

5.2.3.1 Operator preconditioning with standard norms

Operator preconditioning with standard norms leads to a symmetric positive definite block diago-nal preconditioner for the reduced optimality system (5.76), given by

stk Psn =

Ky 0 0 00 Ky 0 00 0 Mp 00 0 0 Mp

, (5.77)

cf. [26, 34]. This preconditioner is robust with respect to the mesh size h but not with respect tothe regularization parameter β.

52


Operator preconditioning with non-standard norms leads to a symmetric positive definite blockdiagonal preconditioner for the reduced optimality system (5.76), given by

stk Pnsn =

My +

βKy 0 0 0

01β(My +

βKy) 0 0

0 0 (K−1p +

βM−1

p )−1 00 0 0 β(K−1

p +

βM−1p )−1

,

(5.78)cf. [26, 35]. This preconditioner is robust with respect to the underlying parameters, i.e., h andβ.

5.2.3.3 Inexact Schur complement approximation

Scaling the reduced optimality system (5.76) gives

AR

ypλµ

≡

My

βKy 0

βBT

βKy −My BT 00

βB 0 0

βB 0 0 0

y1

βλ

p1

βµ

=

b0f

1β

g

. (5.79)

As described earlier, a preconditioner for the above reduced optimality system can be constructedusing the definition (5.7), with transformation matrix

H =

I II −I

.

This gives

P =12(HT A + AH) =

My +

βKy 0

0 My +

βKy

and

R =

B 00 B

My +

βKy 0

0 My +

βKy

−1 B 00 B

T

=

B(Mp +

βKp)−1BT 0

0 B(Mp +

βKp)−1BT

.

(5.80)

Approximating B(Mp +

βKp)−1BT with (K−1p +

βM−1

p )−1 as proposed by Cahouet-Chabardin [22] yields

stk Pixs =

My +

βKy 0 0 00 My +

βKy 0 0

0 0 (K−1p +

βM−1

p )−1 00 0 0 (K−1

p +

βM−1p )−1

. (5.81)

The bounds on the condition number is given by

κ(stk P−1ixs AR) ≤ 4.25,

53

cf. [34]. Further, this preconditioner is robust to the underlying parameters, i.e., h and β andis equivalent to the preconditioner stk Pnsn obtained through operator preconditioning with non-standard norms.

5.2.3.4 Mesh independent Schur complement approximation

For preconditioning the saddle point system (5.75), Rees and Wathen in [17] suggested the blockdiagonal preconditioner5

stk Pbd1 =

A 00 S

(5.82)


stk Pblt1 =

A 0C S

, (5.83)

where

A =

My 0 00 0 00 0 β My

, C =

Ky BT −MyB 0 0

,

and the Schur complement approximation

S =

Ky BT

B 0

M−1

y 00 M−1

p

Ky BT

B 0

T

.

Let us introduce

Ξ =

Ky BT

B 0

, M =

M−1

y 00 M−1

p

.

Then we writeS = ΞM−1ΞT . (5.84)

Further, My and Mp are the approximations of the velocity and the pressure mass matrices,My and Mp respectively; while Ky is the approximation of the velocity stiffness matrix Ky. It ispointed out in [24, 17] that since MINRES is a nonlinear Krylov subspace iterative method, itcannot be used as an inner solver to approximate the block Ξ without running to convergence.It is so suggested to use a stationary method, such as the Inexact Uzawa iteration, as innerpreconditioner. Thus, solving the block Ξ is about solving the system of the form

A BT

B 0

up

=

fg

. (5.85)

For solving the above system using an iterative method, one needs to find a splitting matrix W

such thatρ(I −W

−1Ξ) < 1, (5.86)

where ρ denotes the spectral radius and

W =

Ky 0B −τ Mp

(5.87)

is a block lower triangular matrix with τ as a parameter6 that depends on the eigenvalues ofMp.

5The formulation in [17] also includes the pressure regularization term, which we do not consider here.6The value of τ suggested in [24, 17] is 3/5.

54

Thus, a simple iteration to solve Ξ, also called the inexact Uzawa iteration can be definedby

zk+1 = zk +W−1Ξrk (5.88)

where rk is the residual b − Ξxk. We replace Ξ by Ξn, where n denotes the steps of the inexactUzawa iteration. So we write

S = ΞnM−1ΞT

n . (5.89)

The corresponding algorithm follows:

Algorithm 5 Preconditioned inexact Uzawa iteration1: For j = 1, ..., n, Do:2: yk+1 = yk + K−1

y (f − Kyyk − BT pk)

3: pk+1 = pk + τM−1p (Byk+1 − g)

4: EndFor

Thus the approximation of the Schur complement S is obtained by two Stokes approximationswith the Stokes preconditioners, i.e., on the left we have

Ky BT

B 0

:=

Ky 0B Mp

(5.90)

and on the right Ky BT

B 0

:=

Ky BT

0 Mp

. (5.91)

5.2.3.5 Parameter independent Schur complement approximation

Pearson in [16] re-derives exactly the same preconditioner that is derived by Zulehner in [35]using operator preconditioning with non-standard norms. A block diagonal preconditioner for thereduced optimality system (5.76) is given by

P1 =

A 00 S

.

Noting that block A of the reduced optimality system (5.76) can be related to the distributedoptimal control problem constrained by the Poisson equation, a corresponding preconditioner asproposed by Zulehner [35] can be used to approximate it. Thus

A =

M K

K −1β

M

≡

M +

βK 0

01β

M +1

βK

:= A.

And the Schur complement approximation is given by

S =

B 00 B

M +

βK 0

01β

M +1

βK

−1

BT 00 BT

=

B 00 B

(M +

βK 0

01β

M +1

βK

−1

BT 00 BT

=

B(M +

βK)−1BT 0

0 βB(M +

βK)−1BT

.

(5.92)

55

Approximating B(M +

βK)−1BT with (K−1p +

βM−1

p )−1 as proposed by Cahouet-Chabard[22] gives

S =

(K−1

p +

βM−1p )−1 0

0 β(K−1p +

βM−1

p )−1

(5.93)

This then leads to the preconditioner

stk Pbd2 =

M +

βK 0 0 0

01β(M +

βK) 0 0

0 0 (K−1p +

βM−1

p )−1 00 0 0 β(K−1

p +

βM−1p )−1

. (5.94)

This preconditioner is equivalent to the preconditioners stk Pnsn and stk Pixs.

5.2.3.6 Schur complement approximation using the commutator argument

As we know from earlier discussion that a block diagonal preconditioner for the saddle pointsystem (5.76) is given by

P1 =

A 00 S

.

Since the block A in (5.76) is related to the distributed Poisson control problem, Pearson in [16]approximates it with a preconditioner pson Pbd2 defined earlier, i.e.,

A =

M 00 (K + 1√

βM)M−1(K + 1√

βM)

:= A. (5.95)

The Schur complement approximation is given by

S =

B 00 B

M 00 (K + 1√

βM)M−1(K + 1√

βM)

−1 BT 00 BT

=

BM−1BT 0

0 B(KM−1K +1β

M)−1BT

.

(5.96)

The first block of S can be approximated using the relation

BM−1BT≈ Kp, (5.97)

cf. [18]. The second block of S is approximated using the commutator argument.

Let L = (KM−1K +1β

M)−1. Then we have

Σ := BL−1BT . (5.98)

And the commutator is given byE = (L)∇−∇(L)p, (5.99)

where L = ∆2 +1β

I := ∇4 +1β

I is carefully chosen to approximate Σ.

Discretizing the commutator results in

Eh = (M−1L)M−1BT− M−1BT(M−1

p Lp), (5.100)

56

cf. [16].

Pre-multiplying by BL−1M and post-multiplying with L−1p Mp gives

Eh = (BL−1M)((M−1L)M−1BT)(L−1p Mp)− (BL−1M)M−1BT(M−1

p Lp)(L−1p Mp)

= BM−1BT L−1p Mp − BM−1BT .

(5.101)

Using the crucial assumption that Eh is small gives

BM−1BT L−1p Mp ≈ BM−1BT . (5.102)

Since BM−1BT ≈ Kp, we have

Σ = KTp L−1

p Mp ≈ BL−1BT . (5.103)

And finally

Σ−1≈ M−1

p LpK−1p

= M−1p (KM−1K +

1β

M)−1K−1p

= M−1p Kp M−1

p +1β

K−1p .

(5.104)

This results in the following preconditioner, i.e.,

stk Pcomm =

My 0 0 0

0 (K +1

βMy)M−1

y (K +1

βMy) 0 0

0 0 Kp 0

0 0 (M−1p Kp M−1

p +1β

K−1p )−1

, (5.105)

cf. [16]. This preconditioner is robust to the underlying parameters, i.e., h and β.

5.3 Overview of spectrally equivalent numerical approxima-

tion of discrete differential operators

Improving the spectral properties of A through preconditioning it with P is an important criterionfor a good preconditioner. Yet equally important is that the multiplication of vectors from the leftby P−1 is cost efficient and preserves the upper bound on the condition number, i.e., the practicalapplication of a preconditioner should be spectrally equivalent to its ideal counterpart. Here wepresent the summary of different preconditioner blocks discussed earlier, and refer to them asdiscrete differential operators, followed by an overview of their spectrally equivalent approxima-tions.

• Zero order differential operator

M (the mass matrix)

• Second order differential operators

K (the stiffness matrix)

M +

βK

M +

βF

57

• Fourth order differential operators

(K + 1√β

M)M−1(K + 1√β

M)T

(F + 1√β

M)M−1(F + 1√β

M)T

It is noted in [26] that the approximation of mass matrix M with symmetric Gauss-Seidel iterationis parameter robust. Further, it is noted in [18, 32] that diag(M−1)M is well conditioned forall commonly used finite element basis functions and so M can be approximated through fixednumber of Chebyshev semi-iterations7.

The second order differential operator K, M +

βK, can be robustly approximated by a V-cyclemultigrid iteration, with symmetric Gauss-Seidel or relaxed Jacobi as smoother, according to theanalysis in [36, 37].

5.3.1 Approximation of the discrete differential operators arising in dis-

tributed optimal control of the Poisson equation

In [27], the mass matrix M is approximated using 20 Chebyshev semi iterations. The secondorder differential operator K is approximated with two algebraic multigrid cycles with two pre-smoothing and two post-smoothing steps by the Jacobi (smoother) method. Finally, the 4th or-der differential operator (K + 1√

βM)M−1(K + 1√

βM)T is approximated similarly, i.e., using two

algebraic multigrid cycles with two pre-smoothing and two post-smoothing steps of the Jacobi(smoother) method for each of the 2nd order operator terms.

In [24], M is approximated with 5 Chebyshev semi-iterations and K is approximated using onegeometric multigrid (GMG) V-cycle with three pre-smoothing iterations. Another approach con-sidered for approximating K is through an AMG routine HSL M120 [39], applied via a MATLABinterface with default parameter settings.

In [26, 35, 34], M is approximated using a symmetric Gauss-Seidel iteration, while for K andM +

βK, one step of V-cycle with one step of symmetric Gauss-Seidel iteration for pre and post

smoothing is used. Finally, for the 4th order discrete differential operator (K + 1√β

M)M−1(K +

1√β

M)T, each 2nd order term is approximated with a W-cycle multigrid iteration with symmetric

Gauss-Seidel iteration as smoother.


tributed optimal control of the convection-diffusion equation

In [28], the 4th order differential operator (F + 1√β

M)M−1(F + 1√β

M) is approximated using

the Geometric Multigrid with 4(2-pre and 2-post) smoothing steps with block Gauss-Seidel. Thisimplementation is based on Ramage [38] and further discussed in [18].

The preconditioners proposed by Zulehner [35] and Axelsson [40] have not yet been used to solvefor the optimality system arising in a convection-diffusion control problem. Hence, for approximat-ing the 2nd order differential operator M +

βF that appears in both these preconditions, we

will use one AMG V-cycle with two pre-smoothing and two post-smoothing steps of block Gauss-Seidel (smoother) method in our numerical experiments.

7the observation in [32] is that there is insignificant difference between the convergence of this method and the CGmethod preconditioned with diag(M.)

58


tributed optimal control of the Stokes system

In [24, 27], M is approximated with 20 Chebyshev semi-iterations, while K is approximated withan AMG routine HSL M120 [39], applied via a MATLAB interface with default parameter settings.Further, for approximating each of the two blocks Ξ in (5.84), i.e.,

S = ΞM−1ΞT

it has been suggested to use two inexact Uzawa iterations.

In [16], M is approximated with 20 Chebyshev semi-iterations. Further, both the 2nd orderdifferential operators, M +

βK, and each relevant term in the 4th order differential operator,

(K + 1√β

M)M−1(K + 1√β

M)T, is approximated with an AMG routine HSL M120 [39], applied

via a MATLAB interface using two V-cycles with two pre and post relaxed Jacobi smoothingsteps.

Finally in [26, 35, 34], M is approximated using one step of a symmetric Gauss-Seidel iterationand the 2nd order differential operator, K and M +

βK, are approximated with one step of

AMG V-cycle iteration with one step of symmetric Gauss-Seidel iteration for both pre and postsmoothing processes.

59

That is the reason they are calledlessons," the Gryphon remarked:"because they lessen from day today."


6Numerical results

We solve our three benchmark PDE-constrained optimization problems with appropriate numer-ical preconditioning techniques and compare their efficiency. All the results are obtained withC++ implementation using the open source finite element library deal.ii [58]. Further, deal.iiprovides interface to the Trilinos library [59], giving access to various data structures and precon-ditioners including the algebraic multigrid (AMG) solver that it provides. So whenever we referto the AMG solver in our numerical results, we imply the one provided with the Trilinos library.All experiments were performed on Intel(R) Core(TM) i5 CPU 750 @ 2.67GHz-2.80GHz with in-stalled memory RAM of 4GB. The results presented across all the tables follow the followingconvention:

For each value of β and h, we show the number of outer (MINRES or FGMRES) iterations in thefirst row. The adjacent brackets represent the average inner iterations for each outer iteration; thefirst number to the left shows the average number of Chebyshev semi-iterations and the numberto the right shows the average number of AMG iterations. The number just below the first rowrepresents the CPU times (in seconds) to solve the problem.

6.1 Distributed optimal control problem constrained by the

Poisson equation

Consider the problem

miny,u

12y − y2

L2(Ω) +12

βu2L2(Ω)

s.t. − ∆y = u in Ω,y = y|∂Ω on ∂Ω,

where Ω = [0, 1]2 defines the domain with boundary ∂Ω, β > 0 is the regularization parameter,and y is the desired state given by

y =

(2x1 − 1)2(2x2 − 1)2 if x ∈

0,

12

2,

0 otherwise.

60

This problem was also considered in [24, 25]. Recall that numerically dealing with the distributedoptimal control of the Poisson equation leads to the optimally system

M 0 KT

0 βM −MK −M 0

yuλ

=

b0d

(6.1)


M KT

K −1β

M

.

yλ

=

bd

. (6.2)

The state y, the control u and the adjoint λ are all discretized using the Q1 basis functions.We solve the problem with five different preconditioners discussed earlier, i.e., pson Pbd1 , pson Pblt,pson Pbd2

, pson Pnsn, and pson PF. The relative convergence tolerance of the outer solver is set to10−6 in the L2(Ω) norm. The mass matrix, which corresponds to zero-order discrete differentialoperator, is approximated using 20 Chebyshev semi-iterations with the relative convergence tol-erance set to 10−4 in the L2(Ω) norm. To approximate each block corresponding to higher orderdiscrete differential operators, i.e.,

• M +

βK

• K + 1√β

M

we apply one V-cycle iteration of Algebraic Multigrid (AMG) solver with two pre-smoothing andtwo post-smoothing steps by symmetric Gauss-Seidel (smoother) method with the relative con-vergence tolerance set to 10−4 in the L2(Ω) norm. In case of the block corresponding to thediscrete differential operator

• (K + 1√β

M)M−1(K + 1√β

M)T

the transpose part (K+ 1√β

M)T is approximated in the similar fashion of (K+ 1√β

M). The results

are presented in Tables 6.1-6.5.

6.1.1 Block diagonal preconditioner pson Pbd1, Pearson and Wathen

In Table 6.1, we present the results of preconditioning the saddle point system (6.1) with the blockdiagonal preconditioner

pson Pbd1 =

M 0 00 β M 00 0 (K + 1√

βM)M−1(K + 1√

βM)T

(6.3)

using MINRES as an outer solver.

61

β

h Size 10−2 10−3 10−4 10−5 10−6 10−7 10−8 10−9 10−10

2−4 867 12(17+2) 13(18+2) 13(19+2) 12(21+2) 13(21+2) 12(23+2) 10(24+2) 8(25+2) 6(26+2)0.005 0.005 0.006 0.005 0.006 0.006 0.005 0.004 0.003

2−5 3267 11(14+2) 13(15+2) 13(16+2) 13(18+2) 12(19+2) 12(20+2) 12(22+2) 10(24+2) 8(24+2)0.011 0.013 0.014 0.015 0.014 0.014 0.015 0.013 0.010

2−6 12675 11(12+9) 13(13+8) 13(14+7) 13(15+7) 12(17+6) 12(18+5) 12(19+4) 12(21+4) 10(23+6)0.082 0.089 0.087 0.085 0.076 0.070 0.068 0.067 0.069

2−7 49923 11(10+7) 13(11+7) 13(12+7) 13(13+7) 12(14+6) 11(16+6) 11(17+6) 11(18+4) 10(20+4)0.275 0.325 0.330 0.330 0.301 0.273 0.268 0.234 0.212

2−8 198147 11(9+9) 13(10+9) 13(10+8) 13(11+7) 12(12+7) 11(13+7) 11(15+6) 11(17+6) 9(18+4)1.389 1.626 1.538 1.452 1.377 1.232 1.211 1.232 0.863

Table 6.1: Distributed optimal control problem constrained by the Poisson equation, block diagonal pre-conditioner pson Pbd1 , Pearson and Wathen.

6.1.2 Block lower triangular preconditioner pson Pblt2, Pearson and Wathen

In Table 6.2, we present the results of preconditioning the saddle point system (6.1) with the blocklower triangular preconditioner

pson Pblt =

M 0 00 β M 0K −M −(K + 1√

βM)M−1(K + 1√

βM)T

(6.4)

using FGMRES as the outer solver.

β

h Size 10−2 10−3 10−4 10−5 10−6 10−7 10−8 10−9 10−10

2−4 867 13(12+2) 15(14+2) 15(16+2) 15(17+2) 15(18+2) 15(20+2) 15(22+2) 11(22+2) 9(22+2)0.006 0.007 0.009 0.009 0.009 0.009 0.011 0.008 0.005

2−5 3267 13(10+2) 15(11+2) 15(13+2) 17(14+2) 15(14+2) 17(17+2) 17(19+2) 16(20+2) 13(21+2)0.014 0.016 0.017 0.020 0.017 0.028 0.030 0.021 0.017

2−6 12675 13(8+10) 15(9+9) 17(10+8) 18(11+7) 18(12+6) 17(13+6) 17(14+4) 18(16+4) 17(18+6)0.102 0.109 0.129 0.129 0.121 0.104 0.096 0.010 0.116

2−7 49923 14(7+8) 16(7+8) 18(8+7) 18(9+7) 19(10+6) 19(11+6) 17(11+6) 18(13+4) 18(15+4)0.373 0.435 0.467 0.443 0.459 0.454 0.407 0.363 0.375

2−8 198147 16(6+10) 16(7+10) 18(7+8) 17(6+8) 17(8+8) 17(8+7) 19(9+6) 19(10+6) 21(12+4)2.150 2.132 2.152 2.002 1.995 1.888 1.955 1.980 1.828

Table 6.2: Distributed optimal control problem constrained by the Poisson equation, block lower triangularpreconditioner pson Pblt2 , Pearson and Wathen.

6.1.3 Block diagonal preconditioner pson Pbd2, Pearson and Wathen

In Table 6.3, we present the results of preconditioning the reduced optimality system (6.2) withthe block diagonal preconditioner

pson Pbd2 =

M 00 (K + 1√

βM)M−1(K + 1√

βM)T

. (6.5)

Again, we use MINRES as the outer solver.

62

β

h Size 10−2 10−3 10−4 10−5 10−6 10−7 10−8 10−9 10−10

2−4 578 11(2+9) 14(2+9) 15(2+10) 16(2+11) 16(2+12) 13(2+13) 9(2+13) 5(2+14) 4(2+15)0.003 0.007 0.01 0.01 0.007 0.007 0.006 0.004 0.004

2−4 2178 11(2+8) 14(2+8) 15(2+9) 16(2+9) 17(2+11) 16(2+12) 13(2+13) 9(2+13) 5(2+14)0.008 0.01 0.01 0.011 0.013 0.012 0.01 0.007 0.008

2−4 8450 14(2+11) 14(2+11) 15(2+10) 16(2+10) 17(2+9) 17(2+10) 15(2+11) 13(2+12) 9(2+12)0.053 0.052 0.054 0.057 0.059 0.061 0.057 0.05 0.034

2−4 33282 11(2+11) 13(2+11) 15(2+10) 16(2+10) 17(2+9) 17(2+9) 15(2+9) 15(2+10) 12(2+11)0.167 0.191 0.217 0.228 0.239 0.235 0.21 0.217 0.18

2−4 132098 18(2+11) 15(2+11) 15(2+11) 16(2+10) 17(2+10) 17(2+9) 16(2+9) 15(2+8) 13(2+10)1.162 0.974 0.967 1.01 1.056 1.042 0.97 0.9 0.818

Table 6.3: Distributed optimal control problem constrained by the Poisson equation, block diagonal pre-conditioner pson Pbd2 , Pearson and Wathen.

6.1.4 Block diagonal preconditioner pson Pnsn , W. Zulehner

In Table 6.4, we present the results of preconditioning the reduced optimality system (6.2)

pson Pnsn =

M +

βK 0

01β(M +

βK)

. (6.6)

MINRES is used as the outer solver here.

β

h Size 10−2 10−3 10−4 10−5 10−6 10−7 10−8 10−9 10−10

2−4 578 12(2+0) 14(2+0) 14(2+0) 12(2+0) 12(2+0) 12(2+0) 9(2+0) 7(2+0) 5(2+0)0.007 0.008 0.008 0.007 0.007 0.007 0.007 0.007 0.004

2−5 2178 12(2+0) 14(2+0) 14(2+0) 13(2+0) 12(2+0) 12(2+0) 11(2+0) 9(2+0) 7(2+0)0.005 0.006 0.006 0.005 0.005 0.005 0.005 0.004 0.003

2−6 8450 14(2+0) 14(2+0) 14(2+0) 14(2+0) 12(2+0) 12(2+0) 11(2+0) 11(2+0) 9(2+0)0.035 0.035 0.035 0.035 0.031 0.031 0.03 0.029 0.023

2−7 33282 12(2+0) 14(2+0) 14(2+0) 14(2+0) 13(2+0) 12(2+0) 12(2+0) 11(2+0) 11(2+0)0.125 0.144 0.144 0.145 0.134 0.125 0.125 0.115 0.114

2−8 132098 14(2+0) 14(2+0) 14(2+0) 14(2+0) 13(2+0) 12(2+0) 12(2+0) 11(2+0) 11(2+0)0.607 0.608 0.607 0.609 0.567 0.528 0.528 0.489 0.489

Table 6.4: Distributed optimal control problem constrained by the Poisson equation, block diagonal pre-conditioner pson Pnsn , W. Zulehner.

6.1.5 Block factorization preconditioner pson PF, Axelsson and Neytcheva

Here we use the transformation of (6.2), i.e.,

ART =

M −βKT

K M

(6.7)

and precondition it with

pson PF =

M −βKT

K M + 2

βK

. (6.8)

FGMRES is used as an outer solver here. We use Algorithm (4) described earlier to solve thesystem, which requires one AMG solve to approximate the block M +

βK twice. The results

are presented in Table 6.5.

63

β

h Size 10−2 10−3 10−4 10−5 10−6 10−7 10−8 10−9 10−10

2−4 578 6(2+0) 7(2+0) 7(2+0) 7(2+0) 7(2+0) 7(2+0) 6(2+0) 4(2+0) 3(2+0)0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.001 0.001

2−5 2178 6(2+0) 6(2+0) 7(2+0) 7(2+0) 7(2+0) 7(2+0) 6(2+0) 6(2+0) 4(2+0)0.003 0.003 0.004 0.004 0.004 0.004 0.003 0.007 0.002

2−6 8450 6(2+0) 7(2+0) 7(2+0) 7(2+0) 6(2+0) 6(2+0) 6(2+0) 6(2+0) 5(2+0)0.018 0.02 0.021 0.021 0.018 0.018 0.019 0.018 0.019

2−7 33282 5(2+0) 6(2+0) 6(2+0) 6(2+0) 6(2+0) 6(2+0) 6(2+0) 5(2+0) 5(2+0)0.061 0.072 0.072 0.072 0.072 0.072 0.072 0.061 0.061

2−8 132098 6(2+0) 6(2+0) 6(2+0) 6(2+0) 6(2+0) 6(2+0) 6(2+0) 5(2+0) 5(2+0)0.305 0.306 0.304 0.304 0.305 0.304 0.305 0.261 0.262

Table 6.5: Distributed optimal control problem constrained by the Poisson equation, block preconditionerpson PF, Axelsson and Neytcheva.

6.1.6 Discussion

The regularization parameter β determines how close the state y approaches the desired state y.We now illustrate the behavior of cost functional J for different values of β. For the purpose, wereproduce Table 6.6 as done by Youngsoo in [25], with slight modifications. We observe that as βdecreases, y → y. Another observation is that u increases with decreasing β. The explanationprovided in [25] is that it makes sense, otherwise the cost functional J will be insensitive to uas β becomes small.

β iter u2 y − y2 y − y2/y2 J b −Ax2/b2 time2e-02 5 4.7e+00 3.96e-02 3.96e-01 2.25e-01 3.94e-07 0.0112e-03 6 2.6e+01 2.87e-02 2.87e-01 6.70e-01 1.56e-06 0.0122e-04 6 7.1e+01 1.42e-02 1.42e-01 5.01e-01 1.76e-06 0.0122e-05 6 1.2e+02 4.55e-03 4.55e-02 1.51e-01 1.69e-06 0.0122e-06 6 1.6e+02 1.22e-03 1.22e-02 2.49e-02 1.74e-06 0.0122e-07 6 1.8e+02 3.09e-04 3.08e-03 3.20e-03 1.68e-06 0.0122e-08 6 1.9e+02 8.32e-05 8.32e-04 3.68e-04 1.48e-06 0.0122e-09 6 2.0e+02 3.95e-05 3.94e-04 4.06e-05 1.08e-06 0.0122e-10 5 2.1e+02 3.60e-05 3.60e-04 4.36e-06 2.87e-06 0.010

Table 6.6: Study of the cost functional J of the Distributed optimal control problem con-

strained by the Poisson equation The table is produced using the preconditioner pson PF. For mesh size 2−6,we show the number of outer FGMRES iterations represented as "iter". u represents the L2(Ω) norm of the control u.y − y measures how closely the state y matches the desired state y. y − y/y measures the relative error. J isthe calculated cost functional1. b −Ax/b represents the residual norm of the KKT system of equations to show thesystem converged in the L2(Ω) norm. Finally, the last column tells us the time (sec) it took to solve the system.

Moreover, observing the iteration count presented in Tables 6.1-6.5, it is clear that all five precon-ditioners are robust with respect to mesh size h and the regularization parameter β.

In Figure 6.1, we present the snapshot of performance of different preconditioners in terms ofiterations required to solve the relevant saddle point systems for various values of β when themesh size h = 2−6.

1The differences between the cost functionals using all other preconditioners are insignificant.

64

Pearso

n 3x3 BD

Pearso

n 3x3 BLT

Pearso

n 2x2 BD

Zulehn

er2x

2

Axelss

on2x

2

10

1513

18

16

14

7

itera

tions

(a) β = 10−5

Pearso

n 3x3 BD

Pearso

n 3x3 BLT

Pearso

n 2x2 BD

Zulehn

er2x

2

Axelss

on2x

2

5

10

15

12

1817

12

6

itera

tions

(b) β = 10−6

Pearso

n 3x3 BD

Pearso

n 3x3 BLT

Pearso

n 2x2 BD

Zulehn

er2x

2

Axelss

on2x

25

10

15

12

17 17

12

6

itera

tions

(c) β = 10−7

Pearso

n 3x3 BD

Pearso

n 3x3 BLT

Pearso

n 2x2 BD

Zulehn

er2x

2

Axelss

on2x

25

10

15

12

17

15

11

6

itera

tions

(d) β = 10−8

Figure 6.1: For mesh size h = 2−6, iteration count comparison across different preconditioners for variousvalues of β.

Here we observe that the preconditioner pson PF is the most efficient compared to others. Thepreconditioner pson Pnsn takes the 2nd place. The results of preconditioners, pson Pbd1 and pson Pblt,are in line with what is obtained in [27]. The preconditioner pson Pbd2 being the reduced version ofpson Pbd1 , however, did not appear to perform any better as was expected.

In Figure 6.2, we present the CPU time (sec) required to solve the relevant saddle point systemsfor various values of β when mesh size h = 2−6. Again, preconditioner pson PF appears to bethe most efficient, followed by the preconditioner pson Pnsn. Moreover, the preconditioner pson Pbd2

for the reduced optimality system performed better compared to the preconditioners pson Pbd1 andpson Pblt for the full optimality system. Thus reducing the optimality system shows clear advantagein decreasing the amount of time needed to solve the problem.

65

Pearso

n 3x3 BD

Pearso

n 3x3 BLT

Pearso

n 2x2 BD

Zulehn

er2x

2

Axelss

on2x

2

5 · 10−2

0.18.5 · 10−2

0.13

5.7 · 10−2

3.5 · 10−2

2.1 · 10−2

CP

Utim

e(s

ec)

(a) β = 10−5

Pearso

n 3x3 BD

Pearso

n 3x3 BLT

Pearso

n 2x2 BD

Zulehn

er2x

2

Axelss

on2x

2

5 · 10−2

0.17.6 · 10−2

0.12

5.9 · 10−2

3.1 · 10−2

1.8 · 10−2

CP

Utim

e(s

ec)

(b) β = 10−6

Pearso

n 3x3 BD

Pearso

n 3x3 BLT

Pearso

n 2x2 BD

Zulehn

er2x

2

Axelss

on2x

2

5 · 10−2

0.1

7 · 10−2

0.1

6.1 · 10−2

3.1 · 10−2

1.8 · 10−2

CP

Utim

e(s

ec)

(c) β = 10−7

Pearso

n 3x3 BD

Pearso

n 3x3 BLT

Pearso

n 2x2 BD

Zulehn

er2x

2

Axelss

on2x

2

2 · 10−2

4 · 10−2

6 · 10−2

8 · 10−2

0.1

6.8 · 10−2

9.6 · 10−2

5.7 · 10−2

3 · 10−2

1.9 · 10−2

CP

Utim

e(s

ec)

(d) β = 10−8

Figure 6.2: For mesh size h = 2−6, CPU time (sec) comparison across different preconditioners for variousvalues of β.

Finally, using the preconditioner pson PF to solve the problem, we reproduce the plots for the statey and the control u for various values of β, as obtained by Youngsoo in [25].

(a) β = 2 × 10−2 (b) u, β = 2 × 10−2

66

(c) β = 2 × 10−4 (d) β = 2 × 10−4

(e) β = 2 × 10−6 (f) β = 2 × 10−6

Figure 6.3: State (y) and control (u) distribution for different values of β

.

Figures 6.3(a), 6.3(c) and 6.3(e) represent the state y of the system while Figures 6.3(b), 6.3(d)and 6.3(f) represent the control u.


convection-diffusion equation

Consider the following distributed optimal control problem constrained by the convection-diffusionequation:

miny,u

J (y, u) =12y − y2

L2(Ω)+12

βu2L2(Ω)

s.t. − ε∆y + (w ·∇)y + cy = u in Ω,y = y|∂Ω on ∂Ω,

(6.9)

where Ω = [0, 1]2 defines the domain with boundary ∂Ω, β > 0 is the regularization parameter, wrepresents divergence free wind, and > 0 represents viscosity. We take w = [cosθ, sinθ] for θ =π4 so that the maximum value of w2 = 1 on Ω. Further, y is the desired state given by

67

y =

(2x1 − 1)2(2x2 − 1)2 if x ∈

0,

12

,

0 otherwise.(6.10)

This problem was also considered in (Chapter 6, [24]). As noted earlier, numerically tackling thedistributed optimal control of the convection-diffusion equation using the local projection stabiliza-tion (LPS) scheme leads to the optimality system

M 0 FT

0 βM −MF −M 0

yuλ

=

b0d

(6.11)


M FT

F −1β

M

yλ

=

bd

, (6.12)

where F is a non-symmetric block. The state y, control u and the adjoint λ are discretized usingQ1 basis functions. We solve2 the problem with five different preconditioners discussed earlier,i.e., cd Pbd1 , cd Pblt, cd Pbd2

, cd Pnsn, and cd PF. The relative convergence tolerance of the outersolver (FGMRES) is set to 10−6 in the L2(Ω) norm. Each operation on the mass matrix, whichcorresponds to zero order discrete differential operator, is approximated using 20 Chebyshevsemi-iterations with the relative convergence tolerance set to 10−4 in the L2(Ω) norm. To approx-imate each block corresponding to higher order discrete differential operators, i.e.,

• M +

βF

• F + 1√β

M

we use a V-cycle Algebraic Multigrid (AMG). Each solve with AMG employs one V-cycle iterationwith two pre-smoothing and two post-smoothing steps by block Gauss-Seidel (smoother) methodwith the relative convergence tolerance set to 10−4 in the L2(Ω) norm. In case of the blockcorresponding to the discrete differential operator

• (F + 1√β

M)M−1(F + 1√β

M)T

the transpose part (F + 1√β

M)T is approximated in the similar fashion of (F + 1√β

M).

For = 1/500, the results3 are presented in Tables 6.7-6.11.

6.2.1 Block diagonal preconditioner cd Pbd1, Pearson and Wathen

In Table 6.7, we present the results of preconditioning the saddle point system (6.11) with theblock diagonal preconditioner

pson Pbd1 =

M 0 00 β M 00 0 (F + 1√

βM)M−1(F + 1√

βM)T

. (6.13)

2Solving for the optimality systems (6.11) and (6.12) using the LPS scheme is discussed in Chapter 2.3The LPS scheme requires computing the approximate solutions y and λ, see Chapter 2. In our numerical experi-

ments, we use one outer iteration for this purpose, but it is not accounted for in the iteration count presented in Tables6.7-6.11.

68

β

h Size 10−2 10−3 10−4 10−5 10−6 10−7 10−8 10−9 10−10

2−4 867 19(2+19) 17(2+18) 19(2+21) 18(2+22) 14(2+23) 10(2+24) 8(2+24) 8(2+24) 6(2+25)0.011 0.009 0.011 0.011 0.008 0.006 0.005 0.005 0.004

2−5 3267 23(2+17) 21(2+16) 24(2+20) 20(2+22) 16(2+23) 12(2+23) 9(2+23) 8(2+24) 8(2+25)0.031 0.026 0.033 0.028 0.022 0.016 0.012 0.011 0.011

2−6 12675 29(2+15) 27(2+14) 23(2+17) 19(2+20) 20(2+22) 14(2+22) 11(2+22) 10(2+22) 8(2+23)0.129 0.116 0.103 0.088 0.097 0.067 0.053 0.049 0.04

2−7 49923 37(2+13) 33(2+13) 25(2+16) 19(2+16) 19(2+20) 18(2+21) 14(2+21) 10(2+21) 9(2+22)0.725 0.63 0.486 0.372 0.397 0.385 0.296 0.215 0.196

2−8 198147 48(2+11) 46(2+11) 29(2+12) 19(2+14) 19(2+17) 17(2+19) 15(2+20) 12(2+21) 9(2+20)3.934 3.773 2.472 1.655 1.79 1.667 1.485 1.226 0.937

Table 6.7: Distributed optimal control problem constrained by the convection-diffusion equation, blockdiagonal preconditioner cd Pbd1 , Pearson and Wathen.

6.2.2 Block lower triangular preconditioner cd Pblt , Pearson and Wathen

In Table 6.8, we present the results of preconditioning the saddle point system (6.11) with theblock lower triangular preconditioner

cd Pblt =

M 0 00 β M 0F −M −(F + 1√

βM)M−1(F + 1√

βM)T

. (6.14)

β

h Size 10−2 10−3 10−4 10−5 10−6 10−7 10−8 10−9 10−10

2−4 867 17(2+18) 17(2+17) 19(2+21) 18(2+23) 14(2+24) 10(2+24) 8(2+24) 8(2+25) 6(2+25)0.009 0.009 0.011 0.011 0.008 0.006 0.005 0.005 0.004

2−5 3267 23(2+16) 21(2+15) 21(2+19) 20(2+22) 16(2+23) 12(2+23) 9(2+23) 8(2+23) 8(2+24)0.03 0.026 0.029 0.029 0.023 0.017 0.013 0.012 0.012

2−6 12675 29(2+14) 27(2+13) 23(2+17) 19(2+19) 18(2+21) 14(2+22) 11(2+22) 9(2+22) 8(2+23)0.133 0.116 0.107 0.091 0.089 0.07 0.055 0.046 0.042

2−7 49923 38(2+13) 33(2+11) 25(2+14) 19(2+16) 17(2+19) 16(2+20) 13(2+21) 11(2+22) 9(2+22)0.729 0.614 0.488 0.382 0.359 0.344 0.283 0.245 0.202

2−8 198147 48(2+10) 43(2+8) 27(2+11) 19(2+14) 17(2+16) 17(2+19) 15(2+20) 11(2+20) 11(2+21)3.959 3.435 2.333 1.682 1.584 1.666 1.521 1.099 1.137

Table 6.8: Distributed optimal control problem constrained by the convection-diffusion equation, blocklower triangular preconditioner cd Pblt , Pearson and Wathen.

6.2.3 Block diagonal preconditioner cd Pbd2, Pearson and Wathen


pson Pblt =

M 00 (K + 1√

βM)M−1(K + 1√

βM)T

. (6.15)

69

β

h Size 10−2 10−3 10−4 10−5 10−6 10−7 10−8 10−9 10−10

2−4 578 22(2+8) 25(2+8) 24(2+10) 18(2+10) 11(2+11) 8(2+11) 6(2+11) 5(2+12) 5(2+12)0.011 0.012 0.011 0.007 0.004 0.003 0.002 0.002 0.002

2−5 2178 24(2+7) 37(2+7) 34(2+8) 23(2+9) 14(2+10) 9(2+10) 7(2+10) 5(2+11) 5(2+11)0.022 0.037 0.036 0.022 0.013 0.008 0.007 0.005 0.005

2−6 8450 32(2+6) 47(2+5) 42(2+7) 27(2+8) 20(2+9) 12(2+9) 8(2+9) 7(2+10) 5(2+11)0.097 0.132 0.124 0.083 0.062 0.037 0.025 0.023 0.017

2−7 33282 40(2+5) 70(2+5) 51(2+5) 39(2+7) 24(2+7) 17(2+8) 11(2+8) 8(2+9) 7(2+9)0.534 0.921 0.678 0.544 0.338 0.245 0.158 0.118 0.106

2−8 132098 48(2+4) 105(2+4) 73(2+5) 42(2+5) 28(2+6) 21(2+7) 15(2+8) 10(2+8) 8(2+8)2.751 6.015 4.278 2.544 1.755 1.336 0.972 0.661 0.545

Table 6.9: Distributed optimal control problem constrained by the convection-diffusion equation,

block diagonal preconditioner cd Pbd2 , Pearson and Wathen

6.2.4 Block diagonal preconditioner cd Pnsn , W. Zulehner


cd Pnsn =

M +

βF 0

01β(M +

βF)

. (6.16)

β

h Size 10−2 10−3 10−4 10−5 10−6 10−7 10−8 10−9 10−10

2−4 578 20(2+0) 17(2+0) 20(2+0) 18(2+0) 12(2+0) 8(2+0) 6(2+0) 6(2+0) 4(2+0)0.006 0.005 0.006 0.005 0.003 0.002 0.002 0.002 0.001

2−5 2178 24(2+0) 22(2+0) 22(2+0) 20(2+0) 16(2+0) 10(2+0) 8(2+0) 6(2+0) 6(2+0)0.016 0.014 0.014 0.012 0.01 0.006 0.005 0.004 0.004

2−6 8450 31(2+0) 28(2+0) 26(2+0) 22(2+0) 18(2+0) 14(2+0) 10(2+0) 8(2+0) 6(2+0)0.079 0.068 0.062 0.051 0.041 0.032 0.023 0.019 0.014

2−7 33282 39(2+0) 37(2+0) 31(2+0) 23(2+0) 20(2+0) 16(2+0) 12(2+0) 8(2+0) 6(2+0)0.424 0.404 0.347 0.248 0.214 0.17 0.129 0.088 0.068

2−8 132098 45(2+0) 47(2+0) 35(2+0) 25(2+0) 21(2+0) 19(2+0) 16(2+0) 10(2+0) 8(2+0)2.157 2.251 1.705 1.197 1.023 0.906 0.762 0.484 0.395

Table 6.10: Distributed optimal control problem constrained by the convection-diffusion equation, blockdiagonal preconditioner cd Pnsn , W. Zulehner.

6.2.5 Block factorization preconditioner cd PF, Axelsson and Neytcheva

Here we use the transformation of (6.12), i.e.,

M −βFT

F M

yλ

=

bd

, (6.17)

and precondition it with

cd PF =

M −βFT

F M + 2

βF

. (6.18)

We use Algorithm (4) described earlier to solve the system. It requires one AMG solve to approx-imate the block M +

βF twice. The results are presented in Table 6.11.

70

β

h Size 10−2 10−3 10−4 10−5 10−6 10−7 10−8 10−9 10−10

2−4 578 9(2+0) 9(2+0) 10(2+0) 8(2+0) 6(2+0) 4(2+0) 3(2+0) 3(2+0) 2(3+0)0.003 0.002 0.003 0.002 0.002 0.001 0.001 0.001 0.001

2−5 2178 11(2+0) 11(2+0) 11(2+0) 9(2+0) 7(2+0) 5(2+0) 3(2+0) 3(2+0) 2(3+0)0.007 0.007 0.007 0.005 0.004 0.003 0.002 0.002 0.002

2−6 8450 14(2+0) 14(2+0) 12(2+0) 9(2+0) 8(2+0) 5(2+0) 4(2+0) 3(2+0) 2(3+0)0.032 0.03 0.025 0.019 0.017 0.012 0.01 0.008 0.006

2−7 33282 17(2+0) 16(2+0) 13(2+0) 10(2+0) 8(2+0) 6(2+0) 4(2+0) 3(2+0) 3(2+0)0.19 0.176 0.143 0.111 0.09 0.07 0.05 0.04 0.04

2−8 132098 19(2+0) 18(2+0) 15(2+0) 10(2+0) 8(2+0) 7(2+0) 5(2+0) 4(2+0) 3(2+0)0.946 0.888 0.742 0.504 0.42 0.364 0.272 0.227 0.182

Table 6.11: Distributed optimal control problem constrained by the convection-diffusion equation, blockpreconditioner cd PF, Axelsson and Neytcheva.

6.2.6 Discussion

Observing the iteration count in Tables 6.7-6.11, it is clear that all these preconditioners are robustwith respect to mesh size h and the regularization parameter β.

We now illustrate the behavior of the cost functional J for different values of β. Recall that howclosely the state y approaches the desired state y is determined by the regularization parameterβ. However we observe (across all tested preconditioners) that y − y stops to go down anyfurther around β ≤ 10−6. Further, we observe that u stops increasing further around β ≤ 10−6.This implies that the optimal value of β for the problem is around 10−6.

β iter u2 y − y2 y − y2/y2 J b −Ax2/b2 time1e-02 14 5.79e+01 6.98e-02 6.98e-01 1.67e+01 4.40e-08 0.0321e-03 14 4.94e+01 1.09e-02 1.09e-01 1.22e+00 7.46e-09 0.0301e-04 12 4.94e+01 1.98e-03 1.98e-02 1.22e-01 3.69e-08 0.0251e-05 9 5.08e+01 3.77e-04 3.77e-03 1.29e-02 4.95e-08 0.0191e-06 8 5.19e+01 6.61e-05 6.60e-04 1.34e-03 1.51e-08 0.0171e-07 5 5.22e+01 3.62e-05 3.62e-04 1.36e-04 5.32e-08 0.0121e-08 4 5.22e+01 3.61e-05 3.61e-04 1.36e-05 8.63e-09 0.0101e-09 3 5.22e+01 3.61e-05 3.61e-04 1.36e-06 6.94e-09 0.0081e-10 2 5.22e+01 3.61e-05 3.61e-04 1.37e-07 3.58e-08 0.006

Table 6.12: Study of the cost functional J of the distributed optimal control problem con-

strained by the convection-diffusion equation The table is produced using the preconditioner cd PF. Formesh size 2−6, we show the number of outer FGMRES iterations represented as "iter". u represents the L2(Ω) norm ofthe control u. y − y measures how closely the state y matches the desired state y. y − y/y measures the relativeerror. J is the calculated cost functional4. b −Ax/b represents the residual norm of the KKT system of equations toshow the system converged in the L2(Ω) norm. Finally, the last column tells us the time (sec) it took to solve the system.

In Figure 6.4, we present the snapshot of performance of different preconditioners in terms ofiterations required to solve the relevant saddle point systems for various values of β when themesh size h = 2−6. It is clear that the preconditioner cd PF outperforms all the other four precon-ditioners.

4The differences between the cost functionals using all other preconditioners are insignificant

71

Pearso

n 3x3 BD

Pearso

n 3x3 BLT

Pearso

n 2x2 BD

Zulehn

er2x

2

Axelss

on2x

2

10

20 19 19

27

22

9

itera

tions

(a) β = 10−5

Pearso

n 3x3 BD

Pearso

n 3x3 BLT

Pearso

n 2x2 BD

Zulehn

er2x

2

Axelss

on2x

2

10

15

2020

1820

18

8

itera

tions

(b) β = 10−6

Pearso

n 3x3 BD

Pearso

n 3x3 BLT

Pearso

n 2x2 BD

Zulehn

er2x

2

Axelss

on2x

2

5

10

15 14 14

12

14

5

itera

tions

(c) β = 10−7

Pearso

n 3x3 BD

Pearso

n 3x3 BLT

Pearso

n 2x2 BD

Zulehn

er2x

2

Axelss

on2x

2

4

6

8

10

1211 11

8

10

4

itera

tions

(d) β = 10−8

Figure 6.4: For mesh size h = 2−6, iteration count comparison across different preconditionersfor various values of β.

In Figure 6.5, we present the snapshot of performance of the five preconditioners in terms of theCPU time (sec) required to solve the relevant saddle point systems for various values of β whenmesh size h = 2−6. We find a clear advantage of reducing the optimality system. Again, thepreconditioner cd PF performs exceptionally well.

72

Pearso

n 3x3 BD

Pearso

n 3x3 BLT

Pearso

n 2x2 BD

Zulehn

er2x

2

Axelss

on2x

2

2 · 10−2

4 · 10−2

6 · 10−2

8 · 10−2

0.18.8 · 10−2 9.1 · 10−2

8.3 · 10−2

5.1 · 10−2

1.9 · 10−2

CP

Utim

e(s

ec)

(a) β = 10−5

Pearso

n 3x3 BD

Pearso

n 3x3 BLT

Pearso

n 2x2 BD

Zulehn

er2x

2

Axelss

on2x

2

2 · 10−2

4 · 10−2

6 · 10−2

8 · 10−2

0.1 9.7 · 10−2

8.9 · 10−2

6.2 · 10−2

4.1 · 10−2

1.7 · 10−2

CP

Utim

e(s

ec)

(b) β = 10−6

Pearso

n 3x3 BD

Pearso

n 3x3 BLT

Pearso

n 2x2 BD

Zulehn

er2x

2

Axelss

on2x

2

2

4

6

·10−2

6.7 · 10−2 7 · 10−2

3.7 · 10−2

3.2 · 10−2

1.2 · 10−2

CP

Utim

e(s

ec)

(c) β = 10−7

Pearso

n 3x3 BD

Pearso

n 3x3 BLT

Pearso

n 2x2 BD

Zulehn

er2x

2

Axelss

on2x

2

2

4

6·10−2

5.3 · 10−2 5.5 · 10−2

2.5 · 10−22.3 · 10−2

1 · 10−2

CP

Utim

e(s

ec)

(d) β = 10−8

Figure 6.5: For mesh size h = 2−6, CPU time (sec) comparison across different preconditionersfor various values of β.

The results of our numerical experiments clearly indicate that the preconditioner cd PF is veryeffective for solving convection-diffusion control problems. Further, it is shown that this precon-ditioner is also robust to the mesh size h and the regularization parameter β. When comparedto other preconditioners in terms of iteration count and computational time, it turned out to be aclear winner.

The results for = 1/1500 are presented is Appendix A. We find that all these five preconditionersare also robust to smaller values of . We again find cd PF to be the best in terms of parameterrobustness, iteration count and time requirement.

Finally, we produce the plots for the state y and the control u for two different values of β togive a visual clue to the solutions obtained. The mesh size is set to h = 2−6 and we use thepreconditioner cd PF to solve the problem.

73

(a) β = 10−4 (b) β = 10−4

(c) β = 10−6 (d) β = 10−6

Figure 6.6: State (y) and control (u) distribution for different values of β.

Figures 6.6(a) and 6.6(c) represent the state y of the system while Figures 6.6(b) and 6.6(f)represent the control u.


Stokes system

Consider the following distributed optimal control problem (also referred as the velocity trackingproblem with distributed control) constrained by the Stokes system, cf. [35]:

miny,u

12y − y2

L2(Ω) +12

βu2L2(Ω)

s.t. − ∆y +∇p = u in Ω∇ ·y = 0 in Ω

y = y|∂Ω on ∂Ω.

(6.19)

74

where Ω = [0, 1]2 defines the domain with boundary ∂Ω, β > 0 is the regularization parameter,and the desired state y(x1, x2) = (Y1(x1, x2), Y2(x1, x2)) is given by

Y1(x1, x2) = 10∂

∂x2(ϕ(x1)ϕ(x2)) and Y2(x1, x2) = −10

∂

∂x1(ϕ(x1)ϕ(x2))

withϕ(z) = (1 − cos(0.8πz))(1 − z)2.

As we noted earlier, numerically dealing with the distributed optimal control of the Stokes systemleads to the optimality system

My 0 0 Ky BT

0 0 0 B 00 0 βMy −MT

y 0Ky BT −My 0 0B 0 0 0 0

ypuλµ

=

b00fg

, (6.20)


My Ky 0 BT

Ky −1β

My BT 0

0 B 0 0B 0 0 0

ypλµ

=

b0fg

. (6.21)

We discretize the problem with inf-sup stable [4] Taylor-hood (also known as the Q2-Q1 pair)finite element basis functions. Thus, the state y, control u and the adjoint λ are discretizedusing the Q2 basis functions, while the pressure p and its corresponding adjoint µ are discretizedusing the Q1 basis functions. The results obtained by solving the problem with three differentpreconditioners discussed earlier, i.e., stk Pbd1 , stk Pcomm and stk Pnsn are presented in Tables 6.13-6.15. We use MINRES as the outer solver with relative convergence tolerance set to 10−6 in theL2(Ω) norm. The mass matrix, which corresponds to zero order discrete differential operator,is approximated using 20 Chebyshev semi-iterations with the relative convergence tolerance setto 10−4 in the L2(Ω) norm. To approximate each matrix, corresponding to higher order discretedifferential operators, i.e.,

• K

• M +

βK

• K + 1√β

M

we use a V-cycle Algebraic Multigrid (AMG). Each solve with AMG employs one V-cycle iterationwith two pre-smoothing and two post-smoothing steps by symmetric Gauss-Seidel (smoother)method with the relative convergence tolerance set to 10−4 in the L2(Ω) norm.

6.3.1 Block diagonal preconditioner stk Pbd1, Rees and Wathen

We first solve the saddle point system (6.20) using the block diagonal preconditioner proposed in[24, 17], i.e.,

stk Pbd1 =

A 00 S

, (6.22)

75

where

A =

My 0 00 0 00 0 β My

,

and the Schur complement approximation

S =

Ky BT

B 0

M−1

y 00 M−1

p

Ky BT

B 0

T

,

or simplyS = ΞM−1ΞT .

The problem presented in these papers also contains a pressure regularization term. Howeverin our case, we will solve the problem without it. Further, the authors of the papers present thetable for β = 10−2 only. In Table 6.13, we however present the results for β = 10−2 along withβ = 10−3 and β = 10−4.

The application of this preconditioner requires one solve with Chebyshev semi-iteration eachto approximate the mass matrix My on block (1, 1) and block (3, 3) respectively. Further, theapproximation of the block Ξ requires two inexact Uzawa iterations. Within each inexact Uzawaiteration, one solve with AMG is needed to approximate the stiffness matrix Ky and one solve withChebyshev semi-iteration is required to approximate the mass matrix Mp. Similar operations arerequired to approximate the block ΞT. For a complete algorithm to solve with this preconditioner,see (Algorithm 2, [17]).

β

h Size 10−2 10−3 10−4

2−4 7112 69(4+68) 486(4+70) 3736(4+74)0.445 3.179 24.795

2−5 27528 74(4+63) 513(4+63) 4574(4+70)1.908 13.392 123.145

Table 6.13: Distributed optimal control problem constrained by the Stokes system, block diagonal precon-ditioner stk Pbd1 , Rees and Wathen.

6.3.2 Block diagonal preconditioner stk Pcomm, Pearson

We now solve the system (6.21) with the preconditioner proposed by Pearson [16] using thecommutator argument, i.e.,

stk Pcomm =

My 0 0 0

0 (K +1

βMy)M−1

y (K +1

βMy)

T 0 0

0 0 Kp 0

0 0 (M−1p Kp M−1

p +1β

K−1p )−1

. (6.23)

The results are presented in Table 6.14.

76

β

h Size 10−2 10−3 10−4 10−5 10−6 10−7 10−8 10−9 10−10

2−4 4934 101(3+29) 102(3+29) 93(3+29) 84(3+30) 73(3+31) 57(3+33) 42(3+32) 30(3+32) 24(3+31)0.31 0.308 0.282 0.257 0.232 0.187 0.14 0.105 0.08

2−5 19078 122(3+28) 119(3+28) 108(3+28) 96(3+28) 84(3+29) 63(3+30) 50(3+31) 39(3+31) 28(3+30)1.569 1.509 1.364 1.217 1.095 0.823 0.68 0.538 0.387

2−6 75014 143(3+27) 134(3+27) 122(3+26) 107(3+26) 94(3+27) 75(3+27) 60(3+28) 47(3+28) 36(3+28)8.184 7.612 6.8 5.958 5.239 4.205 3.406 2.681 2.123

2−7 297478 156(3+27) 145(3+26) 140(3+25) 119(3+25) 104(3+25) 91(3+26) 66(3+25) 54(3+27) 41(3+26)39.44 35.767 34.146 29.042 25.136 22.215 16.188 13.281 10.099

Table 6.14: Distributed optimal control problem constrained by the Stokes system, block diagonal precon-ditioner stk Pcomm , Pearson.

6.3.3 Block diagonal preconditioner pson Pnsn , W. Zulehner

Finally, we now solve the system (6.21) with the preconditioner proposed by Zulehner [35] usingthe non-standard norm argument, i.e.,

stk Pnsn =

My +

βKy 0 0 0

01β(My +

βKy) 0 0

0 0 (K−1p +

βM−1

p )−1 00 0 0 β(K−1

p +

βM−1p )−1

.

(6.24)

The results are presented in Table 6.15.

β

h Size 10−2 10−3 10−4 10−5 10−6 10−7 10−8 10−9 10−10

2−4 4934 74(4+23) 74(4+23) 71(4+24) 61(4+24) 53(4+23) 41(4+23) 36(4+22) 31(4+23) 25(4+22)0.16 0.16 0.156 0.134 0.116 0.09 0.079 0.068 0.056

2−5 19078 82(4+22) 82(4+22) 80(4+23) 71(4+23) 59(4+23) 45(4+22) 39(4+22) 33(4+21) 27(4+21)0.789 0.79 0.772 0.688 0.57 0.436 0.38 0.321 0.266

2−6 75014 90(4+21) 89(4+21) 87(4+22) 77(4+22) 65(4+22) 49(4+21) 41(4+21) 35(4+21) 29(4+19)3.885 3.848 3.764 3.338 2.822 2.131 1.788 1.526 1.276

2−7 297478 96(4+21) 97(4+21) 95(4+21) 85(4+21) 72(4+20) 53(4+19) 43(4+19) 37(4+20) 29(4+19)18.654 18.872 18.495 16.574 14.043 10.329 8.391 7.252 5.713

Table 6.15: Distributed optimal control problem constrained by the Stokes system, block diagonal precon-ditioner stk Pnsn, W.Zulehner.

6.3.4 Discussion

Clearly the preconditioner stk Pbd1 is independent with respect to the mesh size h, but not to theregularization parameter β. The other two preconditioners, i.e., stk Pcomm and stk Pnsn, are robustwith respect to the mesh size h as well as the regularization parameter β.

In Figure 6.7, we compare the efficiency of the three preconditioners in terms of the number ofiterations required to solve the relevant saddle point systems for various values of β when meshsize h = 2−5. We find that for β = 10−2, the performance for the preconditioner for full optimalitysystem and the preconditioner for the reduced optimality system based on non-standard normare almost matched. However, we cannot make any conclusive remark here on if reducing theoptimality system results in a better performance in terms of number of iterations since we do not

77

have a parameter robust preconditioner to solve for the full optimality system. We find that thepreconditioner based on the non-standard norm, i.e., stk Pnsn, is a winner here.

Rees 5x

5

Pearso

n 4x4

Zulehn

er4x

4

80

100

120

74

122

82

itera

tions

(a) β = 10−2

Rees 5x

5

Pearso

n 4x4

Zulehn

er4x

4

200

400

513

11982

itera

tions

(b) β = 10−3

Rees 5x

5

Pearso

n 4x4

Zulehn

er4x

4

0

2,000

4,000

4,574

108 80

itera

tions

(c) β = 10−4

Pearso

n 4x4

Zulehn

er4x

4

70

80

90

96

71

itera

tions

(d) β = 10−5

Figure 6.7: For mesh size h = 2−5, iteration count comparisons across different preconditionersfor various values of β.

In Figure 6.8, we compare the efficiency of the three preconditioners in terms of the numberof iterations required to solve the relevant saddle point systems for various values of β whenthe mesh size h = 2−5. Again we cannot draw any conclusion here since the preconditionerstk Pbd1 is not robust with respect to the regularization parameter β. Though, we observe that thepreconditioner based on the non-standard norm, i.e., stk Pnsn, performs the best.

78

Rees 5x

5

Pearso

n 4x4

Zulehn

er4x

4

1

1.5

2 1.91

1.57

0.79

CP

Utim

e(s

ec)

(a) β = 10−2

Rees 5x

5

Pearso

n 4x4

Zulehn

er4x

4

0

5

10

15 13.39

1.51 0.79

CP

Utim

e(s

ec)

(b) β = 10−3

Rees 5x

5

Pearso

n 4x4

Zulehn

er4x

4

0

50

100

123.15

1.36 0.77

CP

Utim

e(s

ec)

(c) β = 10−4

Pearso

n 4x4

Zulehn

er4x

4

0.8

1

1.21.22

0.69

CP

Utim

e(s

ec)

(d) β = 10−5

Figure 6.8: For mesh size h = 2−5, CPU time (sec) comparisons across different preconditionersfor various values of β.

We note that the algorithm (see Algorithm 2, [17]) for applying the preconditioner (6.22) to the fulloptimality system (6.20) is rather complicated compared to its reduced counterparts. So we areof the opinion that reducing the optimality system might also help in reducing the complexity ofsolving the system.

We now illustrate the behavior of the cost functional J for different values of β. As stated earlier,how close the state y approaches the desired state y is determined by the regularization param-eter β. We observe (across all tested preconditioners) that y − y continues to go down withdecreasing β while u stops increasing further around β ≤ 10−4. This implies that the optimalvalue of β for the problem is pretty small, i.e., around 10−10.

79

β iter u2 y − y2 y − y2/y2 J b −Ax2/b2 time1e-02 90 1.4e+02 4.08e-01 9.66e-01 9.70e+01 3.19e-09 3.8811e-03 89 1.1e+03 3.12e-01 7.39e-01 5.61e+02 5.17e-09 3.8351e-04 87 3.2e+03 9.84e-02 2.33e-01 5.00e+02 5.06e-09 3.7531e-05 77 4.1e+03 1.61e-02 3.80e-02 8.25e+01 5.07e-09 3.3651e-06 65 4.3e+03 2.90e-03 6.86e-03 9.10e+00 3.89e-09 2.8061e-07 49 4.3e+03 6.29e-04 1.49e-03 9.39e-01 4.73e-09 2.1251e-08 41 4.4e+03 1.46e-04 3.46e-04 9.53e-02 4.22e-09 1.7811e-09 35 4.4e+03 3.34e-05 7.92e-05 9.61e-03 2.92e-09 1.5241e-10 29 4.4e+03 7.29e-06 1.72e-05 9.64e-04 4.05e-09 1.261

Table 6.16: Study of the cost functional J of the Distributed optimal control problem con-

strained by the Stokes system The table is produced using the preconditioner stk Pnsn, For mesh size2−6, we show the number of outer MINRES iterations represented as "iter". u represents the L2(Ω) normof the control u. y − y measures how closely the state y matches the desired state y. y − y/y mea-sures the relative error. J is the calculated cost functional5. b −Ax/b represents the residual normof the KKT system of equations to show the system converged in the L2(Ω) norm. Finally, the last columntells us the time (sec) it took to solve the system.

Finally, we reproduce the plots from [35] for state y, control u and pressure p for β = 10−6 andmesh size h = 2−6. We use the preconditioner stk Pnsn, developed by Zulehner to generate theseplots.

(a) State u (b) Control u

Clearly, the region with the highest magnitude in Figure 6.9(a) contain the vectors of the highestmagnitude of one for the state y. The region of the highest magnitude in Figure 6.9(b) corre-sponds to the value 57.7 for the control u.

5The differences between the cost functionals using all other preconditioners are insignificant

80

(c) Pressure p

Figure 6.9: State y, control u and pressure p distribution for h = 2−6 and β = 10−6.

Finally, we can see that the pressure p in Figure 6.9(c) lies in the range of -5.7 to 5.7.

81

“Have you guessed the riddle yet?”the Hatter said, turning to Aliceagain. “No, I give it up,” Alice replied:“What’s the answer?” “I haven’t theslightest idea,” said the Hatter”


7Summary and conclusion

We discussed the general framework of PDE-constrained optimization problems. Three bench-mark problems were considered: the distributed optimal control of the Poisson equation, thedistributed optimal control of the convection-diffusion equation and the distributed optimal controlof the Stokes system. We discussed how numerically tackling these problems lead to a largediscrete optimality system with a saddle point structure. Since large saddle point systems areindefinite with bad spectral properties, both direct methods as well as Krylov subspace iterativemethods are ill suited to deal with them. However, Krylov subspace iterative methods can becombined with an efficient preconditioner to speedup their convergence.

We discussed various preconditioners for saddle point systems constructed through various tech-niques in the context of our benchmark problems. An important contribution was to apply thepreconditioner proposed by Axelsson and Neytcheva in [40] to solve for the first two benchmarkproblems. We implemented and tested all these different preconditioners to demonstrate theirrelative efficiency. The convection-diffusion control problem posed certain challenges which wereeffectively dealt with using the local projection stabilization (LPS) technique.

We found the preconditioner [40] to be parameter robust. Moreover, this preconditioner naturallycomes in a reduced form, and its efficiency both in terms of computational time as well as iterationcount was found to be the best compared to all the other preconditioners that were tested. Wenoted that preconditioners for the reduced optimality system performed well compared to their fullcounterparts. Hence we can conclude that reducing the optimality system for certain problems,when such a possibility is available, leads to a better performance. It was observed that in thecase of full optimality system arising in the distributed optimal control of the Stokes system, thepreconditioner and its related algorithm were rather complex. Thus we can say that reducing theoptimality system may also help in designing efficient and easy-to-apply preconditioners.

We have only considered distributed optimal control problems without constraints on the controland the state. Parameter identification and time dependent problems were also not considered.Preconditioning techniques currently available in literature for such problems were highlighted inthe introduction (Chapter 1) of this thesis. As a final remark, we point out that the preconditioner[40] can also be utilized to effectively solve these problems and hence can be a topic for extendingthe current work.

82

Acknowledgments

I would like to thank Owe Axelsson for reading my thesis and providing valuable suggestions andcorrections. I am also thankful to my reviewer, Sverker Holmgren, for his comments on my thesis.I must also thank the opponent of my thesis, Martin Hagelin, for his interesting and thoughtfulcomments. Also thanks to Justin Pearson for arranging my thesis presentation. Special thanks toMurtazo Nazarov for his suggestions on the local projection stabilization (LPS) scheme. Finally, Iwould like to thank my thesis supervisor, Maya Neytcheva, for all her technical and inspirationalsupport throughout the thesis.

83

Bibliography

[1] Owe Axelsson and Maya Neytcheva, Eigenvalue estimates for preconditioned saddle pointmatrices, Linear Algebra Appl. 2006; 13:339-360.

[2] I. Babuska and A. K. Aziz, Survey lectures on the mathematical foundations of the finiteelement method, in The Mathematical Foundations of the Finite Element Method withApplications to Partial Differential Equations, A. K. Aziz, ed., Academic Press, New York,1972, pp. 1-359.

[3] F. Brezzi, On the existence, uniqueness and approximation of saddle-point problems aris-ing from Lagrangian multipliers, R.A.I.R.O. Anal.Numer., 2 (1974), pp. 129-151.

[4] F. Brezzi and M. Fortin, Mixed and Hybrid Finite Element Methods, Springer-Verlag, 1991.

[5] M. Benzi, G. H. Golub, AND J. Liesen, Numerical solution of saddle point problems, ActaNumer., 14 (2005), pp. 1-137.

[6] R. T. Rockafellar, Convex Analysis, Princeton Landmarks Math., Princeton UniversityPress, Princeton, NJ, 1997.

[7] Youcef Saad and Martin H. Schultz. GMRES: A generalized minimal residual algorithm forsolving nonsymmetric linear systems. SIAM J. Sci. Stat. Comput., 7(3):856-869, 1986.

[8] O. Axelsson, A generalized conjugate direction method and its application to a singularperturbation problem, in Proc. 8th biennial Numerical Analysis Conference (Dundee, Scot-land, June 26-29, 1979),(G. A. Watson, ed.). Lecture Notes in Math., vol. 773, Springer-Verlag, Berlin, 1980, pp. 1-11.

[9] O. Axelsson, "Conjugate gradient type methods for unsymmetric and inconsistent systemsof linear equations," Linear Algebra Appl., v. 29, 1980, pp. 1-16.

[10] J.H. Bramble and J. E. Pasciak. A preconditioning technique for indefinite systems result-ing from mixed approximations of elliptic problems. Math. Comp., 50:1-17, 1988.

[11] O. Axelsson, Iterative Solution Methods, Cambridge University Press, New York, NY, USA,1994.

[12] Cornelius Lanczos. Iteration method for the solution of the eigenvalue problem of lineardifferential and integal operators. J. Res. Nat. Bur. Stand., 45:255-282, 1950.

[13] M. R. Hestenes and E. Stiefel. Methods of conjugate gradients for solving linear systems.J. Res. Nat. Bur. Stand., 49:409-436, 1952.

[14] C. C. Paige and M. A. Saunders. Solution of sparse indefinite systems of linear equations.SIAM J. Numer. Anal., 12(4):617-629, 1975.

[15] Yousef Saad, Iterative methods for sparse linear systems, PWS Publishing Company,1996.

[16] John W. Pearson, On the role of commutator arguments in the development of parameter-robust preconditioners for Stokes control problems, to appear in Electronic Transactionson Numerical Analysis, 2014.

84

[17] Rees, Tyrone; Wathen, Andrew J. Preconditioning iterative methods for the optimal controlof the Stokes equations. SIAM J. Sci. Comput. 33 (2011), no. 5, 2903-2926.

[18] H. C. Elman, D. J. Silvester, and A. J. Wathen. Finite elements and fast iterative solvers:

with applications in incompressible fluid dynamics. Numerical Mathematics and ScientificComputation. Oxford University Press, New York, 2005.

[19] Alfio Borzì, Volker Schulz, Computational Optimization of Systems Governed by PartialDifferential Equations, volume 8 of Computational Science and Engineering. Society forIndustrial and Applied Mathematics (SIAM), Philadelphia, PA, 2012.

[20] L.T. Biegler, O. Ghattas, M. Heinkenschloss, D. Keyes, B. van Bloemen Waanders. Real-Time PDE-Constrained Optimization, Computational Science and Engineering. Society forIndustrial and Applied Mathematics (SIAM), 2007.

[21] M. Hinze, R. Pinnau, M. Ulbrich, S. Ulbrich, Optimization with PDE Constrains, Springer2009.

[22] J. Cahouet and J.-P. Chabard. Some fast 3D finite element solvers for the generalizedStokes problem. Internat. J. Numer. Methods Fluids, 8(8):869-895, 1988.

[23] S. S. Collis, and M. Heinkenschloss, Analysis of the Streamline Upwind/Petrov GalerkinMethod Applied to the Solution of Optimal Control Problems, Technical Report CAAMTR02-01, Rice University, 2002.

[24] T. Rees, Preconditioning Iterative Methods for PDE Constrained Optimization, Doctor ofPhilosophy Thesis, University of Oxford, 2010.

[25] Simultaneous Analysis and Design in PDE-constrained Optimization. Doctor of PhilosophyThesis, Stanford University, December 2012.

[26] Markus Kollmann, Efficient Iterative Solvers for Saddle Point Systems arising in PDE-constrained Optimization Problems with Inequality Constraints, Doctor of Philosophy The-sis, Johannes Kepler University Linz, 2013.

[27] John W. Pearson, Andrew J. Wathen, A new approximation of the Schur complement inpreconditioners for PDE-constrained optimization, Numerical Linear Algebra with Applica-tions, 2011.

[28] John W. Pearson, Andrew J. Wathen, Fast Iterative Solvers for Convection-Diffusion Con-trol Problems, Electronic Transactions on Numerical Analysis, Volume 40, pp. 294-310,2013.

[29] R. Becker, and B. Vexler, Optimal Control of the Convection-Diffusion Equation UsingStabilized Finite Element Methods, Numerische Mathematik, 106(3), pp.349-367, 2007.

[30] J.-L. Guermond, Stabilization of Galerkin approximations of transport equations by subgridmodeling, M2AN Math. Model. Numer. Anal., 33 (1999), pp. 1293-1316.

[31] J.-L. Guermond, A. Marra, L. Quartapelle, Subgrid stabilized projection method for 2Dunsteady flows at high Reynolds numbers, Computer Methods in Applied Mechanics andEngineering Volume 195, Issues 44-47, 15 September 2006, Pages 5857-5876.

[32] A. J. Wathen and T. Rees, Chebyshev semi-iteration in preconditioning for problems in-cluding the mass matrix, Electron. Trans. Numer. Anal., 34 (2008/09), pp. 125-135.

[33] R. Hiptmair. Operator preconditioning. Comput. Math. Appl., 52(5):699-706, 2006.

[34] Walter Zulehner, Efficient Solvers for Saddle Point Problems with Applications to PDE-Constrained Optimization, Advanced Finite Element Methods and Applications, LectureNotes in Applied and Computational Mechanics Volume 66, 2013, pp 197-216.

85

[35] Walter Zulehner, Nonstandard norms and robust estimates for saddle-point problems,SIAM J. Matrix Anal. Appl. 32(2011), 536-560.

[36] M. A. Olshanskii and A. Reusken. On the convergence of a multigrid method for linearreaction diffusion problems, Computing, 65(3):193-202, 2000.

[37] M. A. Olshanskii, J. Peters, and A. Reusken. Uniform preconditioners for a parameter de-pendent saddle point problem with application to generalized Stokes interface equations.Numer. Math., 105(1):159-191, 2006.

[38] A. Ramage, A Multigrid Preconditioner for Stabilised Discretisations of Advection-DiffusionProblems, Journal of Computational and Applied Mathematics, 110 (1999), pp.187-203.

[39] J. Boyle, M. D. Mihajlovic, and J. A. Scott. HSL MI20: an efficient amg preconditioner.Technical Report RAL-TR-2007-021, Department of Computational and Applied Mathe-matics, Rutherford Appleton Laboratory, 2007.

[40] O. Axelsson, M. Neytcheva, Operator splittings for solving nonlinear, coupled multiphysicsproblems with an application to the numerical solution of an interface problem, Institute forInformation Technology, Uppsala University TR 2011-009, 2011.

[41] P. Boyanova, M. Do-Quang, M. Neytcheva Efficient preconditioners for large scale binaryCahn-Hilliard models Comput. Methods Appl. Math., 12 (2012), pp. 1-22.

[42] O. Axelsson, P. Boyanova, M. Kronbichler, M. Neytcheva, X. Wu, Numerical and compu-tational efficiency of solvers for two-phase problems, Computers and Mathematics withApplications Volume 65, Issue 3, February 2013, Pages 301-314, Efficient NumericalMethods for Scientific Applications.

[43] O. Axelsson, M. Neytcheva, Bashir Ahmad, A comparison of iterative methods to solvecomplex valued linear algebraic systems, Numerical Algorithms, 2014, Volume 66, Issue4, pp 811-841.

[44] W. Bangerth. Adaptive Finite Element Methods for the Identification of Distributed Param-eters in Partial Differential Equations. PhD thesis, University of Heidelberg, 2002.

[45] Bjorn Fredrik Nielsen and Kent-Andre Mardal, Efficient Preconditioners for Optimality Sys-tems Arising in Connection with Inverse Problems, SIAM J. Control Optim., 48(8), 5143-5177, 2010.

[46] G. Rozza, A. Manzoni and F. Negri, Reduction strategies for PDE-constrained optimizationproblems in haemodynamics, MATHICSE Technical Report Nr. 26.2012, Ecole polytech-nique fédérale de Lausanne, July 2012.

[47] Axelsson, O.; Vassilevski, P. S. Algebraic multilevel preconditioning methods. I. Numer.Math. 56 (1989), no. 2-3, 157-177.

[48] Axelsson, O.; Vassilevski, P. S. Algebraic multilevel preconditioning methods. II. SIAM J.Numer. Anal. 27 (1990), no. 6, 1569-1590.

[49] Axelsson, O.; Vassilevski, P. S. A black box generalized conjugate gradient solver withinner iterations and variable-step preconditioning. SIAM J. Matrix Anal. Appl. 12 (1991),no. 4, 625-644.

[50] Johannes Kraus, Svetozar Margenov, Robust Algebraic Multilevel Methods and Algo-rithms, Radon Series Comp. Appl. Math., vol. 5, Walter de Gruyter, Berlin-NewYork, 2009.

[51] Andrea Toselli, Olof Widlund, Domain Decomposition Methods-Algorithms and TheorySpringer Series in Computational Mathematics, Vol. 34, 2005.

[52] A. Schiela and S. Ulbrich, Operator preconditioning for a class of inequality con-strained optimal control problems, Technical report, Department of Mathematics, TU

86

Darmstadt, Darmstadt, 2014. http://wwwopt.mathematik.tu-darmstadt.de/~ulbrich/papers/schielaulbrichprecfinal.pdf

[53] M. Stoll and A. Wathen, Preconditioning for partial differential equation constrained opti-mization with control constraints. Numer. Linear Algebra Appl. 19 (2012), no. 1, 53-71.

[54] J. W. Pearson, M. Stoll, and A. J. Wathen, Preconditioners for state constrained opti-mal control problems with Moreau–Yosida penalty function, Numer. Linear Algebra Appl.(2011), 00:1-17.

[55] T. Rees, M. Stoll, and A. WATHEN, All-at-once preconditioners for PDE-constrained opti-mization, Kybernetika, 46 (2010), pp. 341–360.

[56] J.W. Pearson, M. Stoll, A. Wathen, Regularization-robust preconditioners for time-dependent PDE constrained optimization problems, SIAM Journal on Matrix Analysis andApplications, 2012.

[57] Stoll, Martin; Wathen, Andy, All-at-once solution of time-dependent Stokes control. J.Comput. Phys. 232 (2013), 498-515.

[58] W. Bangerth, R. Hartmann, and G. Kanschat, deal.II-a general-purpose object-orientedfinite element library, ACM Transactions on Mathematical Software, 33 (2007), pp. Art. 24,27. http://www.dealii.org/

[59] The Trilinos Project http://trilinos.sandia.gov/

[60] The Scalable Linear Solvers (HYPRE) project http://computation.llnl.gov/casc/linear_solvers/sls_hypre.html

[61] The Portable, Extensible Toolkit for Scientific Computation (PETSc) http://www.mcs.anl.gov/petsc/

87

http://wwwopt.mathematik.tu-darmstadt.de/~ulbrich/papers/schielaulbrichprecfinal.pdf

http://wwwopt.mathematik.tu-darmstadt.de/~ulbrich/papers/schielaulbrichprecfinal.pdf

http://www.dealii.org/

http://trilinos.sandia.gov/

http://computation.llnl.gov/casc/linear_solvers/sls_hypre.html

http://computation.llnl.gov/casc/linear_solvers/sls_hypre.html

http://www.mcs.anl.gov/petsc/

http://www.mcs.anl.gov/petsc/

AResults for distributed convection-diffusion

control problem for = 1/1500

In Table A.1, we present the results of preconditioning the saddle point system (6.11), using thepreconditioner cd Pbd1 .

β

h Size 10−2 10−3 10−4 10−5 10−6 10−7 10−8 10−9 10−10

2−4 867 22(2+19) 24(2+19) 24(2+23) 18(2+23) 12(2+24) 10(2+24) 8(2+25) 8(2+25) 6(2+25)0.014 0.014 0.015 0.01 0.007 0.006 0.004 0.005 0.003

2−5 3267 29(2+17) 28(2+17) 28(2+20) 22(2+22) 16(2+23) 12(2+23) 10(2+24) 8(2+24) 7(2+24)0.04 0.038 0.04 0.03 0.022 0.016 0.014 0.011 0.01

2−6 12675 46(2+16) 42(2+16) 36(2+19) 24(2+22) 18(2+22) 12(2+22) 10(2+23) 8(2+23) 8(2+24)0.213 0.19 0.18 0.116 0.086 0.057 0.048 0.039 0.04

2−7 49923 54(2+14) 52(2+14) 46(2+17) 30(2+21) 19(2+21) 16(2+22) 12(2+22) 10(2+22) 8(2+23)0.993 0.966 0.889 0.661 0.398 0.335 0.253 0.214 0.174

2−8 198147 56(2+12) 80(2+12) 49(2+14) 25(2+16) 21(2+19) 17(2+21) 14(2+21) 11(2+22) 9(2+22)4.544 6.494 4.145 2.187 1.991 1.648 1.365 1.105 0.918

Table A.1: Distributed optimal control problem constrained by the convection-diffusion

equation-block diagonal preconditioner cd Pbd1 , Pearson and Wathen: For each value of β andh, we show the number of outer FGMRES iterations in the first row. The adjacent brackets represent theaverage inner iterations for each outer iteration. The first number to the left shows the average number ofiterations for 2 solves with Chebyshev semi-iteration and the number to the right shows the average numberof iterations for two solves with AMG. The number just below the first row represents the CPU times (inseconds) to solve the problem.

In Table A.2, we present the results of preconditioning the saddle point system (6.11), using thepreconditioner cd Pblt .

88

β

h Size 10−2 10−3 10−4 10−5 10−6 10−7 10−8 10−9 10−10

2−4 867 22(2+19) 22(2+18) 21(2+21) 18(2+24) 12(2+24) 10(2+24) 8(2+24) 8(2+25) 6(2+25)0.013 0.012 0.012 0.011 0.007 0.006 0.005 0.005 0.004

2−5 3267 29(2+17) 28(2+16) 28(2+21) 20(2+22) 14(2+23) 12(2+23) 10(2+24) 8(2+24) 6(2+24)0.05 0.038 0.041 0.028 0.019 0.017 0.014 0.011 0.009

2−6 12675 46(2+17) 37(2+15) 34(2+19) 24(2+22) 18(2+21) 12(2+22) 10(2+23) 8(2+23) 8(2+24)0.207 0.172 0.175 0.12 0.088 0.059 0.05 0.041 0.041

2−7 49923 52(2+14) 52(2+12) 37(2+15) 23(2+18) 20(2+20) 16(2+21) 12(2+22) 10(2+22) 8(2+22)0.988 0.957 0.719 0.472 0.422 0.341 0.258 0.219 0.178

2−8 198147 56(2+12) 80(2+10) 49(2+11) 27(2+15) 21(2+20) 17(2+20) 14(2+21) 11(2+22) 9(2+22)4.667 6.491 4.057 2.421 2.02 1.651 1.382 1.091 0.908


equation-block lower triangular preconditioner cd Pblt , Pearson and Wathen: For each valueof β and h, we show the number of outer FGMRES iterations in the first row. The adjacent brackets representthe average inner iterations for each outer iteration. The first number to the left shows the average numberof iterations for 2 solves with Chebyshev semi-iteration and the number to the right shows the averagenumber of iterations for two solves with AMG. The number just below the first row represents the CPU times(in seconds) to solve the problem.

In Table A.3, we present the results of preconditioning the reduced saddle point system (6.12),using the preconditioner cd Pbd2 .

β

h Size 10−2 10−3 10−4 10−5 10−6 10−7 10−8 10−9 10−10

2−4 578 25(2+9) 29(2+9) 25(2+10) 15(2+11) 9(2+11) 7(2+11) 5(2+11) 5(2+12) 4(2+13)0.013 0.017 0.012 0.006 0.003 0.003 0.002 0.002 0.002

2−5 2178 34(2+8) 51(2+7) 40(2+9) 21(2+10) 12(2+10) 8(2+10) 6(2+11) 5(2+12) 5(2+12)0.034 0.048 0.041 0.02 0.011 0.007 0.006 0.005 0.005

2−6 8450 46(2+7) 78(2+6) 54(2+8) 28(2+8) 16(2+9) 9(2+10) 8(2+10) 5(2+10) 5(2+11)0.136 0.228 0.165 0.089 0.049 0.028 0.025 0.017 0.017

2−7 33282 53(2+6) 142(2+5) 105(2+6) 43(2+7) 22(2+8) 13(2+9) 8(2+9) 7(2+9) 6(2+10)0.719 1.913 1.427 0.607 0.313 0.19 0.119 0.106 0.094

2−8 132098 57(2+5) 200(2+5) 168(2+5) 58(2+6) 29(2+7) 18(2+8) 11(2+9) 8(2+9) 7(2+9)3.396 11.876 9.923 3.538 1.829 1.177 0.735 0.549 0.495


equation-block diagonal preconditioner cd Pbd2 , Pearson and Wathen: For each value of β andh, we show the number of outer FGMRES iterations in the first row. The adjacent brackets represent theaverage inner iterations for each outer iteration. The first number to the left shows the average number ofiterations for 1 solve with Chebyshev semi-iteration and the number to the right shows the average numberof iterations for two solves with AMG. The number just below the first row represents the CPU times (inseconds) to solve the problem.

In Table A.4, we present the results of preconditioning the reduced saddle point system (6.12),using the preconditioner cd Pnsn .

89

β

h Size 10−2 10−3 10−4 10−5 10−6 10−7 10−8 10−9 10−10

2−4 578 22(2+0) 21(2+0) 22(2+0) 16(2+0) 10(2+0) 8(2+0) 6(2+0) 6(2+0) 4(2+0)0.008 0.006 0.007 0.004 0.002 0.002 0.001 0.001 0.001

2−5 2178 31(2+0) 26(2+0) 26(2+0) 20(2+0) 14(2+0) 10(2+0) 7(2+0) 6(2+0) 5(2+0)0.023 0.018 0.018 0.012 0.008 0.006 0.004 0.004 0.003

2−6 8450 44(2+0) 39(2+0) 33(2+0) 24(2+0) 18(2+0) 12(2+0) 8(2+0) 6(2+0) 6(2+0)0.103 0.095 0.083 0.057 0.041 0.027 0.019 0.015 0.015

2−7 33282 51(2+0) 53(2+0) 45(2+0) 28(2+0) 22(2+0) 14(2+0) 10(2+0) 8(2+0) 6(2+0)0.55 0.573 0.486 0.306 0.236 0.149 0.108 0.088 0.068

2−8 132098 55(2+0) 80(2+0) 62(2+0) 35(2+0) 24(2+0) 18(2+0) 12(2+0) 8(2+0) 6(2+0)2.641 3.843 3.016 1.705 1.151 0.857 0.575 0.395 0.308


equation-block diagonal preconditioner cd Pnsn , W. Zulehner: For each value of β and h, we showthe number of outer FGMRES iterations in the first row. The adjacent brackets represent the average inner

iterations needed to approximate two diagonal blocks with two solves of AMG for each outer iteration. Thenumber just below the first row represents the CPU times (in seconds) to solve the problem.

In Table A.5, we present the results of preconditioning the reduced saddle point system (6.12),using the preconditioner cd PF.

β

h Size 10−2 10−3 10−4 10−5 10−6 10−7 10−8 10−9 10−10

2−4 578 11(2+0) 11(2+0) 11(2+0) 8(2+0) 5(2+0) 4(2+0) 3(2+0) 2(3+0) 2(3+0)0.003 0.003 0.003 0.002 0.001 0.001 0.001 0.001 0.001

2−5 2178 15(2+0) 14(2+0) 13(2+0) 9(2+0) 6(2+0) 4(2+0) 3(2+0) 3(2+0) 2(3+0)0.01 0.009 0.008 0.005 0.004 0.003 0.002 0.002 0.002

2−6 8450 19(2+0) 17(2+0) 15(2+0) 10(2+0) 7(2+0) 5(2+0) 3(2+0) 3(2+0) 2(3+0)0.047 0.036 0.032 0.022 0.016 0.012 0.008 0.008 0.006

2−7 33282 21(2+0) 23(2+0) 20(2+0) 12(2+0) 8(2+0) 6(2+0) 4(2+0) 3(2+0) 2(3+0)0.235 0.26 0.219 0.132 0.09 0.07 0.05 0.04 0.031

2−8 132098 22(2+0) 32(2+0) 27(2+0) 14(2+0) 9(2+0) 7(2+0) 5(2+0) 3(2+0) 3(2+0)1.097 1.645 1.343 0.693 0.455 0.363 0.274 0.182 0.182


equation-block preconditioner cd PF, Axelsson and Neytcheva: For each value of β and h, weshow the number of outer FGMRES iterations in the first row. The adjacent brackets represent the averageinner iterations needed for two solves with AMG using Algorithm (4) described earlier. The number justbelow the first row represents the CPU times (in seconds) to solve the problem.

90

BLocal projection stabilization (LPS)

Consider the convection-diffusion equation

−ε∆u + (w ·∇)u = f in Ω

uD = g0 on ∂Ω,(B.1)

where Ω is the domain with boundary ∂Ω, is the diffusion coefficient, and the vector w repre-sents the advection field.

The discretization of the convection-diffusion equation with the LPS scheme leads to:

(∇uh,∇vh) + (w.∇uh, vh) + δk((I − Ph)w.∇uh, w.∇vh) = f . (B.2)

or(∇uh,∇vh) + (w.∇uh, vh) + δk(w.∇uh, w.∇vh) = f + δkPh(w.∇uh, vh). (B.3)

The stabilization parameter δk is defined locally on individual elements and depends on the Pecletnumber

Pkh =

hkw

.

Following from Becker and Vexler in [29], we define

δk =

hkw

, if Pkh ≥ 1,

0, otherwise.(B.4)

We now split the equation (B.3) as follows:

(∇uh,∇vh) + δk(w.∇uh, w.∇vh) + (w.∇uh, vh) = f + δkξh. (B.5)

Ph(w.∇uh, vh)ξH = w.(∇uh, vh). (B.6)

where ξH is a cell-wise constant function on the patches and ξh is the interpolation of ξH to Th.

The following algorithm should solve the convection-diffusion equation using the LPS scheme.

91

1. Set ξh = 0.

2. Assemble (B.5) and compute uh using one iteration of FGMRES.

3. Interpolate uh to uH. This takes the nodal values on (see Figure 2.2) from the fine meshTh to the coarse mesh TH.

4. Compute gradient of uH, i.e., ∇uH.

5. Assemble (B.6), i.e., MPh ξH = (w.∇uH , vH), and solve for ξH.

6. Interpolate ξH back to ξh.

7. Assemble (B.5) and solve with FGMRES till convergence.

B.1 The double-glazing problem

We consider the double-glazing problem from (Chapter 3, [18]). Let Ω = [−1, 1]2 represent thedomain. Dirichlet boundary conditions are imposed everywhere, and are given given by

uD =

1 if x2 = 1,0 otherwise. (B.7)

The circulating flow is determined by the advection field represented by the wind

w =12

x2(1 − x21),−

12

x1(1 − x22). (B.8)

The diffusion coefficient is represented by the viscosity =1

500.

With this data, we solve the convection diffusion problem (B.1) using the LPS scheme. For meshside h = 2−6, the solution can be viewed in Figure B.1

Figure B.1: Solution to the double-glazing problem using the LPS scheme.

92

performance comparisons of preconditioned iterative...

Documents