Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Adaptive discretization strategies for theidentification of distributed parameters in partial

differential equations

Barbara Kaltenbacher, University of Klagenfurt

joint work with

Anke Griesbaum, University of HeidelbergAlana Kirchner, Technical University of Munich

Slobodan Veljovic, University of GrazBoris Vexler, Technical University of Munich

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Overview

I motivation: parameter identification in PDEs

I ideas on adaptivity for inverse problems

I principles of goal oriented error estimators

I Tikhonov & discrepancy principle

I regularized Newton iterations & discrepancy principle

I outlook

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Motivation: Parameter Identification in PDEs

I instability: amplification of numerical errorsI computational effort:

I large scale: several PDE solves per regularized inversionI several reg. inversions to determine regularization parameter

Example −∆u = q:refine grid for u and q: • at jumps or large gradients or

• at locations with large error contribution

→ location of large gradients / large errors a priori unknown

→ general strategy for mesh generation separately for q and u(example −∇q(u)∇u) = f )

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Motivation: Parameter Identification in PDEs

I instability: amplification of numerical errorsI computational effort:

I large scale: several PDE solves per regularized inversionI several reg. inversions to determine regularization parameter

Example −∆u = q:refine grid for u and q: • at jumps or large gradients or

• at locations with large error contribution

→ location of large gradients / large errors a priori unknown

→ general strategy for mesh generation separately for q and u(example −∇q(u)∇u) = f )

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Motivation: Parameter Identification in PDEs

I instability: amplification of numerical errorsI computational effort:

I large scale: several PDE solves per regularized inversionI several reg. inversions to determine regularization parameter

Example −∆u = q:refine grid for u and q: • at jumps or large gradients or

• at locations with large error contribution

→ location of large gradients / large errors a priori unknown

→ general strategy for mesh generation separately for q and u(example −∇q(u)∇u) = f )

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Motivation: Parameter Identification in PDEs

I instability: amplification of numerical errorsI computational effort:

I large scale: several PDE solves per regularized inversionI several reg. inversions to determine regularization parameter

Example −∆u = q:refine grid for u and q: • at jumps or large gradients or

• at locations with large error contribution

→ location of large gradients / large errors a priori unknown

→ general strategy for mesh generation separately for q and u(example −∇q(u)∇u) = f )

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Motivation: Parameter Identification in PDEs

I instability: amplification of numerical errorsI computational effort:

I large scale: several PDE solves per regularized inversionI several reg. inversions to determine regularization parameter

Example −∆u = q:refine grid for u and q: • at jumps or large gradients or

• at locations with large error contribution

→ location of large gradients / large errors a priori unknown

→ general strategy for mesh generation separately for q and u(example −∇q(u)∇u) = f )

instability ⇒ regularization necessary !

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Motivation: Parameter Identification in PDEs

I instability: amplification of numerical errorsI computational effort:

I large scale: several PDE solves per regularized inversionI several reg. inversions to determine regularization parameter

Example −∆u = q:refine grid for u and q: • at jumps or large gradients or

• at locations with large error contribution

→ location of large gradients / large errors a priori unknown

→ general strategy for mesh generation separately for q and u(example −∇q(u)∇u) = f )

instability ⇒ regularization necessary !

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Regularization of Inverse Problems

• unstable operator equation: F (q) = g with F : q 7→ u or C (u)

• solution q = F−1(g) does not depend continuously on gi.e.,

(∀(gn), gn → g 6⇒ qn := F−1(gn)→ F−1(g)

)

• only noisy data g δ ≈ g available (measurements): ‖g δ − g‖ ≤ δ

• making ‖F (q)− g δ‖ small 6⇒ good result for q!

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Regularization of Inverse Problems

• unstable operator equation: F (q) = g with F : q 7→ u or C (u)

• solution q = F−1(g) does not depend continuously on gi.e.,

(∀(gn), gn → g 6⇒ qn := F−1(gn)→ F−1(g)

)• only noisy data g δ ≈ g available (measurements): ‖g δ − g‖ ≤ δ

• making ‖F (q)− g δ‖ small 6⇒ good result for q!

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Regularization of Inverse Problems

• unstable operator equation: F (q) = g with F : q 7→ u or C (u)

• solution q = F−1(g) does not depend continuously on gi.e.,

(∀(gn), gn → g 6⇒ qn := F−1(gn)→ F−1(g)

)• only noisy data g δ ≈ g available (measurements): ‖g δ − g‖ ≤ δ

• making ‖F (q)− g δ‖ small 6⇒ good result for q!

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Regularization of Inverse Problems

• unstable operator equation: F (q) = g with F : q 7→ u or C (u)

• solution q = F−1(g) does not depend continuously on gi.e.,

(∀(gn), gn → g 6⇒ qn := F−1(gn)→ F−1(g)

)• only noisy data g δ ≈ g available (measurements): ‖g δ − g‖ ≤ δ

• making ‖F (q)− g δ‖ small 6⇒ good result for q!

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

An Example(Identification of a source term in a 1-d differential equation, δ = 1%)

exact and noisy data g , gδ exact q vs q with ‖F (q)− gδ‖ = 1.e − 14

exact and noisy data g , gδ exact q vs q with ‖F (q)− gδ‖ = 2δ

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Regularization of Inverse Problems• unstable operator equation: F (q) = g with F : q 7→ u or C (u)

• solution q = F−1(g) does not depend continuously on gi.e.,

(∀(gn), gn → g 6⇒ qn := F−1(gn)→ F−1(g)

)• only noisy data g δ ≈ g available (measurements): ‖g δ − g‖ ≤ δ

• making ‖F (q)− g δ‖ small 6⇒ good result for q!

• regularization means approaching solution along stable path

• regularization method:family (Rα)α>0 with parameter choice α = α(g δ, δ)such that worst case convergence as δ → 0:

sup‖gδ−g‖≤δ

‖Rα(gδ,δ)(g δ)− F−1(g)‖ → 0 as δ → 0

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Regularization of Inverse Problems• unstable operator equation: F (q) = g with F : q 7→ u or C (u)

• solution q = F−1(g) does not depend continuously on gi.e.,

(∀(gn), gn → g 6⇒ qn := F−1(gn)→ F−1(g)

)• only noisy data g δ ≈ g available (measurements): ‖g δ − g‖ ≤ δ

• making ‖F (q)− g δ‖ small 6⇒ good result for q!

• regularization means approaching solution along stable path

• regularization method:family (Rα)α>0 with parameter choice α = α(g δ, δ)such that worst case convergence as δ → 0:

sup‖gδ−g‖≤δ

‖Rα(gδ,δ)(g δ)− F−1(g)‖ → 0 as δ → 0

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Regularization of Inverse Problems• unstable operator equation: F (q) = g with F : q 7→ u or C (u)

• solution q = F−1(g) does not depend continuously on gi.e.,

(∀(gn), gn → g 6⇒ qn := F−1(gn)→ F−1(g)

)• only noisy data g δ ≈ g available (measurements): ‖g δ − g‖ ≤ δ

• making ‖F (q)− g δ‖ small 6⇒ good result for q!

• regularization means approaching solution along stable path

• regularization method:family (Rα)α>0 with parameter choice α = α(g δ, δ)such that worst case convergence as δ → 0:

sup‖gδ−g‖≤δ

‖Rα(gδ,δ)(g δ)− F−1(g)‖ → 0 as δ → 0

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Motivation: Parameter Identification in PDEs

I instability: amplification of numerical errorsI computational effort:

I large scale: several PDE solves per regularized inversionI several reg. inversions to determine regularization parameter

Example −∆u = q:refine grid for u and q: • at jumps or large gradients or

• at locations with large error contribution

→ location of large gradients / large errors a priori unknown

→ general strategy for mesh generation separately for q and u(example −∇q(u)∇u) = f )

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Motivation: Parameter Identification in PDEs

I instability: amplification of numerical errorsI computational effort:

I large scale: several PDE solves per regularized inversionI several reg. inversions to determine regularization parameter

Example −∆u = q:refine grid for u and q: • at jumps or large gradients or

• at locations with large error contribution

→ location of large gradients / large errors a priori unknown

→ general strategy for mesh generation separately for q and u(example −∇q(u)∇u) = f )

computational effort ⇒ efficient numerical strategies necessary !

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Motivation: Parameter Identification in PDEs

I instability: amplification of numerical errorsI computational effort:

I large scale: several PDE solves per regularized inversionI several reg. inversions to determine regularization parameter

Example −∆u = q:refine grid for u and q: • at jumps or large gradients or

• at locations with large error contribution

→ location of large gradients / large errors a priori unknown

→ general strategy for mesh generation separately for q and u(example −∇q(u)∇u) = f )

computational effort ⇒ efficient numerical strategies necessary !

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Efficient Methods for PDEs

multilevel iteration:

l = 4

l = 3

l = 2

l = 1 •• • •

• • • •• •

•

start with coarse discretizationrefine successively

adaptive discretization:

coarse discretization where possiblefine grid only where necessary

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Efficient Methods for PDEs

combined multilevel adaptive strategy:

courtesy to [R.Becker&M.Braack&B.Vexler, App.Num.Math., 2005]

start on coarse gridsucessive adaptive refinement

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

An Example: Crack Identification in Piezoelectric Ceramics

−DIV~σ = 0 in Ω−∇D = 0 in Ω

NT~σ = 0 on ∂Ω∪Σφ = φe on Γe = ∂Ω

~n · D = 0 on Σ electric potential distribution

~σ = cE DIV T~d + eT∇φ. . . stress

~D = eDIV T~d − εS∇φ. . . dielectr. displacement

DIV T = 12 (∇+∇T ) , cE , e, εS . . . material tensors

~d = (dx , dy , dz). . . mech. displacement

φ . . . electr. potential

Given additional boundary measurements qe = ~n · Dand/or ~d on ∂Ω, determine Σ. [Steinhorst&BK 2011]

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Different Refinement Strategies

∼ 4000 nodes with . . .

uniform adaptive boundary adaptive(residual based) concentrated & bndy conc.

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Error in crack plane angle with different refinement strategies:

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Error in crack plane offset with different refinement strategies:

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Errors with Different Refinement Strategies

∼ 20.000 nodes:

uniform adaptive boundary adaptive(residual based) concentrated & bndy conc.

error angle 0.8 e-4 0.3 e-3 1.e-6 1.e-6error offset 0.7 e-3 0.3 e-2 0.5e-5 0.3e-5

∼ 80.000 nodes:

uniform adaptive boundary adaptive(residual based) concentrated & bndy conc.

error angle 0.9 e-5 0.2 e-3 0.7 e-7 0.7 e-7error offset 0.3 e-3 0.5 e-3 0.3 e-6 0.2 e-6

Conventional refinement strategies may lead to bad results!

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Errors with Different Refinement Strategies

∼ 20.000 nodes:

uniform adaptive boundary adaptive(residual based) concentrated & bndy conc.

error angle 0.8 e-4 0.3 e-3 1.e-6 1.e-6error offset 0.7 e-3 0.3 e-2 0.5e-5 0.3e-5

∼ 80.000 nodes:

uniform adaptive boundary adaptive(residual based) concentrated & bndy conc.

error angle 0.9 e-5 0.2 e-3 0.7 e-7 0.7 e-7error offset 0.3 e-3 0.5 e-3 0.3 e-6 0.2 e-6

Conventional refinement strategies may lead to bad results!

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Some Ideas on Adaptivity for Inverse Problems

I Haber&Heldmann&Ascher’07: Tikhonov with BV type reg.:Refine for u to compute residual term sufficiently precisely;Refine for q to compute regularization term sufficiently precisely

I Neubauer’03, ’06, ’07: moving mesh reg., adaptive grid reg.:Tikhonov with BV type regularization:Refine where q has jumps or large gradients

I Borcea&Druskin’02: optimal finite difference grids (a priori):Refine close to measurements

I Chavent&Bissell’98, Ben Ameur&Chavent&Jaffre’02, BK&BenAmeur’02:refinement and coarsening indicators

I Becker&Vexler’04, Griesbaum&BK&Vexler’07, Bangerth’08,BK&Kirchner&Vexler’11, BK&Kirchner&Veljovic&Vexler’11:goal oriented error estimators

I . . .

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Some Ideas on Adaptivity for Inverse Problems

I Haber&Heldmann&Ascher’07: Tikhonov with BV type reg.:Refine for u to compute residual term sufficiently precisely;Refine for q to compute regularization term sufficiently precisely

I Neubauer’03, ’06, ’07: moving mesh reg., adaptive grid reg.:Tikhonov with BV type regularization:Refine where q has jumps or large gradients

I Borcea&Druskin’02: optimal finite difference grids (a priori):Refine close to measurements

I Chavent&Bissell’98, Ben Ameur&Chavent&Jaffre’02, BK&BenAmeur’02:refinement and coarsening indicators

I Becker&Vexler’04, Griesbaum&BK&Vexler’07, Bangerth’08,BK&Kirchner&Vexler’11, BK&Kirchner&Veljovic&Vexler’11:goal oriented error estimators

I . . .

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Parameter identification as nonlinear operator equation

F (q) = g

g δ ≈ g . . . given data; noise level δ ≥ ‖g δ − g‖F . . . forward operator: F (q) = (C S)(q) = C (u)where C . . . observation operatorwhere S . . . parameter-to-solution operator, i.e., u = S(q) solves

A(q, u)(v) = (f , v) ∀v ∈ V . . . PDE in weak form

Hilbert spaces Q, V , G : q ∈ QS→ u ∈ V

C→ g ∈ G

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Tikhonov Regularization and the Discrepancy Principle

Minimize jα(q) = ‖F (q)− g δ‖2 + α‖q‖2 over q ∈ Q ,

⇔

Minimize Jα(q, u) = ‖C (u)− g δ‖2 + α‖q‖2 over q ∈ Q , u ∈ V

under the constraint A(q, u)(v) = (f , v) ∀v ∈ V

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Tikhonov Regularization and the Discrepancy Principle

Minimize jα(q) = ‖F (q)− g δ‖2 + α‖q‖2 over q ∈ Q ,

Choice of α: discrepancy principle (fixed constant τ ≥ 1)

‖F (qδα∗)− g δ‖ = τδ

Convergence analysis: [Engl& Hanke& Neubauer 1996] and the

references therein

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Goal Oriented Error Estimators in PDE Constrained Optimization

[Becker&Kapp&Rannacher’00], [Becker&Rannacher’01], [Becker&Vexler ’04, ’05]

Minimize J(q, u) over q ∈ Q , u ∈ V

under the constraints A(q, u)(v) = f (v) ∀v ∈ V ,

Lagrange functional:

L(q, u, z) = J(q, u) + f (z)− A(q, u)(z) .

First order optimality conditions:

L′(q, u, z)[(p, v , y)] = 0 ∀(p, v , y) ∈ Q × V × V (1)

Discretization Qh ⊆ Q, Vh ⊆ V discretized version of (1).

Estimate the error due to discretization in some quantity of interest I :

I (q, u)− I (qh, uh) ≤ η

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Goal Oriented Error Estimators (II)Auxiliary functional:

M(q, u, z , p, v , y) = I (q, u) + L′(q, u, z)[(p, v , y)]

Consider additional equations:

M′(xh)(dxh) = 0 ∀dxh ∈ Xh = (Qh × Vh × Vh)2

Theorem (Becker&Vexler, J. Comp. Phys., 2005):

I (q, u)− I (qh, uh) =1

2M′(xh)(x − xh)︸ ︷︷ ︸

=:η

+O(‖x − xh‖3) ∀xh ∈ Xh .

Error estimator η is a sum of local contributions due to either q, u, z , . . . :

η =

Nq∑i=1

ηqi +

Nu∑i=1

ηui +

Nz∑i=1

ηzi +

Np∑i=1

ηpi +

Nv∑i=1

ηvi +

Ny∑i=1

ηyi

local refinement separately for q ∈ Qh, u ∈ Vh, z ∈ Vh, . . .

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Choice of Quantity of Interest ?

aim: recover infinite dimensional convergence resultsfor Tikhonov + discrepancy principlein the adaptively discretized setting

challenge: carrying over infinite dimensional results is

. . . straightforward if we can guarantee smallness of operator norm‖Fh − F‖ huge number of quantities of interest!

. . . not too hard if we can guarantee smallness of‖Fh(q†)− F (q†)‖ large number of quantities of interest!

. . . but we only want to guarantee precision of ≤ 5 quantities ofinterest

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Choice of Quantity of Interest ?

aim: recover infinite dimensional convergence resultsfor Tikhonov + discrepancy principlein the adaptively discretized setting

challenge: carrying over infinite dimensional results is

. . . straightforward if we can guarantee smallness of operator norm‖Fh − F‖ huge number of quantities of interest!

. . . not too hard if we can guarantee smallness of‖Fh(q†)− F (q†)‖ large number of quantities of interest!

. . . but we only want to guarantee precision of ≤ 5 quantities ofinterest

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Choice of Quantity of Interest ?

aim: recover infinite dimensional convergence resultsfor Tikhonov + discrepancy principlein the adaptively discretized setting

challenge: carrying over infinite dimensional results is

. . . straightforward if we can guarantee smallness of operator norm‖Fh − F‖ huge number of quantities of interest!

. . . not too hard if we can guarantee smallness of‖Fh(q†)− F (q†)‖ large number of quantities of interest!

. . . but we only want to guarantee precision of ≤ 5 quantities ofinterest

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Choice of Quantity of Interest ?

aim: recover infinite dimensional convergence resultsfor Tikhonov + discrepancy principlein the adaptively discretized setting

challenge: carrying over infinite dimensional results is

. . . straightforward if we can guarantee smallness of operator norm‖Fh − F‖ huge number of quantities of interest!

. . . not too hard if we can guarantee smallness of‖Fh(q†)− F (q†)‖ large number of quantities of interest!

. . . but we only want to guarantee precision of ≤ 5 quantities ofinterest

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Convergence Analysis Choice of Quantities of Interest

Theorem [Griesbaum&BK& Vexler’07], [BK&Kirchner&Vexler’11]:

α∗ = α∗(δ, gδ) and Qh × Vh × Vh such that for

I (q, u) := ‖C (u)− g δ‖2G = ‖F (q)− g δ‖2

G

τ2δ2 ≤ I (qδh,α∗ , uδh,α∗

) ≤ τδ2

(i) If additionally

|I (qδh,α∗ , uδh,α∗

)− I (qδα∗ , uδα∗)| ≤ cI (qδh,α∗ , u

δh,α∗

)

for some sufficiently small constant c > 0 then qδα∗→ q† as δ → 0.

Optimal rates under source conditions (logarithic/Holder).

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Convergence Analysis Choice of QoI

Theorem [Griesbaum&BK&Vexler’07], [BK&Kirchner&Vexler’11]:

α∗ = α∗(δ, gδ) and Qh × Vh × Vh such that for

I (q, u) := ‖C (u)− g δ‖2G = ‖F (q)− g δ‖2

G

τ2δ2 ≤ I (qδh,α∗ , uδh,α∗

) ≤ τδ2

(ii) If additionally for

I2(q, u) := Jα(q, u)

|I2(qδh,α∗ , uδh,α∗

)− I2(qδα∗ , uδα∗)| ≤ σδ

2

for some constant C > 0 with τ2 ≥ 1 + σ , then qδh,α∗→ q† as δ → 0.

Optimal rates under source conditions (logarithic/Holder).

see also [Neubauer&Scherzer 1990]

J as quantity of interest [Becker&Kapp&Rannacher’00], [Becker&Rannacher’01], [Bangerth’08]

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Convergence Analysis Choice of QoI

Theorem [Griesbaum&BK&Vexler’07], [BK&Kirchner&Vexler’11]:

α∗ = α∗(δ, gδ) and Qh × Vh × Vh such that for

I (q, u) := ‖C (u)− g δ‖2G = ‖F (q)− g δ‖2

G

τ2δ2 ≤ I (qδh,α∗ , uδh,α∗

) ≤ τδ2

(ii) If additionally for

I2(q, u) := Jα(q, u)

|I2(qδh,α∗ , uδh,α∗

)− I2(qδα∗ , uδα∗)| ≤ σδ

2

for some constant C > 0 with τ2 ≥ 1 + σ , then qδh,α∗→ q† as δ → 0.

Optimal rates under source conditions (logarithic/Holder).

see also [Neubauer&Scherzer 1990]

J as quantity of interest [Becker&Kapp&Rannacher’00], [Becker&Rannacher’01], [Bangerth’08]

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Efficient Computation of α Choice of QoI

Choice of α: discrepancy principle (fixed constant τ ≥ 1)

‖F (qδα∗)− g δ‖ = τδ

1-d nonlinear equation solve by Newton’s method

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Efficient Computation of α Choice of QoI

Theorem [Griesbaum&BK&Vexler’07], [BK& Kirchner&Vexler’11]:

Define

i( 1α) := I (q, u) := ‖C (u)− g δ‖2

G = ‖F (q)− g δ‖2G

I2(q, u) := i ′( 1α)

β∗ sol. to i(β∗) = τ2δ2 (discr.princ.) βk+1 = βk −ikh − τ2δ2

i ′kh(Newton)

for k ≤ k∗ − 1 with k∗ = mink ∈ N | ikh − τ2δ2 ≤ 0 with

|i(βk)− ikh | ≤ εk , |i ′(βk)− i ′kh | ≤ ε′k ,

εk , ε′k sufficiently small.Then βk satisfies quadratic convergence estimate and

(τ2 − τ2)δ2 ≤ i(βk∗) ≤ (τ2 + τ2)δ2

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Remarks

I computation of error estimators for i(β):just one more SQP type step;

I evaluation of i ′(β):can be extracted from quantities computed for errorestimators for i(β)

I error estimators for i ′(β):stationary point of another auxiliary functional

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Newton type Regularization and the Discrepancy Principle

Newton’s method for F (q) = g :

qk+1 = qk − F ′(qk)−1(F (qk)− g)

Iteratively Regularized Gauss-Newton Method IRGNM

qδk+1 = qδk − (F ′(qδk)∗F ′(qδk) + αk I )−1(F ′(qδk)∗(F (qδk)− g δ) + αk(qδk − q0))

or equivalently

qδk+1 = arg minq‖F ′(qδk)(q − qδk) + F (qδk)− g δ‖2 + αk‖q − q0‖2

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Newton type Regularization and the Discrepancy Principle

Newton’s method for F (q) = g :

qk+1 = qk − F ′(qk)−1(F (qk)− g)

Iteratively Regularized Gauss-Newton Method IRGNM

qδk+1 = qδk − (F ′(qδk)∗F ′(qδk) + αk I )−1(F ′(qδk)∗(F (qδk)− g δ) + αk(qδk − q0))

or equivalently

qδk+1 = arg minq‖F ′(qδk)(q − qδk) + F (qδk)− g δ‖2 + αk‖q − q0‖2

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Newton type Regularization and the Discrepancy Principle

Newton’s method for F (q) = g :

qk+1 = qk − F ′(qk)−1(F (qk)− g)

Iteratively Regularized Gauss-Newton Method IRGNM

qδk+1 = qδk − (F ′(qδk)∗F ′(qδk) + αk I )−1(F ′(qδk)∗(F (qδk)− g δ) + αk(qδk − q0))

or equivalently

qδk+1 = arg minq‖F ′(qδk)(q − qδk) + F (qδk)− g δ‖2 + αk‖q − q0‖2

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Newton type Regularization and the Discrepancy PrincipleIteratively Regularized Gauss-Newton Method IRGNM

qδk+1 = arg minq‖F ′(qδk)(q − qδk) + F (qδk)− g δ‖2 + αk‖q − q0‖2

or equivalently

Minimize Jk(q, u,w) = ‖C (w+u)−g δ‖2G +αk‖q−q0‖2 over

q ∈ Qu ∈ Vw ∈ V

under the constraints

A′u(qδk , u)[w ](v) + A′q(qδk , u)[q − qδk ](v) = 0 ∀v ∈ V ,

A(qδk , u)(v) = f (v) ∀v ∈ V ,

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Newton type Regularization and the Discrepancy PrincipleIteratively Regularized Gauss-Newton Method IRGNM

qδk+1 = arg minq‖F ′(qδk)(q − qδk) + F (qδk)− g δ‖2 + αk‖q − q0‖2

or equivalently

Minimize Jk(q, u,w) = ‖C (w+u)−g δ‖2G +αk‖q−q0‖2 over

q ∈ Qu ∈ Vw ∈ V

under the constraints

A′u(qδk , u)[w ](v) + A′q(qδk , u)[q − qδk ](v) = 0 ∀v ∈ V ,

A(qδk , u)(v) = f (v) ∀v ∈ V ,

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Newton type Regularization and the Discrepancy Principle

Iteratively Regularized Gauss-Newton Method IRGNM

qδk+1 = qδk − (F ′(qδk)∗F ′(qδk) + αk I )−1(F ′(qδk)∗(F (qδk)− g δ) + αk(qδk − q0))

a posteriori selection of αk (inexact Newton)

θ‖F (qk)− g δ‖ ≤ ‖F ′(qk)(qk+1 − qk) + F (qk)− g δ‖ ≤ θ‖F (qk)− g δ‖

a posteriori selection of k∗ (discrepancy principle)

k∗ = mink ∈ N : ‖F (qk)− g δ‖ ≤ τδ

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Newton type Regularization and the Discrepancy Principle

Iteratively Regularized Gauss-Newton Method IRGNM

qδk+1 = qδk − (F ′(qδk)∗F ′(qδk) + αk I )−1(F ′(qδk)∗(F (qδk)− g δ) + αk(qδk − q0))

a posteriori selection of αk (inexact Newton)

θ‖F (qk)− g δ‖ ≤ ‖F ′(qk)(qk+1 − qk) + F (qk)− g δ‖ ≤ θ‖F (qk)− g δ‖

a posteriori selection of k∗ (discrepancy principle)

k∗ = mink ∈ N : ‖F (qk)− g δ‖ ≤ τδ

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Newton type Regularization and the Discrepancy Principle

Iteratively Regularized Gauss-Newton Method IRGNM

qδk+1 = qδk − (F ′(qδk)∗F ′(qδk) + αk I )−1(F ′(qδk)∗(F (qδk)− g δ) + αk(qδk − q0))

a posteriori selection of αk (inexact Newton)

θ‖F (qk)− g δ‖ ≤ ‖F ′(qk)(qk+1 − qk) + F (qk)− g δ‖ ≤ θ‖F (qk)− g δ‖

a posteriori selection of k∗ (discrepancy principle)

k∗ = mink ∈ N : ‖F (qk)− g δ‖ ≤ τδ

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Convergence Analysis Choice of QoI

|I k+1i ,h − I k+1

i | ≤ ηk+1i , i ∈ 1, 2, 3, 4 (∗)

by goal oriented adaptivity, where

I k+11,h = ‖F ′h(qH

k )(qhk+1 − qH

k ) + Fh(qHk )− g δ‖2 + αk‖qh

k+1 − q0‖2

I k+12,h = ‖F ′h(qH

k )(qhk+1 − qH

k ) + Fh(qHk )− g δ‖2

I k+13,h = ‖Fh(qH

k )− g δ‖2

I k+14,h = ‖Fh(qh

k+1)− g δ‖2,

Theorem [BK&Kirchner&Veljovic&Vexler’11]:

Let (∗) hold with ηk+1i sufficiently small. Then qδh,k∗ → q† as δ → 0.

Optimal rates under source conditions (logarithic/Holder).

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

However. . .need to solve

2αk(qk+1 − q0) + A′q(qHk , uold)∗v 1 = 0

2C ∗(C (w + uold)− g δ) + A′′uu(qHk , uold)[w , ·](v 1)

+A′′qu(qHk , uold)[q − qH

k , ·](v 1) + A′u(qHk , uold)∗v 2 = 0

2C ∗(C (w + uold)− g δ) + A′u(qHk , uold)∗v 1 = 0

A′u(qHk , uold)[w ] + A′q(qH

k , uold)[q − qHk ] = 0

A(qHk , uold)− f = 0

A(q, u)− f = 0

- second order derivatives of PDEs appear- nonlinear PDEs have to be solved

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

An all-at once Formulation (I)

measurements: C (u) = g in GPDE: A(q, u) = f in V ∗

i.e., F(u, q) = g

Iteratively Regularized Gauss-Newton Method IRGNM(qδk+1

uδk+1

)=

(qδkuδk

)−(

F′(qδk , uδk)∗F′(qδk , u

δk) + αk

(I 00 0

))−1

×(

F′(qδk , uδk)∗(F(qδk , u

δk)− gδ) + αk

(qδk − q0

0

))

or equivalently: unconstrained quadratic minimization

(qδk+1, uδk+1)

= arg minq,u ‖A′q(qk , uk)(q − qk) + A′u(qk , uk)(u − uk) + A(qk , uk)− f ‖2V ∗

+‖Cu − g δ‖2G + αk‖q − q0‖2

Q .

see also [Burger&Muhlhuber’02]: SQP type approach

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

An all-at once Formulation (I)

measurements: C (u) = g in GPDE: A(q, u) = f in V ∗

i.e., F(u, q) = g

Iteratively Regularized Gauss-Newton Method IRGNM(qδk+1

uδk+1

)=

(qδkuδk

)−(

F′(qδk , uδk)∗F′(qδk , u

δk) + αk

(I 00 0

))−1

×(

F′(qδk , uδk)∗(F(qδk , u

δk)− gδ) + αk

(qδk − q0

0

))or equivalently: unconstrained quadratic minimization

(qδk+1, uδk+1)

= arg minq,u ‖A′q(qk , uk)(q − qk) + A′u(qk , uk)(u − uk) + A(qk , uk)− f ‖2V ∗

+‖Cu − g δ‖2G + αk‖q − q0‖2

Q .

see also [Burger&Muhlhuber’02]: SQP type approach

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

An all-at once Formulation (I)

measurements: C (u) = g in GPDE: A(q, u) = f in V ∗

i.e., F(u, q) = g

Iteratively Regularized Gauss-Newton Method IRGNM(qδk+1

uδk+1

)=

(qδkuδk

)−(

F′(qδk , uδk)∗F′(qδk , u

δk) + αk

(I 00 0

))−1

×(

F′(qδk , uδk)∗(F(qδk , u

δk)− gδ) + αk

(qδk − q0

0

))or equivalently: unconstrained quadratic minimization

(qδk+1, uδk+1)

= arg minq,u ‖A′q(qk , uk)(q − qk) + A′u(qk , uk)(u − uk) + A(qk , uk)− f ‖2V ∗

+‖Cu − g δ‖2G + αk‖q − q0‖2

Q .

see also [Burger&Muhlhuber’02]: SQP type approach

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

An all-at once Formulation (I)

measurements: C (u) = g in GPDE: A(q, u) = f in V ∗

i.e., F(u, q) = g

Iteratively Regularized Gauss-Newton Method IRGNM(qδk+1

uδk+1

)=

(qδkuδk

)−(

F′(qδk , uδk)∗F′(qδk , u

δk) + αk

(I 00 0

))−1

×(

F′(qδk , uδk)∗(F(qδk , u

δk)− gδ) + αk

(qδk − q0

0

))or equivalently: unconstrained quadratic minimization

(qδk+1, uδk+1)

= arg minq,u ‖A′q(qk , uk)(q − qk) + A′u(qk , uk)(u − uk) + A(qk , uk)− f ‖2V ∗

+‖Cu − g δ‖2G + αk‖q − q0‖2

Q .

see also [Burger&Muhlhuber’02]: SQP type approach

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

An all-at once Formulation (II)

(qδk+1, uδk+1)

= arg minq,u ‖L(q − qk) + K (u − uk) + A(qk , uk)− f ‖2V ∗

+‖Cu − g δ‖2G + αk‖q − q0‖2

Q .

with K = A′u(qk , uk), L = A′q(qk , uk).

K regular ⇒ Hessian

(L∗L + αk I L∗K

K ∗L C ∗C + K ∗K

)positive definite.

First order optimality system: αk I 0 L∗

0 C ∗C K ∗

L K I

qk+1

uk+1

zk+1

=

αkqk

−C ∗(Cuk − g δ)Lqk + Kuk − A(qk , uk) + f

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

An all-at once Formulation (II)

(qδk+1, uδk+1)

= arg minq,u ‖L(q − qk) + K (u − uk) + A(qk , uk)− f ‖2V ∗

+‖Cu − g δ‖2G + αk‖q − q0‖2

Q .

with K = A′u(qk , uk), L = A′q(qk , uk).

K regular ⇒ Hessian

(L∗L + αk I L∗K

K ∗L C ∗C + K ∗K

)positive definite.

First order optimality system: αk I 0 L∗

0 C ∗C K ∗

L K I

qk+1

uk+1

zk+1

=

αkqk

−C ∗(Cuk − g δ)Lqk + Kuk − A(qk , uk) + f

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

An all-at once Formulation (II)

(qδk+1, uδk+1)

= arg minq,u ‖L(q − qk) + K (u − uk) + A(qk , uk)− f ‖2V ∗

+‖Cu − g δ‖2G + αk‖q − q0‖2

Q .

with K = A′u(qk , uk), L = A′q(qk , uk).

K regular ⇒ Hessian

(L∗L + αk I L∗K

K ∗L C ∗C + K ∗K

)positive definite.

First order optimality system: αk I 0 L∗

0 C ∗C K ∗

L K I

qk+1

uk+1

zk+1

=

αkqk

−C ∗(Cuk − g δ)Lqk + Kuk − A(qk , uk) + f

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

All-At-Once-Newton type Regularization and theDiscrepancy Principle

Iteratively Regularized Gauss-Newton Method IRGNM(qδk+1

uδk+1

)=

(qδkuδk

)−(

F′(qδk , uδk)∗F′(qδk , u

δk) + αk

(I 00 0

))−1

×(

F′(qδk , uδk)∗(F(qδk , u

δk)− gδ) + αk

(qδk − q0

0

))a posteriori selection of αk (inexact Newton)

θ‖F(qk,uk)− gδ‖ ≤ ‖F′(qk,uk)

(qk+1−qk

uk+1−uk

)+ F(qk,uk)− gδ‖ ≤ θ‖F(qk,uk)− gδ‖

a posteriori selection of k∗ (discrepancy principle)

k∗ = mink ∈ N : ‖F(qk , uk)− gδ‖ ≤ τδ

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

All-At-Once-Newton type Regularization and theDiscrepancy Principle

Iteratively Regularized Gauss-Newton Method IRGNM(qδk+1

uδk+1

)=

(qδkuδk

)−(

F′(qδk , uδk)∗F′(qδk , u

δk) + αk

(I 00 0

))−1

×(

F′(qδk , uδk)∗(F(qδk , u

δk)− gδ) + αk

(qδk − q0

0

))a posteriori selection of αk (inexact Newton)

θ‖F(qk,uk)− gδ‖ ≤ ‖F′(qk,uk)

(qk+1−qk

uk+1−uk

)+ F(qk,uk)− gδ‖ ≤ θ‖F(qk,uk)− gδ‖

a posteriori selection of k∗ (discrepancy principle)

k∗ = mink ∈ N : ‖F(qk , uk)− gδ‖ ≤ τδ

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

All-At-Once-Newton type Regularization and theDiscrepancy Principle

Iteratively Regularized Gauss-Newton Method IRGNM(qδk+1

uδk+1

)=

(qδkuδk

)−(

F′(qδk , uδk)∗F′(qδk , u

δk) + αk

(I 00 0

))−1

×(

F′(qδk , uδk)∗(F(qδk , u

δk)− gδ) + αk

(qδk − q0

0

))a posteriori selection of αk (inexact Newton)

θ‖F(qk,uk)− gδ‖ ≤ ‖F′(qk,uk)

(qk+1−qk

uk+1−uk

)+ F(qk,uk)− gδ‖ ≤ θ‖F(qk,uk)− gδ‖

a posteriori selection of k∗ (discrepancy principle)

k∗ = mink ∈ N : ‖F(qk , uk)− gδ‖ ≤ τδ

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Convergence Analysis Choice of QoI

|I k+1i ,h − I k+1

i | ≤ ηk+1i , i ∈ 1, 2, 3, 4 (∗)

by goal oriented adaptivity, where

I k+11,h = ‖F′h(qH

k ,uHk )

(qhk+1−qH

k

uhk+1−uH

k

)+ Fh(qH

k ,uHk )− gδ‖+ αk‖qh

k+1 − q0‖2

I k+12,h = ‖F′h(qH

k ,uHk )

(qhk+1−qH

k

uhk+1−uH

k

)+ Fh(qH

k ,uHk )− gδ‖

I k+13,h = ‖Fh(qH

k , uHk )− gδ‖2

I k+14,h = ‖Fh(qh

k+1, uhk+1)− gδ‖2,

Theorem [BK&Kirchner&Veljovic&Vexler’11]:

Let (∗) hold with ηk+1i sufficiently small. Then qδh,k∗ → q† as δ → 0.

Optimal rates under source conditions (logarithic/Holder).

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Convergence Analysis Choice of QoI

|I k+1i ,h − I k+1

i | ≤ ηk+1i , i ∈ 1, 2, 3, 4 (∗)

by goal oriented adaptivity, where

I k+11,h = ‖F′h(qH

k ,uHk )

(qhk+1−qH

k

uhk+1−uH

k

)+ Fh(qH

k ,uHk )− gδ‖+ αk‖qh

k+1 − q0‖2

I k+12,h = ‖F′h(qH

k ,uHk )

(qhk+1−qH

k

uhk+1−uH

k

)+ Fh(qH

k ,uHk )− gδ‖

I k+13,h = ‖Fh(qH

k , uHk )− gδ‖2

I k+14,h = ‖Fh(qh

k+1, uhk+1)− gδ‖2,

- only first order derivatives of PDEs appear- only residuals of nonlinear PDEs have to be evaluated

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Numerical Tests (Tikhonov & Discrepancy Principle)nonlinear inverse source problem:

−∆u + 1000u3 = q in Ω = (0, 1)2 + homogeneous Dirichlet BC

Identify q from distributed measurements of u at 10× 10 points in Ω

(a) q†(x , y) =1

2πσ2exp

(− (x− 5

11 )2+(y− 511 )2

2σ2

), σ = 0.01

(b) q†(x , y) = q1(x , y) + q2(x , y)

qi =1

2πσ2exp

−1

2

(six − 12

σ

)2

+

(siy − 1

2

σ

)2 ,

σ = 0.1s1 = 2,s2 = 0.8

(c) q†(x , y) =

1 x ≤ 1

20 x > 1

2

Computations with Gascoigne and RoDoBo.

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Numerical Tests

exact par. q:

(a), (b), (c)

exact state u:

(a), (b), (c)

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Numerical Results (I)

Computations with 1% random noise:number of nodes on finest grid:

(a) (b) (c)

uniform 263169 66049 66049adaptive 14157 18035 56409

reduction of CPU time 92% 53% 10%

adaptive grids:

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Numerical Results (II)

exact par. q:

(a), (b), (c)

computed par. q:

(a), (b), (c)

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Numerical Results (III)

Convergence as δ → 0 for example (a),linear inverse source problem

with σ = 0.05 with σ = 0.01

δ‖qδ

α∗−q†‖‖q†‖ 1/α∗

8% 0.761 156.3904% 0.592 660.9302% 0.414 2426.1091% 0.288 7047.4720.5% 0.229 17042.825

δ‖qδ

α∗−q†‖‖q†‖ 1/α∗

8% 0.869 2396.2814% 0.776 9044.3742% 0.744 24364.8941% 0.734 55017.3640.5% 0.731 117560.866

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Summary and Outlook• goal oriented error estimators allow to control the error in

some quantity of interest• Tikhonov& discrepancy principle:

I sufficiently small error in residual norm and Tikh. func.⇒ convergence and rates

I sufficiently small error in residual norm i( 1α ) and in i ′( 1

α )⇒ fast convergence of Newton’s method for finding α∗ (discr.princ.) coarse grids at the beginning of Newton’s method→ save computational effort

• regularized Newton iterations& discrepancy principle:I sufficiently small error in nonlinear residual norm,

linearized residual norm, and linearized Tikhonov functional⇒ convergence and rates

I all-at-once-approach

→ other regularization methods

→ other parameter choice strategies: e.g., balancing principle

Adaptive discretization strategies for the identification of distributed parameters in partial differential equations

Thank you for your attention!