First Edition, 2012 ISBN 978-81-323-1362-5 © All rights reserved. Published by: College Publishing House 4735/22 Prakashdeep Bldg, Ansari Road, Darya Ganj, Delhi - 110002 Email: info@wtbooks.com 

Table of Contents

Chapter 1 - Numerical Ordinary Differential Equations Chapter 2 - Boundary Element Method, Beeman's Algorithm and Adaptive

Stepsize Chapter 3 - Céa's Lemma Chapter 4 - Constraint Algorithm

Chapter 5 - Compact Stencil, Courant–Friedrichs–Lewy Condition and Direct multiple Shooting Method

Chapter 6 - Crank–Nicolson Method

Chapter 7 - Discrete Laplace Operator and Discrete Poisson Equation

Chapter 8 - Euler Method

Chapter 9 - Finite Difference

Chapter 10 - Finite Difference Method

Chapter 11 - Finite Element Method Chapter 12 - Bramble-Hilbert Lemma and Spectral Element Method Chapter 13 - hp-FEM Chapter 14 - Finite Element Method in Structural Mechanics

Chapter 15 - Interval Finite Element

Chapter 16 - Modal Analysis using FEM

Chapter 17 - Domain Decomposition Methods and Additive Schwarz Method

Chapter 1

Numerical Ordinary Differential Equations

Illustration of numerical integration for the differential equation y' = y,y(0) = 1. Blue: the Euler method, green: the midpoint method, red: the exact solution, y = et. The step size is h = 1.0.

The same illustration for h = 0.25. It is seen that the midpoint method converges faster than the Euler method.

Numerical ordinary differential equations is the part of numerical analysis which studies the numerical solution of ordinary differential equations (ODEs). This field is also known under the name numerical integration, but some people reserve this term for the computation of integrals.

Many differential equations cannot be solved analytically, in which case we have to satisfy ourselves with an approximation to the solution. The algorithms studied here can be used to compute such an approximation. An alternative method is to use techniques from calculus to obtain a series expansion of the solution.

Ordinary differential equations occur in many scientific disciplines, for instance in physics, chemistry, biology, and economics. In addition, some methods in numerical partial differential equations convert the partial differential equation into an ordinary differential equation, which must then be solved.

The problem

We want to approximate the solution of the differential equation

where f is a function that maps [t0,∞) × Rd to Rd, and the initial condition y0 ∈ Rd is a given vector.

The above formulation is called an initial value problem (IVP). The Picard–Lindelöf theorem states that there is a unique solution, if f is Lipschitz continuous. In contrast, boundary value problems (BVPs) specify (components of) the solution y at more than one point. Different methods need to be used to solve BVPs, for example the shooting method (and its variants) or global methods like finite differences, Galerkin methods, or collocation methods.

Note that we restrict ourselves to first-order differential equations (meaning that only the first derivative of y appears in the equation, and no higher derivatives). However, a higher-order equation can easily be converted to a system of first-order equations by introducing extra variables. For example, the second-order equation y'' = −y can be rewritten as two first-order equations: y' = z and z' = −y.

Methods

Three elementary methods are discussed to give the reader a feeling for the subject. After that, pointers are provided to other methods (which are generally more accurate and efficient). The methods mentioned here are analysed in the next section.

The Euler method

A brief explanation: From any point on a curve, you can find an approximation of a nearby point on the curve by moving a short distance along a line tangent to the curve.

Rigorous development: Starting with the differential equation (1), we replace the derivative y' by the finite difference approximation

which when re-arranged yields the following formula

and using (1) gives:

This formula is usually applied in the following way. We choose a step size h, and we construct the sequence t0, t1 = t0 + h, t2 = t0 + 2h, … We denote by yn a numerical estimate of the exact solution y(tn). Motivated by (3), we compute these estimates by the following recursive scheme

This is the Euler method (or forward Euler method, in contrast with the backward Euler method, to be described below). The method is named after Leonhard Euler who described it in 1768.

The Euler method is an example of an explicit method. This means that the new value yn+1 is defined in terms of things that are already known, like yn.

The backward Euler method

If, instead of (2), we use the approximation

we get the backward Euler method:

The backward Euler method is an implicit method, meaning that we have to solve an equation to find yn+1. One often uses fixed point iteration or (some modification of) the Newton–Raphson method to achieve this. Of course, it costs time to solve this equation; this cost must be taken into consideration when one selects the method to use. The advantage of implicit methods such as (6) is that they are usually more stable for solving a stiff equation, meaning that a larger step size h can be used.

The exponential Euler method

If the differential equation is of the form

then an approximate explicit solution can be given by

This method is commonly employed in neural simulations and it is the default integrator in the GENESIS neural simulator.

Generalizations

The Euler method is often not accurate enough. In more precise terms, it only has order one (the concept of order is explained below). This caused mathematicians to look for higher-order methods.

One possibility is to use not only the previously computed value yn to determine yn+1, but to make the solution depend on more past values. This yields a so-called multistep method. Perhaps the simplest is the Leapfrog method which is second order and (roughly speaking) relies on two time values.

Almost all practical multistep methods fall within the family of linear multistep methods, which have the form

Another possibility is to use more points in the interval [tn,tn+1]. This leads to the family of Runge–Kutta methods, named after Carl Runge and Martin Kutta. One of their fourth-order methods is especially popular.

Advanced features

A good implementation of one of these methods for solving an ODE entails more than the time-stepping formula.

It is often inefficient to use the same step size all the time, so variable step-size methods have been developed. Usually, the step size is chosen such that the (local) error per step is below some tolerance level. This means that the methods must also compute an error indicator, an estimate of the local error.

An extension of this idea is to choose dynamically between different methods of different orders (this is called a variable order method). Methods based on Richardson extrapolation, such as the Bulirsch–Stoer algorithm, are often used to construct various methods of different orders.

Other desirable features include:

• dense output: cheap numerical approximations for the whole integration interval, and not only at the points t0, t1, t2, ...

• event location: finding the times where, say, a particular function vanishes. • support for parallel computing. • when used for integrating with respect to time, time reversibility

Alternative methods

Many methods do not fall within the framework discussed here. Some classes of alternative methods are:

• multiderivative methods, which use not only the function f but also its derivatives. This class includes Hermite–Obreschkoff methods and Fehlberg methods, as well as methods like the Parker–Sochacki method or Bychkov-Scherbakov method, which compute the coefficients of the Taylor series of the solution y recursively.

• methods for second order ODEs. We said that all higher-order ODEs can be transformed to first-order ODEs of the form (1). While this is certainly true, it may not be the best way to proceed. In particular, Nyström methods work directly with second-order equations.

• geometric integration methods are especially designed for special classes of ODEs (e.g., symplectic integrators for the solution of Hamiltonian equations). They take care that the numerical solution respects the underlying structure or geometry of these classes.

Analysis

Numerical analysis is not only the design of numerical methods, but also their analysis. Three central concepts in this analysis are:

• convergence: whether the method approximates the solution, • order: how well it approximates the solution, and • stability: whether errors are damped out.

Convergence

A numerical method is said to be convergent if the numerical solution approaches the exact solution as the step size h goes to 0. More precisely, we require that for every ODE (1) with a Lipschitz function f and every t* > 0,

All the methods mentioned above are convergent. In fact, convergence is a condition sine qua non for any numerical scheme.

Consistency and order

Suppose the numerical method is

The local error of the method is the error committed by one step of the method. That is, it is the difference between the result given by the method, assuming that no error was made in earlier steps, and the exact solution:

The method is said to be consistent if

The method has order p if

Hence a method is consistent if it has an order greater than 0. The (forward) Euler method (4) and the backward Euler method (6) introduced above both have order 1, so they are consistent. Most methods being used in practice attain higher order. Consistency is a necessary condition for convergence, but not sufficient; for a method to be convergent, it must be both consistent and zero-stable.

A related concept is the global error, the error sustained in all the steps one needs to reach a fixed time t. Explicitly, the global error at time t is yN − y(t) where N = (t−t0)/h. The global error of a pth order one-step method is O(hp); in particular, such a method is convergent. This statement is not necessarily true for multi-step methods.

Stability and stiffness

For some differential equations, application of standard methods—such as the Euler method, explicit Runge–Kutta methods, or multistep methods (e.g., Adams–Bashforth methods)—exhibit instability in the solutions, though other methods may produce stable solutions. This "difficult behaviour" in the equation (which may not necessarily be complex itself) is described as stiffness, and is often caused by the presence of different time scales in the underlying problem. Stiff problems are ubiquitous in chemical kinetics, control theory, solid mechanics, weather forecasting, biology, plasma physics, and electronics.

History

Below is a timeline of some important developments in this field.

• 1768 - Leonhard Euler publishes his method.

• 1824 - Augustin Louis Cauchy proves convergence of the Euler method. In this proof, Cauchy uses the implicit Euler method.

• 1855 - First mention of the multistep methods of John Couch Adams in a letter written by F. Bashforth.

• 1895 - Carl Runge publishes the first Runge–Kutta method. • 1905 - Martin Kutta describes the popular fourth-order Runge–Kutta method. • 1910 - Lewis Fry Richardson announces his extrapolation method, Richardson

extrapolation. • 1952 - Charles F. Curtiss and Joseph Oakland Hirschfelder coin the term stiff

equations.

Chapter 2

Boundary Element Method, Beeman's Algorithm and Adaptive Stepsize

Boundary element method The boundary element method (BEM) is a numerical computational method of solving linear partial differential equations which have been formulated as integral equations (i.e. in boundary integral form). It can be applied in many areas of engineering and science including fluid mechanics, acoustics, electromagnetics, and fracture mechanics. (In electromagnetics, the more traditional term "method of moments" is often, though not always, synonymous with "boundary element method".)

Mathematical basis

The integral equation may be regarded as an exact solution of the governing partial differential equation. The boundary element method attempts to use the given boundary conditions to fit boundary values into the integral equation, rather than values throughout the space defined by a partial differential equation. Once this is done, in the post-processing stage, the integral equation can then be used again to calculate numerically the solution directly at any desired point in the interior of the solution domain.

BEM is applicable to problems for which Green's functions can be calculated. These usually involve fields in linear homogeneous media. This places considerable restrictions on the range and generality of problems to which boundary elements can usefully be applied. Nonlinearities can be included in the formulation, although they will generally introduce volume integrals which then require the volume to be discretised before solution can be attempted, removing one of the most often cited advantages of BEM. A useful technique for treating the volume integral without discretising the volume is the dual-reciprocity method. The technique approximates part of the integrand using radial basis functions (local interpolating functions) and converts the volume integral into boundary integral after collocating at selected points distributed throughout the volume domain (including the boundary). In the dual-reciprocity BEM, although there is no need to discretize the volume into meshes, unknowns at chosen points inside the solution domain are involved in the linear algebraic equations approximating the problem being considered.

The Green's function elements connecting pairs of source and field patches defined by the mesh form a matrix, which is solved numerically. Unless the Green's function is well behaved, at least for pairs of patches near each other, the Green's function must be integrated over either or both the source patch and the field patch. The form of the method in which the integrals over the source and field patches are the same is called "Galerkin's method". Galerkin's method is the obvious approach for problems which are symmetrical with respect to exchanging the source and field points. In frequency domain electromagnetics, this is assured by electromagnetic reciprocity. The cost of computation involved in naive Galerkin implementations is typically quite severe. One must loop over elements twice (so we get n2 passes through) and for each pair of elements we loop through Gauss points in the elements producing a multiplicative factor proportional to the number of Gauss-points squared. Also, the function evaluations required are typically quite expensive, involving trigonometric/hyperbolic function calls. Nonetheless, the principal source of the computational cost is this double-loop over elements producing a fully populated matrix.

The Green's functions, or fundamental solutions, are often problematic to integrate as they are based on a solution of the system equations subject to a singularity load (e.g. the electrical field arising from a point charge). Integrating such singular fields is not easy. For simple element geometries (e.g. planar triangles) analytical integration can be used. For more general elements, it is possible to design purely numerical schemes that adapt to the singularity, but at great computational cost. Of course, when source point and target element (where the integration is done) are far-apart, the local gradient surrounding the point need not be quantified exactly and it becomes possible to integrate easily due to the smooth decay of the fundamental solution. It is this feature that is typically employed in schemes designed to accelerate boundary element problem calculations.

Comparison to other methods

The boundary element method is often more efficient than other methods, including finite elements, in terms of computational resources for problems where there is a small surface/volume ratio . Conceptually, it works by constructing a "mesh" over the modelled surface. However, for many problems boundary element methods are significantly less efficient than volume-discretisation methods (finite element method, finite difference method, finite volume method).

Boundary element formulations typically give rise to fully populated matrices. This means that the storage requirements and computational time will tend to grow according to the square of the problem size. By contrast, finite element matrices are typically banded (elements are only locally connected) and the storage requirements for the system matrices typically grow quite linearly with the problem size. Compression techniques (e.g. multipole expansions or adaptive cross approximation/hierarchical matrices) can be used to ameliorate these problems, though at the cost of added complexity and with a success-rate that depends heavily on the nature of the problem being solved and the geometry involved.

Beeman's algorithm Beeman's algorithm is a method for numerically integrating ordinary differential equations of order 2. Beeman in his paper introduced several predictor-corrector methods adapted to the equation of motion , where x is the position and the velocity. Most popular and known under the name Beeman's method became a reduced explicit variant of the order 3 method, which is closely related to Verlet integration. It produces identical positions to the Verlet iteration, but is more accurate in velocities. It is most commonly used in molecular dynamics simulations.

Equation

The formula used to compute the positions at time t + Δt is:

.

With this prediction of the position vector, the (in the evaluation of a(t + Δt)) implicit system

is iterated one or two times. The values on the right are the old values of the last iterations, resulting in the new values on the left. Beeman also proposed to alternatively replace the velocity update in the last equation by the second order Adams–Moulton method:

where

• t is present time (i.e.: independent variable) • Δt is the time step size • x(t) is the position at time t • v(t) is the velocity at time t • a(t) is the acceleration at time t, computed as a function of x(t) • the last term is the error term, using the big O notation

Predictor-Corrector Modifications

In systems where the forces are a function of velocity in addition to position, the above equations need to be modified into a predictor-corrector form whereby the velocities at time t + Δt are predicted and the forces calculated, before producing a corrected form of the velocities.

An example is:

The velocities at time t =t + Δt are then calculated from the positions.

The accelerations at time t =t + Δt are then calculated from the positions and predicted velocities.

Error term

As shown above, the error term is O(Δt4) for position and O(Δt3) velocity. In comparison, Verlet is O(Δt4) for position and O(Δt2) for velocity. In exchange for greater accuracy, Beeman's algorithm is moderately computationally more expensive.

Memory Requirements

The simulation must keep track of position, velocity, acceleration and previous acceleration vectors per particle (though some clever work-arounds for storing the previous acceleration vector are possible), keeping its memory requirements on par with velocity Verlet and slightly more expensive than the original Verlet method.

Adaptive stepsize Adaptive stepsize is a technique in numerical analysis used for many problems, but mainly for integration. It can be used for both normal integration (i.e. quadrature), or the process of solving an ordinary differential equation.

As usual, an initial value problem is stated:

Here, it is made clear that y and f can be vectors, as they will be when dealing with a system of coupled differential equations.

Suppose we are interested in obtaining a solution at point t = b, given a function f(t,y), an initial time point, t = a, and an initial solution ya = y(a). Of course a numerical solution will generally have an error, so we assume yb + ε = y(b), where ε is the error.

For the sake of simplicity, the following example uses the simplest integration method, the Euler method. Note that the Euler method is almost exclusively useful for educational purposes; in practice, higher-order (Runge-Kutta) methods are used due to their superior convergence and stability properties.

Recall that the Euler method is derived from Taylor's theorem with the intermediate value theorem and the fact that y'(t) = f(t,y):

Which leads to the Euler method:

And its local truncation error

We mark this solution and its error with a (0). Since c is not known to us in the general case (it depends on the derivatives of f), in order to say something useful about the error, a second solution should be created, using a stepsize that is smaller. For example half the original stepsize. Note that we have to apply Euler's method twice now, meaning we get two times the local error (in the worst case). Our new, and presumably more accurate solution is marked with a (1).

Here, we assume error factor c is constant over the interval [t,t + h]. In reality its rate of change is proportional to y(3)(t). Subtracting solutions gives the error estimate:

This local error estimate is third order accurate.

The local error estimate can be used to decide how stepsize h should be modified to achieve the desired accuracy. For example, if a local tolerance of tol is allowed, we could let h evolve like:

The 0.9 is a safety factor to ensure success on the next try. This should, in principle give

an error of about in the next try. If , we consider the step successful, and the error estimate is used to improve the solution:

This solution is actually third order accurate in the local scope (second order in the global scope), but since there is no error estimate for it, this doesn't help in reducing the number of steps. This technique is called Richardson extrapolation.

Beginning with an initial stepsize of h = b − a, this theory facilitates our controllable integration of the ODE from point a to b, using an optimal number of steps given a local error tolerance.

Similar methods can be developed for higher order methods, such as the Runge-Kutta 4th order method. Also, a global error tolerance can be achieved by scaling the local error to global scope. However, you might end up with a stepsize that is prohibitively small, especially using this Euler based method.

Chapter 3

Céa's Lemma

Céa's lemma is a lemma in mathematics. It is an important tool for proving error estimates for the finite element method applied to elliptic partial differential equations.

Lemma statement

Let V be a real Hilbert space with the norm Let be a bilinear form with the properties

• for some constant γ > 0 and all v,w in V (continuity)

• for some constant α > 0 and all v in V (coercivity or V-ellipticity).

Let be a bounded linear operator. Consider the problem of finding an element u in V such that

for all v in

Consider the same problem on a finite-dimensional subspace Vh of V, so, uh in Vh satisfies

for all v in

By the Lax–Milgram theorem, each of these problems has exactly one solution. Céa's lemma states that

for all v in Vh.

That is to say, the subspace solution uh is "the best" approximation of u in Vh, up to the constant γ / α.

The proof is straightforward

for all v in Vh.

We used the a-orthogonality of u − uh and Vh

in Vh

which follows directly from

a(u,v) = L(v) = a(uh,v) for all v in Vh.

Note: Céa's lemma holds on complex Hilbert spaces also, one then uses a sesquilinear form instead of a bilinear one. The coercivity assumption then becomes

for all v in V (notice the absolute value sign around a(v,v)).

Error estimate in the energy norm

The subspace solution uh is the projection of u onto the subspace Vh in respect to the inner product .

In many applications, the bilinear form is symmetric, so

for all v,w in V.

This, together with the above properties of this form, implies that is an inner product on V. The resulting norm

is called the energy norm, since it corresponds to a physical energy in many problems. This norm is equivalent to the original norm

Using the a-orthogonality of u − uh and Vh and the Cauchy–Schwarz inequality

for all v in Vh

Hence, in the energy norm, the inequality in Céa's lemma becomes

for all v in Vh

(notice that the constant γ / α on the right-hand side is no longer present).

This states that the subspace solution uh is the best approximation to the full-space solution u in respect to the energy norm. Geometrically, this means that uh is the projection of the solution u onto the subspace Vh in respect to the inner product

Using this result, one can also derive a sharper estimate in the norm . Since

for all v in Vh,

it follows that

for all v in Vh.

An application of Céa's lemma

We will apply Céa's lemma to estimate the error of calculating the solution to an elliptic differential equation by the finite element method.

A string with fixed endpoints under the influence of a force pointing down

Consider the problem of finding a function satisfying the conditions

where is a given continuous function.

Physically, the solution u to this two-point boundary value problem represents the shape taken by a string under the influence of a force such that at every point x between a and b the force density is (where is a unit vector pointing vertically, while the endpoints of the string are on a horizontal line). For example, that force may be the gravity, when f is a constant function (since the gravitational force is the same at all points).

Let the Hilbert space V be the Sobolev space which is the space of all square integrable functions v defined on [a,b] that have a weak derivative on [a,b] with v' also being square integrable, and v satisfies the conditions v(a) = v(b) = 0. The inner product on this space is

for all v and w in

After multiplying the original boundary value problem by v in this space and performing an integration by parts, one obtains the equivalent problem

for all v in V,

with

(here the bilinear form is given by the same expression as the inner product, this is not always the case), and

It can be shown that the bilinear form and the operator L satisfy the assumptions of Céa's lemma.

A function in Vh (in red), and the typical collection of basis functions in Vh (in blue)

In order to determine a finite-dimensional subspace Vh of V, consider a partition

of the interval [a,b], and let Vh be the space of all continuous functions that are affine on each subinterval in the partition (such functions are called piecewise-linear). In addition, assume that any function in Vh takes the value 0 at the endpoints of [a,b]. It follows that Vh is a vector subspace of V whose dimension is n − 1 (the number of points in the partition that are not endpoints).

Let uh be the solution to the subspace problem

for all v in Vh,

so one can think of uh as of a piecewise-linear approximation to the exact solution u. By Céa's lemma, there exists a constant C > 0 dependent only on the bilinear form

such that

for all v in

To explicitly calculate the error between u and uh, consider the function πu in Vh that has the same values as u at the nodes of the partition (so πu is obtained by linear interpolation

on each interval [xi,xi + 1] from the values of u at interval's endpoints). It can be shown using Taylor's theorem that there exists a constant K that depends only on the endpoints a and b, such that

for all x in [a,b], where h is the largest length of the subintervals [xi,xi + 1] in the partition, and the norm on the right-hand side is the L2 norm.

This inequality then yields an estimate for the error

Then, by substituting v = πu in Céa's lemma it follows that

where C is a different constant from the above (it depends only on the bilinear form, which implicitly depends on the interval [a,b]).

This result is of a fundamental importance, as it states that the finite element method can be used to approximately calculate the solution of our problem, and that the error in the computed solution decreases proportionately to the partition size h. Céa's lemma can be applied along the same lines to derive error estimates for finite element problems in higher dimensions (here the domain of u was in one dimension), and while using higher order polynomials for the subspace Vh.

Chapter 4

Constraint Algorithm

In mechanics, a constraint algorithm is a method for satisfying constraints for bodies that obey Newton's equations of motion. There are three basic approaches to satisfying such constraints: choosing novel unconstrained coordinates ("internal coordinates"), introducing explicit constraint forces, and minimizing constraint forces implicitly by the technique of Lagrange multipliers or projection methods.

Constraint algorithms are often applied to molecular dynamics simulations. Although such simulations are sometimes carried out in internal coordinates that automatically satisfy the bond-length and bond-angle constraints, they may also be carried out with explicit or implicit constraint forces for the bond lengths and bond angles. Explicit constraint forces typically shorten the time-step significantly, making the simulation less efficient computationally; in other words, more computer power is required to compute a trajectory of a given length. Therefore, internal coordinates and implicit-force constraint solvers are generally preferred.

Mathematical background

The motion of a set of N particles can be described by a set of second-order ordinary differential equations, Newton's second law, which can be written in matrix form

where M is a mass matrix and q is the vector of generalized coordinates that describe the particles' positions. For example, the vector q may be a 3N Cartesian coordinates of the particle positions rk, where k runs from 1 to N; in the absence of constraints, M would be the 3Nx3N diagonal square matrix of the particle masses. The vector f represents the generalized forces and the scalar V(q) represents the potential energy, both of which are functions of the generalized coordinates q.

If M constraints are present, the coordinates must also satisfy M time-independent algebraic equations

where the index j runs from 1 to M. For brevity, these functions gi are grouped into an M-dimensional vector g below. The task is to solve the combined set of differential-algebraic (DAE) equations, instead of just the ordinary differential equations (ODE) of Newton's second law.

This problem was studied in detail by Joseph Louis Lagrange, who laid out most of the methods for solving it. The simplest approach is to define new generalized coordinates that are unconstrained; this approach eliminates the algebraic equations and reduces the problem once again to solving an ordinary differential equation. Such an approach is used, for example, in describing the motion of a rigid body; the position and orientation of a rigid body can be described by six independent, unconstrained coordinates, rather than describing the positions of the particles that make it up and the constraints among them that maintain their relative distances. The drawback of this approach is that the equations may become unwieldy and complex; for example, the mass matrix M may become non-diagonal and depend on the generalized coordinates.

A second approach is to introduce explicit forces that work to maintain the constraint; for example, one could introduce strong spring forces that enforce the distances among mass points within a "rigid" body. The two difficulties of this approach are that the constraints are not satisfied exactly, and the strong forces may require very short time-steps, making simulations inefficient computationally.

A third approach is to use a method such as Lagrange multipliers or projection to the constraint manifold to determine the coordinate adjustments necessary to satisfy the constraints. Finally, there are various hybrid approaches in which different sets of constraints are satisfied by different methods, e.g., internal coordinates, explicit forces and implicit-force solutions.

Internal coordinate methods

The simplest approach to satisfying constraints in energy minimization and molecular dynamics is to represent the mechanical system in so-called internal coordinates corresponding to unconstrained independent degrees of freedom of the system. For example, the dihedral angles of a protein are an independent set of coordinates that specify the positions of all the atoms without requiring any constraints. The difficulty of such internal-coordinate approaches is twofold: the Newtonian equations of motion become much more complex and the internal coordinates may be difficult to define for cyclic systems of constraints, e.g., in ring puckering or when a protein has a disulfide bond.

The original methods for efficient recursive energy minimization in internal coordinates were developed by Gō and coworkers.

Efficient recursive, internal-coordinate constraint solvers were extended to molecular dynamics. Analogous methods were applied later to other systems.

Lagrange multiplier-based methods

Resolving the constraints of a rigid water molecule using Lagrange multipliers: a) the unconstrained positions are obtained after a simulation time-step, b) the gradients of each constraint over each particle are computed and c) the Lagrange multipliers are computed for each gradient such that the constraints are satisfied.

In most molecular dynamics simulation, constraints are enforced using the method of Lagrange multipliers. Given a set of n linear (holonomic) constraints at the time t,

where and are the positions of the two particles involved in the kth constraint at the time t and dk is the prescribed inter-particle distance.

These constraint equations, are added to the potential energy function in the equations of motion, resulting in, for each of the N particles in the system

Adding the constraint equations to the potential does not change it, since all should, ideally, be zero.

Integrating both sides of the equations of motion twice in time yields the constrained particle positions at the time t + Δt

where is the unconstrained (or uncorrected) position of the ith particle after integrating the unconstrained equations of motion.

To satisfy the constraints in the next timestep, the Lagrange multipliers must be chosen such that

This implies solving a system of n non-linear equations

simultaneously for the n unknown Lagrange multipliers λk.

This system of n non-linear equations in n unknowns is best solved using Newton's method where the solution vector is updated using

where is the Jacobian of the equations σk:

Since not all particles are involved in all constraints, is blockwise-diagonal and can be solved blockwise, i.e. molecule for molecule.

Furthermore, instead of constantly updating the vector , the iteration is started with

, resulting in simpler expressions for and . After each iteration, the unconstrained particle positions are updated using

.

The vector is then reset to

This is repeated until the constraint equations are satisfied up to a prescribed tolerance.

Although there are a number of algorithms to compute the Lagrange multipliers, they differ only in how they solve the system of equations, usually using Quasi-Newton methods.

The SETTLE algorithm

The SETTLE algorithm solves the system of non-linear equations analytically for n = 3 constraints in constant time. Although it does not scale to larger numbers of constraints, it is very often used to constrain rigid water molecules, which are present in almost all biological simulations and are usually modelled using three constraints (e.g. SPC/E and TIP3P water models).

The SHAKE algorithm

The SHAKE algorithm was the first algorithm developed to satisfy bond geometry constraints during molecular dynamics simulations.

It solves the system of non-linear constraint equations using the Gauss-Seidel method to approximate the solution of the linear system of equations

in the Newton iteration. This amounts to assuming that is diagonally dominant and solving the kth equation only for the k unknown. In practice, we compute

λk

for all iteratively until the constraint equations are solved to a given tolerance.

Each iteration of the SHAKE algorithm costs operations and the iterations themselves converge linearly.

A noniterative form of SHAKE was developed later.

Several variants of the SHAKE algorithm exist. Although they differ in how they compute or apply the constraints themselves, the constraints are still modelled using Lagrange multipliers which are computed using the Gauss-Seidel method.

The original SHAKE algorithm is limited to mechanical systems with a tree structure, i.e., no closed loops of constraints. A later extension of the method, QSHAKE (Quaternion SHAKE) was developed to amend this. It works satisfactorily for rigid loops such as aromatic ring systems but fails for flexible loops, such as when a protein has a disulfide bond.

Further extensions include RATTLE, WIGGLE and MSHAKE. RATTLE works the same way as SHAKE, yet using the Velocity Verlet time integration scheme. WIGGLE extends SHAKE and RATTLE by using an initial estimate for the Lagrange multipliers λk

based on the particle velocities. Finally, MSHAKE computes corrections on the constraint forces, achieving better convergence.

A final modification is the P-SHAKE algorithm for rigid or semi-rigid molecules. P-SHAKE computes and updates a pre-conditioner which is applied to the constraint gradients before the SHAKE iteration, causing the Jacobian to become diagonal or strongly diagonally dominant. The thus de-coupled constraints converge much faster (quadratically as opposed to linearly) at a cost of .

The M-SHAKE algorithm

The M-SHAKE algorithm solves the non-linear system of equations using Newton's method directly. In each iteration, the linear system of equations

is solved exactly using an LU decomposition. Each iteration costs operations, yet the solution converges quadratically, requiring fewer iterations than SHAKE.

This solution was first proposed in 1986 by Ciccotti and Ryckaert under the title "the matrix method", yet differed in the solution of the linear system of equations. Ciccotti and Ryckaert suggest inverting the matrix directly, yet doing so only once, in the first iteration. The first iteration then costs operations, whereas the following iterations cost only operations (for the matrix-vector multiplication). This improvement comes at a cost though, since the Jacobian is no longer updated, convergence is only linear, albeit at a much faster rate than for the SHAKE algorithm.

Several variants of this approach based on sparse matrix techniques were studied by Barth et al..

The LINCS algorithm

An alternative constraint method, LINCS (Linear Constraint Solver) was developed in 1997 by Hess, Bekker, Berendsen and Fraaije, and was based on the 1986 method of Edberg, Evans and Morriss (EEM), and a modification thereof by Baranyai and Evans (BE).

LINCS applies Lagrange multipliers to the constraint forces and solves for the multipliers by using a series expansion to approximate the inverse of the Jacobian :

in each step of the Newton iteration. This approximation only works for matrices with Eigenvalues smaller than 1, making the LINCS algorithm suitable only for molecules with low connectivity.

LINCS has been reported to be 3-4 times faster than SHAKE.

Hybrid methods

Hybrid methods have also been introduced in which the constraints are divided into two groups; the constraints of the first group are solved using internal coordinates whereas those of the second group are solved using constraint forces, e.g., by a Lagrange multiplier or projection method. This approach was pioneered by Lagrange, and result in Lagrange equations of the mixed type.

Chapter 5

Compact Stencil, Courant–Friedrichs–Lewy Condition and Direct multiple Shooting Method

Compact stencil

A 2D compact stencil using all 8 adjacent nodes, plus the center node (in red)

In mathematics, especially in the areas of numerical analysis called numerical partial differential equations, a compact stencil is a type of stencil that uses only nine nodes for its discretization method in two dimensions. It uses only the center node and the adjacent nodes. For any structured grid utilizing a compact stencil in 1, 2, or 3 dimensions the maximum number of nodes is 3, 9, or 27 respectively. Compact stencils may be compared to non-compact stencils. Compact stencils are currently implemented in many partial differential equation solvers, including several in the topics of CFD, FEA, and other mathematical solvers relating to PDE's.

Two Point Stencil Example

The two point stencil for the first derivative of a function is given by:

.

This is obtained from the Taylor series expansion of the first derivative of the function given by:

.

Replacing h with − h, we have:

.

Addition of the above two equations together results in the cancellation of the terms in odd powers of h:

.

.

.

Three Point Stencil Example

For example, the three point stencil for the second derivative of a function is given by:

.

This is obtained from the Taylor series expansion of the first derivative of the function given by:

.

Replacing h with − h, we have:

.

Subtraction of the above two equations results in the cancellation of the terms in even

powers of h:

.

.

.

Courant–Friedrichs–Lewy condition In mathematics, the Courant–Friedrichs–Lewy condition (CFL condition) is a necessary condition for convergence while solving certain partial differential equations (usually hyperbolic PDEs) numerically. (It is not in general a sufficient condition.) It arises when explicit time-marching schemes are used for the numerical solution. As a consequence, the timestep must be less than a certain time in many explicit time-marching computer simulations, otherwise the simulation will produce wildly incorrect results. The condition is named after Richard Courant, Kurt Friedrichs, and Hans Lewy who described it in their 1928 paper.

For example, if a wave is crossing a discrete grid, then the timestep must be less than the time for the wave to travel adjacent grid points. As a corollary, when the grid point separation is reduced, the upper limit for the time step also decreases. In essence, the numerical domain of dependence must include the analytical domain of dependence in order to assure that the scheme can access the information required to form the solution.

The CFL condition is commonly prescribed for those terms in PDEs which represent advection (hyperbolic part of the PDE). For one-dimensional case, the CFL condition is given by

where

u is the velocity (L/T) is the time step (T) is the length interval (L),

and the constant C depends on the particular equation to be solved and not on Δt and Δx.

The number is called the Courant number.

In the two-dimensional case this becomes

The CFL condition can be a very limiting constraint on the time step Δt, to the extent that for certain fourth-order nonlinear partial differential equations it can be of the form

and efforts are often made to avoid it by using implicit methods.

Direct multiple shooting method In the area of mathematics known as numerical ordinary differential equations, the direct multiple shooting method is a numerical method for the solution of boundary value problems. The method divides the interval over which a solution is sought into several smaller intervals, solves an initial value problem in each of the smaller intervals, and imposes additional matching conditions to form a solution on the whole interval. The method constitutes a significant improvement in distribution of nonlinearity and numerical stability over single shooting methods.

Single shooting methods

Shooting methods can be used to solve boundary value problems (BVP) like

in which the time points ta and tb are known but the initial and terminal values ya and yb are unknown and sought for.

Single shooting methods proceed as follows. Let y(t; t0, y0) denote the solution of the initial value problem (IVP)

Define the function F(p) as the difference between y(tb; p) and the specified boundary value yb: F(p) = y(tb; p) − yb. Then for every solution (ya, yb) of the boundary value problem we have ya=y0 while yb corresponds to a root of F. This root be solved by any root-finding method given that certain method-dependent prerequisites are satisfied. This often will require initial guesses to ya and yb. Typically, analytic root finding is impossible and iterative methods such as Newton's method are used for this task.

The application of single shooting for the numerical solution of boundary value problems suffers from several drawbacks.

• For a given initial value y0 the solution of the IVP obviously must exist on the interval [ta,tb] so that we can evaluate the function F whose root is sought.

For highly nonlinear or unstable ODEs, this requires the initial guess y0 to be extremely close to an actual but unknown solution ya. Initial values that are chosen slightly off the true solution may lead to singularities or breakdown of the ODE solver method. Choosing such solutions is inevitable in an iterative root-finding method, however.

• Finite precision numerics may make it impossible at all to find initial values that allow for the solution of the ODE on the whole time interval.

• The nonlinearity of the ODE effectively becomes a nonlinearity of F, and requires a root-finding technique capable of solving nonlinear systems. Such methods typically converge slower as nonlinearities become more severe. The boundary value problem solver's performance suffers from this.

• Even stable and well-conditioned ODEs may make for unstable and ill-conditioned BVPs. A slight alteration of the initial value guess y0 may generate an extremely large step in the ODEs solution y(tb; ta, y0) and thus in the values of the function F whose root is sought. Non-analytic root-finding methods can seldom cope with this behaviour.

Multiple shooting

A direct multiple shooting method partitions the interval [ta, tb] by introducing additional grid points

.

The method starts by guessing somehow the values of y at all grid points tk with 0 ≤ k ≤ N − 1. Denote these guesses by yk. Let y(t; tk, yk) denote the solution emanating from the kth grid point, that is, the solution of the initial value problem

All these solutions can be pieced together to form a continuous trajectory if the values y match at the grid points. Thus, solutions of the boundary value problem correspond to solutions of the following system of N equations:

The central N−2 equations are the matching conditions, and the first and last equations are the conditions y(ta) = ya and y(tb) = yb from the boundary value problem. The multiple shooting method solves the boundary value problem by solving this system of equations. Typically, a modification of the Newton's method is used for the latter task.

Chapter 6

Crank–Nicolson Method

In numerical analysis, the Crank–Nicolson method is a finite difference method used for numerically solving the heat equation and similar partial differential equations. It is a second-order method in time, implicit in time, and is numerically stable. The method was developed by John Crank and Phyllis Nicolson in the mid 20th century.

For diffusion equations (and many other equations), it can be shown the Crank–Nicolson method is unconditionally stable. However, the approximate solutions can still contain (decaying) spurious oscillations if the ratio of time step to the square of space step is large (typically larger than 1/2). For this reason, whenever large time steps or high spatial resolution is necessary, the less accurate backward Euler method is often used, which is both stable and immune to oscillations.

The method

The Crank–Nicolson stencil for a 1D problem

The Crank–Nicolson method is based on central difference in space, and the trapezoidal rule in time, giving second-order convergence in time. For example, in one dimension, if the partial differential equation is

then, letting , the equation for Crank–Nicolson method is the average of that forward Euler method at n and that backward Euler method at n + 1 (note,

however, that the method itself is not simply the average of those two methods, as the equation has an implicit dependence on the solution):

The function F must be discretized spatially with a central difference.

Note that this is an implicit method: to get the "next" value of u in time, a system of algebraic equations must be solved. If the partial differential equation is nonlinear, the discretization will also be nonlinear so that advancing in time will involve the solution of a system of nonlinear algebraic equations, though linearizations are possible. In many problems, especially linear diffusion, the algebraic problem is tridiagonal and may be efficiently solved with the tridiagonal matrix algorithm, which gives a fast direct solution as opposed to the usual for a full matrix.

Example: 1D diffusion

The Crank–Nicolson method is often applied to diffusion problems. As an example, for linear diffusion,

whose Crank–Nicolson discretization is then:

or, letting :

which is a tridiagonal problem, so that may be efficiently solved by using the tridiagonal matrix algorithm in favor of a much more costly matrix inversion.

A quasilinear equation, such as (this is a minimalistic example and not general)

would lead to a nonlinear system of algebraic equations which could not be easily solved as above; however, it is possible in some cases to linearize the problem by using the old value for a, that is instead of . Other times, it may be possible to estimate using an explicit method and maintain stability.

Example: 1D diffusion with advection for steady flow, with multiple channel connections

This is a solution usually employed for many purposes when there's a contamination problem in streams or rivers under steady flow conditions but information is given in one dimension only. Often the problem can be simplified into a 1-dimensional problem and still yield useful information.

Here we model the concentration of a solute contaminant in water. This problem is composed of three parts: the known diffusion equation (Dx chosen as constant), an advective component (which means the system is evolving in space due to a velocity field), which we choose to be a constant Ux, and a lateral interaction between longitudinal channels (k).

where C is the concentration of the contaminant and subscripts N and M correspond to previous and next channel.

The Crank–Nicolson method (where i represents position and j time) transforms each component of the PDE into the following:

Now we create the following constants to simplify the algebra:

and substitute <1>, <2>, <3>, <4>, <5>, <6>, α, β and λ into <0>. We then put the new time terms on the left (j + 1) and the present time terms on the right (j) to get:

To model the first channel, we realize that it can only be in contact with the following channel (M), so the expression is simplified to:

In the same way, to model the last channel, we realize that it can only be in contact with the previous channel (N), so the expression is simplified to:

To solve this linear system of equations we must now see that boundary conditions must be given first to the beginning of the channels:

: initial condition for the channel at present time step : initial condition for the channel at next time step

: initial condition for the previous channel to the one analyzed at present time step

: initial condition for the next channel to the one analyzed at present time step

For the last cell of the channels (z) the most convenient condition becomes an adiabatic one, so

This condition is satisfied if and only if (regardless of a null value)

Let us solve this problem (in a matrix form) for the case of 3 channels and 5 nodes (including the initial boundary condition). We express this as a linear system problem:

where

and

Now we must realize that AA and BB should be arrays made of four different subarrays (remember that only three channels are considered for this example but it covers the main part discussed above).

and

where the elements mentioned above correspond to the next arrays and an additional 4x4 full of zeros. Please note that the sizes of AA and BB are 12x12:

,

,

,

&

The d vector here is used to hold the boundary conditions. In this example it is a 12x1 vector:

To find the concentration at any time, one must iterate the following equation:

Example: 2D diffusion

When extending into two dimensions on a uniform Cartesian grid, the derivation is similar and the results may lead to a system of band-diagonal equations rather than tridiagonal ones. The two-dimensional heat equation

can be solved with the Crank–Nicolson discretization of

assuming that a square grid is used so that Δx = Δy. This equation can be simplified somewhat by rearranging terms and using the CFL number

For the Crank–Nicolson numerical scheme, a low CFL number is not required for stability, however it is required for numerical accuracy. We can now write the scheme as:

Application in financial mathematics

Because a number of other phenomena can be modeled with the heat equation (often called the diffusion equation in financial mathematics), the Crank–Nicolson method has been applied to those areas as well. Particularly, the Black-Scholes option pricing model's differential equation can be transformed into the heat equation, and thus numerical solutions for option pricing can be obtained with the Crank–Nicolson method.

The importance of this for finance, is that option pricing problems, when extended beyond the standard assumptions (e.g. incorporating changing dividends), cannot be solved in closed form, but can be solved using this method. Note however, that for non-smooth final conditions (which happen for most financial instruments), the Crank–Nicolson method is not satisfactory as numerical oscillations are not damped. For vanilla options, this results in oscillation in the gamma value around the strike price. Therefore, special damping initialization steps are necessary (e.g., fully implicit finite difference method).

Chapter 7

Discrete Laplace Operator and Discrete Poisson Equation

Discrete Laplace operator In mathematics, the discrete Laplace operator is an analog of the continuous Laplace operator, defined so that it has meaning on a graph or a discrete grid. For the case of a finite-dimensional graph (having a finite number of edges and vertices), the discrete Laplace operator is more commonly called the Laplacian matrix.

The discrete Laplace operator occurs in physics problems such as the Ising model and loop quantum gravity, as well as in the study of discrete dynamical systems. It is also used in numerical analysis as a stand-in for the continuous Laplace operator. Common applications include image processing, where it is known as the Laplace filter, and in machine learning for clustering and semi-supervised learning on neighborhood graphs.

Definition

On Graphs

There are various definitions of the discrete Laplacian for graphs, differing by sign and scale factor (sometimes one averages over the neighboring vertices, other times one just sums; this makes no difference for a regular graph). The traditional definition of the graph Laplacian, given below, corresponds to the negative continuous Laplacian on a domain with a free boundary.

Let G = (V,E) be a graph with vertices and edges . Let be a ring-valued function of the vertices. Then, the discrete Laplacian Δ acting on φ is defined by

where d(w,v) is the graph distance between vertices w and v. Thus, this sum is over the nearest neighbors of the vertex v. For a graph with a finite number of edges and verticies, this definition is identical to that of the Laplacian matrix. That is, φ can be written as a

column vector; and so Δφ is the product of the column vector and the Laplacian matrix, while (Δφ)(v) is just the v'th entry of the product vector.

If the graph has weighted edges, that is, a weighting function is given, then the definition can be generalized to

where γwv is the weight value on the edge .

Closely related to the discrete Laplacian is the averaging operator:

Approximations of the continuous Laplacian

Approximations of the Laplacian, obtained by the finite difference method or by the finite element method can also be called Discrete Laplacians. For example, the Laplacian in two dimensions can be approximated using the five-point stencil finite difference method, resulting in

where the grid size is h in both dimensions, so that the five point stencil of a point (x, y) in the grid is

If the grid size h=1, the result is the negative discrete Laplacian on the graph, which is the square lattice grid. There are no constraints here on the values of the function f(x,y) on the boundary of the lattice grid, thus this is the case of the homogeneous Neumann boundary condition, i.e., free boundary. Other types of boundary conditions, e.g., the homogeneous Dirichlet boundary condition, where f(x,y)=0 on the boundary of the grid, are rarely used for graph Laplacians, but are common in other applications.

Multidimensional discrete Laplacians on rectangular cuboid regular grids have very special properties, e.g., they are Kronecker sums of one-dimensional discrete Laplacians, Kronecker sum of discrete Laplacians, in which case all its eigenvalues and eigenvectors can be explicitly calculated.

Image Processing

Discrete Laplace operator is often used in image processing e.g. in edge detection and motion estimation applications. The discrete Laplacian is defined as the sum of the second derivatives Laplace operator#Coordinate expressions and calculated as sum of differences over the nearest neighbours of the central pixel.

Implementation in Image Processing

For one, two and three dimensional signals, the discrete laplacian can be given as convolution with the following kernels:

1D-Filter:

2D-Filter:

or, including the diagonals:

2D-Filter:

3D-Filter: is given by: first plane = ; second plane =

; third plane =

These kernels are deduced by using discrete differential quotients.

Spectrum

The spectrum of the discrete Laplacian is of key interest; since it is a self-adjoint operator, it has a real spectrum. For the convention Δ = I − M, the spectrum lies within [0,2] (as the averaging operator has spectral values in [ − 1,1]) and contains 0 (for constant functions). The smallest non-zero eigenvalue is denoted λ1 and is called the spectral gap. There is also the notion of the spectral radius, commonly taken as the largest eigenvalue.

The eigenvectors don't depend on the convention above (for regular graphs), and are the same as for the averaging operator (as they differ by adding a multiple of the identity), though the eigenvalues differ according to the convention.

Theorems

If the graph is an infinite square lattice grid, then this definition of the Laplacian can be shown to correspond to the continuous Laplacian in the limit of an infinitely fine grid. Thus, for example, on a one-dimensional grid we have

This definition of the Laplacian is commonly used in numerical analysis and in image processing. In image processing, it is considered to be a type of digital filter, more specifically an edge filter, called the Laplace filter.

Discrete Schrödinger operator

Let be a potential function defined on the graph. Note that P can be considered to be a multiplicative operator acting diagonally on φ

Then H = Δ + P is the discrete Schrödinger operator, an analog of the continuous Schrödinger operator.

If the number of edges meeting at a vertex is uniformly bounded, and the potential is bounded, then H is bounded and self-adjoint.

The spectral properties of this Hamiltonian can be studied with Stone's theorem; this is a consequence of the duality between posets and Boolean algebras.

On regular lattices, the operator typically has both traveling-wave as well as Anderson localization solutions, depending on whether the potential is periodic or random.

Discrete Green's function

The Green's function of the discrete Schrödinger operator is given in the resolvent formalism by

where δw is understood to be the Kronecker delta function on the graph: δw(v) = δwv; that is, it equals 1 if v=w and 0 otherwise.

For fixed and λ a complex number, the Green's function considered to be a function of v is the unique solution to

ADE classification

Certain equations involving the discrete Laplacian only have solutions on the simply-laced Dynkin diagrams (all edges multiplicity 1), and are an example of the ADE classification. Specifically, the only positive solutions to the homogeneous equation:

Δφ = φ,

in words,

"Twice any label is the sum of the labels on adjacent vertices,"

are on the extended (affine) ADE Dynkin diagrams, of which there are 2 infinite families (A and D) and 3 exceptions (E). The resulting numbering is unique up to scale, and if the smallest value is set at 1, the other numbers are integers, ranging up to 6.

The ordinary ADE graphs are the only graphs that admit a positive labeling with the following property:

Twice any label minus two is the sum of the labels on adjacent vertices.

In terms of the Laplacian, the positive solutions to the inhomogeneous equation:

Δφ = φ − 2.

The resulting numbering is unique (scale is specified by the "2"), and consists of integers; for E8 they range from 58 to 270, and have been observed as early as (Bourbaki 1968).

Discrete Poisson equation In mathematics, the Discrete Poisson Equation is the finite difference analog of the Poisson equation. In it, the discrete Laplace operator takes the place of the Laplace operator. The discrete Poisson equation is frequently used in numerical analysis as a stand-in for the continuous Poisson equation, although it is also studied in its own right as a topic in discrete mathematics.

On a two-dimensional rectangular grid

Using the finite difference numerical method to discretize the 2 dimensional Poisson equation (assuming a uniform spatial discretization) on an m x n grid gives the following formula:

where and . The preferred arrangement of the solution vector is to use natural ordering which, prior to removing boundary elements, would look like:

This will result in an mn x mn linear system:

where

I is the m x m identity matrix, and D, also m x m , is given by:

For each uij equation, the columns of D correspond to the u components:

while the columns of I to the left and right of D correspond to the u components:

and

respectively.

From the above, it can be inferred that there are n block columns of m in A. It is important to note that prescribed values of u (usually lying on the boundary) would have their corresponding elements removed from I and D. For the common case that all the nodes on the boundary are set, we have and , and the system would have the dimensions (m - 2) (n - 2) x (m - 2) (n - 2) , where D and I would have dimensions (m-2) x (m-2) .

Example

For a 5×5 ( m = 5 and n = 5 ) grid with all the boundary nodes prescribed, the system would look like:

with

and

As can be seen, the boundary u's are brought to the right-hand-side of the equation. The entire system is 9 x 9 while D and I are 3 x 3 and given by:

and

Methods of Solution

Because is block tridiagonal and sparse, many methods of solution have been

developed to optimally solve this linear system for . Among the methods are a generalized Thomas algorithm, cyclic reduction, successive overrelaxation, and Fourier transforms. A theoretically optimal O(n) solution can be computed using multigrid methods.

Applications

In computational fluid dynamics, for the solution of an incompressible flow problem, the incompressibility condition acts as a constraint for the pressure. There is no explicit form available for pressure in this case due to a strong coupling of the velocity and pressure fields. In this condition, by taking the divergence of all terms in the momentum equation, one obtains the pressure poisson equation. For an incompressible flow this constraint is given by:

where vx is the velocity in the x direction, vy is velocity in y and vz is the velocity in the z direction. Taking divergence of the momentum equation and using the incompressibility constraint, the pressure poisson equation is formed given by:

where ν is the kinematic viscosity of the fluid and V is the velocity vector.

The discrete Poisson's equation arises in the theory of Markov chains. It appears as the relative value function for the dynamic programming equation in a Markov decision process, and as the control variate for application in simulation variance reduction.

Chapter 8

Euler Method

Illustration of the Euler method. The unknown curve is in blue, and its polygonal approximation is in red.

In mathematics and computational science, the Euler method, named after Leonhard Euler, is a first-order numerical procedure for solving ordinary differential equations (ODEs) with a given initial value. It is the most basic kind of explicit method for numerical integration of ordinary differential equations.

Informal geometrical description

Consider the problem of calculating the shape of an unknown curve which starts at a given point and satisfies a given differential equation. Here, a differential equation can be thought of as a formula by which the slope of the tangent line to the curve can be computed at any point on the curve, once the position of that point has been calculated.

The idea is that while the curve is initially unknown, its starting point, which we denote by A0, is known. Then, from the differential equation, the slope to the curve at A0 can be computed, and so, the tangent line.

Take a small step along that tangent line up to a point A1. If we pretend that A1 is still on the curve, the same reasoning as for the point A0 above can be used. After several steps, a polygonal curve is computed. In general, this curve does not diverge too far from the original unknown curve, and the error between the two curves can be made small if the step size is small enough and the interval of computation is finite.

Derivation

Illustration of numerical integration for the equation y' = y,y(0) = 1. Blue is the Euler method; green, the midpoint method; red, the exact solution, y = et. The step size is h = 1.0.

The same illustration for h = 0.25. It is seen that the midpoint method converges faster than the Euler method.

We want to approximate the solution of the initial value problem

by using the first two terms of the Taylor expansion of y, which represents the linear approximation around the point (t0,y(t0)) . One step of the Euler method from tn to tn+1 = tn + h is

The Euler method is explicit, i.e. the solution yn + 1 is an explicit function of yi for .

While the Euler method integrates a first order ODE, any ODE of order N can be represented as a first-order ODE in more than one variable by introducing N − 1 further variables, y', y'', ..., y(N), and formulating N first order equations in these new variables. The Euler method can be applied to the vector

to integrate the higher-order system.

Example

Given the differential equation y' = y and the initial point y(0) = 1, we would like to use the Euler method to approximate y3 using step size h = 1.

The Euler method is

so first we must compute f(t0,y0). This simple differential equation depends only on y, so we need only worry about inputting the values for y.

By doing the above step, we have found the slope of the line that is tangent to the solution curve at the point (0,1). Recall that the slope is defined as the change in y divided by the change in t, or .

The next step is to multiply the above value by the step size h.

.

Since the step size is the change in t, when we multiply the step size and the slope of the tangent, we get a change in y value. This value is then added to the initial y value to obtain the next value to be used for computations.

.

The above steps should be repeated to find y2 and y3.

Due to the repetitive nature of this algorithm, it can be helpful to organize computations in a chart form, as seen below, to avoid making errors.

yn tn y'(t) h dy yn + 1 1 0 1 1 1 2 2 1 2 1 2 4 4 2 4 1 4 8

Error

The magnitude of the errors arising from the Euler method can be demonstrated by comparison with a Taylor expansion of y. If we assume that f(t) and y(t) are known exactly at a time t0, then the Euler method gives the approximate solution at time t0 + h as:

In comparison, the Taylor expansion in h about t0 gives:

Since we know that y' = f(t,y) it follows that

This, along with f(t0,y(t0)) can be inserted into the Taylor expansion in h about t0.

The error introduced by the Euler method is given by the difference between these equations:

For small h, the dominant error per step, or the local truncation error, is proportional to h2. To solve the problem over a given range of t, the number of steps needed is proportional to 1 / h so it is to be expected that the total error at the end of the fixed time, or the global truncation error, will be proportional to h (error per step times number of steps). Because the global truncation error is proportional to h, the Euler method is said to be first order. This makes the Euler method less accurate (for small h) than other higher-order techniques such as Runge-Kutta methods and linear multistep methods.

As stated in the introduction, decreasing the step size can help to make the approximation more accurate and decrease the error between the two curves. The error to three decimal places for the example in the above section (with step size h = 1)is the following:

When the step size is changed to h = .1 our y3 value becomes 17.449. The error, then, for step size h = .1 is the following:

Although the error has decreased, our approximation is still not particularly accurate. In addition, since the step size decreased with no change in the interval, the number of iterations has increased to thirty. While possible, it is no longer reasonable to do these computations by hand.

Error bound

As with other methods, there is a way for us to determine an error bound for a particular problem. The error bound on the global error is given by:

where h is the step size, M is the upper bound on the second derivative of y on the given interval (which must be estimated), and L is the Lipschitz constant.

If the error bound is computed, it can be seen, once again, that if small error is desired, the step size h must be very small.

For instance, let us calculate the step size required for global truncation error to be εn + 1 = .01, assuming a maximum value for the second derivative of M = 10, a Lipschitz constant of L = 1, and t from zero to four. Using the equation given, we obtain the following:

which means that h must be smaller than the above to get the desired error or less, and 4 / h, or about 1072 iterations will need to be completed to do so. The large number of steps, and thus high computation cost, supports the use of alternative, higher-order methods such as Runge-Kutta methods or linear multistep methods

Numerical stability

The Euler method can also be numerically unstable, especially for stiff equations. This limitation—along with its slow convergence of error with h—means that the Euler method is not often used, except as a simple example of numerical integration. The instability can be avoided by using the Euler-Cromer algorithm.

Chapter 9

Finite Difference

A finite difference is a mathematical expression of the form f(x + b) − f(x + a). If a finite difference is divided by b − a, one gets a difference quotient. The approximation of derivatives by finite differences plays a central role in finite difference methods for the numerical solution of differential equations, especially boundary value problems.

Recurrence relations can be written as difference equations by replacing iteration notation with finite differences.

Forward, backward, and central differences

Only three forms are commonly considered: forward, backward, and central differences.

A forward difference is an expression of the form

Depending on the application, the spacing h may be variable or constant.

A backward difference uses the function values at x and x − h, instead of the values at x + h and x:

Finally, the central difference is given by

Relation with derivatives

The derivative of a function f at a point x is defined by the limit

If h has a fixed (non-zero) value, instead of approaching zero, then the right-hand side is

Hence, the forward difference divided by h approximates the derivative when h is small. The error in this approximation can be derived from Taylor's theorem. Assuming that f is continuously differentiable, the error is

The same formula holds for the backward difference:

However, the central difference yields a more accurate approximation. Its error is proportional to square of the spacing (if f is twice continuously differentiable):

The main problem with the central difference method, however, is that oscillating functions can yield zero derivative. If f(nh)=1 for n uneven, and f(nh)=2 for n even, then f'(nh)=0 if it is calculated with the central difference scheme. This is particularly troublesome if the domain of f is discrete.

Higher-order differences

In an analogous way one can obtain finite difference approximations to higher order derivatives and differential operators. For example, by using the above central difference formula for f'(x + h / 2) and f'(x − h / 2) and applying a central difference formula for the derivative of f' at x, we obtain the central difference approximation of the second derivative of f:

2nd Order Central

Similarly we can apply other differencing formulas in a recursive manner. 2nd Order Forward

More generally, the nth-order forward, backward, and central differences are respectively given by:

Note that the central difference will, for odd n, have h multiplied by non-integers. This is often a problem because it amounts to changing the interval of discretization. The problem may be remedied taking the average of δn[f](x − h / 2) and δn[f](x + h / 2).

The relationship of these higher-order differences with the respective derivatives is very straightforward:

Higher-order differences can also be used to construct better approximations. As mentioned above, the first-order difference approximates the first-order derivative up to a term of order h. However, the combination

approximates f'(x) up to a term of order h2. This can be proven by expanding the above expression in Taylor series, or by using the calculus of finite differences, explained below.

If necessary, the finite difference can be centered about any point by mixing forward, backward, and central differences.

Arbitrarily sized kernels

Using a little linear algebra, one can fairly easily construct approximations, which sample an arbitrary number of points to the left and a (possibly different) number of points to the right of the center point, for any order of derivative. This involves solving a linear system such that the Taylor expansion of the sum of those points, around the center point, well approximates the Taylor expansion of the desired derivative.

This is useful for differentiating a function on a grid, where, as one approaches the edge of the grid, one must sample fewer and fewer points on one side.

The details are outlined in these notes.

Properties

• For all positive k and n

• Leibniz rule:

Finite difference methods

An important application of finite differences is in numerical analysis, especially in numerical differential equations, which aim at the numerical solution of ordinary and partial differential equations respectively. The idea is to replace the derivatives appearing in the differential equation by finite differences that approximate them. The resulting methods are called finite difference methods.

Common applications of the finite difference method are in computational science and engineering disciplines, such as thermal engineering, fluid mechanics, etc.

Calculus of finite differences

The forward difference can be considered as a difference operator, which maps the function f to Δh[f]. This operator satisfies

where Th is the shift operator with step h, defined by Th[f](x) = f(x + h), and I is an identity operator.

Finite difference of higher orders can be defined in recursive manner as or, in operators notation,

Another possible (and equivalent) definition is

The difference operator Δh is linear and satisfies Leibniz rule. Similar statements hold for the backward and central difference.

Formally applying the Taylor series with respect to h gives the formula

where D denotes the derivative operator, mapping f to its derivative f'. The expansion is valid when both sides act on analytic function, for sufficiently small h. Formally inverting the exponential suggests that

This formula holds in the sense that both operators give the same result when applied to a polynomial. Even for analytic functions, the series on the right is not guaranteed to converge; it may be an asymptotic series. However, it can be used to obtain more accurate approximations for the derivative. For instance, retaining the first two terms of the series yields the second-order approximation to f'(x) mentioned at the end of the section Higher-order differences.

The analogous formulas for the backward and central difference operators are

The calculus of finite differences is related to the umbral calculus in combinatorics.

The inverse operator of the forward difference operator is the indefinite sum.

In mathematics, a difference operator maps a function, ƒ(x), to another function, ƒ(x + b) − ƒ(x + a).

The forward difference operator

occurs frequently in the calculus of finite differences, where it plays a role formally similar to that of the derivative, but used in discrete circumstances. Difference equations can often be solved with techniques very similar to those for solving differential equations. This similarity led to the development of time scale calculus. Analogously we can have the backward difference operator

When restricted to polynomial functions f, the forward difference operator is a delta operator, i.e., a shift-equivariant linear operator on polynomials that reduces degree by 1.

n-th difference

The nth forward difference of a function f(x) is given by

where is the binomial coefficient. Forward differences applied to a sequence are sometimes called the binomial transform of the sequence, and have a number of interesting combinatorial properties.

Forward differences may be evaluated using the Nörlund–Rice integral. The integral representation for these types of series is interesting because the integral can often be evaluated using asymptotic expansion or saddle-point techniques; by contrast, the forward difference series can be extremely hard to evaluate numerically, because the binomial coefficients grow rapidly for large n.

Newton series

The Newton series consists of the terms of the Newton forward difference equation, named after Isaac Newton and in essence the Newton interpolation formula first published in his Principia Mathematica in 1687, is the relationship

which holds for any polynomial function f and for some, but not all, analytic functions. Here, the expression

is the binomial coefficient, as

is the "falling factorial" or "lower factorial" and the empty product (x)0 defined to be 1. In this particular case there is an assumption of unit steps for the changes in the values of x. Note also the formal similarity of this result to Taylor's theorem; this is one of the observations that lead to the idea of umbral calculus.

To illustrate how one might use Newton's formula in actual practice consider the first few terms of the Fibonacci sequence f = 2, 2, 4... One can find a polynomial that reproduces these values by first computing a difference table and then substituting the differences which correspond to x0 (underlined) into the formula as follows,

For the case of nonuniform steps in the values of x Newton computes the divided differences,

the series of products,

and the resulting polynomial is the scalar product,

.

In analysis with p-adic numbers, Mahler's theorem states that the assumption that f is a polynomial function can be weakened all the way to the assumption that f is merely continuous.

Carlson's theorem provides necessary and sufficient conditions for a Newton series to be unique, if it exists. However, a Newton series will not, in general, exist.

The Newton series, together with the Stirling series and the Selberg series, is a special case of the general difference series, all of which are defined in terms of scaled forward differences.

Rules for calculus of finite difference operators

Analogous to rules for finding the derivative, we have:

• Constant rule: If c is a constant, then

• Linearity: if a and b are constants,

All of the above rules apply equally well to any difference operator, including as to Δ.

• Product rule:

• Quotient rule:

or

• Summation rules:

Indefinite sum

The inverse operator of the forward difference operator is the indefinite sum.

Generalizations

• A generalized finite difference is usually defined as

where is its coefficients vector. An infinite difference is a further generalization, where the finite sum above is replaced by an infinite series. Another way of generalization is making coefficients μk depend on point x : μk = μk(x), thus considering weighted finite difference. Also one may make step h depend on point x : h = h(x). Such generalizations are useful for constructing different modulus of continuity.

• Difference operator generalizes to Möbius inversion over a partially ordered set.

• As a convolution operator: Via the formalism of incidence algebras, difference operators and other Möbius inversion can be represented by convolution with a function on the poset, called the Möbius function μ; for the difference operator, μ is the sequence (1, −1, 0, 0, 0, ...).

Finite difference in several variables

Finite differences can be considered in more than one variable. They are analogous to partial derivatives in several variables.

Some partial derivative approximations are:

Chapter 10

Finite Difference Method

In mathematics, finite-difference methods are numerical methods for approximating the solutions to differential equations using finite difference equations to approximate derivatives.

Intuitive derivation

Finite-difference methods approximate the solutions to differential equations by replacing derivative expressions with approximately equivalent difference quotients. That is, because the first derivative of a function f is, by definition,

then a reasonable approximation for that derivative would be to take

for some small value of h. In fact, this is the forward difference equation for the first derivative. Using this and similar formulae to replace derivative expressions in differential equations, one can approximate their solutions without the need for calculus.

Derivation from Taylor's polynomial

Assuming the function whose derivatives are to be approximated is properly-behaved, by Taylor's theorem,

where n! denotes the factorial of n, and Rn(x) is a remainder term, denoting the difference between the Taylor polynomial of degree n and the original function. Again using the first derivative of the function f as an example, by Taylor's theorem,

f(x0 + h) = f(x0) + f'(x0)h + R1(x),

which, with some minor algebraic manipulation, is equivalent to

so that for R1(x) sufficiently small,

Accuracy and order

The error in a method's solution is defined as the difference between its approximation and the exact analytical solution. The two sources of error in finite difference methods are round-off error, the loss of precision due to computer rounding of decimal quantities, and truncation error or discretization error, the difference between the exact solution of the finite difference equation and the exact quantity assuming perfect arithmetic (that is, assuming no round-off).

The finite difference method relies on discretizing a function on a grid

To use a finite difference method to attempt to solve (or, more generally, approximate the solution to) a problem, one must first discretize the problem's domain. This is usually done by dividing the domain into a uniform grid. Note that this means that finite-difference methods produce sets of discrete numerical approximations to the derivative, often in a "time-stepping" manner.

An expression of general interest is the local truncation error of a method. Typically expressed using Big-O notation, local truncation error refers to the error from a single application of a method. That is, it is the quantity f'(xi) − f'i if f'(xi) refers to the exact value and f'i to the numerical approximation. The remainder term of a Taylor polynomial is convenient for analyzing the local truncation error. Using the Lagrange form of the remainder from the Taylor polynomial for f(x0 + h), which is

, where x0 < ξ < x0 + h,

the dominant term of the local truncation error can be discovered. For example, again using the forward-difference formula for the first derivative, knowing that f(xi) = f(x0 + ih),

and with some algebraic manipulation, this leads to

and further noting that the quantity on the left is the approximation from the finite difference method and that the quantity on the right is the exact quantity of interest plus a remainder, clearly that remainder is the local truncation error. A final expression of this example and its order is:

This means that, in this case, the local truncation error is proportional to the step size.

Example: ordinary differential equation

For example, consider the ordinary differential equation

The Euler method for solving this equation uses the finite difference quotient

to approximate the differential equation by first substituting in for u'(x) and applying a little algebra to get

The last equation is a finite-difference equation, and solving this equation gives an approximate solution to the differential equation.

Example: The heat equation

Consider the normalized heat equation in one dimension, with homogeneous Dirichlet boundary conditions

(boundary condition)

(initial condition)

One way to numerically solve this equation is to approximate all the derivatives by finite differences. We partition the domain in space using a mesh x0,...,xJ and in time using a mesh t0,....,tN. We assume a uniform partition both in space and in time, so the difference between two consecutive space points will be h and between two consecutive time points will be k. The points

will represent the numerical approximation of u(xj,tn).

Explicit method

The stencil for the most common explicit method for the heat equation

Using a forward difference at time tn and a second-order central difference for the space derivative at position xj ("FTCS") we get the recurrence equation:

This is an explicit method for solving the one-dimensional heat equation.

We can obtain from the other values this way:

where r = k / h2.

So, with this recurrence relation, and knowing the values at time n, one can obtain the corresponding values at time n+1. and must be replaced by the boundary conditions, in this example they are both 0.

This explicit method is known to be numerically stable and convergent whenever . The numerical errors are proportional to the time step and the square of the

space step:

Implicit method

The implicit method stencil

If we use the backward difference at time tn + 1 and a second-order central difference for the space derivative at position xj ("BTCS") we get the recurrence equation:

This is an implicit method for solving the one-dimensional heat equation.

We can obtain from solving a system of linear equations:

The scheme is always numerically stable and convergent but usually more numerically intensive than the explicit method as it requires solving a system of numerical equations on each time step. The errors are linear over the time step and quadratic over the space step.

Crank–Nicolson method

Finally if we use the central difference at time tn + 1 / 2 and a second-order central difference for the space derivative at position xj ("CTCS") we get the recurrence equation:

This formula is known as the Crank–Nicolson method.

The Crank–Nicolson stencil

We can obtain from solving a system of linear equations:

The scheme is always numerically stable and convergent but usually more numerically intensive as it requires solving a system of numerical equations on each time step. The errors are quadratic over the time step and formally are of the fourth degree regarding the space step:

However, near the boundaries, the error is often O(h2) instead of O(h4).

Usually the Crank–Nicolson scheme is the most accurate scheme for small time steps. The explicit scheme is the least accurate and can be unstable, but is also the easiest to implement and the least numerically intensive. The implicit scheme works the best for large time steps.

Chapter 11

Finite Element Method

2D FEM solution for a magnetostatic configuration (lines denote the direction and colour the magnitude of calculated flux density)

2D mesh for the image above (mesh is denser around the object of interest)

The finite element method (FEM) (its practical application often known as finite element analysis (FEA)) is a numerical technique for finding approximate solutions of partial differential equations (PDE) as well as of integral equations. The solution approach is based either on eliminating the differential equation completely (steady state problems), or rendering the PDE into an approximating system of ordinary differential equations, which are then numerically integrated using standard techniques such as Euler's method, Runge-Kutta, etc.

In solving partial differential equations, the primary challenge is to create an equation that approximates the equation to be studied, but is numerically stable, meaning that errors in the input and intermediate calculations do not accumulate and cause the resulting output to be meaningless. There are many ways of doing this, all with advantages and disadvantages. The Finite Element Method is a good choice for solving partial differential equations over complicated domains (like cars and oil pipelines), when the domain changes (as during a solid state reaction with a moving boundary), when the desired precision varies over the entire domain, or when the solution lacks smoothness. For instance, in a frontal crash simulation it is possible to increase prediction accuracy in

"important" areas like the front of the car and reduce it in its rear (thus reducing cost of the simulation); another example would be the simulation of the weather pattern on Earth, where it is more important to have accurate predictions over land than over the wide-open sea.

History

The finite element method originated from the need for solving complex elasticity and structural analysis problems in civil and aeronautical engineering. Its development can be traced back to the work by Alexander Hrennikoff (1941) and Richard Courant (1942). While the approaches used by these pioneers are different, they share one essential characteristic: mesh discretization of a continuous domain into a set of discrete sub-domains, usually called elements. Starting in 1947, Olgierd Zienkiewicz from Imperial College gathered those methods together into what would be called the Finite Element Method, building the pioneering mathematical formalism of the method.

Hrennikoff's work discretizes the domain by using a lattice analogy, while Courant's approach divides the domain into finite triangular subregions to solve second order elliptic partial differential equations (PDEs) that arise from the problem of torsion of a cylinder. Courant's contribution was evolutionary, drawing on a large body of earlier results for PDEs developed by Rayleigh, Ritz, and Galerkin.

Development of the finite element method began in earnest in the middle to late 1950s for airframe and structural analysis and gathered momentum at the University of Stuttgart through the work of John Argyris and at Berkeley through the work of Ray W. Clough in the 1960s for use in civil engineering. By late 1950s, the key concepts of stiffness matrix and element assembly existed essentially in the form used today. NASA issued a request for proposals for the development of the finite element software NASTRAN in 1965. The method was again provided with a rigorous mathematical foundation in 1973 with the publication of Strang and Fix's An Analysis of The Finite Element Method, and has since been generalized into a branch of applied mathematics for numerical modeling of physical systems in a wide variety of engineering disciplines, e.g., electromagnetism, thanks to Peter P. Silvester and fluid dynamics.

Application

Visualization of how a car deforms in an asymmetrical crash using finite element analysis.

A variety of specializations under the umbrella of the mechanical engineering discipline (such as aeronautical, biomechanical, and automotive industries) commonly use integrated FEM in design and development of their products. Several modern FEM packages include specific components such as thermal, electromagnetic, fluid, and structural working environments. In a structural simulation, FEM helps tremendously in producing stiffness and strength visualizations and also in minimizing weight, materials, and costs.

FEM allows detailed visualization of where structures bend or twist, and indicates the distribution of stresses and displacements. FEM software provides a wide range of simulation options for controlling the complexity of both modeling and analysis of a system. Similarly, the desired level of accuracy required and associated computational time requirements can be managed simultaneously to address most engineering applications. FEM allows entire designs to be constructed, refined, and optimized before the design is manufactured.

This powerful design tool has significantly improved both the standard of engineering designs and the methodology of the design process in many industrial applications. The

introduction of FEM has substantially decreased the time to take products from concept to the production line. It is primarily through improved initial prototype designs using FEM that testing and development have been accelerated. In summary, benefits of FEM include increased accuracy, enhanced design and better insight into critical design parameters, virtual prototyping, fewer hardware prototypes, a faster and less expensive design cycle, increased productivity, and increased revenue.

Technical discussion

We will illustrate the finite element method using two sample problems from which the general method can be extrapolated. It is assumed that the reader is familiar with calculus and linear algebra.

P1 is a one-dimensional problem

where f is given, u is an unknown function of x, and u'' is the second derivative of u with respect to x.

The two-dimensional sample problem is the Dirichlet problem

where Ω is a connected open region in the (x,y) plane whose boundary is "nice" (e.g., a smooth manifold or a polygon), and uxx and uyy denote the second derivatives with respect to x and y, respectively.

The problem P1 can be solved "directly" by computing antiderivatives. However, this method of solving the boundary value problem works only when there is only one spatial dimension and does not generalize to higher-dimensional problems or to problems like u + u'' = f. For this reason, we will develop the finite element method for P1 and outline its generalization to P2.

Our explanation will proceed in two steps, which mirror two essential steps one must take to solve a boundary value problem (BVP) using the FEM.

• In the first step, one rephrases the original BVP in its weak form. Little to no computation is usually required for this step. The transformation is done by hand on paper.

• The second step is the discretization, where the weak form is discretized in a finite dimensional space.

After this second step, we have concrete formulae for a large but finite dimensional linear problem whose solution will approximately solve the original BVP. This finite dimensional problem is then implemented on a computer.

Weak formulation

The first step is to convert P1 and P2 into their equivalents weak formulation. If u solves P1, then for any smooth function v that satisfies the displacement boundary conditions, i.e. v = 0 at x = 0 and x = 1,we have

(1)

Conversely, if u with u(0) = u(1) = 0 satisfies (1) for every smooth function v(x) then one may show that this u will solve P1. The proof is easier for twice continuously differentiable u (mean value theorem), but may be proved in a distributional sense as well.

By using integration by parts on the right-hand-side of (1), we obtain

(2)

where we have used the assumption that v(0) = v(1) = 0.

A proof outline of existence and uniqueness of the solution

We can loosely think of to be the absolutely continuous functions of (0,1) that are 0 at x = 0 and x = 1. Such function are (weakly) "once differentiable" and it turns out that the symmetric bilinear map then defines an inner product which turns

into a Hilbert space (a detailed proof is nontrivial.) On the other hand, the left-

hand-side is also an inner product, this time on the Lp space L2(0,1). An application of the Riesz representation theorem for Hilbert spaces shows that there is a unique u solving (2) and therefore P1. This solution is a-priori only a member of

, but using elliptic regularity, will be smooth if f is.

The weak form of P2

If we integrate by parts using a form of Green's identities, we see that if u solves P2, then for any v,

where denotes the gradient and denotes the dot product in the two-dimensional plane. Once more can be turned into an inner product on a suitable space of "once differentiable" functions of Ω that are zero on . We have also assumed that

. Existence and uniqueness of the solution can also be shown.

Discretization

A function in H1

0, with zero values at the endpoints (blue), and a piecewise linear approximation (red).

The basic idea is to replace the infinite dimensional linear problem:

Find such that

with a finite dimensional version:

(3) Find such that

where V is a finite dimensional subspace of . There are many possible choices for V (one possibility leads to the spectral method). However, for the finite element method we take V to be a space of piecewise polynomial functions.

For problem P1, we take the interval (0,1), choose n values of x with 0 = x0 < x1 < ... < xn < xn + 1 = 1 and we define V by

where we define x0 = 0 and xn + 1 = 1. Observe that functions in V are not differentiable according to the elementary definition of calculus. Indeed, if then the derivative is typically not defined at any x = xk, k = 1,...,n. However, the derivative exists at every other value of x and one can use this derivative for the purpose of integration by parts.

A piecewise linear function in two dimensions

For problem P2, we need V to be a set of functions of Ω. In the figure on the right, we have illustrated a triangulation of a 15 sided polygonal region Ω in the plane (below), and

a piecewise linear function (above, in color) of this polygon which is linear on each triangle of the triangulation; the space V would consist of functions that are linear on each triangle of the chosen triangulation.

One often reads Vh instead of V in the literature. The reason is that one hopes that as the underlying triangular grid becomes finer and finer, the solution of the discrete problem (3) will in some sense converge to the solution of the original boundary value problem P2. The triangulation is then indexed by a real valued parameter h > 0 which one takes to be very small. This parameter will be related to the size of the largest or average triangle in the triangulation. As we refine the triangulation, the space of piecewise linear functions V must also change with h, hence the notation Vh. Since we do not perform such an analysis, we will not use this notation.

Choosing a basis

Basis functions vk (blue) and a linear combination of them, which is piecewise linear (red).

To complete the discretization, we must select a basis of V. In the one-dimensional case, for each control point xk we will choose the piecewise linear function vk in V whose value is 1 at xk and zero at every , i.e.,

for k = 1,...,n; this basis is a shifted and scaled tent function. For the two-dimensional case, we choose again one basis function vk per vertex xk of the triangulation of the planar region Ω. The function vk is the unique function of V whose value is 1 at xk and zero at every .

Depending on the author, the word "element" in "finite element method" refers either to the triangles in the domain, the piecewise linear basis function, or both. So for instance, an author interested in curved domains might replace the triangles with curved primitives, and so might describe the elements as being curvilinear. On the other hand, some authors replace "piecewise linear" by "piecewise quadratic" or even "piecewise polynomial". The author might then say "higher order element" instead of "higher degree polynomial". Finite element method is not restricted to triangles (or tetrahedra in 3-d, or higher order simplexes in multidimensional spaces), but can be defined on quadrilateral subdomains (hexahedra, prisms, or pyramids in 3-d, and so on). Higher order shapes (curvilinear elements) can be defined with polynomial and even non-polynomial shapes (e.g. ellipse or circle).

Examples of methods that use higher degree piecewise polynomial basis functions are the hp-FEM and spectral FEM.

More advanced implementations (adaptive finite element methods) utilize a method to assess the quality of the results (based on error estimation theory) and modify the mesh during the solution aiming to achieve approximate solution within some bounds from the 'exact' solution of the continuum problem. Mesh adaptivity may utilize various techniques, the most popular are:

• moving nodes (r-adaptivity) • refining (and unrefining) elements (h-adaptivity) • changing order of base functions (p-adaptivity) • combinations of the above (hp-adaptivity)

Small support of the basis

Solving the two-dimensional problem uxx + uyy = − 4 in the disk centered at the origin and radius 1, with zero boundary conditions. (a) The triangulation.

(b) The sparse matrix L of the discretized linear system.

(c) The computed solution, u(x,y) = 1 − x2 − y2.

The primary advantage of this choice of basis is that the inner products

and

will be zero for almost all j,k. (The matrix containing in the (j,k) location is known as the Gramian matrix.) In the one dimensional case, the support of vk is the interval [xk − 1,xk + 1]. Hence, the integrands of and φ(vj,vk) are identically zero whenever | j − k | > 1.

Similarly, in the planar case, if xj and xk do not share an edge of the triangulation, then the integrals

and

are both zero.

Matrix form of the problem

If we write and then problem (3), taking v(x) = vj(x) for j = 1,...,n, becomes

for j = 1,...,n. (4)

If we denote by and the column vectors (u1,...,un)t and (f1,...,fn)t, and if we let

L = (Lij)

and

M = (Mij)

be matrices whose entries are

Lij = φ(vi,vj)

and

then we may rephrase (4) as

. (5)

It is not, in fact, necessary to assume . For a general function f(x), problem (3) with v(x) = vj(x) for j = 1,...,n becomes actually simpler, since no matrix M is used,

, (6)

where and for j = 1,...,n.

As we have discussed before, most of the entries of L and M are zero because the basis functions vk have small support. So we now have to solve a linear system in the unknown

where most of the entries of the matrix L, which we need to invert, are zero.

Such matrices are known as sparse matrices, and there are efficient solvers for such problems (much more efficient than actually inverting the matrix.) In addition, L is symmetric and positive definite, so a technique such as the conjugate gradient method is favored. For problems that are not too large, sparse LU decompositions and Cholesky decompositions still work well. For instance, Matlab's backslash operator (which uses sparse LU, sparse Cholesky, and other factorization methods) can be sufficient for meshes with a hundred thousand vertices.

The matrix L is usually referred to as the stiffness matrix, while the matrix M is dubbed the mass matrix.

General form of the finite element method

In general, the finite element method is characterized by the following process.

• One chooses a grid for Ω. In the preceding treatment, the grid consisted of triangles, but one can also use squares or curvilinear polygons.

• Then, one chooses basis functions. In our discussion, we used piecewise linear basis functions, but it is also common to use piecewise polynomial basis functions.

A separate consideration is the smoothness of the basis functions. For second order elliptic boundary value problems, piecewise polynomial basis function that are merely continuous suffice (i.e., the derivatives are discontinuous.) For higher order partial differential equations, one must use smoother basis functions. For instance, for a fourth order problem such as uxxxx + uyyyy = f, one may use piecewise quadratic basis functions that are C1.

Another consideration is the relation of the finite dimensional space V to its infinite dimensional counterpart, in the examples above . A conforming element method is one in which the space V is a subspace of the element space for the continuous problem. The example above is such a method. If this condition is not satisfied, we obtain a nonconforming element method, an example of which is the space of piecewise linear functions over the mesh which are continuous at each edge midpoint. Since these functions are in general discontinuous along the edges, this finite dimensional space is not a subspace of the original .

Typically, one has an algorithm for taking a given mesh and subdividing it. If the main method for increasing precision is to subdivide the mesh, one has an h-method (h is customarily the diameter of the largest element in the mesh.) In this manner, if one shows that the error with a grid h is bounded above by Chp, for some and p > 0, then one has an order p method. Under certain hypotheses (for instance, if the domain is convex), a piecewise polynomial of order d method will have an error of order p = d + 1.

If instead of making h smaller, one increases the degree of the polynomials used in the basis function, one has a p-method. If one combines these two refinement types, one obtains an hp-method (hp-FEM). In the hp-FEM, the polynomial degrees can vary from element to element. High order methods with large uniform p are called spectral finite element methods (SFEM). These are not to be confused with spectral methods.

For vector partial differential equations, the basis functions may take values in .

Comparison to the finite difference method

The finite difference method (FDM) is an alternative way of approximating solutions of PDEs. The differences between FEM and FDM are:

• The most attractive feature of the FEM is its ability to handle complicated geometries (and boundaries) with relative ease. While FDM in its basic form is

restricted to handle rectangular shapes and simple alterations thereof, the handling of geometries in FEM is theoretically straightforward.

• The most attractive feature of finite differences is that it can be very easy to implement.

• There are several ways one could consider the FDM a special case of the FEM approach. One might choose basis functions as either piecewise constant functions or Dirac delta functions. In both approaches, the approximations are defined on the entire domain, but need not be continuous. Alternatively, one might define the function on a discrete domain, with the result that the continuous differential operator no longer makes sense, however this approach is not FEM.

• There are reasons to consider the mathematical foundation of the finite element approximation more sound, for instance, because the quality of the approximation between grid points is poor in FDM.

• The quality of a FEM approximation is often higher than in the corresponding FDM approach, but this is extremely problem dependent and several examples to the contrary can be provided.

Generally, FEM is the method of choice in all types of analysis in structural mechanics (i.e. solving for deformation and stresses in solid bodies or dynamics of structures) while computational fluid dynamics (CFD) tends to use FDM or other methods like finite volume method (FVM). CFD problems usually require discretization of the problem into a large number of cells/gridpoints (millions and more), therefore cost of the solution favors simpler, lower order approximation within each cell. This is especially true for 'external flow' problems, like air flow around the car or airplane, or weather simulation in a large area.

Various types of finite element methods

Generalized finite element method

The Generalized Finite Element Method (GFEM) uses local spaces consisting of functions, not necessarily polynomials, that reflect the available information on the unknown solution and thus ensure good local approximation. Then a partition of unity is used to “bond” these spaces together to form the approximating subspace. The effectiveness of GFEM has been shown when applied to problems with domains having complicated boundaries, problems with micro-scales, and problems with boundary layers.

hp-FEM

The hp-FEM combines adaptively elements with variable size h and polynomial degree p in order to achieve exceptionally fast, exponential convergence rates.

hpk-FEM

The hpk-FEM combines adaptively elements with variable size h, polynomial degree of the local approximations p and global differentiability of the local approximations (k-1) in order to achieve best convergence rates.

Other applications of finite elements analysis

FEA has also been proposed to use in stochastic modelling, for numerically solving probability models.

Chapter 12

Bramble-Hilbert Lemma and Spectral Element Method

Bramble-Hilbert lemma In mathematics, particularly numerical analysis, the Bramble-Hilbert lemma, named after James H. Bramble and Stephen R. Hilbert, bounds the error of an approximation of a function by a polynomial of order at most in terms of derivatives of of order

. Both the error of the approximation and the derivatives of are measured by norms on a bounded domain in . This is similar to classical numerical analysis,

where, for example, the error of linear interpolation can be bounded using the second derivative of . However, the Bramble-Hilbert lemma applies in any number of dimensions, not just one dimension, and the approximation error and the derivatives of

are measured by more general norms involving averages, not just the maximum norm.

Additional assumptions on the domain are needed for the Bramble-Hilbert lemma to hold. Essentially, the boundary of the domain must be "reasonable". For example, domains that have a spike or a slit with zero angle at the tip are excluded. Lipschitz domains are reasonable enough, which includes convex domains and domains with continuously differentiable boundary.

The main use of the Bramble-Hilbert lemma is to prove bounds on the error of interpolation of function by an operator that preserves polynomials of order up to

, in terms of the derivatives of of order . This is an essential step in error estimates for the finite element method. The Bramble-Hilbert lemma is applied there on the domain consisting of one element.

The one dimensional case

Before stating the lemma in full generality, it is useful to look at some simple special cases. In one dimension and for a function that has derivatives on interval , the lemma reduces to

where is the space of all polynomials of order at most .

In the case when , , , and is twice differentiable, this means that there exists a polynomial of degree one such that for all ,

This inequality also follows from the well-known error estimate for linear interpolation by choosing as the linear interpolant of .

Statement of the lemma

Suppose is a bounded domain in , , with boundary and diameter .

is the Sobolev space of all function on with weak derivatives of order up to in . Here, is a multiindex,

and denotes the derivative times with respect to , times with respect to , and so on. The Sobolev seminorm on

consists of the norms of the highest order derivatives,

and

is the space of all polynomials of order up to on . Note that for all

. and , so has the same value for any .

Lemma (Bramble and Hilbert) Under additional assumptions on the domain , specified below, there exists a constant independent of and such that for any

there exists a polynomial such that for all ,

The original result

The lemma was proved by Bramble and Hilbert under the assumption that satisfies the strong cone property; that is, there exists a finite open covering of and corresponding cones with vertices at the origin such that is contained in

for any .

The statement of the lemma here is a simple rewriting of the right-hand inequality stated in Theorem 1 in . The actual statement in is that the norm of the factorspace

is equivalent to the seminorm. The norm is not the usual one but the terms are scaled with so that the right-hand inequality in the equivalence of the seminorms comes out exactly as in the statement here.

In the original result, the choice of the polynomial is not specified, and the value of constant and its dependence on the domain cannot be determined from the proof.

A constructive form

An alternative result was given by Dupont and Scott under the assumption that the domain is star-shaped; that is, there exists a ball such that for any , the closed convex hull of is a subset of . Suppose that is the supremum of the diameters of such balls. The ratio is called the chunkiness of .

Then the lemma holds with the constant , that is, the constant depends on the domain only through its chunkiness and the dimension of the space . In addition, v can be chosen as v = Qmu, where is the averaged Taylor polynomial, defined as

where

is the Taylor polynomial of degree at most of centered at evaluated at , and is a function that has derivatives of all orders, equals to zero outside of , and

such that

Such function always exists.

The result can be extended to the case when the domain is the union of a finite number of star-shaped domains, which is slightly more general than the strong cone property, and other polynomial spaces than the space of all polynomials up to a given degree.

Bound on linear functionals

This result follows immediately from the above lemma, and it is also called sometimes the Bramble-Hilbert lemma, for example by Ciarlet . It is essentially Theorem 2 from .

Lemma Suppose that is a continuous linear functional on and its dual norm. Suppose that for all . Then there exists a constant

such that

Spectral element method In mathematics, the spectral element method is a high order finite element method.

Introduced in a 1984 paper by A. T. Patera, the abstract begins: "A spectral element method that combines the generality of the finite element method with the accuracy of spectral techniques..."

The spectral element method is an elegant formulation of the finite element method with a high degree piecewise polynomial basis.

Discussion

The spectral method expands the solution in trigonometric series, a chief advantage is that the resulting method is of very high order. This approach relies on the fact that

trigonometric polynomials are an orthonormal basis for L2(Ω). The spectral element method chooses instead high degree piecewise polynomial basis functions, also achieving a very high order of accuracy.

A-priori error estimate

The classic analysis of Galerkin methods and Céa's lemma holds here and it can be shown that, if u is the solution of the weak equation, uN is the approximate solution and

:

where C is independent from N and s is no larger than the degree of the piecewise polynomial basis. As we increase N, we can also increase the degree of the basis functions. In this case, if u is an analytic function:

where γ depends only on u.

Related methods

• G-NI or SEM-NI: these are the most used spectral methods. The Galerkin formulation of spectral methods or spectral element methods, for G-NI or SEM-NI respectively, is modified and Gaussian numerical integration is used instead of integrals in the definition of the bilinear form and in the functional F. These method are a family of Petrov–Galerkin methods their convergence is a consequence of Strang's lemma.

• The spectral element method uses tensor product space spanned by nodal basis functions associated with Gauss–Lobatto points. In contrast, the p-version finite element method spans a space of high order polynomials by nodeless basis functions, chosen approximately orthogonal for numerical stability. Since not all interior basis functions need to be present, the p-version finite element method can create a space that contains all polynomials up to a given degree with many fewer degrees of freedom. However, some speedup techniques possible in spectral methods due to their tensor-product character are no longer available. The name p-version means that accuracy is increased by increasing the order of the approximating polynomials (thus, p) rather than decreasing the mesh size, h.

• The hp finite element method (hp-FEM) combines the advantages of the h and p refinements to obtain extremely fast, exponential convergence rates.

Chapter 13

hp-FEM

The hp-FEM is a general version of the finite element method (FEM), a numerical method for solving partial differential equations based on piecewise-polynomial approximations that employs elements of variable size (h) and polynomial degree (p). The origins of hp-FEM date back to the pioneering work of Ivo Babuska et al. who discovered that the finite element method converges exponentially fast when the mesh is refined using a suitable combination of h-refinements (dividing elements into smaller ones) and p-refinements (increasing their polynomial degree). The exponential convergence makes the method a very attractive choice compared to most other finite element methods which only converge with an algebraic rate. The exponential convergence of the hp-FEM was not only predicted theoretically but also observed by numerous independent researchers.

Differences from standard FEM

The hp-FEM differs from the standard (lowest-order) FEM in many aspects.

• Choice of higher-order shape functions: To begin with, the higher-degree polynomials in elements can be generated using different sets of shape functions. The choice of such set can influence dramatically the conditioning of the stiffness matrix, and in turn the entire solution process. This problem was first documented by Babuska et al.

• Automatic hp-adaptivity: In the hp-FEM, an element can be hp-refined in many different ways. One way is to just increase its polynomial degree without subdividing it in space. Or, the element can be subdivided geometrically, and various polynomial degrees can be applied to the subelements. The number of element refinement candidates easily reaches 100 in 2D and 1000 in 3D. Therefore, clearly, one number indicating the size of error in an element is not enough to guide automatic hp-adaptivity (as opposed to adaptivity in standard FEM). Other techniques such as reference solutions or analyticity considerations must be employed to obtain more information about the shape of error in every element.

• Ratio of assembling and solution CPU times: In standard FEM, the stiffness matrix usually is assembled quickly but it is quite large. Therefore, typically, the solution of the discrete problem consumes the largest part of the overall computing time. On the contrary, the stiffness matrices in the hp-FEM typically

are much smaller, but (for the same matrix size) their assembly takes more time than in standard FEM. Mostly, this is due to the computational cost of higher-order numerical quadrature.

• Analytical challenges: The hp-FEM is more difficult to understand from the analytical point of view than standard FEM. This concerns numerous techniques, such as the discrete maximum principles (DMP) for elliptic problems. These results state that, usually with some limiting assumptions on the mesh, the piecewise-polynomial FEM approximation obeys analogous maximum principles as the underlying elliptic PDE. Such results are very important since they guarantee that the approximation remain physically admissible, leaving no possibility of computing a negative density, negative concentration, or negative absolute temperature. The DMP are quite well understood for lowest-order FEM but completely unknown for the hp-FEM in two or more dimensions. First DMP in one spatial dimension were formulated recently in , .

• Programming challenges: It is much harder to implement a hp-FEM solver than standard FEM code. The multiple issues that need to be overcome include (but are not limited to): higher-order quadrature formulas, higher-order shape functions, connectivity and orientation information relating shape functions on the reference domain with basis functions in the physical domain, etc.

Example: the Fichera problem

The Fichera problem (also called the Fichera corner problem) is a standard benchmark problem for adaptive FEM codes. One can use it to show the dramatic difference in the performance of standard FEM and the hp-FEM. The problem geometry is a cube with missing corner. The exact solution has a singular gradient (an analogy of infinite stress) at the center. The knowledge of the exact solution makes it possible to calculate the approximation error exactly and thus compare various numerical methods. For illustration, the problem was solved using three different versions of adaptive FEM: with linear elements, quadratic elements, and the hp-FEM.

Fichera problem: singular gradient

Fichera problem: convergence comparison.

The convergence graphs show the approximation error as a function of the number of degrees of freedom (DOF). By DOF we mean (unknown) parameters that are needed to define the approximation. The number of DOF equals the size of the stiffness matrix. The reader can see in the graphs that the convergence of the hp-FEM is much faster than the convergence of both other methods. Actually, the performance gap is so huge that the linear FEM might not converge at all in reasonable time and the quadratic FEM would need hundreds of thousands or perhaps millions of DOF to reach the accuracy that the hp-FEM attained with approximately 17,000 DOF. Obtaining very accurate results using relatively few DOF is the main strength of the hp-FEM.

Why is hp-FEM so efficient?

Smooth functions can be approximated much more efficiently using large high-order elements than small piecewise-linear ones. This is illustrated in the figure below, where a 1D Poisson equation with zero Dirichlet boundary conditions is solved on two different meshes. The exact solution is the sin function.

• Left: mesh consisting of two linear elements. • Right: mesh consisting of one quadratic element.

While the number of unknowns is the same in both cases (1 DOF), the errors in the corresponding norm are 0.68 and 0.20, respectively. This means that the quadratic approximation was roughly 3.5-times more efficient than the piecewise-linear one. When we proceed one step further and compare (a) four linear elements to (b) one quartic element (p=4), then both discrete problems will have three DOF but the quartic approximation will be approximately 40-times more efficient. When performing few more steps like this, the reader will see that the efficiency gap opens extremely fast.

On the contrary, small low-order elements can capture small-scale features such as singularities much better than large high-order ones. The hp-FEM is based on an optimal combination of these two approaches which leads to exponential convergence.

What is hp-adaptivity?

Some FEM sites describe hp-adaptivity as a combination of h-adaptivity (splitting elements in space while keeping their polynomial degree fixed) and p-adaptivity (only increasing their polynomial degree). This is not entirely accurate. The hp-adaptivity is significantly different from both h- and p-adaptivity since the hp-refinement of an element can be done in many different ways. Besides a p-refinement, the element can be subdivided in space (as in h-adaptivity), but there are many combinations for the polynomial degrees on the subelements. This is illustrated in the figure on the right. For example, if a triangular or quadrilateral element is subdivided into four subelements where the polynomial degrees are allowed to vary by at most two, then this yields 3^4 =

81 refinement candidates (not considering polynomially anisotropic candidates). Analogously, splitting a hexahedron into eight subelements and varying their polynomial degrees by at most two yields 3^8 = 6,561 refinement candidates. Clearly, standard FEM error estimates providing one constant number per element are not enough to guide automatic hp-adaptivity.

Higher-order shape functions

In standard FEM one only works with shape functions associated with grid vertices (the so-called vertex functions). In contrast to that, in the hp-FEM one moreover regards edge functions (associated with element edges), face functions (corresponding to element faces - 3D only), and bubble functions (higher-order polynomials which vanish on element boundaries). The following images show these functions (restricted to a single element):

Vertex function

Edge function

Face function

Bubble function

Note: all these functions are defined in the entire element interior!

Chapter 14

Finite Element Method in Structural Mechanics

The Finite element method (FEM) is a powerful technique originally developed for numerical solution of complex problems in structural mechanics, and it remains the method of choice for complex systems. In the FEM, the structural system is modeled by a set of appropriate finite elements interconnected at points called nodes. Elements may have physical properties such as thickness, coefficient of thermal expansion, density, Young's modulus, shear modulus and Poisson's ratio.

History

The origin of finite method can be traced to the matrix analysis of structures where the concept of displacement or stiffness matrix approach was introduced. Finite element concepts were developed based on engineering methods in 50s. The original works such as those by Argyris and Clough became foundation for today’s finite element structural analysis methods. Earlier books such as by Zienkiewicz and more recent books such as by Yang give comprehensive summary of developments in finite element structural analysis.

Element properties

Fig.1 A crude FE example mesh (with triangular and rectangular elements) for a cantilever beam (corbel) with a hole

• Straight or curved one-dimensional elements with physical properties such as axial, bending, and torsional stiffnesses. This type of elements is suitable for modeling cables, braces, trusses, beams, stiffeners, grids and frames. Straight elements usually have two nodes, one at each end, while curved elements will need at least three nodes including the end-nodes. The elements are positioned at the centroidal axis of the actual members.

• Two-dimensional elements for membrane action (plane stress, plane strain) and/or bending action (plates and shells). They may have a variety of shapes such as flat or curved triangles and quadrilaterals. Nodes are usually placed at the element corners and, if needed for higher accuracy, additional nodes can be placed along the element edges or even inside the element. The elements are positioned at the mid-surface of the actual layer thickness.

• Torus-shaped elements for axisymmetric problems such as thin, thick plates, shells, and solids. The cross-section of the elements are similar to the previously described types: one-dimensional for thin plates and shells, and two-dimensional for solids, and thick plates and shells.

• Three-dimensional elements for modeling 3-D solids such as machine components, dams, embankments or soil masses. Common element shapes include tetrahedrals and hexahedrals. Nodes are placed at the vertexes and possibly in the element faces or within the element.

Element interconnection and displacement

The elements are interconnected only at the exterior nodes, and altogether they should cover the entire domain as accurately as possible. Nodes will have nodal (vector) displacements or degrees of freedom which may include translations, rotations, and for special applications, higher order derivatives of displacements. When the nodes displace, they will drag the elements along in a certain manner dictated by the element formulation. In other words, displacements of any points in the element will be interpolated from the nodal displacements, and this is the main reason for the approximate nature of the solution.

Practical considerations

From the application point of view, it is important to model the system such that:

• Symmetry or anti-symmetry conditions are exploited in order to reduce the size of the domain.

• Displacement compatibility, including any required discontinuity, is ensured at the nodes, and preferably, along the element edges as well, particularly when adjacent elements are of different types, material or thickness. Compatibility of displacements of many nodes can usually be imposed via constraint relations—When such a feature is not available in the software package, a physical model that imposes the constraints may be used instead.

• Elements' behaviours capture the dominant actions of the actual system, both locally and globally.

• The element mesh is sufficiently fine in order to have acceptable accuracy. To assess accuracy, the mesh is refined until the important results shows little change. For higher accuracy, the aspect ratio of the elements should be as close to unity as possible, and smaller elements are used over the parts of higher stress gradient.

• Proper support constraints are imposed with special attention paid to nodes on symmetry axes.

Large scale commercial software packages often provide facilities for generating the mesh, graphical display of input and output, which greatly facilitate the verification of both input data and interpretation of the results.

Theoretical overview of FEM-Displacement Formulation: From elements to system to solution

While the theory of FEM can be presented in different perspectives or emphases, its development for structural analysis follows the more traditional approach via the virtual work principle or the minimum total potential energy principle. The virtual work principle approach is more general as it is applicable to both linear and non-linear material behaviours.

The principle of virtual displacements for the structural system expresses the mathematical identity of external and internal virtual work:

The virtual internal work in the right-hand-side of the above equation may be found by summing the virtual work in the individual elements—This is the crucial step where we will need displacement functions written only for the small domain rather than over the entire system. As shown in the subsequent sections, Eq.(1) leads to the following governing equilibrium equation for the system:

where

= vector of nodal forces, representing external forces applied to the system's nodes.

= vector of system's nodal displacements, which will, by interpolation, yield displacements at any point of the finite element mesh.

= vector of equivalent nodal forces, representing all external effects other than the nodal forces which are already included in the preceding nodal force vector R. These external effects may include distributed or concentrated surface forces, body forces, thermal effects, initial stresses and strains.

= system stiffness matrix, which will be established by assembling the elements' stiffness matrices : .

Once the supports' constraints are accounted for, the nodal displacements are found by solving the system of linear equations (2), symbolically:

Subsequently, the strains and stresses in individual elements may be found as follows:

where

= vector of element's nodal displacements--a subset of the system displacement vector r that pertains to the element under consideration.

= strain-displacement matrix that transforms nodal displacements q to strains at any point in the element.

= elasticity matrix that transforms effective strains to stresses at any point in the element.

= vector of initial strains in the element. = vector of initial stresses in the element.

By applying the virtual work equation (1) to the system, we can establish the element matrices , as well as the technique of assembling the system matrices and . Other matrices such as , , and can be directly set up from data input.

Interpolation or shape functions

Let be the vector of nodal displacements of a typical element. The displacements at any point of the element may be found by interpolation functions as, symbolically:

where

= vector of displacements at any point {x,y,z} of the element. = matrix of shape functions serving as interpolation functions.

Equation (6) gives rise to other quantities of great interest:

• Virtual displacements consistent with virtual nodal displacements:

• Strains in the elements:

where = matrix of differential operators that convert displacements to strains using linear elasticity theory. Eq.(7) shows that matrix B in (4) is

• Virtual strains consistent with element's virtual nodal displacements:

Internal virtual work in a typical element

For a typical element of volume Ve, the internal virtual work due to virtual displacements is obtained by substitution of (5) and (9) into (1):

Element matrices

Primarily for the convenience of reference, the following matrices pertaining to a typical elements may now be defined:

Element stiffness matrix Equivalent element load vector

These matrices are usually evaluated numerically using Gaussian quadrature for numerical integration. Their use simplifies (10) to the following:

Element virtual work in terms of system nodal displacements

Since the nodal displacement vector q is a subset of the system nodal displacements r (for compatibility with adjacent elements), we can replace q with r by expanding the size of the element matrices with new columns and rows of zeros:

where, for simplicity, we use the same symbols for the element matrices, which now have expanded size as well as suitably rearranged rows and columns.

System virtual work

Summing the internal virtual work (14) for all elements gives the right-hand-side of (1):

Considering now the left-hand-side of (1), the system external virtual work consists of:

• The work done by the nodal forces R: • The work done by external forces on the part of the elements' edges or

surfaces, and by the body forces

Substitution of (6b) gives:

or where we have introduced additional element's matrices defined below:

Again, numerical integration is convenient for their evaluation. A similar replacement of q in (17a) with r gives, after rearranging and expanding the vectors :

Assembly of system matrices

Adding (16), (17b) and equating the sum to (15) gives:

Since the virtual displacements are arbitrary, the preceding equality reduces to:

Comparison with (2) shows that:

• The system stiffness matrix is obtained by summing the elements' stiffness matrices:

• The vector of equivalent nodal forces is obtained by summing the elements' load vectors:

In practice, the element matrices are neither expanded nor rearranged. Instead, the system stiffness matrix is assembled by adding individual coefficients to Kkl where the subscripts ij, kl mean that the element's nodal displacements match respectively with the system's nodal displacements rk,rl. Similarly, is assembled by adding individual coefficients to where matches rk. This direct addition of into Kkl gives the procedure the name Direct Stiffness Method.

Chapter 15

Interval Finite Element

Maximum von Mises stress in plane stress problem with the interval parameters (calculated by using gradient method).

The interval finite element method (interval FEM) is a finite element method that uses interval parameters. Interval FEM can be applied in situations where it is not possible to get reliable probabilistic characteristics of the structure. This is important in concrete structures, wood structures, geomechanics, composite structures, biomechanics and in many other areas . The goal of the Interval Finite Element is to find upper and lower bounds of different characteristics of the model (e.g. stress, displacements, yield surface etc.) and use these results in the design process. This is so called worst case design, which is closely related to the limit state design.

Worst case design require less information than probabilistic design however the results are more conservative [Köylüoglu and Elishakoff 1998].

Applications of the interval parameters to the modeling of uncertainty

Solution of the following equation

where a and b are real numbers is equal to .

Very often exact values of the parameters a and b are unknown. Let's assume that and . In that case it is necessary to solve the following equation

[1,2]x = [1,4]

There are several definition of the solution set of the equation with the interval parameters.

United solution set

In this approach the solution is the following set

This is the most popular solution set of the interval equation and this solution set will be applied here.

In the multidimensional case the united solutions set is much more complicated. Solution set of the following system of linear interval equations

is shown on the following picture

Exact solution set is very complicated, because of that in applications it is necessary to find the smallest interval which contain the exact solution set

or simply

where

Parametric solution set of interval linear system

Interval Finite Element Method require the solution of parameter dependent system of equations (usually with symmetric positive definite matrix). Example of the solution set of general parameter dependent system of equations

is shown on the picture below (E. Popova, Parametric Solution Set of Interval Linear System ).

Algebraic solution

In this approach x is such interval number for which the equation

[1,2]x = [1,4]

is satisfied. In other words left side of the equation is equal to the right side of the equation. In this particular case the solution is equal to x = [1,2] because

ax = [1,2][1,2] = [1,4]

If the uncertainty is bigger i.e. a = [1,4], then x = [1,1] because

ax = [1,4][1,1] = [1,4]

If the uncertainty is even bigger i.e. a = [1,8], then the solution doesn't exist. It is really hard to find physical interpretation of the algebraic interval solution set. Because of that in applications usually the united solution set is applied.

Truss structure with the interval parameters

Example 1

Let us consider a truss structure with uncertain load . Mid point value of the load is equal to P0 = 10 [kN].

The truss structure contain 69 bars with the length L1 = 1[m] or , Young's modulus E = 210[GPa], area of cross-section A = 0.012[m2]

Relative error of the interval axial forces

is given in the following table

According to the numerical results the relative error of the axial forces is bigger than 100%, however variations of the forces P are equal to only 5% [Pownuk 2004]. Calculation of the range of function is the main objective of the Interval Finite Element Method. ANSYS input files which can be used to the verification of the results can be found on the following web page .

Example 2

Let us consider a thruss structure which is shown below.

The truss structure contain 15 bars with the length L = 1[m], Young's modulus E = 210[GPa], area of cross-section A = 0.012[m2], . Mid point value of the load is equal to P0 = 10 [kN].

Results of the calculations are shown below (compare ). Relative error of the interval axial forces.

Relative error is bigger than 60%.

The method

Consider PDE with the interval parameters

where is a vector of parameters which belong to given intervals

For example the heat transfer eqation

where kx,ky are the interval parameters (i.e. ).

Solution of the equation (1) can be defined in the following way

For example in the case of the heat transfer equation

Solution is very complicated because of that in practice it is more interesting to find the smallest possible interval which contain the exact solution set .

For example in the case of the heat transfer equation

Finite element method lead to the following parameter dependent system of algebraic equations

where K is a stiffness matrix and Q is a right hand side.

Interval solution can be defined as a multivalued function

In the simplest case above system can be treat as a system of linear interval equations.

It is also possible to define the interval solution as a solution of the following optimization problem

In multidimensional case the intrval solution can be written as

History

Ben-Haim Y., Elishakoff I., 1990, Convex Models of Uncertainty in Applied Mechanics. Elsevier Science Publishers, New York

Valliappan S., Pham T.D., 1993, Fuzzy Finite Element Analysis of A Foundation on Elastic Soil Medium. International Journal for Numerical and Analytical Methods in Geomechanics, Vol.17, pp. 771–789

Elishakoff I., Li Y.W., Starnes J.H., 1994, A deterministic method to predict the effect of unknown-but-bounded elastic moduli on the buckling of composite structures. Computer methods in applied mechanics and engineering, Vol.111, pp. 155–167

Valliappan S. Pham T.D., 1995, Elasto-Plastic Finite Element Analysis with Fuzzy Parameters. International Journal for Numerical Methods in Engineering, 38, pp. 531–548

Rao S.S., Sawyer J.P., 1995, Fuzzy Finite Element Approach for the Analysis of Imprecisly Defined Systems. AIAA Journal, Vol.33, No.12, pp. 2364–2370

Köylüoglu H.U., Cakmak A., Nielsen S.R.K., 1995, Interval mapping in structural mechanics. In: Spanos, ed. Computational Stochastic Mechanics. 125-133. Balkema, Rotterdam

Muhanna, R. L. and R. L. Mullen (1995). "Development of Interval Based Methods for Fuzziness in Continuum Mechanics" in Proceedings of the 3rd International Symposium on Uncertainty Modeling and Analysis and Annual Conference of the North American Fuzzy Information Processing Society (ISUMA–NAFIPS'95),IEEE, 705–710

More references can be found here

Interval solution versus probabilistic solution

It is important to know that the interval parameters generate different results than uniformly distributed random variables.

Interval parameter take into account all possible probability distributions (for ).

In order to define the interval parameter it is necessary to know only upper and lower bound .

Calculations of probabilistic characteristics require the knowledge of a lot of experimental results.

It is possible to show that the sum of n interval numbers is times wider than the sum of appropriate normally distributed random variables.

Sum of n interval number is equal to

Width of that interval is equal to

Let us consider normally distributed random variable X such that

Sum of n normally distributed random variable is a normally distributed random variable with the following characteristics

We can assume that the width of the probabilistic result is equal to 6 sigma (compare Six Sigma).

Now we can compare the width of the interval result and the probabilistic result

Because of that the results of the interval finite element (or in general worst case analysis) may be overestimated in comparison to the stochastic fem analysis. However in the case of nonprobabilistic uncertainty it is not possible to apply pure probabilistic methods. Because probabilistic characteristic in that case are not known exactly [Elishakoff 2000].

It is possible to consider random (and fuzzy random variables) with the interval parameters (e.g. with the interval mean, variance etc.). Some researchers use interval (fuzzy) measurements in statistical calculations (e.g. ). As a results of such calculations we will get so called imprecise probability.

Imprecise probability is understood in a very wide sense. It is used as a generic term to cover all mathematical models which measure chance or uncertainty without sharp numerical probabilities. It includes both qualitative (comparative probability, partial preference orderings, …) and quantitative modes (interval probabilities, belief functions, upper and lower previsions, …). Imprecise probability models are needed in inference problems where the relevant information is scarce, vague or conflicting, and in decision problems where preferences may also be incomplete.

Simple example (tension-compression)

1D Example

In the tension-compression problem the relation between the displacement u and the force P is the following

ku = P

where , u = u(L) or simply

(compare the definition of Young's modulus). Let us assume that the Young's modulus and the force are uncertain

In order to find upper and lower bound of the displacement u it is necessary to calculate partial derivatives

Extreme values of the displacement can be calculated in the following way

Strain can be calculated from the following formula

Derivative of the strain can be calculated by using derivative from the displacements (the same approach can be applied in more complex problems)

Extreme values of the strains can be calcuated as

It is also possible to calculate extreme valuse of strain using the displacements

then

The same methodology can be applied to the stress

then

and

If we treat stress as a function of strain then

then

Structure is safe if stress σ is smaller than a given value σ0 i.e.

σ < σ0

this condition is true if

After calculation we know that this relation is satisfied if

The example is very simple but it shows the applications of the interval parameters in mechanics. Interval FEM use very similar methodology in multidimensional cases [Pownuk 2004].

However in the multidimensional cases relation between the uncertain parameters and the solution is not always monotone. In that cases more complicated optimization methods have to be applied.

Multidimensional example

In the case of tension-compression problem the equilibrium equation has the following form

where u is displacement, E is Young's modulus, A is an area of cross-section, and n is a distributed load. In order to get unique solution it is necessary to add appropriate boundary conditions e.g.

u(0) = 0

If Young's modulus E and n are uncertain then the interval solution can be defined in the following way

For each FEM element it is possible to multiply the equation by the test function v

where After integration by parts we will get the equation in the week form

where Let's introduce a set of grid points x0,x1,...,xNe, where Ne is a number of elements, and linear shape functions for each FEM element

where

left endpoint of the element, left endpoint of the element number "e". Approximate solution in the "e"-th element is a linear combination of the shape functions

After substitution to the weak form of the equation we will get the following system of equations

or in the matrix form

K(e)u(e) = Q(e)

In order to assemble the global stiffness matrix it is necessary to consider an equilibrium equations in each node. After that the equation has the following matrix form

Ku = Q

where

is the global stiffness matrix,

is the solution vector,

is the right hand side.

In the case of tension-compression problem

If we neglect the distributed load n

After taking into account the boundary conditions the stiffness matrix has the following form

Right-hand side has the following form

Let's assume that Young's modulus E, area of cross-section A and the load P are uncertain and belong to some intervals

The interval solution can be defined calculating the following way

Calculation of the interval vector is in general NP-hard, however in specific cases it is possible to calculate the solution which can be used in many engineering applications.

The results of the calculations are the interval displacements

Let's assume that the displacements in the column have to be smaller than some given value (due to safety).

The uncertain system is safe if the interval solution satisfy all safety conditions.

In this particular case

or simple

In postprocessing it is possible to calculate the interval stress, the interval strain and the interval limit state functions and use these values in the design process.

The interval finite element method can be applied to the solution of problems in which there is not enough information to create reliable probabilistic characteristic of the structures [Elishakoff 2000]. Interval finite element method can be also applied in the theory of imprecise probability.

Endpoints combination method

It is possible to solve the equation K(p)u(p) = Q(p) for all possible combinations of endpoints of the interval . The list of all vertices of the interval can be written as . Upper and lower bound of the solution can be calculated in the following way

Endpoints combination method gives solution which is usually exact; unfortunately the method has exponential computational complexity and cannot be applied to the problems with many interval parameters [Neumaier 1990].

Taylor expansion method

The function u = u(p) can be expanded by using Taylor series. In the simplest case the Taylor series use only linear approximation

Upper and lower bound of the solution can be calculated by using the following formula

The method is very efficient however it is not very accurate. In order to improve accuracy it is possible to apply higher order Taylor expansion [Pownuk 2004]. This approach can be also applied in the interval finite difference method and the interval boundary element method.

Gradient method

If the sign of the derivatives is constant then the functions ui = ui(p) is monotone and the exact solution can be calculated very fast.

if then

if then

Extreme values of the solution can be calculated in the following way

In many structural engineering applications the method gives exact solution. If the solution is not monotone the solution is usually reasonable. In order to improve accuracy of the method it is possible to apply monotonicity tests and higher order sensitivity analysis. The method can be applied to the solution of linear and nonlinear problems of computational mechanics [Pownuk 2004]. Applications of sensitivity analysis method to the solution of civil engineering problems can be found in the following paper [M.V. Rama Rao, A. Pownuk and I. Skalna 2008]. This approach can be also applied in the interval finite difference method and the interval boundary element method.

Element by element method

Muhanna and Mullen applied element by element formulation to the solution of finite element equation with the interval parameters [Muhanna, Mullen 2001]. Using that method it is possible to get the solution with guaranteed accuracy in the case of truss and frame structures.

Perturbation methods

The solution u = u(p) stiffness matrix K = K(p) and the load vector Q = Q(p) can be expanded by using perturbation theory. Perturbation theory lead to the approximate value of the interval solution [Qiu, Elishakoff 1998]. The method is very efficient and can be applied to large problems of computational mechanics.

Response surface method

It is possible to approximate the solution u = u(p) by using response surface. Then it is possible to use the response surface to the get the interval solution [Akpan 2000]. Using response surface method it is possible to solve very complex problem of computational mechanics [Beer 2008].

Pure interval methods

Several authors tried to apply pure interval methods to the solution of finite element problems with the interval parameters. In some cases it is possible to get very interesting results e.g. [Popova, Iankov, Bonev 2008]. However in general the method generates very overestimated results [Kulpa, Pownuk, Skalna 1998].

Parametric interval systems

[Popova 2001] and [Skalna 2006] introduced the methods for the solution of the system of linear equations in which the coefficients are linear combinations of interval parameters. In this case it is possible to get very accurate solution of the interval equations with guaranteed accuracy.

Chapter 16

Modal Analysis using FEM

The goal of modal analysis in structural mechanics is to determine the natural mode shapes and frequencies of an object or structure during free vibration. It is common to use the finite element method (FEM) to perform this analysis because, like other calculations using the FEM, the object being analyzed can have arbitrary shape and the results of the calculations are acceptable. The types of equations which arise from modal analysis are those seen in eigensystems. The physical interpretation of the eigenvalues and eigenvectors which come from solving the system are that they represent the frequencies and corresponding mode shapes. Sometimes, the only desired modes are the lowest frequencies because they can be the most prominent modes at which the object will vibrate, dominating all the higher frequency modes.

It is also possible to test a physical object to determine its natural frequencies and mode shapes. This is called an Experimental Modal Analysis. The results of the physical test can be used to calibrate a finite element model to determine if the underlying assumptions made were correct (for example, correct material properties and boundary conditions were used).

FEA eigensystems

For the most basic problem involving a linear elastic material which obeys Hooke's Law, the matrix equations take the form of a dynamic three dimensional spring mass system. The generalized equation of motion is given as:

where [M] is the mass matrix, is the 2nd time derivative of the displacement [U] (i.e.,

the acceleration), is the velocity, [C] is a damping matrix, [K] is the stiffness matrix, and [F] is the force vector. The general problem, with nonzero damping, is a quadratic eigenvalue problem. However, for vibrational modal analysis, the damping is generally ignored, leaving only the 1st and 3rd terms on the left hand side:

This is the general form of the eigensystem encountered in structural engineering using the FEM. To represent the free-vibration solutions of the structure harmonic motion is

assumed , so that is taken to equal λ[U], where λ is an eigenvalue (with units of reciprocal time squared, e.g., s − 2), and the equation reduces to:

[M][U]λ + [K][U] = [0]

In contrast, the equation for static problems is:

[K][U] = [F]

which is expected when all terms having a time derivative are set to zero.

Comparison to linear algebra

In linear algebra, it is more common to see the standard form of an eigensystem which is expressed as:

[A][x] = [x]λ

Both equations can be seen as the same because if the general equation is multiplied through by the inverse of the mass, [M] − 1, it will take the form of the latter. Because the lower modes are desired, solving the system more likely involves the equivalent of multiplying through by the inverse of the stiffness, [K] − 1, a process called inverse iteration. When this is done, the resulting eigenvalues, μ, relate to that of the original by:

but the eigenvectors are the same.

Methods of solution

For linear elastic problems that are properly set up (no rigid body rotation or translation), the stiffness and mass matrices and the system in general are positive definite. These are the easiest matrices to deal with because the numerical methods commonly applied are guaranteed to converge to a solution. When all the qualities of the system are considered:

1. Only the smallest eigenvalues and eigenvectors of the lowest modes are desired 2. The mass and stiffness matrices are sparse and highly banded 3. The system is positive definite

a typical prescription of solution is first to tridiagonalize the system using the Lanczos algorithm. Next, use the QR algorithm to find the eigenvectors and eigenvalues of this tridiagonal system. If inverse iteration is used, the new eigenvalues will relate to the old

by , while the eigenvectors of the original can be calculated from those of the tridiagonalized matrix by:

[rn] = [Q][vn]

where [rn] is a Ritz vector approximately equal to the eigenvector of the original system, [Q] is the matrix of Lanczos vectors, and [vn] is the nth eigenvector of the tridiagonal matrix.

Example

The mesh shown below is the frame of a building modeled as beam elements, specifically consisting of 930 elements and 385 nodal points. The building is constrained at its base where displacements and rotations are zero. The next images are that of the first 5 lowest modes of this building during free vibration. This problem can be seen as a depiction of the likeliest deflections a building would take during an earthquake. As expected, the first mode is a swaying of the building from front to back. The next mode is swaying of the building side to side. The third mode is a stretching and compression mode in the vertical y direction. For the fourth mode, the building nearly assumes the shape of half a sine wave. The fifth mode is a twisting mode.

original mesh

mode 1 swaying front to back

mode 1 and original mesh

mode 2 swaying side to side

mode 2 and original mesh

mode 3 stretching and compression

mode 3 and original mesh

mode 4 sine shape

mode 4 and original mesh

mode 5 twisting

mode 5 and original mesh

Chapter 17

Domain Decomposition Methods and Additive Schwarz Method

Domain decomposition methods In mathematics, numerical analysis, and numerical partial differential equations, domain decomposition methods solve a boundary value problem by splitting it into smaller boundary value problems on subdomains and iterating to coordinate the solution between adjacent subdomains. A coarse problem with one or few unknowns per subdomain is used to further coordinate the solution between the subdomains globally. The problems on the subdomains are independent, which makes domain decomposition methods suitable for parallel computing. Domain decomposition methods are typically used as preconditioners for Krylov space iterative methods, such as the conjugate gradient method or GMRES.

In overlapping domain decomposition methods, the subdomains overlap by more than the interface. Overlapping domain decomposition methods include the Schwarz alternating method and the additive Schwarz method. Many domain decomposition methods can be written and analyzed as a special case of the abstract additive Schwarz method.

In non-overlapping methods, the subdomains intersect only on their interface. In primal methods, such as Balancing domain decomposition and BDDC, the continuity of the solution across subdomain interface is enforced by representing the value of the solution on all neighboring subdomains by the same unknown. In dual methods, such as FETI, the continuity of the solution across the subdomain interface is enforced by Lagrange multipliers. The FETI-DP method is hybrid between a dual and a primal method.

Non-overlapping domain decomposition methods are also called iterative substructuring methods.

Mortar methods are discretization methods for partial differential equations, which use separate discretization on nonoverlapping subdomains. The meshes on the subdomains do not match on the interface, and the equality of the solution is enforced by Lagrange multipliers, judiciously chosen to preserve the accuracy of the solution. In the

engineering practice in the finite element method, continuity of solutions between non-matching subdomains is implemented by multiple-point constraints.

Additive Schwarz method In mathematics, the additive Schwarz method, named after Hermann Schwarz, solves a boundary value problem for a partial differential equation approximately by splitting it into boundary value problems on smaller domains and adding the results.

Overview

Partial differential equations (PDEs) are used in all hard sciences to model phenomena. For the purpose of exposition, we give an example physical problem and the accompanying boundary value problem (BVP). Even if the reader is unfamiliar with the notation, the purpose is merely to show what a BVP looks like when written down.

(Model problem) The heat distribution in a square metal plate such that the left edge is kept at 1 degree, and the other edges are kept at 0 degree, after letting it sit for a long period of time satisfies the following boundary value problem: fxx(x,y) + fyy(x,y) = 0 f(0,y) = 1; f(x,0) = f(x,1) = f(1,y) = 0 where f is the unknown function, fxx and fyy denote the second partial derivatives with respect to x and y, respectively.

Here, the domain is the square [0,1] × [0,1].

This particular problem can be solved exactly on paper, so there is no need for a computer. However, this is an exceptional case, and most BVPs cannot be solved exactly. The only possibility is to use a computer to find an approximate solution.

Solving on a computer

A typical way of doing this is to sample f at regular intervals in the square [0,1] × [0,1]. For instance, we could take 8 samples in the x direction at x = 0.1, 0.2, ..., 0.8 and 0.9, and 8 samples in the y direction at similar coordinates. We would then have 64 samples of the square, at places like (0.2,0.8) and (0.6,0.6). The goal of the computer program would be to calculate the value of f at those 64 points, which seems easier than finding an abstract function of the square.

There are some difficulties, for instance it is not possible to calculate fxx(0.5,0.5) knowing f at only 64 points in the square. To overcome this, one uses some sort of numerical approximation of the derivatives, see for instance the finite element method or finite

differences. We ignore these difficulties and concentrate on another aspect of the problem.

Solving linear problems

Whichever method we choose to solve this problem, we will need to solve a large linear system of equations. The reader may recall linear systems of equations from high school, they look like this:

2a + 5b = 12 (*) 6a − 3b = −3

This is a system of 2 equations in 2 unknowns (a and b). If we solve the BVP above in the manner suggested, we will need to solve a system of 64 equations in 64 unknowns. This is not a hard problem for modern computers, but if we use a larger number of samples, even modern computers cannot solve the BVP very efficiently.

Domain decomposition

Which brings us to domain decomposition methods. If we split the domain [0,1] × [0,1] into two subdomains [0,0.5] × [0,1] and [0.5,1] × [0,1], each has only half of the sample points. So we can try to solve a version of our model problem on each subdomain, but this time each subdomain has only 32 sample points. Finally, given the solutions on each subdomain, we can attempt to reconcile them to obtain a solution of the original problem on [0,1] × [0,1].

Size of the problems

In terms of the linear systems, we're trying to split the system of 64 equations in 64 unknowns into two systems of 32 equations in 32 unknowns. This would be a clear gain, for the following reason. Looking back at system (*), we see that there are 6 important pieces of information. They are the coefficients of a and b (2,5 on the first line and 6,−3 on the second line), and the right hand side (which we write as 12,−3). On the other hand, if we take two "systems" of 1 equation in 1 unknown, it might look like this:

System 1: 3a = 15 System 2: 6b = −4

We see that this system has only 4 important pieces of information. This means that a computer program will have an easier time solving two 1×1 systems than solving a single 2×2 system, because the pair of 1×1 systems are simpler than the single 2×2 system. While the 64×64 and 32×32 systems are too large to illustrate here, we could say by analogy that the 64×64 system has 4160 pieces of information, while the 32×32 systems each have 1056, or roughly a quarter of the 64×64 system.

Domain decomposition algorithm

Unfortunately, for technical reason it is usually not possible to split our grid of 64 points (a 64×64 system of linear equations) into two grids of 32 points (two 32×32 systems of linear equations) and obtain an answer to the 64×64 system. Instead, the following algorithm is what actually happens:

1) Begin with an approximate solution of the 64×64 system. 2) From the 64×64 system, create two 32×32 systems to improve the approximate solution. 3) Solve the two 32×32 systems. 4) Put the two 32×32 solutions "together" to improve the approximate solution to the 64×64 system. 5) If the solution isn't very good yet, repeat from 2.

There are two ways in which this can be better than solving the base 64×64 system. First, if the number of repetitions of the algorithm is small, solving two 32×32 systems may be more efficient than solving a 64×64 system. Second, the two 32×32 systems need not be solved on the same computer, so this algorithm can be run in parallel to use the power of multiple computers.

In fact, solving two 32×32 systems instead of a 64×64 system on a single computer (without using parallelism) is unlikely to be efficient. However, if we use more than two subdomains, the picture can change. For instance, we could use four 16×16 problems, and there's a chance that solving these will be better than solving a single 64×64 problem even if the domain decomposition algorithm needs to iterate a few times.

A technical example

Here we assume that the reader is familiar with partial differential equations.

We will be solving the partial differential equation

uxx + uyy = f (**)

The boundary condition is boundedness at infinity.

We decompose the domain R² into two overlapping subdomains H1 = (− ∞,1] × R and H2 = [0,+ ∞) × R. In each subdomain, we will be solving a BVP of the form:

u( j )xx + u( j )

yy = f in Hj u( j )(xj,y) = g(y)

where x1 = 1 and x2 = 0 and taking boundedness at infinity as the other boundary condition. We denote the solution u( j ) of the above problem by S(f,g). Note that S is bilinear.

The Schwarz algorithm proceeds as follows:

1. Start with approximate solutions u( 1 )0 and u( 2 )

0 of the PDE in subdomains H1 and H2 respectively. Initialize k to 1.

2. Calculate u( j )k + 1 = S(f,u(3 − j)

k(xj)) with j = 1,2. 3. Increase k by one and repeat 2 until sufficient precision is achieved.