pcfd j.title 27-08-08arrow.utias.utoronto.ca/~groth/publications/pcfd-2011-gao.pdf · fax: 1...

76 Progress in Computational Fluid Dynamics, Vol. 11, No. 2, 2011

Parallel solution-adaptive method for

two-dimensional non-premixed combusting flows

X. Gao*

MS 50A-1148,Lawrence Berkeley National Lab,1 Cyclotron Rd., Berkeley, CA 94720, USAFax: 1 (510)4866900E-mail: [email protected]*Corresponding author

S. Northrup and C.P.T. Groth

Institute for Aerospace Studies,University of Toronto,4925 Dufferin Street, Toronto, ON M3H 5T6, CanadaE-mail: [email protected]: [email protected]

Abstract: A parallel Adaptive Mesh Refinement (AMR) algorithm is proposed and applied tothe predictions of both laminar and turbulent steady non-premixed compressible combustingflows. The parallel solution-adaptive algorithm solves the system of partial-differentialequations governing two-dimensional axisymmetric laminar and turbulent compressibleflows for reactive thermally perfect gaseous mixtures using a fully coupled finite-volumeformulation on body-fitted multi-block quadrilateral mesh. Numerical results for co-flowlaminar and turbulent diffusion flames are described and compared to available experimentaldata. The numerical results demonstrate the validity and potential of the parallel AMRapproach for predicting both fine-scale features of laminar and complex turbulentnon-premixed flames.

Keywords: parallel solution-adaptive algorithm; AMR; adaptive mesh refinement; laminarcombustion; turbulent combustion; non-premixed flames; diffusion flames.

Reference to this paper should be made as follows: Gao, X., Northrup, S. and Groth, C.P.T.(2011) ‘Parallel solution-adaptive method for two-dimensional non-premixed combustingflows’, Progress in Computational Fluid Dynamics, Vol. 11, No. 2, pp.76–95.

Biographical notes: Xinfeng Gao obtained a MS Degree in Aerospace Engineering fromSyracuse University, USA and a PhD Degree in Aerospace Science and Engineering fromthe University of Toronto. She is currently a Postdoctoral Scientist at Lawrence BerkeleyNational Lab in USA.

Scott Northrup obtained undergraduate Degrees in Mechanical Engineering and AppliedMathematics at the University of Western Ontario and a MASc Degree in Aerospace Scienceand Engineering at the Institute for Aerospace Studies, University of Toronto where he iscurrently a PhD candidate.

Clinton P.T. Groth is a theoretical and computational fluid dynamicist and with expertisein finite-volume methods for compressible, non-reacting, and reacting flows and in thedevelopment of high-order and parallel AMR schemes. He also has expertise in thecomputation of non-equilibrium, rarefied, and magnetised flows, and the development ofgeneralised transport models following from moment closures of kinetic theory. Followinggraduate work and postdoctoral studies at the University of Toronto and the University ofMichigan, respectively, he is now a Professor in aerospace engineering at the University ofToronto Institute for Aerospace Studies.

Copyright © 2011 Inderscience Enterprises Ltd.

Parallel solution-adaptive method for two-dimensional non-premixed combusting flows 77

1 Introduction

In the last 20 years, numerical methods have becomean essential tool for the investigation of combustingflows. The application of Computational Fluid Dynamics(CFD) methods to reactive flows has yielded animproved understanding of combustion processes.Nevertheless, combustion involves a wide range ofcomplicated physical and chemical phenomena (flamebehaviour is dictated by a strong interaction betweenthe flow structure, chemical kinetics, and thermodynamicproperties of the reactants and products), each with theirown characteristic spatial and/or temporal scales. Inmany cases, combusting flows exhibit large disparities inthese characteristic scales, mainly because combustionis usually associated with turbulent flows. Due to moremanageable computational requirements and somewhatgreater ease in handling complex flow geometries,most practical simulation algorithms are based on theReynolds- or Favre-averaged Navier-Stokes equations,where the turbulent flow structure is entirely modelledand not resolved. In spite of simplifications offered bythe time-averaging approach, the system of equationsgoverning combusting flows can be both large andstiff and its solution can still place severe demands onavailable computational resources.

Many approaches have been taken to reducethe computational costs of simulating combustingflows. One successful approach is to make use ofsolution-directed mesh adaptation, such as the AMRalgorithms developed for aerospace applications (Berger,1984; Berger and Colella, 1989; Quirk, 1991; Powellet al., 1993; De Zeeuw and Powell, 1993; Quirk andHanebutte, 1993; Berger and Saltzman, 1994; Aftosmiset al., 1998; Groth et al., 1999, 2000). Computationalgrids that automatically adapt to the solution ofthe governing equations are very effective in treatingproblems with disparate length scales, providing therequired spatial resolution while minimising memoryand storage requirements. Recent progress in thedevelopment and application of AMR algorithmsfor low-Mach-number reacting flows and premixedturbulent combustion is described by Day and Bell(2000) and Bell et al. (2001, 2002b). Another approachfor coping with the computational cost of reactingflow prediction is to apply a domain decompositionprocedure and solve the problem in a parallel fashionusing multiple processors. Large massively paralleldistributed-memory computers can provide many foldincreases in processing power and memory resourcesbeyond those of conventional single-processor computersand would therefore provide an obvious avenue forgreatly reducing the time required to obtain numericalsolutions of combusting flows.

This work seeks to combine these two numericalapproaches, producing a parallel AMR method thatboth reduces the overall problem size and the timeto calculate steady solutions for combusting flowsfrom the laminar to turbulent regimes. In particular,

a highly scalable parallel block-based AMR algorithm isproposed for predicting a wide range of two-dimensionalnon-premixed compressible combusting flows.

In the following sections, the mathematical modellingof non-premixed combustion is presented for turbulentflows and the parallel AMR algorithm is described indetails. Numerical verification of the proposed parallelAMR scheme is then presented by considering thenumerical predictions for three classical non-reactingflow problems and one reactive flow problem. Numericalresults are described and discussed for non-premixedmethane-air laminar and turbulent co-flow axisymmetricdiffusion flames. Finally, conclusions of this work aregiven.

2 Mathematical modelling

The governing conservation equations describing thebehaviour of a thermally perfect compressible reactivegaseous mixture are formulated for both laminarand turbulent reactive flows. For laminar reactiveflows, Navier-Stokes equations are employed and theformulation is rather standard and straightforward.Herein, we will only present mathematical modelling forturbulent combusting flows to keep this section brief.

2.1 Favre-averaged Navier-Stokes equations

A system of Favre-averaged Navier-Stokes equationsdescribing a thermally-perfect compressible turbulentreactive mixture can be formulated as:

∂ρ

∂t+ �∇ · (ρ�u) = 0, (1)

∂

∂t(ρ�u) + �∇ ·

(ρ�u�u + p

��I)

= �∇ ·(��τ + ��λ

), (2)

∂

∂t(ρe) + �∇·

[ρ�u

(e +

p

ρ

)]

= �∇ ·[(

��τ + ��λ)

· �u]+ �∇ · (Dk

�∇k) − �∇ · (�q + �qt), (3)

∂

∂t(ρcn) + �∇ · (ρcn�u) = −�∇ · ( �Jn + �Jtn) + ρwn, (4)

where equations (1)–(3) reflect the conservation of mass,momentum, and energy for the reactive mixture, ρis the Reynolds-averaged mixture density, �u is theFavre-averaged mean velocity of the mixture, p is theReynolds-averaged mixture pressure given by the idealgas law p =

∑Nn=1 ρcnRnT where N is the number of

species, Rn is the species gas constant, and T is themixture temperature. Here, e = |�u|2/2 +

∑Nn=1 cnhn −

p/ρ + k is the Favre-averaged total specific mixtureenergy, k is the specific turbulent kinetic energy and Dk

is the coefficient for the diffusion of the turbulent energy,��τ and ��λ are the fluid stress tensor and the turbulentReynolds stress tensor for the mixture, respectively, �qand �qt are the laminar and turbulent heat flux vector,respectively. Equation (4) describes the time evolution of

78 X. Gao, S. Northrup and C.P.T. Groth

the species mass fraction, where wn is the time-averagedor mean rate of the change of the species mass fractionproduced by the chemical reactions. The molecular stressis given by the general stress-strain relationship

��τ = 2µ

(��S − 1

3��I �∇ · �u

),

where µ is molecular viscosity, ��S is the strain rate tensor,

and ��I is the identity tensor. For species n, the molecularand turbulent diffusive flux, �Jn and �Jtn, are modelled by

�Jln = −ρDn∇cn and �Jtn = −ρDtn∇cn.

The molecular and turbulent heat flux are modelled by

�q = −(κ∇T−

N∑n=1

hn�Jn

)and �qt = −

(κt∇T−

N∑n=1

hn�Jtn

),

where κ and κt are the laminar and turbulent thermalconductivity of the mixture and Dn and Dtn are themolecular and turbulent diffusivity of species n relativeto the major species, respectively. Given laminar andturbulent Schmidt numbers, Sc and Sct, Dn and Dtn

are obtained using Dn = µ/ρSc and Dtn = µt/ρSct.In addition, hn is the absolute (chemical and sensible)internal enthalpy for species n.

2.2 Two-equation k-ω model

The two-equation k-ω model of Wilcox (1998) is usedhere to model the unresolved turbulent flow quantities.In this approach, the Boussinesq approximation is used

to relate the Reynolds stress tensor, ��λ, to the mean flowstrain-rate tensor using a turbulent eddy viscosity, µt,��λ = 2µt(

��S − 13��I �∇ · �u) − 2

3��Iρk with µt =ρk/ω. Transport

equations are solved for turbulent kinetic energy, k, andthe specific dissipation rate, ω, given by

∂

∂t(ρk) + �∇ · (ρk�u)

= ��λ : �∇�u + �∇ · [(µ + µtσ∗)�∇k] − β∗ρkω, (5)

∂

∂t(ρω) + �∇ · (ρω�u)

= αω

k��λ : �∇�u + �∇ · [(µ + µtσ)�∇ω] − βρω2, (6)

where σ∗, β∗, α, σ, and β are closure coefficients for thetwo-equation model and are given by Wilcox (2002).

2.3 Thermodynamic and transport properties

Thermodynamic relationships and transport coefficientsare required to close the systems of equations givenabove for both laminar and turbulent combusting flows.Thermodynamic and molecular transport properties ofeach gaseous species are prescribed using the empiricaldatabase compiled by Gordon and McBride (1994),McBride and Gordon (1996), which provides curve

fits for the species enthalpy, hn; specific heat, cpn;entropy; viscosity, µn; and thermal conductivity, κn, asfunctions of temperature, T . The Gordon-McBride dataset contains curve fits for over 2000 substances, including50 reference elements.

The molecular viscosity, µ, and thermal conductivity,κ, of the reactive mixture are determined using themixture rules of Wilke (1950), Mason and Saxena(1958), respectively. Turbulent contributions to thermalconductivity and species diffusivity are modelled bymaking an analogy between momentum and heat andmass transfer and introducing the turbulent Prandtl andSchmidt numbers, Prt and Sct, both of which are takento be constant (Prt = 0.9 and Sct = 1), and assumingκt = µtcp/Prt and Dtn = µt/ρSct.

2.4 Reduced chemical kinetics

The combustion of methane is considered here.Although several detailed chemical reaction mechanismsare available for describing methane-air combustionprocesses (Smith et al., GRI-Mech 3.0), forcomputational simplicity, our attention shall be restrictedto reduced chemical kinetic schemes. Both one- andtwo-step reduced chemical reaction mechanisms asdescribed by Westbrook and Dryer (1981) are used.

Empirically derived expressions for the reaction ratesin each case are used. The five species considered in theone-step reaction mechanism are methane (CH4), oxygen(O2), carbon dioxide (CO2), water (H2O), and nitrogen(N2). Nitrogen is assumed to be inert. Carbon monoxide(CO) species is also considered in the two-step reactionmechanism. Further details and reaction rates for thesereduced mechanisms are given by Westbrook and Dryer(1981).

2.5 Eddy Dissipation Model

The mean reaction rates, ωn, in equation (4) describethe mean production and consumption of each of thechemical species due to the chemical reactions and stronginteractions between turbulence and chemistry and areestimated using the Eddy Dissipation Model (EDM) ofMagnussen and Hjertager (1976). This model assumesthat turbulence mixing limits the fuel burning and thatthe fuel reaction rate is limited by the deficient species.The individual species mean reaction rate is then takento be the minimum of the rates given by the finite-ratechemical kinetics (i.e., the law of mass action andArrhenius reaction rates) and the EDM value. The latteris related to the turbulence mixing time and is estimatedusing the dissipation rate, ω.

2.6 Treatment of near-wall turbulence

Both low-Reynolds-number and wall-functionformulations of the k-ω model are used for thetreatment of near-wall turbulent flows, with a procedurefor automatically switching from one to the other,


depending on mesh resolution. In the case of thelow-Reynolds-number formulation, it can be shownthat limy→0 ω = 6ν

βoy2 where y is the distance normalfrom the wall. Rather than attempting to solve theω-equation directly, the preceding expression is used tospecify ω for all values of y+ ≤ 2.5, where y+ = uτy/ν,u2

τ = τw/ρ, and τw is the wall shear stress. Providedthere are 3–5 computational cells inside y+ = 2.5,this procedure reduces numerical stiffness, guaranteesnumerical accuracy, and permits the k − ω model to besolved directly in the near-wall region without resortingto wall functions. In the case of the wall-functionformulation, the expressions k = u2

τ√β∗

o

, ω = uτ√β∗

o κy, are

used to fully specify k and ω for y+ ≤ 30 − 250, where κis the von Kármán constant.

A procedure has also been developed to automaticallyswitch between these two approaches, depending on thenear-wall mesh resolution. In this procedure, the valuesof k and ω are approximated by

k =u2

τ√β�

o

(min(y+, 30)

30

)2

and ω = ωo

√1 +

(ωwall

ωo

)2

,

where ωo = 6νβoy2 and ωwall = uτ√

β∗κy. This procedure has

been devised to prescribe k and ω for y+ lying between 2.5and a cutoff value. where ωo and ωwall are the values inthe near-wall sub-layer and in the log layer, respectively.The cutoff is taken to be in the range 30–50 for thisstudy. When y+ is close to the lower limit, 2.5, k andω approach zero and the asymptotic value, respectively.When y+ approaches the cutoff value, the wall functionis recovered. This automatic near-wall treatment readilyaccommodates situations during AMR where the meshresolution may not be sufficient for directly calculatingnear-wall turbulence using the low-Reynolds-numberformulation.

3 Parallel AMR algorithm

3.1 Finite volume scheme

A finite volume scheme is proposed to solve the system ofpartial-differential equations governing two-dimensionalaxisymmetric laminar and turbulent compressible flowsfor reactive thermally perfect gaseous mixtures using afully coupled finite-volume formulation on body-fittedmulti-block quadrilateral mesh. Applying the divergencetheorem to the differential form of the system ofgoverning equations in two-dimensional axisymmetriccoordinates, Equations (1)–(6), one arrives at the integralform

ddt

∫A(t)

UdA +∮

l(t)�n · �Fdl =

∫A(t)

(− Sa

r+ Sc + St

)dA,

(7)

where U is the vector of conserved variables, �F theflux dyad, defined as �F = (F − Fv,G − Gv), Sa thesource term associated with the axisymmetric geometry,Sc and St the source terms due to finite rate chemistryand turbulence modelling to be included for turbulentflows, A the control area, l the closed contour of thecontrol volume, r is the radial distance, and �n is the unitoutward vector normal to the closed surface. For laminarflows, U is given by

U = [ρ, ρvr, ρvz, ρe, ρc1, . . . , ρcN ]T , (8)

and ρk and ρω are included for turbulent flows. Note thatthe last species N is chosen to be Nitrogen and usedto accommodate the numerical errors. In other words,the concentration of Nitrogen is corrected by usingcN = 1.0 −

∑N−1n=1 cn after solving the system.

This fully compressible formulation canreadily accommodate large density variations andthermo-acoustic phenomena. Nevertheless, laminarcombusting flows are in general characterised by verylow Mach numbers (M < 0.2) and nearly incompressiblebehaviour. Therefore, a local preconditioning techniqueproposed by Weiss and Smith (1995) and Turkel(1999) is used here to remove numerical stiffness andmaintain solution accuracy for low-Mach-number flows.The preconditioned system of governing equationsis integrated over quadrilateral cells of a structuredmulti-block quadrilateral mesh. The semi-discrete formof this finite-volume formulation applied to cell (i, j) isgiven by

ΓdUi,j

dt= − 1

Ai,j

∑faces,k

�Fi,j,k ·�ni,j,k∆�i,j,k −Sai,j

ri,j+Sci,j ,

(9)

where Γ is the Weiss-Smith preconditioning matrix forthe conserved variable system, ri,j and Ai,j are the radialdistance and area of cell (i, j), and ∆� is the length of thecell face.

The inviscid (hyperbolic) component of the numericalflux at each cell face is evaluated using limitedlinear reconstruction (Barth, 1993) and one of severalRiemann-solver based flux functions (Roe, 1981; Einfeldt,1988; Linde, 2002). The viscous (elliptic) componentof the numerical flux is evaluated by employing adiamond-path reconstruction procedure as described byCoirier and Powell (1996).

3.2 Time marching method

For the time-invariant calculations performed as partof this study, a multi-stage time-marching scheme isused to solve the coupled set of non-linear ODEs,that arise from the finite-volume spatial discretisationprocedure. The time-marching scheme is based on theoptimally-smoothing multi-stage time marching schemedeveloped by van Leer et al. (1989). The general M


stage optimally smoothing time-marching scheme forintegrating Equation (9) from the time level n to timelevel n + 1 can be written as

m stage:

U0i,j = Un

i,j

Umi,j = U0

i,j − αm∆tn Ri,j(Um−1)

Un+1i,j = UM

i,j

,

where m = 1 · · ·M , ∆tn = tn+1 − tn is the size of thetime step and αm are multi-stage coefficients. Thecoefficients used here have been selected to optimisethe high-frequency damping for first- and second-orderupwind discretisations of the scalar advection equationin multigrid applications (van Leer et al., 1989). Theyare not optimised for diffusion problems or viscousflows.

The source terms associated with finite-rate chemistryand turbulence modelling are usually responsible formuch of the numerical stiffness in the resulting discretisedsystem of equations. The use of semi-implicit timeintegration can be utilised to cope with the stiffness ofthe system. This method treats source terms implicitly,while treating the fluxes explicitly. Hence, this methodavoids solving the large block matrices associatedwith the fully implicit scheme. A local linear systemof equations is then solved to obtain the solutionchange using a dense matrix solver. In this case a LUdecomposition was used.

The inviscid Courant-Friedrichs-Lewy stability,viscous von Neumann stability, and turbulent andchemical time step constraints are imposed when selectingthe time step. Note that, for reacting flows, the inverseof the maximum diagonal of the chemical source termJacobian is added to the time step calculation. The timestep, ∆tn, is then determined by

∆tn = min

(CFL

∆l

|�u|+c,α

2ρ∆l2

max(µ, µt),

(β max

(∂Sc

∂U

))−1)

(10)

where ∆l is the cell-face length of a cell, c is the soundspeed, and µ and µt are molecular viscosity and turbulenteddy viscosity, respectively, and where α and β are scalingfactors.

3.3 Preconditioned multigrid

Application of multigrid to the Favre-averagedNavier-Stokes equations can result in convergencerates that are far from optimal due to the stiff sourceterm associated with the turbulence models. Classicmultigrid remedies for these multigrid difficulties, suchas directional-coarsening, directional implicit smoother,combining directional coarsening and smoothing, andcombining a point-implicit block-Jacobi preconditionerand J-coarsening, etc., are well documented and the

effects are illustrated for some example problems byPierce and Giles (1996, 1997) and Mavriplis (1998).Sheffer et al. (1998) employed the point-implicitformulation of Bussing and Murman (1987, 1988) incombination with an explicit time-stepping multigridsolver for calculating high speed reactive flows. Gerlingerand Brüggemann (1997) and Gerlinger et al. (1998, 2001)investigated the q-ω model and proposed the techniquesof computing the production term and the divergence ofvelocity field only on the finest mesh and restricted thesevalues to coarser meshes. For complicated geometriesand simple flow field initialisations, they initiated thecalculation with several fine mesh iterations beforerestricting to coarser meshes. Refer Gao (2008) for detailson a literature review in terms of multigrid applicationsto turbulent flows.

In this study, to remedy the multigrid difficulties instability and convergence due to the stiff turbulenceproduction terms and chemical source terms and the useof highly stretched meshes, we employed a point-implicitblock-Jacobi preconditioner (matrix preconditioner) incombination with the multigrid solver. The turbulencequantities are restricted to the coarse mesh but notupdated so as to enhance the stability of the schemeand avoid non-physical solutions. Note that we donot believe that the point-Jacobi preconditioning willprovide a fully satisfactory solution to issues withhigh-aspect-ratio meshes; however, our experiences showthat the preconditioning combined with modificationsto the restriction and prolongation operators partiallyalleviates the problem.

3.3.1 Smoothing operator

The point-implicit block-Jacobi preconditioner usedherein is based on the form of the discrete residualoperator, R, and obtained by extracting the termscorresponding to the centre cell in the stencil. Theapplication of the matrix preconditioner to themulti-stage time-marching scheme is rewritten as

m stage:

U0i,j = Un

i,j

Umi,j = U0

i,j − ναm P−1i,j Ri,j (Um−1)

Un+1i,j = UM

i,j

where ν includes the time step and P−1i,j is the inverse of

the matrix preconditioner for cell (i, j). The constructionof the matrix preconditioner is illustrated here usingthe system of governing equations in a two-dimensionalaxisymmetric system. Given the residual function forcell (i, j), Ri,j , which can be written as

Ri,j =dUi,j

dt=

−1Ai,j

Nf∑k=1

�Fk ·�nk ∆lk −Sai,j

r+Sti,j +Sci,j ,

(11)


the matrix preconditioner, Pi,j = ∂R∂U

∣∣i,j, is a N × N

matrix and has the form

Pi,j =∂Ri,j(U, ∇U)

∂Ui,j=

∂R1

∂U1

∂R1

∂U2. . .

∂R1

∂UN

∂R2

∂U1

∂R2

∂U2. . .

∂R2

∂UN

......

. . ....

∂RN

∂U1

∂RN

∂U2. . .

∂RN

∂UN

.

(12)

Each term Pi,j consists of five components and theyresult from the inviscid numerical flux Jacobian, Ci,the viscous flux Jacobian, Cv , and the source Jacobiansdue to axisymmetric coordinate system, Ca, the sourceJacobian due to turbulence, Ct, and the source Jacobiandue to chemistry, Cc. The evaluation of each of theseJacobian matrices is discussed below.

The inviscid numerical flux Jacobian at each cell faceis evaluated by the solutions of a Riemann problem in arotated frame aligned with the normal to the cell face andtakes on the form of

Ci =∂(�F · �n)∂Ui,j

=∂F∂F∗

∂F∗

∂U∗L/R

∂U∗L/R

∂Ui,j, (13)

where U∗L/R and F∗ are the solution state and flux

in the rotated frame. The term ∂F∗

∂U∗L/R

is evaluated for

both the Roe and HLLE flux functions. This inviscidJacobian evaluation is approximated by assuming thatthe eigenvalues and eigenvectors (appearing in Roe andHLLE flux functions) are constant and, when usinghigher order scheme, the gradients of the primitivevariables are also assumed to be constant. Pierceand Giles (1997) also suggested that using the matrixprecondition based on a first-order discretisation forhigher-order schemes is acceptable.

The viscous numerical flux Jacobian is formulateddepending on the method used to evaluate the viscousflux. In this study, the viscous numerical flux Jacobian isdetermined using a diamond-path procedure. The generalform for the viscous flux Jacobian is

Cv =∂�Fv · �n∂Ui,j

=∂(nr Fv + nz Gv)

∂Ui,j. (14)

In the source Jacobians, Cs, the three terms are lumpedtogether as

Cs = −∂

(Sai,j

r

)∂Ui,j

+∂Sti,j

∂Ui,j+

∂Sci,j

∂Ui,j. (15)

The preconditioner was tested using a finite-differencemethod with a first order accurate approximation of the

residual Jacobian:

∂Ri,j(U, ∇U)∂U

∣∣∣∣∣i,j

≈ R(Ui,j + ε) − R(Ui,j)ε

+ O(ε),

where ε = ηU with η = 10−6 ∼ 10−5. The verification hasbeen performed with varied solution fields and differentmesh stretching factors. The observed maximum relativeerror was found to be about

σ =

∣∣∣∣∣∣∂R

∂Ui,j− R(Ui,j+ε)−R(Ui,j)

ε

∂R∂Ui,j

∣∣∣∣∣∣ ≤ 2.0%. (16)

It is felt that this error is acceptable considering theapproximations used in the Jacobian evaluation and thenumerical error associated with the numerical scheme.

3.3.2 Restriction and prolongation operators

In particular, it was found that the multigrid convergencewas significantly affected by the prolongation operator.A standard bi-linear interpolation was used initially totransfer corrections from coarse to fine mesh; however,the performance of the multigrid scheme suffered.Convergence was hampered and, in some cases, theprocedure failed to converge. In this work, effortshave been made to devise more effective prolongationoperators for meshes with large cell aspect ratios.

Figure 1 illustrates a stretched grid. Severalprolongation operators are investigated: simpleinjection, Ui,j fine = Ui,j coarse; area weighted injection,Ui,j fine = (Ai,j coarse/

∑Ai,j fine) Ui,j coarse; a standard

bi-linear interpolation; and finally, a standard bi-linearinterpolation plus a linear interpolation, meaning thatinterpolating the value for the coarse node (i, j) usingstandard bi-linear first and then interpolating for thefine cell (i, j) with values from both coarse node(i, j) and coarse cell (i, j) with a linear interpolation.The effectiveness of using these different prolongationoperators has been studied for a laminar non-reactingflow problem using stretched computational grids. Basedon this study, it is suggested that a cell-aspect-ratio-basedsensor can be applied as a switch in the approach of theprolongation operator. A cell-aspect-ratio-based sensorwas implemented in the multigrid algorithm as follows:simple injection is used for cells with aspect ratio higherthan a cutoff value, and standard bi-linear interpolationis then employed for cells with aspect ratios lower thanthat cutoff value.

Note that the multigrid algorithm is applied directlyto each of the solution blocks without regard to thelevel of refinement for the grid blocks associated witheach solution block, i.e., the grids are not necessarily onthe same refinement level due to the AMR procedure.This block-based approach to the multigrid algorithmcan adversely affect its performance. This performancedegradation may be remedied by utilising additionalmultigrid levels spanning across blocks to bring allsolution content to the same level of refinement, but this


is not considered here. In addition, the system ofgoverning equations is solved on the coarsest grid level.The communication for the multigrid is then carriedout on the coarsest grids between blocks. The numberof inter-block messages on the coarsest meshes is oftennearly the same as that on the fine meshes, thereby theratio of computation to communication may decreaseand the parallel performance would be adversely affected.

Figure 1 Illustration of different prolongation operatorson a stretched body-fitted quadrilateral mesh(see online version for colours)

3.4 Block-based Adaptive Mesh Refinement

A flexible block-based hierarchical data structure hasbeen developed and is used in conjunction with thefinite-volume scheme described above to facilitateautomatic solution-directed mesh adaptation onmulti-block body-fitted quadrilateral mesh accordingto physics-based refinement criteria. In a block-basedAMR strategy, mesh adaptation is accomplished by thedividing and coarsening of appropriate solution blocks.In general, each block also has an equal number ofcells. The basic data structure is then a tree, where anyblock that requires refinement generates a number ofequal sized blocks when a resolution change of twois assumed. The block-based AMR strategy results ina rather light tree data structure for prescribing theconnectivity between blocks as compared to the treestructure generally used for tracking cell connectivity inthe cell-based methods. In addition, the block-based datastructure naturally lends itself towards an efficient andreadily scalable parallel implementation. It amortises theoverhead of communication over entire blocks of cells,instead of over single cells as in cell-based data structures.However, generally larger numbers of refined cells canbe created (i.e., typically more than the correspondingnumber of cells used in cell-based tree data structures)thereby possibly increasing the amount of computationalwork and storage space needed to solve a given problem.

Applications of the block-based approach onCartesian mesh are described, for example, byQuirk and Hanebutte (1993), Berger and Saltzman

(1994), Gombosi et al. (1994), and MacNeice et al.(2000). Following Groth et al. (1999, 2000) forcomputational magnetohydrodynamics, a flexibleblock-based hierarchical data structure to facilitateautomatic solution-directed mesh adaptation onmultiblock body-fitted (curvilinear) meshes for complexflow geometries has been developed. While introducingsome added complications, the use of body-fitted meshespermits more accurate solutions near boundaries, enablesthe use of anisotropic grids with grid point clusteringand stretching, and allows for better resolution of thinboundary and mixing layers. Note that, in this study,the mesh refinement is constrained such that the gridresolution changes by only a factor of two betweenadjacent blocks and the minimum resolution is not lessthan that of the initial mesh. Standard multigrid-typerestriction and prolongation operators are used toevaluate the solution on all blocks created by thecoarsening and division processes, respectively.

We use a heuristic set of refinement criteria basedon our physical understanding of the flow propertiesof interest (so-called physics-based refinement criteria).For the non-reacting flows considered here, the followingmeasures are used ε1 ∝ |�∇ρ|, ε2 ∝ |�∇ · �u|, ε3 ∝ |�∇ ⊗ �u|,in the decision to refine or coarsen a solution block.These three quantities correspond to local measures ofthe density gradient, compressibility, and vorticity ofthe mean flow field and enable the detection of contactsurfaces, shocks, and shear layers. For combusting flows,additional measures were identified for directing the meshadaption. The following four additional measures, ε4 ∝|�∇k|, ε5 ∝ |�∇ω|, ε6 ∝ |�∇T |, ε7 ∝ |�∇cn| are used. The firsttwo measures correspond to gradients of the specificturbulent kinetic energy and dissipation rate per unitturbulent kinetic energy, respectively, and relate to thestructure of the turbulent field. The last two quantitiesmeasure the gradients of mean temperature and meanconcentration for species n, respectively, and providereliable detection of flame fronts and combustion zonesfor reactive flows.

3.5 Domain decomposition and parallelimplementation

Domain decomposition is carried out by farming thesolution blocks out to the separate processors, withmore than one block permitted on each processor.For homogeneous architectures, an effective loadbalancing is achieved by simply distributing the blocksequally among the processors. For heterogeneous parallelmachines, a weighted distribution of the blocks can beadopted to preferentially place more blocks on the fasterprocessors and less blocks on the slower processors.

Placing nearest-neighbour blocks on the sameprocessor can also help to reduce the overallcommunication costs. A Morton ordering space-fillingcurve (Aftosmis et al., 2004) is adopted to providenearest-neighbour ordering of the solution blocks inthe multi-block quadrilateral AMR meshes (Fig. 2),


and improve the parallel performance of the solutionmethod.

Figure 2 Morton ordering space filling curve used to providenearest-neighbour ordering of blocks for moreefficient load balancing of blocks on multipleprocessors. The thick black line represents the spacefilling curve passing through each of the solutionblocks in the multi-block AMR mesh

The parallel implementation of the block-based AMRscheme was developed using the C++ programminglanguage (Stroustrup, 2000) and Message PassingInterface (MPI) (Gropp et al., 1999). Use of thesestandards greatly enhances the portability of thecomputer code. Inter-processor communication is mainlyassociated with block interfaces and involves theexchange of ghost-cell solution values and conservativeflux corrections at every stage of the multi-stagetime-integration procedure. Message passing of theghost-cell values and flux corrections is performed inan asynchronous fashion with gathered wait states andmessage consolidation.

4 Numerical verification

The parallel implementation has been carried out on aparallel cluster of 4-way Hewlett-Packard ES40, ES45,and Integrity rx4640 servers with a total of 244 Alpha andItanium 2 processors. A low-latency Myrinet networkand switch is used to interconnect the servers in thecluster. All of the numerical results reported here wereobtained using this parallel cluster. Initial verificationof the proposed parallel AMR scheme is carried out

by considering the numerical predictions for threeclassical non-reacting flow problems (a laminar Couetteflow, a laminar flat plate boundary layer flow and afully-developed turbulent pipe flow) and one reactive flowproblem (a premixed laminar flame). The solutions forthese problems are well established and can be used toassess the validity and accuracy of the scheme. Herein,we only present a laminar Couette flow, a fully-developedturbulent pipe flow and a premixed laminar flame to keepthis section brief.

4.1 Laminar Couette flow

The computation of non-reacting laminar Couette flowin a channel with a moving wall was considered inorder to demonstrate the accuracy of the viscous spatialdiscretisation scheme. The case with an upper wallvelocity of 29.4 m/s and a favourable pressure gradientof dp/dx = −3,177 Pa/m was investigated and comparedto the analytic solution. The predicted velocity profile(not shown here) matches the exact analytic solution forthis incompressible isothermal flow. The L1-norm of theerror in axial component of velocity is shown in Figure 3for both the uniform and adaptive grids. The slope of thenorm is 2.02, indicating that the finite-volume scheme isindeed second-order accurate.

Figure 3 L1-norms of the solution error as a function ofmesh size for laminar Couette flow

4.2 Fully-developed turbulent pipe flow

A verification of the parallel AMR scheme fornon-reacting turbulent flows has been performed. Thenumerical results of a non-reacting fully-developedturbulent pipe flow with Re = 500,000 are comparedto the experimental data of Laufer (1954). Solutionsfor both the wall function and low-Reynolds-numberformulations of the k-ω turbulence model are comparedto experimental mean axial velocity and turbulent kinetic


energy profiles in Figures 4 and 5. Calculations with thelow-Reynolds-number formulation were performed using80 cells in the radial direction with 3–4 of those cells lyingwithin the laminar sub-layer. The first cell off the wall waslocated at y+ ≈ 0.6. The results using the wall functionswere obtained using 32 cells in the radial direction withthe first cell located at y+ ≈ 43. The agreement betweenthe experimental data and numerical results for this caseis generally quite good. As expected, it is evident that thek-ω model is able to reproduce the characteristic featuresof fully-developed pipe flow.

Figure 4 Comparison of predicted mean axial velocity withexperimental data for fully developed pipe flow,Re = 500,000

Figure 5 Comparison of predicted turbulent kinetic energywith experimental data for fully developed pipeflow, Re = 500,000

4.3 Multigrid acceleration

Convergence acceleration provided by the preconditionedmultigrid algorithm was also examined for the fully

developed turbulent pipe flow problem. A mesh of size1024 cells and having cell aspect ratios in the range of10 × 105 to 2 × 105 and an off-wall spacing of 7.0 ×10−7 m was used in this study. There were 32 cells inthe radial direction and an automatic wall boundarytreatment was employed. The influence of using differentmultigrid levels and cycles on convergence features hasalso been investigated.

Figures 6 and 7 compare the convergence ratesachieved for the turbulent pipe flow using the explicittime-marching scheme with local time-stepping, themultigrid algorithm with explicit smoother, and thepreconditioned multigrid approach. The convergence rateis shown as a function of both the number of iterationsand the number of equivalent Right-Hand Side (RHS)evaluations. Clearly, the preconditioned multigrid resultsin a more efficient convergence rate than the others.Notice from the figures that the preconditioned multigridalgorithm exhibited a convergence stall after the residualin the turbulent kinetic energy dropped to about 104.It is believed that this effect is due to the nonlinearnature of the slope limiters and their activation in smoothregions of the flow field (Venkatakrishnan, 1993). Thisconvergence stall can be alleviated by freezing the limiterafter the residual has dropped to a predefined level;however, this technique was not employed here.

Figure 6 Comparisons of convergence rates as a function ofmultigrid cycles for 4-level V-cycle multigridbetween regular multigrid and preconditionedmultigrid with a 5-stage optimal smoothing schemefor the fully-developed turbulent pipe flow


Figure 7 Comparisons of convergence rates as a function ofthe number of equivalent RHS evaluations for4-level V-cycle multigrid between regular multigridand preconditioned multigrid with a 5-stage optimalsmoothing scheme for the fully-developed turbulentpipe flow

Tables 1–3 summarise some convergence features forthe fully developed turbulent pipe flow problem. Notethat the maximum grid level for these cases was chosento be 3, allowing for relatively smaller-sized solutionblocks. Table 1 lists the numerical data from usingboth the regular and the preconditioned multigrid andthe results for grid-level and multigrid-cycle effects arepresented in Tables 2 and 3. The termWork Unit (WU) isdefined as the time for one RHS evaluation on the finestmesh.

Table 1 Matrix-preconditioner effects on convergence of the4-level V-cycle multigrid for the fully-developedturbulent pipe flow

Method CPU time [min] WUs Speedup

Multigrid 210.5 17542 1Preconditioned multigrid 15 1250 14

Table 2 Grid-level effects on convergence of the V-cyclepreconditioned multigrid for the fully-developedturbulent pipe flow


Single-level 35.8 2983 12-level 10.9 908 3.33-level 9.69 807.5 3.7

Table 3 The V- and W-cycle effects on convergence of the3-level preconditioned multigrid for thefully-developed turbulent pipe flow


W-cycle 38 3166.7 1V-cycle 22 1833.3 1.72

Table 1 indicates that the preconditioned multigridwith 5-stage optimal smoothing scheme produces a14 times speedup over multigrid without a preconditionerand is shown in Figure 8. For both cases, the L2norms of the residual for turbulent kinetic energy dropabout six orders of magnitude. The data from Table 2shows a speedup factor of four between the 3-levelmultigrid and the single-level computation. This grid-level influence on the convergence is shown in Figure 9.The convergence rate of 2-level is the same as that ofa 3-level for this case, and both used the V-cycle. Thesame convergence rates for both 2- and 3-level might bedue to the fact that the turbulence source terms werenot recalculated on the coarse meshes. Figure 10 showsthat the 3-level V- and W-cycle preconditioned multigridalgorithms, using a 5-stage optimal smoothing scheme,have nearly the same speedup in terms of multigridcycles, while Table 3 indicates the V-cycle uses abouthalf the computation time of the W-cycle. The reasonmight be that the W-cycle is expensive in a parallelalgorithm when frequent coarse-level calculations lead topoor processor utilisation. From these results, it appearsthat a 3-level V-cycle preconditioned multigrid shoulddeliver an optimal speed up for computing turbulentflows. For this reason, the 3-level V-cycle preconditionedmultigrid was employed in the numerical predictions ofthe turbulent reactive and non-reactive flows that follow.

4.4 Premixed laminar flame

Verification of the proposed parallel AMR schemefor reactive flows is carried out by considering thenumerical predictions of planar one-dimensionalpremixed methane-air laminar flames for a range ofequivalence ratios and comparing the predictions tothose obtained using the CHEMKIN program PREMIX.The six-species, two-step, reduced kinetic scheme forthe oxidation of methane described above is used.CHEMKIN is a commercial software tool availablefrom Reaction Design for solving complex chemical


kinetics problems and PREMIX is a utility that can beused for predicting one-dimensional premixed flames.A detailed 17-species, 58-reaction kinetic scheme is usedin the PREMIX calculations to represent the oxidationof methane. These comparisons provide a check of thealgorithms ability to predict two key features of laminarflames: the flame temperature and laminar flame speed.

Figure 8 Comparisons of the convergence rates between thepreconditioned multigrid and regular multigrid bothusing 3-level V-cycle with a 5-stage optimalsmoothing scheme for the fully-developed turbulentpipe flow

For the premixed flame predictions, a fixed (non-adapted)one-dimensional mesh with 400 non-uniformly spacecomputational cells is used. The steady state ortime-invariant structure of the flame is then obtained bystarting with uniform fresh and burnt gas solution statesat atmospheric and the adiabatic flame temperatures,respectively, and iterating until a steady-state solutionis achieved with a stationary flame structure. Theupstream and downstream boundary velocity andpressure are adjusted such that the mass flux is constantthroughout the domain.

The numerical results for the premixed laminar flameare summarised in Figures 11 and 12 and Table 4.

Figure 9 Comparisons between the grid-level effects onconvergence of the preconditioned V-cycle multigridwith a 5-stage optimal smoothing scheme for thefully-developed turbulent pipe flow

Figure 10 Comparisons between the V- and W-cyclepreconditioned multigrid convergence rates of a3-level preconditioned multigrid with 5-stageoptimal smoothing scheme for the fully-developedpipe flow


The table gives predictions of both the equilibriumtemperature of the products, T , and the laminarflame velocity, sL, as a function of the equivalenceratio (φ = 0.6, 0.8, 1.0, and 1.2). These figures depictthe predicted flame structure for φ = 1 and showvariation of the velocity, temperature, and mass fractionthrough the flame. The predicted laminar flame speedis sL = 40.6 cm/s in this case and the temperatureof the products (flame temperature) is T = 2256. Theoverall agreement between the two sets of results is verygood, especially considering that the six-species two-stepchemical kinetics scheme used by the parallel solveris greatly simplified in comparison to the 17-species,58-reaction scheme used in the CHEMKIN calculations.This provides strong support for the validity of theproposed reactive flow solver.

Figure 11 Velocity and temperature distributions of steadyone-dimensional premixed methane-air flamestructure for φ = 1

Figure 12 Species mass fraction distributions of steadyone-dimensional premixed methane-air flamestructure for φ = 1

It should be noted that the flow Mach numbers for thepremixed laminar flames are very small (M ≈ 0.001 −0.003) and the low-Mach-number preconditioning is

absolutely necessary for these cases in order to getaccurate predictions of the flame structure with theproposed compressible finite-volume formulation.

Table 4 Comparison of the predictions of the parallel AMRalgorithm using the two-step methane reducedmechanism to those of the CHEMKIN PREMIXprogram with detailed chemistry for variousequivalence ratios. Predictions of both theequilibrium temperature of the products, T , and thelaminar flame velocity, sL, are shown for φ = 0.6,0.8, 1.0, and 1.2

Equivalence ratio, φSolutionmethod 0.6 0.8 1.0 1.2

T (K) PREMIX 1656 1993 2234 2143Current 1650 1995 2256 2221

sL (cm/s) PREMIX 12.15 29.1 41.0 38.6Current 13.3 29.4 40.6 38.5

5 Results and discussions

The parallel AMR method is applied to solutions ofaxisymmetric co-flow methane-air laminar and turbulentdiffusion flames. The six-species, two-step, reducedkinetic scheme for the oxidation of methane is againused for the laminar diffusion flame calculations andthe five-species, one-step, reduced kinetic scheme for theoxidation of methane is used for turbulent diffusion flamecalculations.

5.1 Non-premixed laminar diffusion flame

The computational domain is rectangular in shape withdimensions of 10 cm by 5 cm. The axis of symmetryis aligned with the left boundary of the domain andthe right far-field boundary is taken to be a free-slipboundary along which inviscid reflection boundary datais specified. The top or outlet of the flow domain isopen to a stagnant reservoir at atmospheric pressure andtemperature and Neumann-type boundary conditions areapplied to all properties except pressure which is heldconstant. The bottom or inlet is subdivided into fourregions (a innermost region of the fuel inlet, a small gapassociated with the annular wall separating the fuel andoxidiser, a region of co-flowing oxidiser, and a far-fieldboundary along which free-slip boundary conditions areapplied). Additional details concerning the setup forthis diffusion flame can be found in the papers byMohammed et al. (1998) and Day and Bell (2000). Thesolution domain is initialised with a uniform solutionstate corresponding to quiescent air at 298 K, exceptfor a thin region across the fuel and oxidiser inlets,which is taken to be air at 1500 K so as to ignite theflame. Note that the Mach and Reynolds number basedon the fixed diluted methane flow in the fuel inlet areM = 0.0016 and Re = 169.


Figure 13 shows the computed isotherms and flamestructure obtained using a sequence of adaptively refinedgrids starting from the initial mesh (96 solution blockswith 3072 cells) and proceeding to the final mesh after fivelevels of refinement (396 blocks with 12,672 cells). Thesequence of adaptively refined grids, showing both thesolution blocks and computational cells, is also shown inFigure 14. The effect of the finer resolution can be clearlyseen, as the flame structure becomes much sharpenedand more resolved. Finally, Figure 15 shows the massfractions of the combustion products.

Figure 13 Solution of methane-air axisymmetric laminardiffusion flame showing the computed isothermsand flame structure obtained for a 396 block meshwith 12,672 cells and five levels of refinement.The sequence of adaptively refined grids, showingboth the solution blocks and computational cells,is also shown in the figure

A comparison of the results of Figures 13–15 with thosegiven in the previous studies by Mohammed et al. (1998)and Day and Bell (2000) reveals, that in spite of theinherent simplifications used in the two-step reactionmechanism, the predicted flame structure agrees very wellwith the previous work. The ‘wishbone’ structure of thehigh-temperature region is present and the computedlift-off and flame heights are 0.05 cm and 3.3 cm,respectively, with a maximum centre-line temperature of2080 K. All of these values agree reasonably well withthe previously published results. The predicted value ofthe carbon monoxide, CO, mass fraction concentrationat z = 3 cm along the centre-line is cCO = 0.026 and,considering the limitations of the reduced chemistrymechanism being used, is in reasonable agreement withthose of Mohammed et al. (1998), who report a massfraction of cCO = 0.03 at the same location.

5.2 Non-premixed turbulent diffusion flame

The International Workshops on Measurement andComputation of Turbulent Non-Premixed Flames (TNF)has established an internet library of well-documentedexperimental database for turbulent non-premixed

flames that are appropriate for combustion modelverification and validation. The Sydney bluff-bodyburner configuration shown in Figure 16 that forms partof this experimental database has data available for bothnon-reacting and reacting cases. The bluff-body has adiameter of Db = 50 mm and is located in a co-axial flowof air. Various gases can be injected through an orifice ofdiameter 3.6 mm at the base of the cylindrical bluff body.The bluff-body stabilised flames have a recirculationzone close to the base of the bluff body. This burnerconfiguration produces a relatively extensive and complexturbulent field and causes intense mixing between thereactants and combustion products. The stabilisationmechanisms resemble those of industrial combustors andyet the boundary conditions for the bluff-body flamesare simple and well-defined, making them well suited forinvestigating in great detail the capabilities of models forturbulent non-premixed diffusion flames.

Figure 14 Predicted laminar diffusion flame temperature (K)comparison for three different levels of meshrefinement

Figure 15 Predicted mass fractions of products CO2, H2O,and CO for laminar diffusion flames

5.2.1 Bluff-body burner non-reacting flows

In the cold non-reacting bluff-body burner flow case,air is injected through the orifice at the base of thecylindrical bluff body with a temperature of 300 Kand a bulk velocity of 61 m/s. The bulk velocity andtemperature of the co-flowing air are 20 m/s and 300 K,respectively. The Reynolds and Mach numbers based onthe high-speed jet are Re = 193,000 and Ma = 0.18.


Figure 16 Schematic of the Sydney bluff-body burnershowing the fuel jet, co-flow, and bluff-bodygeometry

We have examined predicted solutions on a sequence ofrefined meshes to establish the grid-convergence of thesolution. The flow field predictions have been computedon a sequence of adaptively refined grids, 5 16 × 16 cellblocks and 1280 cells, 14 16 × 16 cell blocks and 3584cells, 26 16 × 16 cell blocks and 6656 cells, and 53 16 × 16cell blocks and 13,568 cells. The mesh resolution was suchthat the typical size of the computational cells nearest thewall was in the range 0.2 < y+ < 1. It is apparent that themajority of the solution parameters do not change as themesh is refined from 6656 cells to 13,568 cells, as shownin Figures 20–23. The numerical solutions can be said tobe virtually grid independent.

Figure 17 Predicted flow velocity and streamlines fornon-reacting flow field of bluff-body burner

Figure 17 shows the predicted mean velocity andstreamlines and reveal the formation of a double-vortex structure in the re-circulation zone whichare important in controlling fuel/oxidiser mixing.

The calculations indicate that the re-circulation zoneextends to x/Db ≈ 0.8. This is slightly less than theexperimentally observed value of x/Db ≈ 1.0. Theagreement between the predictions and experiment isfurther confirmed by a comparison of the predictedradial profiles of the mean axial velocity componentat x/Db = 0.6 and x/Db = 1.0 downstream from thebase of the bluff-body to the measured data as shownin Figures 18 and 19, and by a comparison ofthe predicted axial (centre-line) profile of the meanaxial velocity component to experimental results asdepicted in Figure 20. The predicted Root Mean Square(RMS) fluctuations of the velocity components andspecific Reynolds stress (u′v′) are also compared to theexperimental data in Figures 21–23. It can be seen thatthere are under- and/or over-predicted regions (r/Rb <0.2). These regions encompass the inner vortex and thevicinity of the outer vortex of a double-vortex structurein the re-circulation zone. Re-circulation zones withcomplex turbulent structures are quite sensitive to theturbulence modelling and a variety of RANS simulationshave addressed this sensitivity to the turbulence modeland/or combustion models (Dally et al., 1998; Xu andPope, 2000; Merci et al., 2001; Liu et al., 2005). Theoverall agreement between the numerical solution and theexperimental data for these turbulence quantities is quitereasonable and is comparable to other results reported inthe literature (Dally et al., 1998; Turpin and Troyes, 2000;Merci et al., 2001).

Figure 18 Comparison of predicted and measured axialvelocity component at x/Db = 0.6 downstreamfrom the base of the bluff-body for non-reactingbluff-body burner with air jet

Figure 19 Comparison of predicted and measured axialvelocity component at x/Db = 1.0 downstreamfrom the base of the bluff-body for non-reactingbluff-body burner with air jet


Figure 20 Comparison of predicted and measured on-axisaxial profiles of the mean axial velocity componentdownstream from the base of the bluff-body fornon-reacting bluff-body burner with air jet

Figure 21 Comparison of predicted and measured

√u′2 at

x/Db = 0.6 downstream from the base of thebluff-body for non-reacting bluff-body burnerwith air jet

Figure 22 Comparison of predicted and measured√

v′2 atx/Db = 0.6 downstream from the base of thebluff-body for non-reacting bluff-body burnerwith air jet

Figure 23 Comparison of predicted and measured u′v′ atx/Db = 0.6 downstream from the base of thebluff-body for non-reacting bluff-body burnerwith air jet

For the ethylene jet case, ethylene (C2H4) is injectedat the base of the bluff-body with a velocity of 50 m/s

and a temperature of 300 K. In this case, the Reynoldsand Mach numbers based on the ethylene flow areRe = 145,000 and Ma = 0.11. Numerical results for theethylene fuel jet are depicted in Figure 24, where thepredicted mass fraction of C2H4 obtained using a meshconsisting of 479 6 × 6 cell blocks and 17,244 cells, withfive levels of refinement, is compared to measured C2H4concentrations. The mesh resolution was also such thatthe typical size of the computational cells nearest thewall was in the range 0.2 < y+ < 1. The predictionsof the mixing field (Fig. 24) also appear to be quitereasonable when compared to the experimental data.Detailed comparisons of the predicted on-axis axial andradial distributions at x/Db = 1.0 of the mean C2H4mass fraction to measured values given in Figures 25and 26 also indicate that the fuel and oxidiser mixingprocess is quite well reproduced.

Figure 24 Predicted mean C2H4 mass fraction fornon-reacting flow field of bluff-body burner

Figure 25 Predicted axial profile of the mean C2H4 massfraction for non-reacting flow field of bluff-bodyburner

Figure 26 Predicted radial profile of the mean C2H4 massfraction at x/Db = 1.0 for non-reacting flow fieldof bluff-body burner


5.2.2 Bluff-body burner reacting flow

For the reacting case, methane (CH4) is injected throughthe orifice at the base of the cylindrical bluff bodywith a temperature of 300 K. The bulk velocities of theco-flowing air and methane fuel are 25 m/s and 108 m/s,respectively. The Reynolds and Mach numbers of themethane jet are Re = 315,000 and Ma = 0.24.

Computations were carried out on a sequence ofadaptively refined grids, 7 16 × 16 cell blocks and 1792cells, 28 16 × 16 cell blocks and 7168 cells, 70 16 × 16 cellblocks and 17,920 cells, 97 16 × 16 cell blocks and 24,832cells, and 112 16 × 16 cell blocks and 31,744 cells toassess the grid independence of the predictions. As in thenon-reacting cases, the mesh resolution was such that thetypical size of the computational cells nearest the wall wasin the range 0.2 < y+ < 1. The results of the refinementstudy are shown in Figures 27–30, and the majority of thesolution parameters, such as, axial velocity, temperature,and major species CO2, do not change as the meshis refined from 24,832 cells to 31,744 cells. The gridconvergence solution is achieved.

Figure 27 Predicted mean axial velocity along the centrelineof the bluff-body for reacting bluff-body burner

Figure 28 Predicted mean axial velocity at x/Db = 1.92downstream from the base of the bluff-body forreacting bluff-body burner

Figure 29 Predicted mass fraction of CO2 at x/Db = 1.92downstream from the base of the bluff-body forreacting bluff-body burner

Figure 30 Predicted mass temperature at x/Db = 1.92downstream from the base of the bluff-body forreacting bluff-body burner

Figure 31 shows the predicted distributions of meantemperature and mean mass fraction of CO2 forthis turbulent non-premixed flame. The predictedflame structure is generally in agreement with theexperimentally observed structure. The flame is quiteelongated and three zones can be identified: there-circulation, neck, and jet-like propagation zones.A vortex structure is formed in the re-circulation zoneand acts to stabilise the flame. The maximum flametemperature is about 2180 K. The predicted meantemperature, 1350 K, and mass fraction of CO2, 0.1,at location of (x/Db = 1.92, r/Rb = 0.4) are comparedto the measured values of the flame temperature,1120 K, and carbon dioxide concentration, 0.07. Thetemperature and hence carbon dioxide concentration aresomewhat over-predicted. However, the agreement withthe experimental values is reasonable considering thelimitations of the simplified reduced chemical kineticsscheme and turbulence/chemistry interaction model usedherein, as well as the fact that radiation transport is nottaken into account in the simulation.

Figure 31 Predicted mean temperature and CO2 massfraction for reacting flow field of bluff-body burner

5.3 Multigrid acceleration

The numerical solutions for both the cold andhot cases of the two-dimensional axisymmetricbluff-body burner flows were obtained using thepreconditioned multigrid technique. The 3-level V-cyclepreconditioned multigrid with a 3-stage optimallysmoothing scheme was employed. A prolongationoperator, based on a cell-aspect-ratio sensor as


discussed above, was also applied. For those cellswith aspect ratios greater than a value of 1000,a simple injection was used. Standard bi-linearinterpolation was employed for cells with aspectratios smaller than this value. The CFL numberwas 0.2.

Both the multigrid and preconditioned multigridalgorithms speed up the convergence rates to thesteady-state solutions quite significantly for bothcases as compared to the convergence rate achievedusing the semi-implicit time-marching method alone(i.e., smoother alone without the multigrid procedure).The preconditioned multigrid seems to have a morepositive effect for the reacting case than for thenon-reacting problem. Although the convergence rateslows after the residual has been reduced by 4–5 ordersof magnitude, and this slow down may be associated withlimitations of the proposed multigrid method when AMRis used, overall it is felt that satisfactory convergencerates have been achieved. The preconditioned multigridprovides a speedup of about two in terms of CPU timeover regular multigrid for both cases.

5.4 Parallel performance

Parallel speedup (strong scaling), parallel scale-up(weak scaling), and parallel efficiency are often usedto measure/evaluate the parallel performance of aparallel algorithm. The parallel speedup, Sp, is definedas Sp = t1/tp, the parallel scale-up, Sφ, defined asSφ = t1/tp p, and the parallel efficiency, Ep is defined asEp = Sp/p, where t1 is the time required to solve theproblem by a single processor, and tp is the time requiredto solve the problem by p processors. While both theparallel speedup and the parallel scale-up are importantto consider, the parallel speedup is probably morerelevant for engineering problems of practical interest.High efficiency for strong scaling is generally harderto achieve than for the weak scaling problem. Herein,the parallel speedup and efficiency of the proposedparallel solution-adaptive algorithm applied to three flowproblems (a laminar diffusion flame, a turbulent pipeflow, and a turbulent diffusion flame) have been assessed.

For the laminar diffusion flame problem, Figure 32shows that the parallel speed-up of the block-basedAMR scheme is linear and is 90% efficient for up to32 processors using the larger (10 × 10) solution blocks.For the smaller (8 × 8) blocks, the efficiency dropsslightly down to 80% efficient. Figure 33 shows that theparallel speedup of the block-based AMR scheme forthe turbulent pipe flow problem is nearly linear and isat least 90% efficient for up to 32 processors using thelarger (10 × 10) solution blocks. For the smaller (8 × 8)blocks, the efficiency drops slightly down to 87% efficient.The rather high level of performance should generallybe expected for the two-dimensional algorithm with theexplicit time marching scheme.

The parallel performance of the proposed algorithm isfurther assessed for a fixed-size turbulent diffusion flame

problem using up to 64 processors. An added differenceof this parallel performance assessment from the previousones is the use of 3-level V-cycle preconditioned multigridtechnique in the computations. It can be observed inFigure 34 that the proposed scheme provides a nearlylinear speedup and is about 76% efficient for up to64 processors using the larger 24 × 24 cell solutionblocks. For the smaller 16 × 16 cell solution blocks,the parallel efficiency drops to 68%. Compared to theestimation shown in Figure 33, the parallel efficiency issomewhat reduced. The performance is affected by thecoarse grid calculations of the multigrid algorithm. Asmentioned earlier, the number of inter-block messages onthe coarse meshes is often nearly the same as that on thefine meshes, thereby decreasing the ratio of computationto communication and adversely affecting the parallelperformance.

Figure 32 Scaled parallel speed-up and parallel efficiencyfor a fixed-size laminar diffusion flame problemusing up to 32 processors

Figure 33 Parallel speedup (strong scaling), Sp and theparallel efficiency, Ep, for a fixed size problemusing up to 32 processors (see online versionfor colours)


Figure 34 Parallel speedup (strong scaling) and efficiency forcomputation of a two-dimensional turbulentdiffusion flame problem with 3-level V-cyclepreconditioned multigrid using up to 64 processors

6 Conclusions

A highly scalable parallel AMR scheme has beendescribed for non-premixed combusting flows. Thecombination of a block-based AMR strategy andparallel implementation has resulted in a powerfulcomputational tool, as demonstrated by the numericalresults for both laminar and turbulent non-premixedflames. The predicted flame structure of an axisymmetricco-flow methane-air laminar diffusion flame, includingthe computed lift-off, flame heights and the maximumcentre-line temperature, agrees reasonably well with thepreviously published results. A quantitative evaluation ofthe parallel AMR algorithm has also been carried outfor a complex turbulent combusting flow in a bluff-bodyburner having a relatively complex physical geometry.For complex turbulent combusting flows, dealing withthe near-wall turbulence is challenging within anAMR procedure. This study proposed a somewhatnovel automatic and smooth switching procedure forcomputing wall turbulence that is well suited to the AMRscheme considered here. In order to provide enhancedconvergence for steady-state problems, a preconditioned(matrix preconditioner) multigrid strategy has beenproposed and developed for two-dimensional turbulentcombustion calculations. It is thought to be one ofthe first applications of a parallel AMR scheme withmultigrid to turbulent-combusting flow calculations inthe literature, and significantly improved convergencewas achieved by devising a cell-aspect-ratio-basedprolongation operator for treating highly stretchedmeshes. This numerical study demonstrates the validityand potential of the parallel AMR approach forpredicting fine-scale features of complex turbulentnon-premixed flames.

Acknowledgement

The first author is very grateful to Professor ClintonP.T. Groth and University of Toronto for thefinancial support provided throughout this research.Computational resources for performing all of thecalculations reported herein were provided by theSciNet High Performance Computing Consortium atthe University of Toronto and Compute/Calcul Canadathrough funding from the Canada Foundation forInnovation (CFI) and the Province of Ontario, Canada.

References

Aftosmis, M.J., Berger, M.J. and Melton, J.E. (1998) ‘Robustand efficient cartesian mesh generation for component-base geometry’, AIAA Journal, Vol. 36, pp.952–960.

Aftosmis, M.J., Berger, M.J. and Murman, S.M. (2004)Applications of Space-Filling Curves to CartesianMethods for CFD, AIAA Paper, 2004–1232.

Barth, T.J. (1993) Recent Developments in High Order K-exactReconstruction on Unstructured Meshes, AIAA Paper,93-0668.

Bell, J.B., Day, M.S., Almgren, A.S., Lijewski, M.J. andRendleman, C.A. (2001) ‘Adaptive numerical simulationof turbulent premixed combustion’, Proceedings of theFirst MIT Conference on Computational Fluid andSolid Mechanics, June, Cambridge, Massachusetts, USA,pp.1073–1076.

Bell, J.B., Day, M.S., Almgren, A.S., Lijewski, M.J. andRendleman, C.A. (2002a) ‘A parallel adaptive projectionmethod for low mach number flows’, InternationalJournal for Numerical Methods in Fluids, Vol. 40,pp.209–216.

Bell, J.B., Day, M.S. and Grcar, J.F. (2002b) ‘Numericalsimulation of premixed turbulent methane combustion’,Proceedings of the Combustion Institute, Vol. 29,pp.1987–1993.

Berger, M.J. (1984) ‘Adaptive mesh refinement for hyperbolicpartial differential equations’, Journal of ComputationalPhysics, Vol. 53, pp.484–512.

Berger, M.J. and Colella, P. (1989) ‘Local adaptive meshrefinement for shock hydrodynamics’, Journal ofComputational Physics, Vol. 82, pp.67–84.

Berger, M.J. and Saltzman, J.S. (1994) ‘AMR on the CM-2’,Applied Numerical Mathematics, Vol. 14, pp.239–253.

Bussing, T.R.A. and Murman, E.M. (1987) ‘Numericalinvestigation of two-dimensional H2-air flameholding overramps and rearward-facing steps’, J. Propulsion, Vol. 3,pp.448–454.

Bussing, T.R.A. and Murman, E.M. (1988) ‘Finite-volumemethod for the calculation of compressible chemicallyreacting flows’, AIAA Journal, Vol. 26, pp.1070–1078.

Coirier, W.J. and Powell, K.G. (1996) ‘An adaptively-refined, Cartesian, cell-based scheme for the Euler andNavier-Stokes equations’, AIAA Journal, Vol. 34, No. 5,May, pp.938–945.


Dally, B.B., Fletcher, D.F. and Masri, A.R. (1998) ‘Flow andmixing fields of turbulent bluff-body jets and flames’,Combustion Theory and Modeling, Vol. 2, pp.193–219.

Day, M.S. and Bell, J.B. (2000) ‘Numerical simulationof laminar reacting flows with complex chemistry’,Combustion Theory and Modelling, Vol. 4, No. 4,pp.535–556.

De Zeeuw, D. and Powell, K.G. (1993) ‘An adaptively refinedCartesian mesh solver for the euler equations’, Journal ofComputational Physics, Vol. 104, pp.56–68.

Einfeldt, B. (1988) ‘On godunov-type methods for gasdynamics’, SIAM Journal on Numerical Analysis, Vol. 25,pp.294–318.

Gao, X. (2008) A Parallel Solution-Adaptive Method forTurbulent Non-Premixed Combusting Flows, PhD Thesis,University of Toronto, Toronto, Canada.

Gerlinger, P. and Brüggemann, D. (1997) ‘Multigridconvergence acceleration for turbulent supersonic flows’,International Journal for Numerical Methods in Fluids,Vol. 24, No. 10, pp.1019–1035.

Gerlinger, P., Möbus, H. and Brüggemann, D. (2001)‘An implicit multigrid method for turbulent combustion’,Journal of Computational Physics, Vol. 167, pp.247–276.

Gerlinger, P., Stoll, P. and Brüggemann, D. (1998) ‘An implicitmultigrid method for the simulation of chemically reactingflows’, Journal of Computational Physics, Vol. 146,pp.322–345.

Gombosi, T.I., Powell, K.G. and De Zeeuw, D.L. (1994)‘Axisymmetric modeling of cometary mass loading onan adaptively refined grid: MHD results’, Journal ofGeophysical Research, Vol. 99, pp.21525–21539.

Gordon, S. and McBride, B.J. (1994) Computer Programfor Calculation of Complex Chemical EquilibriumCompositions and Applications I. Analysis, ReferencePublication 1311, NASA, Cleveland, Ohio, USA.

Groth, C.P.T., De Zeeuw, D.L., Gombosi, T.I. andPowell, K.G. (2000) ‘Global three-dimensional MHDsimulation of a space weather event: CME formation,interplanetary propagation, and interaction with themagnetosphere’, Journal of Geophysical Research,Vol. 105, No. A11, pp.25053–25078.

Groth, C.P.T., De Zeeuw, D.L., Marshall, H.G., Gombosi, T.I.,Powell, K.G. and Stout, Q.F. (1999) A Parallel Solution-Adaptive Scheme for Ideal Magnetohydrodynamics,AIAA Paper, 99-3273, June.

Gropp, W., Lusk, E. and Skjellum, A. (1999) Using MPI,MIT Press, Cambridge, Massachussets.

Laufer, J. (1954) The Structure of Turbulence in FullyDeveloped Pipe Flow, Report 1174, NACA.

Linde, T. (2002) ‘A practical, general-purpose, two-stateHLL Riemann solver for hyperbolic conservation laws’,International Journal for Numerical Methods in Fluids,Vol. 40, pp.391–402.

Liu, K., Pope, S.B. and Caughey, D.A. (2005) ‘Calculationsof bluff-body stabilized flames using a joint probabilitydensity function model with detailed chemistry’,Combustion and Flame, Vol. 141, pp.89–117.

McBride, B.J. and Gordon, S. (1996) Computer Programfor Calculation of Complex Chemical EquilibriumCompositions and Applications II. Users Manual andProgram Description, Reference Publication 1311, NASA.

MacNeice, P., Olson, K.M., Mobarry, C., de Fainchtein, R.and Packer, C. (2000) ‘PARAMESH: A parallel adaptivemesh refinement community toolkit’, Computer PhysicsCommunications Vol. 126, pp.330–354.

Magnussen, B.F. and Hjertager, B.H. (1976) ‘On mathematicalmodeling of turbulent combustion with special emphasison soot formation and combustion’, 16th Symp. (Int.)on Combustion Comb. Inst., Pittsburg, Pennsylvania,pp.719–729.

Mason, E.A. and Saxena, S.C. (1958) ‘An approximate formulafor the thermal conductivity of multicomponent gasmixtures’, Phys. Fluids, Vol. 1, pp.361–369.

Mavriplis, D.J. (1998) ‘Multigrid strategies for viscous flowsolvers on anisotropic unstructured meshes’, Journal ofComputational Physics, Vol. 145, pp.141–165.

Merci, B., Dick, E., Vierendeels, J., Roekaerts, D. andPeeters, T.W.J. (2001) ‘Application of a new cubicturbulence model to piloted and bluff-body diffusionflames’, Combustion and Flame, Vol. 126, pp.1533–1556.

Mohammed, R.K., Tanoff, M.A., Smooke, M.D.,Schaffer, A.M. and Long, M.B. (1998) ‘Computationaland experimental study of a forced, time-varying,axisymmetric, laminar diffusion flame’, Twenty-SeventhSymposium (International) on Combustion, TheCombustion Institute, Pittsburgh, pp.693–702.

Pierce, N.A. and Giles, M.B. (1996) PreconditioningCompressible Flow Calculations on Stretched Meshes,AIAA Paper, 96-0889.

Pierce, N.A. and Giles, M. (1997) ‘Preconditioned multigridmethods for compressible flow calculations on stretchedmeshes’, Journal of Computational Physics, Vol. 136,pp.425–445.

Powell, K.G., Roe, P.L. and Quirk, J.J. (1993) ‘Adaptive-mesh algorithms for computational fluid dynamics’,in Hussaini, M.Y., Kumar, A. and Salas, M.D. (Eds.):Algorithmic Trends in Computational Fluid Dynamics,Springer-Verlag, New York, pp.303–337.

Quirk, J.J. (1991) An Adaptive Grid Algorithm forComputational Shock Hydrodynamics, PhD Thesis,Cranfield Institute of Technology, Bedford, Cranfield,England, January.

Quirk, J.J. and Hanebutte, U.R. (1993) A Parallel AdaptiveMesh Refinement Algorithm, Report 93-63, ICASE,August.

Roe, P.L. (1981) ‘Approximate Riemann solvers,parameter vectors, and difference schemes’, Journal ofComputational Physics, Vol. 43, pp.357–372.

Sheffer, S.G., Martinelli, L. and Jameson, A. (1998)‘An efficient multigrid algorithm for compressible reactiveflows’, J. Comput. Phys., Vol. 144, pp.484–516.

Smith, G.P., Golden, D.M., Frenklach, M., Moriarty, N.W.,Eiteneer, B., Goldenberg, M., Bowman, C.T.,Hanson, R.K., Song, S., Gardiner, W.C., Jr.,Lissianski, V.V. and Qin, Z. GRI-Mech 3.0,http://www.me.berkeley.edu/gri_mech/

Stroustrup, B. (2000) The C++ Programming Language,4th ed., Addison-Wesley, Reading Mass, USA.

Turkel, E. (1999) ‘Preconditioning techniques in computationalfluid dynamics’, Annual Review of Fluid Mechanics,Vol. 31, pp.385–416.


Turpin, G. and Troyes, J. (2000) Validation of a Two-EquationTurbulence Model for Axisymmetric Reacting andNon-Reacting Flows, AIAA Paper, 2000-3463.

van Leer, B., Tai, C.H. and Powell, K.G. (1989) Design ofOptimally-Smoothing Multi-Stage Schemes for the EulerEquations, AIAA Paper, 89-1933-CP.

Venkatakrishnan, V. (1993) On the Accuracy of Limitersand Convergence to Steady State Solutions, AIAA Paper,93-0880.

Weiss, J.M. and Smith, W.A. (1995) ‘Preconditioning appliedto variable and constant density flows’, AIAA Journal,Vol. 33, No. 11, pp.2050–2057.

Westbrook, C.K. and Dryer, F.L. (1981) ‘Simplified reactionmechanisms for the oxidation of hydrocarbon fuels inflames’, Combustion Science and Technology, Vol. 27,pp.31–43.

Wilcox, D.C. (2002) Turbulence Modeling for CFD,DCW Industries, La Cañada.

Wilke, C.R. (1950) ‘A viscosity equation for gas mixtures’,Journal of Chemical Physics, Vol. 18, pp.517–519.

Xu, J. and Pope, S.B. (2000) ‘PDF calculations of turbulentnonpremixed flames with local extinction’, Combustionand Flame, Vol. 123, pp.281–307.

Website

TNF, http://www.ca.sandia.gov/TNF/

pcfd j.title 27-08-08arrow.utias.utoronto.ca/~groth/publications/pcfd-2011-gao.pdf · fax: 1...

Documents