![Page 1: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/1.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
Large-eddy simulation of the flow over a circular cylinder at Reynolds number 3900
Dmitry A. Lysenko1
Ivar S. Ertesvåg1
Kjell Eric Rian2
1Norwegian University of Science and Technology, Trondheim, Norway2Computational Industry Technologies AS, Trondheim, Norway
![Page 2: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/2.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
Agenda
• Motivation & background• Test case description• Numerical aspects and HPC• Results for ReD = 3900
• Results for ReD = 20000• Concluding remarks
![Page 3: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/3.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
Motivation & background (I)
• Advancing the numerical modeling of interactions between turbulent flow and chemical kinetics for Large-eddy simulation, with special emphasis on biomass combustion, efficiency and pollutant formation• The Eddy Dissipation Concept (EDC) model of turbulent combustion has been developed by Bjørn F. Magnussen and coworkers at NTNU/SINTEF over nearly four decades• EDC has become the standard model for turbulent combustion almost in all commercial CFD codes • EDC is most widely-used in engineering due to its robustness, relatively easy implementation in CFD codes, tractable with complex chemistry and low computational cost • Although considerable success, there is still potential for further development ☺
![Page 4: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/4.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
• Excessive validation (I) - Turbulent 2D bluff-body test problems• Intensive comparison with ANSYS FLUENT• Good agreement between OpenFOAM, FLUENT, DNS and other experimental/numerical data • Lysenko, D.A., Ertesvåg, I.S. and Rian K.E., Modeling of turbulent separated flows using OpenFOAM. Comput Fluids (2012), doi:10.1016/j.compfluid.2012.01.015
Flow visualization of the vortex shedding behind a triangular rod: a – the experimental interferogram taken from Nakagawa 2005; b – numerical interferogram (OF)
The contours of instantaneous radiated density gradient field for the laminar flow over a circular cylinder at Re =140
Motivation & background (II)
![Page 5: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/5.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
Test case descriptionLES of the flow over a circular cylinder at Reynolds number ReD=3900
![Page 6: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/6.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
NRBC
Laminar
General scheme (a), description of the grid (b) and zoom into the cylinder region (c): θ is the the circumferential coordinate, x and y are the domain extents in stream-wise and transverse directions
Iso-thermal, non-slipping
• O-type grid• Lx = Ly = 50 × D
• Lz = π × D
Test case description (I)
Run Re M Mesh SGS model
1 3900 0.2 300 × 300 × 64 Conventional Smagorinsky
2 3900 0.2 300 × 300 × 64 Dynamic k-equation
3 20000 0.2 300 × 300 × 64 Dynamic k-equation
4 20000 0.2 440 × 440 × 64 Dynamic k-equation
5 20000 0.2 440 × 440 × 64 Conventional Smagorinsky
![Page 7: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/7.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
Test case description (II)Experimental and DNS references
• PIV by Lourenco & Shih (1993)• PIV by Parnaudeau et al. (2008)• HWA by Ong & Wallace (1996)• HWA by Norberg (1994, 2001)• DNS by Ma et al. (2000)• DNS by Dong et al. (2006) • DNS by Wissink & Rodi (2008) for ReD = 3300 [1206 × 406 × 1024]
• & other LES studies as well
![Page 8: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/8.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
Numerical aspects& High performance computing
![Page 9: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/9.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
Numerical method (I)
• OpenFOAM 1.7/2.1• LES approach
• Dynamic turbulence kinetic energy equation SGS model (TKE hereafter) • Conventional Smagorinsky SGS model with default constants (Cs=0.053)
(SMAG hereafter)• Wall-resolving LES (Y+~1)
• Spatial discretization• rhoPisoFOAM• 2nd order central-differences (CDC2) for viscous terms• 4th order for inviscid terms and pressure gradient
• Temporal discretization• 2nd order implicit backward Euler method (BDF2)• Dynamic adjustable time stepping (CFL<1)
• Linear algebra & accuracy• ICCG (1x10-7)
![Page 10: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/10.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
Numerical method (II)• Thermodynamics
• Ideal gas (sub-grid scales incompressibility hypothesis)• Const for viscosity and other properties
• Boundary conditions• Laminar for inlet / NRBC for outlet• Isothermal non-slip for walls
• Visualization • λ2 (native Foam utility) • Numerical schlieren method [Hadjadj & Kudryavtsev 2005]
• Spectral analysis• MatLab wavelet toolbox• 1D continues wavelet analysis (Morlet wavelet)
[Farge 1992, Addison et al. 2001 … ]• Welch periodogram technique (adaptive-time stepping) [Welch 1967]
![Page 11: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/11.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
HPC (I)
• OpenFOAM is PC software that has been extended to work on supercomputers.• The source code is cluttered with non-MPI versions of libraries• MPI programs normally:
• Scatter input data from process 0 to other processes with MPI• Loop
• Calculate• Synchronize processes with MPI
• Gather partial results on process 0 with MPI• OpenFOAM does:
• decomposePar - creates files with the initial data for each process• rhoPisoFoam - loop, stores partial results in files• reconstructPar - reads the many files and creates end result files
![Page 12: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/12.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
HPC (II)OpenFOAM – scalability study done in PRACE(Thanks to HPC @NTNU team)
«RunTimeModifiable» set to «No»
Number of processes 64 128 256 512 1024
Execution time [s] 686 801 890 1161 2048
Time used for metadatahandling [s]
64 202 274 389 892
The share of time used on Meta datahandling
9.3% 25% 31% 34% 39%
Number of processes 64 128 256 512 1024
Execution time [s] 542 381 343 411 N/A
Time used for metadatahandling [s]
0.99 2.62 6.03 14.4 N/A
The share of time used on Meta datahandling
0.2% 0.7% 1.8% 3.5% N/A
![Page 13: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/13.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
HPC (III)OpenFOAM – scalability study done in PRACE(Thanks to HPC @NTNU team)
Prohibits scaling Large difficulties for high quality visualization HPC is the next big issue after documentation (!?!)
Number of processes 64 128 256 512 1024
Number of files created 512 1024 2048 4096 8192
Number of files read 1089 2177 4353 9729 17409
Average file size 597K 317K 163K 84K 47K
Number of stat()-calls 500 000 1000 000 2 000 000 4 400 000 8 500 000
‘RunTimeModifiable: No’
Number of stat()-calls 5000 10 000 20 000 44 000 N/A
![Page 14: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/14.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
HPC (IV)Next generation NTNU HPC facility
Strong scalability study N eff = 8 of 16 cores per node
Strong scalability per one nodeNumber of cores 1 2 4 8 16
Execution time [s] 188 96 83 46 35
Speedup 1 2 2.3 4.1 5.4
Ideal speedup 1 2 4 8 16
VILJE@NTNU SGI Altix 8600 cluster
Number of nodes 1440
Number of cores 23040
CPU type Intel Sandy Bridge
Cores/node 2x8
Memory/node 32 Gb
Network FDR Infiniband
![Page 15: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/15.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
Results for Re @ 3900
![Page 16: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/16.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
Instantaneous flow fieldTime history of the lift and drag coefficients for SMAG and TKE models for
the flow over a circular cylinder at Re = 3900
Sampled statistics (150 vortex shedding cycles)
The Smagorinsky model:
St = 0.19
<Cd> = 1.18
(Cl)rms = 0.44
The dynamic k-equation model:
St = 0.209
<Cd> = 0.97
(Cl)rms = 0.09
![Page 17: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/17.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
Mean flow features
Mean pressure coefficient (a) and mean, normalized vorticity magnitude (b) distributions on the cylinder surface (θsep = 0 is the stagnation point) for the flow over a circular cylinder at Re = 3900
Mean streamlines in the x-y plane by TKE (a) and SMAG (b) runs for the flow over a circular cylinder at Re = 3900
Mean streamlines in the x-y plane from DNS run by Wissink & Rodi (2008) for the flow over a circular cylinder at Re = 3300 (from private communication)
θ=89° (SMAG) θ=88° (TKE)
<Cp,b> = -0.8 (SMAG) <Cp,b> = -0.91 (TKE)
<Lr>/D = 1.588 (DNS) <Lr>/D = 1.67 (TKE) <Lr>/D = 0.97 (SMAG)
a
b
![Page 18: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/18.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
First order statistics
x/D = 1.06
x/D = 1.54
x/D = 2.02
x/D = 4.00
x/D = 7.00 x/D = 10.00
x/D = 1.06
x/D = 1.54
x/D = 2.02
x/D = 4.00
x/D = 7.00 x/D = 10.00
Mean stream-wise velocity and its fluctuations at different locations in the wake of a circular cylinder at Re = 3900
TKE solution agrees fairly well with PIV by Parnaudeau et al. 2008 and DNS by Wissink & Rodi 2008 (U shape)
SMAG solution agrees fairly well with HWA by Lourenco & Shih 1993 (V shape)
![Page 19: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/19.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
Spectral analysis (I)Vortex shedding and shear-layer instability
c d
The Iso-surfaces of λ2 and density gradient fields (numerical schlieren method) obtained by TKE (a,c) and SMAG (b,d) models for the flow over a circular cylinder at Re = 3900
![Page 20: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/20.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
Spectral analysis (II)One-dimensional spectra
x/D = 5; y/D = 0
x/D = 3; y/D = 0
![Page 21: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/21.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
Spectral analysis (III)Wavelet signatures Fundamental Strouhal frequency (fvs) in the wake at x/D=3, y/D = 0
Shear-layer (Kelvin–Helmholtz) frequency (fsl) at the point x/D = 0.69, y/D = 0.69
fsl/fvs = 7.6 (TKE & SMAG)
fsl/fvs = 7.83 (DNS by Dong et al. 2006)
fsl/fvs = 5.99 (Exp by Prasad and Williamson 1996)
LES-TKE LES-SMAG
![Page 22: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/22.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
Parameters and integral flow features
![Page 23: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/23.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
Results for Re @ 20000
Run Re M Mesh SGS model
1 3900 0.2 300 × 300 × 64 Conventional Smagorinsky
2 3900 0.2 300 × 300 × 64 Dynamic k-equation
3 20000 0.2 300 × 300 × 64 Dynamic k-equation
4 20000 0.2 440 × 440 × 64 Dynamic k-equation
5 20000 0.2 440 × 440 × 64 Conventional Smagorinsky
![Page 24: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/24.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
Flow visualization
Shear-layer instability at Re = 20000λ2 Iso-surface for the flow over a circular cylinder at Re = 20000
Iso-surface of the fluctuating pressure at p’ = −0.1 for the flow over a circular cylinder at Re = 20000
![Page 25: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/25.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
Spectral analysis
Shear-layer frequency (fsl) in the shear layer
Strouhal frequency (fvs) in the wake at x/D=3, y/D = 0
fsl/ /fvs (Re) = C1 × Re0.67
Variation of normalized shear-layer frequency with Reynolds number. Data from Prasad & Williamson 1997
fsl/ /fvs (Re) = C2 × Re0.41
fsl/fvs = 15.08
LES-TKE
Strouhal frequency (fvs) in the wake at x/D=3, y/D = 0
![Page 26: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/26.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
Summary
![Page 27: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/27.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
Concluding remarks (I)Quality & Reliability (LES-IQ)
Index of resolution quality for LES (Celik et al. 2005)• Based on Richardson extrapolation• Sufficient LES resolution ~80% of the TKE (Pope, 2000)• Two sets of grids: 300×300×64 & 440×440×64
![Page 28: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/28.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
Concluding remarks (II)
• Excessive validation effort was carried out for LES of the flow over a circular cylinder at ReD = 3900• Good agreement with recent experimental data and DNS data was achieved as well as fulfilled physics entitlement • The main results were submitted to the Flow, Turbulence & Combustion
(DOI: 10.1007/s10494-012-9405-0)• The next effort will be extended for ReD = 20000 and ReD = 140000
![Page 29: Large-eddy simulation of the flow over a circular cylinder ... · PDF file7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012 Large-eddy simulation of the flow over a circular](https://reader034.vdocument.in/reader034/viewer/2022051721/5a7a29947f8b9adf778d5f77/html5/thumbnails/29.jpg)
7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012
Acknowledgements
• This work was conducted as a part of the CenBio Center for environmentally-friendly energy
• We are very thanked for the uninterrupted HPC computational resources and the useful technical support provided by the Norwegian Meta center for Computational Science (NOTUR).
• Special thanks to Bjørn Lindi & Henrik Nagel from HPC @NTNU team