asme v&v 40

ASME V&V 40.4 Code VerificationExample Problem

Marc Horner Ismail GulerANSYS, Inc. Boston Scientific Corporation

(On behalf of ASME V&V 40.4 Verification Working Group)

May 21, 2020ASME Verification & Validation 2020 Virtual Symposium

ASME V&V 40

1

ASME V&V 40.4 Verification Best Practices Working Group

Verification Best Practices

Code Verification Best Practices Calculation Verification Best Practices

1) Poiseuille flow example problem (complete)2) Womersley flow example problem (active)

1) Nitinol stent FEA example problem (active)2) Hip stem FEA example problem (active)3) Knee tibial tray FEA example problem (active)

Goal: The fundamentals of code and calculation verification are thoroughly reviewed in the ASME V&V 10, 10.1, and 20 standards. However, the computational models used in the evaluation of medical devices can be quite complex and have their own subtleties. The objective of the ASME V&V 40.4 working group is to explore, learn, and employ code and calculation verification best practices on representative examples from the medical device space.

2

“Four ups” of VVUQ (adopted from “integral theory” of Ken Wilber)

Waking up ASME V&V 40 (How much V&V?)Being conscious of COU of model, model risk(model influence, decision consequence), etc.

Growing up ASME V&V 10 & 20 (How to do V&V?)Being well versed in VVUQ methodologies(not just textbook knowledge, but experiential knowledge)

Cleaning up Shadow work: allergies and addictionsallergies → dislike/resistance to rigorous VVUQ because it

puts burden on us (convenience bias), etc.addictions→ urge to follow old way of doing things, etc.

(following legacy standard procedures in VVUQ)

Showing up Progress made in “three ups” listed above allows one to become a better citizen of the modeling world and engagefully in that world.

VVUQ is a process for realizing the reality (credibility) of computational models.

3

Christopher Basciano1, Jeff Bodner2, Constantine Butakoff3, Carlos Corrales4, Chris Delametter5, Swapnil Dindorkar6, Jun Ding7, Beatriz Eguzkitza8, Alex Francois-Saint-Cyr5, Mark Goodin9, Sharath Gopal10,

Ismail Guler11, Marc Horner12, Alireza Kermani13, Kranthi Kolli14, Sanjeev Kulkarni6, Omid Madani5, Anup Paul15, Ashley Peterson16, Sudeep Sastry17, Travis Schauer11, Jaykumar Teli6 , Alireza Vali9

1Becton Dickinson, 2Medtronic, 3ELEM, 4Baxter, 5Siemens, 6Neilsoft, 7Abbott, 8Barcelona Supercomputing Center, 9SimuTech Group, 10Eli Lilly, 11Boston Scientific, 12Ansys Inc ., 13Veryst, 14Weill Cornell Medical College,

15Stress Engineering Services, 16Thornton-Tomasetti,17WL Gore

ASME V&V 40.4 Code Verification Best Practices Working Group

4

Code verification example – Womersley flow

Flow characteristics:

• Circular rigid tube

• Pulsatile

• Single phase

• Laminar

• Incompressible

• Newtonian fluid

4

4

R

L

=

=

mm (radius of pipe)

cm (length of pipe)

0 cmx = 2 cmx = 4 cmx =

inlet axial location forpost-processing

outlet

0.035

1.06

=

=

2

3

dyn s / cm (dynamic viscosity of fluid)

g / cm (density of fluid)

0outletp = Pa0 cos( )

0

0

inletp p t

v

w

=

=

=

0 3 Pa

60 beats/min 1 Hz

2

outletp p p

f

f

= − =

= =

=

wall

5

Relevant dimensionless numbers

(2 )u RRe

=

Reynolds number:

Reynolds number based on max. centerline velocity:

CL,max

max

(2 )30

w RRe

=

u tC

x

=

Courant number:

Courant number based on max. centerline velocity:

CL,max

max

w tC

x

=

Womersley number:

5.5R

=

6

Analytical solutions generated in MATLAB

3/2

0

3/2

0

( / )( , ) Re 1

( )

i t

z

J i r RAu r t e

i J i

= −

7

MATLAB code Python code

Input parameters

Coding analytical solution in MATLAB & Python

3/2

0

3/2

0

( / )( , ) Re 1

( )

i t

z

J i r RAu r t e

i J i

= −

Analytical solution for axial velocity

8

( )

( )

( )

17

17

17

max 8.31 10 m/s

max 7.50 10 m/s

max 8.31 10 m/s

m

( , 0)

( , 0.25)

( , 0.

( , 0)

( , 0.25)

( , 0.5)

(

5

a ,

)

x

z i Python

z i Py

z i MATLAB

z i MATLAB

z i MATLAB

thon

z i Pytho

i

n

i

i

i

z i

u ru r t

u r t

u r

abs

abs

abs

ab

t

u

t

u

s

r t

u

r

r t

−

−

−

− =

− =

=

=

=

=

= −

=

=

( ) 17( , 0.75) 7.50 10 m/0.75) sz i PythMAT oLAB nu r tt − − = = =

Maximum difference in values calculated by MATLAB versus Python codes

16maximum difference in axial velocity profile 10 m/s−

9

Initial conditions for velocity field (consistent initialization)

3/2

0

3/2

0

( ,0) 0

( ,0) 0

( / )1( ,0) ( ,0) 1

( )

r

z

u r

u r

J i r Rpu r w r

L i J i

=

=

= = −

10

Numerical code verification with both spatial & temporal discretization

t

x

h p q r s

h x x t t xt x tg h g h g h h HOT = + + +

Discretization error expansion:

mixed spatial-temporal term

seven unknowns

, , (coefficients)

, , , (observed orders of accuracy)

x t xtg g g

p q r s

Seven independent levels of spatial-temporal refinement are required:

2

2

2

,

/ ,

/ ,

, /

, /

/ , /

/ , /

x

x

t

t

x t

x t

x t

x r t

x r t

x t r

x t r

x r t r

x r t r

: spatial refinement factor

: temporal refinement factor

x

t

r

r

mesh refinement levels used by Kamm et al. 2003

Requires the solution of seven nonlinear algebraic equations

Oberkampf W. L. and Roy C. J., Verification and Validation in Scientific Computing, Cambridge University Press, New York, 2010. 11


t

x

h p q

h x x t tg h g h HOT = + +

Discretization error expansion without a mixed spatial-temporal term:

four unknowns

, (coefficients)

, (observed orders of accuracy)

x tg g

p q

Four independent levels of spatial-temporal refinement are required:

,

/ ,

, /

/ , /

x

t

x t

x t

x r t

x t r

x r t r

: spatial refinement factor

: temporal refinement factor

x

t

r

r

mesh refinement levels recommended by Oberkampf and Roy 2010

( 0)xtg =

Requires the solution of four nonlinear algebraic equations

Oberkampf W. L. and Roy C. J., Verification and Validation in Scientific Computing, Cambridge University Press, New York, 2010.

(additive decomposition of discretization error →spatial discretization error + temporal discretization error)

12


Simple approach suggested by Oberkampf and Roy 2010:

First perform a spatial mesh refinement using three meshes with a fixed time step ( )th t=

t

x

h p

h x xg h = + (fixed temporal error term)q

t tg h =

2 ,

,

,

x x t

x x t

x t

r h h

r h h

h h

( ) ( )2ln /

ln( )

t t t t

x x x x xx x

h h h h

r h r h hr h

x

pr

− − =

( 1)

t t

x x x

h h

r h h

x p p

x x

gh r

−=

−



Simple approach suggested by Oberkampf and Roy 2010 (continued):

Second perform a temporal discretization refinement using three time steps with a fixed spatial mesh ( )xh x=

t

x

h q

h t tg h = + (fixed spatial error term)p

x xg h =

2,

,

,

x t t

x t t

x t

h r h

h r h

h h

( ) ( )2

ln /

ln( )

t t t t t t t

x x x x

r h r h r h h

h h h h

t

qr

− − =

( 1)

t t t

x x

r h h

h h

t q q

t t

gh r

−=

−


Solver details

COMSOL 5.4 – Finite element method (FEM) based software

• Parallel run on 20 cores• 64-bit• Q1+Q1 discretization

• First order (linear) elements for both the velocity components and pressure• Stabilized FEM

• Streamline diffusion (SUPG)• Crosswind diffusion (DC)

• Time-dependent fully coupled solver• Implicit “generalized-alpha” method for time discretization

• Steps taken by solver → fixed (manually specified)• Newton’s method for nonlinear iterations• Relative tolerance of 1E-9 (default = 1E-3) for iterative convergence• Direct solver (PARDISO) for linear system of equations

15

Convergence tolerance for nonlinear iterations in COMSOL

default relative tolerance = 10-3

relative tolerance used in this study = 10-9

16

Expected order of accuracy for spatial discretization (FEM) in COMSOL

• First order (linear) elements were used for both the velocity components and pressure (Q1+Q1).

• Ciarlet P. G., The Finite Element Method for Elliptic Problems, SIAM 2002 (first published in 1978)• Second order (p = k + 1 = 2) accuracy is expected for the error in L2 norm for the second order

elliptic boundary value problems when linear (k = 1) finite elements are used.• COMSOL Multiphysics 5.1 Reference Manual

• “The accuracy of SUPG can be shown to be at least O(h^(k+1/2)) where k >= 1 is the order of the basis functions.”o Johnson C., Numerical Solution of Partial Differential Equations by the Finite Element

Method, Dover 2009 (first published in 1987)

expected order of accuracy for spatial discretization 1.5

17

Expected order of accuracy for temporal discretization (generalized-alpha) in COMSOL

Implicit “generalized-alpha” method implementation in COMSOL → second order accurate

Jansen K. E., Whiting C. H., and Hulbert G. M., A generalized-alpha method for integrating the filtered Navier-Stokes equations with a stabilized finite element method, Computer Methods in Applied Mechanics and Engineering 2000; 190: 305-319.

expected order of accuracy for temporal discretization → second order

18

Discretization levels and Courant number

Courant number based on max. centerline velocity:

CL,max

max

w tC

x

=

Spatial Temporal CourantDiscretization Discretization Number

(mm) (ms)0.25 25 1.251

0.125 25 2.5020.0625 25 5.003

0.5 25 0.6250.5 12.5 0.3130.5 6.25 0.156


(mm) (ms)0.25 6.25 0.313

0.125 6.25 0.6250.0625 6.25 1.251

0.25 3.125 0.1560.25 1.5625 0.078

Study # 1 Study # 2

CL,max 0.012508 m/sw =


(mm) (ms)0.25 6.25 0.313

0.125 6.25 0.6250.0625 6.25 1.2510.0625 3.125 0.6250.0625 1.5625 0.313

Study # 3

Radius of pipe (R) = 4 mmPeriod of pulse (T) = 1 sec

4 mm16

0.25 mm

1 s160

6.25 ms

R

x

T

t

= =

= =

19

axis of symmetry

Finite element mesh for 2D-axisym model (x = 0.25 mm)

20

Monitored quantities of interest

axis of symmetry(centerline)

0 cmx = 2 cmx = 4 cmx =

point for calculation of discretization error in axial component of velocity (cm/s) at centerline

0r =

line for calculation of L2-norm of discretization error in axial component of velocity (cm/s) and discretization error in flow rate (mL/s)

10.00 st =

10.25 st =

10.50 st =

10.75 st =

t

x

wh

h

t

x

Qh

h2

t

x

wh

h

21

Tabular summary of results (Study # 2)

Quantity of Interest Time Spatial Discretization Temporal Discretization Spatial Temporal

(s) gx p gt q axial velocity at centerline (cm/s) 10.00 6.04E-02 1.8743 7.99E-07 1.9071 4.49E-03 2.32E-05

axial velocity at centerline (cm/s) 10.25 1.44E-02 2.4824 5.93E-07 2.0124 4.57E-04 2.03E-05

axial velocity at centerline (cm/s) 10.50 6.04E-02 1.8743 7.95E-07 1.9088 4.49E-03 2.32E-05axial velocity at centerline (cm/s) 10.75 1.44E-02 2.4824 5.95E-07 2.0108 4.57E-04 2.04E-05L2-norm of axial velocity (cm/s) 10.00 6.45E-02 1.9941 -2.16E-07 2.0415 4.07E-03 -9.54E-06L2-norm of axial velocity (cm/s) 10.25 3.54E-02 2.0384 -7.73E-07 1.9542 2.12E-03 -2.02E-06L2-norm of axial velocity (cm/s) 10.50 6.45E-02 1.9941 -2.16E-07 2.0415 4.07E-03 -9.54E-06L2-norm of axial velocity (cm/s) 10.75 3.54E-02 2.0384 -7.73E-07 1.9542 2.12E-03 -2.00E-06

flow rate (mL/s) 10.00 1.12E-02 2.0017 1.63E-07 1.1308 6.99E-04 8.12E-07flow rate (mL/s) 10.25 9.91E-03 2.4025 -8.14E-07 2.0021 3.82E-04 -4.29E-06



(mm) (ms)0.25 6.25 0.313

0.125 6.25 0.6250.0625 6.25 1.251

0.25 3.125 0.1560.25 1.5625 0.078

t

x

h p q

h x x t tg h g h

= +

2

t

x

h p q

h x x t tg h g h

= +

1.5

2.0

f

f

p

q

=

22

Tabular summary of results (Study # 3)

Quantity of Interest Time Spatial Discretization Temporal Discretization Spatial Temporal

(s) gx p gt q axial velocity at centerline (cm/s) 10.00 6.04E-02 1.8743 6.39E-07 2.0109 3.32E-04 2.32E-05

axial velocity at centerline (cm/s) 10.25 1.44E-02 2.4824 6.35E-07 1.9998 1.03E-05 2.03E-05

axial velocity at centerline (cm/s) 10.50 6.04E-02 1.8743 6.35E-07 2.0128 3.32E-04 2.32E-05axial velocity at centerline (cm/s) 10.75 1.44E-02 2.4824 6.37E-07 1.9984 1.03E-05 2.04E-05L2-norm of axial velocity (cm/s) 10.00 6.45E-02 1.9941 -2.90E-07 1.8593 2.56E-04 -9.54E-06L2-norm of axial velocity (cm/s) 10.25 3.54E-02 2.0384 -2.94E-05 0.1874 1.64E-04 -2.02E-06L2-norm of axial velocity (cm/s) 10.50 6.45E-02 1.9941 -2.90E-07 1.8595 2.56E-04 -9.54E-06L2-norm of axial velocity (cm/s) 10.75 3.54E-02 2.0384 -2.97E-05 0.1854 1.64E-04 -2.00E-06




(mm) (ms)0.25 6.25 0.313

0.125 6.25 0.6250.0625 6.25 1.2510.0625 3.125 0.6250.0625 1.5625 0.313

t

x

h p q

h x x t tg h g h

= +

2

t

x

h p q

h x x t tg h g h

= +

1.5

2.0

f

f

p

q

=

23

Time history of discretization error (spatial refinement)

Discretization error for axial component of velocity (cm/s) at centerline (x=2cm & r=0cm)

( , )t

x

h exact

h x tf h h f = −

6.25 msth t= =6.25 msth t= =

(Study # 2) (Study # 2)

24



25 msth t= = 6.25 msth t= =

( , )t

x

h exact

h x tf h h f = −


25

Time history of discretization error (temporal refinement)


62.5 mxh x = =0.25 mmxh x= =

( , )t

x

h exact

h x tf h h f = −


26



( , )t

x

h exact

h x tf h h f = −

62.5 mxh x = =0.25 mmxh x= =


27

Time history of discretization error (temporal refinement) – zoom-in


0.25 mmxh x= =0.25 mmxh x= =

error decreases with temporal refinement (“apparent convergence”)

error decreases with temporal refinement (“apparent convergence”)

( , )t

x

h exact

h x tf h h f = −


28

DE for axial component of velocity (cm/s) at centerline (t = 10.00 s)

21.8743 & 6.04 10xp g −= = 71.9071 & 7.99 10tq g −= =

spatial refinement temporal refinement

t

x

h p

h x xg h − = t

x

h q

h t tg h − =

0.25 mmxh x= =

6.25 msth t= =

t

x

h p q

h x x t tg h g h

= +(Study # 2)

29


22.4824 & 1.44 10xp g −= = 72.0124 & 5.93 10tq g −= =


0.25 mmxh x= =

6.25 msth t= =

t

x

h p

h x xg h − = t

x

h q

h t tg h − =

t

x

h p q

h x x t tg h g h

= +(Study # 2)

30


21.8743 & 6.04 10xp g −= = 71.9088 & 7.95 10tq g −= =


0.25 mmxh x= =

6.25 msth t= =

t

x

h p

h x xg h − = t

x

h q

h t tg h − =

t

x

h p q

h x x t tg h g h

= +(Study # 2)

31


22.4824 & 1.44 10xp g −= = 72.0108 & 5.95 10tq g −= =


0.25 mmxh x= =

6.25 msth t= =

t

x

h p

h x xg h − = t

x

h q

h t tg h − =

t

x

h p q

h x x t tg h g h

= +(Study # 2)

32


L2-norm of discretization error for axial component of velocity (cm/s) at x=2cm1/2

2

2

1( ) where ( , )t t t

x x x

h h h exact

h h h x td f h h fA

= = −

6.25 msth t= =6.25 msth t= =


33



2

2

1( ) where ( , )t t t

x x x

h h h exact

h h h x td f h h fA

= = −

0.25 mmxh x= = 62.5 mxh x = =


34



2

2

1( ) where ( , )t t t

x x x

h h h exact

h h h x td f h h fA

= = −

0.25 mmxh x= = 62.5 mxh x = =


35

Time history of discretization error (temporal refinement) – zoom-in


2

2

1( ) where ( , )t t t

x x x

h h h exact

h h h x td f h h fA

= = −

error increases with temporal refinement (“apparent divergence”)

0.25 mmxh x= =0.25 mmxh x= =

The term “apparent divergence” was coined by Luis Eca during a discussion of the results on November 20, 2019 at the ASME V&V committee meeting in San Antonio, TX.



36



2

2

1( ) where ( , )t t t

x x x

h h h exact

h h h x td f h h fA

= = −


2

t

x

h p q

h x x t tg h g h

= +

2if 0 ( 0), but 0 ( 0) is approaching from belowt

x

h

x t hg g →2

as t

x

h

h th

37

L2-norm of DE for axial component of velocity (cm/s) at x=2cm (t = 10.00 s)

21.9941 & 6.46 10xp g −= =


2

t

x

h p

h x xg h − =2

t

x

h q

h t tg h − =

0.25 mmxh x= =

6.25 msth t= =

72.2.0415 & 1 16 0tq g −= − =

2

t

x

h p q

h x x t tg h g h

= +(Study # 2)

38

77.1.9542 & 1 73 0tq g −= − =


22.0384 & 3.54 10xp g −= =


2

t

x

h p

h x xg h − =2

t

x

h q

h t tg h − =

0.25 mmxh x= =

6.25 msth t= =

2

t

x

h p q

h x x t tg h g h

= +(Study # 2)

39


21.9941 & 6.45 10xp g −= =


2

t

x

h p

h x xg h − =2

t

x

h q

h t tg h − =

0.25 mmxh x= =

6.25 msth t= =

72.2.0415 & 1 16 0tq g −= − =

2

t

x

h p q

h x x t tg h g h

= +(Study # 2)

40

77.1.9542 & 1 73 0tq g −= − =


22.0384 & 3.54 10xp g −= =


2

t

x

h p

h x xg h − =2

t

x

h q

h t tg h − =

0.25 mmxh x= =

6.25 msth t= =

2

t

x

h p q

h x x t tg h g h

= +(Study # 2)

41


Discretization error for flow rate (mL/s) at x=2cm

6.25 msth t= =6.25 msth t= =

( , )t

x

h exact

h x tf h h f = −


42



25 msth t= = 6.25 msth t= =

( , )t

x

h exact

h x tf h h f = −


43



0.25 mmxh x= =

( , )t

x

h exact

h x tf h h f = −

62.5 mxh x = =


44



0.25 mmxh x= =

( , )t

x

h exact

h x tf h h f = −

62.5 mxh x = =


45

Discretization error for flow rate (mL/s) at x=2cm (t = 10.00 s)

22.0017 & 1.12 10xp g −= =


t

x

h p

h x xg h − = t

x

h q

h t tg h − =

0.25 mmxh x= =

6.25 msth t= =

71.1308 & 1.63 10tq g −= =

t

x

h p q

h x x t tg h g h

= +(Study # 2)

46


32.4025 & 9.91 10xp g −= =


0.25 mmxh x= =

6.25 msth t= =

78.2.0021 & 1 14 0tq g −= − =

t

x

h p q

h x x t tg h g h

= +

t

x

h p

h x xg h − = t

x

h q

h t tg h − =

(Study # 2)

47


22.0017 & 1.12 10xp g −= =


0.25 mmxh x= =

6.25 msth t= =

71.1328 & 1.61 10tq g −= =

t

x

h p q

h x x t tg h g h

= +

t

x

h p

h x xg h − = t

x

h q

h t tg h − =

(Study # 2)

48


32.4029 & 9.92 10xp g −= =


0.25 mmxh x= =

6.25 msth t= =

78.2.0019 & 1 15 0tq g −= − =

t

x

h p q

h x x t tg h g h

= +

t

x

h p

h x xg h − = t

x

h q

h t tg h − =

(Study # 2)

49

Thank you!

Questions?

[email protected]

Acknowledgements: Members of ASME V&V 40.4 Verification Working Group

Prof. Chris Roy at Virginia Tech

Prof. Luis Eça at Technical University of Lisbon

Dr. Constantine Butakoff at ELEM Biotech 50

mailto:[email protected]

asme v&v 40

Documents