1
Mark F. AdamsH.H. Bayraktar, T.M. Keaveny, P. Papadopoulos and Atul Gupta
Ultrascalable Implicit Finite Element Analyses in Solid
Mechanics with over a Half a Billion Degrees of Freedom
(excerpts)
2
Trabecular Bone
5-mm Cube
Cortical bone
Trabecular bone
3
Micro-Computed Tomography
CT @ 22 m resolution
3D image
Mechanical TestingE, yield, ult, etc.
2.5 mm cube44 m elements
FE mesh
Methods: FE modeling
4
the vertebral body you are showing is pretty healthy from a 80 year old female and it is a T-10 that is thoracic. So it is pretty close to the mid-spine. Usually research is done from T-10 downward to the lumbar vertebral bodies. There are 12 thoracic VB's and 5 lumbar. The numbers go up as you go down.
5
1 mm slice from vertebral body
6
80 µm w/ shell
Vertebral Body With Shell
Large deformation elast. 6 load steps (3% strain) Scaled speedup
~131K dof/processor 7 to 537 million dof 4 to 292 nodes IBM SP Power3
14 of 16 procs/node
7
Athena: Parallel FE ParMetis
Parallel Mesh Partitioner (Univerisity of Minnesota)
Prometheus Multigrid Solver
FEAP Serial general purpose
FE application (University of California)
PETSc Parallel numerical
libraries (Argonne National Labs)
FE MeshInput File
Athena ParMetis
FE input file(in memory)
FE input file(in memory)
Partition to SMPs
Athena Athena ParMetis
File File File File
FEAPFEAPFEAPFEAP Material Card
Silo DBSilo DB
Silo DBSilo DB
Visit
Prometheus
PETScParMetis
METISMETISMETISMETIS
pFEAP
Computational Architecture
Olympus
8
ParMetis partitions
9
131K dof / proc.47 Teraflops - 4088 processors