new twinkle, twinkle little star using 18.000 gpus to simulate jets in … · 2013. 11. 26. ·...

Text optional: Institutsname Prof. Dr. Hans Mustermann www.fzd.de Mitglied der Leibniz-Gemeinschaft

Twinkle, twinkle little star

using 18.000 GPUs to simulate jets in the cosmos

Text optional: Institutsname Prof. Dr. Hans Mustermann www.fzd.de Mitglied der Leibniz-Gemeinschaft

Twinkle, twinkle little star

using 18.000 GPUs to simulate jets in the cosmos

Michael Bussmann1, Heiko Burau1, Thomas E. Cowan1, Alexander Debus1, Axel Hübl1,

Guido Juckeland2, Thomas Kluge1, Wolfgang E. Nagel1, Richard Pausch1, Felix Schmitt2,

Ulrich Schramm1, Joseph Schuchart2,3, René Widera1

1 Helmholtz-Zentrum Dresden – Rossendorf 2 ZIH, Technical University Dresden 3 Oak Ridge National Laboratory

Michael Bussmann · Computational Radiation Physics · picongpu.hzdr.de [email protected]

© Miguel Claro Night Sky Photography www.miguelclaro.com

We understand the Universe …



… by the Light we see from Earth.



We have built Telescopes to see Objects far away …


… and at any Wavelength of the Spectrum.

X-Ray: NASA/CXC/J.Hester (ASU); Optical: NASA/ESA/J.Hester & A.Loll (ASU); Infrared: NASA/JPL-Caltech/R.Gehrz (Univ. Minn.)


Monsters in the Sky ─ Pulsars, Active Galactic Nuclei & Black Holes

Relativistic Speeds

Turbulent Dynamics

Particle Acceleration

Relativistic Speeds

Turbulent Dynamics

Particle Acceleration

Video by CHANDRA X-Ray Observatory


Jets are Sources of Accelerated Particles

http://imgs.xkcd.com/comics/fountain.png


Can we have a Look inside these Jets?

NASA Publich Domain: http://en.wikipedia.org/wiki/File:Messier_87_Hubble_WikiSky.jpg


The Model Jet

v ≈ c

Gas around the Jet

Jet

Gas around the Jet

v ≈ c


Hydrogen Plasma

Hydrogen Plasma

Hydrogen Plasma

v ≈ c

The Model Jet


Hydrogen Plasma

Hydrogen Plasma

Hydrogen Plasma

vrelative

vrelative

vrelative

The Model Jet


+ +

+ +

+ + + +

─

─

─

─

─ ─

─ ─

The Model Jet


+

+ +

+

+

+

+

+

+

+ +

+

+

+

+ +

+

+

+ +

+

+

─

─

─

─

─

─

─

─

─ ─

─

─ ─

─

─

─

─

─

─

─

─


A Many-GPGPU Particle-in-Cell Code

Built to simulate Laser-Plasma Interaction

Compact Accelerators for Cancer Therapy

Ideal for Astro-Plasma-Physics

Open Source, free for Download


+

+ +

+

+

+

+

+

+

+ +

+

+

+

+ +

+

+

+ +

+

+

─

─

─

─

─

─

─

─

─ ─

─

─ ─

─

─

─

─

─

─

─

─

Moving Particles create Fields

Fields act back on Particles

Particles change Cells


// calculate electromagnetic field energy

algorithm::kernel::Foreach<math::CT::Int<TILE_WIDTH,TILE_HEIGHT,1> >()(

energyDBuffer.zone(),

energyDBuffer.origin(),

cursor::tools::slice(fieldE.origin()),

cursor::tools::slice(fieldB.origin()),

_1 = (_abs2(_2) + _abs2(_3) * MUE0_EPS0) * 0.5 * EPS0);

In-Kernel STL Magic


HOST NODE (CPU)

Communication Thread 1

GPU 1

GPU 2

Communication Thread 2

MPI

MPI


GPU

GPU

GPU

GPU

GPU

GPU

GPU

GPU

GPU

GPU

GPU

GPU


Computations on GPU

CPU for I/O only

Fast GPU Memory Access

Data + Task Parallelism

Concurrent Kernels

Asynchronous I/O


The largest kinetic Simulation of the relativistic KHI yet

46 × larger ─ 4.7 × higher resolution ─ 75 Billion Particles

Visualization by David Pugmire


There‘s more to it than Particles ─ Fields


Field and Particles show similar Structures


But we can‘t SEE Electrons from Millions of Lightyears away!

─


512 Frequencies, 481 Directions

─

Offline Analysis: 4 Pbyte of Trajectories


KHI-Skymap


Skymap with Particles


Skymap with Fields


Gordon-Bell Results


PIConGPU ─ Speedup from 16 to 18,432 Nodes

*ideal: 1152


PIConGPU ─ Strong Scaling 16 to 18,432 Nodes


PIConGPU ─ Weak Scaling 1 to 18,432 Nodes


PIConGPU ─ Performance MEASURED (Peak / Sustained)

CUPTI Metrics: flops_sp, flops_dp

7.176 PFLOP/s (dp)

+

1.449 PFLOP/s (sp)


René

Axel

Heiko Richard

new twinkle, twinkle little star using 18.000 gpus to simulate jets in … · 2013. 11. 26. ·...

Documents