mcma 2017 - napoli fredpeople.na.infn.it/~mettivie/mcma presentation/16 aula a/14_15_schi… · a...
TRANSCRIPT
![Page 1: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/1.jpg)
Fred a new GPU-based fast-MC code
and its applications in proton beam therapy
MCMA 2017 - Napoli
A. Schiavi
![Page 2: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/2.jpg)
Fast paRticle thErapy Dose evaluator
Collaboration• A. Schiavi, V. Patera, M. Senzacqua, Univ. La Sapienza
Roma /INFN (Italy)
• - G. Battistoni, S. Pioli - INFN (Italy)
• - I. Rinaldi, N. Krah - CNRS/IN2P3 and Lyon 1 University (France)
• A. Rucinski, J. Gajewski - PAN, Krakow (Poland)
![Page 3: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/3.jpg)
FRED fast-MC platform• MC for protons in voxel geometry
• Tabulated total stopping power in water (PSTAR-NIST), energy straggling (Gaussian and Landau-Vavilov regimes)
• MCS models: single-,double-,triple-gaussian, 2 gauss+Rutherford
• Nuclear interactions: elastic and inelastic; fragmentation; local deposition of heavy ions; tracking of secondary protons and deuterons
• HU to density conversion (Schneider) and stoppow calibration
• MC-TPS: dose optimization using DDO (Lomax)
• RBE models = fixed 1.1, LETd-based (Wedenberg, Carabe, Wilkens, Chen), table-based (LEM1, MKMPIDE)
![Page 4: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/4.jpg)
Fred - no nucl
Fred
Fluka
200 MeV protons in liquid waterDose map of a pencil beam
![Page 5: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/5.jpg)
Water model: energy deposition
100 MeV 150 MeV 200 MeV 250 MeV
nucl. off
elasticnucl. only
elasticnucl. only
nucl. on
GeantFLUKAFred
22
20
18
16
14
1224 25 26 27
A. Schiavi et al, PMB 62 (2017) 7482–7504
![Page 6: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/6.jpg)
Longitudinal profile and lateral tails
c)
d)
a)
Dose map
FLUKA
b)
![Page 7: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/7.jpg)
QA SOBPz x
y
head-onside-on
Water tank CT
3 cm cube at 15 cm depth
![Page 8: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/8.jpg)
QA SOBP: dose profiles and measurements
![Page 9: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/9.jpg)
E = 226.61 MeV/u at 20 cm depth
Field size factor
![Page 10: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/10.jpg)
• standard codes
• expensive (€€€€€)
• maintenance (staff)
• low budget (€€€€)
• redundancy
• in-house maintenance
+
Hardware for rapid MC recalculation
![Page 11: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/11.jpg)
Parallel execution model in Fred
Fred front-end
ExtranodeMPI
IntranodeMulti-GPUOpenCl
IntranodePOSIX
Multi-threads
![Page 12: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/12.jpg)
Queues and timeline
Execution timeline for 8 queues on 4 GPUs using OpenCL.
Host-to-device transfers (green), kernel execution (red), and device-to-host (blue) transfers
![Page 13: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/13.jpg)
Hardware and Performance
full-MC * * FLUKA or Geant4C
PU
benchmark: 150 MeV protons in a water phantom on a 1 mm3 dose scoring grid
![Page 14: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/14.jpg)
Hardware and Performance
full-MC * * FLUKA or Geant4C
PU
GP
U
![Page 15: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/15.jpg)
• Patient-specific QA protocol at CNAO
• Patient-specific HU-RSP calibration
• Commissioning of CCB proton center in Krakow
• Dose monitoring using secondary protons
Applications to proton therapy
see S. Muraro talk this afternoon (ID 67)
![Page 16: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/16.jpg)
Fast-MC recalculation of patient verification plans at CNAO
![Page 17: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/17.jpg)
Fast-MC recalculation of patient verification plans at CNAO
![Page 18: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/18.jpg)
18
Patient verification planTPS
FRED
TPS
FRED
Gy
tps
fred
head-on side-on
fred
tps
99.6% @ 2mm/2% 96.7% @ 1mm/1%
ɣ-index pass rates
![Page 19: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/19.jpg)
Patient-specific and treatment-specific HU-RSP
calibration
Fred
![Page 20: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/20.jpg)
Fred commissioning @ CCB Krakow Proton Beam Therapy Centre
FRED central cross-section
0 50 100 150 200 250
Z [mm]
-30
-20
-10
0
10
20
30
X [m
m]
0 50 100 150 200 250
Z [mm]
0
0.2
0.4
0.6
0.8
1
Dose
norm
. [-
]
Bragg Peak comparison
TPS Beam Model
(Rmax
=156.59mm, R 2
bort=1.000)
FRED MC sim.
(Rmax
=156.59mm, R 2
bort=1.000)
0 50 100 150 200 250
Z [mm]
2
3
4
5
6
7
Spot si
ze [m
m]
Spot primary sigma at depth
TPS σmean
Prim
FRED σmean
Prim
0 50 100 150 200 250
Z [mm]
5
10
15
20
25
Spot si
ze [m
m]
Spot secondary sigma at depth
TPS σmean
Sec
FRED σmean
Sec
-50 -40 -30 -20 -10 0 10 20 30 40 50
X [mm]
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Dose
norm
. [-
]
σXTPS=5.61mm (R 2=0.9999)
σXFRED=5.65mm (R 2=0.9999)
TPS Eclipse profile
FRED MC sim. profile
Signle Gauss fit to TPS
Signle Gauss fit to FRED
-50 -40 -30 -20 -10 0 10 20 30 40 50
X [mm]
10 -6
10 -5
10 -4
10 -3
10 -2
10 -1
10 0
Log d
ose
norm
. [-
]
Prim. σXTPS=5.74mm
Sec. σXTPS=16.50mm
(R2=0.9994)
Prim. σXFRED=5.55mm
Sec. σXFRED=11.06mm
(R2=0.9943)
TPS Eclipse profile
FRED MC sim. profile
Double Gauss fit to TPS
Double Gauss fit to FRED
-50 -40 -30 -20 -10 0 10 20 30 40 50
Y [mm]
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Dose
norm
. [-
]
σYTPS=5.19mm (R 2=0.9999)
σYFRED=5.54mm (R 2=0.9999)
TPS Eclipse profile
FRED MC sim. profile
Signle Gauss fit to TPS
Signle Gauss fit to FRED
-50 -40 -30 -20 -10 0 10 20 30 40 50
Y [mm]
10 -6
10 -5
10 -4
10 -3
10 -2
10 -1
10 0
Log d
ose
norm
. [-
]
Prim. σYTPS=5.38mm
Sec. σYTPS=16.54mm
(R2=0.9993)
Prim. σYFRED=5.47mm
Sec. σYFRED=11.08mm
(R2=0.9950)
TPS Eclipse profile
FRED MC sim. profile
Double Gauss fit to TPS
Double Gauss fit to FRED
Fred code is currently being commissioned at CCB as a quality assurance tool.Preliminary results show good agreement of single beam dose distributions calculated with Eclipse and Fred, indicating an accurate implementation of CCB beam model in the Fred MC-TPS code. Dose distributions for a complete plan can be obtained in about one minute using Fred on GPU. Robustness studies of treatment plan strategy can be conducted on the HPC cluster Prometheus.
![Page 21: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/21.jpg)
Top performance on Prometheus
• 72 Nodes with 24 CPUs and 2 Tesla K40d GPUs
• Up to 144 GPUs in parallel + 1728 CPUs
perfect linear scaling
up to 0.3 billion primary/s
2.4 mln/s per GPU
This research was supported in part by PL-Grid Infrastructure.
![Page 22: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/22.jpg)
• clinical validation of fast-recalculation tool
• applications to clinical routine
• extensions to include other ions (Carbon, Helium) and secondary particles (alphas, delta-rays and neutrons)
• dose monitoring using charged secondary particles
Future developmentsand perspectives
![Page 23: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/23.jpg)
![Page 24: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/24.jpg)
24
Patient recalculation plan
tps
fred
gamma-index 97% @ 2mm/2% gamma-index 92% @ 1mm/1%
recalculation at 1% = 700 million primary protons
simulation time = 72 s
![Page 25: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/25.jpg)
Hardware: 4x GPU NVIDIA GTX 10801x CPU Intel i7-5930K @ 3,50 GHz with 12 cores
20 mln primary/s
Water-cooled 4 GPU workstation
4x NVIDIA Titan-Xp40 mln primary/s
budget: 10 kEuro
compare with newNVIDIA DGX-1(8x Tesla P100)
expected performance:80 mln primary/s
budget: 125 kEuro
![Page 26: MCMA 2017 - Napoli Fredpeople.na.infn.it/~mettivie/MCMA presentation/16 Aula A/14_15_Schi… · a new GPU-based fast-MC code and its applications in proton beam therapy MCMA 2017](https://reader036.vdocument.in/reader036/viewer/2022070212/6107728de4d9301dd7240e1f/html5/thumbnails/26.jpg)
Case study: 3D raytracing for legacy F77 hydrocode
From 1 to 2 Mray/s(equivalent to 800 MPI processes)
Raytracing step well belowhydrodynamic step
Duty-cycle 30%
Duty-cycle 100%
4 water-cooled GPU
2 air-cooled GPU