intelligent trigger for hyper-k with gpus akitaka ariga university of bern, switzerland
DESCRIPTION
Noise rate in Hyper-K SK -> HK : Smaller signal and larger background – Detector size -> larger -> gate width longer 200ns ->500ns – # of sensors -> larger N 12k -> 20k ~ 80k – Noise rate -> larger N 4kHz -> 10kHz – Photo coverage -> smaller smaller S 40% -> 15% ~ 20% SK: 200ns x 12,000PMTs x 4kHz = 10 hits/gate (SK threshold = 33 hits) HK:500ns x 20,000PMTs x 10kHz = 100 hits/gate Direct impact on low energy neutrino physics, supernova and partially on proton decayTRANSCRIPT
![Page 1: Intelligent trigger for Hyper-K with GPUs Akitaka Ariga University of Bern, Switzerland](https://reader036.vdocument.in/reader036/viewer/2022062401/5a4d1b1e7f8b9ab0599946e0/html5/thumbnails/1.jpg)
Intelligent trigger for Hyper-K with GPUs
Akitaka ArigaUniversity of Bern, Switzerland
![Page 2: Intelligent trigger for Hyper-K with GPUs Akitaka Ariga University of Bern, Switzerland](https://reader036.vdocument.in/reader036/viewer/2022062401/5a4d1b1e7f8b9ab0599946e0/html5/thumbnails/2.jpg)
Recent changes in design• Conventional design
– 10 compartments– Noise rate in each of them is about
SK scale
• Recently coming back to SK style– For cost optimization– 1 (or a few) large detector– Longer gate width– Larger number of PMT per
detector– Large noise rate to cope
![Page 3: Intelligent trigger for Hyper-K with GPUs Akitaka Ariga University of Bern, Switzerland](https://reader036.vdocument.in/reader036/viewer/2022062401/5a4d1b1e7f8b9ab0599946e0/html5/thumbnails/3.jpg)
Noise rate in Hyper-K• SK -> HK : Smaller signal and larger background
– Detector size -> larger -> gate width longer• 200ns ->500ns
– # of sensors -> larger N• 12k -> 20k ~ 80k
– Noise rate -> larger N• 4kHz -> 10kHz
– Photo coverage -> smaller smaller S• 40% -> 15% ~ 20%
• SK: 200ns x 12,000PMTs x 4kHz = 10 hits/gate (SK threshold = 33 hits)• HK:500ns x 20,000PMTs x 10kHz = 100 hits/gate
Direct impact on low energy neutrino physics, supernova and partially on proton decay
![Page 4: Intelligent trigger for Hyper-K with GPUs Akitaka Ariga University of Bern, Switzerland](https://reader036.vdocument.in/reader036/viewer/2022062401/5a4d1b1e7f8b9ab0599946e0/html5/thumbnails/4.jpg)
Signal in SK (40%)
Signal in HK (20%)
Noise level in HK
Noise level in SK
Solar neutrino
Supernova
Signal / background• Signal: 6 hits/MeV (SK,40%), 3 hits/MeV (HK,20%)• Noise level: expected number of hits in a gate– SK: 200ns x 12,000PMTs x 4kHz = 10 hits/gate– HK:500ns x 20,000PMTs x 10kHz = 100 hits/gate
Noise hits will be dominant at low energy (E<30MeV)
![Page 5: Intelligent trigger for Hyper-K with GPUs Akitaka Ariga University of Bern, Switzerland](https://reader036.vdocument.in/reader036/viewer/2022062401/5a4d1b1e7f8b9ab0599946e0/html5/thumbnails/5.jpg)
Solar neutrino
Supernova
Detectable energy• Detectable : Signal+Noise > Noise + noise fluctuation• Noise issue is essential to access low energy physics
below 20 MeV, where most of supernova, solar neutrino, some of proton decay signals exist.
Signal + noise in SK
Signal + noise in HK
Noise + 5s fluctuation= realistic threshold
detectable
![Page 6: Intelligent trigger for Hyper-K with GPUs Akitaka Ariga University of Bern, Switzerland](https://reader036.vdocument.in/reader036/viewer/2022062401/5a4d1b1e7f8b9ab0599946e0/html5/thumbnails/6.jpg)
Need to improve trigger quality• Be intelligent!– Use of 4D information hits, (x,y,z,t)
• Many ideas– Exploit TOF information to narrow gate
width next page– Vertex calculation: 2 hits can make a
hyperbolic surface, 4 hits can make unique identification of vertex position
– Ring pattern fitting
AB
C
Hyperbolic by A, B
Hyperbolic by B, C
),,,( tzyx
![Page 7: Intelligent trigger for Hyper-K with GPUs Akitaka Ariga University of Bern, Switzerland](https://reader036.vdocument.in/reader036/viewer/2022062401/5a4d1b1e7f8b9ab0599946e0/html5/thumbnails/7.jpg)
One of many ideas: Sub-volume triggering
• The largest factor of noise increase is gate width due to large detector Let’s make it small.
• Sub-volume triggering– Divide detector into several sub-volumes– In each sub-volume, perform inversion of
hit-time using distance from hit-positions– smaller gate width, canceling detector
size increase• Large computing power required
– triggering in O(100) sub-volumes
),,,( tzyx
)||,,,( 000 cAVtzyx
A
V),,( 000 zyx
center of sub-volume
projected params
A’
t t’
![Page 8: Intelligent trigger for Hyper-K with GPUs Akitaka Ariga University of Bern, Switzerland](https://reader036.vdocument.in/reader036/viewer/2022062401/5a4d1b1e7f8b9ab0599946e0/html5/thumbnails/8.jpg)
Intelligent trigger with GPUs• To profit of 4D data, need more computing power• GPU is an ideal solution: Expertise in LHEP-Bern– GPU: Graphic Processing Unit– Parallel processing with O(1000) processing cores– Triggering code can be highly parallelized
![Page 9: Intelligent trigger for Hyper-K with GPUs Akitaka Ariga University of Bern, Switzerland](https://reader036.vdocument.in/reader036/viewer/2022062401/5a4d1b1e7f8b9ab0599946e0/html5/thumbnails/9.jpg)
Parallel processing
• GPU allow you a parallel processing with thousands of processing cores.
Serial processCPU
Parallel processGPU
task 1task 2...
![Page 10: Intelligent trigger for Hyper-K with GPUs Akitaka Ariga University of Bern, Switzerland](https://reader036.vdocument.in/reader036/viewer/2022062401/5a4d1b1e7f8b9ab0599946e0/html5/thumbnails/10.jpg)
High computing power
1 full tower of CPU based computing cluster = 5-10 TFLOPS
NVIDIA GeforceTitan Z= 8 TFLOPS
FLOPS = floating-point operations per second
![Page 11: Intelligent trigger for Hyper-K with GPUs Akitaka Ariga University of Bern, Switzerland](https://reader036.vdocument.in/reader036/viewer/2022062401/5a4d1b1e7f8b9ab0599946e0/html5/thumbnails/11.jpg)
CMOS camera0.5 – 2.4 Gbyte/s
Experience of LHEP-Bern 1: High speed emulsion reconstruction
Custom-made real-time scanning microscope
(Real time) 3D track reconstruction with GPUs
x90 faster
Geforece GTX TITAN x 32688 cores, 6GB memory, 4.5 TFLOPs in each
JINST 9 P04002 (2014), GTC2014, GPU in high energy physics (2014)
![Page 12: Intelligent trigger for Hyper-K with GPUs Akitaka Ariga University of Bern, Switzerland](https://reader036.vdocument.in/reader036/viewer/2022062401/5a4d1b1e7f8b9ab0599946e0/html5/thumbnails/12.jpg)
• Hough transform with GPU• x 50 faster processing achieved
x 50 faster
LAr detector (ArgonTube at LHEP-Bern)
Experience of LHEP-Bern 2: Reconstruction of LAr-TPC
![Page 13: Intelligent trigger for Hyper-K with GPUs Akitaka Ariga University of Bern, Switzerland](https://reader036.vdocument.in/reader036/viewer/2022062401/5a4d1b1e7f8b9ab0599946e0/html5/thumbnails/13.jpg)
Possible hardware for HK• Data will be distributed to several nodes equipped
with GPUs• O(100) processes run with O(100,000) GPU cores
4U processing server2 CPU x 10 cores8 GPUs (24,000 cores)
Processing machine
GPU
2.5 Gbyte/s
CPUCPU
Processing machine
GPU
CPUCPU
Processing machine
![Page 14: Intelligent trigger for Hyper-K with GPUs Akitaka Ariga University of Bern, Switzerland](https://reader036.vdocument.in/reader036/viewer/2022062401/5a4d1b1e7f8b9ab0599946e0/html5/thumbnails/14.jpg)
Improve WIT?
• One of the bottlenecks with current algorithm is number of combinations.– To calculate a vertex with 4 hits– nC4 quickly increase like n4
– 10C4 = 210 (SK level), 100C4 = 3.9x106 (HK level)– (according to Michael Smy, a hit selection can
reduce n4 -> n3, which is implemented in WIT)• A comparison of processing time is quickly
studied.
![Page 15: Intelligent trigger for Hyper-K with GPUs Akitaka Ariga University of Bern, Switzerland](https://reader036.vdocument.in/reader036/viewer/2022062401/5a4d1b1e7f8b9ab0599946e0/html5/thumbnails/15.jpg)
Vertexing by 4-hits combination• Using a WCSim-simulated data provided by Yano
– H 100m, D 69m, electrons start from center– Only signal hits are used, 5000 events.
• Implement code in CPU and GPU• Equivalent result is, of course, obtained in GPU
CPU GPUVertices are reconstructed at center of detector (0,0,0), as it should be.
![Page 16: Intelligent trigger for Hyper-K with GPUs Akitaka Ariga University of Bern, Switzerland](https://reader036.vdocument.in/reader036/viewer/2022062401/5a4d1b1e7f8b9ab0599946e0/html5/thumbnails/16.jpg)
First comparison in speed• Basic optimization done for CPU code• Factor 35 faster with GPU• In my experience, it can be additional factor 2-5 faster with
further optimization.
3MeV 5 711
1315 MeV(about 500,000 combinations / event)9
20 MeV(about 1.6 million combinations / event)cpu 788 secgpu 22.71 sec
![Page 17: Intelligent trigger for Hyper-K with GPUs Akitaka Ariga University of Bern, Switzerland](https://reader036.vdocument.in/reader036/viewer/2022062401/5a4d1b1e7f8b9ab0599946e0/html5/thumbnails/17.jpg)
Sub-volume triggering
• In each sub-volume, perform inversion of hit-time using distance from hit-positions– smaller gate width, canceling
detector size increase• Test with simulated data– H 100m, D 50m– electron emitted from center to
x direction
),,,( tzyx
)||,,,( 000 cAVtzyx
A
V),,( 000 zyx
center of sub-volume
projected params
A’
t t’
xz
y
(0,0,0)
![Page 18: Intelligent trigger for Hyper-K with GPUs Akitaka Ariga University of Bern, Switzerland](https://reader036.vdocument.in/reader036/viewer/2022062401/5a4d1b1e7f8b9ab0599946e0/html5/thumbnails/18.jpg)
Sub-volume triggering
),,,( tzyx
)||,,,( 000 cAVtzyx
A
V
),,( 000 zyxpredefined vertex
projected params
A’
xz
y
• time back-calculation to predefined vertices along xx axis = [500, 1500] ns, 10 ns binning, blue histogram = event related
100 m height, 69 m diameter, 19 k PMTs, 9 MeV
Center
![Page 19: Intelligent trigger for Hyper-K with GPUs Akitaka Ariga University of Bern, Switzerland](https://reader036.vdocument.in/reader036/viewer/2022062401/5a4d1b1e7f8b9ab0599946e0/html5/thumbnails/19.jpg)
Subvolume triggering• time back-calculation to predefined vertices along Z
),,,( tzyx
)||,,,( 000 cAVtzyx
A
V
),,( 000 zyxpredefined vertex
projected params
A’
xz
y
x axis = [500, 1500] ns, 10 ns binning, blue histogram = event related
100 m height, 69 m diameter, 19 k PMTs, 9 MeV
Center
軸方向に vertexを並べたときに比べてピークが局在化。高い値を持つ領域は楕円球状に存在する trackingできる、そしていくつかの subvolumeの連続することを要求すればBGも落とせる。
![Page 20: Intelligent trigger for Hyper-K with GPUs Akitaka Ariga University of Bern, Switzerland](https://reader036.vdocument.in/reader036/viewer/2022062401/5a4d1b1e7f8b9ab0599946e0/html5/thumbnails/20.jpg)
Signal/BG Separation• Predefine vertices every 5m in detector
volume(~3000 vertices)• Find vertex which has highest entry in one
of time bin• 9 MeV electron from center x 5000 events Predefine vertex
every 5m
Simply counting # of hits in 500 ns gate width
Number of hits in 10 ns in the most probable predefined vertex (time-space)数字上 2.7から 7シグマに向上するが思ったよりセパレーションがよくない。。。そもそもガウシアンではない。 Noise onlyに対しても3000個の Vertex で最大値を取ると chance coincidenceで高く出てしまうことが原因。要改良。
noise
meanss
s=2.7 s=7.0
noise only noise + signal
![Page 21: Intelligent trigger for Hyper-K with GPUs Akitaka Ariga University of Bern, Switzerland](https://reader036.vdocument.in/reader036/viewer/2022062401/5a4d1b1e7f8b9ab0599946e0/html5/thumbnails/21.jpg)
スピード
![Page 22: Intelligent trigger for Hyper-K with GPUs Akitaka Ariga University of Bern, Switzerland](https://reader036.vdocument.in/reader036/viewer/2022062401/5a4d1b1e7f8b9ab0599946e0/html5/thumbnails/22.jpg)
Summary
• Noise rate is a crucial issue for low energy neutrino, supernova and proton decay
• We are investigating an intelligent trigger by exploiting 4D data from detector
• Larger computing power of >O(100) could be necessary An use of GPUs is a promising solution