![Page 1: Extending Petascale I/O with Data Services Hasan Abbasi Karsten Schwan Matthew Wolf Jay Lofstead Scott Klasky (ORNL)](https://reader036.vdocument.in/reader036/viewer/2022081813/56649e9d5503460f94b9d817/html5/thumbnails/1.jpg)
Extending Petascale I/O with Data Services
Hasan AbbasiKarsten SchwanMatthew Wolf
Jay LofsteadScott Klasky (ORNL)
![Page 2: Extending Petascale I/O with Data Services Hasan Abbasi Karsten Schwan Matthew Wolf Jay Lofstead Scott Klasky (ORNL)](https://reader036.vdocument.in/reader036/viewer/2022081813/56649e9d5503460f94b9d817/html5/thumbnails/2.jpg)
Motivation
• I/O bottleneck• Petascale data sizes• Data overload• Faster solution
![Page 3: Extending Petascale I/O with Data Services Hasan Abbasi Karsten Schwan Matthew Wolf Jay Lofstead Scott Klasky (ORNL)](https://reader036.vdocument.in/reader036/viewer/2022081813/56649e9d5503460f94b9d817/html5/thumbnails/3.jpg)
Observations
• Fast Extraction• Flexibility in where we execute operations• Managed output to data consumer• Flexible resource utilization
![Page 4: Extending Petascale I/O with Data Services Hasan Abbasi Karsten Schwan Matthew Wolf Jay Lofstead Scott Klasky (ORNL)](https://reader036.vdocument.in/reader036/viewer/2022081813/56649e9d5503460f94b9d817/html5/thumbnails/4.jpg)
Compute Area Using ADIOS for flexibility in
choosing output method Data is serialized using FFS COD provides a processing hook
within the compute application SmartTap generates the output
buffer through a user defined function
DataTap moves the data to the staging area
![Page 5: Extending Petascale I/O with Data Services Hasan Abbasi Karsten Schwan Matthew Wolf Jay Lofstead Scott Klasky (ORNL)](https://reader036.vdocument.in/reader036/viewer/2022081813/56649e9d5503460f94b9d817/html5/thumbnails/5.jpg)
Staging Area
Additional resources for buffering before storage Simple operations like aggregation Complex analysis and compression operations Domain specific services
Combination of extraction, processing and storage Placement to optimize performance
![Page 6: Extending Petascale I/O with Data Services Hasan Abbasi Karsten Schwan Matthew Wolf Jay Lofstead Scott Klasky (ORNL)](https://reader036.vdocument.in/reader036/viewer/2022081813/56649e9d5503460f94b9d817/html5/thumbnails/6.jpg)
Runtime Overhead
2k 16k 64k 112k0
10
20
30
40
50
60
70
80
90
POSIXDataTap
Number of Nodes
Runti
me
Ove
rhea
d (%
)