![Page 1: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/1.jpg)
Towards Automating the Configuration of a
Distributed Storage System
Lauro B. Costa Matei Ripeanu{lauroc, matei}@ece.ubc.ca
NetSysLabUniversity of British Columbia
![Page 2: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/2.jpg)
2
Distributed File Systems
Different workloads:Read/write only, high data similarity
Different optimizations: Temp local storage, deduplication,
replication
One size fits all: each choice may be the optimal for a specific workload, not for all
![Page 3: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/3.jpg)
3
Configurable OptimizationsMosaStore1 and UrsaMinor2 propose file systems with configurable optimizations
User must choose the optimizations
1The Case for a Versatile Storage System, S. Al-Kiswany, A. Gharaibeh, M. Ripeanu, SOSP Workshop on Hot Topics in Storage and File Systems (HotStorage’09)2 Ursa Minor: versatile cluster-based storage. M. Abd-El-Malek, W. V. Courtright et al. In Proceedings of the 4th Conference on USENIX Conference on File and Storage Technologies - FAST’05
![Page 4: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/4.jpg)
4
Tuning File System
Indentify the system parametersDefine a target metricdo
Define a target value
Configure the parameters
Measure and analyze the performance
while not satisfied
![Page 5: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/5.jpg)
5
Tuning is Hard
Defining metrics and target values can be complex
Lack of knowledge of distributed systems, application or application’s workloads
Workload or infrastructure can change
Tuning is time-consuming
![Page 6: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/6.jpg)
6
Deduplication: Detecting Similarity
Only the firstblock is different
File AX
Y
Z
BlocksHashing
AAAA
BBBB
CCCC
File B
W
Y
Z
HashingBlocks
DDDD
BBBB
CCCC
![Page 7: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/7.jpg)
7
Deduplication for Checkpointing?Checkpointing applications write multiple snapshotsSuccessive snapshots may have high data similarity
Similarity depends on number of factors, e.g.:
– Process or application level– Frequency of checkpointing
![Page 8: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/8.jpg)
8
How can we configure the file system parameters (optimizations) with minimal human intervention?
![Page 9: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/9.jpg)
9
Agenda
Motivation: Configurable file systemsArchitecture to automatically configure a
FSFirst Case: Checkpointing applicationsImplementationEvaluationSummary and Future Work
![Page 10: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/10.jpg)
10
Requirements
Be easy to configureMinimal human intervention
Be able to choose a satisfactory performancePerformance close to administrator’s intention
Have a reasonable automated configuration costOverhead small enough to make sense to use
![Page 11: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/11.jpg)
11
Loop for Automated ConfigurationApplication
Distributed Storage System
Monitor
Actuator
Controller
PredictorUtility
Function
Metrics Metrics
Best Configuration
Change Configuration
Data
![Page 12: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/12.jpg)
12
Controller
Utility function captures the metrics’ utility
It is simple for one target metric, e.g. timeIt reduces several target metrics to just one
dimension
Predictor estimates how a change affects the target metrics
Controller
PredictorUtility
Function
![Page 13: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/13.jpg)
13
Controller
PredictorUtility
Function
Controller decides the configuration by comparing the utility of
current and predicted metrics
![Page 14: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/14.jpg)
14
Agenda
Motivation: Configurable file systemsArchitecture to automatically configure a
FSFirst Case: Checkpointing applicationsImplementationEvaluationConclusionsFuture Work
![Page 15: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/15.jpg)
15
Data Deduplication
Can save storage space and network bandwidth
Has high computational cost to hash data
Mechanism to choose among two options:data deduplication on or off
![Page 16: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/16.jpg)
16
Control Loop for DeduplicationMetrics: time spent and storage space
Keep history for writes:– total time– number of blocks received– number of blocks similar
![Page 17: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/17.jpg)
17
Utility Function Controller
PredictorUtility
Function
Administrator gives weights to capture the relative importance, e.g.:
– 1 x time + 0 x storage– 0.5 x time + 0.5 x storage
![Page 18: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/18.jpg)
18
Predictor Controller
PredictorUtility
Function
Space
Time
No deduplication
Deduplication
number of blocks
I/O operations
consider similarity
+ time for hashing
data
![Page 19: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/19.jpg)
19
Evaluation
Three aspects:– Effort to configure– Performance– Overhead
Experimental setup10 storage nodes, 1Gbps NICs
![Page 20: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/20.jpg)
20
Evaluation: Workload
Synthetic
Similarity variedFor each similarity level, write 100 snapshots
Similar results for several snapshot sizes32, 64, 128, 256, 512 MBPlots for 256MB
![Page 21: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/21.jpg)
21
Effort to Configure
Small effortAdministrator specifies the weights for each
metric
No effort in the default caseSystem optimizes for time
![Page 22: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/22.jpg)
22
Optimizing for Time
![Page 23: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/23.jpg)
23
Optimizing for Time
Hashing cost paid off by savings with I/O operations
![Page 24: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/24.jpg)
24
Optimizing for Time
![Page 25: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/25.jpg)
25
Overhead
Memoryless than 1KB
ComputationalLow similarity - within 5% in evaluated
casesHigh similarity – negligible
![Page 26: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/26.jpg)
26
Summary
Initial study on automatically configuring a file system
Data deduplication configured properly with low overheads
![Page 27: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/27.jpg)
27
Future Work
More parameters for similarity detectionvariable block boundary, block sizes, offload to
GPU
Constraints for utility functionse.g., best time for a maximum storage space
More optimizations and metricsreplication, buffer size, caching policiesreliability, energy
![Page 28: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/28.jpg)
28
![Page 29: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/29.jpg)
29
Mixing time and storage space
![Page 30: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/30.jpg)
30
Application
Distributed Storage System
Monitor
Actuator
time, # blocks, #similar blocks
Deduplication on or off
Set deduplication
on or off
Data
time, # blocks, #similar blocks
Controller
PredictorUtility
Function
weights for metrics
![Page 31: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/31.jpg)
31
MosaStore Architecture
Metadata Manager
Benefactors(Storage nodes)
Client(FS interface)
...
![Page 32: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/32.jpg)
32
Prototype in MosaStore
Deduplication can be turned on and off on the fly
Write flow collects the measurements
Monitor and Controller are co-located with the client
![Page 33: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/33.jpg)
33
Utility
Utility is a measure of the relative satisfaction: How happy the administrator is
Money is a good proxy, but complicatedFocus on simple cases
- 100% time + 0% space- 50% time + 50% space- Constraint on space, optimize for time
Function cannot use different units: normalize
![Page 34: Towards Automating the Configuration of a Distributed Storage System Lauro B. Costa Matei Ripeanu {lauroc, matei}@ece.ubc.ca NetSysLab University of British](https://reader030.vdocument.in/reader030/viewer/2022013011/5514d93055034693478b53a5/html5/thumbnails/34.jpg)
34
MosaStore Architecture
Storage space aggregated from nodes in a network
Naming scheme: BYHASH, BYSEQ
File creation/write Collects metricIt has the option to activate similarity
detection