workload analysis of a large-scale key-value store
DESCRIPTION
Berk Atikoglu, Yuehai Xu , Eitan Fracthenberg , Song Yiang , Mike Paleczny. Workload Analysis of a Large-Scale Key-Value Store. Analyze Memcached at Facebook. +284,000,000,000 requests 5 different use cases Workload characteristics, locality, cache effectiveness. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Workload Analysis of a Large-Scale Key-Value Store](https://reader035.vdocument.in/reader035/viewer/2022081503/56814cf3550346895db9f6d1/html5/thumbnails/1.jpg)
Workload Analysis of a Large-Scale Key-Value Store
Berk Atikoglu, Yuehai Xu, Eitan Fracthenberg, Song Yiang, Mike Paleczny
![Page 2: Workload Analysis of a Large-Scale Key-Value Store](https://reader035.vdocument.in/reader035/viewer/2022081503/56814cf3550346895db9f6d1/html5/thumbnails/2.jpg)
2
Analyze Memcached at Facebook
+284,000,000,000 requests
5 different use cases Workload characteristics, locality,
cache effectiveness
![Page 3: Workload Analysis of a Large-Scale Key-Value Store](https://reader035.vdocument.in/reader035/viewer/2022081503/56814cf3550346895db9f6d1/html5/thumbnails/3.jpg)
3
Why Is Caching Important?
Cache ServersWeb Servers
Database
![Page 4: Workload Analysis of a Large-Scale Key-Value Store](https://reader035.vdocument.in/reader035/viewer/2022081503/56814cf3550346895db9f6d1/html5/thumbnails/4.jpg)
4
Motivation
Understand workload characteristics Identify factors affecting
performance Provide a benchmark for future
studies
![Page 5: Workload Analysis of a Large-Scale Key-Value Store](https://reader035.vdocument.in/reader035/viewer/2022081503/56814cf3550346895db9f6d1/html5/thumbnails/5.jpg)
5
Memcached
Distributed memory caching system Key-value store for small objects
Hash Function
Memcached Servers
Key
![Page 6: Workload Analysis of a Large-Scale Key-Value Store](https://reader035.vdocument.in/reader035/viewer/2022081503/56814cf3550346895db9f6d1/html5/thumbnails/6.jpg)
6
Tracing Methodology
Capture traces through a Linux Kernel Module (LKM)
Process traces with Hive
MemcachedTransport (TCP/UDP)NetworkEthernet
LKM
![Page 7: Workload Analysis of a Large-Scale Key-Value Store](https://reader035.vdocument.in/reader035/viewer/2022081503/56814cf3550346895db9f6d1/html5/thumbnails/7.jpg)
7
Facebook Deployment
Pool Size DescriptionUSR Few User-account status informationAPP Dozens Object metadata of a popular
applicationSYS Few System data on service locationVAR Dozens Server-side browser informationETC Hundreds Nonspecific, general purpose
Contains server related information
Anything that doesn’t belong to a specific pool goes to ETC
![Page 8: Workload Analysis of a Large-Scale Key-Value Store](https://reader035.vdocument.in/reader035/viewer/2022081503/56814cf3550346895db9f6d1/html5/thumbnails/8.jpg)
8
Analysis
Workload Characteristics
Locality, Cache Behavior
![Page 9: Workload Analysis of a Large-Scale Key-Value Store](https://reader035.vdocument.in/reader035/viewer/2022081503/56814cf3550346895db9f6d1/html5/thumbnails/9.jpg)
9
Request Composition> 99.8% GET
GET:UPDATE = 30:1
![Page 10: Workload Analysis of a Large-Scale Key-Value Store](https://reader035.vdocument.in/reader035/viewer/2022081503/56814cf3550346895db9f6d1/html5/thumbnails/10.jpg)
10
Key Size Distribution90% of VAR keys are 31B
USR keys are 16B or 21B
ETC is heterogeneous
![Page 11: Workload Analysis of a Large-Scale Key-Value Store](https://reader035.vdocument.in/reader035/viewer/2022081503/56814cf3550346895db9f6d1/html5/thumbnails/11.jpg)
11
Value Size DistributionUSR values are only 2B
90% of values are smaller than 500B
![Page 12: Workload Analysis of a Large-Scale Key-Value Store](https://reader035.vdocument.in/reader035/viewer/2022081503/56814cf3550346895db9f6d1/html5/thumbnails/12.jpg)
12
Value Size Dist. By Overall Weight
90% of data is generated by values of 500B or smaller except ETC
90% is 10KB or smaller values for ETC
![Page 13: Workload Analysis of a Large-Scale Key-Value Store](https://reader035.vdocument.in/reader035/viewer/2022081503/56814cf3550346895db9f6d1/html5/thumbnails/13.jpg)
13
Request Rate Over TimeAll pools show diurnal pattern except SYS
![Page 14: Workload Analysis of a Large-Scale Key-Value Store](https://reader035.vdocument.in/reader035/viewer/2022081503/56814cf3550346895db9f6d1/html5/thumbnails/14.jpg)
14
Request Rate Over Time (ETC)
Night time in Western Semiphere
North America starts its day
![Page 15: Workload Analysis of a Large-Scale Key-Value Store](https://reader035.vdocument.in/reader035/viewer/2022081503/56814cf3550346895db9f6d1/html5/thumbnails/15.jpg)
15
Analysis
Workload Characteristics
Locality, Cache Behavior
![Page 16: Workload Analysis of a Large-Scale Key-Value Store](https://reader035.vdocument.in/reader035/viewer/2022081503/56814cf3550346895db9f6d1/html5/thumbnails/16.jpg)
16
Repeating Keys0.0003% of keys in 10% of requests in ETC
1% of keys in 55% of requests in ETC
Least frequent 50% of keys in 1% of requests in ETC
![Page 17: Workload Analysis of a Large-Scale Key-Value Store](https://reader035.vdocument.in/reader035/viewer/2022081503/56814cf3550346895db9f6d1/html5/thumbnails/17.jpg)
17
Locality Over Time
USR APP ETC VAR SYS020406080
100
% of unique keys out of total in unit time
5min 60min
![Page 18: Workload Analysis of a Large-Scale Key-Value Store](https://reader035.vdocument.in/reader035/viewer/2022081503/56814cf3550346895db9f6d1/html5/thumbnails/18.jpg)
18
Reuse Period of Keys99.9% of SYS keys are reused in 1hr
88.5% of ETC keys are reused in 1hr
96.4% of ETC keys are reused in 6hr
![Page 19: Workload Analysis of a Large-Scale Key-Value Store](https://reader035.vdocument.in/reader035/viewer/2022081503/56814cf3550346895db9f6d1/html5/thumbnails/19.jpg)
19
Hit Rate98.2% 92.9% 81.4%
93.7% 98.7%
Why?
![Page 20: Workload Analysis of a Large-Scale Key-Value Store](https://reader035.vdocument.in/reader035/viewer/2022081503/56814cf3550346895db9f6d1/html5/thumbnails/20.jpg)
20
Causes of ETC Cache Misses
Compulsory
Capacity
Invalidation
70% 22% 8%
81%
13%4% 2%hit miss: compulsory miss: capacity
miss: invalidation
![Page 21: Workload Analysis of a Large-Scale Key-Value Store](https://reader035.vdocument.in/reader035/viewer/2022081503/56814cf3550346895db9f6d1/html5/thumbnails/21.jpg)
21
Conclusion
Analyzed 5 different memcached use cases
Different applications of memcached have extreme variations in access patterns
Answered pertinent questions to improve Facebook’s memcached usage
![Page 22: Workload Analysis of a Large-Scale Key-Value Store](https://reader035.vdocument.in/reader035/viewer/2022081503/56814cf3550346895db9f6d1/html5/thumbnails/22.jpg)
22
Thank You
Questions?