non-volatile memory dbmspavlo/courses/fall2013/static/slides/project-team… · comparison with...
TRANSCRIPT
![Page 1: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/1.jpg)
NON-VOLATILE MEMORY DBMS
Joy Arulraj 15-799 : Final Presentation
![Page 2: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/2.jpg)
TRADITIONAL DBMS • Long Transactions – Interactive workload
• Small Memory Capacity – Disk latency
![Page 3: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/3.jpg)
REALITY CHECK • Short Transactions • Repetitive Workloads • Large Memory Capacity
![Page 4: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/4.jpg)
MAIN-MEMORY DBMS
![Page 5: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/5.jpg)
MAIN-MEMORY DBMS • Disk used only for logging/recovery • High-throughput OLTP
![Page 6: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/6.jpg)
CHALLENGES • DRAM SCALING LIMIT – Reliable sensing
• RECOVERY LATENCY – Throughput
![Page 7: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/7.jpg)
NVM DBMS
![Page 8: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/8.jpg)
PROPERTIES
10/ 18
DRAM NVM SSD DISK
READ LATENCY 1x 2-5x 500x 105x
WRITE LATENCY 1x 2-5x 5000x 105x
PERSISTENCE û ü ü ü
SCALABLITY û ü ü ü
BYTE-LEVEL ACCESS ü ü û û
![Page 9: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/9.jpg)
STORAGE CHOICES
11/ 18
DRAM
DISK
DRAM DRAM
NVM
II I III
NVM
IV
![Page 10: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/10.jpg)
ENVIRONMENT • INTEL NVM EMULATOR – Instrumented motherboard • PERSISTENT MEMORY FILE SYSTEM – MMAP interface to PM
![Page 11: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/11.jpg)
NVM HARDWARE EMULATOR • READ LATENCY – LLC Miss Stalls
• WRITE BANDWIDTH – Throttling in memory controller
![Page 12: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/12.jpg)
PM FILE SYSTEM
APPS
OS VMM PMFS Direct to PM
PM DRAM
MMAP READ/WRITE MALLOC/FREE
![Page 13: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/13.jpg)
GOALS • MMAP-BASED STORAGE MANAGER • EVALUATION ON NVM EMULATOR • MOVE INDEX STORAGE TO NVM
18/ 18
![Page 14: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/14.jpg)
IMPLEMENTATION • STORAGE MANAGER – H-Store Table – Per-table memory mapped file – Metadata for recovery
![Page 15: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/15.jpg)
IMPLEMENTATION • STORAGE MANAGER – Pool Storage – String Pool (VARCHAR)
![Page 16: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/16.jpg)
IMPLEMENTATION • STL ALLOCATOR – Index Storage – On top of Storage Manager – Ordered and Unordered map
![Page 17: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/17.jpg)
EXPERIMENTS
![Page 18: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/18.jpg)
SETUP • INTEL NVM EMULATOR – 62 GB DRAM
• YCSB BENCHMARK – Zipfian distribution – Read Only (100% Reads) – Update Heavy (50% Updates, 50% Reads)
![Page 19: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/19.jpg)
READ-ONLY WORKLOAD
![Page 20: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/20.jpg)
0 50000
100000 150000 200000 250000 300000 350000 400000
HStore (MMAP)
MySQL
THRO
UGHP
UT (T
XNS./
SEC)
0.5
1.5
2X DRAM LATENCY
![Page 21: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/21.jpg)
0 50000
100000 150000 200000 250000 300000 350000 400000
HStore (MMAP)
MySQL
THRO
UGHP
UT (T
XNS./
SEC)
0.5
1.5
16X DRAM LATENCY
![Page 22: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/22.jpg)
UPDATE-HEAVY WORKLOAD
![Page 23: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/23.jpg)
IMPACT OF NVM LATENCY
0
10000
20000
30000
40000
50000
1X 2X 4X 8X
THRO
UGHP
UT (T
XNS.
/SEC
)
NVM LATENCY SLOWDOWN (W.R.T. DRAM LATENCY)
Skew
0.5
1
1.5
![Page 24: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/24.jpg)
COMPARISON WITH DISK DBMS
0
10000
20000
30000
40000
50000
1X 2X 4X 8X
THRO
UGHP
UT (T
XNS.
/SEC
)
NVM LATENCY SLOWDOWN (W.R.T. DRAM LATENCY)
H-Store (MMAP)
MySQL
![Page 25: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/25.jpg)
CONCLUSION • Throughput comparison with MySQL – 4.5X on read-only workloads – 1.5X on update-heavy workloads
• Update-heavy workload – msync overhead
![Page 26: NON-VOLATILE MEMORY DBMSpavlo/courses/fall2013/static/slides/project-team… · comparison with disk dbms 0 10000 20000 30000 40000 50000 1x 2x 4x 8x throughput (txns./sec) nvm latency](https://reader034.vdocument.in/reader034/viewer/2022042218/5ec3ab595e5c0f53e3380a87/html5/thumbnails/26.jpg)
CONCLUSION • A new design ? – Recovery – Concurrency Control
THANKS !
NVM
IV