ramcloud scalable high-performance storage entirely in dram

16

RAMCloud Scalable High-Performance Storage Entirely in DRAM John Ousterhout, Parag Agrawal, David Erickson, Christos Kozyrakis, Jacob Leverich, David Mazières, Subhasish Mitra, Aravind Narayanan, Guru Parulkar, Mendel Rosenblum, Stephen M. Rumble, Eric Stratmann, and Ryan Stutsman Stanford University Presented by Sangjin Han (many slides are stolen from https://ramcloud.stanford.edu) 1

Upload: cormac

Post on 22-Feb-2016

51 views

Category:

Documents

0 download

Report

Download

Tags:

Embed Size (px):

DESCRIPTION

RAMCloud Scalable High-Performance Storage Entirely in DRAM. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: RAMCloud Scalable High-Performance Storage Entirely in DRAM

1

RAMCloudScalable High-Performance Storage Entirely in DRAM

John Ousterhout, Parag Agrawal, David Erickson, Christos Kozyrakis, Jacob Leverich, David Mazières, Subhasish Mitra, Aravind Narayanan,

Guru Parulkar, Mendel Rosenblum, Stephen M. Rumble, Eric Stratmann, and Ryan Stutsman

Stanford University

Presented by Sangjin Han(many slides are stolen from https://ramcloud.stanford.edu)

Page 2: RAMCloud Scalable High-Performance Storage Entirely in DRAM

2

Features (or Goals)

• Low latency: 5-10 µs (not milliseconds)• High throughput: 1M operations/s• Key-value storage with 1000s of servers• No replicas• Entirely in DRAM– Disks only for backup

• Fast recovery (1-2 secs)

Page 3: RAMCloud Scalable High-Performance Storage Entirely in DRAM

3

Why DRAM?

Mid-1980’s 2009 ChangeDisk capacity 30 MB 500 GB 16667xMax. transfer rate 2 MB/s 100 MB/s 50xLatency (seek & rotate) 20 ms 10 ms 2x

Capacity/bandwidth(large blocks) 15 s 5000 s 333x

Capacity/bandwidth(1KB blocks) 600 s 58 days 8333x

Jim Gray’s Rule (1KB) 5 min. 30 hours 360x

Today 5-10 years# servers 1000 4000GB/server 64GB 256GBTotal capacity 64TB 1PBTotal server cost $4M $6M$/GB $60 $6

Page 4: RAMCloud Scalable High-Performance Storage Entirely in DRAM

create(tableId, blob) => objectId, versionread(tableId, objectId) => blob, versionwrite(tableId, objectId, blob) => versioncwrite(tableId, objectId, blob, version) => versiondelete(tableId, objectId)

June 3, 2011 RAMCloud Overview & Status Slide 4

Data Model

Tables

Identifier (64b)Version (64b)

Blob (≤1MB)

Object

(Only overwrite ifversion matches)

Page 5: RAMCloud Scalable High-Performance Storage Entirely in DRAM

Master

Backup

Master

Backup

Master

Backup

Master

Backup…

Appl.

Library

Appl.

Library

Appl.

Library

Appl.

Library…

DatacenterNetwork Coordinator

1000 – 10,000 Storage Servers

1000 – 100,000 Application Servers

32-64GB/server

5

Overall Architecture

Page 6: RAMCloud Scalable High-Performance Storage Entirely in DRAM

6

Per-Node Architecture

• One server per data• 2-3 backups for

buffered logging

Page 7: RAMCloud Scalable High-Performance Storage Entirely in DRAM

7

Coordinator

• Centralized server for data placement

• E.g,

• Clients obtain a map from the coordinator– And cache

Table # First Key Last Key Server

12 0 264 - 1 192.168.0.1

47 63,742 5,723,742 192.168.0.2

… … …

Page 8: RAMCloud Scalable High-Performance Storage Entirely in DRAM

8

Fast Recovery (First Try)RecoveryMaster

Backups

CrashedMaster

• Reconstruct data from backup logs

• Bottleneck: disk B/W• Solution: more disks

Page 9: RAMCloud Scalable High-Performance Storage Entirely in DRAM

9

Fast Recovery (Second Try)

• Randomly distribute log shards• Bottleneck: recovery master• Solution: no single recovery master

RecoveryMaster

~1000Backups

CrashedMaster

Page 10: RAMCloud Scalable High-Performance Storage Entirely in DRAM

10

Fast Recovery (Third Try)

• Temporarily distributed recovery masters• Happy?– 35 GB recovery in 1.6s using 60 nodes (SOSP 2011)

RecoveryMasters

Backups

CrashedMaster

Page 11: RAMCloud Scalable High-Performance Storage Entirely in DRAM

11

Typical Datacenter Latency

Page 12: RAMCloud Scalable High-Performance Storage Entirely in DRAM

12

RAMCloud Latency

Page 13: RAMCloud Scalable High-Performance Storage Entirely in DRAM

13

Publication History

• Unveiling: SOSP 2009 WIP• White paper: Operating Systems Review 2009– What you read

• (Call for) low latency: HotOS 2011• Fast recovery: SOSP 2011• Ongoing work: index, transaction, transport, …

• Visit http://ramcloud.stanford.edu

Page 14: RAMCloud Scalable High-Performance Storage Entirely in DRAM

14

Thoughts & Discussion

• Garbage collection?• Infiniband vs. Ethernet– HW support for transport layer?

• Killer app?• Non volatile memory?• Synchronous vs. asynchronous query• Moving data vs. Moving code• How many papers out of this project? :P

Page 15: RAMCloud Scalable High-Performance Storage Entirely in DRAM

15

Thanks

Page 16: RAMCloud Scalable High-Performance Storage Entirely in DRAM

16

SORRY

• My English still s^^ks• Would you repeat the question– SLOWLY and– CLEARLY

???

Scalable Many-Core Memory Systems Topic 1: DRAM Basics …users.ece.cmu.edu/~omutlu/pub/onur-ACACES2013-Topic1-dram-basic… · Solution 1: Tolerate DRAM ! Overcome DRAM shortcomings

Scalable Many-Core Memory Systems Lecture 1, Topic 1: DRAM ...users.ece.cmu.edu/.../onur-ACACES2013-Lecture1-dram... · Solution 1: Tolerate DRAM ! Overcome DRAM shortcomings with

Ramcloud deview2013

Memory Scaling : A Systems Architecture PerspectiveSolution 1: Tolerate DRAM ! Overcome DRAM shortcomings with " System-DRAM co-design " Novel DRAM architectures, interface, functions

HY425 Lecture 15: DRAM Technology - University of Cretehy425/2012f/lectures/lecture15...DRAM basics Advanced DRAM technology Virtual memory HY425 Lecture 15: DRAM Technology Dimitrios

Fine-Grained DRAM: Energy-Efficient DRAM for Extreme ...niladrish.org/pubs/micro17.pdf · Fine-Grained DRAM: Energy-Efficient DRAM for Extreme Bandwidth Systems Mike O’Connor∗†‡

RAMCloud Overview John Ousterhout Stanford University

Analysis of RAMCloud Crash Probabilities Asaf Cidon Stanford University

Fast Crash Recovery in RAMCloud - SIGOPS

ARCHITECTURAL TECHNIQUES TO ENHANCE DRAM …research.ece.cmu.edu/safari/thesis/ykim_defense_slides.pdfCO-DESIGN: CPU & DRAM CPU Memory Management DRAM Controller DRAM Arch & Interface

Co-Architecting Controllers and DRAM to Enhance DRAM ...camelab.org/uploads/Main/Co-Architecting Controllers and DRAM to... · Co-Architecting Controllers and DRAM to Enhance DRAM

Indexing in RAMCloud - Stanford University...January 21, 2014 Secondary Indexing in RAMCloud Slide 11 Failure/Restoration: Restoration Time Recovery: Similar to RAMCloud data recovery:

Solar-DRAM: Reducing DRAM Access Latency by Exploiting …...reads to DRAM cache lines containing “weak” or “strong” cells, and 2) writes to all of DRAM. We evaluate Solar-DRAM

Gather-Scatter DRAM: In-DRAM Address ranslationT …omutlu/pub/GSDRAM-gather...Gather-Scatter DRAM: In-DRAM Address ranslationT to Improve the Spatial Locality of Non-unit Strided

DDR3 Synchronous DRAM Memory - … · DDR3 Synchronous DRAM Memory ... Read/Write leveling . DDR3 Synchronous DRAM 3 8-bit prefetch/burst length . DDR3 Synchronous DRAM 4 Commands

DRAM Memory Controllerscs7810/pres/dram-cs7810-mem-ctlr-x2.pdf · DRAM Memory Controllers Reference: “Memory Systems: Cache, DRAM, Disk Bruce Jacob, Spencer Ng, & David Wang Today’s

DRAM Disturbance Errors - Carnegie Mellon Universityomutlu/pub/dram-row... · DRAM Disturbance Errors. DRAM Chip Row of Cells Row Row Row Row Wordline VV LOWHIGH Victim Row Victim

ADVANTECH DRAM Memory Module Portfolio Introductionags.advantech.com/PTDFiles/ADVANTECH DRAM Memory...ADVANTECH DRAM Memory Module Portfolio Introduction PAPS Product Management Q3

DRAM Technology

High Speed DRAM Interface - Hanyangiclab.hanyang.ac.kr/research/data/High-speed DRAM interface.pdf · High Speed DRAM Interface Changsik Yoo DRAM Design 3 Samsung Electronics. [email protected]@ieee.org

7 The RAMCloud Storage System - Stanford Universityweb.stanford.edu/~ouster/cgi-bin/papers/ramcloud-tocs.pdf7 The RAMCloud Storage System JOHN OUSTERHOUT, ... Storage Management

Gather-Scatter DRAM: In-DRAM Address ranslationT …people.inf.ethz.ch/omutlu/pub/GSDRAM-gather-scatter-dram...Gather-Scatter DRAM: In-DRAM Address ranslationT to Improve the Spatial

Fast Crash Recovery in RAMCloud - Stanford Universityweb.stanford.edu/~ouster/cgi-bin/papers/ramcloud-recovery.pdf · Fast Crash Recovery in RAMCloud Diego Ongaro, Stephen M. Rumble,

RAMCloud: Scalable Datacenter Storage Entirely in DRAM

ProactiveDRAM: A DRAM-initiated Retention Management ......DRAM cell DRAM row sense amp DRAM bank subarray wordline bitline Fig. 2. DRAM internal bank architecture. banks. A rank can

RAMCloud Scalable High-Performance Storage Entirely in DRAM John Ousterhout, Parag Agrawal, David Erickson, Christos Kozyrakis, Jacob Leverich, David Mazières,

DRAM Parameters - Duke Electrical and Computer Engineeringpeople.ee.duke.edu/~sorin/prior-courses/ece152-spring... · 2009. 3. 22. · DRAM Parameters • DRAM parameters • Large

Fast Crash Recovery in RAMCloud - Stanford Universityouster/cgi-bin/papers/ramcloud-recovery.pdf · crash recovery. RAMCloud’soverallgoalistoenable anewclassofapplications that

A Compilation of RAMCloud Slides (through Oct. 2012) John Ousterhout Stanford University

Memory and Object Management in RAMCloud Steve Rumble December 3 rd, 2013

Staged-Reads : Mitigating the Impact of DRAM Writes on DRAM Reads