byzantine fault tolerant cloud storage for storing sensor data

Cloud Storagefor storing sensor data

Jos van der Til

Byzantine fault tolerant

WHAT WAS THAT FIRST SLIDE?

I KNOW SOME OF THESE WORDS!SENSOR DATA

I KNOW SOME OF THESE WORDS!SENSOR DATAVARIES BY MEASUREMENT INTERVAL


VARIES BY DIMENSIONALITY



VARIES BY SIZE



VARIES BY SIZE

IMPORTANT: IMAGES AND VIDEO ARE ALSO SENSOR DATA!

I KNOW SOME OF THESE WORDS!

CLOUD STORAGESENSOR DATA


CLOUD STORAGESENSOR DATAUNLIMITED STORAGE



ACCESSIBLE FROM ANYWHERE

UNLIMITED STORAGE




ACCESSIBLE ANYTIME

UNLIMITED STORAGE




ACCESSIBLE ANYTIME

UNLIMITED STORAGE

PAY FOR WHAT YOU USE!


CLOUD STORAGEFAULT TOLERANT

SENSOR DATA



SENSOR DATAPROCESSES ONLY CRASH



SENSOR DATAPROCESSES ONLY CRASH…RIGHT?



HOW BAD CAN IT GET? SENSOR DATA



BYZANTINE

SENSOR DATA

HOW DO PROCESSES FAIL?

Fail stop Crash


Fail stop Crash

Send OmissionReceive

Omission

General

Omission


Fail stop Crash


Omission

General

Omission

Arbitrary failures

with message

authentication



Fail stop Crash


Omission

General

Omission

Arbitrary failures

with message

authentication

Arbitrary

(Byzantine)

failures

Storage clouds

Sensor Network

Measurements

Sensor server

Sto

rage

Lib

Measurements

Storage Lib

Processing server

Writer

Reader

READERS ARE PROCESSES




WRITERS ARE PROCESSES



WRITERS ARE PROCESSESbut they are cool.



WRITERS ARE PROCESSESbut they are cool.

can fail without causing damage.

are only expected to fail by crashing.

HOW DO PROCESSES FAIL?CLOUD PROVIDERS ARE PROCESSES


but they are NOT cool.



can leak your data



can corrupt your data



can delete your data



can stop responding to your requests



HAS FULL CONTROL OVER YOUR DATA BUT BEHAVES BYZANTINE

YOUR DATA IS STORED

IN A PROCESS THAT CAN FAIL

BYZANTINE

HOW TO ACHIEVE

BYZANTINE FAULT TOLERANCE?

DO NOT TRUST A SINGLE CLOUD!

DO TRUST MULTIPLE CLOUDS!

UPLOAD DATA TO ALL THE CLOUDS!

HOW MANY CLOUDS DO WE NEED?

𝑛 ≥ 3𝑓 + 1

HOW IS DATA STORED?

DATA IS STORED IN A QUORUM OF CLOUD PROVIDERSQUORUM OF 2f+1 PROVIDERS TO BE EXACT


DATA CAN NOT BE RETRIEVED BY LESS THAN f+1 PROVIDERS



ENCRYPTION

SECRET SHARING



SHOULD NOT REQUIRE n TIMES THE STORAGE SPACE




ERASURE CODING




ERASURE CODING𝑛

𝑓 + 1

−1

=𝑓 + 1

𝑛

lim𝑓→∞

𝑓 + 1

𝑛=

𝑓 + 1

3𝑓 + 1=1

3

SPACE EFFICIENCY

MAXIMUM SPACE EFFICIENCY

lim𝑓→∞

1

𝑛= 0NOT BAD COMPARED TO

TRADITIONAL APPROACHDATA IS A BLOCK!


I WANT TO READ THIS BLOCK


I WANT TO ENCRYPT THIS BLOCK


I WANT TO HASH THIS BLOCK


I WANT TO UPLOAD THIS BLOCK


I WANT TO DOWNLOAD THIS BLOCK


BLOCK DOES NOT FIT IN MEMORY


BLOCK DOES NOT FIT IN MEMORY

:(

NOW WHAT?

MY APPROACHDATA IS A STREAM!

MY APPROACHDATA IS A STREAM…of blocks!


Every block should fit into memory



Every block is processed independent



Every block is processed independent

Every block has a checksum (think BitTorrent)

MY APPROACHHAS MORE OVERHEAD IN MEMORY AND STORAGE


REQUIRES LESS MEMORY FOR PROCESSING


REQUIRES LESS MEMORY FOR PROCESSING

CAN FAIL FASTER, SAVING BANDWIDTH

WHEN IS THIS USED?

WHEN IS THIS USED?

DATA IS PETABYTE SCALE


…OR ATLEAST A COUPLE OF 100 TERABYTES A YEAR

WHEN IS THIS USED?



DATA IS CONTINOUSLY ADDED

WHEN IS THIS USED?



DATA IS CONTINOUSLY ADDED

TIME BETWEEN BATCHES IS TOO LONG

WHEN IS THIS USED?



DATA IS CONTINOUSLY ADDEDTIME BETWEEN BATCHES IS TOO LONG

KEEPING THOUSANDS OF MACHINES RUNNING IS EXPENSIVE

WHEN IS THIS USED?



DATA IS CONTINOUSLY ADDEDTIME BETWEEN BATCHES IS TOO LONG

KEEPING THOUSANDS OF MACHINES RUNNING IS EXPENSIVE

WHAT IF MY HADOOP CLUSTER IS DESTROYED?

WHEN IS THIS USED?

New Data

All data

Streaming cluster

Batch cluster Batch view

Realtime view

ClientQuery

Query

WHY NOT HADOOP?

WHY NOT HADOOP?

HADOOP STILL HAS ITS PLACE

WHY NOT HADOOP?

HADOOP STILL HAS ITS PLACE

JUST NOT FOR STORAGE

New Data

All data

Streaming cluster

Batch cluster Batch view

Realtime view

ClientQuery

Query

OK…WHY NOT HADOOP STORAGE?

OK…WHY NOT HADOOP STORAGE?BATCH LAYER IS AN EXCELLENT PLACE FOR HADOOP PROCESSING


BUT HADOOP STORAGE IS EXPENSIVE!


BUT HADOOP STORAGE REQUIRES MAINTENANCE!


BUT HADOOP STORAGE IS ONLINE EVEN WHEN IDLE!


BUT HADOOP STORAGE CONSUMES LOTS OF ENERGY!

DOES THIS WORK?

PERFORMANCE

Requests done by 16 threads concurrently

PERFORMANCE


8 core virtual machine

at least 4 GB RAM (but often > 16GB)

PERFORMANCE



At least 4 GB RAM (but often > 16GB)

f = 1, thus n = 4

PERFORMANCE



At least 4 GB RAM (but often > 16GB)

f = 1, thus n = 4

Two implementations:

Streaming DepSky-A

Streaming DepSky-CA

PERFORMANCE

Throughput downstream (per thread):

Filesize 4MB, 750 KB/second (90th percentile)

Filesize 8MB, 1 MB/second (90th percentile)

Throughput upstream (per thread):

Filesize 4MB, 1.2 MB/second (90th percentile)

Filesize 8MB, 1.7 MB/second (90th percentile)

0.9997

0.9998

0.9999

1.0000

0 5 10 15 20 25

log2(Filesize (b))

Success r

ate

HTTP Verb

GET

PUT

DELETE

LIST

Streaming DepSky-A

0.994

0.996

0.998

1.000

0 5 10 15 20 25

log2(Filesize (b))

Success r

ate

HTTP Verb

GET

PUT

DELETE

LIST

Streaming DepSky-CA

AVAILABILITY

thanks!

Thesis available at:

http://www.cs.rug.nl/~aiellom/tesi/vdtil.pdf

http://www.cs.rug.nl/~aiellom/tesi/vdtil.pdf

byzantine fault tolerant cloud storage for storing sensor data

Technology