reactive supply to changing demand

114
Reactive Supply to Changing Demand Jonas Bonér Typesafe CTO & co-founder @jboner Scalable Trait: React 2014

Upload: jonas-boner

Post on 19-Aug-2014

946 views

Category:

Engineering


13 download

DESCRIPTION

This is the talk I gave at React Conf 2014 in London. Recording available here: https://www.youtube.com/watch?v=mBFdj7w4aFA Abstract: Reactive applications need to be able to respond to demand, be elastic and ready to scale up, down, in and out—taking full advantage of mobile, multi-core and cloud computing architectures—in real time. In this talk we will discuss the guiding principles making this possible through the use of share-nothing and non-blocking designs, applied all the way down the stack. We will learn how to deliver systems that provide reactive supply to changing demand.

TRANSCRIPT

Page 1: Reactive Supply To Changing Demand

Reactive Supply to Changing Demand

Jonas Bonér Typesafe

CTO & co-founder @jboner

Scalable Trait: React 2014

Page 2: Reactive Supply To Changing Demand

Outline

1. Introduction

2. Scale Up

3. Scale Out

4. Bring It Together

2

Page 3: Reactive Supply To Changing Demand

Outline

1. Introduction

2. Scale Up

3. Scale Out

4. Bring It Together

3

Page 4: Reactive Supply To Changing Demand

4

Page 5: Reactive Supply To Changing Demand

4

Page 6: Reactive Supply To Changing Demand

What is Scalability?

Page 7: Reactive Supply To Changing Demand

6

“The house in which Amdahl wakes up very early each day and rules with an iron fist.”

- Martin Thompson (originally Gil Tene)

Page 8: Reactive Supply To Changing Demand

6

“The house in which Amdahl wakes up very early each day and rules with an iron fist.”

- Martin Thompson (originally Gil Tene)

Page 9: Reactive Supply To Changing Demand

7

“Capable of being easily expanded or upgraded on demand” - Merriam Webster Dictionary

Page 10: Reactive Supply To Changing Demand

8

“A service is said to be scalable if when we increase the resources in a system, it results in increased performance

in a manner proportional to resources added.” - Werner Vogels

Page 11: Reactive Supply To Changing Demand

Scalability vs Performance

Page 12: Reactive Supply To Changing Demand

Why do we need to Scale on Demand?

Page 13: Reactive Supply To Changing Demand

The rules of the game

have changed

Page 14: Reactive Supply To Changing Demand

12

Apps in the 60s-90s were written for

Apps today are written for

Page 15: Reactive Supply To Changing Demand

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines

Page 16: Reactive Supply To Changing Demand

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Page 17: Reactive Supply To Changing Demand

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors

Page 18: Reactive Supply To Changing Demand

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Page 19: Reactive Supply To Changing Demand

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM

Page 20: Reactive Supply To Changing Demand

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM Cheap RAM

Page 21: Reactive Supply To Changing Demand

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM Cheap RAM

Expensive disk

Page 22: Reactive Supply To Changing Demand

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM Cheap RAM

Expensive disk Cheap disk

Page 23: Reactive Supply To Changing Demand

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM Cheap RAM

Expensive disk Cheap disk

Slow networks

Page 24: Reactive Supply To Changing Demand

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM Cheap RAM

Expensive disk Cheap disk

Slow networks Fast networks

Page 25: Reactive Supply To Changing Demand

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM Cheap RAM

Expensive disk Cheap disk

Slow networks Fast networks

Few concurrent users

Page 26: Reactive Supply To Changing Demand

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM Cheap RAM

Expensive disk Cheap disk

Slow networks Fast networks

Few concurrent users Lots of concurrent users

Page 27: Reactive Supply To Changing Demand

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM Cheap RAM

Expensive disk Cheap disk

Slow networks Fast networks

Few concurrent users Lots of concurrent users

Small data sets

Page 28: Reactive Supply To Changing Demand

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM Cheap RAM

Expensive disk Cheap disk

Slow networks Fast networks

Few concurrent users Lots of concurrent users

Small data sets Large data sets

Page 29: Reactive Supply To Changing Demand

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM Cheap RAM

Expensive disk Cheap disk

Slow networks Fast networks

Few concurrent users Lots of concurrent users

Small data sets Large data sets

Latency in seconds

Page 30: Reactive Supply To Changing Demand

12

Apps in the 60s-90s were written for

Apps today are written for

Single machines Clusters of machines

Single core processors Multicore processors

Expensive RAM Cheap RAM

Expensive disk Cheap disk

Slow networks Fast networks

Few concurrent users Lots of concurrent users

Small data sets Large data sets

Latency in seconds Latency in milliseconds

Page 31: Reactive Supply To Changing Demand
Page 32: Reactive Supply To Changing Demand

14

Page 33: Reactive Supply To Changing Demand

14

Page 34: Reactive Supply To Changing Demand

14

Page 35: Reactive Supply To Changing Demand

14

Page 36: Reactive Supply To Changing Demand

Cost Gravity is at Work

15

Page 37: Reactive Supply To Changing Demand

Cost Gravity is at Work

15

Page 38: Reactive Supply To Changing Demand

Outline

1. Introduction

2. Scale Up

3. Scale Out

4. Bring It Together

16

Page 39: Reactive Supply To Changing Demand

UPScale

Page 40: Reactive Supply To Changing Demand

18

Modern CPU architecture

Page 41: Reactive Supply To Changing Demand

18

Modern CPU architecture

Page 42: Reactive Supply To Changing Demand

18

Modern CPU architecture

Page 43: Reactive Supply To Changing Demand

The CPU is gambling. Taking bets.

19

Page 44: Reactive Supply To Changing Demand

Maximize Locality of Reference

Page 45: Reactive Supply To Changing Demand

Minimize Contention

Page 46: Reactive Supply To Changing Demand

Common points of

22

ApplicationPhysical

contention

Page 47: Reactive Supply To Changing Demand

23

Neverever

Page 48: Reactive Supply To Changing Demand

23

Neverever

Page 49: Reactive Supply To Changing Demand

Block

23

Neverever

Page 50: Reactive Supply To Changing Demand

24

GO

Page 51: Reactive Supply To Changing Demand

Async

24

GO

Page 52: Reactive Supply To Changing Demand

Divide & Conquer

25

Page 53: Reactive Supply To Changing Demand

26

Page 54: Reactive Supply To Changing Demand

27

Single Writer Principle

Page 55: Reactive Supply To Changing Demand

27

Single Writer PrincipleIO deviceProducers

SERIAL & CONTENDED

Page 56: Reactive Supply To Changing Demand

27

Single Writer PrincipleIO deviceProducers

SERIAL & CONTENDED

IO deviceProducers Actor or Queue

BATCHED & UNCONTENDED

Page 57: Reactive Supply To Changing Demand

The Role of Immutable State

28

Page 58: Reactive Supply To Changing Demand

The Role of Immutable State

28

Page 59: Reactive Supply To Changing Demand

The Role of Immutable State

• Great to represent facts

• Events

• Database snapshots

28

Page 60: Reactive Supply To Changing Demand

The Role of Immutable State

• Great to represent facts

• Events

• Database snapshots

• Less ideal for a “working” data set

• Persistent data structures can increase contention

• Instead use a Share Nothing Architecture with mutable state within each isolated processing unit

28

Page 61: Reactive Supply To Changing Demand

29

Needs to be async and non-blocking all the way down

Page 62: Reactive Supply To Changing Demand

29

Needs to be async and non-blocking all the way down

…or Amdahl’s Law will hunt you down

Page 63: Reactive Supply To Changing Demand

29

Needs to be async and non-blocking all the way down

…or Amdahl’s Law will hunt you down

Page 64: Reactive Supply To Changing Demand

30

NOTHINGshare

Page 65: Reactive Supply To Changing Demand

Scale on Demand

Page 66: Reactive Supply To Changing Demand

Outline

1. Introduction

2. Scale Up

3. Scale Out

4. Bring It Together

32

Page 67: Reactive Supply To Changing Demand

OUTScale

Page 68: Reactive Supply To Changing Demand

Distributed Computing is the !

• Mobile

• Cloud Services, REST etc.

• NOSQL DBs

• Big Data

34

NEW NORMAL

Page 69: Reactive Supply To Changing Demand

What is the essence of distributed computing?

35

Page 70: Reactive Supply To Changing Demand

What is the essence of distributed computing?

To try to overcome that 1. Information travels at the speed of light

2. Independent things fail independently

35

Page 71: Reactive Supply To Changing Demand

36

Page 72: Reactive Supply To Changing Demand

Node Node Node

36

Node

Page 73: Reactive Supply To Changing Demand

Middleware

Node Node Node

36

Node

Page 74: Reactive Supply To Changing Demand

Cluster/Rack/Datacenter

Cluster/Rack/Datacenter

Cluster/Rack/Datacenter

Middleware

Node Node Node

36

Node

Page 75: Reactive Supply To Changing Demand

37

Peter Deutsch’s 8 Fallacies of

Distributed Computing

Page 76: Reactive Supply To Changing Demand

37

1. The network is reliable

Peter Deutsch’s 8 Fallacies of

Distributed Computing

Page 77: Reactive Supply To Changing Demand

37

1. The network is reliable2. Latency is zero

Peter Deutsch’s 8 Fallacies of

Distributed Computing

Page 78: Reactive Supply To Changing Demand

37

1. The network is reliable2. Latency is zero3. Bandwidth is infinitePeter Deutsch’s

8 Fallacies of Distributed Computing

Page 79: Reactive Supply To Changing Demand

37

1. The network is reliable2. Latency is zero3. Bandwidth is infinite4. The network is secure

Peter Deutsch’s 8 Fallacies of

Distributed Computing

Page 80: Reactive Supply To Changing Demand

37

1. The network is reliable2. Latency is zero3. Bandwidth is infinite4. The network is secure5. Topology doesn't change

Peter Deutsch’s 8 Fallacies of

Distributed Computing

Page 81: Reactive Supply To Changing Demand

37

1. The network is reliable2. Latency is zero3. Bandwidth is infinite4. The network is secure5. Topology doesn't change6. There is one administrator

Peter Deutsch’s 8 Fallacies of

Distributed Computing

Page 82: Reactive Supply To Changing Demand

37

1. The network is reliable2. Latency is zero3. Bandwidth is infinite4. The network is secure5. Topology doesn't change6. There is one administrator7. Transport cost is zero

Peter Deutsch’s 8 Fallacies of

Distributed Computing

Page 83: Reactive Supply To Changing Demand

37

1. The network is reliable2. Latency is zero3. Bandwidth is infinite4. The network is secure5. Topology doesn't change6. There is one administrator7. Transport cost is zero8. The network is homogeneous

Peter Deutsch’s 8 Fallacies of

Distributed Computing

Page 84: Reactive Supply To Changing Demand

The Graveyard of Distributed Systems

• Distributed Shared Mutable State

•⇒ EVIL (where N is number of nodes)

• Serializable Distributed Transactions

• Synchronous RPC

• Guaranteed Delivery

• Distributed Objects • “Sucks like an inverted hurricane” - Martin Fowler

38

N

Page 85: Reactive Supply To Changing Demand

Maximize Locality of Reference

Page 86: Reactive Supply To Changing Demand

40

Sticky Load Balancing & Sharding

Page 87: Reactive Supply To Changing Demand

40

Page 88: Reactive Supply To Changing Demand

41

Buddy Replication

Page 89: Reactive Supply To Changing Demand

41

Page 90: Reactive Supply To Changing Demand

42

Consistent Hashing

Page 91: Reactive Supply To Changing Demand

42

Page 92: Reactive Supply To Changing Demand

Minimize Contention (Coordination in the Cluster)

Page 93: Reactive Supply To Changing Demand

Strong Consistency

Page 94: Reactive Supply To Changing Demand

45

Linearizability

“Under linearizable consistency, all operations appear to have executed atomically in an order that is consistent with the

global real-time ordering of operations.” - Herlihy & Wing 1991

Page 95: Reactive Supply To Changing Demand

Strong Consistency Protocols

Page 96: Reactive Supply To Changing Demand

47

CAPTheorem

Page 97: Reactive Supply To Changing Demand

47

CAPTheorem

Page 98: Reactive Supply To Changing Demand

Eventual Consistency

Page 99: Reactive Supply To Changing Demand

49

CRDTCvRDTs/CmRDTs

Page 100: Reactive Supply To Changing Demand

50

“In general, application developers simply do not implement large scalable applications assuming

distributed transactions.” -­‐  Pat Helland

Life beyond Distributed Transactions: an Apostate’s Opinion

Page 101: Reactive Supply To Changing Demand

51

“State transitions are an important part of our problem space and should be modeled within our domain.”  

- Greg Young

Domain Events

Page 102: Reactive Supply To Changing Demand

52

"The database is a cache of a subset of the log.” - Pat Helland

The Event Log

Page 103: Reactive Supply To Changing Demand

The Event Log

• Append-Only Logging

• Database of Facts

• Two models:

• One single Event Log

• Strong Consistency

• Multiple sharded Event Logs

• Strong + Eventual Consistency

53

Page 104: Reactive Supply To Changing Demand

NOTHING

54

share

Page 105: Reactive Supply To Changing Demand

Scale on Demand

Page 106: Reactive Supply To Changing Demand

TRANSPARENCY

56

location

Page 107: Reactive Supply To Changing Demand

Outline

1. Introduction

2. Scale Up

3. Scale Out

4. Bring It Together

57

Page 108: Reactive Supply To Changing Demand

58

Page 109: Reactive Supply To Changing Demand

58

Page 110: Reactive Supply To Changing Demand

59

Page 111: Reactive Supply To Changing Demand

Data Center

59

Data Center

ClusterClusterMachineMachineJVMJVMNodeNodeThreadThreadCPUCPUCPU Socket

CPU Socket

CPU Core

CPU Core

CPU L1/L2 Cache

CPU L1/L2 Cache

Page 112: Reactive Supply To Changing Demand

60

Scaling Up and Out is essentially

the same thing

Page 113: Reactive Supply To Changing Demand

Questions?

Page 114: Reactive Supply To Changing Demand

Reactive Supply to Changing Demand

Jonas Bonér Typesafe

CTO & co-founder @jboner

Scalable Trait: React 2014

Thanks for listening