the 5 pillars of cloudiness
TRANSCRIPT
5 PILLARS OF CLOUDINESS
wayne wallscloud evangelist
alex brandtsr developer
Wednesday, August 21, 13
FIVE PILLARS OF CLOUDINESS
1 2 3 4 5
Parallel Modular Horizontal Agile Secure
2
Wednesday, August 21, 13
Pillar #1:Parallel
Wednesday, August 21, 13
1MRecords
Answer
Node
4
Wednesday, August 21, 13
1MRecords
Answer
Node1-250K
Node251K-500K
Node501K-750K
Node751K-1M
5
Wednesday, August 21, 13
6
1MRecords
Answer
Node1-250K
Node251K-500K
Node501K-750K
Node751K-1M
Wednesday, August 21, 13
6
1MRecords
Answer
Node1-250K
Node251K-500K
Node501K-750K
Node751K-1M
Wednesday, August 21, 13
7
Wednesday, August 21, 13
DATA
CALCULATIONMAP
CALCULATIONREDUCE
DATA
CALC
D
C
R
D
D
C
R
D
D
C
R
D
SHUFFLE
DATA
7
Wednesday, August 21, 13
8
Wednesday, August 21, 13
Thread 1
Deadlock
9
Thread 2
Wednesday, August 21, 13
Thread 1
Deadlock
9
Thread 2
Wednesday, August 21, 13
File A
Thread 1
Deadlock
9
Thread 2
Wednesday, August 21, 13
File A
Thread 1
Deadlock
9
Thread 2
Wednesday, August 21, 13
File A File B
Thread 1
Deadlock
9
Thread 2
Wednesday, August 21, 13
File A File B
Thread 1
Deadlock
9
Thread 2
Wednesday, August 21, 13
File A File B
Thread 1
Deadlock
9
File B
Thread 2
Wednesday, August 21, 13
File A File B
Thread 1
Deadlock
9
File B
Thread 2
Wednesday, August 21, 13
File A File B
File A
Thread 1
Deadlock
9
File B
Thread 2
Wednesday, August 21, 13
File A File B
File A
Thread 1
Deadlock
9
File B
Thread 2
Wednesday, August 21, 13
File A File B
File A
Thread 1
Deadlock
9
X
File B
Thread 2
Wednesday, August 21, 13
File A File B
File A
Thread 1
Deadlock
9
X
File B
Thread 2
Wednesday, August 21, 13
File A File B
File A
Thread 1
Deadlock
9
X X
File B
Thread 2
Wednesday, August 21, 13
Solution? Scheduling
Y
XA
B
A
B
Y
X
1
2
3
4
10
Wednesday, August 21, 13
1M records
Answer
Node
1M records
Answer
Node1-250K
Node251K-500K
Node501K-750K
Node750K-1M
11
Wednesday, August 21, 13
1M records
Answer
Node
1M records
Answer
Node1-250K
Node251K-500K
Node501K-750K
Node750K-1M
11
Wednesday, August 21, 13
12
Wednesday, August 21, 13
13
Wednesday, August 21, 13
pillar #2: modular
Wednesday, August 21, 13
ONE PERSON BUILDS A CAR
15
Wednesday, August 21, 13
CHASSIS ENGINE BODY INTERIOR
16
Wednesday, August 21, 13
CHASSIS ENGINE BODY INTERIOR
17
Wednesday, August 21, 13
CHASSIS ENGINE BODY INTERIOR
18
Wednesday, August 21, 13
CHASSIS ENGINE BODY INTERIOR
SEATS FLOOR
DASHBOARD
GASAND
BRAKE PEDALS
18
Wednesday, August 21, 13
CHASSIS ENGINE BODY INTERIOR
19
Wednesday, August 21, 13
CHASSIS ENGINE BODY INTERIOR
20
Wednesday, August 21, 13
defect
CHASSIS ENGINE BODY INTERIOR
20
Wednesday, August 21, 13
defect defect
CHASSIS ENGINE BODY INTERIOR
20
Wednesday, August 21, 13
defect defect defect
CHASSIS ENGINE BODY INTERIOR
20
Wednesday, August 21, 13
defect defect defectdefectCHASSIS ENGINE BODY INTERIOR
20
Wednesday, August 21, 13
defect defect defectdefect LEMONCHASSIS ENGINE BODY INTERIOR
20
Wednesday, August 21, 13
Pillar #3:horizontal
Wednesday, August 21, 13
2 GB2 CORES
22
8 GB4 CORES
Wednesday, August 21, 13
2 GB2 CORES
VERTICAL
22
8 GB4 CORES
Wednesday, August 21, 13
2 GB2 CORES
VERTICAL
22
8 GB4 CORES
Wednesday, August 21, 13
2 GB2 CORES
VERTICAL
HORIZONTAL
22
8 GB4 CORES
Wednesday, August 21, 13
2 GB2 CORES
VERTICAL
HORIZONTAL
22
8 GB4 CORES
Wednesday, August 21, 13
2 GB2 CORES
2 GB2 CORES
VERTICAL
HORIZONTAL
22
8 GB4 CORES
Wednesday, August 21, 13
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
VERTICAL
HORIZONTAL
22
8 GB4 CORES
Wednesday, August 21, 13
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
VERTICAL
HORIZONTAL
22
8 GB4 CORES
Wednesday, August 21, 13
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
VERTICAL
HORIZONTAL
22
8 GB4 CORES
Wednesday, August 21, 13
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
VERTICAL
HORIZONTAL
22
8 GB4 CORES
Wednesday, August 21, 13
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
VERTICAL
HORIZONTAL
22
8 GB4 CORES
Wednesday, August 21, 13
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
VERTICAL
HORIZONTAL
SHARED NOTHING THROUGH STATELESSNESS
22
8 GB4 CORES
Wednesday, August 21, 13
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
VERTICAL
HORIZONTAL
SHARED NOTHING THROUGH STATELESSNESS
22
8 GB4 CORES
Wednesday, August 21, 13
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
VERTICAL
HORIZONTAL
SHARED NOTHING THROUGH STATELESSNESS
2 GB2 CORES
22
8 GB4 CORES
Wednesday, August 21, 13
23
Wednesday, August 21, 13
24
Wednesday, August 21, 13
ONE SERVER NEAR
CAPACITY
24
Wednesday, August 21, 13
ONE SERVER NEAR
CAPACITY
ADD ANOTHER SERVER
24
Wednesday, August 21, 13
ONE SERVER NEAR
CAPACITY
ADD ANOTHER SERVER
25
Wednesday, August 21, 13
1234 MESSAGE QUEUE
26
Wednesday, August 21, 13
1234MESSAGE QUEUE
27
Wednesday, August 21, 13
1
2
3
4
28
MESSAGE QUEUE
Wednesday, August 21, 13
1
2
3
4
28
56789101112 MESSAGE QUEUE
Wednesday, August 21, 13
29
5678 9101112MESSAGE QUEUE
1
2
3
4
Wednesday, August 21, 13
29
5678 9101112
ADD MORE SERVERSBASED ON
MESSAGE QUEUE
MESSAGE QUEUE
1
2
3
4
Wednesday, August 21, 13
MESSAGE QUEUE
30
5
6
7
8
9
10
11
12
13141516
ADD MORE SERVERSBASED ON
MESSAGE QUEUE
1
2
3
4
Wednesday, August 21, 13
31
5678 9101112
13141516
ADD MORE SERVERSBASED ON
MESSAGE QUEUE
1234
MESSAGE QUEUE
Wednesday, August 21, 13
32
5678 9101112
13141516
ADD MORE SERVERSBASED ON
MESSAGE QUEUE
1234
MESSAGE QUEUE
Wednesday, August 21, 13
33
5678 9101112
13141516
ADD MORE SERVERSBASED ON
MESSAGE QUEUE1234
MESSAGE QUEUE
Wednesday, August 21, 13
33
5678 9101112
13141516
ADD MORE SERVERSBASED ON
MESSAGE QUEUE1234
MESSAGE QUEUE
DECREASE SERVERSBASED ON
MESSAGE QUEUE
Wednesday, August 21, 13
MESSAGE QUEUE
34
5678 9101112
13
14
15
16
ADD MORE SERVERSBASED ON
MESSAGE QUEUE1234
DECREASE SERVERSBASED ON
MESSAGE QUEUE
Wednesday, August 21, 13
Pillar #4:agile
Wednesday, August 21, 13
36
Wednesday, August 21, 13
Features In Progress Blocked Tabled Testing Complete
Feature A
Feature B
Feature C
37
Monday
Wednesday, August 21, 13
Features In Progress Blocked Tabled Testing Complete
Feature A
Feature B
Feature C
38
Tuesday
Wednesday, August 21, 13
Features In Progress Blocked Tabled Testing Complete
Feature A
Feature B
Feature C
Feature E
Feature D
38
Tuesday
Wednesday, August 21, 13
Features In Progress Blocked Tabled Testing Complete
Feature AFeature BFeature C
Feature E
Feature G
39
Wednesday
Wednesday, August 21, 13
Features In Progress Blocked Tabled Testing Complete
Feature AFeature BFeature C
Feature E
Feature D
Feature F
Feature G
39
Wednesday
Wednesday, August 21, 13
Features In Progress Blocked Tabled Testing Complete
Feature A
Feature B
Feature C
Feature E
Feature D
Feature F
Feature G
40
Friday
Wednesday, August 21, 13
Features In Progress Blocked Tabled Testing Complete
Feature A
Feature B
Feature C
Feature E
Feature D
Feature F
Feature G
Feature H
Feature I
40
Friday
Wednesday, August 21, 13
Features In Progress Blocked Tabled Testing Complete
Feature A
Feature B
Feature C
Feature EFeature D
Feature F
Feature G
Feature H
Feature I
41
Friday - WK 2
Wednesday, August 21, 13
57%
14%
29%
Failed Successful Challenged
49%
42%
9%
Waterfall Agile
Source: The CHAOS Manifesto, The Standish Group, 2012
42
Wednesday, August 21, 13
43
Continuous Integration/Continuous Delivery
Wednesday, August 21, 13
44
Quality
Wednesday, August 21, 13
45
Release Cycle
Wednesday, August 21, 13
IMAGE: blogs.bu.edu
46
Wednesday, August 21, 13
47
Lint Testing
Test Driven Development
Functional Testing
Black Box
Integration Testing
Unit Testing
Behavior Driven Development
White Box
Smoke Testing
Acceptance Testing
Wednesday, August 21, 13
ENSURES QUALITY
47
Lint Testing
Test Driven Development
Functional Testing
Black Box
Integration Testing
Unit Testing
Behavior Driven Development
White Box
Smoke Testing
Acceptance Testing
Wednesday, August 21, 13
48
Lint Testing
Test Driven Development
Functional Testing
Black Box
Integration Testing
Unit Testing
Behavior Driven Development
White Box
Smoke Testing
Acceptance Testing
Wednesday, August 21, 13
ENSURES CONFIDENCE
48
Lint Testing
Test Driven Development
Functional Testing
Black Box
Integration Testing
Unit Testing
Behavior Driven Development
White Box
Smoke Testing
Acceptance Testing
Wednesday, August 21, 13
Pillar #5:secure
Wednesday, August 21, 13
50
Wednesday, August 21, 13
51
PRESENTATION
APPLICATION
STORAGE
Wednesday, August 21, 13
51
PRESENTATION
APPLICATION
STORAGE
INTERFACESECURITYACROSS ALL LAYERS
Wednesday, August 21, 13
52
Wednesday, August 21, 13
53
Host-based Authentication
Wednesday, August 21, 13
54
User Authentication
Wednesday, August 21, 13
55
Hybrid Cloud
Wednesday, August 21, 13
56
Trustworthiness as an Organization
Wednesday, August 21, 13