super scale systems
DESCRIPTION
Visit itscales.org Watch http://www.youtube.com/watch?v=HeXDvOZcdD4 The social web is alive! And with it all of its technical challenges. Never had the servers to answer that many requests, never to store so much data. While the technological leaders like Facebook and Google use highly customized super scale systems that run on provincial town sized server farms distributed all over the world current web frameworks barely offer a single strategy to scale. Where the leaders use highly efficient caching mechanisms, frameworks recalculate responses over and over again. Where the leaders automatically add new machines for processing, frameworks allow to manually add machines without a designated field of work. Scales comes to the rescue! It’s a super scale web framework developed with the knowledge of this thesis that clearly splits up responsibilities and makes them individually scaleable. No response is calculated twice as 100% of the resources are stored in a highly efficient, distributed key-value storage. The content itself is created with common, well-known frameworks like the beautiful Ruby on Rails and the whole system can be monitored using a website that updates in real-time. The universal design goals for the framework are examined using load tests and experiments that put a light on the latest system limits, I/O strategies, protocols and caching technologies.TRANSCRIPT
![Page 1: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/1.jpg)
Thomas FankhauserMaster Thesis Colloquium
![Page 2: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/2.jpg)
Super Scale Systems
System Limits
I/O Strategies
Protocols & Technologies
Caching
Scales
Knowledge Stack
Scalability
OperatingFigures
Key Principles
IntegerLimits
DatabaseLimits
RequestLimits
FrameworkLimits
Concurrency
Sync
Async
HTTP
Server-SentEvents
WebSockets
Tailor vs.Superstore
Techno-logies
DesignGoals
Modules
Architecture
Application
API
3D-Hosting
Demo
![Page 3: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/3.jpg)
Super Scale Systems
System Limits
I/O Strategies
Protocols & Technologies
Caching
Scales
Knowledge Stack
Scalability
OperatingFigures
Key Principles
IntegerLimits
DatabaseLimits
RequestLimits
FrameworkLimits
Concurrency
Sync
Async
HTTP
Server-SentEvents
WebSockets
Tailor vs.Superstore
Techno-logies
DesignGoals
Modules
Architecture
Application
API
3D-Hosting
Demo
![Page 4: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/4.jpg)
Super Scale Systems
1
![Page 5: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/5.jpg)
Super Scale Systems
1Ultra Large Web Applications
![Page 6: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/6.jpg)
How large?
1
5 trillion 235 billion 943 million
310 billion 390 million
2,078 million
170 million
160 million
youTube
Google+
![Page 7: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/7.jpg)
How much do they grow per day?
1
4,335 million
341 million
1 million
500 thousand
50 thousand
youTube
Google+
![Page 8: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/8.jpg)
Scaleability!
1Structural Load
Space Distance
Bondi (2000)
![Page 9: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/9.jpg)
Super Scale Systems
System Limits
I/O Strategies
Protocols & Technologies
Caching
Scales
Knowledge Stack
Scalability
OperatingFigures
Key Principles
IntegerLimits
DatabaseLimits
RequestLimits
FrameworkLimits
Concurrency
Sync
Async
HTTP
Server-SentEvents
WebSockets
Tailor vs.Superstore
Techno-logies
DesignGoals
Modules
Architecture
Application
API
3D-Hosting
Demo
Load Scalability
Structural Scalability
![Page 10: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/10.jpg)
Super Scale Systems
System Limits
I/O Strategies
Protocols & Technologies
Caching
Scales
Knowledge Stack
Scalability
OperatingFigures
Key Principles
IntegerLimits
DatabaseLimits
RequestLimits
FrameworkLimits
Concurrency
Sync
Async
HTTP
Server-SentEvents
WebSockets
Tailor vs.Superstore
Techno-logies
DesignGoals
Modules
Architecture
Application
API
3D-Hosting
Demo
Load Scalability
Structural Scalability
![Page 11: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/11.jpg)
System Limits
2
![Page 12: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/12.jpg)
Infinitely Increment an Integer
2
JavaC
Ruby
OverflowOverflowRunning and running and ...
Danger! Overflows happen without notice!
![Page 13: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/13.jpg)
Database Rows and Query Times
2
Total Request Maximum Total Query Maximum
Single Query Time
1000ms (Nielson)500ms133ms at 3 Queries
1,441,419 Rows 3,632,492 Dedicated DB Servers for Facebook
![Page 14: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/14.jpg)
Request Limits
2
NginxNodeJS
12,049 Requests/s11,922 Requests/s
Amazon AWS Vertical : Horizontal Scaling
IO Bound2.56 : 1
Simple HTML Index Page
![Page 15: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/15.jpg)
Framework Limits
2
m1.smallm1.large
12 Requests/s77 Requests/s
Amazon AWS Vertical : Horizontal Scaling
CPU Bound1.14 : 1
Social Rails App with 100 Users, 1,000 Statuses and 1,300 Friend Connections
![Page 16: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/16.jpg)
Summary
2
CachingBottlenecks
FromTo
is EssentialShifted, never solved
request-build-respondrequest-lookup-respond
![Page 17: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/17.jpg)
Super Scale Systems
System Limits
I/O Strategies
Protocols & Technologies
Caching
Scales
Knowledge Stack
Scalability
OperatingFigures
Key Principles
IntegerLimits
DatabaseLimits
RequestLimits
FrameworkLimits
Concurrency
Sync
Async
HTTP
Server-SentEvents
WebSockets
Tailor vs.Superstore
Techno-logies
DesignGoals
Modules
Architecture
Application
API
3D-Hosting
Demo
Load Scalability
Structural Scalability
Ruby
Caching
Scale Horizontal
Preprocess
![Page 18: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/18.jpg)
Super Scale Systems
System Limits
I/O Strategies
Protocols & Technologies
Caching
Scales
Knowledge Stack
Scalability
OperatingFigures
Key Principles
IntegerLimits
DatabaseLimits
RequestLimits
FrameworkLimits
Concurrency
Sync
Async
HTTP
Server-SentEvents
WebSockets
Tailor vs.Superstore
Techno-logies
DesignGoals
Modules
Architecture
Application
API
3D-Hosting
Demo
Load Scalability
Structural Scalability
Ruby
Caching
Scale Horizontal
Preprocess
![Page 19: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/19.jpg)
I/O Strategies
3
![Page 20: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/20.jpg)
Concurrency
3
920 KB Memory
100 KB Memory
Process
Thread
4 KB Memory Fiber
![Page 21: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/21.jpg)
Sync vs. Async
3
Easy to codeBig Memory Footprint
Context Switching
Hard to codeSmall Memory FootprintReactor Loop
![Page 22: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/22.jpg)
Super Scale Systems
System Limits
I/O Strategies
Protocols & Technologies
Caching
Scales
Knowledge Stack
Scalability
OperatingFigures
Key Principles
IntegerLimits
DatabaseLimits
RequestLimits
FrameworkLimits
Concurrency
Sync
Async
HTTP
Server-SentEvents
WebSockets
Tailor vs.Superstore
Techno-logies
DesignGoals
Modules
Architecture
Application
API
3D-Hosting
Demo
Load Scalability
Structural Scalability
Ruby
Caching
Scale Horizontal
Preprocess
Fibers
Async I/O
![Page 23: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/23.jpg)
Super Scale Systems
System Limits
I/O Strategies
Protocols & Technologies
Caching
Scales
Knowledge Stack
Scalability
OperatingFigures
Key Principles
IntegerLimits
DatabaseLimits
RequestLimits
FrameworkLimits
Concurrency
Sync
Async
HTTP
Server-SentEvents
WebSockets
Tailor vs.Superstore
Techno-logies
DesignGoals
Modules
Architecture
Application
API
3D-Hosting
Demo
Load Scalability
Structural Scalability
Ruby
Caching
Scale Horizontal
Preprocess
Fibers
Async I/O
![Page 24: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/24.jpg)
Protocols and Technologies
4
![Page 25: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/25.jpg)
HTTP 1.0 4
SYN 1
AC
K 1
FIN 1
FIN-A
CK
1
SYN 2
AC
K 2
FIN 2
FIN-A
CK
2
SYN 3
AC
K 3
FIN 3
FIN-A
CK
3
40 60 80 100 120 140 160 180 200 220 240 260 280 300 320 340 360 380 400 420Client
Server
Time in ms
Open 1 Request 1 Close 1 Open 2 Request 2 Close 2 Open 3 Request 3 Close 3
TCP HandshakeTerminate Connection
40ms Processing 20ms Latency
3 Requests Each 160ms Total 480ms
![Page 26: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/26.jpg)
HTTP 1.1 4Keep-AlivePipelining
3 Requests Each 160ms Total 320ms
SYN
AC
K
FIN
FIN-A
CK
40 60 80 100 120 140 160 180 200 220 240 260 280Client
Server
Time in ms
TCP Handshake Request 1 Request 2 Request 3 TCP Termination
![Page 27: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/27.jpg)
HTTP 1.1 4Keep-AlivePipelining
3 Requests Each 160ms Total 160ms
SYN
AC
K
FIN
FIN-A
CK
40 60 80 100 120 140Client
Server
Time in ms
TCP Handshake Request 1 Request 2 Request 3 TCP Termination
![Page 28: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/28.jpg)
Server-Sent Events vs. WebSockets
4
Push onlyDOM Events
Same Origin Policy
BidirectionalPublish/SubscribeAny Origin
![Page 29: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/29.jpg)
Super Scale Systems
System Limits
I/O Strategies
Protocols & Technologies
Caching
Scales
Knowledge Stack
Scalability
OperatingFigures
Key Principles
IntegerLimits
DatabaseLimits
RequestLimits
FrameworkLimits
Concurrency
Sync
Async
HTTP
Server-SentEvents
WebSockets
Tailor vs.Superstore
Techno-logies
DesignGoals
Modules
Architecture
Application
API
3D-Hosting
Demo
Load Scalability
Structural Scalability
Ruby
Caching
Scale Horizontal
Preprocess
Fibers
Async I/O
HTTP 1.1
WebSockets
![Page 30: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/30.jpg)
Super Scale Systems
System Limits
I/O Strategies
Protocols & Technologies
Caching
Scales
Knowledge Stack
Scalability
OperatingFigures
Key Principles
IntegerLimits
DatabaseLimits
RequestLimits
FrameworkLimits
Concurrency
Sync
Async
HTTP
Server-SentEvents
WebSockets
Tailor vs.Superstore
Techno-logies
DesignGoals
Modules
Architecture
Application
API
3D-Hosting
Demo
Load Scalability
Structural Scalability
Ruby
Caching
Scale Horizontal
Preprocess
Fibers
Async I/O
HTTP 1.1
WebSockets
![Page 31: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/31.jpg)
Caching
5
![Page 32: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/32.jpg)
Tailor vs. Superstore
5
Current WebOn Demand
Highly Customized
Future WebOn StockPreselected Goods
![Page 33: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/33.jpg)
Storage Technologies
5
1,000 5,000 10,000 15,000 20,000 25,000
5
10
15
20
25
30
35
Total Items
Write Time in Seconds
File Memcached MongoDB Redis
Write Time for X 64 KB Files
![Page 34: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/34.jpg)
5
1,000 5,000 10,000 15,000 20,000 25,000
1
2
3
4
5
6
Total Items
Read Time in Seconds
File Memcached MongoDB Redis
Storage TechnologiesRead Time for X 64 KB Files
![Page 35: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/35.jpg)
Super Scale Systems
System Limits
I/O Strategies
Protocols & Technologies
Caching
Scales
Knowledge Stack
Scalability
OperatingFigures
Key Principles
IntegerLimits
DatabaseLimits
RequestLimits
FrameworkLimits
Concurrency
Sync
Async
HTTP
Server-SentEvents
WebSockets
Tailor vs.Superstore
Techno-logies
DesignGoals
Modules
Architecture
Application
API
3D-Hosting
Demo
Load Scalability
Structural Scalability
Ruby
Caching
Scale Horizontal
Preprocess
Fibers
Async I/O
HTTP 1.1
WebSockets
Redis
![Page 36: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/36.jpg)
Super Scale Systems
System Limits
I/O Strategies
Protocols & Technologies
Caching
Scales
Knowledge Stack
Scalability
OperatingFigures
Key Principles
IntegerLimits
DatabaseLimits
RequestLimits
FrameworkLimits
Concurrency
Sync
Async
HTTP
Server-SentEvents
WebSockets
Tailor vs.Superstore
Techno-logies
DesignGoals
Modules
Architecture
Application
API
3D-Hosting
Demo
Load Scalability
Structural Scalability
Ruby
Caching
Scale Horizontal
Preprocess
Fibers
Async I/O
HTTP 1.1
WebSockets
Redis
![Page 37: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/37.jpg)
6
![Page 38: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/38.jpg)
Design Goals 6
Load Scalability
Structural Scalability
Ruby
Caching
Scale Horizontal
Preprocess
Fibers
Async I/O
HTTP 1.1
WebSockets
Redis
Knowledge StackMultiple Machine SupportO(1) Time ComplexityAsync I/O with FibersMultiple Cores Support100% Cache100% Timeliness of DataDecouple Reads, Processing and WritesSupport Existing FrameworksEvent SystemVersatile Content TypesResource Overview
![Page 39: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/39.jpg)
http://www.youtube.com/watch?v=HeXDvOZcdD4Animation
![Page 40: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/40.jpg)
API
Update
6
![Page 41: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/41.jpg)
API
Update
6@html = "<html><head></head><body><p>Hello World</p></body></html>"@xml = "<tracks><track><name>Islandary</name>
<artist>Thomas Fankhauser</artist></track></tracks>"@json = '[{ name : "Islandary", artist : "Thomas Fankhauser" }]'
# Push a HTMLScales.push :html => @html, :to => "/hello"
# Push a XMLScales.push :xml => @xml, :to => "/hello.xml"
# Push a JSONScales.push :json => @json, :to => "/hello.json"
# Push an Image (PNG)Scales.push :png => IO.read("path/to/image.png"), :to => "/hello.png"
Push
![Page 42: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/42.jpg)
API
Update
6# HTML Update URLsScales.update "/", "/tracks", "/overview", :format => :html
# XML Update URLsScales.update "/tracks.xml", "/overview.xml", :format => :xml
# JSON Update URLsScales.update "/tracks.json", "/overview.json", :format => :json
Update
![Page 43: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/43.jpg)
API
Update
6# HTML ModificationsScales.append :html => '<p id="track2">Track 2</p>', :to => "/tracks", :select => "#tracks"Scales.prepend :html => '<p id="track2">Track 2</p>', :to => "/tracks", :select => "#tracks"Scales.set :html => '<p id="track2">Track 2</p>', :at => "/tracks", :select => "#tracks"Scales.replace :html => '<p id="track2">Track 2</p>', :at => "/tracks", :select => "#tracks"Scales.remove :at => "/tracks", :select => "#tracks"
# XML ModificationsScales.append :xml => '<track>Track 2</track>', :to => "/tracks.xml", :select => "/tracks"Scales.prepend :xml => '<track>Track 2</track>', :to => "/tracks.xml", :select => "/tracks"Scales.set :xml => '<track>Track 2</track>', :at => "/tracks.xml", :select => "/tracks"Scales.replace :xml => '<track>Track 2</track>', :at => "/tracks.xml", :select => "/tracks/track"Scales.remove :at => "/tracks.xml", :select => "/tracks/track"
# JSON ModificationsScales.append :json => '{ "id":2, "name":"Wait for it" }', :to => "/tracks.json", :select => "$.tracks"Scales.prepend :json => '{ "id":2, "name":"Wait for it" }', :to => "/tracks.json", :select => "$.tracks"Scales.set :json => '{ "id":2, "name":"Wait for it" }', :to => "/tracks.json", :select => "$.tracks"Scales.replace :json => '{ "id":2, "name":"Wait for it" }', :to => "/tracks.json", :select => "$.tracks"Scales.remove :at => "/tracks.json", :select => "$.tracks"
Modify
![Page 44: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/44.jpg)
API
Update
6
# Destroy URLsScales.destroy "/", "/tracks", "/overview", "/overview.json", "/overview.xml"
Destroy
![Page 45: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/45.jpg)
Application
Update
6
# GET /tracks/1def show @track = Track.find(params[:id]) Scales.push :html => render("show"), :to => "/tracks/#{@track.id}"end
How does the code change? (1)
# GET /tracks/1def show @track = Track.find(params[:id]) render "show"end
![Page 46: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/46.jpg)
Application
Update
6
# PUT /tracks/1def update @track = Track.find(params[:id]) if @track.update_attributes(params[:track]) Scales.update "/tracks/#{@track.id}", "/tracks/#{@track.id}/edit", "/tracks", :format => :html redirect_to @track, notice: 'Track was successfully updated.' else render action: "edit" endend
How does the code change? (2)
# PUT /tracks/1def update @track = Track.find(params[:id]) if @track.update_attributes(params[:track]) redirect_to @track, notice: 'Track was successfully updated.' else render action: "edit" endend
![Page 47: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/47.jpg)
Application
Update
6Scale Up
require 'scales/up/rails'
desc "Scale up the cache"Scales::Up.new do |scales| # Stylesheets scales.push :css, :to => "/assets/application.css?body=1" scales.push :css, :to => "/assets/scaffolds.css?body=1" scales.push :css, :to => "/assets/tracks.css?body=1" # Javascripts scales.push :js, :to => "/assets/jquery.js?body=1" scales.push :js, :to => "/assets/jquery_ujs.js?body=1" scales.push :js, :to => "/assets/tracks.js?body=1" scales.push :js, :to => "/assets/application.js?body=1" # Images scales.push :png, :to => "/assets/rails.png" # Tracks scales.update "/", "/tracks", "/tracks/new", :format => :html Track.all.each{ |track| scales.update "/tracks/#{track.id}", "/tracks/#{track.id}/edit", :format => :html } end
![Page 48: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/48.jpg)
Live Demo
Update
6
![Page 49: Super Scale Systems](https://reader033.vdocument.in/reader033/viewer/2022042613/54868e925806b59f588b49be/html5/thumbnails/49.jpg)
Thank you!
Update
Contact [email protected]
@tommylefunksouthdesign.de
Buy My AppsPianoTabsBeatfreak
QuestionPad
Read Full Thesishttps://dl.dropbox.com/u/889313/
thomas_fankhauser_-_super_scale_systems.pdf
gem install scales
itscales.orggithub.com/scales/scales
Try
Take Away Note Preprocess everything!