skalowalna architektura na przykładzie soccerway.com
DESCRIPTION
"Skalowalna architektura na przykładzie soccerway.com" - Adam Brodziak - Global Sports Media b.v.TRANSCRIPT
![Page 1: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/1.jpg)
By Adam BrodziakGlobal Sports Media b.v.
Scalable architecture
![Page 2: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/2.jpg)
Abstract
Adam Brodziak
An overview of modern web-based application architecture - from hardware infrastructure, through PHP/SQL code, HTML/CSS markup distribution. All of this spiced up by cache, loadbalancing and CDN.
![Page 3: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/3.jpg)
Who is this guy?
Lead developer at Global Sports Media GSM collects and process sports data GSM owns soccerway.com portal
Linux user Interested in frameworks, design patterns Semantic Web enthousiast Football (soccer) fan
![Page 4: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/4.jpg)
Topics
The Challenge Infrastructure Code Cache CDN
![Page 5: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/5.jpg)
Topics
The Challenge Infrastructure Code Cache CDN
![Page 6: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/6.jpg)
Raw numbers
7 millions visits / month 52 millions pageviews / month 1 billion request / month 6TB of traffic / month 300k users at peak time Quite a few clients using the same hardware
![Page 7: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/7.jpg)
Not so much, but...
700 leagues Livescores Game events Match statistics Rankings Editorials
![Page 8: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/8.jpg)
Traffic growth
![Page 9: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/9.jpg)
The Challenge
Loads of data to process Scores Events Stats
In real-time (livescores) Growing number of visitors 13K hits/sec at peak-time
![Page 10: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/10.jpg)
10 servers to run it all
![Page 11: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/11.jpg)
Topics
The Challenge Infrastructure Code Cache CDN
![Page 12: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/12.jpg)
It starts with one
![Page 13: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/13.jpg)
Load balancing
![Page 14: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/14.jpg)
Loadbalancing caveats
Don't relay on the local filesystem Temporary files, session, logs
Avoid assuming exclusive/single cache APC, Zend Cache
Use distributed session storage Memcache, database
Encalsulate above
![Page 15: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/15.jpg)
Separate database server
![Page 16: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/16.jpg)
DB replication
![Page 17: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/17.jpg)
Replicaton caveats
Writes only on master Reads from slaves Data consistency Replication lag
Don't do
$master->query('UPDATE session SET logged = 1');$slave->query('SELECT logged FROM session');
![Page 18: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/18.jpg)
Whole image
![Page 19: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/19.jpg)
Topics
The Challenge Infrastructure Code Cache CDN
![Page 20: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/20.jpg)
PHP is slow!
Yes, but it does not matter! Database access is slower Cache over network is slower Disk access is slower HTTP requests are slower Webservice calls are slower Discover bottlenecks before blaming PHP
![Page 21: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/21.jpg)
It's about architecture
Heavy tasks in background CRON, Gearman
Pregenerate stuff Move some code to SQL
Calculations in queries Stored procedures Triggers
C/C++ or Java for heavy computation Use PHP to glue it together
![Page 22: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/22.jpg)
PHP Frameworks
Hundreds of others Which one to choose?
![Page 23: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/23.jpg)
Framework? Think again!
Raw performance matters Support for master-slave replication Multiple layers of cache Working with accelerators (HipHop!) Beware of bottlenecks
i.e. core part of framework is slow
Designed to scale
![Page 24: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/24.jpg)
Topics
The Challenge Infrastructure Code Cache CDN
![Page 25: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/25.jpg)
Cache is everywhere
CPU: L1, L2 Disk buffer Linux filesystem MySQL PHP (APC) Smarty HTTP Proxy Browser cache
![Page 26: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/26.jpg)
Where to cache?
![Page 27: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/27.jpg)
Memory is cheap
Pre-generate stuff Store results in memory
APC, memcached
App config in memory APC with stat=off
Increase RAM for MySQL Disk is the new tape
![Page 28: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/28.jpg)
Memcached for the rescue!
Dead simple Key-value Distributed storage pool Automatic invalidation after X sec
No garbage collecting invoked
Store arrays, objects, simple values Easy integration
![Page 29: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/29.jpg)
Topics
The Challenge Infrastructure Code Cache CDN
![Page 30: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/30.jpg)
Reverse-proxy
First line of cache Returns content if resource is up-to-date Works on HTTP level
Can be integrated into existing infrastructure
Can do load balancing In-memory cache storage Squid, Nginx, Varnish
![Page 31: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/31.jpg)
Content Delivery Network
Network of servers Worldwide Automatic loadbalancing Fast access (low ping time) Data redundancy gratis Ideal for static resources
But not only
Must-have for worldwide websites
![Page 32: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/32.jpg)
CDN as reverse-proxy
HTTP request / response chain Embraces REST architecture Requests are distributed Reduces latency Lowers traffic volume Increases availability i.e. Akamai Edge Suite
![Page 33: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/33.jpg)
CDN at soccerway.com
All of the content is served via CDN Images, CSS, JS Generated HTML JSON for Ajax
90% of traffic via CDN Origin requests only from Europe Site online even if servers are down Can't live without ;)
![Page 34: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/34.jpg)
Thank you for listening
Questions?
![Page 35: Skalowalna architektura na przykładzie soccerway.com](https://reader034.vdocument.in/reader034/viewer/2022042714/54b7b7594a7959bf688b473f/html5/thumbnails/35.jpg)
Interested?
Contact me: [email protected] www.goldenline.pl/adam-brodziak www.linkedin.com/in/adambrodziak
We're hiring! Web developers Football / sport fans