habits of highly scalable web applications
DESCRIPTION
Eli White's "Habits of Highly Scalable Web Applications" slides from Dutch PHP Conference 2009TRANSCRIPT
![Page 1: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/1.jpg)
Habits of Highly Scalable Web Applications
Eli WhiteZend
http://eliw.com/
![Page 2: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/2.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Scaling?
Enabling your application to grow as traffic grows
Only do what you need but …Don't code yourself into a corner
![Page 3: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/3.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Various aspects
Web ServerDatabaseCaching
![Page 4: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/4.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
In the Beginning …
One Web Server:
![Page 5: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/5.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Load Balanced
![Page 6: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/6.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Numerous Options
DNS RotationApache ProxyCloud Services
NetScaler
BIG-IP
![Page 7: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/7.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Preparation
Ensure you don't preclude this, for example:If using local caching (APC / Zend Server)
Avoid assuming exclusive/single cacheDon't rely on the filesystem
Temporary files, Sessions, etc
If you do have code assuming one server: Encapsulate
![Page 8: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/8.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Database Scaling
Everyone starts with just one server:
Multiple steps to take as you move forward
![Page 9: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/9.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Step One: Master/Slave
![Page 10: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/10.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Master/Slave Preparation
Even with one server:Make code write to master and read from slave
![Page 11: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/11.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Avoid Slave Lag
Don't write code that would fail with slave lag:
$master->query('update users set comments += 1'); $slave->query('select comments from users');
![Page 12: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/12.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Step Two: Multiple Slaves
![Page 13: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/13.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
One Slave per Web Server?
Not as flexible
![Page 14: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/14.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Better Solution: Random
![Page 15: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/15.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Code to Select Random Slaveclass DB { private static $cfg = array( 'write' => array('mysql:dbname=MyDB;host=10.1.2.3'), 'read' => array('mysql:dbname=MyDB;host=10.1.2.7', 'mysql:dbname=MyDB;host=10.1.2.8', 'mysql:dbname=MyDB;host=10.1.2.9'); );
public static function getConnection($pool) { $max = count(self::$cfg[$pool]) - 1; $dsn = self::$cfg[$pool][mt_rand(0, $max)]; return new PDO($dsn, USER, PASS); }}
$db = DB::getConnection('read');
![Page 16: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/16.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Step Three: Slave Pools
Virtually divide your slaves into pools
Use this to isolation high database load
Potentially enhance query caches
![Page 17: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/17.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Possible Pool Layout
![Page 18: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/18.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Step Four: Partitioning
Simplest Definition:Break your tables or databases into smaller ones
![Page 19: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/19.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Multiple Masters
![Page 20: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/20.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Cons of Partitioning
Loss of direct SQL supportIncreased Web Server / PHP loadMore complicated programming
![Page 21: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/21.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Main Types of Partitioning
HorizontalVertical
Application Level
![Page 22: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/22.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Vertical Partitioning
“Moving various columns of your table into different tables”
Various methodologies:● Move rarely used columns into auxiliary table● Move often empty columns into auxiliary table● Move columns that are not used in where clauses
Usually done within the same database/server
![Page 23: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/23.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Vertical Partitioning
![Page 24: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/24.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Horizontal Partitioning
“Moving various rows of your table into different tables”
Various methodologies:● Range Based● Date Based● Interlaced● User Based
Can be done on one server, or break into multiple masters
![Page 25: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/25.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Horizontal Partitioning
![Page 26: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/26.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Application Level Partitioning
“Moving various tables of your DB onto different servers”
Various methodologies:● Move single tables to specific servers● Move groups of related tables together to allow joining
![Page 27: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/27.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Application Level Partitioning
![Page 28: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/28.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Multiple Datacenters
You thought worrying about slave lag was bad
Data from multiple sources all needs integrated
Good luck!
![Page 29: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/29.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Brief Touch on Caching
Often considered performance
Can absolutely be a scalability factor, especially when combined with smaller discrete DB queries
Allows you to get around DB scalability by ignoring the DB
![Page 30: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/30.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Type of Caching
Single server memory caches
APC or Zend Server Data CacheLimited due to lack of sync'd cache
Distributed
Memcached or Zend PlatformRequired for true scalability enhancement
![Page 31: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/31.jpg)
Eli White — Habits of Highly Scalable Web Applications — Dutch PHP Conference 2009 — 6/13/2009
Caching Best Practices
Write through cache
Choose small, discrete, reusable data units
Don't store data you can't recreate
Store data in as close to final processed form
![Page 32: Habits of Highly Scalable Web Applications](https://reader033.vdocument.in/reader033/viewer/2022061118/5468bda8af79593b558b47d2/html5/thumbnails/32.jpg)
Questions?
For this presentation & more:
http://eliw.com/
Zend's DevZone:
http://dz.zend.com/
Twitter: @eliw
Rate Me: http://joind.in/591