scalable architecture 101
TRANSCRIPT
- 1. Mike Willbanks Blog:http://blog.digitalstruct.com Twitter : mwillbanks IRC : lubs on freenode Scalable Architectures 101 MNPHP
2. Scalability?
- Your application is growing, your systems are slowing and growth is inevitable...
- Where do we go from here?
- Web Servers
3. Database Servers 4. Cache Servers 5. Job Servers 6. .... 7. The Beginning...
- Single Server Syndrome
- One Server Many Functions
- Web Server, Database Server, DNS Server, File Server
8. The Next Step...
- Single Separation Syndrome
- Separation of Web and Database
- Fix the main disk I/O bottleneck.
However, we can't handle our current I/O on our web server.Let's start there... 9. Web Servers 10. Load Balancing Our Environment 11. Several Options
- DNS Rotation
- Not very reliable, but works on a small scale.
Software Based
- Squid, Wackamole, HAProxy, Apache Proxy, Perlbal...
Hardware Based
- Several vendors ranging based on need.
12. What We Need to Remember
- Files
- All web servers need our files.
13. Static content could be tagged in version control. 14. Static content may need a file server / CDN / etc. 15. User Generated content on NFS mount or served from the cloud or a CDN. Sessions
- All web servers need access to our sessions.
16. Remember disk is slow and the database will be a bottleneck.How about distributed caching? 17. Other Thoughts
- Running PHP on your web server may be a resource hog, you may want to offload static content requests to nginx, lighttpd or some other lightweight web server.
- Running a proxy to your main web servers works great for hardworking processes.While serving static content from the lightweight server.
18. Database Servers 19. Where We All Start
- Single Database Server
- Lots of options and steps as we move forward.
20. Replication
- Single Master, Single Slave
- Write code that can write to the master and read from the slave.
- Exception: Be smart, don't write to the master and read from the slave on the table you just wrote to.
21. Multiple Slaves
- Single Master, Multiple Slaves
- It is a great time to start to implement connection pooling.
22. Multiple Masters
- Multiple Master, Multiple Slaves
- Now we can pool on our masters as well.
23. Be warned, auto-incrementing now should change so you do not conflict. 24. Partitioning
- Segmenting your Data
- Vertical Partitioning
- Move less accessed columns, large data columns and columns not likely in the where to other tables.
Horizontal Partitioning
- Done by moving rows into different tables.
- Based on Range, Date, User or Interlaced
25. Vertical Partitioning id uri name content 1 / homepage TEXT 2 /contact contact TEXT id uri 1 / 2 /contact id name content 1 homepage TEXT 2 contact TEXT 26. Horizontal Partitioning id uri name content 1 / homepage TEXT 2 /contact contact TEXT 3 /about about TEXT 4 /services services TEXT id uri name content 1 / homepage TEXT 3 /about about TEXT id uri name content 2 /contact contact TEXT 4 /services services TEXT 27. Cache Servers 28. Caching
- Speed Up Access
- Caching is imperative in scaling and performance both.
- Single Server
- APC / Xcache / etc
29. Not highly scalable, great for configuration files. Distributed
- Redvis, Memcached, etc.
30. Setup consistent hashing. Do not cache what cannot be re-created. 31. Caching
- In The Beginning
- Single Caching Server
32. Start to cache fetches, invalidate cache on write and write new cache, always reading from the cache. 33. Distributed Caching
- Distributed Mania
- Write based on consistent hashing (hash of a key that you are writing)
34. Server depends on the hash. 35. Hint use the memcached pecl extension. 36. The Read / Write Process
- In the most simple form...
37. Job Servers 38. Job Servers (Message Queues)
- Use job servers for asynchronousand synchronous jobs.
39. Do nothing in real-time if you do not have to.
- Email, Image Resizing, Video Processing, etc.
Hint gearman rocks and there is a pecl extension. 40. Mike Willbanks Blog:http://blog.digitalstruct.com Twitter : mwillbanks IRC : lubs on freenode Talk:http://joind.in/1375 Questions?