jazoon'13 - abdelmonaim remani - the economies of scaling software
DESCRIPTION
TRANSCRIPT
![Page 1: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/1.jpg)
The Economies of Scaling SoftwareAbdelmonaim Remani@PolymathicCoder
![Page 2: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/2.jpg)
Creative Commons Attribution Non-Commercial License 3.0 Unported
The graphics and logos in this presentation belong to their rightful owner
![Page 3: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/3.jpg)
• Platform Architect at just.me Inc.
• JavaOne RockStar and frequent speaker at many developer events and
conferences including JavaOne, JAX, OSCON, OREDEV, 33rd Degree, etc...
• Open-source advocate and contributor
• Active Community member• The NorCal Java User Group• The Silicon Valley Dart Meetup
Bio:
http://about.me/PolymathicCoder
Twitter: @PolymathicCoder
Email: abdelmonaim.remani
@gmail.com
SlideShare:
http://www.slideshare.net/PolymathicCoder/
| @PolymathicCoder
About Me
![Page 4: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/4.jpg)
Follow @PolymathicCoder
http://speakerscore.com/jazoon-scalability
![Page 5: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/5.jpg)
• The Economies of Scale
• “In microeconomics, economies of scale are the cost
advantages that enterprises obtain due to size [...] often
operational efficiency is [...] greater with increasing
scale [...]” -Wikipedia
| @PolymathicCoder
The Title of the Talk
![Page 6: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/6.jpg)
Let’s Go!
![Page 7: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/7.jpg)
• Only the enterprise worried about scalability• The rise of social and the abundance of mobile
• An exponential growth of internet traffic• The creation of a spoiled user-base
• I want to see the closest Moroccan restaurants to my current location on a map along with consumer ratings and whether any of my friends has recently checked-in in the last 30 days
• The lines are blurred between consumer applications and the enterprise applications
| @PolymathicCoder
Blurred Lines…
![Page 8: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/8.jpg)
Scalability is everyone’s problem…
| @PolymathicCoder
The Bar Is Higher!
![Page 9: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/9.jpg)
What is
Scalability?
![Page 10: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/10.jpg)
• The ability of an application to handle an increasing amount of work without performance degradation
• Not a good definition! It implies:• You’ll need to scale forever
• Scalability is relative; It is bound by one’s specific needs
• You’ll need to be fully scalable from day one• Scalability is evolutionary; It is a gradual process
• There are no external constraints• Unrealistic
| @PolymathicCoder
The Common Definition
![Page 11: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/11.jpg)
• The ability of an application to gracefully evolve within the constraints of its ecosystem in order to handle the maximum potential amount of work without performance degradation
• Work?• Simultaneous requests
• Performance degradation?• Increased latency or decreased throughput
| @PolymathicCoder
A Better Definition
![Page 12: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/12.jpg)
• Don’t be surprised if• Your application supports one
million users• You add one more feature• 500,000 user load crashes your
system or renders it unusable
| @PolymathicCoder
A Black Art!
![Page 13: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/13.jpg)
Latency Is Your Enemy
![Page 14: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/14.jpg)
• To scale is to reduce latency• To reduce latency is to address bottlenecks• To scale is to address bottlenecks
• The usual suspects• The CPU• The Storage I/O• The Network I/O
• Inter-related
| @PolymathicCoder
Syllogismo
![Page 15: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/15.jpg)
OvercomingThe CPU
Bottleneck
![Page 16: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/16.jpg)
• Nothing affects the CPU more than the instructions it is summoned to execute
• This is about your application• How it is written (Architecture, code base,
etc..)• How it is deployed
| @PolymathicCoder
Overcoming the CPU Bottleneck
![Page 17: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/17.jpg)
A Scalable
Architecture
![Page 18: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/18.jpg)
• “Things that people perceive as hard-to-change” -Martin Flower• http://martinfowler.com/ieeeSoftware/
whoNeedsArchitect.pdf• Decision you commit to; the ones that will be
stuck with you forever
| @PolymathicCoder
Architecture?
![Page 19: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/19.jpg)
• Choose the right technologies• Platform• Languages
• Frameworks• Libraries
• Make the right abstractions• Loosely-coupled components
• Functional abstractions• Technical abstractions• Make sure that the latter is subordinate to the former and not the
other way around
| @PolymathicCoder
Be Wise… Think Twice…
![Page 20: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/20.jpg)
Write Good
Code
![Page 21: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/21.jpg)
• Think your algorithms through and mind their complexity (Asymptotic Complexity, Cyclomatic Complexity, etc…)
• SOLIDify your design• Single Responsibility, Open-Closed, Liskov Substitution,
Interface Segregation, and Dependency Inversion• Understand the limitation of your technology and
leverage its strengths
| @PolymathicCoder
Write Good Code
![Page 22: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/22.jpg)
• Obsess with testing• TDD/BDD
• Tools• Static code analyzers (PMD, FindBugs, etc…)• Profilers (Detect memory leaks, bottlenecks, etc…)
• Etc…
| @PolymathicCoder
Quality… Quality…
Quality!
![Page 23: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/23.jpg)
• Read• The Classics (The Mythical Man-Mouth, etc…)• GoF’s “Design Patterns”• Eric Evans’ “Domain-Driven Design”• Every book by Martin Fowler• Uncle Bob’s “Clean Code”• Josh Bloch’s “Effective Java”• Brian Goetz’s “Java Concurrency in Practice”• Tech Papers/Blogs• Etc...
| @PolymathicCoder
Know Thy S#!t
![Page 24: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/24.jpg)
The
Inevitable
![Page 25: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/25.jpg)
You’ll end up with…
At best…The fading tradition of making cow dung piles
http://news.ukpha.org/2011/01/the-fading-tradition-of-making-cow-dung-piles/
| @PolymathicCoder
You do all that…
![Page 26: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/26.jpg)
| @PolymathicCoder
Still better than…
![Page 27: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/27.jpg)
• What is it?• The quick-and-dirty you are not proud of• What you would have done differently haven't you had
time• It’s a matter of time before it starts to smell really
bad• What to do?
• The fact you recognize it as debt is good thing in itself• Keep tabs and refactor often• Cut the right corners
• Don’t mortgage architecture (Don’t lock yourself out) | @PolymathicCoder
Technical Debt
![Page 28: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/28.jpg)
Write Code That Scales
Up
![Page 29: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/29.jpg)
• Vertical Scaling (Scaling Up)• On a single-node system• Adding more computing resources to the node (Getting
a beefier machine)• Writing code to harness the full power of the one
node
| @PolymathicCoder
Vertical Scaling
![Page 30: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/30.jpg)
• Writing concurrent code of simultaneously executing code
• Simple business logic within containers is already multi-threaded
• Executing complex business logic within a reasonable time
• Break it into smaller steps• Execute them in parallel• Aggregate data back
| @PolymathicCoder
Parallelism At The Node Level
![Page 31: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/31.jpg)
• Moore’s Law• Performance gain is automatically realized by software
(Code is faster on faster hardware)• Nothing is forever…
• The era of the multi-core chip• We need to write code to take advantage of all
cores
| @PolymathicCoder
Easier Said Than Done…
![Page 32: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/32.jpg)
• Synchronize state across threads across multiple cores• Good luck!
• Relay on frameworks and libraries (Fork/Join, Akka, etc…)
• Go immutable• Not always straightforward or possible
• Go functional (Scala, Clojure, etc…)
| @PolymathicCoder
Easier Said Than Done…
![Page 33: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/33.jpg)
• Amdahl’s Law• Throwing more cores does not necessarily result in
performance gain• Diminishing return at some point no matter how many
cores you throw in
| @PolymathicCoder
It Gets More Interesting…
![Page 34: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/34.jpg)
• Leverage Probabilistic data structures and algorithms• Bloom Filters, Quotient filters, etc…
• Go Reactive• http://www.reactivemanifesto.org/• RxJava, Spring Reactor, etc…
| @PolymathicCoder
Miscellaneous
![Page 35: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/35.jpg)
Write Code That Scales
Out
![Page 36: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/36.jpg)
• Horizontal Scaling• On a distributed system (A cluster)• Adding more nodes
• Writing code to harness the full power of the cluster
| @PolymathicCoder
Horizontal Scaling
![Page 37: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/37.jpg)
• A typical cluster consists of• A number of identical application server nodes behind
a load balancer
| @PolymathicCoder
Topology
![Page 38: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/38.jpg)
• A typical cluster consists of• A number of identical application server nodes behind
a load balancerA number?
• It depends on how many you actually need and can afford
• Elastic Scaling / Auto-Scaling• The number of live nodes within the cluster shrinks and
grows depending on the load• New ones are provisioned or terminated as needed
| @PolymathicCoder
Topology
![Page 39: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/39.jpg)
• A typical cluster consists of• A number of identical application server nodes behind
a load balancerIdentical?
• Application nodes are cloned off of image files (Ex. AWS Ec2 AMIs, etc...)
• Configuration Management tool (Chef, Puppet, Salt, etc...)
| @PolymathicCoder
Topology
![Page 40: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/40.jpg)
• A typical cluster consists of• A number of identical application server nodes behind
a load balancer
Load balancer?• Load is evenly distributed across live nodes
according to some algorithm (Round-Robin typically)
| @PolymathicCoder
Topology
![Page 41: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/41.jpg)
• Session data• Session Replication• Session Affinity / Sticky Session
• Requests from the same client are routed to the same node
• When the node dies, the session data dies with it• Shared Session / Distributed Session
• Session data is in a “centralized” location• Go Stateless
• No session data (Any node would do)
| @PolymathicCoder
Managing State
![Page 42: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/42.jpg)
• Leverage Map/Reduce• “A programming model for processing large
data sets with a parallel, distributed algorithm on a cluster”
• Apache Hadoop
| @PolymathicCoder
Parallelism At The Cluster Level
![Page 43: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/43.jpg)
• How to HTTPS?• End at load balancer• Wildcard SSL
• Distributed Lock Manager (DLM)• Synchronize access to shared resources
• (Google Chubby, Apache Zookeeper, etc…)• Distributed Transactions
• X/Open XA
| @PolymathicCoder
Miscellaneous
![Page 44: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/44.jpg)
Deployment
![Page 45: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/45.jpg)
• Multiple Environments• Development, Test, Stage, and Production• Automatic Configuration Management
• Practice Continuous Delivery• Leverage The Cloud
• IaaS, PaaS, SaaS, and NaaS
| @PolymathicCoder
Deployment
![Page 46: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/46.jpg)
OvercomingThe Storage
I/OBottleneck
![Page 47: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/47.jpg)
• The storage I/O is usually the most significant
| @PolymathicCoder
The Storage I/O Bottleneck
![Page 48: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/48.jpg)
The Persistent
Datastore
![Page 49: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/49.jpg)
• Relational of course!• Normalized schema guaranteeing data integrity• ACID Transactions• No biased towards specific access patterns• Flexible query language
• As datasets grow• Scale up (Buy beefier machines)• Database tuning / query optimization• Create materialized views• De-normalize• Etc…
| @PolymathicCoder
What Datastore to Use?
![Page 50: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/50.jpg)
• No other choice but scaling out RDBMS• Master/Slave clusters• Sharding
• Failed big time!• RDBMS is designed to run on one machine• Eric Brewer’s CAP Theorem of distributed systems
• Pick 2 out of 3: Consistency, Availability, and Partition Tolerance
• The relational model is designed to favor CA, hence can never support P
| @PolymathicCoder
Mucho Data!
![Page 51: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/51.jpg)
• A wide range of specialized datastores with the goal of addressing the challenges of the relational model
• “The whole point of seeking alternatives is that you need to solve a problem that relational databases are a bad fit for” –Eric Evans
• A wide variety• Key-Value Datastores• Columnar Datastores• Document Datastores• Graph Datastores
| @PolymathicCoder
NoSQL
![Page 52: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/52.jpg)
• Within the application• Data is complex and accessed in many different ways• Why should we fit it into one storage model?
• Polyglot Persistence is about• Leveraging multiple data stores based on the specific
way the data is stored and accessed• For more info:
• Checkout my talk on YouTube from JAX Conf 2012• “The Rise of NoSQL and Polyglot Persistence”
• http://bit.ly/PCWtWi
| @PolymathicCoder
Polyglot Persistence
![Page 53: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/53.jpg)
Caching
![Page 54: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/54.jpg)
• A cache is typically a simple key-value data structure• Instead of incurring the overhead of data retrieval or
computation every time, you check the cache first• You can’t cache everything, caches can be configured to
use multiple algorithms depending on the use case (LRU, LFU, Bélády's Algorithm, etc...)
• Use aggressively!• What to cache?
• Frequently accessed data (Session data, feeds, etc…)• Results of intensive computations
| @PolymathicCoder
Caching
![Page 55: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/55.jpg)
• Where to cache?• On disk
• File System: Slow and sequential access• DB: A bit better (Data is arranged in structures
designed for efficiant access, indexes, etc…)• Generally a terrible idea (SSDs make things a bit
better)• In-Memory: Fast and random access, but volatile• Something in between: Persistence caches (Redis,
etc…)• What type of cache?
• Local, Replicated, Distributed, and Clustered| @PolymathicCoder
Caching
![Page 56: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/56.jpg)
• How to cache?• Most caches implement a very simple interface• Always attempt to get from cache first using a key
• If it is a hit, you saved yourself the overhead• If it is a miss, compute or read from the data store then
put in cache for subsequent gets• When you update you can evict stale data• You can set a TTL when you put
• Many other common operations...
| @PolymathicCoder
Caching
![Page 57: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/57.jpg)
• Caching Query Results• Key: Hash of the query itself• How about parameterized queries?
• Key: Hash of the query itself + Hash of parameter values
• Method/Function Memoization• Key: Method name• How methods with parameters?
• Key: Hash of the method name + Hash of parameter values
• Caching Objects• Key: Identity of the object| @PolymathicCoder
Caching Patterns
![Page 58: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/58.jpg)
• Time-series datasets (Ex. Real-time feed)• Most of the time pseudo/near real-time is enough• Use caching to throttle access to resources
• Cache query result with a t expiry• Fresh data is only read every t
| @PolymathicCoder
Caching Patterns
![Page 59: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/59.jpg)
• Profile your code to assess what to cache, and whether you need to to begin with
• Stale state might bite you hard• Incoherence: Inconsistent copies of objects cached with
multiple keys• Stale nested aggregates
• Network overhead of misses might outweighs the performance gain of hits
• Consider writing/updating cache when writing/updating the persistence store
| @PolymathicCoder
Caching Gotchas
![Page 60: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/60.jpg)
• EhCache• Memcahed• Oracle Coherence• Redis
• A persistence NoSQL datastore• Built-in data structures like sets and lists• Supports intelligent keys and namespaces
| @PolymathicCoder
Featured Solutions
![Page 61: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/61.jpg)
OvercomingThe Network
I/OBottleneck
![Page 62: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/62.jpg)
• The Network I/O is can bring you down as much
| @PolymathicCoder
The Network I/O
Bottleneck
![Page 63: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/63.jpg)
Asynchronous
Processing
![Page 64: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/64.jpg)
• Resource-intensive tasks cannot be handled practically during an HTTP session
• Synchronous processing is overused and not necessary most of the time
| @PolymathicCoder
Asynchronous Processing
![Page 65: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/65.jpg)
• Pseudo-Asynchronous Processing• Flow
• Process data / operations in advance• User requests data or operation• Respond synchronously with pre-processed result
• Sometimes not possible (Dynamic content, etc...)
| @PolymathicCoder
Asynchronous Processing Patterns
![Page 66: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/66.jpg)
• True Asynchronous Processing• Flow
• User request data or operation• Acknowledge
• Ex. A REST that return an “202 Accepted” HTTP status code
• Do Processing at your own convenience• Allow the user to check progress
• Optionally notify when processing is completed
| @PolymathicCoder
Asynchronous Processing Patterns
![Page 67: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/67.jpg)
• Leverage Job/Work/Task Queues• JMS (Java Messaging Service) – JSR 914• AMQP (Advanced Message Queuing Protocol): RabbitMQ, ActiveMQ,
etc…• AWS SQS• Redis Lists• Etc…
• Task Scheduling• Jobs triggered periodically (Cron, Quartz, etc…)
• Batch Processing
| @PolymathicCoder
Techniques
![Page 68: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/68.jpg)
Content Delivery
Network
![Page 69: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/69.jpg)
• Static content• Binary (Video, Audio, etc…)• Web objects (HTML, JavaScript, CSS, etc…)
• Do NOT serve through your application server• Use a CDN
• “A large distributed system of servers deployed in multiple data centers across the internet”• Akamai• AWS CloudFront
| @PolymathicCoder
Content Delivery Network (CDN)
![Page 70: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/70.jpg)
• Dirty Caches• script.js is a script file deployed on CDN• Multiple copies of script.js will be replicated across all
edge nodes of the CDN• Clients/browsers will their own copies of script.js locally• We update script.js• Since the new and old version have the same URI
• New clients will be served the old version by the CDN
• Old clients will continue to use the old version from their local cache
| @PolymathicCoder
CDN Gotchas
![Page 71: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/71.jpg)
• Dirty Caches• What to do?
• Simply append version number to file names• script-v1.js, script-v2.js, etc…
• Force invalidation of all copies on edge nodes• Set HTTP caching headers properly
| @PolymathicCoder
CDN Gotchas
![Page 72: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/72.jpg)
Domain Name
Service
![Page 73: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/73.jpg)
• Do NOT rely on your free domain name registrar DNS• Use a scalable DNS solution
• AWS Route 53• DynECT• UltraDNS• Etc…
• Domain Sharding• Browsers limit the number of connections per host (Max of 6 usually)
• Creating multiple subdomains (CNAME entries) allow for more resources to be downloaded in parallel
• Watch out for: DNS lookup overhead, HTTPS cost, Browser’s Same-Origin Policy, etc…
| @PolymathicCoder
Domain Name Service (DNS)
![Page 74: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/74.jpg)
Remoting
![Page 75: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/75.jpg)
• In a SOA (Service Oriented Architecture)• RPC calls to multiple services• Data Exchange (Plain vs. Binary)
• SOAP / REST with XML or JSON• Google Protocol Buffers, Apache Thrift, Apache Avro,
etc…• Protocol
• JMS• HTTP• SPDY
| @PolymathicCoder
Remoting
![Page 76: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/76.jpg)
QualifyingScalability
![Page 77: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/77.jpg)
• Instrumentation: Bake it into the code early• Monitoring
• Health (Application / Infrastructure)• Key Performance Indicators (KPIs)
• Number of request handled, throughput, latency, Apdex Index, etc ...
• Logs• Testing
• Load/Stress testing
| @PolymathicCoder
Qualifying Scalability
![Page 78: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/78.jpg)
Disaster Recovery
![Page 79: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/79.jpg)
• Goal• Fault-tolerant system• Restore service and recover data ASAP in case of a
disaster• Be proactive
• Develop a Disaster Recovery Plan (DRP)• Practice and test your DRP by doing failure drills
| @PolymathicCoder
When Disaster Hits…
![Page 80: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/80.jpg)
Scaling Teams
![Page 81: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/81.jpg)
• Hiring• Always hire top talent
• You are as strong as your weakest link• Develop a process to bring people in
• Turnkey Hardware/Software Setup (Vagrant, etc...)• Arrange for proper access/accounts
• Develop a knowledge base (Architecture documentation, FAQs, etc...)
• Development Process• Be Agile• Refine in the spirit of Six Sigma
| @PolymathicCoder
Scaling Teams
![Page 82: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/82.jpg)
• Team Structure• Small is good• Form ad-hoc teams from pools of Agile breeds
• Product Owners• Team Members
• Team Lead (Scrum Master)• Engineers• QAs
• Architecture Owners
• Give them ownership of their DevOps
| @PolymathicCoder
Scaling Teams
![Page 83: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/83.jpg)
The Take-home
![Page 84: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/84.jpg)
• The early-bird gets the worm• Design to scale from day one• Plan for capacity early
• Your needs determine how scalable “your scalable” needs to be• Do not over-engineer
• Do not bite more than you can chew• Building scalable system is process
• Commit to a road map around bottlenecks• Guided by planned business features
• Learn from others’ experiences (Twitter, Netflix, etc...) | @PolymathicCoder
The Take-home Message
![Page 85: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/85.jpg)
Work smarter not harder…
| @PolymathicCoder
Take it slow… You’ll get there…
![Page 86: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/86.jpg)
![Page 87: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/87.jpg)
Questions?
![Page 88: JAZOON'13 - Abdelmonaim Remani - The Economies of Scaling Software](https://reader033.vdocument.in/reader033/viewer/2022051611/54b55ff04a795900388b45c8/html5/thumbnails/88.jpg)
Thanks for the attention!
Follow @PolymathicCoder
[email protected]://blog.polymathiccoder.com
http://speakerscore.com/jazoon-scalability