design patterns for scaling web applications

Design patterns for scaling web applications

SysAdmin Bulgaria meetup #01

Ivan DimitrovManager Technical OperationsXogito Group, Inc

ivan.dimitrov@xogito.com

tl;dr

Web application architecture

Life’s too short for bad software- Lew Cirne

tl;dr

Scalability patterns

The only real problem is scaling.Everything else is a sub-problem.

- O’Dell’s Axiom

Let’s start with hardware

Web application architecture

● Single machine web server● Two-tier web application● Multi-Tier application● Message Bus architecture● Service-Oriented architecture

Single machine web server

Single machine web server

● All in one servero Real IPo Web servero Application servero Databaseo Storageo Compute

● All in one servero Security problemo Shared CPUo Shared memoryo SPOFo Maintenance =

Downtime

Single machine web server

Two-tier web application

Two-tier web application

● Dedicated resources

● Multiple POF● Internal network● Maintenance =

downtime

● Different hardware generations

● Expensive

Multi Tier web application

Multi-tier web application

● Separate responsibility to teams● Most of security at first tier● Independant scaling● Front-end can talk with APIs internaly● Every tier can have it’s own hardware

requirements● Global Load Balancer

Global Load Balancer

Message Bus application

Message Bus application

● Many to Many communication between servers

● Replaces constantly polling the database● Asyncronious, not sorted● Loosly coupled systems● Support different hardware

Service Oriented Architecture

Service Oriented Architecture

● Loosly coupled systems● Service oriented company - small teams● Can provide services to third parties● Can subscribe to services from third parties

Scaling Up

● Bottleneck● Measure Results● Horizontal

duplication● Functional/Service

split● Lookup-oriented

split

● Caching● Data sharding● Threading● Queueing● Content Delivery

Networks

Theory of constraints

● Identify the Bottleneck● Utilize the Bottleneck● Adjust Non-Bottlenecks● Elevate the Bottleneck● Review the Process

Finding the bottleneck

● Accumulation● Throughput● Full Capacity● Wait Times

Measure results

Caching

cache hit ratio = cache lookups / cache hits● Store most recent data● Store most used data● Store only repeatedly used data

Cache invalidation - TTL

Threads and Queues

● Single thread process vs Multi thread process

● Queues FIFO, LIFO

Content delivery network

Additional reading