scaling web application in the cloud

SCALING WEB APPLICATIONS ON THE CLOUD

Federico Feroldi - federico@coderloop.com

0M PVs

125M PVs

250M PVs

375M PVs

500M PVs

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

user demand servers capacity

1/3 of your infrastructure

costis USELESS

TOPICS FOR TODAY‣ What is Scalability?‣ Amazon Web Services‣ Building a web application on AWS‣ Tools and Best Practices

WHO’S TALKING?Geek by nature, working on the web since 1996

PROFESSIONAL ENTREPRENEUR

SCALABILITY

WHAT IS SCALABILITY?A desirable property of a system, which indicates its ability to...Gracefully handle increasing demand.Increase its capacity (transactions, processing, storage, throughput) proportionally with the addition of more resources (nodes, CPU, storage, bandwidth).

SCALING UPAdd resources to a single node in the system.Pros: transparent to the software, almost “linear” scale, quickly fix performance issuesCons: risky, may impact on service, can be very expensive, cannot scale “infinitely”

“It's  the  Stay  Puft  Marshmallow  Man!”

SCALING OUTAdd more nodes to the system.Pros: cheap, almost no impact on service, can scale “infinitely”Cons: the system must be properly designed, could be complex to operate

“The  best  thing  about  being  me  -­‐  there's  so  

many  of  me.”

SCALE FAST OR DIE!If your application is accessible from the Internet you can't decide how many people are going to use it.More time it takes for your systems to adapt to the user demand, more time in advance you have to provision your system to be able to handle the future usage (and most of the time your forecast will be wrong).

ELASTICITYThe ability to quickly and gracefully increase capacity by adding more resources and also to quickly and gracefully release resources when the required capacity decreases.In a few words: the ability to quickly scale a system capacity up and down based on the demand in an almost real-time fashion.

INTRODUCING AWS

INTRODUCINGAmazon launched Amazon Web Services in 2001 Currently the most mature, feature rich and flexible IaaS platformIn a few words: AWS is the Data Center in the Cloud (and much more)

5 REASONS TO USE AWS1. It’s cheap: pay per use, simple cost model.2. It’s efficient: VMs, DBs, NAS, storage, messaging,

CDNs, LBs, etc...3. It’s safe: proven infrastructure, Amazon itself builds it's

services on the same technologies.4. It’s flexible: everything can be managed with an API

call, you can build your own tools or use one of the many available.

5. It’s unlimited: virtually no limit on VM instances or storage you can use.

WEB APPS COMPONENTSWeb/Application servers: to serve dynamic pagesDatabases: to store user dataFile Storage: to store images, videos, documentsComputing nodes: to execute background tasks like image conversion, video transcoding, email deliveryMessaging infrastructure: asynchronous and reliable communication between nodesLoad balancers: distribute requestsMonitoring infrastructure: to check that everything is working well and track the system usage

BUILDING ON AWS

EXAMPLE: CODERLOOPCoderloop is a community where programmers can practice their skills and Load balancers: distribute requests against each other to solve complex programming problems.Users send computer programs to solve certain puzzles, Coderloop execute the programs, verifies the correctness and gives a rating.

Serve Web Application

Store User Data

Process Submissions

FIRST A WEB SERVEREC2 is the Elastic Compute Cloud.Users can launch virtual machines (called instances) with a CLI tool or an API.They have dynamic IPs but you can reserve fixed IPs and associate them to an instance.Instances are created from customizable “images” and they come in many sizes (memory and CPU).You pay for the CPU time and the bandwidth.When an instance is turned off, you lose the data in it (volatile storage).

PERSISTENT USERS’ DATARDS is the Relational Database Service.It’s a fully managed MySQL database.You can scale it as needed (CPU and available storage).It’s automatically patched when needed.Supports multi-master and replication.You can create backups periodically or with a single API call.You pay for CPU usage, dedicated primary storage, backup storage (optional) and bandwidth used.

THE NOSQL ALTERNATIVESimpleDB is the non relational data store.It’s a fully managed, scalable “data store”.Simple web service API no SQLNo need to define schema in advance.Scales automatically, you just keep adding data.You pay for CPU time, used storage and bandwidth.25 hours of CPU and 1GB of storage for free each month.

STORING FILES AND IMAGESS3 is the Simple Storage Service.Simple API (REST and SOAP) to write objects (files) in buckets (similar to directories).Objects can be from 1 byte to 5GB in sizeStandard (99.999999999%) or reduced (99.99%) availability options.You pay for the storage used and the bandwidth.

PROCESSING SUBMISSIONSWe use EC2 with SQS (Simple Queue Service)Reliable, highly scalable message queue.Web servers queue “job requests” that are picked by EC2 instances and processed.EC2 instances can be added or removed based on the amount of requests to be processed.You pay for the amount of requests made to the service and the amount of data transferred.The first 100K requests are free each month.

SCALING OUT THE APPWe can distribute the requests with the Elastic Load Balancing serviceMonitors the available instances and routes incoming requests to “healthy” instancesSupports sticky sessions and SSL termination.You pay for the time and the bandwidth transferred.With RDS we can easily setup read replicas for our database to scale the read capacity.

ENABLING ELASTICITYWith Cloud Watch we can monitor our servers’ usage in real time: CPU, Disk and Network for EC2 instances and databases. This information can be used to enable Auto Scaling: automatically scale your Amazon EC2 capacity up or down according to conditions you define (based on average CPU utilization, network activity or disk utilization).New EC2 instances get automatically added to or removed from the Elastic Load Balancer

INCREASING AVAILABILITYAWS is deployed on multiple Regions and Availability Zones.AZs are distinct locations, insulated from failures in other AZs and provide inexpensive, low latency network connectivity to other AZ in the same Region.Regions consist of one or more AZs, are geographically dispersed, and will be in separate geographic areas or countries: currently Northern Virginia, Northern California, Ireland, and Singapore.

Simple Q

ueue Services

CloudWatch

ElasticLoad Balancer

AutoScaling

S3RDS

EC2

SQSEC2

AWS ARCHITECTURE

TOOLS YOU NEED

PUPPETCentralized configuration managementRapid creation of an instance of a pre-defined type: web server, email server, etc...Ensuring uniform configuration of the entire set of instances of the same type at all times.A Puppet Master, guardian of configurations.Multiple Puppet clients installed on EC2 instances.

CAPISTRANOApplication deployment and parallel execution of automated tasksScripting a certain number of tasks, whether complex or not (deliveries, backups, site publication/maintenance, etc.) executing them rapidly in parallel on X instances with a single command.Webistrano (web interface) for 1-click deployments

MONITORING AND LOGSSupervision verifies the state of a host or a service and sends out an alarm upon detecting any abnormal behavior: NagiosMetrology enables instrumentation data to be archived, and if necessary, processed or filtered, before it is presented in the form of graphs or reports: CactiThe more instances you have, the more scattered logs there’ll be on the various instances, implement centralized log collection: Syslog-NG

TIPS & BEST PRACTICES

6 TIPS TO REDUCE COSTSKeep machines in the same availability zoneUse spot instances (can cost up to 3 time less)Choose your instance types wisely (c1.medium cost 2x m1.small by offers 5x computing power)Choose the smallest possible storage (it’s very easy to expand the capacity of an RDS instance)Use Autoscaling (have always the least needed amount of instances to handle the traffic)Reserve your instances (you can reduce your costs significantly by reserving number of instances for a year or for three years).

CONCLUSIONS

Amazon Web Services is an excellent platform to build your first prototype or even run your production service.It has a very flexible and proven API that let you manage your infrastructure in ways you never though were possible before.It provides you with a lot of building blocks that scale well and that you can trust, so you don’t have to waste time building your infrastructure and you can focus on making your service the best in the world.And when the day comes that you hit the Techcrunch homepage, you infrastructure is ready to scale in minutes.

THANK YOUfederico@cloudify.me

twitter.com/cloudify

CODERLOOP IS HIRING!Check out http://www.coderloop.com/jobs