aws summit 2011: customer presentation - nytimes
DESCRIPTION
TRANSCRIPT
NYTimes.com
Vadim Jelezniakov
AWS Summit 2011
Next 10 min
The Company
Why Cloud
The App Examples
Progress So Far
What’s Next
The New York TimesThe International Herald Tribune
The Boston Globe15 daily newspapers
50 Web sitesNYTimes.comBoston.comAbout.com
The New York Times
Company
NYTimes.com’s
Stats
In the US:38.1 mil unique users /
month810 mil page views / month
Worldwide:62 mil unique users /
month984 mil page views / month#1
individual newspaper site#2 media
site bloggers link to the most*#4 news
and information Web site
comScore Report, April 2011
*Technorati's index
Why Cloud?
On Demand Availability
Scalability
Rapid Deployment
Why AWS?
Cost-Effective
IaaS = Flexible
Comprehensive
Why Cloud?
On Demand Availability
Scalability
Rapid Deployment
Why AWS?
Cost-Effective
IaaS = Flexible
Comprehensive
Why Cloud?
On Demand Availability
Scalability
Rapid Deployment
Why AWS?
Cost-Effective
IaaS = Flexible
Comprehensive
On Demand Capacity -
TimesMachine
Flexibility - Times Skimmer
Scalability - User Generated
Content (Comments)
Examples
TimesMachine (2007, 2008)
130 years of archives in 36 hours
400K objects, 4T of data
S3, EC2, Hadoop
http://timesmachine.nytimes.com
Skimmer Private Beta (Feb 14th, 2009)
http://www.nytimes.com/skimmer/
11:00 News leaks to TechCrunch, Google Finance11:10 Average load on the instance is 4011:15 Snapshotting the instance, configuring the LB, testing12:00 Four Skimmer instances load balanced, quality is restored1 hour - from prototype to production
“SkimmerGate”
Comments on Articles and Blogs
Rate and Review for Movies,
Theater, Dining and Travel
We like
To be able to scalequickly on demand
We don’t like
calls from Systems teamat 6PM on Fridays
Traffic Spikes - Add Capacity
Load Balancing / Security View
Everyone’s Favorites:
Development Instances!
Gap Analysis
• Load Balancing
• Communication between instances
• Talking back to existing Datacenter
• Scaling up and back
• Securing the instances
Nimbul
Light Cloud Manager
http://github.com/nimbul/nimbul
CloudSource
Simple SVN Deployment
http://github.com/nimbul/cloudsourc
ebased on ServerMattic developed by WordPress
Emissary
Fast AMQP Messaging
http://github.com/nimbul/emissar
y
Tools Developed for the
Cloud
Progress So Far
NYTimes.com EC2
Expansion
Clusters: from 2 to 100. Instances: from 30 to 400.
NYTimes.com EC2
Expansion
EC2 DC in comparison with other DCs
from 5% to 40% (VMs/instances count)
2009 2011
NYTimes.com EC2
Expansion
EC2 contributes ~26% of total operational cost across DCs
(space, power, bandwidth)
What’s Next
Multi-AZ and Multi-Region
SNS, SNS2SQS - in Dev
Multi-AZ MySQL RDS
Need 'smarter' ELBNeed
multi-AZ support in VPC
email: [email protected]: @vadimj
http://www.nytimes.com
http://www.nytimes.com/skimmer/http://timesmachine.nytimes.com
http://open.nytimes.com
http://github.com/nimbul/
images licensed fromhttp://www.dreamstime.com/