Download - Building Internet-scale Applications
Building Internet-scale Applications:The Beginning
By @gaveen At #RefreshColombo 22 September 2011
Gaveen Prabhasara
twitter.com/gaveen
SysAdmin
DevOps
Works at Thinkcubegaveen.owain.org
github.com/gaveen
Standing on theShoulders of Giants
If I claim to be a wise man, wellIt surely means that I don't know
“
“
- Kansas Carry On Wayward Son
This is a 25 minute talk
Which sounds like a collection of
Random thoughts about
Building Internet-scale Systems
You probably already know
Internet-scale
What is it?
Applications
Does not always mean WebApps
Internet-scale > Enterprise-scale
Facebook,YouTube, Twitter,Gmail, Flikr
Eg:
Case: Facebook750M+ active users350M+ daily users100B+ daily hits30B+ monthly objects shared2.5M+ websites integrated130TB+ daily logs*Sources: http://goo.gl/zT1bL
http://goo.gl/cdFPI
Case: Akamai95800+ servers1000+ networks70+ countries15-30% WWW traffic*Source: http://goo.gl/yVmG9
Culture
Culture
Culture
How
Know Your Trade
Learn
Acquire domain-specific knowledge
Learn how people build similar services
Learn about Technologies
Stay updated
RTFM / Use Search Engines ;)
Get Right People
Team
Find balance in the team
Know their strengths
Nurture them
Set a proper environment
Don't step on others toes
Go for the Best Solution
Architecture & Design
Have a solid architecture
Find a good engineering design
Don't over-engineer / over-complicate
Avoid architecture degradation
Right Tool for the Right Job
Technology
You favourite tool can't do everything. Face it
'Prejudice against' & 'religious for' isn't good for you
Brand names aren't for worshiping
Cloud isn't a magical silver bullet
This is Spar... the Internet. It's big
Do Things Right
Process
Use best practices
Automate. Automate. Automate.
Plan for scaling from the start
Don't let the process be a burden
Little more detail...
Test All the Time or better,do Continuous Integration
Little more detail...
Testing isn't a QA job
Let Devs take responsibility
XP, TDD, BDD, Cucumber, etc.
Don't push untested code
Works-in-my-machine doesn't count
Process
Fight the Smell, Fight Anti-Patterns
Little more detail...
Re-factoring is good
Code smell isn't something to behold
Don't let the code age on you
Don't let the architecture degrade
Bad code is bad code
Process
Use Version Control
Little more detail...
Use a proper VCS
Git, SVN, Hg, Bzr, etc.
Helps automation
No version control, no deploying
Learn to use a VCS properly
Process
Use Configuration Management
Little more detail...
Infrastructure is Code, therefore Automate
User a Configuration Management System
Puppet, Chef, CFEngine, MCollective, etc.
One-off work is (usually) a myth
Managed Infra opposed to mucked-about infra
DevOpsProcess
Security isn't a Task
Little more detail...
You can't apply security later
Both infra & code needs to be secure
“Noboday wants to hack me” is a myth
HTTPS isn't a fad
# chmod 777 kills puppies
SELinux / AppArmor isn't your enemy
Process
Scaling starts from the beginning
Little more detail...
Plan early. Start from basics.
Balance load & plan for HA
Use Caching
Do stress / load testing
Not every technology scales gracefully
Use messaging, async jobs when needed
Etc., etc.
Process
Monitor & Measure
Little more detail...
Capture important metrics
Measure, analyze & improve
Monitoring sucks. But still...
Home grown a NMS aren't necessarily better
Have notification alerting in place
Process
Q & A
Anybody who wants ISO's of the following Linux distros,meet me with USB drives after the session.
BackTrack, CentOS, Debian, Fedora, Hanthana, MeeGo, Mint,Ubuntu, Ubuntu Server