scaling apps - karl bunyan 28th november
TRANSCRIPT
S c a lin g a n a p p lic a t io n
F a c e b o o k D e v e lo p e r ’s G a r a g eN o v e m b e r 2 8 t h 2 0 0 7
Scaling an application
What I’m covering
• Facebook specific issues
• A couple of problems solved• A couple of hacks
• Other issues around scaling
Scaling an application
What I’m not covering:
• Writing good code
• Hosting and hardware• Server optimising
Scaling an application
About Exponetic
• Web application developers
• Based in London• 12 members of staff
• Founded in 2002
• 9 Facebook applications live
• 5 currently in development
The application
Two approaches
• A useful app that’s tricky to scale
• A trivial app that uses no resources
The application
Two approaches
• A trivial app that uses no resources– Which Vegetable Are You?
– 155k users
• A useful app that’s tricky to scale– Six Degrees of Separation– 205k users
The application
Six degrees of separation
• Started as an R&D project: 1.5 days work
• Designed to test the end-to-end process of releasing an application
• We didn’t expect it to go anywhere
• Built in PHP on Windows, with SQL Server (to make use of views and triggers)
The application
Six degrees of separation
Original functionality consisted of:
• Calculate how many degrees of separation from the user to me
• Give them some other numbers
The application
What kind of scale?
Active users:
• Small: < 200
• Smallish: < 1,000• Bigish: < 5,000
• Big: < 25,000 We are here
• Huge: …
What kind of scale?
• Currently 205k users
• 87 users after 2 days
• 2000 users after 1 week• 9700 users after 2 weeks
• 128k users after 2 months
What kind of scale?
0
20000
40000
60000
80000
100000
120000
140000
160000
29/08
/200
7
05/09
/200
7
12/09
/200
7
19/09
/200
7
26/09
/200
7
03/10
/200
7
10/10
/200
7
17/10
/200
7
24/10
/200
7
Installs over time
How to get scale
Two different approaches to application design...
How to get scale
Six Degrees:
• Nothing spammy
• No feeds or notifications used• The sign-up process is:
– User sign-up– Shown results– User is encouraged to invite friends “in order
to connect better”
How to get scale
Which Vegetable:
• Give the user a reward for sending invites:– “Invite your friends to find out which
vegetables they are”
• App promoted on “Six Degrees” for 2 days
• Viral spread thereafter
Overview of the code
How “Six Degrees” is put together
Overview of the code
Version 1:
• A handful of FBML pages
• A lot of database triggers• User profiles updated in real time
Week 1 – 2,000 users
• A few bugs to fix
• Scaling is a minimal issue
• Time to add a new feature:– Show connections back
• Moaning users: “I guess you all are in a "repair stage", and the application is unusable at this point? Please notify us when it's been fixed. Thank you!”
Beginning week 2
• Starting to slow down
• “A few kinks”
• Time to streamline the API calls• … and the database
• Use fb:ref to minimise profile updates
• Use an iframe to perform background tasks
Hack #1: Iframe tasks
• FBML is useful, but Facebook doesn’t wait around...
• ...whereas the API may make you wait• Put tasks that may take time into an iframe
• Pass parameters through the fb:iframe src attribute e.g. <fb:iframe src=“find.php?user1=12345&user2=23456" />
The application
[Iframe]
Week 2 – 10,000 users
• It’s getting busy
• New features have new demands
• Almost every calculation is in an iframe• Queuing of profile updates
• “select id from users with (nolock)”
• More RAM for the server
• The “Calculating…” script
Hack #2: Calculating…
Hack #2: Calculating…
• One iframe:– contains a dynamic ‘calculating’ message– and another iframe
• which does the calculation• and tells its parent when it’s finished
• All done through DHTML
Week 4 – 100,000 users
• Separate server for the database– Dual core– 4 gig ram
• Remove the triggers and run “cleanup” scripts
Week 12 – 200,000 users
• Outgrowing the database server
• Queries are often taking minutes
• 20,000 page views/day, but performance is poor
Week 12 – 200,000 users
More user comments:
“It used to work and it was brilliant. The other day, I was showing it to my friends and it didn't work. I tried the same names that gave results before and even they didn't work. Sometimes it does not find the person, sometimes it times out and etc...Make it work, plz!”
Week 12 – 200,000 users
Next steps:
• The concept was never intended to scale
• Needs revision and rethinking
I’d rather have to change the application once I have the users.
How to scale?
How to scale?
• Design for scale?– Don’t know what’s going to be popular– It’s not a bad problem to solve
• Give the user something on the screen
• Limit API calls in FBML
• Use queuing for non-critical updates• Use the “about” wall to communicate
How to scale?
All condensed into one simple PHP function:
How to scale?
if($app->isScalable())
{
$server++;
}else{
break;
}
S c a lin g a n a p p lic a t io nT h a n k s
F a c e b o o k D e v e lo p e r ’s G a r a g eN o v e m b e r 2 8 t h 2 0 0 7