lessons learned at wakoopa
DESCRIPTION
The talk I gave at RubyEnRails 2008TRANSCRIPT
Lessons Learnedwhile building Wakoopa
Menno van der SmanDeveloper at Wakoopa
since May 2007
Wakoopa is about software
MacPC
200million
Does it scale?
Scaling the front is easy
Scaling the back is harder
Limits spawn creativity
One server
at Railsmachinesliced into staging, production and db
MySQL 4.1Apache + Mongrel
Background queue
Lots of tools already
BackgroundRB, DelayedJob, Daemons etc
Rolling our own
database basedusing script/runner
processing multiple items in one loopone cache update per loop
Smart queries
INSERT INTO hourly_usage (...) VALUES (...) ON DUPLICATE KEY UPDATE active_seconds = active_seconds + ..., idle_seconds = idle_seconds + ...
UPDATE developers SET active_seconds = active_seconds + ...
Allows for concurrency
Size = rowsize x rows
Make rowsize smaller
sometimes `id` shouldn’t be the
Primary Key
Enter composite_primary_keys
Overrides default PK
Original idea by Dr NicEnhanced for Rail 2.1 by Darrin Holsthttp://compositekeys.rubyforge.org/
class HourlyUsage < ActiveRecord::Base set_primary_keys :user_id, :software_id, :used_at ...end
Archive old datathat’s not frequently used
HourlyUsageto
HourlyUsageStorage
SELECT ... FROM hourly_usage WHERE used_at < '...' INTO OUTFILE '/path/to/file'
LOAD DATA INFILE '/path/to/file' REPLACE INTO TABLE hourly_usage_storage
avoids long locks
Upcoming improvements
MySQL 5.1Partitioning
CREATE TABLE hourly_usage_storage ( ... )PARTITION BY RANGE (software_id) ( PARTITION p0 VALUES LESS THAN (...), PARTITION p1 VALUES LESS THAN (...), PARTITION p2 VALUES LESS THAN (...), PARTITION p3 VALUES LESS THAN (...), .... PARTITION pn VALUES LESS THAN MAXVALUE);
Available in MySQL 5.1
Makes ‘WHERE software_id = ...’ very fast
And many more plans
MySQL 5.1Memcached
SphinxRails 2.1
Amazon EC2
Designing a new infrastructure
Announcing a new blog
Inside Wakoopahttp://inside.wakoopa.com
Questions?
http://wakoopa.com/jobs
Looking for C++ and frontend awesomeness
or suggestions?