meetup performance

Meetup PerformanceGreg Whalin, CTO Meetup (@gwhalin), Justin Cataldo, Lead UI Engineer (@jcataldo), Will Howard, Lead UI Engineer

Meetup

Platform for local groups

Mission is MEME (Meetup Everywhere About Most Everything)

~6.2m members~70k Groups~500k group joins every month~5 million Meetups have happened~53 million RSVPs

General Architecture and Back-end Performance

(just a tiny bit - this could and should be another presentation)

DataMySQL (any RDBMs store) biggest pain

replication use InnoDB!smart indexing (take advantage of clustered indexes)server side conn pools and short timeouts on connectionssmart about fs choice on Linux (we use XFS after benchmarking)

Hardware - relatively expensive boxes

multi-core (8 or 16) Opteronlots of ram (32/64GB)fast drives (lots of spindles, RAID10)

Cache, cache, cache!

innodb buffer cachememcachelocal app server memory

Shrink data when possiblearchive unused datacustom serialization when serializingdata partitioning/sharding

Storage

Over half a million photos uploaded to Meetup every month

Scaled and processed into 4 different sizes (plus original)

Storage solutions

Options for growth include NAS, SAN, or something else

NAS and SAN are single point of failure and possibly $$$

Only postpones problem

MogileFSdeveloped by Brand Fitzpatrick (i.e. Memcached) OSS distributed filesystem (built in Perl)any hard drive on network can easily be added to clusterscales easily and cheaply

Much much much more going on

but...

UI Performance(much of our focus here)

Why does performance matter?

Why does performance matter?

Slow site

Why does performance matter?

Slow site

Bad User Experience

Why does performance matter?

Slow site

Bad User Experience

Drop in Member Activity

Why focus on front end performance?

Back end only accounts for 10-15% of the response timeLess time and resourcesCosts less

http://developer.yahoo.net/blog/archives/2007/03/high_performanc.html

Case Study: Event Details

Load time = 6.321s

www.webpagetest.org

Event Details: Load Time

Lots of javascript being loaded

Event Details: Requests

How do we improve performance?

3 Steps to improving performance

1. Externalize script2. Move scripts to the bottom of the page3. Reduce requests

3 Steps to improving performance

1. Externalize script2. Move scripts to the bottom of the page3. Reduce requests

Why externalize scripts

Prevents blockingInline scripts prevent asynchronous downloadsDownloads must wait for the script to be executed

CachingInline JavaScript is downloaded every timeExternal scripts are cached by the browserReduced overall page size

ReusableCan use the same code somewhere else on the site easily

Pull out inline script

<script type="text/javascript">if(typeof Meetup.EventDetails == 'undefined') Meetup.EventDetails = {}; (function(){ var self = Meetup.EventDetails;...})();</script>

> EventDetails.js

3 Steps to improving performance

1. Externalize script2. Move scripts to the bottom of the page3. Reduce requests

Web:scriptCustom tag built in houseMoves inline and external script to the bottom of the pageAllows UI engineers to not have to worry about where they place scriptsCompresses inline script using YUICompressor

/***** Load External Script *****/ <web:script src="/script/Meetup/packed/EventDetails.js" />

/***** Load Inline Script *****/ <web:script> Meetup.Copy.noMembersMarkedAttended = "<trn:message key="event.attendance.noMembersMarkedAttended">No members have been marked attended</trn:message>"; Meetup.Copy.noMembersMarkedAttendedDynam = '<trn:message key="event.attendance.noMembersMarkedAttendedDynam"><trn:param name="GROUPING">__GROUPING__</trn:param>No members in "{GROUPING}" have been marked attended</trn:message>'; Meetup.Copy.noMembersMarkedAbsent = "<trn:message key="event.attendance.noMembersMarkedAbsent">No members have been marked absent</trn:message>";</web:script>

3 Steps to improving performance

1. Externalize script2. Move scripts to the bottom of the page3. Reduce requests

Reduce Requests

Concatenate as much as possibleLoad only what we need upfront

Concatenation using Sprockets

Sprockets (www.getsprockets.com)Created by 37SignalsRuby library that preprocesses and concatenates JavaScript filesBaked into our build process

Concatenation using Sprockets (cont.)

EventDetails.jsCommentDeleteConfirm.jsBubbleTips.jsPlacesManager.jsMaxCharactersEnforcer.js > EventDetails.js

Using Sprockets/******* Sprockets Directives *******/ //= require <CommentDeleteConfirm>//= require <DomDeco/BubbleTips>//= require <DomDeco/PlacesManager>//= require <DomDeco/MaxCharactersEnforcer> /******* Begin Event Details Code *******/ if(typeof(Meetup.EventDetails) == 'undefined') Meetup.EventDetails = {}; (function(){ var self = Meetup.EventDetails;...........})();

/******* Build Process *******/ <exec executable="sprocketize" failonerror="true" output="${image.dir}/script/Meetup/packed/EventDetails.js"> <arg value="-I"/> <arg path="${image.dir}/script/Meetup/"/> <arg path="${image.dir}/script/Meetup/EventDetails.js"/></exec>

Lazy Loading

Defer loading of javascript files until they are neededReduces the initial upfront requestsHelps reduce blocking by downloading files asynchronouslyPrecaching

Lazy Loading: How it works

Inserts scripts into the head dynamically

Meetup.Script.include("http://static2.meetupstatic.com/script/Meetup/DomDeco/LinkDecorator.js",callback);

var scriptNode = function(src) { return createDOM("script", { "type": "text/javascript", "src": src }); }

var load = function(id, src, n) { var script = scriptNode(url); head.appendChild(script); script.onload = script.onreadystatechange = function() { if (!this.readyState || this.readyState == "loaded" || this.readyState == "complete") { script.onload = script.onreadystatechange = null; } } }

Did it make a difference?

Javascript requests cut by 50%

Reduced requests

Event Details Page Load After Old Load Time = 6.321s New Load time = 4.643s

Event Details Page Load After Old Load Time = 6.321s New Load time = 4.643s

Load time decreased by

27%!

But that's not all

Execute when the dom is ready

Execute early

Execute early: DOMReadyLibraries that have a DOM ready solution:

jQueryYUIPrototypePretty much every modern JS library (not MochiKit)

Meetup uses MochiKit, so we rolled our own.

Execute early: DOMReady

Meetup.DOMReady.ready(function(){Meetup.EventDetails.init();if(Meetup.EventDetails.isCanceled != 4 && Meetup.EventDetails.rsvp != 0){deletePopup = new Meetup.CommentDeleteConfirm();deletePopup.pagerOffsetFieldName = "p_commentsList";deletePopup._decorate();}});

And by rolled our own, I mean we're using the Dean Edwards/Matthias Miller/John Resig implementation. http://dean.edwards.name/weblog/2006/06/again/#comment5338

With a few changes.

Execute early: Even earlier

Do you need to wait for the DOM to be ready?

If you aren't manipulating the DOM, there's no reason to wait until it's ready.

Automated Image Optimization

./filescan.sh /usr/local/meetup/static/img/ 'smusher -q @' 'jpg,png'&

Using smush.ithttp://developer.yahoo.com/yslow/smushit/

Smusher Ruby gemhttp://github.com/grosser/smusher (gem install smusher)

BASH script that watches our image directories for changes and executes smusher.

Event Delegation

Run less JavaScript up front

Event Delegation

From: http://www.quirksmode.org/js/events_order.html

But first, a little bit about event bubbling...

Event DelegationPros

Much faster on load (not connecting DOM elements)

No need to disconnect / reconnect with AJAX calls

Fewer memory leaks

Cons�

�Does not work well with nested elements

Doesn't work with all events

Slight performance hit with execution

A lot of JS libraries already have plug-ins for event delegation (jQuery, YUI, prototype).

But, it's pretty easy to write your own (we did).

Event Delegation: Meetup.Dispatcher<div id="C_page"> ...

<span class="meetup-topic"><a class="topic-id-7029 topic-link J_onClick topic-info-hover" href="http://javascript.meetup.com/cities/us/ny/brooklyn/">JavaScript</a></span> ...</div>

var mdp = Meetup.Dispatcher.init("C_page", "onmouseover");

mdp.registerFunc("topic-info-hover", Meetup.UI.InfoHover.mouseOver);

Meetup.UI.infoHover.mouseOver = function(e) {topicId = _getTopicId(e.target());if (!topicId || topicId == "") return;_primeCache(topicId);var activeEl = _getActiveEl(e.target());var pos = getElementPosition(activeEl);

...}

<div id="C_page"> ...

<span class="meetup-topic"><a class="topic-id-7029 topic-link J_onClick topic-info-hover" href="http://javascript.meetup.com/cities/us/ny/brooklyn/">JavaScript</a></span> ...</div>

Event Delegation: Meetup.Dispatcher

Event Delegation: Meetup.Dispatcher<div id="C_page"> ...

<span class="meetup-topic"><a class="topic-id-7029 topic-link J_onClick topic-info-hover" href="http://javascript.meetup.com/cities/us/ny/brooklyn/">JavaScript</a></span> ...</div>

// Inits a new instance of dispatcher// Connects a mouseover event to the parent container "C_page"var mdp = Meetup.Dispatcher.init("C_page", "onmouseover");

// Calls Meetup.UI.infoHover.mouseOver() when target element has "topic-info-hover" class.mdp.registerFunc("topic-info-hover", Meetup.UI.InfoHover.mouseOver);

Meetup.UI.infoHover.mouseOver = function(e) {topicId = _getTopicId(e.target());if (!topicId || topicId == "") return;_primeCache(topicId);var activeEl = _getActiveEl(e.target());var pos = getElementPosition(activeEl);

...}

<div id="C_page"> ...

<span class="meetup-topic"><a class="topic-id-7029 topic-link J_onClick topic-info-hover" href="http://javascript.meetup.com/cities/us/ny/brooklyn/">JavaScript</a></span> ...</div>

Speeding up DOM crawling with Sizzle

Internet Explorer 7

MochiKit: 6623.94ms

Sizzle: 306.03ms

Firefox 3.5MochiKit: 210.524ms

Sizzle: 111.553ms

sizzlejs.com

Where do we go from here?

More concatenation and lazy loading where it makes sense Defer image loading where it makes senseReduce DOM elementsReduce CSS and improved selector efficiencyand more

Deployment and ServingAs it pertains to css/html/js

Launches

Launch multiple times a day (sometimes)

Need launches to be quick / no downtime

Optimize static resources only at deploy time and only if modified

Deployment of static content

Sprockets (reduce requests)

YUICompressor for js (local mod to speed up optimizing multiple files) Pre-compress css and jspSet cache-control to be fresh for over a year (indefinite)

All links on site generate programatically and versioned

Link generation

Custom JSTL page functionhttp vs https domain sharding and cookie free domaincontent versioning

<link rel="stylesheet" href="${mfn:staticUrl( "/style/meetup.css", state.context.isSecure )}" type="text/css" /> <img src="${mfn:imgUrl( '/img/noPhoto_50.gif', state.context.isSecure )}" alt=""class="noPhoto" />

<link rel="stylesheet" href="http://static1.meetupstatic.com/050991196173395491322880/style/meetup.css" type="text/css" /> <img src="http://img1.meetupstatic.com/39194172310009655/img/noPhoto_50.gif" alt="" class="noPhoto"/>

Versioning static content

MD5 checksum of contents of fileRun w/ each launchStore versions in db tied to release

mysql> select * from resource_version where filename = '/style/base.css'; | 394328 | /style/base.css | 39020083689267241 | 567 | | 398052 | /style/base.css | 8487620432388779772669 | 568 | | 401776 | /style/base.css | 357470606563045379 | 569 | | 405506 | /style/base.css | 3068234199748867 | 571 | | 409240 | /style/base.css | 024745310801291061590 | 572 | | 412974 | /style/base.css | 024745310801291061590 | 573 | | 416708 | /style/base.css | 09972542737049101325 | 574 |

Static content serving Served off CDN (reverse proxy)Anycast DNS for hostname resolution Origin servers running lighttpdStrip versioning from url using rewrite rulesCache set to 12 months outCompress

expire.url = ( "/" => "access plus 12 months") compress.cache-dir = "/var/cache/lighttpd/"compress.filetype = ("text/plain", "text/html", "text/css", "application/x-javascript", "text/javascript") url.rewrite-once = ( "^/script/(.*/)?[0-9]+/(.+).js$" => "/script/$1$2.js", "^/style/(.*/)?[0-9]+/(.+).css$" => "/style/$1$2.css", "^/img/(.*/)?[0-9]+/(.+).(gif|png|jpg)$" => "/img/$1$2.$3", "^/\d+/script/(.+).js" => "/script/$1.js", "^/\d+/style/(.+).css" => "/style/$1.css", "^/\d+/img/(.+).(gif|jpg|png)" => "/img/$1.$2", "^/photos/(([^\/]+)/.+/(.+)\.jpeg)(?:\?.*)?$" => "/cgi-bin/photos.fcgi?type=$2&key=$3", "^/photos/([^\/]+)/([^\/]+)\.jpeg(?:\?.*)?$" => "/cgi-bin/photos.fcgi?type=$1&key=$2", "^/file.*" => "/")

Questions?Also, we need help! Hiring for:

Linux Systems AdministratorSoftware EngineersUI EngineersQA EngineersCommunity SpecialistPR RenegadeSponsorship SalesAccount Coordinator

http://www.meetup.com/jobs/