Website Optimisation

CLUG March 2012

contents

Network

Compression

HTTP Caching

Structuring

Measurement

Application Caching

Monitoring

contents

Network

Compression

HTTP Caching

Structuring

Measurement

Application Caching

Database tuning

Monitoring

Client

Server

Network

mtr -c 10 -r www.openquery.com

HOST: spaceman.localhost Loss% Snt Last Avg Best Wrst StDev 1.|-- 192.168.1.1 0.0% 10 0.8 2.6 0.7 19.3 5.8 2.|-- 192.168.3.254 0.0% 10 1.1 1.2 1.1 1.5 0.1 3.|-- lns20.cbr1.internode.on.n 0.0% 10 22.6 22.7 22.3 23.2 0.2 4.|-- gi0-0-2.cor3.cbr1.interno 0.0% 10 22.2 22.5 22.0 23.1 0.4 5.|-- te6-0-0.bdr1.syd4.interno 0.0% 10 207.1 207.7 207.1 208.4 0.4 6.|-- te0-0-0.bdr1.syd7.interno 0.0% 10 208.4 210.1 207.1 228.2 6.4 7.|-- po3-0-0.bdr1.sjc2.interno 0.0% 10 207.5 207.2 206.7 207.9 0.4 8.|-- 10gigabitethernet2-3.core 0.0% 10 185.9 187.0 185.0 190.7 2.0 9.|-- 10gigabitethernet1-1.core 0.0% 10 186.3 205.2 186.0 367.5 57.1 10.|-- linode-llc.10gigabitether 10.0% 10 186.1 189.1 185.4 210.8 8.1 11.|-- bluefish.lentz.com.au 10.0% 10 186.5 186.6 186.0 187.9 0.6

Network (DNS)

Check whois is showing the right nameserver

whois openquery.com NS1.LINODE.COM NS2.LINODE.COM NS3.LINODE.COM NS4.LINODE.COM NS1.CREATIVECONTINGENCIES.COM NS2.CREATIVECONTINGENCIES.COM NS3.CREATIVECONTINGENCIES.COM Ensure correct record is returned by querying each nameserver

dig www.openquery.com @ns1.creativecontingencies.com

Network (SSL/TLS)

17 packets 4 roundtrips before application data

Network (SSL/TLS)

Session Tickets - 8 packets 3 roundtrips before application dataEDH crypto computational expensive by has perfect forwards secrecy

Network (HTTP keep-alive)

Request / Response header

Connection:keep-alive

On by default. Can't handle variable length pages

Compression

Gzip/deflate

Compression

Apache's HTTPdEnable mod_deflate

SetOutputFilter DEFLATE

LightyEnable mod_compress in server.modules

Nginxgzip on;

Compression

PHPPhp.ini zlib.output_compression On

HTTP Caching

RFC 2616 Hypertext Transfer Protocol HTTP/1.1

HTTP Caching

EtagUnique string usual some form of digest

Browsers request with HTTP header:If-None-Match: uniquestring....

Server response: 304 not modified if nothing changes

HTTP Caching (Etag)

HTTPdFileETag Mtime

HTTP Caching (Expires)

HTTP Response headerExpires: Thu, 01 Dec 2012 16:00:00 GMTDate:Thu, 22 Mar 2012 01:00:17 GMTLast-Modified:Sun, 11 Mar 2012 20:55:16 GMT

HTTP RequestIf-Modified-Since: Sat, 29 Oct 1994 19:43:31 GMT

HTTP Caching (Cache-Control)

Cache-Control = "Cache-Control" ":" 1#cache-directive

cache-directive = cache-request-directive | cache-response-directive

cache-request-directive = "no-cache" ; Section 14.9.1 | "no-store" ; Section 14.9.2 | "max-age" "=" delta-seconds ; Section 14.9.3, 14.9.4 | "max-stale" [ "=" delta-seconds ] ; Section 14.9.3 | "min-fresh" "=" delta-seconds ; Section 14.9.3 | "no-transform" ; Section 14.9.5 | "only-if-cached" ; Section 14.9.4 | cache-extension ; Section 14.9.6

cache-response-directive = "public" ; Section 14.9.1 | "private" [ "=" ] ; Section 14.9.1 | "no-cache" [ "=" ]; Section 14.9.1 | "no-store" ; Section 14.9.2 | "no-transform" ; Section 14.9.5 | "must-revalidate" ; Section 14.9.4 | "proxy-revalidate" ; Section 14.9.4 | "max-age" "=" delta-seconds ; Section 14.9.3 | "s-maxage" "=" delta-seconds ; Section 14.9.3 | cache-extension ; Section 14.9.6

Gzip + Caching

trunk/roundcubemail/.htaccess SetOutputFilter DEFLATE # replace 'append' with 'merge' for Apache version 2.2.9 and later Header append Cache-Control public env=!NO_CACHE ExpiresActive On ExpiresDefault "access plus 1 month" FileETag MTime

SavingsOn initial response: 104.2K (vs 255.6K) / 44 requests)

On next page: 6.2K 2 requests (vs 204.7K / 44 requests)

Structuring

Use Yslow / Firebug

Measurement

Measurement (Application)

create table `timings` ( `id` int(4) NOT NULL AUTO_INCREMENT, 'host' varchar(20),'page' varchar(120), int t1,int t2, int t3.....)

And in the index.php

$sample = rand(0,100) < 5; # 5% sample rate

if ($sample) $t0 = microtime(TRUE);

..bits of code

if ($sample) $t1 = microtime(TRUE) - $t0;..

if ($sample) $t2 = microtime(TRUE) - $t1;

...

if ($sample) { insert into timings .....$t1*1000, $t2*1000....}

Application Caching

Be strategic cost / benefit

Hard to generate graphs, database queries, complex maths

Frequent pages: front pages, rss, ajax

Application Caching (duplicate)

if (!$db->count('name', $name, DB_CACHE)) { // if no cache entry exists, fetch the appropriate html and insert...

} else { // row exists - determine if it is still valid $last_update = $db->getRecord('time', DB_CACHE, "name='$name'"); ....

if (strtotime($last_update) >= strtotime($max_age)) // value is still valid - retrieve existing { $result = $db->getRecord('value', DB_CACHE, "name='$name'");Fetching the same database entry three times.

Application Caching

Cache an entire page

Memcache is your friend RAM, autoexpire, overload intelligent expire

Share between servers

Nice API in all languagese.g. PHPpublic mixed Memcached::get ( string $key [, callable $cache_cb [, float &$cas_token ]] )

Application scaling

If the application scales by adding webservers you will save on hosting on high power servers

Ensure application is stateless

Reverse proxy state-full bits to central server if needed

CDNs or separated servers for static content saves lots of cpu (latency, keep alives)

Application APC

Graph from apc.php as part of the php_apc package

Database

Know what indexes are and which queries aren't using them.

EXPLAIN {sql expression} (Postgresql and Mysql)

EXPLAIN ANALYSE {sql expression}

Slow query log enable

Know differences between MySQL engines

Tune your serverDefaults well out of proportion with your sever.

Database (continued)

Searching text in SQL Use Sphinxsearch instead

Same sql interfaces, pulls off database much quicker at full text search

Databases (application)

Eventually one DB will be too little

Plan to buy big hardware (Oracle) or plan to use a replicated database (for reads)

Some reads will need to occur on the master. The ones where the data was likely just changed very recently

Monitoring

Monitoring

Include resource usage

Cpu / RAM / processes / forking / swap / IO / Bandwidth...

Web server pages/second, threads

Database query cache hits/misses, connections, replication delay

Business value: page load time.

Alert if boundaries exceeded