clug 2012 march web server optimisation

Website Optimisation

CLUG March 2012

contents

● Network● Compression● HTTP Caching● Structuring● Measurement● Application Caching● Monitoring

contents

● Network● Compression● HTTP Caching● Structuring● Measurement● Application Caching● Database tuning● Monitoring

Client

Server

Networkmtr ­c 10 ­r www.openquery.com

HOST: spaceman.localhost          Loss%   Snt   Last   Avg  Best  Wrst StDev

  1.|­­ 192.168.1.1                0.0%    10    0.8   2.6   0.7  19.3   5.8

  2.|­­ 192.168.3.254              0.0%    10    1.1   1.2   1.1   1.5   0.1

  3.|­­ lns20.cbr1.internode.on.n  0.0%    10   22.6  22.7  22.3  23.2   0.2

  4.|­­ gi0­0­2.cor3.cbr1.interno  0.0%    10   22.2  22.5  22.0  23.1   0.4

  5.|­­ te6­0­0.bdr1.syd4.interno  0.0%    10  207.1 207.7 207.1 208.4   0.4

  6.|­­ te0­0­0.bdr1.syd7.interno  0.0%    10  208.4 210.1 207.1 228.2   6.4

  7.|­­ po3­0­0.bdr1.sjc2.interno  0.0%    10  207.5 207.2 206.7 207.9   0.4

  8.|­­ 10gigabitethernet2­3.core  0.0%    10  185.9 187.0 185.0 190.7   2.0

  9.|­­ 10gigabitethernet1­1.core  0.0%    10  186.3 205.2 186.0 367.5  57.1

 10.|­­ linode­llc.10gigabitether 10.0%    10  186.1 189.1 185.4 210.8   8.1

 11.|­­ bluefish.lentz.com.au     10.0%    10  186.5 186.6 186.0 187.9   0.6

Network (DNS)

● Check whois is showing the right nameserver

whois openquery.com NS1.LINODE.COM

NS2.LINODE.COM

NS3.LINODE.COM

NS4.LINODE.COM

NS1.CREATIVECONTINGENCIES.COM

NS2.CREATIVECONTINGENCIES.COM

NS3.CREATIVECONTINGENCIES.COM

● Ensure correct record is returned by querying each nameserver

● dig www.openquery.com @ns1.creativecontingencies.com

Network (SSL/TLS)

Session Tickets - 8 packets 3 roundtrips before application data

EDH crypto – computational expensive by has perfect forwards secrecy

Network (HTTP keep-alive)

● Request / Response header

Connection:keep-alive

On by default. Can't handle variable length pages

Compression● Gzip/deflate●

Compression● Apache's HTTPd

● Enable mod_deflate

<IfModule mod_deflate.c>

SetOutputFilter DEFLATE

</IfModule>● Lighty

● Enable “mod_compress” in server.modules

● Nginx

gzip on;

Compression

● PHP● Php.ini zlib.output_compression On

HTTP Caching

● RFC 2616 Hypertext Transfer Protocol – HTTP/1.1

HTTP Caching

● Etag● Unique string – usual some form of digest

Browsers request with HTTP header:

If-None-Match: “uniquestring....”● Server response: 304 not modified if nothing

changes

HTTP Caching (Etag)

● HTTPd

FileETag Mtime

HTTP Caching (Expires)

● HTTP Response header

Expires: Thu, 01 Dec 2012 16:00:00 GMT

Date:Thu, 22 Mar 2012 01:00:17 GMT

Last-Modified:Sun, 11 Mar 2012 20:55:16 GMT

● HTTP Request

If-Modified-Since: Sat, 29 Oct 1994 19:43:31 GMT

HTTP Caching (Cache-Control) Cache-Control = "Cache-Control" ":" 1#cache-directive

cache-directive = cache-request-directive

| cache-response-directive

cache-request-directive =

"no-cache" ; Section 14.9.1

| "no-store" ; Section 14.9.2

| "max-age" "=" delta-seconds ; Section 14.9.3, 14.9.4

| "max-stale" [ "=" delta-seconds ] ; Section 14.9.3

| "min-fresh" "=" delta-seconds ; Section 14.9.3

| "no-transform" ; Section 14.9.5

| "only-if-cached" ; Section 14.9.4

| cache-extension ; Section 14.9.6

cache-response-directive =

"public" ; Section 14.9.1

| "private" [ "=" <"> 1#field-name <"> ] ; Section 14.9.1

| "no-cache" [ "=" <"> 1#field-name <"> ]; Section 14.9.1

| "no-store" ; Section 14.9.2

| "no-transform" ; Section 14.9.5

| "must-revalidate" ; Section 14.9.4

| "proxy-revalidate" ; Section 14.9.4

| "max-age" "=" delta-seconds ; Section 14.9.3

| "s-maxage" "=" delta-seconds ; Section 14.9.3

| cache-extension ; Section 14.9.6

Gzip + Caching

trunk/roundcubemail/.htaccess ¶

<IfModule mod_deflate.c>

SetOutputFilter DEFLATE

</IfModule>

<IfModule mod_headers.c>

# replace 'append' with 'merge' for Apache version 2.2.9 and later

Header append Cache-Control public env=!NO_CACHE

</IfModule>

<IfModule mod_expires.c>

ExpiresActive On

ExpiresDefault "access plus 1 month"

</IfModule>

FileETag MTime

SavingsOn initial response: 104.2K (vs 255.6K) / 44 requests)

On next page: 6.2K 2 requests (vs 204.7K / 44 requests)

StructuringUse Yslow / Firebug

Measurement (Application)

create table `timings` ( `id` int(4) NOT NULL AUTO_INCREMENT, 'host' varchar(20),'page' varchar(120), int t1,int t2, int t3.....)

And in the index.php

$sample = rand(0,100) < 5; # 5% sample rate

if ($sample) $t0 = microtime(TRUE);

..bits of code

if ($sample) $t1 = microtime(TRUE) - $t0;

..

if ($sample) $t2 = microtime(TRUE) - $t1;

...

if ($sample) {

insert into timings .....$t1*1000, $t2*1000....

}

Application Caching

● Be strategic – cost / benefit● Hard to generate – graphs, database queries,

complex maths● Frequent pages: front pages, rss, ajax

Application Caching (duplicate)

if (!$db->count('name', $name, DB_CACHE))

{

// if no cache entry exists, fetch the appropriate html and insert

...

} else {

// row exists - determine if it is still valid

$last_update = $db->getRecord('time', DB_CACHE, "name='$name'");

....

if (strtotime($last_update) >= strtotime($max_age)) // value is still valid - retrieve existing

{

$result = $db->getRecord('value', DB_CACHE, "name='$name'");

Fetching the same database entry three times.

Application Caching

● Cache an entire page● Memcache is your friend – RAM, autoexpire,

overload intelligent expire● Share between servers● Nice API in all languages

● e.g. PHP

public mixed Memcached::get ( string $key [, callable $cache_cb [, float &$cas_token ]] )

Application scaling

● If the application scales by adding webservers you will save on hosting on high power servers

● Ensure application is stateless● Reverse proxy state-full bits to central server if

needed● CDNs or separated servers for static content

saves lots of cpu (latency, keep alives)

Application APC

Graph from apc.php as part of the php_apc package

Database

● Know what indexes are and which queries aren't using them.

● EXPLAIN {sql expression} (Postgresql and Mysql)

● EXPLAIN ANALYSE {sql expression}● Slow query log – enable● Know differences between MySQL engines● Tune your server

● Defaults well out of proportion with your sever.

Database (continued)

● Searching text in SQL – Use Sphinxsearch instead

● Same sql interfaces, pulls off database – much quicker at full text search

Databases (application)

● Eventually one DB will be too little● Plan to buy big hardware (Oracle) or plan to

use a replicated database (for reads)● Some reads will need to occur on the master.

The ones where the data was likely just changed very recently

Monitoring

Monitoring

● Include resource usage● Cpu / RAM / processes / forking / swap / IO /

Bandwidth...● Web server – pages/second, threads● Database – query cache hits/misses,

connections, replication delay● Business value: page load time.● Alert if boundaries exceeded