Download - Clug 2012 March web server optimisation
Website Optimisation
CLUG March 2012
contents
● Network● Compression● HTTP Caching● Structuring● Measurement● Application Caching● Monitoring
contents
● Network● Compression● HTTP Caching● Structuring● Measurement● Application Caching● Database tuning● Monitoring
Client
Server
Networkmtr c 10 r www.openquery.com
HOST: spaceman.localhost Loss% Snt Last Avg Best Wrst StDev
1.| 192.168.1.1 0.0% 10 0.8 2.6 0.7 19.3 5.8
2.| 192.168.3.254 0.0% 10 1.1 1.2 1.1 1.5 0.1
3.| lns20.cbr1.internode.on.n 0.0% 10 22.6 22.7 22.3 23.2 0.2
4.| gi002.cor3.cbr1.interno 0.0% 10 22.2 22.5 22.0 23.1 0.4
5.| te600.bdr1.syd4.interno 0.0% 10 207.1 207.7 207.1 208.4 0.4
6.| te000.bdr1.syd7.interno 0.0% 10 208.4 210.1 207.1 228.2 6.4
7.| po300.bdr1.sjc2.interno 0.0% 10 207.5 207.2 206.7 207.9 0.4
8.| 10gigabitethernet23.core 0.0% 10 185.9 187.0 185.0 190.7 2.0
9.| 10gigabitethernet11.core 0.0% 10 186.3 205.2 186.0 367.5 57.1
10.| linodellc.10gigabitether 10.0% 10 186.1 189.1 185.4 210.8 8.1
11.| bluefish.lentz.com.au 10.0% 10 186.5 186.6 186.0 187.9 0.6
Network (DNS)
● Check whois is showing the right nameserver
whois openquery.com NS1.LINODE.COM
NS2.LINODE.COM
NS3.LINODE.COM
NS4.LINODE.COM
NS1.CREATIVECONTINGENCIES.COM
NS2.CREATIVECONTINGENCIES.COM
NS3.CREATIVECONTINGENCIES.COM
● Ensure correct record is returned by querying each nameserver
● dig www.openquery.com @ns1.creativecontingencies.com
Network (SSL/TLS)
Session Tickets - 8 packets 3 roundtrips before application data
EDH crypto – computational expensive by has perfect forwards secrecy
Network (HTTP keep-alive)
● Request / Response header
Connection:keep-alive
On by default. Can't handle variable length pages
Compression● Gzip/deflate●
Compression● Apache's HTTPd
● Enable mod_deflate
<IfModule mod_deflate.c>
SetOutputFilter DEFLATE
</IfModule>● Lighty
● Enable “mod_compress” in server.modules
● Nginx
gzip on;
Compression
● PHP● Php.ini zlib.output_compression On
HTTP Caching
● RFC 2616 Hypertext Transfer Protocol – HTTP/1.1
HTTP Caching
● Etag● Unique string – usual some form of digest
Browsers request with HTTP header:
If-None-Match: “uniquestring....”● Server response: 304 not modified if nothing
changes
HTTP Caching (Etag)
● HTTPd
FileETag Mtime
HTTP Caching (Expires)
● HTTP Response header
Expires: Thu, 01 Dec 2012 16:00:00 GMT
Date:Thu, 22 Mar 2012 01:00:17 GMT
Last-Modified:Sun, 11 Mar 2012 20:55:16 GMT
● HTTP Request
If-Modified-Since: Sat, 29 Oct 1994 19:43:31 GMT
HTTP Caching (Cache-Control) Cache-Control = "Cache-Control" ":" 1#cache-directive
cache-directive = cache-request-directive
| cache-response-directive
cache-request-directive =
"no-cache" ; Section 14.9.1
| "no-store" ; Section 14.9.2
| "max-age" "=" delta-seconds ; Section 14.9.3, 14.9.4
| "max-stale" [ "=" delta-seconds ] ; Section 14.9.3
| "min-fresh" "=" delta-seconds ; Section 14.9.3
| "no-transform" ; Section 14.9.5
| "only-if-cached" ; Section 14.9.4
| cache-extension ; Section 14.9.6
cache-response-directive =
"public" ; Section 14.9.1
| "private" [ "=" <"> 1#field-name <"> ] ; Section 14.9.1
| "no-cache" [ "=" <"> 1#field-name <"> ]; Section 14.9.1
| "no-store" ; Section 14.9.2
| "no-transform" ; Section 14.9.5
| "must-revalidate" ; Section 14.9.4
| "proxy-revalidate" ; Section 14.9.4
| "max-age" "=" delta-seconds ; Section 14.9.3
| "s-maxage" "=" delta-seconds ; Section 14.9.3
| cache-extension ; Section 14.9.6
Gzip + Caching
trunk/roundcubemail/.htaccess ¶
<IfModule mod_deflate.c>
SetOutputFilter DEFLATE
</IfModule>
<IfModule mod_headers.c>
# replace 'append' with 'merge' for Apache version 2.2.9 and later
Header append Cache-Control public env=!NO_CACHE
</IfModule>
<IfModule mod_expires.c>
ExpiresActive On
ExpiresDefault "access plus 1 month"
</IfModule>
FileETag MTime
SavingsOn initial response: 104.2K (vs 255.6K) / 44 requests)
On next page: 6.2K 2 requests (vs 204.7K / 44 requests)
StructuringUse Yslow / Firebug
Measurement (Application)
create table `timings` ( `id` int(4) NOT NULL AUTO_INCREMENT, 'host' varchar(20),'page' varchar(120), int t1,int t2, int t3.....)
And in the index.php
$sample = rand(0,100) < 5; # 5% sample rate
if ($sample) $t0 = microtime(TRUE);
..bits of code
if ($sample) $t1 = microtime(TRUE) - $t0;
..
if ($sample) $t2 = microtime(TRUE) - $t1;
...
if ($sample) {
insert into timings .....$t1*1000, $t2*1000....
}
Application Caching
● Be strategic – cost / benefit● Hard to generate – graphs, database queries,
complex maths● Frequent pages: front pages, rss, ajax
Application Caching (duplicate)
if (!$db->count('name', $name, DB_CACHE))
{
// if no cache entry exists, fetch the appropriate html and insert
...
} else {
// row exists - determine if it is still valid
$last_update = $db->getRecord('time', DB_CACHE, "name='$name'");
....
if (strtotime($last_update) >= strtotime($max_age)) // value is still valid - retrieve existing
{
$result = $db->getRecord('value', DB_CACHE, "name='$name'");
Fetching the same database entry three times.
Application Caching
● Cache an entire page● Memcache is your friend – RAM, autoexpire,
overload intelligent expire● Share between servers● Nice API in all languages
● e.g. PHP
public mixed Memcached::get ( string $key [, callable $cache_cb [, float &$cas_token ]] )
Application scaling
● If the application scales by adding webservers you will save on hosting on high power servers
● Ensure application is stateless● Reverse proxy state-full bits to central server if
needed● CDNs or separated servers for static content
saves lots of cpu (latency, keep alives)
Application APC
Graph from apc.php as part of the php_apc package
Database
● Know what indexes are and which queries aren't using them.
● EXPLAIN {sql expression} (Postgresql and Mysql)
● EXPLAIN ANALYSE {sql expression}● Slow query log – enable● Know differences between MySQL engines● Tune your server
● Defaults well out of proportion with your sever.
Database (continued)
● Searching text in SQL – Use Sphinxsearch instead
● Same sql interfaces, pulls off database – much quicker at full text search
Databases (application)
● Eventually one DB will be too little● Plan to buy big hardware (Oracle) or plan to
use a replicated database (for reads)● Some reads will need to occur on the master.
The ones where the data was likely just changed very recently
Monitoring
Monitoring
● Include resource usage● Cpu / RAM / processes / forking / swap / IO /
Bandwidth...● Web server – pages/second, threads● Database – query cache hits/misses,
connections, replication delay● Business value: page load time.● Alert if boundaries exceeded