About Me• Apache
– Committer and PMC for Apache Traffic Server (ATS) – Apache Member
• IETF– HTTP/2– TLS– TCP Enhancement
• Yahoo– WebRing, GeoCities, People Search, Personals, Tiger Team, Platform
Architect, Edge Group, Network Research, Apache/IETF
Reverse Proxy• Proxy in front of your own web servers• Caching?• Geographic location?• Connection handling?• SSL termination?• SPDY support?• Adding business logic?
Forward / Intercepting Proxy• Proxy in front of the Internet• Configure clients to use proxy?• Caching?• SSL - CONNECT?• SSL - termination?
FeaturesATS NGiNX Squid Varnish Apache httpd
mod_proxy
Reverse Proxy Y Y Y Y Y
Forward Proxy Y N Y N Y
Transp. Proxy Y N Y N Y
Plugin APIs Y Y partial Y Y
Cache Y Y Y Y Y
ESI Y N Y partial N
ICP Y N Y N N
SSL Y Y Y N Y
SPDY Y Y N N partial
How things can go wrong: Vary$ curl -D - -o /dev/null -s --compress http://10.118.73.168/HTTP/1.1 200 OKServer: nginx/1.3.9Date: Wed, 12 Dec 2012 18:00:48 GMTContent-Type: text/html; charset=utf-8Content-Length: 8051Connection: keep-aliveCache-Control: public, max-age=900Last-Modified: Wed, 12 Dec 2012 17:52:42 +0000Expires: Sun, 19 Nov 1978 05:00:00 GMTVary: Cookie,Accept-EncodingContent-Encoding: gzip
How things can go wrong: Vary$ curl -D - -o /dev/null -s http://10.118.73.168/HTTP/1.1 200 OKServer: nginx/1.3.9Date: Wed, 12 Dec 2012 18:00:57 GMTContent-Type: text/html; charset=utf-8Content-Length: 8051Connection: keep-aliveCache-Control: public, max-age=900Last-Modified: Wed, 12 Dec 2012 17:52:42 +0000Expires: Sun, 19 Nov 1978 05:00:00 GMTVary: Cookie,Accept-EncodingContent-Encoding: gzip
EPIC FAIL!
Note: no gzip request
CoAdvisor HTTP protocol quality tests for reverse proxies
ATS 3.3.1
Nginx 1.3.9
Squid 3.2.5
Varnish 3.0.3
0 100 200 300 400 500 600
Failures Violations Success
49%
81%
51%
68%
CoAdvisor HTTP protocol quality tests for reverse proxies
ATS 3.3.1
Nginx 1.3.9
Squid 3.2.5
Varnish 3.0.3
0 100 200 300 400 500 600
Failures Violations Success
25%
6%
27%
15%
Architecture And Process Models• Multithreading• Events• Process• Fibers
– Co-operative multitasking, getcontext/setcontext
Threads• Pros
– Easy to share memory– Lightweight context switching
• Cons– Easy to (accidently) share memory
• Overwriting another threads memory
– Locking• Deadlocks, race conditions, starvation
Problems with Event Processing• Doesn’t work well with
blocking APIs– open(), locking
• It doesn’t scale on SMP by itself
Process Model And ArchitectureATS NGiNX Squid Varnish Apache httpd
mod_proxy
Threads X X X
Events X X X partial X
Processes X X X
Cache• Mainly two types
– File system– Database like
• In memory index– Bytes per object
• Minimize disk seeks and system calls
CacheATS NGiNX Squid Varnish Apache httpd
mod_cache
File system X X X
mmap X
Raw disk/direct IO X X
Ram cache X X
Memory index X X X*
Persistent cache X X X X
ATS Configurationetc/trafficserver/remap.config:
map / http://origin.example.com
etc/trafficserver/records.config:CONFIG proxy.config.http.server_ports STRING 80CONFIG proxy.config.accept_threads INT 3
NGiNX Configurationworker_processes 24;access_log logs/access.log main;
proxy_cache_path /mnt/nginx_cache levels=1:2 keys_zone=my-cache:8m max_size=16384m inactive=600m;proxy_temp_path /mnt/nginx_temp;
server { set $ae ""; if ($http_accept_encoding ~* gzip) { set $ae "gzip"; }
location / { proxy_pass http://origin.example.com; proxy_cache my-cache; proxy_set_header If-None-Match ""; proxy_set_header If-Modified-Since ""; proxy_set_header Accept-Encoding $ae; proxy_cache_key $uri$is_args$args$ae; }
location ~ /purge_it(/.*) { proxy_cache_purge example.com $1$is_args$args$myae }
Squid Configurationhttp_access allow allhttp_port 80 accelworkers 24
cache_mem 4096 MBmemory_cache_shared oncache_dir rock /usr/local/squid/cache 1000 max-size=32768cache_peer origin.example.com parent 80 0 no-query originserver
Varnish Configuration (Cont)
sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=4000sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=2000 -p thread_pool_add_delay=2 -p thread_pool_min=200sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=2000 -p thread_pool_add_delay=2 -p thread_pool_min=1000 -p session_linger=0sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=2000 -p thread_pool_add_delay=2 -p thread_pool_min=1000 -p session_linger=10
Apache httpd ConfigurationLoadModule cache_module modules/mod_cache.soLoadModule cache_disk_module modules/mod_cache_disk.soLoadModule proxy_module modules/mod_proxy.soLoadModule proxy_http_module modules/mod_proxy_http.soInclude conf/extra/httpd-mpm.confProxyPass / http://origin.example.com/
<IfModule mod_cache_disk.c> CacheRoot /usr/local/apache2/cache CacheEnable disk / CacheDirLevels 5 CacheDirLength 3</IfModule>
MaxKeepAliveRequests 10000
• Squid used the most CPU and the worst median latency
• 95th percentile latency with NiGNX, Squid and httpd
ATS NGiNX Squid Varnish httpd0
500
1000
1500
2000
2500
RPS / CPU Usage
ATS NGiNX Squid Varnish httpd0
20000
40000
60000
80000
100000
120000
Requests Per Second
ATS NGiNX Squid Varnish httpd0
2
4
6
8
10
12
14
16
18
Latency
Median
95th
• Squid used the most CPU again
• NGiNX had latency issues• ATS most throughput
ATS NGiNX Squid Varnish httpd0
500
1000
1500
2000
2500
RPS / CPU Usage
ATS NGiNX Squid Varnish httpd0
5000
10000
15000
20000
25000
30000
Requests Per Second
ATS NGiNX Squid Varnish httpd0
5
10
15
20
25
30
35
40
Latency
Median
95th
ATS• Pros
– Scales well automatically, little config needed– Best cache implementation
• Cons– Too many config files– Too many options in the default config files (5.x
fixed this)
NGiNX• Pros
– Lots of plugins– FastCGI support
• Cons– HTTP/1.1 compliance– Latency issues around accepting new connections– Rebuild server for new plugins
Squid• Pros
– Best HTTP/1.1 compliance• Cons
– Memory index for cache using 10x that of ATS– Least efficient with CPU– Worst median latency for keep-alive benchmarks
Varnish• Pros
– VCL (Varnish Configuration Language)• Can do a lot without writing plugins
• Cons– Thread per connection– mmap for cache
• Persistence is experimental
– No SSL or SPDY support
Apache httpd• Pros
– Lots of plugins– Most used http server– Best 95th percentile latency for non-keep-alive
• Cons– SPDY Support
Why ATS?• Scales well
– CPU Usage, auto config• Cache scales well
– Efficient memory index, minimizes seeks• Apache Community• Plugin support
– Dynamic loading of plugins and easy to port existing plugins over
References• ATS - http://trafficserver.apache.org/• NGiNX - http://nginx.org/• Squid - http://www.squid-cache.org/• Varnish - https://www.varnish-cache.org/• Apache httpd - http://httpd.apache.org/