Traffic Control in Apache

Jed Reynolds Blog.Bitratchet.com

Where in your web site do you need traffic control?

Handling Traffic Efficiently

Right and Wrong Traffic Reduce redundant traffic Eliminate unwanted traffic Protect your dynamic content

Caching and tiered delivery

Spectrum of Traffic Control

DNS

Content Delivery Network (CDN)

Proxies and load balancers

Application v static content servers Application server File system and disk

Right Traffic: URI Stability

The URLs our site has might be precious The location on disk might be moved

20,000 sub directories 10,000 files

Google and Sitemaps

Redirects v Rewrites

Redirect sends a HTTP header, Location: Rewrites are server or application configurations

Redirects with mod_alias

Redirect permanent ^/xml/ < uri http://xml.news.com/ < header

RedirectMatch permanent ^/xml/(.*)\.xml http://xml.news.com/$1.xml

Rewrites with mod_alias

Alias /newspaper < uri /home/newspaper < disk

AliasMatch /(newspaper)/(*\.htm) /home/$1/today/$2

Redirects with mod_rewrite

RewriteCond HTTP_HOST ^(.*)\.news.org$RewriteRule ^(.*)$ < uri http://%1.news.com/$1? < header [R=301,L] < rules

Rewrites with mod_rewrite

RewriteRule /date/(.*) < uri /home/day/$1/index.htm < disk [L] < rule

Avoid Combinations

Order of operations:

1) <Directory>2) <DirectoryMatch>3) <Files>,<FilesMatch> <<

rewrites

4) <Location>,<LocationMatch> << aliases

5) <VirtualHost> << repeat the above order inside VH’s after global scope

Rewrite Scripts

Might be easier to write Possible performance or memory concerns What can you best maintain?

Reducing Redundant Traffic with your Headers

Cache-Control: max-age=3600, must-revalidate

Expires: Fri, 30 Oct 1998 14:19:41 GMT

Last-Modified: Mon, 29 Jun 1998 02:28:12 GMT

ETag: "3e86-410-3596fbbc"

Etags encourage freshest content Encourage client caching

Extend Expires with mod_expires

<LocationMatch "/stuff/*">ExpiresActive onExpiresDefault "access plus 1 hour"</Location>

Disable Etags

Mtime Inode File size

Header unset EtagFileETag none

Encourage 304 responses

Apache is good with file content File content is fast Scripts should detect If-Modified-Since Scripts should send 304 Not Modified PHP Cache_Lite

Banning Traffic with <Directory>

<Directory /home/web>Order Allow,DenyDeny from 10.10.10.10</Directory>

Banning Traffic with mod_rewrite

RewriteCond %{HTTP_USER_AGENT} ^NameOfBadRobot.*RewriteCond %{REMOTE_ADDR} ^123\.45\.67\.[8-9]$RewriteRule ^/news/.+ < uri - < placeholder [F] < rule

Vastly More Traffic

Multiple servers, but where? Load balancer mod_proxy HA Proxy Reverse Proxy Caching Squid CDNs

Thank you

Jed Reynolds has been a programmer analyst since 1996 and enjoys bicycling and his Pentax K10D

blog.bitratchet.com @jed_reynolds jed@bitratchet.com