altitude sf 2017: logging at the edge
TRANSCRIPT
• How Fastly’s logs work What to log
• Set up a service and log endpoint
• What to Log
• JSON logging
• Fun with logs
• Serverless Applicationless Edge Logging
Agenda
• Log endpoint configured
• Customized Fastly log string
• JSON formatted
• BigQuery data import
Expectations at the end
159847 TxHeader c Connection: keep-alive
159847 TxHeader c X-Served-By: cache-jfk8125-JFK
159847 TxHeader c X-Cache: HIT
159847 TxHeader c X-Cache-Hits: 2
159847 TxHeader c X-Timer: S1489171719.109755,VS0,VE0
159847 VCL_call c log deliver
159847 VCL_call c log
159847 VCL_Log c syslog 4rqBj4oy1YChTPgdapiWS4 gcs-test :: {"client.ip":"207.38.219.230","req.request":"GET","req.http.host":"altitude-fastly.s3-website-us-east-1.amazonaws.com","req.request":"GET","req.url":"\/","req.bytes_read":94,"resp.status":200,"resp.bytes_written":545,"resp.http.X-Cache":"HIT","fastly_info.state":"HIT","time.start.usec":1489171719109755,"time.start.iso8601":"2017-03-10 18:48:39","time.end.usec":1489171719109876,"time.elapsed.usec":121}
159847 VCL_return c deliver
https://docs.fastly.com/guides/streaming-logs/custom-log-formats#version-2-log-format
resp.http.Set-Cookie
server.datacenter
server.identity
server.region
if(req.http.Fastly-FF,"1","0")
resp.http.X-Cache
fastly_info.state
Useful things to log
ERRORHITHIT-CLUSTERHIT-REFRESHHIT-STALEHIT-STALE-CLUSTERHIT-STALE-WAIT-CLUSTERHIT-SYNTHHIT-WAITHIT-WAIT-CLUSTERHITPASS
fastly_info.state
MISSMISS-CLUSTERMISS-WAITMISS-WAIT-CLUSTERPASSPASS-WAIT
regsub(fastly_info.state,"-.*","")
obj.hits
geoip.continent_code
geoip.country_code
geoip.region(codeforregionwithincountry)
req.restarts
Useful things to log
Somewhat limited support for Apache log format variables
VCL variables can just dropped in
No (good) support for string literals, let alone things like JSON
Log format v1
Full support for Apache log format variables
Some will never return a value
VCL variables (or code) through Fastly specific extension: %{...}V
Supports string literals, and thus JSON is much easier to do
Default for all newly created endpoints
Legacy endpoints can be upgraded to v2
Log format v2
• <134>2017-03-19T12:47:40Zcache-jfk8127foobarlogs2[39694]:
{"client_ip":"207.237.138.218","req_url":"/"}
Results
Originally hardcoded syslog prefix
Now 4 options:
syslog classic syslog prefix, RFC 3164 (default)
loggly modern syslog structured prefix, RFC 5424
logplex Heroku style prefix
blank no prefix, good for JSON and CSV
Currently available through API — and through the Portal TODAY
See https://docs.fastly.com/guides/streaming-logs/changing-log-line-formats
Prefixes
<134>2017-03-19T13:13:33Zcache-jfk8147foobarlogs2[453796]:{"client_ip":"207.237.138.218","req_url":"/"}
<134>12017-03-19T13:13:33Zcache-jfk8147-453796foobarlogs2-{"client_ip":"207.237.138.218","req_url":"/"}
114<134>12017-03-19T13:13:33+00:00cache-jfk8147-453796foobarlogs2-{"client_ip":"207.237.138.218","req_url":"/"}
{"client_ip":"207.237.138.218","req_url":"/"}
Prefixes
log{"syslog60idOs66l4AbzuqvgBypS2
foobarlogs::"}{"{"client_ip":""}
client.ip{"","req_url":""}req.url
{"""}"}";
Generated VCL
log"syslog60idOs66l4AbzuqvgBypS2foobarlogs::
{"
{""client_ip":""}client.ip{"","}
{""req_url":""}req.url{"""}
"}";
Custom VCL
if(randombool(std.atoi(table.lookup(logging,"percentage"))),100){log{"syslog4rqBj4oy1YChTPgdapiWS4gcs-test::"}var.logstr;}
curl-XPATCH-H"Fastly-Key:<api_token>"-d"item_value=5""https://api.fastly.com/service/<service_id>/dictionary/<dict_id>/item/percentage"
Adjustable percentage
Per URL matching
if(table.lookup(panic_mode_logging,req.url)){
logsyslog4rqBj4oy1YChTPgdapiWS4gcs-test::
}
tablepanic_mode_logging{
“/my_broken_url”:“1”
}
declarelocalvar.logstrSTRING;
setvar.logstr=…;
declarelocalvar.endpointSTRING;setvar.endpoint=randomstr(1,"1234");
if(var.endpoint=="1"){log{"syslog4rqBj4oy1YChTPgdapiWS4gcs-test1::"}var.logstr;}elseif(var.endpoint=="2"){log{"syslog4rqBj4oy1YChTPgdapiWS4gcs-test2::"}var.logstr;}elseif(var.endpoint=="3"){log{"syslog4rqBj4oy1YChTPgdapiWS4gcs-test3::"}var.logstr;}else{/*only"4"left*/log{"syslog4rqBj4oy1YChTPgdapiWS4gcs-test4::"}var.logstr;}
Load spreading
backend default { .host = “127.0.0.1”; .port = “8080”; .connect_timeout = 60s; .first_byte_timeout = 60s; .between_bytes_timeout = 60s; .max_connections = 800;}
acl purge { “127.0.0.1”; “localhost”;}
sub vcl_recv { set req.grace = 2m;
#Set X-Forwarded-For header for logging in nginx remove req.http.X-Forwarded-For; set req.http.X-Forwarded-For = client.ip;
Sample syntax highlighting style guide
Typography - Roboto MonoBlue - actionRed - variables Green - stringOrange - valueGrey - comment
Colors are in keynote