building observable applications w/ node.js -- baynode meetup, march 2014
Post on 14-Jul-2015
1.226 Views
Preview:
TRANSCRIPT
Observable Applications w/ Node.js
Yunong Xiao, Senior Node.js Engineer, UI Platform, Netflix
@yunongx, yunong@netflix.com
March 2015BayNode The Bay Area Node.js Meetup
Node.js @ Netflix
• Website www.netflix.com
• Discovery
• Playback
• Acquisition
• Account Management
• Internal Services
• TV/Mobile/Devices
What Now?
“It is a capital mistake to theorize before one has DATA. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts.”
Sherlock Holmes
-A Scandal in Bohemia
vasync
• https://github.com/davepacheco/node-vasync
• Similar to caolan/async.
• Added observability.
vasync
• Make API requests.
• Persist state to file system.
• Query database.
• Update caches.
Async workflow management is error prone
node-bunyan
• https://github.com/trentm/node-bunyan
• Streaming JSON logging library for JS. (node and browser)
Streaming JSON
• One JSON object per line. e.g. Twitter’s streaming API.
• Perfect for machine processing. i.e. works with Unix tools such as grep(1), cut(1), …
• Illegible for Humans
Features
• Lightweight API
• Log levels: trace, debug, info, warn, error, fatal
• Extensible Streams interface.
• Custom object rendering with serializers.
• DTrace support.
DTrace
• Dynamic tracing framework.
• Available on Mac OS X (dev env). `man dtrace`
• Node.js DTrace USDT provider: https://github.com/chrisa/node-dtrace-provider
• Pure JS probes.
Production Tracing• Production environment is Ubuntu Linux on EC2.
• Future work: get probe access. Possibilities:
• SystemTap
• LTTng (Linux Tracing Toolkit Next Gen)
• perf_events markers (currently a proposed patch)
restify• http://restifyjs.com
• Production tested REST framework.
• Observability.
• Metrics.
• First class Bunyan integration.
• DTrace.
Request Capture Stream
• Tradeoffs, Perf vs Verbosity.
• Captures all log statements including trace in memory.
• Dump all logs only on error.
Audit Logsstatus code request ID (UUID)URL
request headers
response headers
individual handler timers
req latency
Logging
• Native Bunyan integration.
• Request capture stream.
• Audit logs with each req.
• Scoped child loggers with each req.
restify + Bunyan• Streaming JSON.
• Processing using Unix tools is easy.
• Helpful tools:
• Unix: cut(1), wc(1), grep(1), awk(1), perl(1), …
• JSON: https://github.com/trentm/json (npm install -g json)
• daggr: https://github.com/joyent/daggr (npm install -g daggr)
Examples
• Find all logs for a specific request.
• Count the # of non-200 responses.
• Show all requests that took longer than 200ms.
distributed processing
• elasticsearch • spark • hive
bunyanrestify
audit logs
req id req headers
req latency
handler latenciesURL
res codeErrors
res headers
Processing w/ Unix tools
• grep(1) • awk(1) • cut(1) • wc(1) • sort(1) • json(1) • bunyan(1) • …
req scoped logs application specific
context
req capture stream dumps all logs on error
DTrace*req latencyhandler
latencies
* Where available
restify metrics
The Killer Combo• vasync: https://github.com/davepacheco/node-vasync
• Observable async operations.
• bunyan: https://github.com/trentm/node-bunyan
• Streaming JSON logs.
• restify: restifyjs.com
• Observable REST applications.
• Unix Tools
• Stream of text model. Many tools to manipulate JSON logs.
Thanks
• Questions?
• yunong@netflix.com
• @yunongx
• http://restifyjs.com
March 2015BayNode The Bay Area Node.js Meetup
top related