measure your app internals with influxdb and symfony2
TRANSCRIPT
Measure your app internals with InfluxDB(time-series database)Symfony Day 2015 - Reggio Emilia
Trust me
Why we measure things?
Measure - Learn - Build - Measure
Measure for a change
Measure for discovery
Measure for compare
What we need?* A data provider (our application)
* A database that will collects data
* A dashboard for representing information
What is a time-series database?A database optimized for handling time series data:
arrays of numbers indexed by time(a datetime or a datetime range)
sign elem is 4 -> @10:13 elem is 4
A time series is a sequence of data points, measured typically atsuccessive points in time spaced at uniform time intervals
InfluxDBEasy to useWritten in Golang (no external dependencies)ready for bilion of rows (scalable)UDP/IP and HTTP network adaptersSeveral client libraries (PHP, Golang, etc...)Integrated query web panelReady for external dashboards (grafana, chronograf, etc...)
Write information in your InfluxDBcurl XPOST 'http://localhost:8086/write?db=mydb' \ d 'cpu load=2 1434055562000000000'
Read data back from InfluxDBcurl G http://localhost:8086/query?pretty=true dataurlencode "db=mydb" \ dataurlencode "q=SELECT * FROM cpu WHERE host='server01' AND time < now() 1d"
For example:
InfluxDB tagsYou can mark new points with a tag information
sign @10.32 key=value tag1=value tag2=value
sign @10.32 temperature=21.3 location=home env=test
sign @10.32 cpu=21.3,mem=123.3 location=euwest1 env=dev instance=ia91afcd1
Use tags in your queriesExample
SELECT * FROM home_sensors WHERE location="home" AND temperature > 19.2 AND time >= now() 10m
Example for websites:SELECT * FROM index_page WHERE env="prod" AND load_time > 19.2 AND time >= now() 1d
Just use them like string column values
Play from the InfluxDB web consolehttp://your-domain.tld:8083/
GRAFANA DASHBOARD
An open source, feature rich, metrics dashboard and graph editorfor several time series databases (InfluxDB, Graphite, etc.)
Annotate your graphs
Thanks to annotations you can use a serieas an annotation list
Mark a new software release in your graph (Continuous Delivery)Mark any critic event in all your graphs (Information Point)
CHRONOGRAF DASHBOARD
A new data visualization tool for InfluxDB.
How to send my application data?* Send every single point directly in real-time
* Collect data during the application life-cycle and send thosepoints once per request
InfluxDB supports two network adapters
HTTP or UDP/IPWhich one we should have to use?
UDP/IP must be enabled by configuration
HTTP* TCP/IP based
* Data point delivery guaranteed
* Error handling
* ~ 374 Ops/second (for compare with UDP/IP)
UDP/IP* No delivery guarantees
* No error handling
* ~ 22917 Ops/second (for compare with TCP/IP)
HTTP* Sensitive data
* Good for once at a time strategy (or if we do not care about monitor impact on our application)
UDP/IP* Good for immediate delivery
InfluxDB-PHP-SDKhttps://github.com/corley/influxdb-php-sdk
Supports both: HTTP and UDP/IPStar 46 Fork 13
composer require corley/influxdb-sdk
How it works$client>mark("serie_name", [ "customer_name" => "walter", "item_price" => 100.12, //...]);
Concise Format (good for immediate delivery)
How it works (2)$client>mark([ "points" => [ [ "measurement" => "instance", "fields" => [ "cpu" => 18.12, "free" => 712423, ], ], [ "measurement" => "page_load", "fields" => [ "listeners_speed" => 1.12, "counted_elements" => 341143, ], ], ]]);
Extended Format (good for a bulk strategy)
InfluxDB ClientUDP/IP + HTTP
$httpOptions = new Options();$reader = new Http\Reader($httpOptions);$udpOptions = new Options();$writer = new Udp\Writer($udpOptions);$client = new Client($reader, $writer);
HTTP only
$http = new \GuzzleHttp\Client();
$options = new Options();$reader = new Http\Reader($http, $options);$writer = new Http\Writer($http, $options);$client = new Client($reader, $writer);
Supports Guzzle versions: ~4, ~5, ~6
Symfony: Dependency Injection Containerparameters: influxdb_host: "localhost" influxdb_port: 4444
services: influxdb: class: InfluxDB\Client arguments: ["@influxdb_reader", "@influxdb_writer"]
influxdb_writer: class: InfluxDB\Adapter\Writer\Udp arguments: ["@influxdb_udp_options"]
influxdb_reader: class: InfluxDB\Adapter\Http\Reader arguments: ["@influxdb_http_options"]
Symfony: Dependency Injection Container#Only options
influxdb_udp_options: class: InfluxDB\Options calls: ["setHost", ["%influxdb_host%"]] # InfluxDB host ["setPort", ["%influxdb_udp_port%"]] # UDP/IP Port ["setTags", ["env": "%kernel.environment%"]] # tag every point with env
influxdb_http_options: class: InfluxDB\Options calls: ["setHost", ["%influxdb_host%"]] # InfluxDB host ["setPort", ["%influxdb_http_port%"]] # HTTP Port
parameters: influxdb_host: "influxdb.domain.tld" influxdb_udp_port: 4444 influxdb_http_port: 8086
Don't try this at homeclass DefaultController extends Controller public function indexAction(Request $request) $client = $this>container>get("influxdb");
$client>mark(...);
Goals: single-responsibility, reduce coupling, simplifing testing
Controllers as a service?For a monitor? Make sense?
With eventsclass DefaultController extends Controller public function indexAction(Request $request) //... $dispatcher>dispatch("my_event_name", $event);
Event listener
class MyMonitorAwareForMyEventListener public function __construct(Client $client) ...
public function onMyEvent(Event $event) $this>client>mark(...);
Deregister this listener in order to stop data collection
Compose your listenermonitor_listener: class: AppBundle\Listener\MonitorListener arguments: ["@influxdb", "@debug.stopwatch"] tags: name: kernel.event_listener, event: kernel.request, method: startStopwatch name: kernel.event_listener, event: kernel.response, method: stopStopwatch
The stopwatch will help us to collect page timing information
A listener example with stopwatchclass MonitorListener public function startStopwatch(GetResponseEvent $event) $routeName = $event>getRequest()>attributes>get("_route"); $this>stopwatch>start($routeName); public function stopStopwatch(FilterResponseEvent $event) $routeName = $event>getRequest()>attributes>get("_route"); $events = $this>stopwatch>stop($routeName); foreach ($events>getPeriods() as $measure)
$this>client>mark($routeName, [ "memory" => $measure>getMemory(), //... ]);
Check your data via InfluxDB
Prepare your dashboard with grafana
Checkout out the app examplehttps://github.com/wdalmut/symfonyday2015-example
Thanksgithub.com/wdalmut