REAL-TIME SPORTS STATISTICSWITH NODE.JS, WEBSOCKETS AND SYMFONY2

#42 TOBIAS JOSEFSSON

Chief Technology Officer”Bobbe” / ”Bobby”@TobiasJosefsson

#8 EMANUEL WINBLAD

Soware Engineer”Manne” / ”Manny”

@ManneW

SPORTS EDITING SWEDENA MEDIA HOUSE FOCUSED ON SPORTS

Overview: Data flow from Operators to Clients

Overview: Data flow from Operators to Clients

Why this complex setup?

”Independent” decoupled parts- Easier to test- Easier to maintain / replace- Each part only does one thing

Multiple languages- There is no silver bullet

Messaging: ZeroMQ

"ZeroMQ is a messaging library, which allows you to design a complex communication system without much effort."

– Nicholas Piël

”Messaging makes everything possible!”– Alvaro Videla, JSDay 2013 (yesterday)

Messaging: ZeroMQ

RabbitMQ ZeroMQBroker Brokerless

Messaging System

Messaging Library

Why:- Lightweight- Minimalistic- Small feature set needed

Messaging: ZeroMQ

Core pattern used:

Overview: Statistics input

Statistics input: External system

Seven Operators per arena, logging events.

- Shots on net, shots wide, shots blocked- Goals and assists- Players currently on the ice, shifts- Penalties- Hits- ...and much more...

Data is transferred using XML files and http(s).

Overview: Parser

Parser: PHP, mongoDBPHP: Familiar, XML processingDoctrine ODM to share documentswith CMS system (Symfony2).

MongoDB: Different attributes/fields depending on event type.

Shot Goal PenaltyPlayer Player Player

Goal section Goal section

Assists

POP, NEP (players on ice)

Penalty reason

Parser: PHP, mongoDB

Possible improvements (when needed):Scale - one parser process per game

(only changes in this component)

Alternatives (PHP based):- Symfony2 (full-stack)- Silex (micro)…

Overview: Editorial input

Editorial input: Symfony2, MySQL

Multiple Operators located in different places

- Live text comments describing the events- Data tagged with games and teams for filtering

- Flexible, adjustable, custom made functionality- CMS also includes REST-ish API used by clients

Overview: Queue Controller

Queue Controller: PHP

Decides what messages that should go where.

Isn’t this a broker!?Yes, kind of…

- Decides which queue(s) to send to.- No information about actual end points.

How?

Queue Controller: PHP// Prepare our context and socket$context = new ZMQContext(); // Subscribe to messages from parser$parser = $context->getSocket(ZMQ::SOCKET_SUB); // Connect to publishing socket$parser->connect('tcp://192.168.0.11:9900');

// Define the channel(s) for// delivery servers to subscribe to$channel = $context->getSocket(ZMQ::SOCKET_PUB); // Bind each channel to an IP and port$channel->bind('tcp://192.168.0.12:5555');

Queue Controller: PHP

// All in an infinite loop// ...// Receive messages from the parser$msg = $parser->recv(); // Which channels will the message go to$recievingChannels = getChannelsForMsg($msg); // Send message to appropriate channelsforeach ($recievingChannels as $channel) {    $channel->send($msg);}

Overview: Delivery

Delivery: Node.js, SockJS

Why Node.JS?- High concurrency- Same language as client

Why SockJS?- Lightweight, but still fallbacks for Web Sockets- Option to get ”raw” sockets

Alternatives:- socket.io (or simply polling...)

http://bit.ly/143An2a

NodeJS server code

var server = sockjs.createServer();    server.on(‘connection’, function(conn) {    conn.on(‘data’, function(msg) {        // Check the action passed        switch (msg.action) {         // ...         // Do the corresponding thing...         // I.e. let the client join a channel         // ...        }    });});

// Game event message received from Queue Controllervar gameEventMessage = ”blablabla”; // Send a message to a clientvar conn = getNextClient(clients); conn.write(gameEventMessage);

NodeJS server code (cont.)

Delivery: Node.js, SockJSSockJS client code

var sockjs = new SockJS('http://192.168.0.13:8080');sockjs.onopen = function() {    // SockJS connection opened}; sockjs.onmessage = function(e) {    // Message recieved    var msg = JSON.parse(e.data);}; sockjs.onclose = function() {  // Connection closed  // Offer the user possibility to reconnect    $('button#reconnect').show();};

Overview: Monitoring and Logging

Overview: Monitoring and Logging

Overview: Clients

Several versions- Web- Mobile version- Mobile version for arenas (using local WiFi)

The different versions share a lot of functionality.

Clients: Symfony2, HTML5, JavaScript

Clients: Desktop version

Clients: Mobile versions

What have we learned?

- Good to keep things separated

- Don’t over engineer the solution!

But......by keeping things separated you can re-engineer later – when needed!

- Use the tools you know how to handle

- Select the most appropriate tool from your own toolchain

SoftwareZeroMQ: http://www.zeromq.orgNodeJS: http://nodejs.orgSockJS: https://github.com/sockjs

ExamplesWeb: http://www.hockeyligan.seMobile: http://m.hockeyligan.se

REFERENCES AND RESOURCES