max: realtime messaging and activity stream engine

Activity stream and conversation engine

(MAX)Víctor Fernández de Alba (@sneridagh)

Carles Bruguera (@sunbit)

MAX is a project founded by @UPCnet and @BarcelonaTech

History• First commit on August, 2011

• Initially designed as the key feature for the BarcelonaTech university concept of social intranet

• The university’s social intranet concept is on the IT roadmap since 2007 and it has been developed with limited resources since then

Social intranet concept

Activity StreamActivity

CommentsLikes

FavoritesImages

Files

Conversations

1to1GroupImages

Files

Realtime using STOMP RabbitMQ plugin

Push notifications

iOS

Android

External sources aggregator

Twitter

AuthenticationoAuth 2.0

Resource Owner Password Credentials flow (https://pypi.python.org/pypi/osiris)

LDAPand other pluggable

auth backends

Activity stream• Stores activity from

users and applications

• The applications can impersonate users to feed the stream with useful information

oAuth server

API

MongoDB

Context oriented

• Subscriptions are made against contexts

• A context is something with an unique URL

• An user can have n subscriptions to contexts

• The user timeline is composed of all the activity generated in these contexts

Conversations

• Realtime conversations and private messaging

• Support for sending images and files

oAuth server

API (WSGI client)

MongoDB

RabbitMQ STOMP

Conversation queues

InfrastructureoAuth server

API

MongoDB RabbitMQ

Queues

push Twitter

conversations

Consumers

Frontends

MAXUI.js

iOS

Android

MAX API

88 RESTfulendpoints

> 600 tests and 96% coverage

MAX API

Pyramid powered

gevent optimized

BackendsMongoDB

RabbitMQ

General storage

Realtime conversation backendProcessing queues

Queues and consumers• Designed for huge loads (conversations specially)

• RabbitMQ queues

• gevent consumers with WSGI API client

• Push (iOS and Android notifications)

• Twitter streaming traking consumers for contexts and users

• Conversations

FrontendsMAXui.js

• jQuery, no framework but using modern JS development good practices

• Can be instantiated virtually in any web site given the username and oAuth token and MAX server

• Stomp.js (STOMP over websockets) powered

• Native apps

• Stream, conversations, subscriptions

• Support for push notifications

FrontendsiOS & Android

Performance4000 concurrent users

100 messages/sec

200 conversations 20 users/conversation

1 message every 2 seconds

Plone integration

What’s next?

• Follow people and activity publishing not related to any particular context

• Sharing

• Improve documentation!!

Resources

https://upcnet.github.io/max

https://github.com/UPCnet/max

https://github.com/UPCnet/maxserver