Ømq - an introduction

MILANO 1863 POLITECNICO

ØMQ - ZeroMQ

AN INTRODUCTION

Carlo Bernaschina – carlo.bernaschina@polimi.it

“We took a normal TCP socket, injected it with a mix of radioactive

isotopes stolen from a secret Soviet atomic research project,

bombarded it with 1950-era cosmic rays, and put it into the hands

of a drug-addled comic book author with a badly-disguised fetish

for bulging muscles clad in spandex. Yes, ZeroMQ sockets are the

world-saving superheroes of the networking world.”

History HOW IT BEGAN

The original Ø meant:

• Zero Broker

• Zero Latency (as close as possible)

With time it also started meaning:

• Zero Administration

• Zero Cost

• Zero Waste

The Real History WHAT Ø STANDS FOR

Let’s build a simple lock-stepped

Client Server communication.

Lock-Stepped Communication:

Each end of the communication sends a message in turn

REQ - REP A BASIC EXAMPLE

Context ctx = ZMQ.context(1);

Socket socket = ctx.socket(ZMQ.REP);

socket.bind("tcp://*:5555");

while (true) {

byte[] req = socket.recv();

socket.send(req);

}

REQ - REP A BASIC EXAMPLE (2)

Context ctx = ZMQ.context(1);

Socket socket = ctx.socket(ZMQ.REQ);

socket.connect("tcp://localhost:5555");

socket.send("hello".getBytes());

byte[] rep = socket.recv();

REQ - REP

A BASIC EXAMPLE (3)

In ZeroMQ all the IO operations are done in the background by a thread pool.

A Context groups a set of sockets which share the same pool.

ZMQ.Context ctx = ZMQ.context(1);

The parameter of the ZMQ.context Factory is the number of threads active in the thread-pool. (1 is a good default for most of the cases)

Context HOW IS IO HANDLED?

In ZeroMQ Sockets are logical endpoints of a communication system. They abstract different physical connections and protocols.

Socket socket = ctx.socket(ZMQ.REP);

They can be of different types:

Socket HOW TO SEND AND RECEIVE DATA?

• PUB

• SUB

• REQ

• REP

• ROUTER

• DEALER

• PUSH

• PULL

• PAIR

ZeroMQ socket types are agnostic w.r.t. the endpoint responsible of initiating the communication, any socket type can be a TCP Server or Client.

One socket should bind to one (or more) address.

socket.bind("tcp://*:5555");

The other should connect to it (or them).

socket.connect("tcp://localhost:5555");

General role of thumb the endpoint which is “static” should bind the “dynamic” ones should connect.

Binding vs Connecting WHO IS THE SERVER?

Each ZeroMQ socket type pair has its own rules

related to sending and receiving data. Different type

pairs allow developers to achieve different result.

The API to send and receive, though, are independent

from the socket type.

byte[] data = socket.recv();

socket.send(data);

Sending vs Receiving HOW DO THEY COMMUNICATE?

ZeroMQ is a message based system even though it

can use stream based transport protocol like TCP.

The message on the wire follows a simple format:

• Length

• Content

Packet Format HOW ARE THE DATA OF THE WIRE?

ZeroMQ packets can contain more than one Frame

each one following the [length | content] format.

Here is the default message envelope.

Frames & Envelops CAN WE SEND DIFFERENT PARTS IN THE SAME MESSAGE?

A ZMsg is convenient Java Class which hides away the complexity of decoding and the message envelope.

ZMsg msg = ZMsg.recvMsg(socket);

msg.send(socket);

Frames are managed as a stack and they can be pushed/popped in/from the message.

Byte[] frame= msg.pop();

msg.push(frame);

ZMsg IS THERE SOME HELP IN MANAGING COMPLEXITY?

ZeroMQ sockets identify each connected endpoint with

a unique identifier called Identity.

The message envelope can be extended to contain

one (or more) Identities.

They are stored before the “empty delimiter frame”.

Identifies HOW CAN WE IDENTIFY THE ORIGIN OF A PACKET?

We will focus on Request Response pairs.

The Valid Combinations are the following:

• REQ to REP

• DEALER to REP

• REQ to ROUTER

• DEALER to ROUTER

• DEALER to DEALER

• ROUTER to ROUTER

Valid Socket Combinations HOW CAN WE PUT BUILDING BLOCKS TOGETHER?

REQ Sockets are synchronous.

After connection/binding they can just send a

message, trying to receive in this state will produce an

exception.

Once one (and only one) message has been sent, they

can just receive a message, trying to send in this state

will produce an exception.

Once one (and only one) message has been received,

the sockets go back to its initial state.

REQ REQUEST AND WAIT

REP Sockets are synchronous.

After connection/binding they can just receive a message,

trying to send in this state will produce an exception.

The Identity of the source endpoint is stored into an

internal state and not exposed to user code.

Once one (and only one) message has been received, the

sockets can just send a message, which will be sent to the

stored Identity.

Once one (and only one) message has been sent, the

sockets go back to its initial state.

REP WAIT AND REPLY

DEALER Sockets are asynchronous.

After connection/binding they can send as many

messages as necessary.

Any number of messages can also be received, no

identity though is attached to them.

DEALER REQUEST AND FORGET

ROUTER Sockets are asynchronous.

After connection/binding they can receive many messages.

Each message is enriched with and extra frame containing the

Identity of the source endpoint.

These sockets can send messages just to other endpoints from

which they received messages.

The messages which are going to be sent MUST contain an

initial frame with the Identity of the target endpoint.

ROUTER ENREACH THE MESSAGE

DEALER – ROUTER Socket Pairs can be used to orchestrate a classic Client - Server communication channels with one (or more) clients and one servers.

Once the DEALER sends a message it can start waiting for answers. The single server architecture guarantees the origin of the message.

DEALER - ROUTER ASYNCHRONOUS CLIENT SERVER COMMUNICATION

Once the ROUTER receives a message it can store the Identity and use it to forward the message back to the correct endpoint when the answer is ready. Meanwhile it can go on waiting for other requests.

Download Source Code

https://github.com/zeromq/jeromq

Download the JAR file

http://central.maven.org/maven2/org/zeromq/jeromq

JeroMQ A PURE JAVA IMPLEMENTATION

Let’s open our IDEs

DEMO TIME

PRACTICE NEVER HURTS

• http://zguide.zeromq.org

• https://github.com/zeromq/jeromq

Reference