Ømq - an introduction
TRANSCRIPT
“We took a normal TCP socket, injected it with a mix of radioactive
isotopes stolen from a secret Soviet atomic research project,
bombarded it with 1950-era cosmic rays, and put it into the hands
of a drug-addled comic book author with a badly-disguised fetish
for bulging muscles clad in spandex. Yes, ZeroMQ sockets are the
world-saving superheroes of the networking world.”
History HOW IT BEGAN
The original Ø meant:
• Zero Broker
• Zero Latency (as close as possible)
With time it also started meaning:
• Zero Administration
• Zero Cost
• Zero Waste
The Real History WHAT Ø STANDS FOR
Let’s build a simple lock-stepped
Client Server communication.
Lock-Stepped Communication:
Each end of the communication sends a message in turn
REQ - REP A BASIC EXAMPLE
Context ctx = ZMQ.context(1);
Socket socket = ctx.socket(ZMQ.REP);
socket.bind("tcp://*:5555");
while (true) {
byte[] req = socket.recv();
socket.send(req);
}
REQ - REP A BASIC EXAMPLE (2)
Context ctx = ZMQ.context(1);
Socket socket = ctx.socket(ZMQ.REQ);
socket.connect("tcp://localhost:5555");
socket.send("hello".getBytes());
byte[] rep = socket.recv();
REQ - REP
A BASIC EXAMPLE (3)
In ZeroMQ all the IO operations are done in the background by a thread pool.
A Context groups a set of sockets which share the same pool.
ZMQ.Context ctx = ZMQ.context(1);
The parameter of the ZMQ.context Factory is the number of threads active in the thread-pool. (1 is a good default for most of the cases)
Context HOW IS IO HANDLED?
In ZeroMQ Sockets are logical endpoints of a communication system. They abstract different physical connections and protocols.
Socket socket = ctx.socket(ZMQ.REP);
They can be of different types:
Socket HOW TO SEND AND RECEIVE DATA?
• PUB
• SUB
• REQ
• REP
• ROUTER
• DEALER
• PUSH
• PULL
• PAIR
ZeroMQ socket types are agnostic w.r.t. the endpoint responsible of initiating the communication, any socket type can be a TCP Server or Client.
One socket should bind to one (or more) address.
socket.bind("tcp://*:5555");
The other should connect to it (or them).
socket.connect("tcp://localhost:5555");
General role of thumb the endpoint which is “static” should bind the “dynamic” ones should connect.
Binding vs Connecting WHO IS THE SERVER?
Each ZeroMQ socket type pair has its own rules
related to sending and receiving data. Different type
pairs allow developers to achieve different result.
The API to send and receive, though, are independent
from the socket type.
byte[] data = socket.recv();
socket.send(data);
Sending vs Receiving HOW DO THEY COMMUNICATE?
ZeroMQ is a message based system even though it
can use stream based transport protocol like TCP.
The message on the wire follows a simple format:
• Length
• Content
Packet Format HOW ARE THE DATA OF THE WIRE?
ZeroMQ packets can contain more than one Frame
each one following the [length | content] format.
Here is the default message envelope.
Frames & Envelops CAN WE SEND DIFFERENT PARTS IN THE SAME MESSAGE?
A ZMsg is convenient Java Class which hides away the complexity of decoding and the message envelope.
ZMsg msg = ZMsg.recvMsg(socket);
msg.send(socket);
Frames are managed as a stack and they can be pushed/popped in/from the message.
Byte[] frame= msg.pop();
msg.push(frame);
ZMsg IS THERE SOME HELP IN MANAGING COMPLEXITY?
ZeroMQ sockets identify each connected endpoint with
a unique identifier called Identity.
The message envelope can be extended to contain
one (or more) Identities.
They are stored before the “empty delimiter frame”.
Identifies HOW CAN WE IDENTIFY THE ORIGIN OF A PACKET?
We will focus on Request Response pairs.
The Valid Combinations are the following:
• REQ to REP
• DEALER to REP
• REQ to ROUTER
• DEALER to ROUTER
• DEALER to DEALER
• ROUTER to ROUTER
Valid Socket Combinations HOW CAN WE PUT BUILDING BLOCKS TOGETHER?
REQ Sockets are synchronous.
After connection/binding they can just send a
message, trying to receive in this state will produce an
exception.
Once one (and only one) message has been sent, they
can just receive a message, trying to send in this state
will produce an exception.
Once one (and only one) message has been received,
the sockets go back to its initial state.
REQ REQUEST AND WAIT
REP Sockets are synchronous.
After connection/binding they can just receive a message,
trying to send in this state will produce an exception.
The Identity of the source endpoint is stored into an
internal state and not exposed to user code.
Once one (and only one) message has been received, the
sockets can just send a message, which will be sent to the
stored Identity.
Once one (and only one) message has been sent, the
sockets go back to its initial state.
REP WAIT AND REPLY
DEALER Sockets are asynchronous.
After connection/binding they can send as many
messages as necessary.
Any number of messages can also be received, no
identity though is attached to them.
DEALER REQUEST AND FORGET
ROUTER Sockets are asynchronous.
After connection/binding they can receive many messages.
Each message is enriched with and extra frame containing the
Identity of the source endpoint.
These sockets can send messages just to other endpoints from
which they received messages.
The messages which are going to be sent MUST contain an
initial frame with the Identity of the target endpoint.
ROUTER ENREACH THE MESSAGE
DEALER – ROUTER Socket Pairs can be used to orchestrate a classic Client - Server communication channels with one (or more) clients and one servers.
Once the DEALER sends a message it can start waiting for answers. The single server architecture guarantees the origin of the message.
DEALER - ROUTER ASYNCHRONOUS CLIENT SERVER COMMUNICATION
Once the ROUTER receives a message it can store the Identity and use it to forward the message back to the correct endpoint when the answer is ready. Meanwhile it can go on waiting for other requests.
Download Source Code
https://github.com/zeromq/jeromq
Download the JAR file
http://central.maven.org/maven2/org/zeromq/jeromq
JeroMQ A PURE JAVA IMPLEMENTATION
Let’s open our IDEs
DEMO TIME
PRACTICE NEVER HURTS
• http://zguide.zeromq.org
• https://github.com/zeromq/jeromq
Reference