1/36

Communication

server

client

rpc

communication

rmiremote

class

procedure

stub

messagemulticast network object

calls

implementations

program

xdr

asynchronous

datadistributed

request

system

address

copydeclarations

interface java

parameters passing

protocol

big-endian

code

language little-

endian machine method

result running send

service skeleton start

array base bytes

defined

operations

pointers

return rmiregistry

2/36

Overview

I Communication types and role of MiddlewareI Remote Procedure Call (RPC)I Message Oriented CommunicationI Multicasting

3/36

Communication Types

I Transient communication. A message is stored by the communicationmiddleware as long as the sending and receiving applications areexecuting.

I Persistent communication. A message that has been submitted fortransmission is stored by the communication middleware for as long asit takes to deliver it to the receiver. E.g. Email.

I Synchronous communication. The sender is blocked until its request isknown to be accepted. This can happen at three places.

I The sender may be blocked until the middleware notifies that it willtake over the transmission of the message.

I The sender may be blocked until the request has been delivered to theintended recipient.

I The sender waits until its request has been fully processed, that is, upto the time that the recipient returns a response.

I Asynchronous communication. The sender continues immediatelyafter it has submitted its message for transmission. This means thatthe message is (temporarily) stored immediately by the middlewareupon transmission.

4/36

Role of Middleware

5/36

Remote Procedure Call (RPC)

I A remote procedure call (RPC) allows a program totransparently call procedures located on another machine. Nomessage passing is visible to the programmer.

I A widely used technique that underlies many distributedsystems.

6/36

Conventional Procedure Call

count = read(fd, buf, nbytes); //in main

7/36

Parameter Passing

I Call-by-value: The variable is copied to the stack. The originalvalue is left unchanged

I Call-by-reference: The address of the variable is copied to thestack so the called procedure modifies the same copy as theprocedure calling

I Call-by-copy/restore: The variable is copied to the stack bythe caller and then copied back after the call, overwriting thecaller’s original value

I In-class Exercise. How is call-by-copy/restore different fromcall-by-reference? Give a concrete example.

8/36

Basic Principles of RPC

I Transparency is achieved by using a client stub and a serverstub.

I A client stub is a local library procedure that translates the callinto a request to the server and sets up the results as a returnto the call from the client.

I A server stub is a procedure (local to the server) thattransforms requests coming over the network into localprocedure calls.

9/36

Remote Procedure Call: Step by Step (1)

I The client procedure calls the client stub in the normal way.I The client stub builds a message and calls the local operating

system.I The client’s OS sends the message to the remote OS.I The remote OS gives the message to the server stub.I The server stub unpacks the parameters and calls the server.

10/36

Remote Procedure Call: Step by Step (2)

I The server does the work and returns the result to the serverstub.

I The server stub packs it in a message and calls it’s local OS.I The server’s OS sends the message to the client’s OS.I The client’s OS gives the message to the client stub.I The stub unpacks the result and returns to the client.

11/36

Parameter Passing in RPC

Passing parameters to RPCs can be tricky since it requires packingparameters into a message (parameter marshaling) to be interpretedcorrectly on another system (unmarshaling). Here are some issues:

I Machines may use different character codes.I Different representation of integers and floating-point numbersI Different byte addressing: little-endian versus big-endian

formatI How to pass reference parameters (like pointers) since an

address will be meaningless on another system? Can we usecall-by-copy/restore?

I How to pack types that take less space than a word? Forexample: a short or a char.

I How to pack an array?I How to pack an object?

12/36

RPC: Passing Parameters

13/36

Representation: Little-Endian vs Big-Endian (1)

I Little-endian machine: the bytes in a word are numbered rightto left. E.g. Intel processors.

I Big-endian machine: the bytes in a word are numbered left toright. E.g. IBM z/Architecture.

I Protocols like IPv4, IPv6, TCP and UDP use big-endian so itis also known as the network byte order.

I Bi-endian machine: Choose either setting via software orhardware. E.g. ARM processors, SPARC, POWER PC.

14/36

Representation: Little-Endian vs Big-Endian (2)

a. Little endian (data before sending)b. Big endian (data after receiving) What is the integer 5 read as

on the big-endian machine?c. Big endian (after reversing bytes on received data)

Does reversing the bytes fix the problem for all data types?

15/36

Representation: Little-Endian vs Big-Endian (3)

I A C program to determine the endianness of the underlyingsystem. See example byteorder.c in examples/sockets-C/miscfolder.

I In-class Exercise. Java uses big-endian representation in theJVM. Do we need to worry about endianness for server andclients written in Java? What if it is running on a litte-endianmachine? What if they deal with files written on a machinewith different endianness?

16/36

Passing Reference Parameters

I Passing an array or simple structure by reference: Copy thearray into the message. The server stub then calls the serverwith a pointer to this copy. Changes to the array happen inthe message buffer in the server stub that can then send backto the client stub, which then copies it back to the client.

I Replaces call-by-reference with call-by-copy/restore. Works inmost cases.

I How about arbitrary data structures with pointers?

17/36

RPC Protocol and Stub Generation

I The RPC protocol would have to define the format of the message,the representation of primitive types and arrays and other datastructures. Are integers stored in 2’s complement, characters in 16-bitUnicode, floats/doubles in IEEE standard #754 and if everything isbig-endian or little-endian?

I An example:

:I Interface Definition Language (IDL) is used to define interfaces for

RPC. IDL is then compiled into client and server stub along with theappropriate compile/run-time interfaces.

18/36

Asynchronous RPC (1)

I Asynchronous RPC: A client immediately continues once theserver accepts the RPC request. The server executes the RPCrequest after the acknowledgment.

I Deferred asynchronous RPC: The client calls the server with aRPC request and the server immediately acknowledges it.Later the server does a callback to the client with the result.

19/36

Asynchronous RPC (2)

I Synchronous (default choice) versus Asynchronous RPC

I Deferred Asynchronous RPC

20/36

Classic RPC Implementations

I Distributed Computing Environment/Remote Procedure Calls(DCE/RPC) was commissioned by the Open SoftwareFoundation, an industry consortium now renamed as the OpenGroup. The DCE/RPC specifications were adopted inMicrosoft’s base for distributed computing, DCOM.

I Open Network Computing Remote Procedure Call (ONC RPC)is a widely deployed remote procedure call system. ONC wasoriginally developed by Sun Microsystems as part of theirNetwork File System project, and is sometimes referred to asSun ONC or Sun RPC.

21/36

Building a RPC Server and Client

22/36

Binding a Client to a RPC Server

Note: ONC RPC uses a rpcbind in place of the DCE daemon. Itdoesn’t use a directory server.

23/36

Issues with RPCs

How to deal with network and system errors?

I At-most-once operation: No call is carried out more than onceeven if the system crashes.

I At-least-once operation: The call is tried multiple times until itsucceeds. It is possible for the call to happen multiple times.

I This only works for idempotent operations: something thatcan be done multiple times without harm.

24/36

More RPC Implementations

I Java RMI (Remote Method Invocation).I XML-RPC and its successor SOAP: An RPC protocol that uses

XML to encode its calls and HTTP as a transport mechanism.I Microsoft .NET Remoting offers RPC/RMI facilities for

distributed systems implemented on the Windows platform.I CORBA provides remote procedure invocation through an

intermediate layer called the object request broker.I Facebook’s Thrift protocol and framework.I Google Protocol Buffers combined with gRPC.I Apache Avro is a RPC and data serialization framework

developed for the Apache Hadoop project.

25/36

Message-Oriented Persistent Communication

I Message-Oriented Middleware (MOM) or message-queuing systemssupport persistent asynchronous communication that is looselycoupled and reliable.

I These systems provide intermediate-term storage capacity formessages without requiring either the sender or receiver to be activeduring the message transmission.

I Examples: IBM WebSphere MQ, Oracle AQ, Microsoft MessagingQueue, Amazon Simple Queue Service, JBoss, Apache Active MQ.Part of Erlang/Elixir, local versions available in most operatingsystems.

I Java Messaging Service (JMS) is a Java Message Oriented Middleware(MOM) API for sending messages between two or more clients. It is amessaging standard that allows application components based on theJava Enterprise Edition (Java EE) to create, send, receive, and readmessages.

I Providers of JMS: Amazon SQS, Apache AcitveMQ, Oracle AQ, JBoss,IBM WebSphere MQ and several others.

26/36

Message Queuing Model (1)

I Four combinations of loosely-coupled communications usingqueues.

27/36

Message Queuing Model (2)

I Basic interface to a queue in a message queuing system.

28/36

Message Queuing System Architecture (1)

Queue-level and Network-level Addressing

29/36

Message Queuing System Architecture (2)

Message Queue System with Routers

30/36

Message Brokers

Message broker: An application-level gateway that convertsincoming messages so that they can be understood by thedestination application.

31/36

Message Queuing Systems

I Message queuing systems can be used to implement email,workflow, groupware and batch processing.

I Enterprise Application Integration (EAI): the integration of acollection of databases and applications into a federatedsystem.

I The broker is responsible for matching applications based onmessages that are being exchanged.

I Uses a publish/subscribe model.I The broker uses a repository of rules and programs that can

transform a message of type T1 into one of type T2.

32/36

Multicast

I A multicast is a message that is delivered to multiple listenerson multiple systems simultaneously. It can be much moreefficient than having to send point-to-point messages.

I A broadcast is a message that is delivered to all listeners on alocal area network and is a special case of multicasting.

I Multicasting requires support from networking hardware suchas routers.

I The most common implementation is IP Multicast, used forstreaming media. No prior knowledge of who or how manyreceivers there are is required. Widely used in enterprises,stock exchanges and multimedia content delivery networks.

33/36

IP Multicast

I IP Multicast addresses have the leading four bits as 1110. Thus theprefix for this group of addresses is 224.0.0.0/4.

I The addresses in the range 224.0.0.0 to 239.255.255.255 are reservedfor multicast addresses.

I Some reserved IPv4 multicast addresses:

224.0.0.0 reserved base address224.0.0.1 all hosts on the same network segment224.0.0.2 all routers on the same network segment224.0.0.251 multicast DNS address224.0.1.1 multicast Network Time Protocol address

I The most common implementation is using UDP (User DataGramProtocol), which isn’t reliable — messages may be lost or deliveredout of order.

34/36

Java Examples

The main class we will use is java.net.MulticastSocket. Seeexamples in examples/multicasting

I ex0-setup: Shows how to find out information about networkinterfaces and if they support multicasting.

I ex1-mcast-hello: Streaming hello world using multicasting!I ex2-mcast-time: Multicast time serverI ex3-mcast-group: Multicast group membership example

35/36

Exercises

I Assume that a client calls an asynchronous RPC to a server,and subsequently waits until the server returns a result usinganother asynchronous RPC. Is this approach the same asletting the client execute a normal RPC? What if we replacethe asynchronous RPC with a synchronous RPC?

I Study the example rmi/ex7-PassingArgsInRMI to see howparameter passing happens in RMI calls. Is it call bycopy-restore?

I Write a example server/client program that copies a largeamount of data (say 500MB) from the server to eightclients.Then write another example that does the same thingusing a MulticastSocket. Compare the time required totransmit the data to all clients.

36/36

References

I Wikipedia article on EndiannessI Linus Torvalds on EndiannessI Multicast AddressI IP Multicast