ctp

DivConq Framework’s CTP

A file and message transfer protocol that transcends the network protocol.

• Common Transfer Protocol provides a complete spec for messaging

commands as well as file transfers (SIGN ON, DIR, GET, PUT)

• It is abstracted from network layer, could be implemented in TCP, UDP or

through either HTTP or Web Socket

• It does not assume full duplex communications

• It does not (necessarily) associate network layer client credentials with the file

transfer, which can lead to scalability and efficiency issues

• It is platform and language neutral

• Server could be implemented in Java, C, Python or similarly suitable

languages

• Client can be implemented in any language with TCP or HTTP capabilities

• It is open and free

Client Server

CTP Session provides the desired abstraction by living just outside of the network

protocol. A CTP Session represents one authenticated user, it can accept duplex

or half-duplex interactions and otherwise doesn’t care about the network layer.

The CTP Service processes the commands sent to the session and provides

support for the file systems the server exposes.

CTP Session

CTP Service

Network

Layer

App

CTP Session

Connector

Client Server

The CTP Session Connector may be JavaScript in a web page, a Python or Java

library, or otherwise. It keeps track of session context for all calls to the server.

The session connector may be paired with a full-duplex or a half-duplex

networking library. To the App developer the API calls will look pretty much the

same either way (with a few exceptions for straight HTTP support).

CTP Session

CTP Service

Network

Layer

App

CTP Session

Connector

The past has many examples of file transfer protocols.

• FTP – commands and network completely intertwined

• SFTP – the file transfer protocol *is* separate from the network protocol, a

good model but

• Effectively stuck on TCP/IP because of SSH

• Assumes full duplex communications

• Associates the communications client credentials with the file transfer,

which can lead to scalability issues

• Can be challenging to debug/trace, especially if custom extensions are

present

• HTTP POST/GET – commands and network not separated

• Existing accelerated transfer protocols – commands and network not

separated

No existing candidates for a Common Transfer Protocol, but a lot to learn from all

of these.

DivConq CTP

Client 1 Server

Session Connectors paired with networking

protocols. A single server can listen for different

networking protocols on different ports – but

again CTP Session and CTP Service can be

mostly ignorant of the network protocol.

CTP Session

CTP Service

HTTPS

Port 443 App CTP Session

Connector

Client 2

App CTP Session

Connector

CTP Session

CTP Session

Client 3

App CTP Session

Connector

UDP

Port 2020

HT

TP

U

DP

WebSocket

Web Socket

Port 2022

UD

P

HT

TP

WSS

Server

Full duplex communications allow for much more efficient use of a TCP

connection. Traditionally (think FTP or HTTP) we follow a send request then

wait for response (half-duplex) approach to issuing commands. If you just

send a request and assume a response will come back when it is ready, then

you can keep sending more commands and data blocks.

Because of this efficiency, and because we do not tie a user to the network

connection, we could be servicing 15 “users” (Sessions) on the client on as

little as 3 TCP connections.

CTP Service

Client 2

15 App Threads

15 CTP Session

Connectors 15 CTP Sessions

TCP

Port 2021

TC

P T

CP

3 Connections

Server

Though not limited to only these options, the design of CTP has support the

following options within its design:

• HTTP/HTTPS – henceforth called H(yperText)CTP

• WS/WSS – henceforth called W(ebSocket)CTP [using WS Binary mode]

• TCP - henceforth called S(tandard)CTP

• UDP - henceforth called A(ccelerated)CTP

CTP Service

UDP 2020 (ACTP)

TCP 2021 (SCTP)

WSS 443 (WCTP)

HTTPS 443 (HCTP)

DivConq CTP

Many of the following slides present details and examples of a reference

implementation of CTP. As long as the results are the same over the wire, it

doesn’t really matter how the CTP server or client is implemented. But

understanding a reference implementation will help expose the design ideas.

Server

There are four options for network connectivity, but there are really only three

paths through the system. Each message is required to have a session identity

by the time it reaches the Session Handler. Typically this means each message

contains a session identity, except with HTTP which has the identity in a secure

HTTP only cookie.

UDP (ACTP)

Session Handler

CTP

Connectors

WS (WCTP)

HTTP (HCTP)

Common Handler

HTTP Handler

Common Security

TCP (SCTP)

Web Security

Server

With Common Security the host name provided on initial network connection

becomes the default host name, but individual messages may override that name

when authenticating a user. Although encryption of these connections resembles

SSH, the client key given at connection is not a user key – it is just a way to confirm

that the client app is allowed. User keys are provided during user authentication. A

single network connection may support multiple users/sessions.

UDP

Session Handler

Web Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

Web Security

CTP

Connectors

Server

With Web Security the host name provided on connection may not be overridden

by a message. Encryption of these connections uses SSL, the client key (if any)

*is* considered to be the user’s key. There can only be one user and one session

per connection. An Authenticate message must still be sent, however, before a

Session is ready to use.

UDP

Session Handler

Web Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

Web Security

CTP

Connectors

Server

With Common Handler communications are expected to be full duplex. The

server or client may push messages at any time. There is no need to wait for

responses. Messages entering Session Handler can be either “send and forget”

or “send”, there is no “send and wait”.

UDP

Session Handler

Web Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

Web Security

CTP

Connectors

Server

With HTTP Handler communications are expected to be half duplex, the server

may NOT push messages at all. Caller must wait on responses. Messages

entering Session Handler can be “send and forget” or “send and wait”. HTTP

Handler sets a SessionId cookie to track the session identity.

UDP

Session Handler

Web Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

Web Security

CTP

Connectors

DivConq CTP

Example Authentication Message: { Service: "Sessions", Feature: "Manager", Op: "Start", Body: { Credentials: { UserName: "fredsmith", Password: "temp123" } } }

To the client and server message structure is JSON-

like (exact details vary depending on network layer).

Actually messages are YAML-like in that data types

may be associated with fields, but for practical

purposes messages are JSON-like.

Messages contain fields. Fields may hold a scalar

value (string, number, true, false or null) or an object

or an array. Just like JSON. Structures may be

nested.

Web page scripts may deal with messages “natively”

using JSON. But JSON is also easy enough to work

with in many programming languages.

When not dealing with large binary data the

message structure works fine – smallish binary data

can be base64 encoded and treated as text.

When dealing with large binary data though, base64 is not the way to go. With

UPD (ACTP), TCP (SCTP) and Web Sockets (WCTP) we can take advantage

of the fact that there are data types and build that awareness into the network

layer. The network layer then sees a sort of “binary JSON” or more accurately

a “binary YAML”.

Web Sockets and JavaScript have binary capability (in HTML5), which is why

WCTP is listed above. HTTP, however, is still better off in text (XML/JSON)

based calls to the server (XHR) – especially for legacy support. Therefore

HTTP will use JSON/XHR to perform all CTP commands, and will open a

separate connection for pure upload or download of binary content.

Although this exception for HTTP makes the implementation of CTP more

complex, it does ensure highest compatibility since any HTTP 1.1 library can

access this service. It also avoids some of the biggest mistakes of FTP

because there is no “active mode” data transfer and there is no separate port

for data transfer – use the same port just a different URL. So proxies and fire

walls are not as big an issue here as with FTP.

Typical Fields: { Service: "XXX", Feature: "YYY", Op: "ZZZ", RespondTag: "010A0D0502", Body: { ... } }

When making a request message (client) there are

some fields that will appear frequently in the

following examples. A quick overview:

Service – the name of the destination service

Feature – a part of the service

Op – operation to perform with Feature

RespondTag – only present if you want a response

to the message, provides a way for the client to

associate a response message with a callback

Body – the payload (or parameters) for the

feature/operation requested

DivConq CTP

Server

UDP

Session Handler

RPC

Web Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

RPC

Web Security

Example Message as seen by Session Handler: { Service: "Sessions", Feature: "Manager", Op: "Start", RespondTag: "010A0D0502", Body: { Domain: "divconq.com", Origin: "rpc:[ipaddress]", Credentials: { UserName: "nnnn", Password: "nnnn" } } }

When the Message gets to Session Handler an

Origin field has been injected by Common Security.

If no Domain was provided by the original sender,

then that too is injected (based on the default Host

for that network connection).

Server

UDP

Session Handler

Web Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

Web Security

When the Message gets to Session Handler the

Origin, Domain and SslThumb (optional) fields have

been injected by Web Security.

RPC

Example Message: { Service: "Session", Feature: "Manager", Op: "Start", RespondTag: "010A0D0502", Body: { Domain: "divconq.com", Origin: "rpc:[ipaddress]", Credentials: { UserName: "nnnn", Password: "nnnn", SslThumb: "010A0D0502" } } }

Server

UDP

Session Handler

Web

Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

Web Security

When the Message gets to Session Handler the

Origin, Domain and SslThumb (optional) fields have

been injected by Web Security. [more next slide]

RPC

Example Message: { Service: "Session", Feature: "Manager", Op: "Start", RespondTag: "SendWait", Body: { Domain: "divconq.com", Origin: "rpc:[ipaddress]", Credentials: { UserName: "nnnn", Password: "nnnn", SslThumb: "010A0D0502" } } }

Server

UDP

Session Handler

Web

Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

Web Security

Both browser requests for web pages as well as XHR, require a Session Context.

To support both models the session context is extracted by HTTP handler from a

secure HTTP [server only] cookie. Server only is used to protect the cookie from

cross domain attacks and such.

RPC + Web

Example Message: { Service: "Session", Feature: "Manager", Op: "Start", RespondTag: "SendWait", Body: { Domain: "divconq.com", Origin: "rpc:[ipaddress]", Credentials: { UserName: "nnnn", Password: "nnnn", SslThumb: "010A0D0502" } } }

DivConq Collab

App Server

UDP

Session Handler

Web

Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

Web Security

Initial RPC/Web calls cause a Session to be created in Session Handler. Calls with

a SessionId cookie will be connected to an existing Session.

Although the App Server cannot push messages over HTTP, it can queue

messages for an (HTTP originated) Actor of a Session.

RPC

A

App Server

UDP

Session Handler

Web

Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

Web Security

Example of an Initial RPC call. Credentials are

optional, if not present then the session will be a

guest session. It takes Trust points to create a

session, too many attempts will block the client.

RPC

A

Example RPC Request Message: { Service: "Session", Feature: "Manager", Op: "Start", RespondTag: "SendWait", Body: { Credentials: { UserName: "nnnn", Password: "mmmm" } } }

App Server

UDP

Session Handler

Web

Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

Web Security

Client gets back User info (JSON) and Session info (cookie). An Authentication

Token as well as the User info are stored in the Session. Anyone with the session

Id and Key can now act as this authenticated user (see guest next slide)

RPC

A

Example Response Message: { Result: 0, // success Body: { UserId: "119", UserName: "awhite", FirstName: "Andy", LastName: "White", Email: "[email protected]", Chronology: "/America/Chicago", Locale: "en-US", Session: { Id: "nnnn", Key: "010A0D0502" } } } Response Session Cookie: Session: "nnnn_mmmm_010A0D0502"

Stored in Session: Context: { Domain: "divconq.com", Origin: "http:[ipaddress]", UserId: "119", UserName: "awhite", FirstName: "Andy", LastName: "White", Email: "[email protected]", AuthToken: "010A0D0502", Chronology: "/America/Chicago", Locale: "en-US", Credentials: { UserName: "nnnn", Password: "mmmm" } }

App Server

UDP

Session Handler

Web

Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

Web Security

If a guest session (no credentials), then all we get

back is session info. The default (guest)

Chronology and Locale are returned.

RPC

A

Example Response Message: { Result: 0, // success Body: { Chronology: "/America/Chicago", Locale: "en-US", Session: { Id: "nnnn", Actor: "mmmm", Key: "010A0D0502" } } } Response Session Cookie: Session: "nnnn_mmmm_010A0D0502“ Stored in Session: Context: { Domain: "divconq.com", Origin: "http:[ipaddress]", Chronology: "/America/Chicago", Locale: "en-US" }

App Server

UDP

Session Handler

Web

Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

Web Security

Message requesting service X is processed through the session. The session

becomes the proxy for the client. User Context is restored from session cookie by a

visit to the session state before passing the message on to service X. Service X

has context and a destination for response, thanks to the session. When the

session gets the response it then sends it back to the client (with HTTP the client is

blocking until the response comes).

RPC

A

Example RPC Request Message: { Service: "X", Feature: "Y", Op: "Z", RespondTag: "SendWait", Body: { ... } } Session Cookie: Session: "nnnn_mmmm_010A0D0502"

X

App Server

UDP

Session Handler

Web

Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

Web Security

Nothing special is returned. Result is non-zero if error. Tag for the RespondTag

field sent. Message for any error message. Body for the data of the response.

RPC

A

Example RPC Response Message: { Result: 0, // success Tag: "SendWait", Body: { ... } }

X

App Server

UDP

Session Handler

Web

Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

Web Security

Someone sends a message for session A. A is configured (by HTTP Handler) to

queue incoming messages.

A

ABC

App Server

UDP

Session Handler

Web

Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

Web Security

Client must actively decide to check mail (queue) by issuing a request. The request

may be long polling. Messages on the queue are popped and returned in a list.

RPC

A

Example Request Message: { Service: "Session", Feature: "Mail", Op: "Check", RespondTag: "SendWait", Body: { ... } } Session Cookie: Session: "nnnn_mmmm_010A0D0502"

ABC

App Server

UDP

Session Handler

Web

Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

Web Security

Each response has a target Service, Feature, Op, Body. FromHub, RespondTo and

RespondTag are allowed. ToHub and Tag are removed as those where values

associated with the Session.

RPC

A

Example Response Message: { Body: [ { Service: "Desktop", Feature: "App", Op: "Open", Body: ... } ] }

ABC

DivConq CTP

App Server

UDP

Session Handler

Web Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

Web Security

Each request via Common Handler must contain a session context or be requesting

a session context.

A

RPC

RPC

RPC

App Server

UDP

Session Handler

Web Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

Web Security

A

RPC

RPC

RPC

Example of an Initial RPC call. Credentials are optional, if

not present then the session will be a guest session. It

takes Trust points to create a session, too many attempts

will (deplete Trust) block the client.

Example RPC Request Message: { Service: "Session", Feature: "Manager", Op: "Start", RespondTag: "010A0D0502", Body: { Domain: "divconq.com", Credentials: { UserName: "nnnn", Password: "mmmm" } } }

App Server

UDP

Session Handler

Web Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

Web Security

A

RPC

RPC

RPC

Client gets back User and Session info. Authentication Token and User info are

stored in Session. Anyone with the session Id and Key can now act as this

authenticated user (see guest next slide)

Example Response Message: { Result: 0, // success Tag: "010A0D0502", Body: { UserId: "119", UserName: "awhite", FirstName: "Andy", LastName: "White", Email: "[email protected]", Chronology: "/America/Chicago", Locale: "en-US", Session: { Id: "nnnn", Key: "010A0D0502" } } }

Stored in Session: Context: { Domain: "divconq.com", Origin: "http:[ipaddress]", UserId: "119", UserName: "awhite", FirstName: "Andy", LastName: "White", Email: "[email protected]", AuthToken: "010A0D0502", Chronology: "/America/Chicago", Locale: "en-US", Credentials: { UserName: "nnnn", Password: "mmmm" } }

App Server

UDP

Session Handler

Web Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

Web Security

A

RPC

RPC

RPC

If a guest session, then all we get back is session

info. The default (guest) Chronology and Locale are

returned.

Example Response Message: { Result: 0, // success Tag: "010A0D0502", Body: { Chronology: "/America/Chicago", Locale: "en-US", Session: { Id: "nnnn", Actor: "mmmm", Key: "010A0D0502" } } } Stored in Session: Context: { Domain: "divconq.com", Origin: "http:[ipaddress]", Chronology: "/America/Chicago", Locale: "en-US" }

App Server

UDP

Session Handler

Web Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

Web Security

A

RPC

RPC

RPC

User Context is restored from session field before passing the message on to

service X. Service X sees a RespondTo and RespondTag corresponding to the

HubId and Session_Actor. If the incoming request has a RespondTag, it is added to

the new RespondTag as in Session_Actor_Tag. The RespondTag is properly

restored to Tag on response.

Example RPC Request Message: { Service: "X", Feature: "Y", Op: "Z", RespondTag: "abcd", Session: { Id: "nnnn", Actor: "mmmm", Key: "010A0D0502" }, Body: { ... } }

X

App Server

UDP

Session Handler

Web Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

Web Security

A

RPC

RPC

RPC

X

Nothing special is returned. Tag is present if a RespondTag was sent. Body for

the response payload. Service X does not have to know anything about the client.

Example RPC Response Message: { Result: 0, // success Tag: "abcd", Body: { ... } }

Someone sends a message for session A. A is configured (by Common Handler) to

directly route incoming messages over the full duplex network layer. Tag is adjusted

to remove the session id, but otherwise the message is fairly consistent with what

ABC sent.

App Server

UDP

Session Handler

Web Socket

HTTP Server

Common Handler

HTTP Handler

Common Security

TCP

Web Security

A

RPC

RPC

RPC

ABC

DivConq CTP

Common Path Request Message: { Service: "Session", Feature: "Manager", Op: "Start", RespondTag: "010A0D0502", Body: { Domain: "divconq.com", Credentials: { UserName: "nnnn", Password: "mmmm", SignedKey: "mmmm" } } }

HTTP Path Request Message: { Service: "Session", Feature: "Manager", Op: "Start", RespondTag: "abcd", Body: { Credentials: { UserName: "nnnn", Password: "mmmm", SslThumb: "mmmm" } } }

WS Path Request Message: { Service: "Session", Feature: "Manager", Op: "Start", RespondTag: "abcd", Body: { Credentials: { UserName: "nnnn", Password: "mmmm", SslThumb: "mmmm" } } }

Common Path Request Message: { Result: 0, // success Tag: "010A0D0502", Body: { UserId: "119", UserName: "awhite", FirstName: "Andy", LastName: "White", Email: "[email protected]", Chronology: "/America/Chicago",

Locale: "en-US", Session: { Id: "nnnn", Actor: "mmmm", Key: "010A0D0502" } } }

HTTP Path Request Message: { Result: 0, // success Tag: "010A0D0502", Body: { UserId: "119", UserName: "awhite", FirstName: "Andy", LastName: "White", Email: "[email protected]", Chronology: "/America/Chicago",

Locale: "en-US", Session: { Id: "nnnn", Actor: "mmmm", Key: "010A0D0502" } } } Response Session Cookie: Session: "nnnn_mmmm_010A0D0502"

WS Path Request Message: { Result: 0, // success Tag: "010A0D0502", Body: { UserId: "119", UserName: "awhite", FirstName: "Andy", LastName: "White", Email: "[email protected]", Chronology: "/America/Chicago",

Locale: "en-US", Session: { Id: "nnnn", Actor: "mmmm", Key: "010A0D0502" } } }

Common Path Request Message: { Service: "X", Feature: "Y", Op: "Z", RespondTag: "abcd", Session: { Id: "nnnn", Actor: "mmmm", Key: "010A0D0502" }, Body: { ... } }

HTTP Path Request Message: { Service: "X", Feature: "Y", Op: "Z", RespondTag: "abcd", Body: { ... } } Session Cookie: Session: "nnnn_mmmm_010A0D0502"

WS Path Request Message: { Service: "X", Feature: "Y", Op: "Z", RespondTag: "abcd", Session: { Id: "nnnn", Actor: "mmmm", Key: "010A0D0502" }, Body: { ... } }

Common Path Request Message: { Result: 0, // success Tag: "abcd", Body: { ... } }

HTTP Path Request Message: { Result: 0, // success Tag: "abcd", Body: { ... } }

WS Path Request Message: { Result: 0, // success Tag: "abcd", Body: { ... } }

DivConq RPC

Since the “real” user context/authentication is stored in the session, and because other vital data may be stored with a session, if a session is lost then the client must start a new one. 1) When client makes a call it will fail because the session id cannot be matched (or

server is down). 2) The client should raise an event “session lost” or “connectivity lost”. Every part of the

client app that relies on session data should clear their flags/states/cache. This should not trigger new RPC’s, wait for “session connected”

3) Try to reconnect and start session 4) Whether 3 works or not, return error to caller because we don’t know if caller relied

on session info 5) If 3 worked then trigger “session connected” event.

DivConq CTP

Client App using API App Server

One Connector represents one user (set of credentials), one or more connections

and one session on server. Connections pooled by browser, just simple

SendWait or SendForget support. Uploads and Downloads have their own URLs.

Session

App Service

A RPC

Web App

Connector Upload

Download


Each Connector represents one user (set of credentials), one connection and one

session on server. No connection pool, just simple Send or SendForget support.

App Server may issue a “Session Moving” message to inform client to use a new

server (e.g. from a server farm).

Session

App Service

A RPC

App

Connector


Each client Session represents one user

(set of credentials) and one session on

server. Connections are pooled, message

from client session to server session may

use any connection to that server. New

client session may start on any connected

server, but session stays with that server.

App Server may issue a “Session Moving”

message to inform client to use a new

server (e.g. from a server farm).

Sessions App Services

A

RPC

App

Pool Sessions App Server


M

App Server


X

B C

N O

Y Z

DivConq CTP

This first example (just one server) may appear overly complex until we review the second example (three servers). CTP is designed to work well within a distributed application environment where some parts may be running on separate servers. But in this first example all parts are running on one server.

App Server

RPC caller sends service message to service Z asking for a download. Hub and

Session Id are included as part of the request so that Z knows where to route the

download to.

Session

Service Z

A RPC

App Server

If Z approves the download, then it creates a Plan which will coordinate all tasks

required for the download(s). Plan holds File Store Access Code, download

Milestones and all details on sessions participating in download.

Session

Service Z

A RPC

Plan

App Server

Plan starts a session on the File Store (same server in the example) by passing

the FS Access Code. Meanwhile the RPC client told that transfer was approved

and which Plan holds the milestones.

Session

B

Session

Service Z

A RPC

Plan

App Server

Session A registers with Plan, waits for Milestone 1. Meanwhile Session B

prepares the download and then registers Milestone 1. This means that the

Source (B) is ready.

Session

B

Session

Service Z

A RPC

Plan

App Server

Session A (Destination) calls Session B (Source) requesting the next (starting with

first) block of the file to transfer. As a block is received by A, it immediately asks for

the next block from B and pushes the current block to the client. A will never have

more than two blocks in holding – if client is slow to accept blocks then B waits until

at least one is accepted by client before proceeding to the next block from B.

Session

B

Session

Service Z

A RPC

Plan

App Server

B sends “final block” flag with final block of file. A passing that flag on to client

when possible (with HTTP download, the response is streamed (chunked) so we

just finish the response to indicate done).

Session

B

Session

Service Z

A RPC

Plan

App Server

After client has file session A sets Milestone 2 with the Plan. Plan knows that this

means it is done, so it kills session B and itself.

Session

B

Session

Service Z

A

Plan

DivConq Database

LA

N F

irewall

DMZ Server DMZ Server

RPC caller sends service message to service Z asking for a download. Hub and

Session Id are included as part of the request so that Z knows where to route the

download to.

Session

Service Z

A RPC

File Store Server

If Z approves the download, then it creates a Plan which will coordinate all tasks

required for the download(s). Plan holds File Store Access Code, download

Milestones and all details on sessions participating in download.

LA

N F

irewall


Session

Service Z

A RPC

File Store Server Plan


the FS Access Code. Meanwhile the RPC client told that transfer was approved

and which Plan holds the milestones.

LA

N F

irewall


Session

Service Z

A RPC


Session

B

Session A registers with Plan, waits for Milestone 1. Meanwhile Session B

prepares the download and then registers Milestone 1. This means that the

Source (B) is ready.

LA

N F

irewall


Session

Service Z

A RPC


Session

B

Session A (Destination) calls Session B (Source) requesting the next (starting with

first) block of the file to transfer. As a block is received by A, it immediately asks for

the next block from B and pushes the current block to the client. A will never have

more than two blocks in holding – if client is slow to accept blocks then B waits until

at least one is accepted by client before proceeding to the next block from B.

LA

N F

irewall


Session

Service Z

A RPC


Session

B

B sends “final block” flag with final block of file. A passing that flag on to client

when possible (with HTTP download, the response is streamed (chunked) so we

just finish the response to indicate done).

LA

N F

irewall


Session

Service Z

A RPC


Session

B

After client has file session A sets Milestone 2 with the Plan. Plan knows that this

means it is done, so it kills session B and itself.

LA

N F

irewall


Session

Service Z

A RPC


Session

B

DivConq Database

Client sends service Z a message asking for an upload. Hub and Session Id are

included as part of the request so that Z knows where to route the download to.

LA

N F

irewall


Session

Service Z

A RPC

File Store Server

If Z approves the upload, then it creates a Plan which will coordinate all tasks

required for the upload. Plan holds File Store Access Code, upload Milestones

and all details on sessions participating in upload.

LA

N F

irewall


Session

Service Z

A RPC



the FS Access Code. Meanwhile the client is told that transfer was approved and

which Plan holds the milestones.

LA

N F

irewall


Session

Service Z

A RPC


Session

B

Session A registers with Plan, sets Milestone 1. This means that the Source (A) is

ready. Meanwhile Session B prepares for the upload, registers with Plan and

then waits for Milestone 1.

LA

N F

irewall


Session

Service Z

A RPC


Session

B

Session B (Destination) calls Session A (Source) requesting the next (starting with

first) block of the file to transfer. As a block is received by B, it immediately asks for

the next block from S and writes the current block to the store. B will never have

more than two blocks in holding – if the store is slow to accept blocks then B waits

until at least one is written before proceeding to the next block from A.

LA

N F

irewall


Session

Service Z

A RPC


Session

B

A sends “final block” flag with final block of file. B closes the file after final write.

LA

N F

irewall


Session

Service Z

A RPC


Session

B

After client has finished upload file session A sets Milestone 2 with the Plan. Plan

knows that this means it is done, so it kills session B and itself.

LA

N F

irewall


Session

Service Z

A RPC


Session

B

DivConq CTP

Client App App Server

To speed up a large file upload, use “Parts”. In this case we’ll transfer three parts at the

same time.

To transfer parts the requests go to the same server (or same squad/farm) for each

session. Only one file name is given for upload, each session transfers a different set of

blocks for the file.

Collab Session File Store

A

RPC

FT App

Pool Sessions

M X

A M X

DivConq CTP

Client App App Server

Each client Session represents one

session on a different server (squad/farm).

One file name is given. Partial files are

verified independently – client knows that

all of part A is intact, all of part M is intact

and that all of part X is intact. Also knows

that the offsets are correct. It all adds up

to one complete file even if that file is in

multiple geographies.


A

RPC

FT App

Pool Sessions App Server


M

App Server


X

A M X

Common Transfer Protocol can meet its goals.

• A complete spec for messaging commands as well as file transfers (SIGN ON,

DIR, GET, PUT)

• It is abstracted from network layer, could be implemented in TCP, UDP or

through either HTTP or Web Socket

• It does not assume full duplex communications

• It does not (necessarily) associate network layer client credentials with the file

transfer, which can lead to scalability and efficiency issues

• It is platform and language neutral

• Server could be implemented in Java, C, Python or similarly suitable

languages

• Client can be implemented in any language with TCP or HTTP capabilities

• It is open and free

ctp

Technology