Using SCTP with Scamper and NettyUsing SCTP with Scamper and Netty

Tim BoudreauTim Boudreau

http://timboudreau.comhttp://timboudreau.com

October 2015October 2015

What is SCTP?What is SCTP?● Stream control transmission protocol● Alternative to TCP

– Yes, that far down the network stack!● Message-oriented like UDP● Multiple streams per connection

– No “head of line” blocking● Multiple addresses per connection● Heartbeats built-in

Why SCTP?Why SCTP?● Designed for multimedia & telephony● But good for anything message-oriented

– Metrics / logging data in particular● Supported by most OS's

– Need lksctp-tools on Linux

Anatomy of an SCTP connectionAnatomy of an SCTP connection● Multiple “channels” per connection

– Send data in sequence on a channel– Does not block sends on other channels

● Association – endpoint = more than one socket address

– Will take the shortest path– Provides network-level fault tolerance

What That MeansWhat That Means● You can decide if you care about message-order● Messages don't (have to) block each other● Messages are either completely delivered or not

– The application doesn't have to deal with partial messages● Message size not limited (unlike UDP)● Reliable (unlike UDP)● Heartbeat built-in

Why isn't SCTP more used?Why isn't SCTP more used?● People use what they know● Home routers don't always support it well via

NAT– Not a problem for server-server communication

Use Case – statsd-like metricsUse Case – statsd-like metrics● Etsy's Statsd is a popular daemon for collecting real-time

metrics● Receives fire-and-forget small UDP packets that update

counters, etc.● Statsd stores them in a back-end for visualization &

reporting

Statsd issuesStatsd issues● You don't know if a UDP packet was

delivered● Packet size must be < network MTU

– Or the packet may be silently dropped● Connection not redundant

SCTP Support in JavaSCTP Support in Java● Very low-level

– com.sun.nio.sctp.SctpChannel

● Complete but hard to work with– Implement message loop, handlers– Typical NIO network API

SCTP Support in NettySCTP Support in Netty● Also low-level

– But better abstractions for writing async servers– Buffer memory pooling to reduce allocation and

other goodies– Still low-level

Why Scamper?Why Scamper?● I wanted some higher level abstractions● Writing asynchronous servers doesn't have to feel like doing something hard

– Follow the NodeJS model: Build a highly concurrent system out of small blocks of simple synchronous code

Scamper – Higher Level Abstractions

Scamper – Higher Level Abstractions

● Messages are objects● Protocols are simple to write

– 5 byte message header + payload● Compression & encryption can be layered on top of any

protocol● Standards-based wire-format for messages

– BSON / JSON / Java serialization / roll-your-own

Scamper – DetailsScamper – DetailsDetermines payload encoding – BSON, JSON, Serializaton +/- compression or encryption.

You define message types, map them to object classes.

The payload, which is encoded/decoded for you

public static final MessageType WHAT_TIME_IS_IT = new MessageType("dateQuery", 1, 1);

public static final MessageType THE_TIME_IS = new MessageType("dateResponse", 1, 2);

new SctpServerAndClientBuilder("datedemo").bind(WHAT_TIME_IS_IT, DateQueryHandler.class).bind(THE_TIME_IS, DateResponseHandler.class).start();

Using ScamperUsing Scamper● Define some POJOs (or use a Map)● Write a handler to receive POJOs● Register a MessageType

– 2 shorts, and a name (for logging)– Identifies which handler should be called with the message

● Create an SctpServerAndClientBuilder● Bind MessageType+Handler pairs● Start the server, or retrieve a Sender if a client

Demo

A trivial client + serverA trivial client + server

Writing a Message TypeWriting a Message Typepublic static final MessageType THE_TIME_IS = new MessageType("dateResponse", 1, 2);

● The 1 and 2 are message identifiers – you define them (16 bit Java shorts)

● The name is just for logging (not sent over the wire)● To create a message:

Message<DateResponse> msg = THE_TIME_IS.newMessage(new DateResponse());

sender.send(address, msg);

Writing a Message HandlerWriting a Message Handler

● Says what type it takes in its constructor● Implements onMessage() to receive POJOS● Can reply with a new message

static class DateResponseHandler extends MessageHandler<Map, DateRecord> { DateResponseHandler() { super(DateRecord.class); }

public Message<Map> onMessage(Message<DateRecord> data, ChannelHandlerContext ctx) { System.out.println("RECEIVE " + new Date(data.body.when) + " from " +ctx.channel().remoteAddress()); return null; //we could reply here }}public static class DateRecord implements Serializable { public long when = System.currentTimeMillis();}

Is It Baked Yet?Is It Baked Yet?● The things that work, work● Uses pre-release Netty 5

– Which has been very stable● Getting multi-homing working on a branch● In-progress patches for Netty for One-to-Many

support

Scamper – How to Get ItScamper – How to Get It● Maven dependency : <dependency> <groupId>com.mastfrog</groupId> <artifactId>scamper</artifactId> <version>1.3-dev</version> </dependency>

● Need my maven repo in your pom.xml– Follow instructions here: http://timboudreau.com/builds– Will be in Maven central Real Soon Now[tm]

LinksLinks● Sources: https://github.com/timboudreau/scamper● Info: http://timboudreau.com/blog/sctp/● Chat Demo:

https://github.com/timboudreau/scamper-chat● Maven repo: http://timboudreau.com/builds/● Me: http://timboudreau.com @kablosna

Thanks!

Demos, Q&ADemos, Q&A

According to the RFCAccording to the RFC● Acknowledged error-free non-duplicated transfer of user

data● Data fragmentation to conform to discovered path MTU size● Sequenced delivery of user messages within multiple

streams, with an option for order-of-arrival delivery of individual user messages

● Optional bundling of multiple user messages into a single SCTP packet, and

● Network-level fault tolerance through supporting of multi-homing at either or both ends of an association.