DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

Chief Technology OfficerPrismTech

DDS SIG Co-ChairObject Management Group (OMG)

Angelo Corsaro, Ph.D.[angelo.corsaro@prismtech.com]

Efficiency and Extensibility

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

./type-matters

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

The Type Systems Debate

‣ The debate around type Programming Languages Type Systems has animated computer scientist over several decades and is far from being resolved

‣ New Programming Languages are continuing to appear on both sides of the fence

F#Statically Typed (ex.) Dynamically Typed (ex.)

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

Are you a Type-Phobic Type?

‣ Proponents of Dynamically Typed Programming Languages advocate against strong typing in favor of reduced verbosity and added flexibility

Yet... Are they so sure that a type system always gets between you and what you are trying to achieve?

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

Strongly-Typed Type-Systems

‣ Well designed type-systems don’t add unnecessary verbosity to your application since use sophisticated inference to derive types

‣ Allow the detection of many errors at compile time, thus improving productivity, safety, code quality and reducing the potential for run-time errors

‣ Enable the generation of more efficient code

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

Types and Distributed Systems

‣ The “Type System debate” has not spared distributed systems

‣ Today we can find different technologies taking different approaches: from completely untyped to strongly typed

‣ The differences are in this case more complex to organize:

‣ Some technologies support only one type, being a string or an array of bytes (e.g. AMQP)‣ Other technologies support some predefined types (e.g JMS)‣ Other technologies allow the definition of user-specified types (e.g DDS)

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

Types and Distributed Systems

T1

T2

T3

T4

T1

T3

T3

T2

T1

T4

T1

T2

?

T1 T2 T3 T4

User Defined Types Example: T1 = RadarTrack T3 = FlightPlanT2 = ClassificationInfo T4 = ...

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

Types and Distributed Systems

T2

T3

T4

T1

T3

T2

T1

T4

T1

T2

T1T3

AMQP

octect sequence

T1 T2 T3 T4

User Defined Types

Pub/Sub Types

Example: T1 = RadarTrack T3 = FlightPlanT2 = ClassificationInfo T4 = ...

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

Types and Distributed Systems

T2

T3

T4

T1

T3

T2

T1

T4

T1

T2

T1T3

JMS

ObjectMessage TextMessage

ByteMessage

StreamMessage

MapMessage

Pub/Sub Types

Example: T1 = RadarTrack T3 = FlightPlanT2 = ClassificationInfo T4 = ...

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

Types and Distributed Systems

T2

T3

T4

T1

T3

T2

T1

T4

T1

T2

T1T3

DDS

T2 T3

T4

T1

T3

T1

Pub/Sub Types == User Types!

Example: T1 = RadarTrack T3 = FlightPlanT2 = ClassificationInfo T4 = ...

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

./dds-types

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

Topic: the unit of Information

Topic Type

‣ Extensible and Evolvable Unit of information exchanged between Publisher and Subscribers.

‣ An association between a unique name, a type and a QoS setting

TrackedObject

Track, UFO, AirCraft, ...

{ Reliability Deadline, Priority Transient, ...}

TopicType QoS

Name

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

Topic: the unit of InformationTopic Type

‣ Extensible and evolvable Type describing the data associated with one or more Topics

‣ A key, made by one or more attributes

‣ Annotations express attribute properties

‣ Expressed in IDL, Java, XML, or UML

struct Track { long tid; //@key long x; long y;};

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

Writing Tracksusing dds::core; using dds::domain; using dds::pub; using dds::topic;

DomainId id = 0;

DomainParticipant dp = theParticipantFactory().create_participant(id);

Publisher pub = dp.create_publisher();

Topic<RadarTrack> topic = dp.create_topic("RadarTrackTopic");DataWriter<RadarTrack> dw = pub.create_datawriter();

RadarTrack t("T101", 100, 200);

dw.write(t);

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

Reading Tracksusing dds::core; using dds::domain; using dds::pub; using dds::topic;

DomainId id = 0;DomainParticipant dp(id);

DomainParticipant dp = theParticipantFactory().create_participant();

Subscriber sub = dp.create_subscriber();

Topic<RadarTrack> topic = dp.create_topic("RadarTrackTopic");DataReader<RadarTrack> reader = sub.create_datareader();

std::vector<RadarTrack> t(max_size);std::vector<SampleInfo> i(max_size);

reader.read(t.begin(), i.begin(), max_size);

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

./efficiency

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

Encoding Efficiency

‣ DDS serialization is several times more efficient than JSON, XML and YAML

‣ Differences can be quite extreme when dealing with complex types

Size for Serialized CoFlight Flight Data Plan

see Esposito et al. OMG RTWS 2008

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

Encoding Impact on RTT

see Esposito et al. OMG RTWS 2008

DDS RTT with different Serialization Format

‣ Inefficient encoding can easily dominate RTT

‣ DDS native encoding is the most efficient among extensible encodings!

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

Size (bytes)

Lat

en

cy (

use

c)

‣ DDS implementations are capable of delivering very low and predictable latencies

‣ No other Pub/Sub technology can approach similar level of performance!

DDS is Very Efficient!

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

./extensibility

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

Type Extensibility

Problem

‣ What if we need to extend the RadarTrack type to add the derivative on the x and y direction?

‣ How can we do this with no impact on running applications?

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

Type ExtensibilitySolution‣ Simply extend the existing RadarTrack and properly annotate new

attributes

‣ Existing application will detect that the new type is an extension and will “project/extend” it to the known type

‣ Type “projection/extensions” can be controlled via QoS

struct Track { long tid; //@key long x, y;};

struct Track { long tid; //@key long x, y; long dx, dy;};

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

Type Extensibility

Problem

‣ OK, that was good. But now I need to add an optional plot to my track

‣ How can I add an optional field? Furthermore, I don’t want the plot to be copied into the topic, I’d like it to be a reference. Can I do this to limit the number of copies and reduce he memory utilization?

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

Type ExtensibilitySolution

‣ Simply extend the existing RadarTrack and properly annotate new attributes

struct Track { long tid; //@key long x, y; long dx, dy;};

struct Track { long tid; //@key long x, y; long dx, dy; sequence<octet> plot; //@optional @shared};

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

In Summary

‣ DDS provides a strongly typed extensible type system

‣ Types can be evolved and extended without breaking backward compatibility nor loosing type-safety

‣ Type “Projection/Promotions” can be fully controlled via QoS

‣ This flexibility does not come at the cost of introducing the time+space overhead typical of XML-based approaches

DDS provides extensibility without compromising efficiency!

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

./coming-soon

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

./web-enabled

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DDS

Radar

TrackClassifier

Display

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DDSJMS

Radar

TrackClassifier

TrackClassifier

Display

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DDSJMS REST

Radar

TrackClassifier

TrackClassifier

Display

Web Browser

iPhone

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DDSJMS

REST

WS-*

Radar

ControllerWorkingPosition

Tuner

Web Browser

iPhone

TrackClassifier

TrackClassifier

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

Web-Enabled DDS

Standardize access to the DDS Global Data Space via:

‣ RESTful Web Services

‣ W3C Web Services

‣ RSS

‣ Atom

‣ XMPP

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

./demo

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

./security

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

Secure DDS

‣ An RFP is close to be finalized for standardizing a security solution for DDS

‣ Key use cases taken into consideration come from Military and Civilian systems

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

Concluding Remarks

‣ DDS is a standard that was designed for time/space efficiency as well as real-time predictability

‣ DDS provides support for end-to-end type propagation and type checking with a very powerful type extensibility mechanism

‣ Upcoming standards for DDS are coming to address integration with Web/Internet technologies as well as security

© 2

010

Prism

Tech

. All

Rig

hts

Res

erve

d.

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

SO

MG

DD

S

Learn More

‣ Data Distribution Service for Real-Time Systems

‣ Extensible and Dynamic Topic Types Specification

‣ Web-Enabled DDS RFP

http://www.omgwiki.org/dds/content/page/specifications