the present and future of dds

Angelo Corsaro, Ph.D. Chief Technology Officer!OMG DDS Sig Co-Chair PrismTech [email protected]!

The Present and Future "of DDS

The Present

The DDS Standard

Copyrig

ht 2011, PrismTech – A

ll Rights Reserved.

Data Distribution for Real-Time Systems

¨  Introduced in 2004 to address the Data Distribution challenges faced by a wide class of Defense and Aerospace Applications

¨  Key requirement for the standard were to deliver very high, predictable, performance while scaling from embedded to ultra-large-scale deployments

¨  Recommended by key administration worldwide

¨  Widely adopted across several different domains, e.g., Automated Trading, Simulations, SCADA, Telemetry, etc.

Copyrig


ll Rights Reserved.

DDS Standards Family

DDS

App

2004

Standard API

§  Standard API for Data-

Centric Real-Time Pub/Sub §  Automatic fail-over §  Persistence §  Dynamic Discovery §  Content Filtering / Queries

Copyrig


ll Rights Reserved.


DDS

App Standard API

DDSI/RTPS network

Interoperable Wire Protocol

2004

2006

DDS

App

DDSI/RTPS

2004

2006

§  High Performance §  Interoperable Wire Protocol

Copyrig


ll Rights Reserved.


DDS

App Standard API

DDSI/RTPS network


UML4DDS

2004

2006

2008

DDS

App

DDSI/RTPS

2004

2006

UML4DDS 2008

§  UML-Based Modeling of DDS applications

Copyrig


ll Rights Reserved.


DDS

App Standard API

DDSI/RTPS network


UML4DDS

2004

2006

2008

X-T

ype

s 20

10

DDS

App

DDSI/RTPS

UML4DDS

2004

2006

2008

X-T

ype

s 20

10

§  Extensible/Evolvable Type System

§  Dynamic Topic Types, Data Readers & Writers

§  Encoding Negotiation

Copyrig


ll Rights Reserved.


DDS

App Standard API

DDSI/RTPS network


UML4DDS

ISO-C++ PSM 2010 2004

2006

2008

X-T

ype

s 20

10

DDS

App

DDSI/RTPS

UML4DDS

2004

2006

2008

X-T

ype

s 20

10

§  ISO C++ DDS PSM §  Simple, Safe, Efficient,

Elegant and Ergonomic API

Copyrig


ll Rights Reserved.


DDS

App Standard API

DDSI/RTPS network


UML4DDS

ISO-C++ PSM 2010

Java5 PSM 2010 2004

2006

2008

X-T

ype

s 20

10

DDS

App

DDSI/RTPS

UML4DDS

2004

2006

2008

X-T

ype

s 20

10

§  Java 5 DDS PSM §  Simple, Safe, Efficient,

Elegant and Ergonomic API

Copyrig


ll Rights Reserved.

OMG DDS + DDSI/RTPS ¨  Language Independent, OS

and HW architecture independent

¨  Standard API Topic-Based, Real-Time Publish/Subscribe

¨  Built-in support for fail-over, filtering, query, data durability

¨  Standard wire protocol allowing interoperability between different implementations of the DDS standard

Object/Relational Mapping

Ownership DurabilityContent

Subscription

Minimum Profile

Data Centric Publish/Subscribe (DCPS)

Data Local Reconstruction Layer (DLRL)

DDS Interoperability Wire Protocol

Application

UDP/IP

Real-Time Publish/Subscribe Protocol

How does it work?

Copyrig


ll Rights Reserved.

What is Data Distribution?

¨  Data distribution is about making application defined data available where needed and when needed

¨  Data is a first class concept, it can be created, updated, read and eventually disposed

¨  The last value (or last N-values) of a Data is available

Global Data Space

[1/2]

Copyrig


ll Rights Reserved.

What is Data Distribution?

¨  The details of how application data is encapsulated and propagated are dealt by the data distribution technology

[2/2]

Copyrig


ll Rights Reserved.

DDS Topics ¨  The definition of Data in DDS is

tied to the definition of a Topic

¨  A Topic is the association between: ¨  Unique Name

¨  Type

¨  QoS

¨  A Topic can have associated a key in order to express data instances identities

TopicType QoS

Name

com.myco.VehiclePositionTopic!

com::myco::VehiclePosition! {Reliability, ! Deadline, ! Priority, ! Durability} !

Copyrig


ll Rights Reserved.

Topic Types ¨  Topic types can be defined

in IDL, XML, Java

¨  Topics types are extensible and evolvable

¨  Rich set of annotations to deal with fields that might be optional, shared, etc.

¨  Each unique key value identifies a Topic Instance

¨  Topic Instance, can be Created, Read, Updated, and Disposed (CRUD)

struct VehiclePosition{ ! string plate; //@Key ! long x; ! longy; !} !

VehiclePositionVehiclePositionVehiclePosition

plate

“A123”

“B456”

“C789”

x y

101 202

303 202

101 606

TrakCo.VehiclePositionTopic

Topic Type

Topic Instances

Topic Key

Copyrig


ll Rights Reserved.

Topic Instances and Samples ¨  An update for a Topic Instance is called Sample

¨  DDS can maintain a configurable history of samples per Topic Instance

DDS in Action Demo#1

DDS X-Types

Copyrig


ll Rights Reserved.

The Type System Debate ¨  The debate around type Programming Languages Type

Systems has animated computer scientist over several decades and is far from being resolved

¨  New Programming Languages are continuing to appear on both sides of the fence

F#Statically Typed (ex.) Dynamically Typed (ex.)

Copyrig


ll Rights Reserved.

Are you a Type-Phobic Type?

¨  Proponents of Dynamically Typed Programming Languages advocate against strong typing in favor of reduced verbosity and added flexibility

Yet... Are they so sure that a type system always gets between you and what you are trying to achieve?

Copyrig


ll Rights Reserved.

Strongly Typed Type Systems

¨  Well designed type-systems don’t add unnecessary verbosity to your application since use sophisticated inference to derive types

¨  Allow the detection of many errors at compile time, thus improving productivity, safety, code quality and reducing the potential for run-time errors

¨  Enable the generation of more efficient code

Copyrig


ll Rights Reserved.

Types and Distributed Systems ¨  The “Type System debate” has not spared distributed systems

¨  Today we can find different technologies taking different approaches: from completely untyped to strongly typed

¨  The differences are in this case more complex to organize: ¨  Some technologies support only one type, being a string or an array

of bytes (e.g. AMQP) ¨  Other technologies support some predefined types (e.g JMS) ¨  Other technologies allow the definition of user-specified types (e.g

DDS)

Copyrig


ll Rights Reserved.

Types and Distributed Systems T1

T2

T3

T4

T1

T3

T3

T2

T1

T4

T1

T2

?

T1 T2 T3 T4

User Defined Types Example: T1 = RadarTrack T3 = FlightPlanT2 = ClassificationInfo T4 = ...

Copyrig


ll Rights Reserved.

Types and Distributed Systems

T2

T3

T4

T1

T3

T2

T1

T4

T1

T2

T1T3

AMQP

octect sequence

T1 T2 T3 T4

User Defined Types

Pub/Sub Types

Example: T1 = RadarTrack T3 = FlightPlanT2 = ClassificationInfo T4 = ...

Copyrig


ll Rights Reserved.


T2

T3

T4

T1

T3

T2

T1

T4

T1

T2

T1T3

JMS

ObjectMessage TextMessage

ByteMessage

StreamMessage

MapMessage

Pub/Sub Types


Copyrig


ll Rights Reserved.


T2

T3

T4

T1

T3

T2

T1

T4

T1

T2

T1T3

DDS

T2 T3

T4

T1

T3

T1

Pub/Sub Types == User Types!


Copyrig


ll Rights Reserved.

“Pre X-Types” DDS Type System

¨  Nominal Type-System w/o sub-typing

¨  Designed with the Closed-World Assumption in mind, e.g., the information model is global and immutable

¨  Changes to the information model are either non-transparent or non-interoperable with legacy, e.g. require compile/deploy cyce

Copyrig


ll Rights Reserved.

Example ¨  In pre X-Types DDS systems the

RadarTrack and RadarTrack3D cannot be associated with the same Topic by different DDS entities

¨  Nor it is possible for Readers expecting RadarTrack types to match Writers producing RadarTrack3D

struct RadarTrack { ! string id; //@Key ! long x; ! longy; !} !

struct RadarTrack3D { ! string id; //@Key ! long x; ! longy; ! long z; !} !

Copyrig


ll Rights Reserved.

DDS X-Types Type System ¨  Structural Type System

¨  Support extensible and evolvable types

¨  A structural sub-type relationship is used to match DataReader and DataWriters

¨  Annotations + QoS Policy provide control on the sub-typing rule

Copyrig


ll Rights Reserved.

Example ¨  Under the X-Types type-system:

¨  RadarTrack3D <: RadarTrack

¨  As a result Subscribers to RadarTrack match Publishers of RadartTrack3D

¨  How about the dual case? ¨  QoS allow to control whether type

widening are allowed and in provide control on widened attributes have to be initialized

struct RadarTrack { ! string id; //@Key ! long x; ! long y; !} !

struct RadarTrack3D { ! string id; //@Key ! long x; ! long z; ! long y; !} !

Copyrig


ll Rights Reserved.

More on X-Types

The DDS X-Types standard also provides

¨  An API for dynamically registering topic types and creating DataReader/Writer for these types

¨  A rich set of annotations for declaring shared and optional attributes, etc.

X-Types Efficiency

Copyrig


ll Rights Reserved.

Encoding Efficiency Size for Serialized CoFlight Flight Data Plan

DDS encoding is 10x more space efficient than XML!

¨  DDS serialization is several times more efficient than JSON, XML and YAML

¨  Differences can be quite extreme when dealing with complex types

Copyrig


ll Rights Reserved.

Encoding Impact on RTT DDS RTT with different Serialization Format

¨  Inefficient encoding can easily dominate RTT

¨  DDS native encoding is the most efficient among extensible encodings!

Copyrig


ll Rights Reserved.

DDS is very Efficient!

Size (bytes)L

ate

ncy

(u

sec)

¨  DDS implementations are capable of delivering very low and predictable latencies

¨  No other Pub/Sub technology can approach similar level of performance!

ISO C++ PSM

Copyrig


ll Rights Reserved.

PIM/PSM Approach

¨  The DDS standard is composed by a PIM specified in UML and A PSM specified in IDL

¨  Language specific APIs are derived via IDL-based mappings (e.g. IDL2C++, etc.)

PIM (UML)

PSM (IDL)

C JavaC++

IDL2CIDL2C++ IDL2Java

Copyrig


ll Rights Reserved.

What is the Problem?

¨  IDL is very good for describing DDS Topic Types, yet...

¨  IDL’s biggest strength, namely (programming) language independence, becomes its biggest weakness when trying to define APIs that are well integrated with a programming language

¨  For some programming languages (e.g. C++) the IDL2C++ mapping is “seasoned”

Copyrig


ll Rights Reserved.

What is the Problem?

¨  Resulting DDS APIs are more complicated than they should

¨  API don’t feel natural or ergonomic to programmers

¨  API (esp. for C++) don’t integrate well with standard libraries nor implements common idioms/patterns.

Copyrig


ll Rights Reserved.

Native C++/Java PSM ¨  The new C++/Java PSM are

derived starting directly from the PIM

¨  The new C++/Java PSM take advantage of all the features available in C++/Java to provide a natural, productive, efficient, safe and portable API

PIM (UML)

ISO C++ PSM Java 5 PSM

Copyrig


ll Rights Reserved.

ISO C++ PSM in a Nutshell ¨  Provides transparent, safe, precise and real-time automatic memory

management

¨  Takes advantage of C++ Template Meta-programming to automate tasks such as type registration and provide a strongly typed API (no downcast ever!)

¨  Provides a DDS API that exploits Iterators and containers as well as other C++ Standard Types

¨  API is extensible, efficient and portable

Copyrig


ll Rights Reserved.

Type Mapping

¨  DDS primitive types are mapped to C99/C++ standard types

¨  DDS container types are mapped to standard C++ containers, namely std::vector and std::map

Copyrig


ll Rights Reserved.

Topic Type Mapping

struct VehiclePosition {! string plate;! long x;! long y;};#pragma keylist VehiclePosition plate

IDL Topic Type Definition

class VehiclePosition {

! VehiclePosition();! VehiclePosition(const std::string& plate,! ! ! int32_t x, int32_t y);

! virtual ~VehiclePosition();

! const std::string& plate() const;! void plate(const std::string& s);

! const int32_t x() const;! void x(int32_t i);

! const int32_t y() const;! void y(int32_t i);

private:! // Encapsulated state representation};

C++ Topic Type Mapping

Copyrig


ll Rights Reserved.

Writing Samples // ================== DataWriter ===================try { DomainId id = 0; DomainParticipant dp = TheParticipantFactory().create_participant(id);

pub::qos::PublisherQos pqos; pqos << policy::Partition("Tracks");

pub::Publisher pub = dp.create_publisher(pqos);

topic::qos::TopicQos tqos; tqos << policy::Reliability::Reliable() << policy::Durability::Transient() << policy::History::KeepLast(10) << policy::TransportPriority(14);

dds::topic::Topic<RadarTrack> topic = dp.create_topic<RadarTrack>("TrackTopic", "RadarTrack", tqos);

pub::qos::DataWriterQos dwqos(tqos);

pub::DataWriter<RadarTrack> dw = pub.create_datawriter(topic, dwqos);

RadarTrack track("alpha", 100, 200);

dw.write(track); // or dw << track; } catch (const dds::core::Exception& e) { }

Copyrig


ll Rights Reserved.

Reading Samples // ================== DataReader===================

try { DomainId id = 0; DomainParticipant dp = TheParticipantFactory().create_participant(id);

sub::qos::SubscriberQos sqos; sqos << policy::Partition("Tracks");

sub::Subscriber sub = dp.create_subscriber(sqos);

topic::qos::TopicQos tqos; tqos << policy::Reliability::Reliable() << policy::Durability::Transient() << policy::History::KeepLast(10) << policy::TransportPriority(14);

dds::topic::Topic<RadarTrack> topic = dp.create_topic<RadarTrack>("TrackTopic", "RadarTrack", tqos);

sub::qos::DataReaderQos dwqos(tqos);

sub::DataReader<RadarTrack> dr = sub.create_datareader(topic, drqos); std::vector<RadarTrack> samples(MY_MAX_LEN); std::vector<SampleInfo> info(MY_MAX_LEN); dr.read(samples.begin(), info.begin(), MY_MAX_LEN);

} catch (const dds::core::Exception& e) { }

OpenSplice DDS

Copyrig


ll Rights Reserved.

Modern History 2008 • Gartner selects DDS as a Cool Technology and Prismtech/OpenSplice

DDS as the cool vendor among DDS provider

2009 • To propel and accelerate the adoption of DDS beyond Aerospace and

Defense PrismTech Open Sources OpenSplice DDS

2010 • With over 3000 new users in the last year OpenSplice DDS establishes

itself as the most widely used and deployed DDS implementation

OpenSplice DDS#1

Copyrig


ll Rights Reserved.

Product Adoption

¨  ~20000 Unique downloads since October 2009

¨  ~3000 Unique OpenSplice DDS Users

0

5000

10000

15000

20000

Dow

nloa

ds

OpenSplice Downloads

0

750

1500

2250

3000

OpenSplice DDS Registered Users

Oct 2010Oct 2009

Copyrig


ll Rights Reserved.

Product Organization

¨  Four different editions with growing sets of functionalities

¨  The Community Edition is Open Source. Compact, Professional and Enterprise Edition are available through Commercial Subscriptions / Licensing

Enterprise Edition

Professional Edition

Compact Edition

Community Edition

Copyrig


ll Rights Reserved.

Community Edition

¨  OMG DDS v1.2

¨  Real-Time Networking

¨  DDSI/RTPS v2.1

Licensing ¨  LGPLv3


Subscription

Minimum Profile

Real-Time Pub/Sub (DCPS)

Networking Technology

Application

UDP/IP

Interoperable Wire Protocol (DDSI)

DD

SI v

2.1

DD

S v

1.2

Real-Time Networking

Copyrig


ll Rights Reserved.

Compact Edition ¨  All Community Ed. features

¨  Power Tools ¨  Eclipse Visual Modeling Tool

¨  Tuner Tool

Licensing ¨  Commercial Subscription / Licenses

¨  PrismTech Source Code License (non-copy-left)


Subscription

Minimum Profile



Application

UDP/IP


DD

SI v

2.1

DD

S v

1.2


PowerTools

MDE

Tuner

Copyrig


ll Rights Reserved.

Professional Edition

¨  All Compact Ed. Features

¨  DLRL v1.2

¨  Connectors ¨  SOAP





Subscription

Minimum Profile


Object-Oriented Pub/Sub (DLRL)

Application

UDP/IP

DD

SI v

2.1

DD

S v

1.2

SOAP

Connectors

PowerTools

MDE

Tuner




Copyrig


ll Rights Reserved.

Enterprise Edition ¨  All Professional Ed. Features

¨  Security

¨  Connectors

¨  DBMS




Ownership Persistence Content Subscription

Minimum Profile

Real-Time Pub/Sub(DCPS)

Object-Oriented Pub/Sub (DLRL)


Application

UDP/IP


DD

SI

v2.1

DD

S v

1.2

Real-Time NetworkingSOAP Secure Networking

DBMS

Connectors

PowerTools

MDE

Tuner

Connectors

OpenSplice DDS [Users & Use Cases]

Copyrig


ll Rights Reserved.

Some Selected Users

Copyrig


ll Rights Reserved.

Some Use Cases Aerospace & Defense

¨  Naval Combat Management Systems

¨  Submarines

¨  Vetronics (Military Vehicles)

¨  Tactical Links

¨  Simulation

¨  Cybercrime

¨  Flycatcher Systems

¨  Data Fusion

¨  Battle Transformation Center

Copyrig


ll Rights Reserved.

Some Use Cases Transportation

¨  Drones

¨  Air Traffic Control & Management

¨  Metropolitan Transportation

Financial Services

¨  Automated Trading Firms

¨  Risk Management Firms

The Future

Copyrig


ll Rights Reserved.

DDS Standard Evolution

DDS

App Standard API

DDSI/RTPS network


UML4DDS

ISO-C++ PSM 2010

Java5 PSM 2010 2004

2006

2008

X-T

ype

s 20

10

DDS

App

DDSI/RTPS

UML4DDS

2004

2006

2008

X-T

ype

s 20

10

Copyrig


ll Rights Reserved.


DDS

App Standard API

DDSI/RTPS network


UML4DDS

ISO-C++ PSM 2010

Java5 PSM 2010 2004

2006

2008

X-T

ype

s 20

10

DDS

App

DDSI/RTPS

UML4DDS

2004

2006

2008

X-T

ype

s 20

10

We

b-D

DS

2011

We

b-D

DS

2011

§  Standardized way of accessing DDS from Web Technologies

§  REST, W3C WS-*, RSS, etc.

Copyrig


ll Rights Reserved.


DDS

App Standard API

ULS-DDSI network


UML4DDS

ISO-C++ PSM 2010

Java5 PSM 2010 2004

2011

2008

X-T

ype

s 20

10

DDS

App

ULS-DDSI

UML4DDS

2004

2011

2008

X-T

ype

s 20

10

We

b-D

DS

2011

We

b-D

DS

2011

§  Ultra-Large Scale Extensions to the DDSI/RTPS wire-protocol

§  New Discovery §  Support for TCP, etc.

Copyrig


ll Rights Reserved.

ULS-DDSI ULS-DDSI


DDS

App Standard API

network


UML4DDS

ISO-C++ PSM 2010

Java5 PSM 2010 2004

2006

2008

X-T

ype

s 20

10

DDS

App

UML4DDS

2004

2006

2008

X-T

ype

s 20

10

We

b-D

DS

2011

We

b-D

DS

2011

§  Interoperable Security Architecture

§  Pluggable Policy / Labeling and Tagging

Sec

urity

20

12

Sec

urity

20

12

Copyrig


ll Rights Reserved.

ULS-DDSI ULS-DDSI


DDS

App

Standard API

network


UML4DDS

ISO-C++ PSM 2010

Java5 PSM 2010 2004

2006

2008

X-T

ype

s 20

10

DDS

App

UML4DDS

2004

2006

2008

X-T

ype

s 20

10

We

b-D

DS

2011

We

b-D

DS

2011

§  Remote Method Invocations over DDS

§  Synchronous, Asynchronous and one ways RMI

Sec

urity

20

12

Sec

urity

20

12

DDS-RMI 2012

DDS-RMI 2012

Copyrig


ll Rights Reserved.

Concluding Remarks ¨  The DDS specification provides you the most

advanced middleware infrastructure for building mission-critical systems

¨  The Specification is evolving to support System of Systems and Cloud Messaging

¨  OpenSplice DDS is emerging as the leading implementation in the DDS market, quickly expanding well beyond its traditional core market

http://www.opensplice.com/

http://www.opensplice.org/

emailto:[email protected]

http://www.youtube.com/OpenSpliceTube http://opensplice.blogspot.com

http://bit.ly/1Sreg

http://www.slideshare.net/angelo.corsaro

http://twitter.com/acorsaro/

D e l i v e r i n g P e r f o r m a n c e , O p e n n e s s , a n d F r e e d o m

OpenSplice DDS

the present and future of dds

Technology

dds topics

dds standard api

dds sig

dds standard udpip

dds xtypes

dds ddsxtypesx

topic instancea123

dds psm2008 simple