tweeting with opensplice dds

47
Angelo Corsaro, Ph.D. Chief Technology Officer PrismTech OMG DDS SIG Co-Chair [email protected] :: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com :: Splice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS Splice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS Tweeting with OpenSplice DDS A w alk a bout on using Open Splice DDS in the Cloud

Upload: angelo-corsaro

Post on 26-Jun-2015

1.815 views

Category:

Technology


0 download

DESCRIPTION

"angelo corsaro","cloud messaging",dds,"omg dds","opensplice dds",twitter

TRANSCRIPT

Page 1: Tweeting with OpenSplice DDS

Angelo Corsaro, Ph.D.Chief Technology Officer

PrismTechOMG DDS SIG Co-Chair

[email protected]

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS

OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS

Tweeting with

OpenSplice DDSA

walk

about on u

sing

OpenS

plice DDS

in

the Clo

ud

Page 3: Tweeting with OpenSplice DDS

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

What is DDS?

‣ A Standard for High-Performance Publish/Subscribe

‣ Defines a standard API (for mainstream languages) and a Wire-Protocol

‣ DDS uses a highly efficient and extensible binary protocol and encoding

Object/Relational Mapping

Ownership DurabilityContent

Subscription

Minimum Profile

Data Centric Publish/Subscribe (DCPS)

Data Local Reconstruction Layer (DLRL)

DDS Interoperability Wire Protocol

Application

UDP/IP

Real-Time Publish/Subscribe Protocol

Page 4: Tweeting with OpenSplice DDS

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

What is DDS?

‣ Used heaving in complex mission/business critical systems

‣ Transportation ‣ Aerospace & Defense‣ Financial Market‣ SCADA & Utilities‣ ...

Object/Relational Mapping

Ownership DurabilityContent

Subscription

Minimum Profile

Data Centric Publish/Subscribe (DCPS)

Data Local Reconstruction Layer (DLRL)

DDS Interoperability Wire Protocol

Application

UDP/IP

Real-Time Publish/Subscribe Protocol

Page 5: Tweeting with OpenSplice DDS

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

How Does it Work?

‣ DDS is based around the concept of a fully distributed Global Data Space (GDS)

‣ Applications can autonomously and asynchronously read/written data in the GDS

Brokers

Note: DDS can be seen as a

relaxation of the LINDA

coordination model

DDSGlobal Data Space

Page 6: Tweeting with OpenSplice DDS

Brokers

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

How Does it Work?

‣ Publishers and Subscribers can join and leave the GDS at any time

Subscriber

Subscriber

Subscriber

Publisher

Publisher

Publisher

DDSGlobal Data Space

Page 7: Tweeting with OpenSplice DDS

Brokers Subscriber

Subscriber

Subscriber

Publisher

Publisher

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

How Does it Work?

‣ Publishers and Subscribers express their intent to produce/consume specific type of data, e.g., Topics

Publisher

DDSGlobal Data Space

Page 8: Tweeting with OpenSplice DDS

Brokers Subscriber

Subscriber

Subscriber

Publisher

Publisher

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

How Does it Work?

‣ Subscriptions are matched by taking into account data types and QoS

Publisher

DDSGlobal Data Space

Page 9: Tweeting with OpenSplice DDS

Brokers

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

How Does it Work?

‣ Subscriptions are dynamically matched and Data flows from Publisher to Subscribers

Subscriber

Subscriber

Subscriber

Publisher

Publisher

Publisher

DDSGlobal Data Space

Page 11: Tweeting with OpenSplice DDS

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

A “Tweet” with DDS

Topic:

‣ Unit of information exchanged between Publisher and Subscribers.

‣ An association between a unique name, a type and a QoS setting TweetType

Tweet

{ Reliable, Persistent, ...}

TopicType QoS

Name

Page 12: Tweeting with OpenSplice DDS

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

A “Tweet” with DDS

Topic Type:

‣ Type describing the data associated with one or more Topics

‣ A Topic type can have a key represented by an arbitrary number of attributes

‣ Expressed in IDL (or XML)

struct TweetType { string userId; string tweet;};#pragma keylist Tweet userId

Page 15: Tweeting with OpenSplice DDS

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

Tweeting With DDS

@bird

@oiseau

@ocell

@vogel

@lintu

@lind

{@ocell, @oiseau}

{@vogel}

@birdTopic

@oiseauTopic

@lindTopic@lintuTopic

@vogelTopic

@ocellTopic‣ Each Tweeter

can be represented with a unique topic {@oiseau, @bird, ocell}

Page 16: Tweeting with OpenSplice DDS

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

Tweeting With DDS

@bird

@oiseau

@ocell

@vogel

@lintu

@lind

{@ocell, @oiseau}

{@vogel}

@TweetTopic@TweetTopic

@TweetTopic

@TweetTopic

@TweetTopic

@TweetTopic

tweeter.bird tweeter.ocel

tweeter.oiseau

tweeter.lind

tweeter.vogel

tweeter.lintu

‣ ...or each Tweeter can be represented with a unique partition

‣ All tweeters use the same topic

Note: DDS partitions can

be specified by using

wildcards, e.g. tweeter.*

{@oiseau, @bird, ocell}

Page 17: Tweeting with OpenSplice DDS

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

Tweeting With DDS

@bird

@oiseau

@ocell

@vogel

@lintu

@lind

{@ocell, @oiseau}

{@vogel}

@TweetTopic

‣ ...or all Tweeters use the same topic

‣ Content filtering or queries are used to extract relevant data

{@oiseau, @bird, ocell}

Page 18: Tweeting with OpenSplice DDS

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

Tweeting With DDS

dds::Topic<TweetType> topic("TweetTopic");

std::vector<std::string> partitions = {"tweeter.ocel", "tweeter.oiseau"};

dds::PublisherQos pubQoS;pubQoS.set_partition(partitions);dds::Publisher pub(pubQoS);

dds::DataReader<TweetType> dr(pub);

std::vector<TweetType> data;std::Vector<SampleInfo> info;

dr.read(data, info);

dds::Topic<TweetType> topic("TweetTopic");

dds::PublisherQos pubQoS;pubQoS.set_partition("Tweeter.bird");dds::Publisher pub(pubQoS);

dds::DataWriter<TweetType> dw(pub);

TweetType tt = { "@bird", "Writing next-gen tweeter in DDS"};

dw.write(tt);

Writing Tweets Reading Tweets

Page 19: Tweeting with OpenSplice DDS

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

Are you a Persistent Tweeter?

‣ How do we deal with persistence?

‣ How do we deal with Time Decoupling?

‣ Tweeter will always want to see the history of what received

@bird

@oiseau

@ocell

{@ocell, @oiseau}

DDSGlobal Data Space

Page 27: Tweeting with OpenSplice DDS

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

Yes, I am More than Persistent...

‣ OpenSplice DDS provides a native (non-dbms-based) high performance persistence implementation

‣ Data Persistence can be controlled at a very fine granularity to tune scalability, performance and resource usage

Let’s see this in act

ion!

Let’s see this in act

ion!Let’s see this in a

ction!

Let’s see this

in action!

Page 28: Tweeting with OpenSplice DDS

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

What Else Can I Do?

‣ Content Filtering

‣ Queries

‣ Security

‣ Connection to DBMS

‣ Connector to Apache CAMEL

‣ REST Connector

‣ ... Apache Camel

DBMS

RESTfulConnector

Page 30: Tweeting with OpenSplice DDS

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

Why the Cloud?

‣ Sometimes you need to scale your application world-wide

‣ Relying on proprietary infrastructure can be very expensive

‣ For some small companies this is not an option either

Page 31: Tweeting with OpenSplice DDS

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

Why OpenSplice DDS on the Cloud?

‣ Adopt a standard-based and Open Source technology to avoid lock-in int Cloud-Provided Messaging/Caching/Data Distribution

‣ Leverage OpenSplice DDS high performance within the cloud

‣ Leverage OpenSplice DDS scalability to Ultra Large Scale Systems across clouds

‣ Leverage OpenSplice DDS technology ecosystem to get your data where you need and how you need (DBMS, REST, etc.)

Page 32: Tweeting with OpenSplice DDS

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

tweet... tweet...

tweet...tweet...

tweet...

DDS

DDS

DDStweet...DDS

tweet...

DDS

DDStweet...

Page 33: Tweeting with OpenSplice DDS

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

World Wide Tweeting with OpenSplice

‣ OpenSplice within and across centers ensures performance and scalability

‣ Rich set of OpenSplice DSD connectors

tweet... tweet...

tweet...

tweet...

tweet...

DDS DDS

DDStweet...DDS

tweet...

DDS

DDStweet...

Page 39: Tweeting with OpenSplice DDS

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

Air Traffic Control

CoFlight eFDP‣ Next-Generation European Flight

Data Processor

‣ All data distribution performed using OpenSplice DDS

‣ Deployed starting from 2011 in France, Italy, and Switzerland

ATCCBrest

ATCCReimsATCC

Paris

ATCCMilano

ATCCBrindisi

ATCCAix-en-

Provence

ATCCBordeaux

ATCCRoma

ATCCPadova

ATCCGeneve

ATCCZurich

Some Numbers‣ Hundreds of DDS Publisher/Subscribers per Centers‣ 11 Centers Spread across 3 Countries

Page 40: Tweeting with OpenSplice DDS

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

Air Traffic Control

SESAR

‣ Operational integration of all Air Traffic Control Centers Pan-European

‣ DDS selected as the standard to distribute real-time information Pan-EU

PENS(Pan-Eropean Network Service)

Page 42: Tweeting with OpenSplice DDS

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

OpenSplice DDS

‣ Open Source implementation of the full OMG DDS Standard

‣ Commercially supported by PrismTech

‣ Four different supported Editions to address the from the most typical to the most advanced users

Enterprise Ed.

Community Ed.

Compact Ed.

Professional Ed.

Page 43: Tweeting with OpenSplice DDS

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

OpenSplice DDS Performances

!"#$%&#'

()(!%(*'

&")&%+)'

&!(,%!)'

),,%$#'

!,*%))'(("%!"'

&&)%*$' $"%"$' ,+%($',($%!+'

,*+%&)'

$&"%,,'

*!#%,&'

)$,%$*'

)"#%*)'

"!+%++'

"*(%)+' ")(%&!' ""&%,&'

,++%++'

!++%++'

$++%++'

#++%++'

*++%++'

)++%++'

"++%++'

&+++%++'

&&++%++'

(+%++'

&+(+%++'

(+(+%++'

,+(+%++'

!+(+%++'

$+(+%++'

#+(+%++'

)' &#' ,(' #!' &()' ($#' $&(' &+(!' (+!)' !+"#'

!"#$%

&'$($)$*+%

!*$$,(*%-./*%

-./0/1/23' 456/'

Test Scenario

‣ Single Threaded Application (multi-threaded networking service)

‣ 8192 bit message batches

ThroughputLatency

Inter-Node Latency

‣ 60 usec

Inter-Core Read-Latency

‣ 2 usec

Inter-Core Latency

‣ <10 usec

HW:

‣ Dell blade-server

‣ Dual-core, Dual-CPU, AMD Opteron 2.4 GhzOS

‣ Linux 2.6.21-1.3194.fc7Network

‣ Gigabit Ethernet cards

‣ Dell PowerConnect 5324 switch

Page 45: Tweeting with OpenSplice DDS

Ope

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

SOpe

nSpl

ice

DD

S

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

Key Points

‣ OpenSplice DDS is a very powerful Open Source and Standard/Based High Performance Data Distribution Technology

‣ OpenSplice DDS is widely used in systems at any scale, from embedded to ultra large scale systems

‣ OpenSplice DDS provides a rich set of features that make it the ideal choice as the data distribution technology for a traditional deployment and equally for a Cloud-based deployment

Page 47: Tweeting with OpenSplice DDS

:: http://www.opensplice.org :: http://www.opensplice.com :: http://www.prismtech.com ::

Tweeting with

OpenSplice DDSA

walk

about on u

sing

OpenS

plice DDS

in

the Clo

ud

THANK You!P.S. Enjoyed the talk? Found it

Bloody boring? Send me your

comments at:

[email protected]