advanced opensplice programming - part i

Post on 15-Jan-2015

1.128 Views

Category:

Technology

5 Downloads

Preview:

Click to see full reader

DESCRIPTION

OpenSplice DDS enables seamless, timely, scalable and dependable data sharing between distributed applications and network-connected devices. Its technical and operational benefits have propelled adoption across multiple industries, such as Defence and Aerospace, SCADA, Gaming, Cloud Computing, Automotive, etc. If you want to learn about OpenSplice DDS or discover some of its advanced features, this webcast is for you! In this two-parts webcast we will cover all the aspects tied to architecting and developing OpenSplice DDS systems. We will look into Quality of Services, data selectors concurrency and scalability concerns. We will present the brand-new, and recently finalized, C++ and Java APIs for DDS, including examples of how this can be used with C++11 features. We will show how, increasingly popular, functional languages such as Scala can be used to efficiently and elegantly exploit the massive HW parallelism provided by modern multi-core processors. Finally we will present some OpenSplice specific extensions for dealing very high-volumes of data – meaning several millions of messages per seconds.

TRANSCRIPT

Ope

nSpl

ice

DD

S

Angelo CORSARO, Ph.D.Chief Technology Officer OMG DDS Sig Co-Chair

PrismTechangelo.corsaro@prismtech.com

Advanced OpenSplice DDS Programming- Part I -

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Before we get into technical details let’s try to answer two questions:

1. What does OpenSplice DDS do?

2. What kind of applications are using it?

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

OpenSplice DDS enables seamless, loosely-coupled, timely, scalable and dependable data sharing between distributed applications and network connected devices.

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Some Use Cases

Integrated Modular Vetronics Training & Simulation Systems Naval Combat Systems

Air Traffic Control & Management Unmanned Air Vehicles Aerospace Applications

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Some Use Cases

Agricultural Vehicle Systems

Train Control Systems Complex Medical Devices

Smart CitiesLarge Scale SCADA Systems

Big Data (In-Memory) Analytics

Ope

nSpl

ice

DD

S

A Fresh Look at OpenSplice

Ope

nSpl

ice

DD

S

From Local to Distributed Data

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Data in Programming Languages

☐ Statically Typed Programming Languages provide mechanisms to declare a variable for given type as well as defining new types

☐ Scoping rules are then used to define what values and types are accessible at any given point in a program text

   int  x;      class  Rational  {  ...  }  

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

“Local” Data☐ Consider the Following code fragment below in an imperative

programming language:

12345678910111213

{    int  x;    x  =  7;    {        int  y;        y  =  18;        double  z;        z  =  3.141593        x  =  x  -­‐  4;        y  =  x  +  y;        z  =  2  *  z  *  y;    }}

State  {  }  {  int  x  }  {  x  =  7  }  {  x  =  7,  {  }  }  {  x  =  7,  {  int  y  }  }  {  x  =  7,  {  y  =  18  }  }  {  x  =  7,  {  y  =  18,  double  z  }  }  {  x  =  7,  {  y  =  18,  z  =  3.1416  }  }  {  x  =  3,  {  y  =  2  }  }  {  x  =  3,  {  y  =  21  }  }  {  x  =  3,  {  y  =  21,  z  =  131.9472  }  }  {  x  =  3,  {    }}  {  }

12345678910111213

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

“Local” Data☐ One final consideration to do is that variables in a programming

language are in essence alias to a memory location

int  x;int  y;

...

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Distributed Data☐ Extending the elegant model provided by programming

languages to distributed systems has been and still is a quite popular research topic

☐ Historically the solutions to the distributed data problem where divided into those based on Distributed Shared Memory and those based on Message Passing

☐ OpenSplice DDS provide a new interesting way of addressing this problem. Let’s see...

Ope

nSpl

ice

DD

S

OpenSplice Foundations

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Distributed Data in OpenSplice☐ OpenSplice DDS provides the abstraction

of an “Active” Global Data Space

☐ The Global Data Space is fully distributed and does not suffer Single Point of Failure issues

☐ Applications can autonomously join and leave this data space as well as declare new “Data Types”

☐ In OpenSplice, “data types” are called Topics

OpenSplice DDS Global Data Space

...

TopicA

TopicB

TopicC

TopicD

TopicE

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Global Data Space☐ The Global Data Space is actually called “Domain” in

OpenSplice DDS

☐ Domains are identified by integer. An application can join a domain by creating a domain participant as follows:

OpenSplice DDS Global Data Space

...

TopicA

TopicB

TopicC

TopicD

TopicE//  Create  a  Topicint  domain_id  =  18;auto  dp  =  DomainParticipant(domain_id);

☐ A Domain defines a top-level scope thus the content of a domain is not accessible from another domain

☐ Finally an application can join multiple domains by creating one domain participant per-domain

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Topics☐ A Topic is defined by means of a

(name, type, qos) tuple, where ☐ name: identifies the topic within

OpenSplice’s Global Data Space

☐ type: is the programming language type associated with the topic. This type can be defined in IDL, XML, and Java. Types are extensible and evolvable

☐ qos: is a collection of policies that control non-functional properties of this topic, such as its durability, reliability, etc.

DURABILITY,DEADLINE,PRIORITY,

“Circle”, Square”, “Triangle”

TopicTypeName

QoS

ShapeType

struct ShapeType { string color; long x; long y; long size;};

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Defining a Topic

☐ In code, if we wanted to define the Square topic we would do (with the new DDS-PSM-Cxx v1.0 API):

// Create a Topicauto dp = DomainParticipant(18);auto topic = Topic<ShapeType>(dp, “Square”, dp.default_topic_qos());

// Or equivalently and more concisely:auto topic = Topic<ShapeType>(dp, “Square”);

☐ The effect of this code is to define a topic called “Square” with type ShapeType and default QoS within the OpenSplice Global Data Space

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Looking with the Spyglassdomain 18

auto topic = Topic<ShapeType>(dp, “Square”);

Squarestruct&ShapeType&{&&string&&&color;&&&&&&&&long&&&x;&&&&&long&&&y;&&long&&&size;};

Policies

Topic

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Topics Scope☐ Topic definitions are (somewhat) equivalent to type definitions in a programming

language. In the sense that they define a new, generalized, type to be used within the global data space

☐ The scope of a Topic definition is the entire Domain, thus one could say that there is only one level of scoping. To make the parallel with Java, one could say that it is as if Topics where defined in a namespace named after the domain-id

domain0  {TopicA,  TopicB,  ...}

domain1  {Topic1,  Topic2,  ...}

☐ As a consequence topic definitions in one domain are not visible from a different domain

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

If Topic are Types what are Variables?

☐ As described earlier a Topic definition can be thought as a Type definition in a programming language

☐ In a statically typed programming language (e.g. Java), given a type Foo I can define a variable of type Foo as:

   Foo  x;  

x  =  new  Foo(...);  ☐ Then I can initialize it as:

☐ How are instance of a Topic defined?

☐ Then I can mutate it, by: x.setBar(...);

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DataWriter

☐ OpenSplice DDS uses DataWriters to declare and write instances of a Topic

☐ A DataWriter for our Square topic can be defined as follows:

// Create a Publisherauto pub = Publisher(dp);

// Create a DataWriterauto dw = DataWriter<ShapeType>(pub, topic);

// Write datadw.write(ShapeType(“RED”,11,77,99));

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Looking with the Spyglass

auto dw = DataWriter<ShapeType>(pub, topic);

Squarestruct&ShapeType&{&&string&&&color;&&&&&&&&long&&&x;&&&&&long&&&y;&&long&&&size;};

Policies

Topic

color sizex y

dw

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Looking with the Spyglass

Squarestruct&ShapeType&{&&string&&&color;&&&&&&&&long&&&x;&&&&&long&&&y;&&long&&&size;};

Policies

Topic

color sizex y

"RED" 9911 77

dw

dw.write(ShapeType(“RED”,11,77,99));

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

An Experiment...☐ What does the following code do?

// Create a Publisherauto pub = Publisher(dp);

// Create DataWritersauto dw1 = DataWriter<ShapeType>(pub, topic);auto dw2 = DataWriter<ShapeType>(pub, topic);

// Write Datadw1.write(ShapeType(“RED”,10,20,30));dw2.write(ShapeType(“BLUE”,30,20, 10));

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Looking with the Spyglass

Squarestruct&ShapeType&{&&string&&&color;&&&&&&&&long&&&x;&&&&&long&&&y;&&long&&&size;};

Policies

Topic

color sizex y

dw2

dw1

auto dw1 = DataWriter<ShapeType>(pub, topic);auto dw2 = DataWriter<ShapeType>(pub, topic);

Data Writers for the same topic are aliased to the same variable!

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Looking with the Spyglass

dw1.write(ShapeType(“RED”,10,20,30));

Squarestruct&ShapeType&{&&string&&&color;&&&&&&&&long&&&x;&&&&&long&&&y;&&long&&&size;};

Policies

Topic

color sizex y

RED 3010 20

dw2

dw1

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Looking with the Spyglass

dw2.write(ShapeType(“BLUE”,30,20, 10));

Squarestruct&ShapeType&{&&string&&&color;&&&&&&&&long&&&x;&&&&&long&&&y;&&long&&&size;};

Policies

Topic

color sizex y

RED 1030 20

dw2

dw1

Squarestruct&ShapeType&{&&string&&&color;&&&&&&&&long&&&x;&&&&&long&&&y;&&long&&&size;};

Policies

Topic

color sizex y

BLUE 1030 20

dw2

dw1

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Topic Instances☐ Topic type can identify an arbitrary number of attributes as defining the

topic key

☐ In this case, each unique key value defines a topic instance

☐ Example:

struct ShapeType { @Key long color; long x; long y; long size;};

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Let’s Experiment...☐ What does the following code do?

// Create a Publisherauto pub = Publisher(dp);

// Create DataWritersauto dw = DataWriter<ShapeType>(pub, topic);

// Write Datadw.write(ShapeType(“RED”,10,20,30));dw.write(ShapeType(“BLUE”,30,20,10));

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Looking with the Spyglass

Squarestruct&ShapeType&{&&string&&&color;&&&&&&&&long&&&x;&&&&&long&&&y;&&long&&&size;};

Policies

Topic

color sizex y

RED 3010 20

dw

dw.write(ShapeType(“RED”,10,20,30));

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Looking with the Spyglass

Squarestruct&ShapeType&{&&string&&&color;&&&&&&&&long&&&x;&&&&&long&&&y;&&long&&&size;};

Policies

Topic

color sizex y

RED 3010 20

dw

BLUE 1030 20

dw.write(ShapeType(“BLUE”,30,20,10));

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

LongVariable Example

struct LongVariable { @Key string name; long value; };

// Create a Publisherauto lvt = Topic<LongVariable>(dp, “TLongVariable”);

// Create DataWritersauto dw = DataWriter<LongVariable>(pub, topic);

// Write Datadw.write(LongVariable(“x”,10));dw.write(LongVariable(“y”,20));

☐ The “distributed” version of the following code:int  x;int  y;x  =  10;y  =  20;

☐ Becomes: Topic Type

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Looking with the Spyglass

TLongValue

Policies

Topic

dw

struct LongVariable { @Key string name; long value; };

valuename

10x

20y

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DataReader

☐ OpenSplice DDS uses DataReaders to read instances of a Topic

☐ A DataReader for our Square topic can be defined as follows:

// Create a Subscriberauto sub = Subscriber(dp);

// Create a DataWriterauto dr = DataReader<ShapeType>(sub, topic);

// Read dataauto data = dr.read();

☐ The read operation will read by default all instance values. Special selector operation are available to pick a specific instance. More later

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DataReader/DataWriter Matching

☐ DataWriters and DataReaders are automatically and dynamically matched by the DDS Dynamic Discovery

☐ A rich set of QoS Policies allows to control existential, temporal, and spatial properties of data and impacts DataReader/DataWriter matching DDS Global Data Space

...

TopicA

TopicBTopicC

TopicD

Data Writer

Data Writer

Data Writer

Data Writer

Data Reader

Data Reader

Data Reader

Data Reader

Ope

nSpl

ice

DD

S

Publishers and Subscribers

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Publishers / Subscribers

Publishers and Subscribers in DDS play two roles:

☐ Coordinate actions across DataReaders/DataWriters☐ e.g. Atomic writes across data-writers

☐ Control access to DDS partitions, a mechanism provided by DDS to further

Domain (e.g. Domain 123)

Domain Participant

Topic

Publisher

DataWrter

Subscriber

DataReader

Partition (e.g. “Telemetry”, “Shapes”, )

Topic Instances/Samples

TaTb

Tc

Tx

Ty

T1

T1 T3

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DDS Partitions☐ DDS Publishers and

Subscribers can specify a list of regular expressions representing the partitions to join

☐ Actual partition can be any string, but some structure can be given in order mimic hierarchy

M:0:0 M:0:1 M:0:2

M:1:0 M:1:1 M:1:2

M:2:0 M:2:1 M:2:2

Domain

Partitions

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

M:0:0 M:0:1 M:0:2

M:1:0 M:1:1 M:1:2

M:2:0 M:2:1 M:2:2

Domain

Partitions

DDS Partitions Matching

M:0:*

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

M:0:0 M:0:1 M:0:2

M:1:0 M:1:1 M:1:2

M:2:0 M:2:1 M:2:2

Domain

Partitions

DDS Partitions Matching

M:*:2

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

M:0:0 M:0:1 M:0:2

M:1:0 M:1:1 M:1:2

M:2:0 M:2:1 M:2:2

Domain

Partitions

DDS Partitions Matching

M:0:0, M:1:1, M:2:2

Ope

nSpl

ice

DD

S

Anatomy of a DDS Application

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Anatomy of a DDS Application

Domain

Reader/Writers for User Defined for Types

Session

// Create a DataWriter/DataWriterauto writer = DataWriter<ShapeType>(pub, topic);auto reader = DataReader<ShapeType>(sub, topic);

Reader/Writer for application defined

Topic Types

Domain Participant

Publisher

DataWriter

Topic Subscriber

DataReader

[DDS C++ API 2010]

auto dp = DomainParticipant(domainId);

// Create a Topicauto topic = Topic<ShapeType>(dp, “Circle”);// Create a Publisher / Subscriberauto pub = Publisher(dp);auto sub = Subscriber(dp);

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Anatomy of a DDS Application

Domain

Reader/Writers for User Defined for Types

Session

// Write datawriter.write(ShapeType(“RED”, 131, 107, 89));// But you can also write like this...writer << ShapeType(“RED”, 131, 107, 89);

// Read new data (loaned)auto data = reader.read();

Reader/Writer for application defined

Topic Types

auto dp = DomainParticipant(domainId); Domain

Participant

Publisher

DataWriter

Topic Subscriber

DataReader

// Create a Topicauto topic = Topic<ShapeType>(dp, “Circle”);// Create a Publisher / Subscriberauto pub = Publisher(dp);auto sub = Subscriber(dp);

[DDS C++ API 2010]

Ope

nSpl

ice

DD

S

Data Reader/Writer Caches

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

write( ) Application Application

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Topic Instance

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Topic Instance Topic Instance

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Applicationwrite( )

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Writer History

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Reader History Reader History

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Applicationwrite( )

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

‣History‣Destination Order‣ Presentation‣ Partition‣ Time Based Filter‣Ownership

QoS Policies

‣Reliability‣History‣ Latency Budget

QoS Policies‣Durability‣ Transport Priority‣ Time Based Filter

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Writer Cache Properties☐ An entry for each topic instance

☐ Each cache entry might contain one or more samples, the number of samples is controlled by the History QoS Policy

☐ Depending on the setting of the Durability QoS Policy, the content of the cache may be made available to late joiners. The depth of the cache controls how many samples per instance will be delivered to late joiners

DataWriter Cache

DataWriter

...

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader Cache Properties☐ An entry for each topic instance

☐ Each cache entry might contain one or more samples, the number of samples is controlled by the History QoS Policy☐ History.KeepLast(n) keeps the last n samples

per instance☐ History.KeepAll maintains all the samples,

modulo resource constraints

☐ For a given instance:☐ Samples from the same writer are inserted in

write-order☐ Samples from multiple-writers are ordered

using time-stamps (source or reception depending on configurable a QoS Policy)

DataReader Cache

DataReader

...

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Eventual Consistency☐ DDS caches provide eventual consistency semantics

☐ This means that a read will see the effect of a preceding write eventually

☐ Furthermore, given a data-writer that is currently matching N readers, we can think of DDS as providing eventual consistency with W=0 and R=1☐ W: the number of Acks expected in order to return from a write☐ R: the number of sources from which a read access data

Ope

nSpl

ice

DD

S

Understanding History

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

write( )1

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

1

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

1

1

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

1 1 1

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

1 1 1

write( )2

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

2 1 1

Application

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

2 1 1

Application

2

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

2 1 1

Application

2 2

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

2 1 12 2

write( )3

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

3 1 12 2

Application

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

3 1 12 2

Application

3

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

3 2 23 3

Application

Ope

nSpl

ice

DD

S

Caches or Streams?

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Dynamic View of a Stream

Stream: Set of samples written over time for a given topic instance.

...

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Eventual View of a Stream

‘Past’ Samples

Stream: Set of samples written over time for a given topic instance.

Reader HistoryWriter History

Assumptions (Default Settings): Reader History = KeepLast (1) WriterHistory = KeepLast (1)

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Eventual View of a Stream

‘Past’ Samples

Stream: Set of samples written over time for a given topic instance.

Reader HistoryWriter History

Assumptions: Reader History = KeepLast (n) with n > 1WriterHistory = KeepLast (1)

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Eventual View of a Stream

‘Past’ Samples

Stream: Set of samples written over time for a given topic instance.

Reader HistoryWriter History

Assumptions: Reader History = KeepLast (n) with n > 1WriterHistory = KeepLast (m) with n > m > 1

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Eventual View of a Stream

Reader HistoryWriter History

Assumptions: Reader History = KeepAllWriterHistory = KeepLast (m) with n > m > 1

...

Ope

nSpl

ice

DD

S

Interacting with the DataReader Cache

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DataReader Cache

DataReader

...

Reading Data Samples☐ Samples can be read from the Data Reader History Cache

☐ The action of reading a sample is non-destructive. Samples are not removed from the cache

DataReader Cache

DataReader

... read

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Taking Data Samples☐ Samples can be taken from the Data Reader History Cache

☐ The action of taking a sample is destructive. Samples are removed from the cache

DataReader Cache

DataReader

...

DataReader Cache

DataReader

... take

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Read vs. Take

☐ The read operation should always be access the latest know value for topics that represent distributed state

☐ The take operation should be used to get the last notification from a topic that represent an event

Ope

nSpl

ice

DD

S

Data Selectors

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Cherry Picking in DDS

☐ DDS provides some very flexible mechanisms for selecting the data to be read:☐ Data Content☐ Data Status

☐ These mechanisms are composable

Ope

nSpl

ice

DD

S

Content-Based Data Selection

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Filters and Queries☐ DDS Filters allow to control what gets

into a DataReader cache

☐ DDS Queries allow to control what gets out of a DataReader cache

☐ Filters are defined by means of ContentFilteredTopics

☐ Queries operate in conjunction with read operations

☐ Filters and Queries are expressed as SQL where clauses

DataReader Cache

DataReader

...... ... ...

Filter

Query

Application

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Filters/** * NOTE: The Scala API if not provided with DP/Sub/Pub assumes * default domains and default partition. **/// Create a Topicauto topic = Topic<ShapeType>(dp, “Circle”);

// Define filter expression and parametersauto filter = Filter(“x < 100 AND y < 200”);

// Define content filtered topicauto cftopic = ContentFilteredTopic<ShapeType>(“CFCircle”, topic, filter)

// Create a DataReader for the content-filtered Topicauto dr = DataReader<ShapeType>(sub,cftopic)

[C++ API]

struct ShapeType { @Key string color; long x; long y; long shapesize;};

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Query

// Define filter expression and parametersauto dr = DataReader<ShapeType>(sub, topic) val query = Query(dr, “x < 100 AND y < 200”);

dr.select() .content(query) .read();

struct ShapeType { @Key string color; long x; long y; long shapesize;};[C++ API]

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Instances☐ DDS provides a very efficient way of reading data belonging to a

specific Topic Instance

☐ Obviously, one could use queries to match the key’s value, but this is not as efficient as the special purpose instance selector

auto handle = dr.lookup_instance(ShapeType(“RED”, 0, 0, 0));

auto data = dr.select() .instance(handle) .read();

Ope

nSpl

ice

DD

S

State-Based Selection

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Sample, Instance, and View State☐ The samples included in the DataReader cache have associated

some meta-information which, among other things, describes the status of the sample and its associated stream/instance

☐ The Sample State (READ, NOT_READ) allows to distinguish between new samples and samples that have already been read

☐ The View State (NEW, NOT_NEW) allows to distinguish a new instance from an existing one

☐ The Intance State (ALIVE, NOT_ALIVE_DISPOSED, NOT_ALIVE_NO_WRITERS) allows to track the life-cycle transitions of the instance to which a sample belongs

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

State Selector in Action

// Read only new samplesauto data = dr.read()

// Read any samples from live instancesauto data = dr.select() .state(DataState::any_data()) .read();

[C++ API]

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

auto data = dr.select() .content(query) .state(data_state) .instance(handle) .read();

Putting all Together

☐ Selectors can be composed in a flexible and expressive manner

[C++ API]

Ope

nSpl

ice

DD

S

Summing Up

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Concluding Remarks☐ OpenSplice DDS provides a very powerful abstraction for building

distributed applications grounded on the concept of a fully distributed global data space

☐ Topics are used to define the types that populate the global data space while DataWriters and DataReaders are used to produce and consume values

☐ DataReaders allow very flexible data access through cache access operations

Ope

nSpl

ice

DD

S

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

References

¥Fastest growing JVM Language¥Open Source¥www.scala-lang.org

¥ #1 OMG DDS Implementation¥ Open Source¥ www.opensplice.org

OpenSplice | DDS¥Scala API for OpenSplice DDS¥Open Source¥github.com/kydos/escalier

Escalier

¥Simple C++ API for DDS¥Open Source¥github.com/kydos/simd-cxx

¥DDS-PSM-Java for OpenSplice DDS¥Open Source¥github.com/kydos/simd-java

¥ DDS-based Advanced Distributed Algorithms Toolkit

¥Open Source¥github.com/kydos/dada

top related