advanced opensplice programming - part i

101
OpenSplice DDS Angelo CORSARO, Ph.D. Chief Technology Ocer OMG DDS Sig Co-Chair PrismTech [email protected] Advanced OpenSplice DDS Programming - Part I -

Upload: angelo-corsaro

Post on 15-Jan-2015

1.128 views

Category:

Technology


5 download

DESCRIPTION

OpenSplice DDS enables seamless, timely, scalable and dependable data sharing between distributed applications and network-connected devices. Its technical and operational benefits have propelled adoption across multiple industries, such as Defence and Aerospace, SCADA, Gaming, Cloud Computing, Automotive, etc. If you want to learn about OpenSplice DDS or discover some of its advanced features, this webcast is for you! In this two-parts webcast we will cover all the aspects tied to architecting and developing OpenSplice DDS systems. We will look into Quality of Services, data selectors concurrency and scalability concerns. We will present the brand-new, and recently finalized, C++ and Java APIs for DDS, including examples of how this can be used with C++11 features. We will show how, increasingly popular, functional languages such as Scala can be used to efficiently and elegantly exploit the massive HW parallelism provided by modern multi-core processors. Finally we will present some OpenSplice specific extensions for dealing very high-volumes of data – meaning several millions of messages per seconds.

TRANSCRIPT

Page 1: Advanced OpenSplice Programming - Part I

Ope

nSpl

ice

DD

S

Angelo CORSARO, Ph.D.Chief Technology Officer OMG DDS Sig Co-Chair

[email protected]

Advanced OpenSplice DDS Programming- Part I -

Page 2: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Before we get into technical details let’s try to answer two questions:

1. What does OpenSplice DDS do?

2. What kind of applications are using it?

Page 3: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

OpenSplice DDS enables seamless, loosely-coupled, timely, scalable and dependable data sharing between distributed applications and network connected devices.

Page 4: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Some Use Cases

Integrated Modular Vetronics Training & Simulation Systems Naval Combat Systems

Air Traffic Control & Management Unmanned Air Vehicles Aerospace Applications

Page 5: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Some Use Cases

Agricultural Vehicle Systems

Train Control Systems Complex Medical Devices

Smart CitiesLarge Scale SCADA Systems

Big Data (In-Memory) Analytics

Page 6: Advanced OpenSplice Programming - Part I

Ope

nSpl

ice

DD

S

A Fresh Look at OpenSplice

Page 7: Advanced OpenSplice Programming - Part I

Ope

nSpl

ice

DD

S

From Local to Distributed Data

Page 8: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Data in Programming Languages

☐ Statically Typed Programming Languages provide mechanisms to declare a variable for given type as well as defining new types

☐ Scoping rules are then used to define what values and types are accessible at any given point in a program text

   int  x;      class  Rational  {  ...  }  

Page 9: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

“Local” Data☐ Consider the Following code fragment below in an imperative

programming language:

12345678910111213

{    int  x;    x  =  7;    {        int  y;        y  =  18;        double  z;        z  =  3.141593        x  =  x  -­‐  4;        y  =  x  +  y;        z  =  2  *  z  *  y;    }}

State  {  }  {  int  x  }  {  x  =  7  }  {  x  =  7,  {  }  }  {  x  =  7,  {  int  y  }  }  {  x  =  7,  {  y  =  18  }  }  {  x  =  7,  {  y  =  18,  double  z  }  }  {  x  =  7,  {  y  =  18,  z  =  3.1416  }  }  {  x  =  3,  {  y  =  2  }  }  {  x  =  3,  {  y  =  21  }  }  {  x  =  3,  {  y  =  21,  z  =  131.9472  }  }  {  x  =  3,  {    }}  {  }

12345678910111213

Page 10: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

“Local” Data☐ One final consideration to do is that variables in a programming

language are in essence alias to a memory location

int  x;int  y;

...

Page 11: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Distributed Data☐ Extending the elegant model provided by programming

languages to distributed systems has been and still is a quite popular research topic

☐ Historically the solutions to the distributed data problem where divided into those based on Distributed Shared Memory and those based on Message Passing

☐ OpenSplice DDS provide a new interesting way of addressing this problem. Let’s see...

Page 12: Advanced OpenSplice Programming - Part I

Ope

nSpl

ice

DD

S

OpenSplice Foundations

Page 13: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Distributed Data in OpenSplice☐ OpenSplice DDS provides the abstraction

of an “Active” Global Data Space

☐ The Global Data Space is fully distributed and does not suffer Single Point of Failure issues

☐ Applications can autonomously join and leave this data space as well as declare new “Data Types”

☐ In OpenSplice, “data types” are called Topics

OpenSplice DDS Global Data Space

...

TopicA

TopicB

TopicC

TopicD

TopicE

Page 14: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Global Data Space☐ The Global Data Space is actually called “Domain” in

OpenSplice DDS

☐ Domains are identified by integer. An application can join a domain by creating a domain participant as follows:

OpenSplice DDS Global Data Space

...

TopicA

TopicB

TopicC

TopicD

TopicE//  Create  a  Topicint  domain_id  =  18;auto  dp  =  DomainParticipant(domain_id);

☐ A Domain defines a top-level scope thus the content of a domain is not accessible from another domain

☐ Finally an application can join multiple domains by creating one domain participant per-domain

Page 15: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Topics☐ A Topic is defined by means of a

(name, type, qos) tuple, where ☐ name: identifies the topic within

OpenSplice’s Global Data Space

☐ type: is the programming language type associated with the topic. This type can be defined in IDL, XML, and Java. Types are extensible and evolvable

☐ qos: is a collection of policies that control non-functional properties of this topic, such as its durability, reliability, etc.

DURABILITY,DEADLINE,PRIORITY,

“Circle”, Square”, “Triangle”

TopicTypeName

QoS

ShapeType

struct ShapeType { string color; long x; long y; long size;};

Page 16: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Defining a Topic

☐ In code, if we wanted to define the Square topic we would do (with the new DDS-PSM-Cxx v1.0 API):

// Create a Topicauto dp = DomainParticipant(18);auto topic = Topic<ShapeType>(dp, “Square”, dp.default_topic_qos());

// Or equivalently and more concisely:auto topic = Topic<ShapeType>(dp, “Square”);

☐ The effect of this code is to define a topic called “Square” with type ShapeType and default QoS within the OpenSplice Global Data Space

Page 17: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Looking with the Spyglassdomain 18

auto topic = Topic<ShapeType>(dp, “Square”);

Squarestruct&ShapeType&{&&string&&&color;&&&&&&&&long&&&x;&&&&&long&&&y;&&long&&&size;};

Policies

Topic

Page 18: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Topics Scope☐ Topic definitions are (somewhat) equivalent to type definitions in a programming

language. In the sense that they define a new, generalized, type to be used within the global data space

☐ The scope of a Topic definition is the entire Domain, thus one could say that there is only one level of scoping. To make the parallel with Java, one could say that it is as if Topics where defined in a namespace named after the domain-id

domain0  {TopicA,  TopicB,  ...}

domain1  {Topic1,  Topic2,  ...}

☐ As a consequence topic definitions in one domain are not visible from a different domain

Page 19: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

If Topic are Types what are Variables?

☐ As described earlier a Topic definition can be thought as a Type definition in a programming language

☐ In a statically typed programming language (e.g. Java), given a type Foo I can define a variable of type Foo as:

   Foo  x;  

x  =  new  Foo(...);  ☐ Then I can initialize it as:

☐ How are instance of a Topic defined?

☐ Then I can mutate it, by: x.setBar(...);

Page 20: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DataWriter

☐ OpenSplice DDS uses DataWriters to declare and write instances of a Topic

☐ A DataWriter for our Square topic can be defined as follows:

// Create a Publisherauto pub = Publisher(dp);

// Create a DataWriterauto dw = DataWriter<ShapeType>(pub, topic);

// Write datadw.write(ShapeType(“RED”,11,77,99));

Page 21: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Looking with the Spyglass

auto dw = DataWriter<ShapeType>(pub, topic);

Squarestruct&ShapeType&{&&string&&&color;&&&&&&&&long&&&x;&&&&&long&&&y;&&long&&&size;};

Policies

Topic

color sizex y

dw

Page 22: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Looking with the Spyglass

Squarestruct&ShapeType&{&&string&&&color;&&&&&&&&long&&&x;&&&&&long&&&y;&&long&&&size;};

Policies

Topic

color sizex y

"RED" 9911 77

dw

dw.write(ShapeType(“RED”,11,77,99));

Page 23: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

An Experiment...☐ What does the following code do?

// Create a Publisherauto pub = Publisher(dp);

// Create DataWritersauto dw1 = DataWriter<ShapeType>(pub, topic);auto dw2 = DataWriter<ShapeType>(pub, topic);

// Write Datadw1.write(ShapeType(“RED”,10,20,30));dw2.write(ShapeType(“BLUE”,30,20, 10));

Page 24: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Looking with the Spyglass

Squarestruct&ShapeType&{&&string&&&color;&&&&&&&&long&&&x;&&&&&long&&&y;&&long&&&size;};

Policies

Topic

color sizex y

dw2

dw1

auto dw1 = DataWriter<ShapeType>(pub, topic);auto dw2 = DataWriter<ShapeType>(pub, topic);

Data Writers for the same topic are aliased to the same variable!

Page 25: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Looking with the Spyglass

dw1.write(ShapeType(“RED”,10,20,30));

Squarestruct&ShapeType&{&&string&&&color;&&&&&&&&long&&&x;&&&&&long&&&y;&&long&&&size;};

Policies

Topic

color sizex y

RED 3010 20

dw2

dw1

Page 26: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Looking with the Spyglass

dw2.write(ShapeType(“BLUE”,30,20, 10));

Squarestruct&ShapeType&{&&string&&&color;&&&&&&&&long&&&x;&&&&&long&&&y;&&long&&&size;};

Policies

Topic

color sizex y

RED 1030 20

dw2

dw1

Squarestruct&ShapeType&{&&string&&&color;&&&&&&&&long&&&x;&&&&&long&&&y;&&long&&&size;};

Policies

Topic

color sizex y

BLUE 1030 20

dw2

dw1

Page 27: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Topic Instances☐ Topic type can identify an arbitrary number of attributes as defining the

topic key

☐ In this case, each unique key value defines a topic instance

☐ Example:

struct ShapeType { @Key long color; long x; long y; long size;};

Page 28: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Let’s Experiment...☐ What does the following code do?

// Create a Publisherauto pub = Publisher(dp);

// Create DataWritersauto dw = DataWriter<ShapeType>(pub, topic);

// Write Datadw.write(ShapeType(“RED”,10,20,30));dw.write(ShapeType(“BLUE”,30,20,10));

Page 29: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Looking with the Spyglass

Squarestruct&ShapeType&{&&string&&&color;&&&&&&&&long&&&x;&&&&&long&&&y;&&long&&&size;};

Policies

Topic

color sizex y

RED 3010 20

dw

dw.write(ShapeType(“RED”,10,20,30));

Page 30: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Looking with the Spyglass

Squarestruct&ShapeType&{&&string&&&color;&&&&&&&&long&&&x;&&&&&long&&&y;&&long&&&size;};

Policies

Topic

color sizex y

RED 3010 20

dw

BLUE 1030 20

dw.write(ShapeType(“BLUE”,30,20,10));

Page 31: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

LongVariable Example

struct LongVariable { @Key string name; long value; };

// Create a Publisherauto lvt = Topic<LongVariable>(dp, “TLongVariable”);

// Create DataWritersauto dw = DataWriter<LongVariable>(pub, topic);

// Write Datadw.write(LongVariable(“x”,10));dw.write(LongVariable(“y”,20));

☐ The “distributed” version of the following code:int  x;int  y;x  =  10;y  =  20;

☐ Becomes: Topic Type

Page 32: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Looking with the Spyglass

TLongValue

Policies

Topic

dw

struct LongVariable { @Key string name; long value; };

valuename

10x

20y

Page 33: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DataReader

☐ OpenSplice DDS uses DataReaders to read instances of a Topic

☐ A DataReader for our Square topic can be defined as follows:

// Create a Subscriberauto sub = Subscriber(dp);

// Create a DataWriterauto dr = DataReader<ShapeType>(sub, topic);

// Read dataauto data = dr.read();

☐ The read operation will read by default all instance values. Special selector operation are available to pick a specific instance. More later

Page 34: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DataReader/DataWriter Matching

☐ DataWriters and DataReaders are automatically and dynamically matched by the DDS Dynamic Discovery

☐ A rich set of QoS Policies allows to control existential, temporal, and spatial properties of data and impacts DataReader/DataWriter matching DDS Global Data Space

...

TopicA

TopicBTopicC

TopicD

Data Writer

Data Writer

Data Writer

Data Writer

Data Reader

Data Reader

Data Reader

Data Reader

Page 35: Advanced OpenSplice Programming - Part I

Ope

nSpl

ice

DD

S

Publishers and Subscribers

Page 36: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Publishers / Subscribers

Publishers and Subscribers in DDS play two roles:

☐ Coordinate actions across DataReaders/DataWriters☐ e.g. Atomic writes across data-writers

☐ Control access to DDS partitions, a mechanism provided by DDS to further

Domain (e.g. Domain 123)

Domain Participant

Topic

Publisher

DataWrter

Subscriber

DataReader

Partition (e.g. “Telemetry”, “Shapes”, )

Topic Instances/Samples

TaTb

Tc

Tx

Ty

T1

T1 T3

Page 37: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DDS Partitions☐ DDS Publishers and

Subscribers can specify a list of regular expressions representing the partitions to join

☐ Actual partition can be any string, but some structure can be given in order mimic hierarchy

M:0:0 M:0:1 M:0:2

M:1:0 M:1:1 M:1:2

M:2:0 M:2:1 M:2:2

Domain

Partitions

Page 38: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

M:0:0 M:0:1 M:0:2

M:1:0 M:1:1 M:1:2

M:2:0 M:2:1 M:2:2

Domain

Partitions

DDS Partitions Matching

M:0:*

Page 39: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

M:0:0 M:0:1 M:0:2

M:1:0 M:1:1 M:1:2

M:2:0 M:2:1 M:2:2

Domain

Partitions

DDS Partitions Matching

M:*:2

Page 40: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

M:0:0 M:0:1 M:0:2

M:1:0 M:1:1 M:1:2

M:2:0 M:2:1 M:2:2

Domain

Partitions

DDS Partitions Matching

M:0:0, M:1:1, M:2:2

Page 41: Advanced OpenSplice Programming - Part I

Ope

nSpl

ice

DD

S

Anatomy of a DDS Application

Page 42: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Anatomy of a DDS Application

Domain

Reader/Writers for User Defined for Types

Session

// Create a DataWriter/DataWriterauto writer = DataWriter<ShapeType>(pub, topic);auto reader = DataReader<ShapeType>(sub, topic);

Reader/Writer for application defined

Topic Types

Domain Participant

Publisher

DataWriter

Topic Subscriber

DataReader

[DDS C++ API 2010]

auto dp = DomainParticipant(domainId);

// Create a Topicauto topic = Topic<ShapeType>(dp, “Circle”);// Create a Publisher / Subscriberauto pub = Publisher(dp);auto sub = Subscriber(dp);

Page 43: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Anatomy of a DDS Application

Domain

Reader/Writers for User Defined for Types

Session

// Write datawriter.write(ShapeType(“RED”, 131, 107, 89));// But you can also write like this...writer << ShapeType(“RED”, 131, 107, 89);

// Read new data (loaned)auto data = reader.read();

Reader/Writer for application defined

Topic Types

auto dp = DomainParticipant(domainId); Domain

Participant

Publisher

DataWriter

Topic Subscriber

DataReader

// Create a Topicauto topic = Topic<ShapeType>(dp, “Circle”);// Create a Publisher / Subscriberauto pub = Publisher(dp);auto sub = Subscriber(dp);

[DDS C++ API 2010]

Page 44: Advanced OpenSplice Programming - Part I

Ope

nSpl

ice

DD

S

Data Reader/Writer Caches

Page 45: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Page 46: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

write( ) Application Application

Page 47: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Topic Instance

Page 48: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Page 49: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Topic Instance Topic Instance

Page 50: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Applicationwrite( )

Page 51: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Writer History

Page 52: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Page 53: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Reader History Reader History

Page 54: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Applicationwrite( )

Page 55: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Page 56: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Page 57: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Page 58: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

‣History‣Destination Order‣ Presentation‣ Partition‣ Time Based Filter‣Ownership

QoS Policies

‣Reliability‣History‣ Latency Budget

QoS Policies‣Durability‣ Transport Priority‣ Time Based Filter

Page 59: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Writer Cache Properties☐ An entry for each topic instance

☐ Each cache entry might contain one or more samples, the number of samples is controlled by the History QoS Policy

☐ Depending on the setting of the Durability QoS Policy, the content of the cache may be made available to late joiners. The depth of the cache controls how many samples per instance will be delivered to late joiners

DataWriter Cache

DataWriter

...

Page 60: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader Cache Properties☐ An entry for each topic instance

☐ Each cache entry might contain one or more samples, the number of samples is controlled by the History QoS Policy☐ History.KeepLast(n) keeps the last n samples

per instance☐ History.KeepAll maintains all the samples,

modulo resource constraints

☐ For a given instance:☐ Samples from the same writer are inserted in

write-order☐ Samples from multiple-writers are ordered

using time-stamps (source or reception depending on configurable a QoS Policy)

DataReader Cache

DataReader

...

Page 61: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Eventual Consistency☐ DDS caches provide eventual consistency semantics

☐ This means that a read will see the effect of a preceding write eventually

☐ Furthermore, given a data-writer that is currently matching N readers, we can think of DDS as providing eventual consistency with W=0 and R=1☐ W: the number of Acks expected in order to return from a write☐ R: the number of sources from which a read access data

Page 62: Advanced OpenSplice Programming - Part I

Ope

nSpl

ice

DD

S

Understanding History

Page 63: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

Page 64: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

write( )1

Page 65: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

1

Page 66: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

1

1

Page 67: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

1 1 1

Page 68: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

1 1 1

write( )2

Page 69: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

2 1 1

Application

Page 70: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

2 1 1

Application

2

Page 71: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

2 1 1

Application

2 2

Page 72: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

2 1 12 2

write( )3

Page 73: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

3 1 12 2

Application

Page 74: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

3 1 12 2

Application

3

Page 75: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader/Writer Caches

DataWriter Cache

DataWriter

DataReader Cache

DataReader

DataReader Cache

DataReader

Network

...

Application Application

Assumptions: DataReader configured with history depth of 2 and DataWriter with default history depth of 1

3 2 23 3

Application

Page 76: Advanced OpenSplice Programming - Part I

Ope

nSpl

ice

DD

S

Caches or Streams?

Page 77: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Dynamic View of a Stream

Stream: Set of samples written over time for a given topic instance.

...

Page 78: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Eventual View of a Stream

‘Past’ Samples

Stream: Set of samples written over time for a given topic instance.

Reader HistoryWriter History

Assumptions (Default Settings): Reader History = KeepLast (1) WriterHistory = KeepLast (1)

Page 79: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Eventual View of a Stream

‘Past’ Samples

Stream: Set of samples written over time for a given topic instance.

Reader HistoryWriter History

Assumptions: Reader History = KeepLast (n) with n > 1WriterHistory = KeepLast (1)

Page 80: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Eventual View of a Stream

‘Past’ Samples

Stream: Set of samples written over time for a given topic instance.

Reader HistoryWriter History

Assumptions: Reader History = KeepLast (n) with n > 1WriterHistory = KeepLast (m) with n > m > 1

Page 81: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Eventual View of a Stream

Reader HistoryWriter History

Assumptions: Reader History = KeepAllWriterHistory = KeepLast (m) with n > m > 1

...

Page 82: Advanced OpenSplice Programming - Part I

Ope

nSpl

ice

DD

S

Interacting with the DataReader Cache

Page 83: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DataReader Cache

DataReader

...

Reading Data Samples☐ Samples can be read from the Data Reader History Cache

☐ The action of reading a sample is non-destructive. Samples are not removed from the cache

DataReader Cache

DataReader

... read

Page 84: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Taking Data Samples☐ Samples can be taken from the Data Reader History Cache

☐ The action of taking a sample is destructive. Samples are removed from the cache

DataReader Cache

DataReader

...

DataReader Cache

DataReader

... take

Page 85: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Read vs. Take

☐ The read operation should always be access the latest know value for topics that represent distributed state

☐ The take operation should be used to get the last notification from a topic that represent an event

Page 86: Advanced OpenSplice Programming - Part I

Ope

nSpl

ice

DD

S

Data Selectors

Page 87: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Cherry Picking in DDS

☐ DDS provides some very flexible mechanisms for selecting the data to be read:☐ Data Content☐ Data Status

☐ These mechanisms are composable

Page 88: Advanced OpenSplice Programming - Part I

Ope

nSpl

ice

DD

S

Content-Based Data Selection

Page 89: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Filters and Queries☐ DDS Filters allow to control what gets

into a DataReader cache

☐ DDS Queries allow to control what gets out of a DataReader cache

☐ Filters are defined by means of ContentFilteredTopics

☐ Queries operate in conjunction with read operations

☐ Filters and Queries are expressed as SQL where clauses

DataReader Cache

DataReader

...... ... ...

Filter

Query

Application

Page 90: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Filters/** * NOTE: The Scala API if not provided with DP/Sub/Pub assumes * default domains and default partition. **/// Create a Topicauto topic = Topic<ShapeType>(dp, “Circle”);

// Define filter expression and parametersauto filter = Filter(“x < 100 AND y < 200”);

// Define content filtered topicauto cftopic = ContentFilteredTopic<ShapeType>(“CFCircle”, topic, filter)

// Create a DataReader for the content-filtered Topicauto dr = DataReader<ShapeType>(sub,cftopic)

[C++ API]

struct ShapeType { @Key string color; long x; long y; long shapesize;};

Page 91: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Query

// Define filter expression and parametersauto dr = DataReader<ShapeType>(sub, topic) val query = Query(dr, “x < 100 AND y < 200”);

dr.select() .content(query) .read();

struct ShapeType { @Key string color; long x; long y; long shapesize;};[C++ API]

Page 92: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Instances☐ DDS provides a very efficient way of reading data belonging to a

specific Topic Instance

☐ Obviously, one could use queries to match the key’s value, but this is not as efficient as the special purpose instance selector

auto handle = dr.lookup_instance(ShapeType(“RED”, 0, 0, 0));

auto data = dr.select() .instance(handle) .read();

Page 93: Advanced OpenSplice Programming - Part I

Ope

nSpl

ice

DD

S

State-Based Selection

Page 94: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Sample, Instance, and View State☐ The samples included in the DataReader cache have associated

some meta-information which, among other things, describes the status of the sample and its associated stream/instance

☐ The Sample State (READ, NOT_READ) allows to distinguish between new samples and samples that have already been read

☐ The View State (NEW, NOT_NEW) allows to distinguish a new instance from an existing one

☐ The Intance State (ALIVE, NOT_ALIVE_DISPOSED, NOT_ALIVE_NO_WRITERS) allows to track the life-cycle transitions of the instance to which a sample belongs

Page 95: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

State Selector in Action

// Read only new samplesauto data = dr.read()

// Read any samples from live instancesauto data = dr.select() .state(DataState::any_data()) .read();

[C++ API]

Page 96: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

auto data = dr.select() .content(query) .state(data_state) .instance(handle) .read();

Putting all Together

☐ Selectors can be composed in a flexible and expressive manner

[C++ API]

Page 97: Advanced OpenSplice Programming - Part I

Ope

nSpl

ice

DD

S

Summing Up

Page 98: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Concluding Remarks☐ OpenSplice DDS provides a very powerful abstraction for building

distributed applications grounded on the concept of a fully distributed global data space

☐ Topics are used to define the types that populate the global data space while DataWriters and DataReaders are used to produce and consume values

☐ DataReaders allow very flexible data access through cache access operations

Page 99: Advanced OpenSplice Programming - Part I

Ope

nSpl

ice

DD

S

Page 100: Advanced OpenSplice Programming - Part I

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

References

¥Fastest growing JVM Language¥Open Source¥www.scala-lang.org

¥ #1 OMG DDS Implementation¥ Open Source¥ www.opensplice.org

OpenSplice | DDS¥Scala API for OpenSplice DDS¥Open Source¥github.com/kydos/escalier

Escalier

¥Simple C++ API for DDS¥Open Source¥github.com/kydos/simd-cxx

¥DDS-PSM-Java for OpenSplice DDS¥Open Source¥github.com/kydos/simd-java

¥ DDS-based Advanced Distributed Algorithms Toolkit

¥Open Source¥github.com/kydos/dada