advanced opensplice programming - part ii

75
OpenSplice DDS Angelo CORSARO, Ph.D. Chief Technology Ocer OMG DDS Sig Co-Chair PrismTech [email protected] Advanced OpenSplice DDS Programming - Part II -

Upload: angelo-corsaro

Post on 15-Jan-2015

1.389 views

Category:

Technology


1 download

DESCRIPTION

OpenSplice DDS enables seamless, timely, scalable and dependable data sharing between distributed applications and network-connected devices. Its technical and operational benefits have propelled adoption across multiple industries, such as Defence and Aerospace, SCADA, Gaming, Cloud Computing, Automotive, etc. If you want to learn about OpenSplice DDS or discover some of its advanced features, this webcast is for you! In this two-parts presentation we will cover most of the aspects tied to architecting and developing OpenSplice DDS systems. We will look into Quality of Services, data selectors concurrency and scalability concerns. We will present the brand-new, and recently finalized, C++ and Java APIs for DDS, including examples of how this can be used with C++11 features. We will show how, increasingly popular, functional languages such as Scala can be used to efficiently and elegantly exploit the massive HW parallelism provided by modern multi-core processors. Finally we will present some OpenSplice specific extensions for dealing very high-volumes of data – meaning several millions of messages per seconds.

TRANSCRIPT

Ope

nSpl

ice

DD

S

Angelo CORSARO, Ph.D.Chief Technology Officer OMG DDS Sig Co-Chair

[email protected]

Advanced OpenSplice DDS Programming- Part II -

Ope

nSpl

ice

DD

S

Part I - Recap

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Key Concepts to Remember☐ OpenSplice DDS provides a mechanism for efficiently sharing user-

defined data among distributed applications and network connected devices

Agricultural Vehicle Systems

Train Control Systems Complex Medical Devices

Smart CitiesLarge Scale SCADA Systems

Big Data (In-Memory) Analytics

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Key Concepts to Remember☐ OpenSplice DDS allows to build distributed systems by reasoning in

terms of application types as opposed to messages

OpenSplice DDS Global Data Space

...

TopicA

TopicB

TopicC

TopicD

TopicE

PS. long in IDL is a 32 bit integer

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Key Concepts to Remember☐ User defined data types are accessible through “local caches”.

These local caches can have a depth and store multiple updates for the same data item

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Key Concepts to Remember

☐ Built-in dynamic discovery automatically establishes associations between data writers and data readers

DDS Global Data Space

...

TopicA

TopicBTopicC

TopicD

Data Writer

Data Writer

Data Writer

Data Writer

Data Reader

Data Reader

Data Reader

Data Reader

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Anatomy of a DDS Application

Domain

Reader/Writers for User Defined for Types

Session

// Create a DataWriter/DataWriterauto writer = DataWriter<ShapeType>(pub, topic);auto reader = DataReader<ShapeType>(sub, topic);

Reader/Writer for application defined

Topic Types

Domain Participant

Publisher

DataWriter

Topic Subscriber

DataReader

[DDS C++ API 2010]

auto dp = DomainParticipant(domainId);

// Create a Topicauto topic = Topic<ShapeType>(dp, “Circle”);// Create a Publisher / Subscriberauto pub = Publisher(dp);auto sub = Subscriber(dp);

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Anatomy of a DDS Application

Domain

Reader/Writers for User Defined for Types

Session

//  Write  datawriter.write(ShapeType(“RED”,  131,  107,  89));//  But  you  can  also  write  like  this...writer  <<  ShapeType(“RED”,  131,  107,  89);

//  Read  new  data  (loaned)auto  data  =  reader.read();

Reader/Writer for application defined

Topic Types

auto  dp  =  DomainParticipant(domainId);   Domain Participant

Publisher

DataWriter

Topic Subscriber

DataReader

//  Create  a  Topicauto  topic  =  Topic<ShapeType>(dp,  “Circle”);//  Create  a  Publisher  /  Subscriberauto  pub  =  Publisher(dp);auto  sub  =  Subscriber(dp);

[DDS C++ API 2010]

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Anatomy of a DDS Application

Domain

Reader/Writers for User Defined for Types

Session

//  Create  a  DataWriter/DataWriterDataWriter<ShapeType>  dw  =pub.createDataWriter(topic);  DataReader<ShapeType>  dr  =pub.createDataReader(topic);  

Reader/Writer for application defined

Topic Types

Domain Participant

Publisher

DataWriter

Topic Subscriber

DataReader

[DDS Java 5 API 2010]

DomainParticipantFactory  factory  =  DomainParticipantFactory.getInstance(env);DomainParticipant  dp  =  factory.createParticipant();

//  Create  a  TopicTopic<ShapeType>  topic  =      dp.createParticipant(“Circle”,  ShapeType.class);//  Create  a  Publisher  /  SubscriberPublisher    pub  =  dp.createPublisher();Subscriber  sub  =  dp.createSubscriber();

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Anatomy of a DDS Application

Domain

Reader/Writers for User Defined for Types

Session

//  write  datadw.write(new  ShapeType(“RED”,  10,  20,  50);  //  read  dataSample.Iterator<ShapeType>  data  =  dr.read();

Reader/Writer for application defined

Topic Types

Domain Participant

Publisher

DataWriter

Topic Subscriber

DataReader

[DDS Java 5 API 2010]

DomainParticipantFactory  factory  =  DomainParticipantFactory.getInstance(env);DomainParticipant  dp  =  factory.createParticipant();

//  Create  a  TopicTopic<ShapeType>  topic  =      dp.createParticipant(“Circle”,  ShapeType.class);//  Create  a  Publisher  /  SubscriberPublisher    pub  =  dp.createPublisher();Subscriber  sub  =  dp.createSubscriber();

Ope

nSpl

ice

DD

S

Part II

Ope

nSpl

ice

DD

S

Quality of Service

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

QoS Model☐ QoS-Policies control local and

end-to-end properties of DDS entities

☐ Local properties controlled by QoS are related resource usage

☐ End-to-end properties controlled by QoS are related to temporal and spatial aspects of data distribution

☐ Some QoS-Policies are matched based on a Request vs. Offered Model thus QoS-enforcement

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

......

QoS QoS

Type Matching

DomainParticipant DomainParticipant

QoS QoS

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

QoS PoliciesQoS Policy Applicability RxO Modifiable

USER_DATATOPIC_DATA

GROUP_DATADURABILITYDURABILITY

SERVICEHISTORY

PRESENTATIONRELIABILITYPARTITION

DESTINATION ORDER

LIFESPAN

DP, DR, DW N Y

ConfigurationT N Y ConfigurationP, S N Y

Configuration

T, DR, DW Y N

Data AvailabilityT, DW N N

Data Availability

T, DR, DW N N

Data Availability

P, S Y N

Data Delivery

T, DR, DW Y N

Data DeliveryP, S N Y Data DeliveryT, DR, DW Y N

Data Delivery

T, DW N Y

Data Delivery

[T: Topic] [DR: DataReader] [DW: DataWriter] [P: Publisher] [S: Subscriber] [DP: Domain Participant]

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

QoS PoliciesQoS Policy Applicability RxO ModifiableDEADLINELATENCY BUDGET

TRANSPORT PRIORITY

TIME BASED FILTER

OWNERSHIPOWNERSHIP STRENGTHLIVELINESS

T, DR, DW Y Y

Temporal/Importance

Characteristics

T, DR, DW Y YTemporal/

Importance Characteristics

T, DW N YTemporal/

Importance Characteristics

DR N Y

Temporal/Importance

Characteristics

T, DR, DW Y NReplicationDW N Y Replication

T, DR, DW Y N Fault-Detection

[T: Topic] [DR: DataReader] [DW: DataWriter] [P: Publisher] [S: Subscriber] [DP: Domain Participant]

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Data Delivery

Data Delivery

Reliability

Presentation

Destination OrderPartition

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reliability QoS Policy

The Reliability Policy controls the level of guarantee offered by the DDS in delivering data to subscribers

☐ Reliable. In steady-state, and no data writer crashes, the middleware guarantees that all samples in the DataWriter history will eventually be delivered to all the DataReader

☐ Best Effort. Indicates that it is acceptable to not retry propagation of any samples

QoS Policy Applicability RxO ModifiableRELIABILITY T, DR, DW Y N

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Setting the Reliability Policy

DataWriterQos  dwqos  =        pub.default_datawriter_qos()  <<  Reliability.Reliable();

-­‐  or  -­‐

DataWriterQos  dwqos  =        pub.default_datawriter_qos()  <<  Reliability.BestEffort();

[DDS C++ API 2010]

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Setting the Partition Policy

final  PolicyFactory  pf  =  ...

DataWriterQos  dwqos  =        pub.getDefaultDataWriterQos()            .withPolicies(pf.Reliability.withReliable());  

-­‐  or  -­‐

DataWriterQos  dwqos  =        pub.getDefaultDataWriterQos()            .withPolicies(pf.Reliability.withBestEffort());  

[DDS Java 5 API 2010]

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Partition QoS Policy☐ The Partition QoS Policy can

be used as subjects for organizing the flows of data

☐ The Partition QoS Policy is used to connect Publishers/Subscribers to a Partitions’ List which might also contain wildcards, e.g. tracks.*

☐ Topics instances are published and subscribed across one or more Partitions

QoS Policy Applicability RxO ModifiablePARTITION P, S N Y

Subscriber

Subscriber

Subscriber

"tracks.kfo" "tracks.ufo"

Publisher

Publisher

Publisher

Partition

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Setting the Partition Policy

PublisherQos  pqos  =        dp.default_publisher_qos()  <<  Partition(“MyPartition”);

-­‐  or  -­‐

PublisherQos  pqos  =        dp.default_publisher_qos()  <<  Partition(myPartitionList);

[DDS C++ API 2010]

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Setting the Partition Policy

final  PolicyFactory  pf  =  ...

PublisherQos  pqos  =        dp.getDefaultPublisherQos()            .withPolicies(pf.Partition(“MyPartition”));  

-­‐  or  -­‐

PublisherQos  pqos  =        dp.getDefaultPublisherQos()            .withPolicies(pf.Partition(myPartitionList));  

[DDS Java 5 API 2010]

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Data Availability

Data Availability

History

Ownership

DurabilityLifespan

OwnershipStrength

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Durability QoS Policy

The DURABILITY QoS controls the data availability w.r.t. late joiners, specifically the DDS provides the following variants:

☐ Volatile. No need to keep data instances for late joining data readers

☐ Transient Local. Data instance availability for late joining data reader is tied to the data writer availability

☐ Transient. Data instance availability outlives the data writer

☐ Persistent. Data instance availability outlives system restarts

QoS Policy Applicability RxO ModifiableDURABILITY T, DR, DW Y N

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Volatile

DDS Global Data Space

TopicA

Data Writer

Data Reader

1

‣ No Time Decoupling‣ Readers get only data produced after they joined the Global Data Space

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Volatile

DDS Global Data Space

TopicA

Data Writer

Data Reader

1

Data Reader

Late Joiner

‣ No Time Decoupling‣ Readers get only data produced after they joined the Global Data Space

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Volatile

DDS Global Data Space

TopicA

Data Writer

Data Reader

1

22

Data Reader

‣ No Time Decoupling‣ Readers get only data produced after they joined the Global Data Space

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Transient Local

DDS Global Data Space

TopicA

Data Writer

Data Reader

11

‣ Some Time Decoupling‣ Data availability is tied to the availability of the data writer and the history settings

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Transient Local

DDS Global Data Space

TopicA

Data Writer

Data Reader

1

Data Reader

Late Joiner

11

‣ Some Time Decoupling‣ Data availability is tied to the availability of the data writer and the history settings

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Transient Local

DDS Global Data Space

TopicA

Data Writer

Data Reader

1

2

Data Reader

11

22

‣ Some Time Decoupling‣ Data availability is tied to the availability of the data writer and the history settings

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Transient

DDS Global Data Space

TopicA

Data Writer

Data Reader

11

‣ Time Decoupling‣ Data availability is tied to the availability of the durability service -- a fully distributed

fault-tolerant service in the case of OpenSplice DDS.

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Transient

DDS Global Data Space

TopicA

Data Writer

Data Reader

1

Data Reader

Late Joiner

11

‣ Time Decoupling‣ Data availability is tied to the availability of the durability service -- a fully distributed

fault-tolerant service in the case of OpenSplice DDS.

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

‣ Time Decoupling‣ Data availability is tied to the availability of the durability service -- a fully distributed

fault-tolerant service in the case of OpenSplice DDS.

Transient

DDS Global Data Space

TopicAData

Reader

Data Reader

1

1

1

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

‣ Time Decoupling‣ Data availability is tied to the availability of the durability service -- a fully distributed

fault-tolerant service in the case of OpenSplice DDS.

Transient

DDS Global Data Space

TopicAData

Reader

Data Reader

1

1

1

Data Reader

Late Joiner

1

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

History QoS Policy

For DataWriters, the HISTORY QoS policy controls the amount of data that can be made available to late joining DataReaders under TRANSIENT_LOCAL Durability

For DataReader, the HISTORY QoS policy controls how many samples will be kept on the reader cache

☐ Keep Last. DDS will keep the most recent “depth” samples of each instance of data identified by its key

☐ Keep All. The DDS keep all the samples of each instance of data identified by its key -- up to reaching some configurable resource limits

QoS Policy Applicability RxO ModifiableHISTORY T, DR, DW N N

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Ownership QoS Policy

Availability of data producers can be controlled via two QoS Policies

☐ OWNERSHIP (SHARED vs. EXCLUSIVE)

☐ OWNERSHIP STRENGTH

☐ Instances of exclusively owned Topics can be modified (are owned) by the higher strength writer

☐ Writer strength is used to coordinate replicated writers

QoS Policy Applicability RxO ModifiableOWNERSHIPSTRENGTH

T, DR, DW Y NDW N Y

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Temporal Properties

Throughput

TimeBasedFilter

[Inbound]

[Outbound]Latency

Deadline

TransportPriority

LatencyBudget

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Latency Budget QoS Policy☐ The LATENCY_BUDGET QoS

policy specifies the maximum acceptable delay from the time the data is written until the data is inserted in the receiver's application-cache

☐ A non-zero latency-budget allows a DDS implementation to batch samples and improve CPU/Network utilization

QoS Policy Applicability RxO ModifiableLATENCY BUDGET

T, DR, DW Y Y

DataWriter DataReaderT1

T2

T3

Latency = T1+T2+T3

Batching

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Deadline QoS Policy

☐ The DEADLINE QoS policy allows to define the maximum inter-arrival time between data samples

☐ DataWriter indicates that the application commits to write a new value at least once every deadline period

☐ DataReaders are notified by the DDS when the DEADLINE QoS contract is violated

QoS Policy Applicability RxO ModifiableDEADLINE T, DR, DW Y Y

DataWriter DataReaderDeadlineDeadlineDeadlineDeadlineDeadline

Deadline Violation

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Transport Priority QoS Policy

☐ The TRANSPORT_PRIORITY QoS policy is a hint to the infrastructure as to how to set the priority of the underlying transport used to send the data.

QoS Policy Applicability RxO ModifiableTRANSPORT

PRIORITYT, DW N Y

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Time-Based Filter QoS Policy

☐ The Time Based Filter allows to control the throughput at which data is received by a data reader

☐ Samples produced more often than the minimum inter-arrival time are not delivered to the data reader

QoS Policy Applicability RxO ModifiableTIME BASED

FILTERDR N Y

DataWriter DataReader

mit

T2

T3

Latency = T1+T2+T3

discarded sample

mit

mit

mit

mit = minimum inter-arrival time

produced sample delivered sample

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

QoS Provider☐ The new C++ and Java APIs introduce the concept of a QoS

Provider

☐ This class allows to externally define policies and decouples the mechanism used to define and access policy definition with policy creation

           //  QosProvider...            QosProvider  qos_provider(                        "http://www.opensplice.org/demo/config/qos.xml",                        "ishapes-­‐profile");

           DataReader<ShapeType>  dr(sub,  topic,  qos_provider.datareader_qos());

Ope

nSpl

ice

DD

S

QoS and Data Modeling Patterns

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Soft State☐ In distributed systems you often need to model soft-state -- a state

that is periodically updated☐ Examples are the reading of a sensor (e.g. Temperature Sensor),

the position of a vehicle, etc.

☐ The QoS combination to model Soft-State is the following:

Reliability            =>    BestEffortDurability              =>    Volatile  History                    =>    KeepLast(n)  [with  n  =  1  in  most  of  the  cases]Deadline                  =>    updatePeriodLatencyBudget        =>    updatePeriod/3  [rule  of  thumb]DestinationOrder  =>    SourceTimestamp  [if  multiple  writers  per  instance]

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Hard State☐ In distributed systems you often need to model hard-state -- a state

that is sporadically updated and that often has temporal persistence requirements

☐ Examples are system configuration, a price estimate, etc.

☐ The QoS combination to model Hard-State is the following:

Reliability            =>    ReliableDurability              =>    Transient  |  Persistent  History                    =>    KeepLast(n)  [with  n  =  1  in  most  of  the  cases]DestinationOrder  =>    SourceTimestamp  [if  multiple  writers  per  instance]

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Events☐ In distributed systems you often need to model events -- the

occurrence of something noteworthy for our system

☐ Examples are a collision alert, the temperature beyond a given threshold, etc.

☐ The QoS combination to model Events is the following:

Reliability            =>    ReliableDurability              =>    any                [depends  on  system  requirements]  History                    =>    KeepAll  [on  both  DataWriter  and  DataReader!]DestinationOrder  =>    SourceTimestamp

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Streams

☐ At times you need to deal with high volumes of frequently changing soft-state

☐ You may have from several hundreds of thousands to several millions of data readings that need to be distributed in your system... Where each reading is usually from few tends to few hundreds of bytes

☐ OpenSplice provide a special abstraction to deal with this situation called Streams

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Streams☐ OpenSplice Streams

transparently perform batching and un-batching of data based on samples count or time

☐ OpenSplice Streams also provide explicit flush operation to force the middleware to send data

StreamDataWriter

Temporal or Samples-Count Driven Batching

Batching

StreamDataReader

Un-Batching

High Frequency Samples

High Frequency Samples

struct SCADASensor { long sid; float value; };#pragma stream SCADASensor

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

StreamDataWriter

StreamDataWriterQos  sqos(max_samples,  max_delay);

SCADASensorStreamDataWriter  sdw  =        new  SCADASensorStreamDataWriter(did,  sqos,  “ScadaSensorT”);

StreamId  sid  =  0;  //  Your  stream  IdSCADASensor  data  =  ...sdw.append(sid,  data);...//  Explicitely  flush  the  streamsdw.flush(sid);

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

StreamDataReaderSCADASensorStreamDataReader  sdr  =        new  SCADASensorStreamDataReader(did,  “ScadaSensorT”);

StreamId  sid  =  0;  //  Your  stream  IdSCADASensorSeq  data;//  Issue  a  blocking  getsdw.get(sid,  data,  DDS::LENGTH_UNLIMITED,  DDS::DURATION_INFINITE);

//  Note:  //    -­‐  get_with_filter  is  also  available  to  filter  data//    -­‐  The  Stream  Examples  are  shown  using  the  DDS  C++  2004  API  //        since  these  are  OpenSplice  specific  extensions.  //        DDS  C++  2010  counterparts  will  be  soon  available.

Ope

nSpl

ice

DD

S

Resource Management

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

LongVariable Example

struct LongVariable { @Key string name; long value; };

// Create a Publisherauto lvt = Topic<LongVariable>(dp, “TLongVariable”);

// Create DataWritersauto dw = DataWriter<LongVariable>(pub, topic);

// Write Datadw.write(LongVariable(“x”,10));dw.write(LongVariable(“y”,20));

☐ The “distributed” version of the following code:int  x;int  y;x  =  10;y  =  20;

☐ Becomes: Topic Type

PS. long in IDL is a 32 bit integer

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Looking with the Spyglass

TLongValue

Policies

Topic

dw

struct LongVariable { @Key string name; long value; };

valuename

10x

20y

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Declaration vs. Assignement

dw.write(LongVariable(“x”,10));{      int  x;    x  =  10;}

Declaration

Assignment

Assignment

Resources Reclamation

☐ Can we declare topic instances in DDS?

☐ Can we control resource reclamation?

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Instance Registration

☐ Instance registration provides a way of informing the system that the given data writer will be writing the given instance

☐ This operation has an impact on instance life-cycle☐ The write operation implicitly register the instance being written, if necessary

auto key = LongVariable(“x”,0)auto handle = dw.register_instance(key);

dw.write(LongVariable(“x”,10));

{      int  x;    x  =  10;}

Declaration

Assignment

Declaration

Resources Reclamation

Assignment

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Instance Un-registration☐ DDS provides a way for a DataWriter to “un-register” an

instance thus informing the system that he won’t be writing that specific instance any more

☐ This operation has an impact on instance life-cycle

dw.unregister_instance(handle);

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Instance Disposal

☐ Allows to control resource reclamation for a given instance.

☐ The action of disposing an instance signifies that the system is done with it...

auto key = LongVariable(“x”,0)auto handle = dw.register_instance(key);

dw.write(LongVariable(“x”,10));dw.dispose_instance(handle)

{      int  x;    x  =  10;}

Declaration

Assignment

Declaration

Resources Reclamation

Assignment

Resources Reclamation

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Writer Data Lifecycle☐ The life-cycle of data on the Writer side is controlled by means of

QoS policy we’ve not introduced yet, the WriterDataLifecycle

☐ This policy controls wether unregistered instances are automatically disposed or not

☐ This is quite important since when a writer is disposed all its instances are automatically unregistered and depending on the setting of this QoS might also be disposed... Which in some cases is not what you want!

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Reader Data Lifecycle☐ The lifecycle of the data-instances received by the DataReader is

controlled by the ReaderDataLifecycle policy

☐ This policy allow to control two properties

☐ autopurge_nowriter_samples_delay: the maximum duration for which the DataReader will maintain information on an instance once its state becomes NOT_ALIVE_NO_WRITERS

☐ autopurge_disposed_samples_delay: the maximum duration for which the DataReader will maintain samples for an instance once its state becomes NOT_ALIVE_DISPOSED

PS. The default setting for this policy is INFINITE for both delays

Ope

nSpl

ice

DD

S

DDS and Functional Programming

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Why Functional Programming (FP)?

Some of the main reasons for the steep increase in the adoption of Functional Programming languages are:

☐ Referential Transparency (no side effects): makes it easier to write reusable code and facilitates concurrency (increasingly more important due to multi-cores)

☐ Declarative style: makes code easier to understand (once you understand the language)

☐ Higher Order abstractions: Higher order functions, pattern matching, algebraic data types, Higher Kinds, makes it easier to express solutions in our domain

☐ Domain Specific Languages (DSL): Functional programming languages like Haskel and Scala make it easy to develop DSLs

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Why DDS and FP?☐ Data Centricity and Stream Computing match very naturally with

functional programming

☐ All of a sudden your system can be easily modeled as streams of data transformed by composable functions

☐ Referential transparency makes it easily parallelizable

F(x)G(y,z)

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Example☐ Compute the moving average for temperature and humidity

struct  TempSensor  {      long  id;      float  temp;      float  hum;};#pragma  keylist  TempSensor  id

val  tst    =  Topic[TempSensor]("TempSensorT")val  atst  =  Topic[TempSensor]("AverageTempSensorT")

val  rqos  =  DataReaderQos()  <=  History.KeepLast(windowSize)val  dr  =  DataReader[TempSensor](tst,  rqos)val  dw  =  DataWriter[TempSensor](atst)

reader.reactions  +=  {    case  DataAvailable(_)  =>  {                val  window  =  dr.history.data        val  tempAvg  =  (0F  /:  window)(_  +  _.temp)  /  window.length        val  humAvg    =  (0F  /:  window)(_  +  _.hum)  /  window.length        dw  !  new  TempSensor(-­‐1,  tempAvg,  humAvg)      }}

Ope

nSpl

ice

DD

S

List Comprehension

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

List Comprehension in DDS

☐ When using the SCALA API, the DDS readers expose data as LinearSeqOptimized thus making possible list comprehension to address quite a few task

☐ Example☐ dr read foreach (println)☐ dr read map (transformer)☐ ...

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Examples☐ Compute the average shape and write it as a Gray Square:

val data = dr historyval s0 = new ShapeType(“GRAY”, 0, 0 ,0)

dw ! scaleShape((s0 /: data) (sumShape), data.size)

☐ Filter samples:

(dr read) filter (_.x <= 100 && _.y <= 200)

Ope

nSpl

ice

DD

S

References

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DDS ISO C++ API 2010

☐ github.com/kydos

☐ New C++ API

☐ Demo applications

☐ Alpha reference implementation (simd-cxx)

☐ online docs ☐ kydos.github.com/dds-psm-cxx/

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DDS Java 5 API 2010

☐ github.com/kydos

☐ New Java API

☐ Demo applications

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

DDS Scala API☐ github.com/kydos

☐ Scala API for DDS

☐ Set of demo showing how the API can be used

☐ Notice that this API can also be used for scripting DDS applications

Ope

nSpl

ice

DD

S

Summing Up

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

Concluding Remarks☐ OpenSplice DDS provides a very powerful abstraction for building

distributed applications grounded on the concept of a fully distributed global data space

☐ Topics are used to define the types that populate the global data space while DataWriters and DataReaders are used to produce and consume values

☐ DataReaders allow very flexible data access through cache access operations

Ope

nSpl

ice

DD

S

Copyrig

ht  2011,  PrismTech  –    A

ll  Rights  Reserved.

Ope

nSpl

ice

DD

S

References

¥Fastest growing JVM Language¥Open Source¥www.scala-lang.org

¥ #1 OMG DDS Implementation¥ Open Source¥ www.opensplice.org

OpenSplice | DDS¥Scala API for OpenSplice DDS¥Open Source¥github.com/kydos/escalier

Escalier

¥Simple C++ API for DDS¥Open Source¥github.com/kydos/simd-cxx

¥DDS-PSM-Java for OpenSplice DDS¥Open Source¥github.com/kydos/simd-java

¥ DDS-based Advanced Distributed Algorithms Toolkit

¥Open Source¥github.com/kydos/dada