beyond discovery

Beyond Discovery

Shaun FoleyPrincipal Applications Engineer, RTI

Topics

• Unicast Discovery– How can applications discover each other when

multicast is unavailable or undesirable?• Static Endpoint Discovery

– How can I minimize the configuration burden when using RTI Limited Bandwidth Static Discovery Plugins?

• User Discovery– How does DDS discovery fit into the broader

picture of system architecture?

Topics

• Unicast Discovery– How can applications discover each other when

multicast is unavailable or undesirable?• Static Endpoint Discovery• User Discovery

Tell Nathan I am in room 205

AngelaHotel

Reception

Track guests and room numbers.

Nathan

Where is Angela?

Communication established: decide where to meet for beer

Angela : room 205

Call room 205

Tell everyone I am address 205

Application ADiscovery

Proxy

A: 205 B: 408 …

Application B

Where is everyone?

Contact 205Contact 202Contact 408…

Communication established:Begin app biz-logic

Track applications and addresses

Tell everyone I am address 205

Application ADiscovery

Proxy

A: 205 B: 408 …

Application B

Where is everyone?

Contact 205Contact 202Contact 408…

Track applications and addresses

Contact proxy at well-known address

Communication established:Begin app biz-logic


Call add_peer() on received addresses

athena

Domain Participant 0





Remote Participant

Remote Participant

peer list: 3@athena

peer list: 1@athena

discovery announcements

An Implementation Aside:Peer Indices and Addresses

struct PeerEntry { string addr; //@key ulong pidx; //@key};

struct PeerEntry { string addr; //@key sequence<ulong> pidxs;};

struct PeerEntry { string addr; //@key ulong max_pidx;};

Data Types

Publishes unneeded data.Resource limits depend on # of expected DPs on node.

Must choose arbitrary sequence bound.

Last value cache.Relies on idempotent add_peer().Only removes peer when last DP on machine dies.

Apps only care about the highest participant index at a given address:add_peer(“[email protected]”);remove_peer(“[email protected]”);

Discovery Proxy

Address 128Participant Idx 0

Peer Map – published durably

Initial peers: 0@128Multicast rx addrs: empty

Addr

PIdx

Application

Discovery Proxy





128 1

Addr 128

PIdx 1

Application

Discovery Proxy


Application





128 1

128 2

Addr 128 128

PIdx 1 2

ApplicationDiscovery

Proxy

Normal application functions


When proxy tells us about new applications, call add_peer()

proxy_client.h

Three lines of code to change normal application to use unicast discovery proxy

Application

Discovery Proxy


Application





128 1

128 2

Addr 128 128

PIdx 1 2

Application

Discovery Proxy


Application


rtiddsspy





128 1

10

128 2

Addr 128 128 1

PIdx 1 2 0

Topics

• Unicast Discovery• Static Endpoint Discovery


• User Discovery

Why Static Discovery?

• Unicast Discovery• Static Endpoint Discovery


• User Discovery

A Participant A sends to initial peers

Participant B respondsB

~400 bytes, but varies from…- Propagated properties- Entity name- # of Locators

RWExchange DW/DR info

Participant A Participant B

~800 bytes, but varies from…- Propagated properties- Entity name- Type description

Participant Announcement

Endpoint Announcement

Why Static Discovery?

Federation with Routing Service

Routing Service

Fully meshed discovery within cluster

Routing Service

Routing Services act as gateways for cluster

• May be difficult to choose meaningful clusters• All cross-cluster traffic passes through Routing Services

A Participant A sends to initial peers

Participant B respondsB

Participant A Participant B

Endpoint Discovery with Limited Bandwidth Plugins

Read LBED configuration to learn about remote participant’s datareaders and datawriters

R

RW

No endpoint data sent on the wire!

Participant X

DW Aroid 1

X

DW Broid 2

LBED Configurationparticipant name = “Y” reader rtps_object_id = 3 topic_name = A qos = ... reader rtps_object_id = 4 topic_name = B qos = ... participant name = “X” writer rtps_object_id = 1 topic_name = A qos = ... writer rtps_object_id = 2 topic_name = B qos = ...

Participant Y

DR Aroid 3

DR Broid 4

Look up participant by name “X”

B

AAssert X’s endpointsinto discovery database

Discovery data: QoS, Type description, … User samples

Configuration Challenges

• Applications do not use the LBED configuration to create DDS entities

• Actual configuration and LBED configuration must remain synchronized

• DDS relies on each datawriter or datareader to have a unique object ID

• Object ID assignment must be deterministic– If created serially: maintain DW/DR count– If created in parallel: hash topic name

Participant X

DW Aroid 10

X

DW Broid 20

LBED Misconfigurationparticipant name = “Y” reader rtps_object_id = 3 topic_name = A qos = ... reader rtps_object_id = 4 topic_name = B qos = ... participant name = “X” writer rtps_object_id = 1 topic_name = A qos = ... writer rtps_object_id = 2 topic_name = B qos = ...

Participant Y

DR Aroid 3

DR Broid 4

Look up participant by name “X”

B

AAssert X’s endpointsinto discovery database

Discovery data: QoS, Type description, … User samples

Who is ROID 10 and 20???

X

Participant XDiscovery Monitor

Endpoint A XEndpoint B

Participant Y

Endpoint C Endpoint D

Y Collect Builtin Topic data and save to XML file

BA

CD

Discovery data: QoS, Type description, …

Generating LBED Configuration

Two Steps to Use LBED Configuration

• Tell applications to use load LB plugins• Select previously generated LBED

configuration

• These can be set in participant QoS section via XML

Topics

• Unicast Discovery• Static Endpoint Discovery• User Discovery

– How does DDS discovery fit into the broader picture of system architecture?

Selective Discovery

• DDS participants communicate with and store data about all endpoints, even those they do not match.

• How can we avoid this?

Participant X

Discovery Proxy

Endpoint A

X

User data

Discovery metadata: Address, DP index, Topic name, …


C

Endpoint B

Participant Y


Y

B

A

Y

X

Y

D

process data

process metadata

process metadata

Proxied Participant Location

normal communication

X

Participant X

Discovery Proxy

Endpoint A

X

Endpoint B

Participant Y


f(A,B)

Y

process data

process metadata

process metadata

Proxied Topic Data- no match scenario

BA

C

D

f(C,D)

User data



X

Y

Participant X

Discovery Proxy

Endpoint A

X

Endpoint B

Participant Y


f(A,B)

Y

process data

process metadata

process metadata

Proxied Topic Data- match scenario

BA

C

D

f(C,D)

normal communication

User data



A

C

X

Y

Participant X

Discovery Proxy

Endpoint A

X

Endpoint B

Participant Y


Y

X

Y

process data

process metadata

process metadata

BA

C

Dnormal communication

User data



A

B

A

Control

Data

Discovery Control

• Control information need not be distributed the same way as data– LBED plugin uses files to describe endpoints, DDS

to communicate– WSDL to describe services, SOAP to communicate– DNS to find, {HTTP, DDS, … } to communicate

• How can we maximize control plane flexibility?

Discovery Data

• Fixed data type– Necessary for DDS interoperability

• Uses [internal] DDS writers and readers• Has associated QoS

Participant Built-inData Reader

PublicationBuilt-inData Writer

SubscriptionBuilt-inData Reader

ParticipantBuilt-in Data Writer

SubscriptionBuilt-inData Writer

PublicationBuilt-inData Reader

ParticipantBuilt-inData Reader

PublicationBuilt-inData Writer

SubscriptionBuilt-inData Reader

Participant Built-inData Writer

SubscriptionBuilt-inData Writer

PublicationBuilt-inData Reader

Best Effort Reliable

Application participant

Plugin

Userdisc Plugin participant

PublicationDataWriter

Subscription DataWriter

PublicationDataReader

Subscription DataReader

Discovery database

Bridging Discovery Data to Normal DDS Topics

Application participant

Hello Writer

Plugin

Userdisc Plugin participant

PublicationDataWriter

Subscription DataWriter

PublicationDataReader

Subscription DataReader

core calls…afterLocalWriterEnabled

Topic: “Hello”keyparticipant_key...

Topic: “Hello”keyparticipant_key...

call into core…assertRemoteReader

• Routing Service– System federation and bridging– Interacting with non-DDS data– Mediation and transformation

• Recording and Replay Services• Persistence Service

– Implement DDS durability– Build more complex messagine on top of DDS

• User code– Infrastructure frameworks, programmer familiarity

“Inherit, not to reuse, but to be reused” 1

1: Sutter and Alexandrescu, C++ Coding Standards

Endpoint DescriptionKeyTopic nameType nameQoSType description

Does not change for static discovery

Unused by 95% of applications

Endpoint HeaderKeyTopic nameType name

Endpoint QoSKeyQoS reference or data

Endpoint TypeKeyType description

Published separately

Accessible via pub-sub or request-reply

Easily reconstructed on other side

Today’s way

The Right Way

beyond discovery

Documents

participant idx

highest participant

peer map

durablyinitial peers

multicast rx addrs

peer indices

application adiscovery

given address