data distribution service dds tutorial building internet of things systems with ease
TRANSCRIPT
AngeloCorsaro,PhDCTO,ADLINKTech.Inc.Co-Chair,[email protected]
The DDS TutorialBuildingIoTSystemswithEase
Internet of Things
The Industrial Internet Consortium (IIC) and the Industrie 4.0 (I4.0)
have defined reference models and architectures for IIoT systems.
As IIoT requirements are a superset of CIoT’s we’ll
investigate the need of the former
CIoT and IIoT
Interoperability. Machines, devices, sensors, and people can freely communicate with each other
Information Transparency. A virtual representation of the physical world is made available by enriching digital plant models with sensor data
Technical assistance. Leverage information to make more informed decisions and solving urgent problems on short notice. Physically support humans by conducting a range of tasks that are unpleasant, too exhausting, or unsafe for humans.
Decentralised Decisions. Autonomous decisions are the norm. Higher level delegation happens only in presence of interferences or conflicting goals
Industrie 4.0Design Principles
The Industrie 4.0 Reference Architecture (RAMI) is three dimensional and organises
the life-cycle/value streams and the manufacturing
hierarchy levels across the six layers of the IT
representation of Industrie 4.0
RAMI 4.0
Imagefrom:“ReferenceArchitectureModelIndustrie4.0”
The ability to virtualise physical entities and make
information available is key to RAMI4.0 and captured
as part of the Virtual Representation layer
RAMI 4.0
Imagefrom:“ReferenceArchitectureModelIndustrie4.0”
The IIC Reference architecture is defined on a three tier model and
across four different view points
The key requirements for each viewpoint and tier are identified by
the IIRA
IIRA
Imagefrom:“IndustrialInternetConsortiumReferenceImplementationv1.7”
IIRA connectivity foresees the use of a Connectivity
Core Standard (such as DDS) and then Gateways
to integrate other connectivity technologies
IIRA Connectivity
Imagefrom:“IndustrialInternetReferenceArchitecturev1.7”
IIRA/I4.0 relationship
Connectivity, syntactic interoperability and ubiquitously available virtual representation of physical entities are common needs of both the IIRA and RAMI4.0
DDS, OPC-UA, MQTT, AMQP are some of the key the standards that have been identified as supporting some or all the connectivity and data sharing needs of IIRA/RAMI4.0
IIRA /RAMI4.ocommonalities
Desirable Properties of a Large Scale Distributed System
Performance
Performance requirements in IIoT
can vary from real-time low latency to
soft real-time and interactive Rea
l-Time
SoftRe
al-Time
Interact
ive
Scalability
The architecture should be designed to promote
scale-out and scale-down as opposed to
scale up
Loose Coupling
Functionalities should be completely isolated
from each other in time and space
Unconstrained
Innovation should not be constrained with respect to
supporting devices, new target platforms, new visualisation
engines, new control or optimisation strategies, etc.
Functionalities should be implemented in the most
effective programming language
Innovation
Incremental
The platform should be as modular as possible to facilitate the individual
functionalities updates and upgrade
Additionally, adding new features should be as much as
possible transparent for the currently running system
Change
Fault-Tolerance
The system should be self-healing and
autonomous
DDS: The Data Distribution Service
DDS. Describes the semantics of the information sharing
abstraction supported by DDS. Defines a nominal type system for
describing DDS information models and ensuring syntactical
interoperability.
DDSI-RTPS. Defines a protocol for interoperable wire implementation
of the DDS semantics.
Standard Structure
DDS-XTypes. Extends the DDS type system with support for structural typing as well as a
dynamic type definition.
DDS-Security. Introduces data centric security in DDS for data in
movement as as well as data at rest.
Standard Structure
DDS-RPC. Extends DDS with support for Remote Procedure Calls.
DDS-PSM-*. Defines highly ergonomic and optimised API
mapping for specific programming languages instead of deriving those
for the DDS-PSM-IDL
DLRL. Defines a language independent Object/Relational
Mapping for DDS
Standard Structure
DDS functionalities span from the Session to the Application level in the
ISO OSI Stack
DDS & ISO OSI
UDP TCP
DDSI-RTPS
DDS
User App.
7. ApplicationHigh-level APIs, including resource sharing, remote file access, directory services and virtual terminals
6. Presentation
Translation of data between a networking service and an application; including character encoding, data compression and encryption/decryption
5. Session
Managing communication sessions, i.e. continuous exchange of information in the form of multiple back-and-forth transmissions between two nodes
4. Transport
Reliable transmission of data segments between points on a network, including segmentation, acknowledgement and multiplexing
3. NetworkStructuring and managing a multi-node network, including addressing, routing and traffic control
2. Data LinkReliable transmission of data frames between two nodes connected by a physical layer
1. PhysicalTransmission and reception of raw bit streams over a physical medium
IP
Programming language, Operating System,
and HW architecture Independent
PlatformIndependent
DDS Abstractions
Applications can autonomously and
asynchronously read and write data enjoying
spatial and temporal decoupling
Virtualised Data Space
DDS Global Data Space
...
Data Writer
Data Writer
Data Writer
Data Reader
Data Reader
Data Reader
Data Reader
Data Writer
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
Virtualised Data Space
Virtualised Data Space
Data Writer
Virtualised Data Space
Data Writer
Virtualised Data Space
Virtualised Data Space
Data Reader
DDS’s virtualised data space is key in enabling loose coupling essential to deal with the fault-tolerance, scale and the heterogeneity needs of IIoT systems
Virtualised Data Space
DDS’s Data Space is eventually consistent with
respect to writes
That means that readers of some kind of data will
eventually see a write, but they may not observe it at
the “same time”
CONSISTENCYMODEL
DDS Global Data Space
...
Data Writer
Data Writer
Data Writer
Data Reader
Data Reader
Data Reader
Data Reader
Data Writer
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
A Topic defines a domain-wide information’s class by a
<name, type, qos> triple
DDS Topics allow to express functional and non-
functional properties of a system information model
Topic
DDS Global Data Space
...
Data Writer
Data Writer
Data Writer
Data Reader
Data Reader
Data Reader
Data Reader
Data Writer
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
TopicType
Name
QoS
Topic types can be expressed using
different syntaxes, including IDL and
ProtoBuf
Topic Type
struct TempSensor { long sid; float temp; float hum; float precision; }; #pragma keylist CarDynamics cid
IDL
Built-in dynamic discovery isolates applications from
network topology and connectivity details
Dynamic
DDS Global Data Space
...
Data Writer
Data Writer
Data Writer
Data Reader
Data Reader
Data Reader
Data Reader
Data Writer
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
Discovery
locationtransparency
Cloud Computing
Fog Computing
Device-to-Cloud Communication
Device-to-Device Communication
Fog-to-Cloud Communication
Cloud-to-Cloud Communication
Device-to-Device Communication
Collect | Store | Analyse | Share
Fog Computing
Fog Computing
No single point of failure or bottleneck
Decentralised
Data Writer
Data Writer
Data Writer
Data Reader
Data Reader
Data Reader
Data Writer
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
TopicDQoS
TopicDQoS
TopicAQoS
Data-Space
DDS Global Data Space
...
Data Writer
Data Writer
Data Writer
Data Reader
Data Reader
Data Reader
Data Reader
Data Writer
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
Connectivity is dynamically adapted
to choose the most effective way of
sharing data
Adaptive
Data Writer
Data Writer
Data Writer
Data Reader
Data Reader
Data Reader
Data Writer
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
TopicDQoS
TopicDQoS
TopicAQoS
ThecommunicationbetweentheDataWriterandmatchingDataReaderscanbepeer-to-peerexploitingUDP/IP(UnicastandMulticast)orTCP/IP
ThecommunicationbetweentheDataWriterandmatchingDataReaderscanbe“brokered”butstillexploitingUDP/IP(UnicastandMulticast)orTCP/IP
Connectivity
QoS policies allow the expression and control over data’s temporal
and availability constraints
QoS Enabled
DDS Global Data Space
...
Data Writer
Data Writer
Data Writer
Data Reader
Data Reader
Data Reader
Data Reader
Data Writer
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
QoS Policies controlling end-to-end
properties follow a Request vs. Offered
QoS Domain
Participant
DURABILITY
OWENERSHIP
DEADLINE
LATENCY BUDGET
LIVELINESS
RELIABILITY
DEST. ORDER
Publisher
DataWriter
PARTITION
DataReader
Subscriber
DomainParticipant
offered QoS
Topicwrites reads
Domain Idjoins joins
produces-in consumes-from
RxO QoS Policies
requested QoS
DDS support for peer-to-peer communication along with its rich set of QoS enable extremely high and controlled performances
Performance
DDS has applicability across the 6 IT levels
Concerning the SCADA and the life-cycle layers, DDS
applicability depends on the constraints of the device
DDS-XRCE will bring connectivity to extremely small devices, i.e. at
most 100KB of RAM
Applicability
DDS
DDSD
DS
Due to its performance and scalability
characteristics DDS is applicable across the
three theirs identified by the IIRA
Applicability
DD
S
Data Modeling Idioms
A state that is periodically updated
Examples are the reading of a sensor (e.g. Temperature
Sensor), the position of a vehicle, etc.
Soft State
Reliability=>BestEffortDurability=>VolatileHistory=>KeepLast(n)Deadline=>updatePeriodLatencyBudget=>updatePeriod/3DestinationOrder=>SourceTimestamp
A state that is sporadically updated and that often has
temporal persistence requirements
Examples are system configuration, a price
estimate, etc.
Hard State
Reliability=>ReliableDurability=>Transient|PersistentHistory=>KeepLast(n)DestinationOrder=>SourceTimestamp
The occurrence of something noteworthy for
our system
Examples are a collision alert, the temperature
beyond a given threshold, etc.
Events
Reliability=>ReliableDurability=>anyHistory=>KeepAllDestinationOrder=>SourceTimestamp
Data-centric design leverage the same principle of Feedback-
control loops to assert a state
In other terms, the desired state is asserted by writing a topic and
the actual state is monitored.
A control action is taken when the desired and the actual state
differ
Control-Loop
Data-centric design leverage the same principle of Feedback-
control loops to assert a state
In other terms, the desired state is asserted by writing a topic and
the actual state is monitored.
A control action is taken when the desired and the actual state
differ
Control-LoopHard State
microservice
Soft State
DDS. Microservices. IoT.
microservice“The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms.” [2]
Architecturalstyle
Server Tier
ProcessFunctionality
Microservice
Individual functionalities become unit of
deployment and run in their own process
Architectures
Microservice Architectures
Microservices communicate through
some lightweight mechanism
Server Tier
ProcessFunctionality
Server TierBenefits
Scaling microservice applications is easier as
we can scale out individual functionalities
Benefits
Unconstrained Innovation as we
choose the technologies stack that makes the
most sense for implementing a given
feature
Server Tier
ProcessFunctionality
Incremental Change is facilitated allowing
incremental deployment of new functionalities.
Potentially different version of the same micro service
could be running at the same time!
Benefits
Server Tier
ProcessFunctionality
Loose Coupling and High Cohesion are
easier to achieved to preserve as the
“barriers” between functionalities are very
thick
Server Tier
Process Boundary / Share Nothing
Benefits
Performance of microservice
architectures may be degraded by the higher
cost of communication if the right technology is
not used
Server Tier
Challenges
Monolithic Implementation
Microservice Implementation
In-Process Communication
Inter-Process/Host Communication
Deployment and Operation of systems
based on the microservice
architectures is inherently more complex,
especially at scale and when the right
technologies are not used
Server Tier
Challenges
Monolithic Implementation
Microservice Implementation
Putting it All Together
The architectural style promoted by DDS is ideal for Microservices
architectures
Microservices are one of the emerging paradigms to design
IIoT systems
Additionally DDS mitigates some of the challenges posed by
Microservices
DDS + Microsvcs
A DDS-based Microservice Framework
First class support for Data-Centric modelling idioms, i.e.
Soft/Hard State and Events
Currently supporting only JVM as a target
AgentV
AgentV provides a set of microsvcs that facilitate
the development and deployment of IoT
applications with the Intel Edison IoT board
Agentv and Edison IOT
Thanks to Vortex AgentV micro services can be
seamless deployed on anything, devices, fog
nodes and cloud
Agentv and Vortex
Coding Lab
Your First AgentV Microsvc
IIRA and RAMI4.0 define the key architectural attributes and concerns that need to be addressed in IoT systems
DDS provides a very good match for providing the connectivity and syntactical interoperability
DDS-based Microservices provide a very elegant way of composing IoT systems
Summing Up
AgentV