vortex: the data-driven iot platform
DESCRIPTION
This presentation provides a gentle introduction to the features and advantages of a DDS-enabled data-driven architecture.TRANSCRIPT
Title Summary
Vortex: The Data Driven IoT Platform
Sander Mertens, Solutions Architect October, 2014
Contents
The OMG Data Distribution Service
The Data-‐centric paradigm
The Vortex Product Family
Industry adoption
OMG DDSData Distribution Service
About the OMG-‐DDS standard
Designed for mission / business critical systems
Maintained by the Object Management Group
Programming language/OS independent
Has been in use for 20+ years
OMG DDS comparison
Deliver Unpack Interpret Build
OMG DDSMQTT, XMPP, WebSockets, JMS Google Protocol Buffers DBMS
Layer 1-‐5 Layer 6 Layer 7 Layer 7
Network transport Serialization Data centricity Business logic
OMG DDS comparison (continued)
DDS
AMQP, MQTT, JMS
Sockets Connectivity
Messaging
Information
= Higher level of abstraction, less code
= Lower level of abstraction, more code
Persistence, Content filtering, Ownership
Reliability, Queues
IP, TCP, UDP
The data-‐centric paradigm: conceptual model
Message-‐centric Data-‐centric• Participants “chat” with each other
• Information is derived from messages
• Participants condense messages to a
local state
• Participants subscribe to information
• Information is managed in global dataspace
• Participants obtain global state from DDS
Local state
Global dataspace
The data-‐centric paradigm: consistency
Message-‐centric Data-‐centric• Participants must do the bookkeeping to
keep track of a (consistent) state
• Forces complexity to participants
• Global dataspace ensures consistency
throughout the system
• Offloads complexity from participants to
infrastructure
Participant-specific state
Middleware-managed state
The data-‐centric paradigm: fault tolerance
Message-‐centric Data-‐centric
This state is now lost!
No state is lost
• High potential for information loss
• Errors propagate to other participants
• Replacing participants is non-‐trivial
• No information is lost
• Other participants are unaffected
• Replacing participants is seamless
The data-‐centric paradigm: Hello World datamodel
module Hello { struct World { long id; string value; }; #pragma keylist World id };
The data-‐centric paradigm: Hello World example (C++)
int main(int argc, char *arv[]) { // Create DDS DomainParticipant and topic dds::domain::DomainParticipant dp( org::opensplice::domain::default_id() ); dds::topic::Topic<Hello::World> topic( dp, “HelloWorld" ); ! // Create Publisher and DataWriter dds::pub::Publisher pub( dp ); dds::pub::DataWriter<Hello::World> dw( pub, topic, topic.qos() ); !
// Write sample Hello::World sample; sample.id( 0 ); sample.value( "Hello World” ); dw << sample;
return 0; }
The data-‐centric paradigm: how does it work?
Publisher Subscriber
P2P, No message broker!
Automatic discovery on LAN
1 subscriber, unicast
The data-‐centric paradigm: how does it work?
Publisher Subscriber
>1 subscribers, multicast
Participants are unaware of peers. Connectivity is established based on interest.
Automatic switching from unicast to multicast (and vice versa) with multiple subscribers.
The data-‐centric paradigm: how does it work?
Publisher Subscriber
1 subscriber, unicast
2 subscribers, multicast
1 subscriber, unicast
2 subscribers, multicast
Applications can express interest through “topics” of which there can be many.
no subscriber, nothing send
The data-‐centric paradigm: how does it work?
Best effort
Participants do RxO QoS matching
Publisher Subscriber
Best effort
Reliable
No connection: offered < requested
The data-‐centric paradigm: iShapes demo
struct ShapeType { string color; long x; long y; long shapesize; }; #pragma keylist ShapeType color
Each shape-‐kind (Circle, Square, Triangle) represents a Topic
Each shape-‐color represents an Instance (analogous to records in a database)
The data-‐centric paradigm: durability
Publisher Subscriber
Durable data is stored in RAM
Data is replicated on participants
The data-‐centric paradigm: durability
Publisher Subscriber
Data is aligned to late joiners
The data-‐centric paradigm: durability
Publisher Subscriber
Redundant storage for fault tolerance
Save RAM, don’t store.
The data-‐centric paradigm: iShapes durability demo
The data-‐centric paradigm: ownership
20 20
Publisher Subscriber
10
Publishers are annotated with a strength
Subscribers only accept data from highest strength publisher
The data-‐centric paradigm: ownership
20 10
Publisher Subscriber
10
When a publisher stops, leaves or crashes the next publisher in line takes ownership.
The data-‐centric paradigm: iShapes ownership demo
10
20
20
The data-‐centric paradigm: Content filters
Publisher Subscriber
DDS supports a subset of SQL with which data can be filtered
SELECT FROM Circle WHERE color = GREEN
Subscriber only receives data that matches the filter
The data-‐centric paradigm: Time based filters
Publisher Subscriber
Publisher writes 2 samples per second
interval period = 1 sec
Subscriber receives 1 sample per second
The data-‐centric paradigm: iShapes filtering demo
Filter on content
Filter on time
The Intelligent Data-‐Sharing Platform for Business-‐Critical IoT Systems
share information in real time from anywhere with anything using open standards
Typical IoT device platforms
www TCP/DDSITCP/DDSI
TCP/DDSI
UDP/Multicast/DDSI
TCP/DDSI
Datacenters
Webbrowser
Mobile devices
WAN Embedded devices
The Vortex product family
www
Webbrowser
Mobile devices
Embedded devices
Datacenters
The Vortex product family: scaling with Vortex Cloud
Vortex Clouds seamlessly connects applications over a WAN
Vortex Cloud is designed for systems with high availability requirements
Vortex Cloud enables IoT systems of massive scale.
The Vortex product family: scaling with Vortex Cloud
Data won’t leave the device.
When there is no interest, no data is send to/from Vortex Cloud.
The Vortex product family: scaling with Vortex Cloud
Publisher only sees one “proxy” subscriber.
Vortex Cloud brokers discovery and forwards data
There is no limit on the actual number of subscribers
The Vortex product family: scaling with Vortex Cloud
Publisher only pushes relevant content at lowest possible rate.
Vortex Cloud automatically forwards content & time filtering of subscribers to publishers (Q1 2015)
The Vortex product family: scaling with Vortex Cloud
Dynamically increase redundancy to increase capacity and fault-tolerance
The Vortex product family: scaling with Vortex Cloud
Seamlessly interconnect multiple data centers to act as one.
Industry Adoption
Defense systems
PrismTech Defense Systems Solutions !Vortex addresses the real-‐time data distribution needs of defense systems by satisfying a range of requirements at different temporal and geographical scale for applications such as Radar Processors, Naval Combat Management Systems, Land Systems, and next-‐generation Network Centric Systems.
Vortex sets new levels of performance, reliability, scalability, interoperability and fault-‐tolerance for defense systems
Smart Cities
PrismTech Smart Cities Solutions !Vortex facilitates ubiquitous access to real-‐time data from a variety of sensors, enabling every aspect of the city to be monitored and analyzed. The availability of rich information helps drive service innovation; improving the quality of life for residents and fostering a productive environment for business.
Vortex is a key technology solution used to meet the
multi-‐facetted requirements of real-‐time urban space management
Transportation
PrismTech Transportation Solutions !Vortex facilitates ubiquitous access to real-‐time data from a variety of sensors, enabling every aspect of the city to be monitored and analyzed. The availability of rich information helps drive service innovation; improving the quality of life for residents and fostering a productive environment for business.
Vortex provides a unique ability to address the real-‐time data distribution requirements of large scale, complex transport management systems
Smart Energy
PrismTech Smart Energy Solutions !By enabling a high quality, scalable, low-‐latency, real-‐time information infrastructure, Vortex provides a proven solution for smart energy grids. This in turn is helping deliver improved performance, data quality, data compliance and at the same time reduces costs.
Vortex is used to meet the demanding real-‐time data sharing requirements of next generation distributed power management
systems
Healthcare
PrismTech Healthcare Solutions !The proliferation of smart medical devices is creating a new set of challenges with respect to how networks of devices are integrated together. Vortex provides a proven solution for the secure transfer of real-‐time data between medical devices and other information systems, critical in a clinical environment.
Vortex enables a patient’s data to be securely shared in real-‐time between healthcare professionals and other clinical information systems via the Cloud and using mobile technologies
Industrial Automation
PrismTech Industrial Automation Solutions !Internet Protocol-‐based architectures are bringing modularity to Supervisory Control and Data Acquisition (SCADA) applications and enabling systems that are much more flexible and easier to update. Vortex provides a flexible, performant and highly scalable real-‐time data sharing solution for Industrial Automation
Vortex provides the integration, scalability and flexibility to support complex, highly sophisticated and distributed control systems
IT and Networks
PrismTech IT and Network Solutions !The proliferation of both Enterprise and Public networks is driving the growth in the equipment required to support new networking functions (switching, routing, load balancing etc). Vortex provides a proven standards based solution to support the integration, monitoring and management challenges faced by NEPs and their customers.
Vortex meets the data sharing needs of the Internet of Things by providing a real-‐time link between
the edge, the Cloud and mobile
Financial
PrismTech Financial Solutions !The surge in fully automated trading, along with the steady growth of the financial sector, have led to an unprecedented growth in the market data rates. Vortex provides an ultra-‐low latency data distribution solution that can support the millions of messages per second required by next generation financial systems.
With DDS becoming an accepted technology within finance, Vortex is well positioned to meet
the needs of high performance trading applications
Mining and Drilling
PrismTech Mining and Drilling Solutions !PrismTech’s Vortex OpenSplice is a best-‐in-‐class technology that is helping customers meet the demanding requirements of next generation distributed systems in the mining, drilling and production sectors.
Vortex OpenSplice provides exceptional performance, scalability, and reliability required by advanced mining and drilling operations
Robotics
PrismTech Robotics Solutions !Vortex OpenSplice is an ideal solution for organizations seeking to support the development, distribution, and adoption of open source software for use in robotics research, education, and product development.
Vortex OpenSplice is ideal for organizations seeking to implement genuine Open Source
Solutions within the Robotics sector
Thank You!