vortex cloud beyond cloud messaging
TRANSCRIPT
Angelo Corsaro, PhDChief Technology Officer
Beyond Cloud Messaging…Cloud
Cop
yrig
ht P
rism
Tech
, 201
4
Infrastructure as as Service (IaaS): Elastic compute fabric that you can “consume” as opposed to own and manage, e.g., Azure, Amazon EC2, Google Compute Engine, Linode, etc.
Platform as a Service (PaaS): Enhances IaaS with a series of managed middleware and storage services ready to be “consumed” by your application, e.g., Azure Service Bus, Amazon Dynamo, Vortex Cloud
Software as a Service (IaaS): A full application as a service available for you to be consumed, e.g., SalesForces
Cloud ComputingThe main idea behind cloud computing incrementally reduce the number of layers that you are required to own and manage in order to build applications
Cop
yrig
ht P
rism
Tech
, 201
4
Cloud Messaging replaces local infrastructure and integration technologies with an Ubiquitous Internet Service
Cloud Messaging allows companies to elastically and transparently deal with changes of workload and scale
Cloud MessagingCloud Messaging provides a ubiquitous and universally accessible “Internet Service” for distributing messages from producers to consumers
Cop
yrig
ht P
rism
Tech
, 201
4
Virtually all the Cloud Messaging implementation available on the market focus on:
Reliable Messaging
Device-to-Cloud communication pattern, i.e., no cloud no communication
Low update rates
Additionally, a large set of Cloud Messaging implementations impose:
HTTP-based communication, e.g. Amazon SQS
Limit on message payload
Limit on deployment across “regions”
Limit on the Deployment/Licensing model, e.g. no OEM options
“Traditional” Cloud Messaging
Cop
yrig
ht P
rism
Tech
, 201
4
“Traditional” Cloud messaging was designed to address the requirements of IT systems. As such its main focus is on highly reliable messaging and device-to-cloud interactions
The surge of the Internet of Things has posed a series of new requirements for Cloud Messaging, such as:
- Real-Time Data Delivery, i.e. freshness of data is more important than its reliability
- High Data Rates
- Last Value Reliability
- Support for Cloud and Fog Computing architectures
- Support for embedded and real-time platforms
- Network Efficiency
Beyond Traditional Cloud Messaging?
Vortex Cloud
Cop
yrig
ht P
rism
Tech
, 201
4VORTEX Cloud provides a universally accessible “Internet Service” for sharing data between DDS-enabled applications
VORTEX Cloud
Cop
yrig
ht P
rism
Tech
, 201
4
Elastic and Fault-Tolerant
Public/Private Clouds Deployments
Reliable and Best-Effort Data Sharing
Last value Reliability
Unicast and Multicast Communication
Support for enterprise, embedded, and mobile platforms
Source Filtering
Customisable Load-Balancing
VORTEX Cloud
Cop
yrig
ht P
rism
Tech
, 201
4
Efficient Binary Protocol (DDSI)
Multiple Transports:- UDP/IP- TCP/IP- WebSockets
C/C++, Java, Scala, JavaScript, CoffeScript, and C# API
Connectivity to MQTT, AMQP, etc., via VORTEX Gateway
VORTEX Cloud
Cop
yrig
ht P
rism
Tech
, 201
4
VORTEX supports both the Cloud and the Fog Computing Paradigm
VORTEX natively supports:
- Device-to-Device Communication
- Device-to-Cloud Communication
Cloud, Fog and Edge Computing
Cloud Computing
Fog Computing
Device-to-Cloud Communication
Device-to-Device Communication
Fog-to-Cloud Communication
Cloud-to-Cloud Communication
Device-to-Device Communication
Collect | Store | Analyse | Share
Collect | Store | Analyse | Share
Fog Computing
Fog Computing
Cop
yrig
ht P
rism
Tech
, 201
5
Performance
Device-2-Device
• Peer-to-Peer Intra-core latency as low as 8 µs
• Peer-to-Peer latency as low as 30 µs
• Point-to-Point throughput well over 2.5M msg/sec
Device-2-Cloud
• Routing latency as low as 4 µs
• Linear scale out
• 44K* msgs/sec with a single router, 4x times more the average Tweets per second in the world (~6000 tweets/sec)!
*2048 bytes message payload
Deployment Models
Cop
yrig
ht P
rism
Tech
, 201
4
Cloud ComputingDevice-to-Cloud Communication
Peer-to-Peer (Brokerless)
Device-to-Device Communication
Device communicate peer-to-peer within a fog-domain and through Cloud across fog-domains
Some device concurrently communicate with peers and the cloud
Device-to-Cloud Communication
Device-to-Cloud Communication
Cop
yrig
ht P
rism
Tech
, 201
4
Cloud and Fog ComputingDevice communicate peer-to-peer within a fog-domain
A Vortex-Fog controls which data is exchanged with the could
Device-to-Cloud Communication
Peer-to-Peer (Brokerless)
Device-to-Device Communication
Fog Computing Fog ComputingFog Computing
Cop
yrig
ht P
rism
Tech
, 201
4
Fog Computing
Peer-to-Peer (Brokerless)
Device-to-Device Communication
Fog domain are transparently federated by Vortex-Fog instances
A Vortex-Fog controls which data is exchanged with the could
Fog Computing
Fog Computing
Fog Computing
Vortex Platform
Cop
yrig
ht P
rism
Tech
, 201
5
Specialised device implementations optimally addressing requirements of OT and IT platforms
VORTEX can readily deal with data ingestion seamlessly integrating with other protocols, e.g. MQTT, CoAP, etc.
VORTEX leverages the DDS standard for interoperability and uniquely extends it with support for Internet Scale systems, mobility and Web 2.0 applications
The VORTEX Platform
PaaS/MaaS
Vortex Cloud Abstraction
Cop
yrig
ht P
rism
Tech
, 201
4
Vortex provides a Distributed Data Space abstraction where applications can autonomously and asynchronously read and write data
Its built-in dynamic discovery isolates applications from network topology and connectivity details
DDS’ Data Space is completely decentralised
High Level Abstraction
DDS Global Data Space
...
Data Writer
Data Writer
Data Writer
Data Reader
Data Reader
Data Reader
Data Reader
Data Writer
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
Cop
yrig
ht P
rism
Tech
, 201
4
DDS supports the definition of Common Information Models.
These data models allow to naturally represent physical and virtual entities characterising the application domain
DDS types are extensible and evolvable, thus allowing incremental updates and upgrades
Data Centricity
Cop
yrig
ht P
rism
Tech
, 201
4
A Topic defines a domain-wide information’s classA Topic is defined by means of a (name, type, qos) tuple, where
• name: identifies the topic within the domain
• type: is the programming language type associated with the topic. Types are extensible and evolvable
• qos: is a collection of policies that express the non-functional properties of this topic, e.g. reliability, persistence, etc.
Topic
TopicTypeName
QoS
struct TemperatureSensor { @key long sid; float temp; float hum;}
Cop
yrig
ht P
rism
Tech
, 201
4Vortex “knows” about application data types and uses this information provide type-safety and content-based routing
Content Awarenessstruct TemperatureSensor { @key long sid; float temp; float hum;}
sid temp hum101 25.3 0.6507 33.2 0.7913 27,5 0.55
1307 26.2 0.67
“temp > 25 OR hum >= 0.6”
sid temp hum101 25.3 0.6507 33.2 0.7
1307 26.2 0.67
Type
TempSensor
Cop
yrig
ht P
rism
Tech
, 201
4
For data to flow from a DataWriter (DW) to one or many DataReader (DR) a few conditions have to apply:
The DR and DW domain participants have to be in the same domain
The partition expression of the DR’s Subscriber and the DW’s Publisher should match (in terms of regular expression match)
The QoS Policies offered by the DW should exceed or match those requested by the DR
Quality of ServiceDomain
Participant
DURABILITY
OWENERSHIP
DEADLINE
LATENCY BUDGET
LIVELINESS
RELIABILITY
DEST. ORDER
Publisher
DataWriter
PARTITION
DataReader
Subscriber
DomainParticipant
offered QoS
Topicwrites reads
Domain Idjoins joins
produces-in consumes-from
RxO QoS Policies
requested QoS
Building App with Vortex
Example1: Strong Typing and Extensibility
Cop
yrig
ht P
rism
Tech
, 201
5
The first step required to build a Vortex application is to define the kind of information associated with a Topic
This can be done using either IDL or Google Protocol Buffer
Topic Definition
Cop
yrig
ht P
rism
Tech
, 201
5
Temperature Sensor
message TempSensor { option (.omg.dds.type) = {name: "dds.TempSensor"}; required string sid = 1 [(.omg.dds.member).key = true]; required float temp = 2; }
struct TempSensor { string sid; float temp;}; #pragma keylist TempSensor sid
IDL Google Protocol Buffer
Cop
yrig
ht P
rism
Tech
, 201
5
Data Producerobject TempSensorPub { val builder = TempSensor.newBuilder() def main(args: Array[String]): Unit = { if (args.length > 1) { val sid = args(0) val period = args(1).toInt val topic = Topic[TempSensor]("TempSensor") val dw = DataWriter[TempSensor](topic) while (true) { val s = readTempSensor(sid) dw.write(s) println(show (s)) Thread.sleep(period) } } else { println("USAGE:\n\tTempSensorPub <sid> <period>") } }}
def readTempSensor(sid: String): TempSensor = { val minTemp = -20 val tdelta = 100; val minHum = 0.1F val hdelta = 9 val t = (tdelta*Math.random()).toInt + minTemp val h = (hdelta*Math.random()).toInt + minHum builder.setSid(sid) builder.setTemp(t) builder.setHum(h) builder.build() }
Cop
yrig
ht P
rism
Tech
, 201
5
Data Consumerobject TempSensorSub { def main(args: Array[String]): Unit = { val topic = Topic[TempSensor]("TempSensor") val dr = DataReader[TempSensor](topic) dr.listen { case DataAvailable(_) => dr.take() .filter(s => s.getData != null) .map(_.getData) .foreach(ts => println(show(ts))) } Thread.currentThread().join() }}
Cop
yrig
ht P
rism
Tech
, 201
5
Suppose that we want now to use a more advanced sensor that provides an estimate of the humidity in addition to the temperature
At the same time, while extending the type of our Temperature Sensor we want older application to continue to work seamlessly with the new sensor
Dealing with Type Evolution
Cop
yrig
ht P
rism
Tech
, 201
5
The humidity attribute is declared optional to allow older consumer to match this type
At the same time, new consumer will be able to detect that old temperature sensor are not providing humidity value
Extended Temperature Sensor Type
message TempSensor { option (.omg.dds.type) = {name: "dds.TempSensor"}; required string sid = 1 [(.omg.dds.member).key = true]; required float temp = 2; optional float hum = 3; }
Example2: Real-Time Web Applications
Cop
yrig
ht P
rism
Tech
, 201
5
Vortex Cloud provides a very effective way of sharing data between HTML5/JavaScript Applications
Additionally, Vortex Cloud can be used to seamlessly share data between Web and embedded applications
Real-Web Apps with Vortex Cloud
Cop
yrig
ht P
rism
Tech
, 201
4Let’s see the steps required to build a Web Chat that may look like this
But before let’s play with it a bit
Web Chat
demo.prismtech.com
Cop
yrig
ht P
rism
Tech
, 201
4
When using Vortex Cloud with web application i topic types can be declared JavaScript/CoffeScript
Topic Declaration
# The post type used by the chat applicationclass Post constructor: (@name, @msg) ->
Cop
yrig
ht P
rism
Tech
, 201
4
The Chat CoffeeScript# Create useful alias for coffez and jQueryroot = thisz_ = coffez$ = jQuery
server = “ws://demo-eu.prismtech.com:9999"
# The post type used by the chat applicationclass Post constructor: (@name, @msg) ->
# Create the runtimeruntime = new dds.runtime.Runtime()
# Define the Post topic used to send and receive chat postspostTopic = new dds.Topic(0, "Post")
# Define the QoS for the DataReader/WriterdrQos = new dds.DataReaderQos(dds.Reliability.Reliable)dwQos = new dds.DataWriterQos(dds.Reliability.Reliable)
Cop
yrig
ht P
rism
Tech
, 201
4
The Chat CoffeeScriptpostReader = z_.NonepostWriter = z_.None
avatar = "avatar" + Math.floor((Math.random() * 10000) + 1);
# Add post to the chat and format it to show it is from mecreateMyPost = (post) -> ...
# Add post to the chat and format it to show it is from otherscreateOtherPost = (post) -> ...
# Add post to the chat and format it to show it is from othersprocessPost = () -> msg = $("#ChatMessage").val() post = new Post(avatar, msg) # Publish the post (notice that postWriter is an Option Monad) # Take a look at (http://en.wikibooks.org/wiki/Haskell/Understanding_monads/Maybe) # or (http://www.scala-lang.org/api/2.11.0/index.html#scala.Option) postWriter.map((dw) -> dw.write(post)) $("#ChatMessageList").append(createMyPost(post)) $("#ChatMessage").val("")
Cop
yrig
ht P
rism
Tech
, 201
4
The Chat CoffeeScript# Deal with click and keys events…
$("#ChatMessage").keyup( (e) -> if(e.keyCode is 13) then processPost())
$("#SendMsgButton").click( (evt) -> console.log("Send Button has been clicked") processPost())
$("#SelectAvatarButton").click( (evt) -> s = $("#AvatarName").val() if (s isnt "") avatar = s)
Cop
yrig
ht P
rism
Tech
, 201
4
The Chat CoffeeScript# Handle the runtime onconnect eventruntime.onconnect = () -> # Create DataReader and DataWriter for our posts dr = new dds.DataReader(runtime, postTopic, drQos) dw = new dds.DataWriter(runtime, postTopic, dwQos)
# Register a listener with the data reader to post messages # in our chat dr.addListener( (post) -> if (post.name isnt avatar) $("#ChatMessageList").append(createOtherPost(post))) postReader = z_.Some(dr) postWriter = z_.Some(dw)
connectRuntime = () -> $("#AvatarName").val(avatar) runtime.connect(server, "uid:pwd")
$(document).ready(() -> connectRuntime())
Putting it all Together
Cop
yrig
ht P
rism
Tech
, 201
4
Cop
yrig
ht P
rism
Tech
, 201
4
Vortex Cloud goes beyond traditional Cloud Messaging solutions by providing support for:
- Real-Time Data Delivery, i.e. freshness of data is more important than its reliability
- High Data Rates
- Last Value Reliability
- Support for Cloud and Fog Computing architectures
- Support for embedded and real-time platforms
- High Network Efficiency
Vortex Cloud is the ideal PaaS infrastructure for Industrial and Consumer IoT applications
Summing Up
Cop
yrig
ht P
rism
Tech
, 201
5