devising your data movement strategy for iot
TRANSCRIPT
1confidential
Devising your Data Movement Strategy
for IoTShawn McAllisterSeptember, 2016
2confidential
All Aboot ShawnCTO of Solace SystemsMember of OASIS MQTTTechnical CommitteeEarly member of AMQPTechnical CommitteeHockey EnthusiastFrom Canada, eh!
3confidential
Data Movement Between Elements of IoT
Top level system considerations
Architectures for different types of data movement
Which standards to use;where and why
4confidential
Relevant System Requirements
Scale of Devices
Capabilities ofDevices
Message Exchange Patterns
Messaging QoS
TrafficCharacteristics
SecurityIT Application Architecture & Technologies
5confidential
ProblemStatement
Firewall
Firewall
LoadBalancer
? ? ? ?
6confidential
7confidential
8confidential
Message Bus
9confidential
Firewall
Connection Tier
LoadBalancer
EDGE
FOG
Typical IoT CloudDeploymentArchitecture
Enterprise BusFirewall
CORE
10confidential
MQTT Paho
Proprietary Proprietary/JMS/JCA
HTTP/REST In Runtime
AMQP 1.0 Qpid / JMS
CoAP Several OS
P R O T O C O L A P IRelevantProtocols& APIs
11confidential
o Device Constraintso Message Exchange
Patternso Messaging QoSo Traffic Characteristicso Security
Fog
12confidential
Fog
MQTT Paho
Proprietary Proprietary/JMS/JCA
HTTP/REST In Runtime
AMQP 1.0 Qpid / JMS
CoAP Several OS
P R O T O C O L A P I
13confidential
Edgeo # Concurrent
Connectionso “Always on”
Connections?o Burstinesso Congestion /
Prioritizationo Multi-protocol?o QoSo Security
14confidential
Edge
MQTT Paho
Proprietary Proprietary/JMS/JCA
HTTP/REST In Runtime
AMQP 1.0 Qpid / JMS
CoAP Several OS
P R O T O C O L A P I
15confidential
Coreo Less scale, more
concentrationo Need enterprise
featureso Enterprise grade,
popular APIso Support streaming &
microservices
16confidential
Core
MQTT Paho
Proprietary Proprietary/JMS/JCA
HTTP/REST In Runtime
AMQP 1.0 Qpid / JMS
CoAP Several OS
P R O T O C O L A P I
17confidential
BigDataLake
Big Data River BigDataLake
Big Data
18confidential
Enterprise Bus
Connection Tier
Addressing Scalabilityo Command & Control
sending message to a specific device
19confidential
Enterprise Bus
Connection Tier
Addressing Scalabilityo Command & Control
sending message to a specific device
o Device-initiated request/reply
o Among millions of deviceso Across 2 middleware tierso Need Integrated
addressing between Enterprise Bus & Connection Tier at massive scale
20confidential
Enterprise Bus
Connection Tier
Addressing Scalability
Load Balancer
21confidential
Enterprise Bus
Connection Tier
Addressing Scalabilityo Connection Tier
Addressing‐ MQTT Topics
o Enterprise Bus Addressing‐ Other topics & queues
o Need to scale ‐ In an integrated manner‐ Update in realtime‐ Handling bursts of changes
22confidential
Singapore Land Transport Authority – ERP2http://www.straitstimes.com/singapore/transport/ncs-mhi-to-build-islandwide-satellite-based-erp-for-556m
Routing Server
Payments
Analytics
0 0 0
• 200K Concurrent Connections
• MQTT Termination
• Request/reply forfirmware upgrades
• Push Notifications
• Persistent Messaging
• Shock Absorber
• High Throughput
• Fanout
• Slow Consumer Handling
MQTT JMS
MHI OBU
Next-Gen Electronic Road Pricingo Connect 1.5M vehicles
(200K at a time) in real-time for pay per use road tolls
o Suggest better routes, traffic management – real time and long term and value-added services/analytics
23confidential
Bus Monitoringo Collect real-time
location, loading information and more thousands of buses
o Optimize bus movement, fleet management, avoid bunching, forecast arrival time and more
REST/MQTT
Regulator’s Dashboard
Commuter’s Apps and
RIAs
Analytics
DWH
ESB (if needed)
CEP
JMS
Legacy REST
Legacy Gateway
REST/MQTT
24confidential
Can’t predict coming evolution of IoT,and changing infrastructure ain’t cheapNeed flexible data movement platform
25confidential
Strong Architecture• Decoupling of concerns
• Scales to meetneeds of clients
• Right featuresat each layer
Smart Use of Standards• Right standards• Features & exchange patterns
• Optimized for needs of users
• Security• Supported by key products andopen source projects
