building open source iot cloud

BUILDING OPEN SOURCE IOT CLOUD

Dejan Bosanac

WHO AM I?

Dejan Bosanac

Senior Software Engineer at Red Hat Messaging and Integration background (ActiveMQ, Camel, Fabric8) Focused more on messaging and backend for IoT lately

AGENDA

Describe typical IoT application Describe all the components and common pitfalls Introduce Eclipse IoT projects to the rescue

Kura Hono Kapua

Future

IOT APPLICATIONS

Devices Cloud

Applications

IOT APPLICATIONS

Device

Device

Applica+on

Device Applica+on

Applica+onIOTCloud

DEVICES

Everything from simple sensors to small computers Connectivity

Some are IP enabled Some have just near-range connectivity, like Bluetooth Variety of network protocols used (MQTT, LWM2M,)

Functions Sensors send data temperature sensor Actuators control environment A/C Unit

APPLICATIONS

Enterprise applications, micro-services and everything in between Want to use data generated by devices Want to control devices Want to control the cloud environment more about that in a minute

CLOUD

Connects devices and applications Provides connectivity layer Provides security layer Provides device state Handle device data

PITFALLS

Create a silo application Hard to upgrade and maintain Solution don't scale Security is an afterthought

SOLUTION

Build well maintained open source stack Scalable, secure and maintainable Developers should focus on devices and applications

STACK

Device Gateway - Eclipse Kura - https://www.eclipse.org/kura/ IoT Connector Eclipse Hono - https://projects.eclipse.org/projects/iot.hono IoT Cloud Eclipse Kapua - https://projects.eclipse.org/projects/iot.kapua

GATEWAY

DEVICE GATEWAY

Device IP onboarding Data pre-processing Control devices Operational management

GATEWAY ARCHITECTURE

Data Center

Gateway

Application

Sensors

INSERT DESIGNATOR, IF NEEDED 15

KURA ARCHITECTURE

KURA SERVICES

I/O Services serial, Bluetooth, GPS, Data services - Store and forward data using MQTT, Apache Camel Cloud services request/reply Configuration service OSGi configuration Web administration interface

KURA APPLICATION

OSGi Bundle Deployed and managed by Kura Using Kura APIs to communicate with devices and cloud Apache Camel Integration

KURA APPLICATION public void setCloudService(CloudService cloudService) { cloudService = cloudService;}protected void activate(ComponentContext componentContext, Map properties) { ... // Acquire a Cloud Application Client for this Application cloudClient = cloudService.newCloudClient("greenhouse"); cloudClient.addCloudClientListener(this); ...}protected void doPublish() { ... KuraPayload kuraPayload = new KuraPayload(); kuraPayload.addMetric("temperature", temperature); cloudClient.publish("sensors", kuraPayload, DFLT_QOS, DFLT_RETAIN,DFLT_PRIORITY); ...}

KURA CAMEL

public class MyKuraRouter extends KuraRouter { @Override public void configure() throws Exception {

from("timer://heartbeat"). setBody(constant("Hello")). to("kura-cloud:myApplication/myTopic"); }}

CONNECTOR

CONNECTOR

Messaging infrastructure that connects gateways and applications Eclipse Kura uses MQTT Usually combined with message brokers like Apache ActiveMQ or Eclipse Paho

MQTT

OASIS standard (v3.1.1) Created by IBM and Eurotech Lightweight

Small network message Simple protocol

Pub / Sub Quality of Service Connection failures Very popular in IoT scenarios

LIMITATIONS

Scalability Connections Destinations

Security based on broker addresses General purpose messaging

No message format

ECLIPSE HONO GOALS

Tailored general messaging for IoT solutions Solve recurring problems Provide messaging APIs for common operations Support multiple IoT protocols (MQTT, AMQP, LWM2M,) Support any underlying messaging infrastructure

JMS Kafka

ECLIPSE HONO FEATURES

Scalability Multi-tenancy Device-based security Multi-protocol support

ECLIPSE HONO APIS

AMPQ 1.0 based Defines message formats coming in and out of Hono Defines message exchange patterns

INSERT DESIGNATOR, IF NEEDED 27

AMQP

International Standard (ISO/IEC ISO 19464) Binary Protocol Rich feature set:

conversation multiplexing advanced flow control Type system QoS Guarantees

Symmetrical message exchange No Broker required

INSERT DESIGNATOR, IF NEEDED 28

AMQP

Message(properties: {

correlation-id: 1,to: "$management",reply-to: "/myaddress"

},application-properties: {

"name" -> "newQueue","operation" -> "CREATE","type" -> "org.example.queue"

},application-data: AmqpValue(

Map("max_size" -> "2000Mb"

))

)

It is not a broker It never owns a message It propagates AMQP transfer, settlement and disposition frames between endpoints Message based or link based routing

AMQP ROUTER

Router

/device1

/device2

/device2

INSERT DESIGNATOR, IF NEEDED 30

AMQP ROUTER

It can be deployed in multiple router-broker-endpoint topology Redundant paths Benefits

Better scaling due to more focused tasks Smart routing can be used to partition the traffic Ideal candidate for gateway into the system

INSERT DESIGNATOR, IF NEEDED 31

SCALABLE MESSAGING

Combination of brokers and routers provides powerful tool box Brokers should focus on storing messages Routers should do the rest Allows for horizontal scaling topologies that can solve IoT challenges

ECLIPSE HONO- APIS

Telemetry Command and Control Device Registration Device Lifecycle

ECLIPSE HONO ARCHITECTURE

MQTTDevice

LWM2M Device

AMQP Device

HONO

ProtocolAdapter

ProtocolAdapter

Device Management

DataCollec+on

AMQP

AMQP

AMQP

AMQPAMQP

ECLIPSE HONO ARCHITECTURE

Protocol Adapters Stateless Provide conversion to common protocols used in IoT

MQTT LWM2M HTTP/Rest

Clients devices and Cloud services Connects using AMQP using well defined APIs

ECLIPSE HONO ARCHITECTURE

Client

Client

Client

Router Network

HonoServer

HonoServer

App

App

HonoServer

Brokers

ECLIPSE HONO ARCHITECTURE

Hono server Stateless can be scaled Validates message format Does device-based authentication

ECLIPSE HONO ARCHITECTURE

Router Network Provide connection scalability Routes AMQP messages through the system

Brokers Any AMQP 1.0 compatible system Used to save persistent messages

ECLIPSE HONO TECHNOLOGY

Hono server Vert.x + Qpid Proton Spring Boot Docker

Scalable and Cloud Ready!

ECLIPSE HONO TECHNOLOGY

Scalable messaging Apache Qpid Dispatch Router Apache ActiveMQ Artemis

Scalable and Cloud Ready!

ECLIPSE KAPUA

ECLIPSE KAPUA GOALS

Provide complete IoT Cloud solution Define and Implement needed services Ready to run

ECLIPSE KAPUA ARCHITECTURE

ECLIPSE KAPUA BACKHAND SERVICES

Data Management Device Registry Device Management

ECLIPSE KAPUA FRONTEND SERVICES

Management Console API Gateway

ECLIPSE KAPUA - IMPLEMENTATION

Micro-services oriented Pluggable service locator

Single JVM OSGi Cloud Deployment

ECLIPSE KAPUA 1.0

MQTT based Apache ActiveMQ in the messaging layer JDBC store for services Elasticsearch for data store Single VM deployment

FUTURE

INSERT DESIGNATOR, IF NEEDED 47

INSERT DESIGNATOR, IF NEEDED 48

FUTURE - KURA

More data pre-processing BPM integration

AMQP support Gateway support proxy

INSERT DESIGNATOR, IF NEEDED 49

KAPUA

Micorservice implementation Docker images REST/AMQP APIs

Hono support

KAPUA + HONO

CONCLUSION

Together Kura, Hono and Kapua will be able to answer even the most challenging IOT demands Lots of work ahead Everything open source Backed by big companies, like Red Hat, Eurotech, Bosch, Join the effort

THANK YOU

https://www.eclipse.org/kura/

https://projects.eclipse.org/projects/iot.hono

https://projects.eclipse.org/projects/iot.kapua