from drones to cars - amazon s3 · pdf filefrom drones to cars ... communication protocol...

© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Jan Metzner Solutions Architect, Amazon Web Services

From Drones to Cars Connecting the Devices in Motion to the Cloud

Pop-up Loft

What to Expect from the Session

•  Understand the challenges •  Architecture building blocks •  Architecture patterns •  Examples •  Demos •  Practical experience and guidelines

IoT in Motion at Monsanto

IoT creates new opportunities in precision agriculture

4

Plan Experiment

Plant

Monitor/apply Harvest

Analyze

Monsanto tests products all around the world

5

Data needs to be reliably gathered from various equipment in remote locations where data connectivity is unreliable

Plan

Plant

Monitor Harvest

Analyze

Plan

Plant

Monitor Harvest

Analyze

Plan

Plant

Monitor Harvest

Analyze

Plan

Plant

Monitor Harvest

Analyze

Plan

Plant

Monitor Harvest

Analyze

The formula for reliable data transfer and processing

6

Store and forward on

edge

Low overhead communication

protocol

Broker-based processing in

cloud

Data ingestion and storage

Analytics Cloud code

Communication Rest API Visualizations

HTTP

Devices in motion

Devices in motion carry a lot of value

•  Automated harvesting •  Automated transport •  Connected cars and vehicles •  Aviation •  Sports, gaming, leisure •  Medical and public service •  Tourism

Technical characteristics

•  Mobile networks: kBit/s •  Lightweight:

Every gram, every watt counts Low RAM, low CPU, low power

•  Near real-time: Seconds count •  Location-based •  Large-impact use cases

Dash streams more than 1 TB of real-time data per day

Dash connects your smartphone to your car with a simple, low-cost device which provides real-time road

safety information to drivers.

As a startup, using AWS has allowed us to scale nicely and use resources without spending a lot

of capital. Brian Langel CTO, Dash

”

“ •  Real-time information for drivers

•  Streams more than 1 TB of real-time data per day over Amazon Kinesis

•  Processes billions records using Amazon DynamoDB

•  Thousands of updates per second during spikes

•  Reduced operating costs by $200,000 per year

Devices in motion: Challenges

•  How to handle constrained resources? •  How to manage millions of things? •  How to communicate securely? •  How to deal with unreliable connections? •  How to handle geo-location? •  How to deal with map data?

?


•  How to handle constrained resources? •  How to manage millions of things? •  How to communicate securely? •  How to deal with unreliable connections? •  How to handle Geo-Location? •  How to deal with map data?

?

MQTT

MQTT vs HTTPS: •  93x faster throughput •  11.89x less battery to send •  170.9x less battery to receive •  50% less power to keep connected •  8x less network overhead

Source: http://stephendnicholas.com/archives/1217

•  OASIS standard protocol (v3.1.1) •  Lightweight, pub-sub, transport protocol

that is useful for connected devices •  MQTT is used on oil rigs, connected

trucks, and many more sensitive and resource-sensitive scenarios

•  Customers have needed to build, maintain, and scale a broker to use MQTT with cloud applications



? ✔

Highly scalable Pub/Sub broker

MQTT

Subscribers

Publishers

Secure by default Connect securely via X509 certs and TLS v1.2 client mutual auth

Multi-protocol message gateway Millions of devices and apps can connect over MQTT or HTTP

topic Elastic Pub/Sub broker Go from 1 to 1 billion long-lived, bi-directional connections with zero provisioning

AWS IoT: Securely connect devices

Devices

Apps

Secure Pub/Sub broker

AWS IoT Pub/Sub broker

1. AWS services (direct integration)

Rules Engine

Actions

AWS IoT Rules Engine

Lambda SNS SQS

S3 Amazon Kinesis DDB RDS

Amazon Redshift

Amazon Glacier

EC2

3. External endpoints (via Lambda and SNS)

Rules Engine connects AWS IoT to external endpoints and AWS services.

2. Rest of AWS (via Amazon Kinesis, Lambda, S3, and more)



? ✔ ✔

AWS IoT security

Authentication Authorization

MQTT/HTTP over TLS v1.2 (X.509 certs) cipher suites TLS mutual auth: (Client/server certificates for things) HTTPS over sigv4 (for apps) Amazon Cognito/IAM (for mobile apps)

Certificates are attached to policies Policies (JSON) give granular access to AWS IoT control and data plane Policies allow and deny permissions on specific resources (rules, shadows, topics)

Devices

Apps


AWS IoT Pub/Sub broker

MQTT/TLS

HTTP/TLS

MQTT/TLS

MQTT/TLS

MQTT/TLS



? ✔ ✔ ✔

Apps Shadow


Thing SDK

AWS IoT Device Shadows

AWS IoT Device Shadows flow

Shadow

Thing SDK

1. Device publishes current state

2. Persist JSON data store

3. App requests device’s current state

4. App requests change the state 5. Device Shadow syncs updated state

6. Device publishes current state 7. Device Shadow confirms state change



? ✔ ✔ ✔ ✔

Geo-location and mapping use cases

•  Tracking •  Geo-fencing •  Find nearby objects •  Alert nearby things •  Map matching •  Routing

Geohashing

•  Easy to compute •  Arbitrary precision •  Similar positions yield

similar prefixes •  Removing characters reduces

precision •  Simple way to look up and manage

geo objects

Picture: https://github.com/stephenlb/geohash-chat-by-proximity

Distribute messages to geo-located things with topic tiles

•  One topic per geohash •  Things subscribe to their

current geohash tile’s topic as they move

•  Send messages to tile topics •  Use geohash lengths to

change area impact of message

Geo library for Amazon DynamoDB

•  Ready-to-use Java JAR •  Library and sample apps

available on GitHub •  Supports GeoJSON •  Uses geohashes for

DynamoDB local secondary indexes

•  Simplifies box and radius queries

AWS IoT Service and DynamoDB with Geohashing Architecture

Rules Engine Send lat/lon AWS Lambda +

Geo library for Amazon DynamoDB

Look up geohash

Moving thing

Amazon DynamoDB

Return matches

1 2

3 4

5

Amazon RDS for PostgreSQL with PostGIS

•  Powerful support for: •  Geo data formats •  Geometric projections •  Transform operations •  Query functions

•  Well understood, easy to use

•  Supported on Amazon RDS for PostgreSQL

AWS IoT and Amazon RDS for PostgreSQL

Rules Engine Send lat/lon AWS Lambda

Geo- queries

Moving thing

Amazon RDS for PostgreSQL

Return matches

1 2

3 4

5

Processing map graphs with TitanDB

•  Scalable graph database •  Import maps as graphs •  Supports routing and

other complex map queries

•  Amazon DynamoDB storage back end for TitanDB available

AWS IoT and Amazon DynamoDB for TitanDB


Map queries

Moving thing

Amazon DynamoDB

Return matches

1 2

3 4

5

TitanDB + Amazon DynamoDB

back end

Amazon Elasticsearch Service

•  Powerful, real-time, distributed, open-source •  Supports geolocation searches:

•  Geo-points, geohashes, geo-aggregations, geoshapes

Amazon Elasticsearch Service •  Managed service •  Easy to set up, scale, operate

AWS IoT and Amazon Elasticsearch Service architecture


Run queries

Moving thing

Amazon Elasticsearch

Service

Return matches

1 2

3 4

5



✔ ✔ ✔ ✔ ✔ ✔

Summary

•  Use AWS IoT to manage and communicate with your things

•  Scalable, easy to use MQTT pub/sub message broker •  Powerful AWS IoT Rules Engine

•  Use AWS IoT Device Shadows for unreliable connections •  Take advantage of AWS Lambda to connect to geo-aware

services: •  Amazon DynamoDB with geohash and/or TitanDB support •  Amazon RDS for PostgreSQL with PostGIS •  Amazon Elasticsearch Service

Demo: Connected car safety Search for MBL203

http://www.iteris.com/cvria/html/applications/app74.html#tab-3

https://github.com/smartdevicelink

OBE Vehicle

RSE

Notification function

DMS

Vehicle

GIS

Rules engine Rules engine Notification

topic

Traffic information center

Partner notification

Shadow

Reported state

Reported state

Desired state

Environment situation stream

Traffic incident service

OBE

Prediction API

Reported state

Desired state

Desired state

from drones to cars - amazon s3 · pdf filefrom drones to cars ... communication protocol...

Documents