ibm_iot_architecture_and_capabilities

33
© 2015 IBM Corporation IBM Analytics IBM Internet of Things - Architecture and Capabilities Pradeep Natarajan - Manager, IoT Solutions and Services April 30th, 2015

Upload: ibminfomanagement

Post on 08-Aug-2015

1.079 views

Category:

Technology


0 download

TRANSCRIPT

Page 1: Ibm_IoT_Architecture_and_Capabilities

© 2015 IBM Corporation

IBM Analytics

IBM Internet of Things - Architecture and Capabilities

Pradeep Natarajan - Manager, IoT Solutions and Services April 30th, 2015

Page 2: Ibm_IoT_Architecture_and_Capabilities

Agenda

• IoT Architectural Components Overview • Informix for the Intelligent Gateway • Informix for the Cloud • IoT Partners and Platforms

2

Page 3: Ibm_IoT_Architecture_and_Capabilities

3

Informix for the Intelligent Gateway

Page 4: Ibm_IoT_Architecture_and_Capabilities

4

Internet of Things Topography

Streams

Deep Analytics Zone

Device/Sensors

Smart Gateways

Sensor Analytics Zone

4

Page 5: Ibm_IoT_Architecture_and_Capabilities

• Gateways can reduce the cost of the backend cloud • Reduces cloud storage by filtering/aggregating/analyzing data locally • Reduces cloud CPU requirements by precomputing values

• Reduces latency since actions can be taken immediately • Intelligent gateways can detect and respond to local events as they

happen rather than waiting for transfer to the cloud

• Some users are not comfortable putting all their data in the cloud • Gateways allow customers to capture and get value from their sensors

without sending data to the cloud

• Protocol Consolidation • Cloud does not need to implement the 100’s of IoT protocols

Over time more and more of the processing will move from the cloud to gateway devices

How Do Gateways Help IoT Solutions?

5

Page 6: Ibm_IoT_Architecture_and_Capabilities

What are the Requirements for a Gateway Database?• The database management system must:

▪ Have a small install footprint, less than 100 MB ▪ Run with low memory requirements – less than 256 MB ▪ Use lossless compression or other techniques to minimize

storage space ▪ Have built-in support for common types of IoT data like time series

and spatial/GIS data ▪ Simple application development supporting both NoSQL and SQL ▪ Driverless, easy access to the data ▪ Require absolutely no administration ▪ Ideally should be able to network multiple gateways together to

create a single distributed database

6

The database must be powerful enough to ingest, process and analyze data in real-time

Page 7: Ibm_IoT_Architecture_and_Capabilities

IBM Informix: The Ideal Database for Gateways

Simple to use ▪ Hands-Free operation – No administration ▪ Supports popular interfaces such as REST and Mongo as well

as ODBC/JDBC ▪ Handles SQL and JSON data in the same database

Performance ▪ One of a kind support for Time Series and Spatial data ▪ Stream data continuously into the database ▪ Run analytics as data arrives ▪ Dynamically add and update analytics when needed ▪ Storage is typically 1/3 the size compared to other vendors

Invisible

Agile

7

Informix is the only database management system perfectly suited to run in Gateways

Page 8: Ibm_IoT_Architecture_and_Capabilities

Sensor Data is Time Series Data• What is a Time Series?

▪ A logically connected set of records ordered by time

• What are the Key Strengths of Informix TimeSeries? ▪ Much less space required

• Typically about 1/3 the space required by other vendors ▪ Queries run orders of magnitude faster

• Unique optimized storage means codes paths are shorter and more data fits in memory

▪ Purpose built streaming data loader for sensor data • Automatically run analytic and/or aggregate functions on new data

▪ Can store structured (SQL) or unstructured (JSON) data for quick application development • REST/ODBC/JDBC/JSON interfaces available

▪ 100’s of functions predefined • Programming APIs available to create your own analytics

8

Page 9: Ibm_IoT_Architecture_and_Capabilities

Traditional Table Approach

Informix TimeSeries Approach

Meter_ID Time KWH Voltage ColN1 1-1-11 12:00 Value 1 Value 2 ……… Value N2 1-1-11 12:00 Value 1 Value 2 ……… Value N3 1-1-11 12:00 Value 1 Value 2 ……… Value N… … … … ……… …1 1-1-11 12:15 Value 1 Value 2 ……… Value N2 1-1-11 12:15 Value 1 Value 2 ……… Value N3 1-1-11 12:15 Value 1 Value 2 ……… Value N… … … … ……… …

Meter_ID Series

1 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]

2 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]

3 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]

4 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]

Traditional Sensor data storage vs Informix TimeSeries Storage

9

Page 10: Ibm_IoT_Architecture_and_Capabilities

IoT Requirements for SpatioTemporal Data

• Many IoT applications have a spatial component to them

▪ Vehicles, cell phones, even pets… tracking is common

• In these cases both location and time is important ▪ Show me the vehicles that have

passed by location X in the last hour

▪ Where has my car been over the last few hours?

• Informix allows you to combine Time series and Spatial data in the same query

10

Page 11: Ibm_IoT_Architecture_and_Capabilities

Informix Handles Structured and Unstructured Data• There is no common format for sensor data

▪ Vendor 1 sends just temperature, Vendor 2 sends temperature plus pressure

• Informix is the only product to handle structured and unstructured data simultaneously in the same database ▪ Store meta data about your devices (e.g. manufacturer) as JSON ▪ Store device history data (e.g. temperature) as SQL time series

• Informix translates between structured and unstructured on the fly ▪ Allows SQL apps to access the unstructured data and Mongo apps to access

the structured data ▪ Structured and unstructured data can be combined in a single query

• “Give me all the temperature data for sensors made by manufacturer X between Jan 1 and Feb 1”

11

Page 12: Ibm_IoT_Architecture_and_Capabilities

Data Access Options

12

MongoDB Client

REST Client

SQLI Client

DRDA Client

Informix DBMS

Informix NoSQL Listener

Informix

• NoSQL ↔ SQL Translation • REST, MongoDB Protocol

Support

• SQLI, DRDA Protocol Support • Relational, Collection, Time

Series, and Spatial Data Support

Spatial

Time Series

JSON Collection

Relational Table

A REST client is any program capable of

making a HTTP request

Page 13: Ibm_IoT_Architecture_and_Capabilities

Data Access Options

13

MongoDB Client

REST Client

SQLI Client

DRDA Client

Informix DBMS

Informix NoSQL Listener

Informix

• NoSQL ↔ SQL Translation • REST, MongoDB Protocol

Support

• SQLI, DRDA Protocol Support

• Relational, Collection, Time Series, and Spatial Data Support

Spatial

Time Series

JSON Collection

Relational Table

You can use the MongoDB PHP drivers

are available for MongoDB with the

NoSQL Listener

Page 14: Ibm_IoT_Architecture_and_Capabilities

What is Node-Red

• nodered.org • A visual tool for wiring the Internet of

Things • Node.js application. Uses building

blocks of reusable code to control flow of data

14

• “Flows” are created to process data.

• Example: Take data in, transpose, process and store in a database and/or publish to a message broker

• Collaboration among node developers

Page 15: Ibm_IoT_Architecture_and_Capabilities

Node-Red Benefits

• Rapid Development • Simple use with JSON • Simple REST • Simple MQTT • Contributor Nodes – simple to use other services

15

Page 16: Ibm_IoT_Architecture_and_Capabilities

Node-RED

16

Page 17: Ibm_IoT_Architecture_and_Capabilities

IoT Developers - Get Started!

• Gateway starter - https://ibm.biz/BdXr2W • Code samples - https://ibm.biz/BdX4QV • Github - https://github.com/IBM-IoT/

17

Page 18: Ibm_IoT_Architecture_and_Capabilities

18

Informix for the Cloud and Operational Zone

Page 19: Ibm_IoT_Architecture_and_Capabilities

19

What are the IoT Requirements for the Cloud?

• Requirements - similar to gateways (but for different reasons): • Potentially 1000’s of servers means zero administration is a must • Data volume adds up very quickly so low storage overhead is required • Data flows into the cloud continuously and must be processed in real-

time • Must be able to handle time series, spatial, and NoSQL data natively

• Additional requirements • Must be able to scale-out • Must be available as a service

The database must be able to ingest, process and analyze data in real-time

Page 20: Ibm_IoT_Architecture_and_Capabilities

20

Why use Informix in the “Operational Zone”?

Simple to use • Hands-Free operation • Supports REST and Mongo APIs as well as ODBC/JDBC • Stores SQL and JSON database in the same database

Highly Available • Close to zero down time • Partition or Hash your data across servers in the cloud • Dynamically add/remove additional servers

Performance • Continuous High Performance Analytics • Specialized support for Time Series and Spatial data

Invisible

Agile

Resilient

Page 21: Ibm_IoT_Architecture_and_Capabilities

21

Shards: Scale-out your Database across Servers or Gateways

• Distribute data among servers by range or hash partitioning

• Each shard can have an associated secondary server for high availability

• Run queries across all shards or a subset of the shards

• Only shards that could qualify are searched

• Shards are searched in parallel • Ignores shards that are offline

Shards in a

Page 22: Ibm_IoT_Architecture_and_Capabilities

22

IoT Analytics - Operational and Big Data Analytics

• Operational Analytics • Needed when results are required in (near) real-time

• Real-time monitoring, situational detection, correlation of events, e.g. • Dynamic advertising based on customer movement • Real-time equipment failure prediction

• Operational analytics are required in gateways as well as the cloud • Gateways need to aggregate, filter, monitor for conditions

• Analytics must run efficiently while new data is being loaded • Must be able to dynamically add and update analytics in the cloud

and gateways

• Big Data analytics are required when you have the time to do “Deeper/Wider” analysis

• Latency between data arrival and results not an issue • Efficient ETL process from the operational repository is a must

Page 23: Ibm_IoT_Architecture_and_Capabilities

23

TCP/IP

Bulk Loader

SQL Queries (from apps)

Informix Warehouse Accelerator

Compressed DB partition

Query Processor

Data Warehouse

Informix SQL

Query Router Results

Informix Warehouse Accelerator: • Connects to Informix via TCP/IP & DRDA • Analyzes, compresses, and loads to memory

• Copy of (portion of) warehouse • Processes routed SQL query and

• returns answer to Informix

Informix Warehouse Accelerator (IWA) In-Memory Analytics

Informix: • Routes SQL queries to accelerator • User need not change SQL or apps. • Can always run query in Informix

• Too short an est. execution time

Page 24: Ibm_IoT_Architecture_and_Capabilities

24

TimeSeries Service on BlueMix

Every IoT deployment will need to store time series data and possibly spatial data

• Provides REST, JSON, ODBC/JDBC based services to:

• Store, retrieve, and join time series, spatial, relational, and NoSQL data

• Built-in time series analytics

• Multi-tenancy support

• Scales out automatically as needed

• No Administration needed

• Quickly and simply add new sensors

• Interface for continuously loading sensor data

• Adapters for MQTT and MessageSight

IoT Foundation Service on BlueMix

BlueMix Services

Page 25: Ibm_IoT_Architecture_and_Capabilities

Docker Engine

• Open source project

• Supported on every major Linux distro (MS Windows in 2015)

• Client-server architecture with daemon deployed on physical or virtual host

• Uses Linux kernel cgroups and namespaces for process resource management and isolation

• Uses copy-on-write filesystem for git-like image change management

25

A portable, lightweight application runtime and packaging tool built on top of kernel container primitives

Build Ship Run

Docker Engine

Page 26: Ibm_IoT_Architecture_and_Capabilities

Containers

26

App A’

Host OSServer

Bins/LibsApp A App B App B’

App B’

App B’

Container

What is a Container • An isolated user space within a running Linux OS • Shared kernel across containers • Direct device access • All packages and data in an isolated run-time, saved as a

filesystem. • Resource management implemented with cgroups • Resource isolation through namespaces

Run

Bins/Libs

Page 27: Ibm_IoT_Architecture_and_Capabilities

Docker Hub

• Enable sharing and collaboration of Docker Images • Private and public repositories of images • Certified base images by ISVs

27

Ship

Docker Hub

$ docker push $ docker pull

Page 28: Ibm_IoT_Architecture_and_Capabilities

Informix on Docker Hub

https://registry.hub.docker.com/u/ibmcom/informix-innovator-c/

• IBM Informix Innovator-C • 12.10.FC4

28

Docker Hub

$docker pull ibmcom/informix-innovator-c

Page 29: Ibm_IoT_Architecture_and_Capabilities

IoT Partners & Platforms

29

Page 30: Ibm_IoT_Architecture_and_Capabilities

IBM Informix and Intel Deliver Leading Edge Solutions for IoT

▪ Informix support for Intel’s new Quark processor, now supporting Intel family, from Quark to Xeon.

▪ Quark port in the free Informix Developer Edition

▪ Informix Developer Edition available as part of standard Intel Gateway developer SDK

Combining IBM and Intel’s strength at the Edge, Gateway and in the Cloud provides an intelligent, e2e IoT solution for rapid time to market.

Intel® Quark™ SoC / Intel® Atom™ / Intel® Xeon™ Windriver McAfee Security Supports OSGI/TR69 Informix Database

30

Page 31: Ibm_IoT_Architecture_and_Capabilities

Summary• IBM Informix - best fit for IoT architecture

• IoT gateway • IoT cloud analytics

• Supported on a wide array of platforms • Best in class embeddability • Native support for sensor data - TimeSeries & Spatial data • Native support for unstructured (JSON) data • Ease of application development - REST access • High availability and dynamic scaling

31

Page 32: Ibm_IoT_Architecture_and_Capabilities

32

Pradeep Natarajan – [email protected]

Page 33: Ibm_IoT_Architecture_and_Capabilities

IBM Analytics © 2015 IBM Corporation33

Legal Disclaimer

• © IBM Corporation 2015. All Rights Reserved. • The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in

this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software.

• References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results.

• If the text contains performance statistics or references to benchmarks, insert the following language; otherwise delete:Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

• If the text includes any customer examples, please confirm we have prior written approval from such customer and insert the following language; otherwise delete:All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer.

• Please review text for proper trademark attribution of IBM products. At first use, each product name must be the full name and include appropriate trademark symbols (e.g., IBM Lotus® Sametime® Unyte™). Subsequent references can drop “IBM” but should include the proper branding (e.g., Lotus Sametime Gateway, or WebSphere Application Server). Please refer to http://www.ibm.com/legal/copytrade.shtml for guidance on which trademarks require the ® or ™ symbol. Do not use abbreviations for IBM product names in your presentation. All product names must be used as adjectives rather than nouns. Please list all of the trademarks that you use in your presentation as follows; delete any not included in your presentation. IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International Business Machines Corporation in the United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both.

• If you reference Adobe® in the text, please mark the first use and include the following; otherwise delete:Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.

• If you reference Java™ in the text, please mark the first use and include the following; otherwise delete:Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

• If you reference Microsoft® and/or Windows® in the text, please mark the first use and include the following, as applicable; otherwise delete:Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.

• If you reference Intel® and/or any of the following Intel products in the text, please mark the first use and include those that you use as follows; otherwise delete:Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.

• If you reference UNIX® in the text, please mark the first use and include the following; otherwise delete:UNIX is a registered trademark of The Open Group in the United States and other countries.

• If you reference Linux® in your presentation, please mark the first use and include the following; otherwise delete:Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others.

• If the text/graphics include screenshots, no actual IBM employee names may be used (even your own), if your screenshots include fictitious company names (e.g., Renovations, Zeta Bank, Acme) please update and insert the following; otherwise delete: All references to [insert fictitious company name] refer to a fictitious company and are used for illustration purposes only.