cloudbrew 2016 - building iot solution with service fabric
TRANSCRIPT
![Page 1: CloudBrew 2016 - Building IoT solution with Service Fabric](https://reader035.vdocument.in/reader035/viewer/2022070603/587155971a28ab8e5b8b5133/html5/thumbnails/1.jpg)
Onsight
Building IoT solution with Service Fabric
![Page 2: CloudBrew 2016 - Building IoT solution with Service Fabric](https://reader035.vdocument.in/reader035/viewer/2022070603/587155971a28ab8e5b8b5133/html5/thumbnails/2.jpg)
Onsight
Teemu Tapanila Microsoft Azure MVP Architect, OnSight Helsinki oy Microsoft Certified Trainer
[email protected]@TapanilaT
![Page 3: CloudBrew 2016 - Building IoT solution with Service Fabric](https://reader035.vdocument.in/reader035/viewer/2022070603/587155971a28ab8e5b8b5133/html5/thumbnails/3.jpg)
Onsight
Topics Why Azure Service Fabric Device Registration Data Ingestion Data processing Data analysis
![Page 4: CloudBrew 2016 - Building IoT solution with Service Fabric](https://reader035.vdocument.in/reader035/viewer/2022070603/587155971a28ab8e5b8b5133/html5/thumbnails/4.jpg)
Onsight
Microsoft Azure Service FabricA platform for reliable, hyperscale, microservice-based applications
Microservices
Application Programming Models
Azure
WindowsServer Linux
Hosted Clouds
WindowsServer Linux
Service Fabric
Private Clouds
WindowsServer Linux
High Availability
Hyper-Scale
Hybrid Operations
High Density
Rolling Upgrades Stateful
services
Low Latency Fast startup & shutdown
Container Orchestration & lifecycle management Replication &
FailoverSimple
programming models
Load balancing
Self-healingData Partitioning
Automated Rollback
Health Monitoring
Placement Constraints
![Page 5: CloudBrew 2016 - Building IoT solution with Service Fabric](https://reader035.vdocument.in/reader035/viewer/2022070603/587155971a28ab8e5b8b5133/html5/thumbnails/5.jpg)
Onsight
Battle-hardened for over 5 years
Azure Core Infrastructure
thousands of machines
Power BI
Intune
800k devices
Azure SQL Database
1.4 million databases
Bing Cortana
500m evals/sec
Azure Document
DB
billions transactions/wee
k
Skype for Business
Hybrid Ops
Event Hubs
20bn events/day
![Page 6: CloudBrew 2016 - Building IoT solution with Service Fabric](https://reader035.vdocument.in/reader035/viewer/2022070603/587155971a28ab8e5b8b5133/html5/thumbnails/6.jpg)
Onsight
What is a microservice? Is (logic + state) that is independently versioned,
deployed, and scaled Has a unique name that can be resolved
e.g. fabric:/myapplication/myservice Interacts with other microservices over well defined
interfaces and protocols like REST Remains always logically consistent in the presence of
failures Hosted inside a “container” (code + config) Can be written in any language and framework Developed by a small engineering team
![Page 7: CloudBrew 2016 - Building IoT solution with Service Fabric](https://reader035.vdocument.in/reader035/viewer/2022070603/587155971a28ab8e5b8b5133/html5/thumbnails/7.jpg)
Onsight
Types of microservices Stateless microservice
Has either no state or it can be retrieved from an external store
There can be N instances e.g. web frontends, protocol gateways etc.
Stateful microservice Maintain hard, authoritative state N consistent copies achieved through replication and
local persistence e.g. database, documents, workflow, user profile,
shopping cart etc.
![Page 8: CloudBrew 2016 - Building IoT solution with Service Fabric](https://reader035.vdocument.in/reader035/viewer/2022070603/587155971a28ab8e5b8b5133/html5/thumbnails/8.jpg)
Stateless microservice
Demo
![Page 9: CloudBrew 2016 - Building IoT solution with Service Fabric](https://reader035.vdocument.in/reader035/viewer/2022070603/587155971a28ab8e5b8b5133/html5/thumbnails/9.jpg)
A set of independent machines; physical or virtual
Typical datacenter
![Page 10: CloudBrew 2016 - Building IoT solution with Service Fabric](https://reader035.vdocument.in/reader035/viewer/2022070603/587155971a28ab8e5b8b5133/html5/thumbnails/10.jpg)
A set of machines that Service Fabric stitches together to form a cluster
Clusters can scale to 1000s of machines
Cluster: A federation of machines
Node
Node
Node
Node
Node
Node
![Page 11: CloudBrew 2016 - Building IoT solution with Service Fabric](https://reader035.vdocument.in/reader035/viewer/2022070603/587155971a28ab8e5b8b5133/html5/thumbnails/11.jpg)
Machine failure detection
Time = t1
83 76 50 4664 New Node arrived61
Time = t2
8361 50 46
Failures Detected
cluster reconfigured
83 76 64 50 46
Time = t0
Nodes failed
![Page 12: CloudBrew 2016 - Building IoT solution with Service Fabric](https://reader035.vdocument.in/reader035/viewer/2022070603/587155971a28ab8e5b8b5133/html5/thumbnails/12.jpg)
Cluster: System viewSystem Services Failover manager
Cluster manager
Naming
Image store
![Page 13: CloudBrew 2016 - Building IoT solution with Service Fabric](https://reader035.vdocument.in/reader035/viewer/2022070603/587155971a28ab8e5b8b5133/html5/thumbnails/13.jpg)
Service Fabric Explorer
Demo
![Page 14: CloudBrew 2016 - Building IoT solution with Service Fabric](https://reader035.vdocument.in/reader035/viewer/2022070603/587155971a28ab8e5b8b5133/html5/thumbnails/14.jpg)
Queues Storage
3-Tier service pattern
Front End(StatelessWeb)
StatelessMiddle-tierCompute
Cache
Scale with partitioned storage
Increase reliability with queues
Reduce read latency with caches
Manage your own transactions for state consistency
Many moving parts each managed differently
Load Balancer
![Page 15: CloudBrew 2016 - Building IoT solution with Service Fabric](https://reader035.vdocument.in/reader035/viewer/2022070603/587155971a28ab8e5b8b5133/html5/thumbnails/15.jpg)
StatefulMiddle-tierCompute
Stateful services: Simplify design, reduce latency
Front End(StatelessWeb)
data stores used for analytics and disaster recovery
Application state lives in the compute tier
Low Latency reads and writes
Partitions are first class for scale-out
Built in transactions
Fewer moving parts
Load Balancer
![Page 16: CloudBrew 2016 - Building IoT solution with Service Fabric](https://reader035.vdocument.in/reader035/viewer/2022070603/587155971a28ab8e5b8b5133/html5/thumbnails/16.jpg)
Onsight
Device Registration Device needs to be registered when it gets
connected to the system. This includes normally authentication keys and
serial number When device gets registered you should assing
it value which you will later use for sharding
![Page 17: CloudBrew 2016 - Building IoT solution with Service Fabric](https://reader035.vdocument.in/reader035/viewer/2022070603/587155971a28ab8e5b8b5133/html5/thumbnails/17.jpg)
Onsight
Data Ingestion Gathered data from 1 data source is not so
important normally Real-time data ingestion vs batch ingestion
![Page 18: CloudBrew 2016 - Building IoT solution with Service Fabric](https://reader035.vdocument.in/reader035/viewer/2022070603/587155971a28ab8e5b8b5133/html5/thumbnails/18.jpg)
Onsight
Data Ingestion
![Page 19: CloudBrew 2016 - Building IoT solution with Service Fabric](https://reader035.vdocument.in/reader035/viewer/2022070603/587155971a28ab8e5b8b5133/html5/thumbnails/19.jpg)
Onsight
Data processing Visibility to data averages on grouping level
(Machine, Building, Floor or etc) Logic based on value changes
![Page 20: CloudBrew 2016 - Building IoT solution with Service Fabric](https://reader035.vdocument.in/reader035/viewer/2022070603/587155971a28ab8e5b8b5133/html5/thumbnails/20.jpg)
StatefulMiddle-tierCompute
Data ingestion & Processing
Front End(StatelessWeb)
data stores used for analytics and disaster recovery
Load Balancer
![Page 21: CloudBrew 2016 - Building IoT solution with Service Fabric](https://reader035.vdocument.in/reader035/viewer/2022070603/587155971a28ab8e5b8b5133/html5/thumbnails/21.jpg)
Onsight
Data processing
Room Stateful Service
Floor Stateful Service
Building Stateful Service
Campus Stateful Service
![Page 22: CloudBrew 2016 - Building IoT solution with Service Fabric](https://reader035.vdocument.in/reader035/viewer/2022070603/587155971a28ab8e5b8b5133/html5/thumbnails/22.jpg)
Onsight
Data Analysis Data is stored outside of service fabric for
processing Most cases makes sense of using other tools to
build the analysis platform than Service Fabric
![Page 23: CloudBrew 2016 - Building IoT solution with Service Fabric](https://reader035.vdocument.in/reader035/viewer/2022070603/587155971a28ab8e5b8b5133/html5/thumbnails/23.jpg)
Microsoft Azure Service FabricA platform for reliable, hyperscale, microservice-based applications
Microservices
Application Programming Models
Azure
WindowsServer Linux
Hosted Clouds
WindowsServer Linux
Service Fabric
Private Clouds
WindowsServer Linux
High Availability
Hyper-Scale
Hybrid Operations
High Density
Rolling Upgrades Stateful
services
Low Latency Fast startup & shutdown
Container Orchestration & lifecycle management Replication &
FailoverSimple
programming models
Load balancing
Self-healingData Partitioning
Automated Rollback
Health Monitoring
Placement Constraints