serverless architecture - windows user group …...roman kiss microsoft integration mvp (2003...
TRANSCRIPT
Roman KissMicrosoft Integration MVP (2003 – 2013)
August 2017
Serverless ArchitectureWith
Azure FunctionsEvent driven model for modern applications
Serverless architecture with Azure Functions (AF)
• Concept & Design• Architecture • Triggers & Bindings • Runtime compile AF• Pre-compile AF• Continues integration• Templatizing & Declarative Programming
• Usage• Using AF in the Azure Backend processes• Using AF in the Azure Event Grid model• Using AF in the stream pipeline• Using AF in the non-telemetry pipeline• Using AF for workflow mediators
• Deploying, Testing, Monitoring, Optimizing and Pricing
Serverless architecture with Azure Functions (AF)
Gen 3 - FaaS
Serverless architecture with Azure Functions (AF)Function as a Service
Thanks to https://www.opsgility.com/blog/2017/02/23/evolutions-of-serverless-compute-in-microsoft-azure/
Serverless architecture with Azure Functions (AF)Break down your monolithic application
Thanks to https://vkbiztalk.wordpress.com/tag/serverless-architecture/
Serverless Principles
• Abstract completely underlying infrastructure• Break business logic to functions• Focus only on application code• True auto scaling in seconds• Integrate with external systems
Serverless Advantages
• Rapid Development• Truly Polyglot• Event & Metadata Driven• Reduces the complexity of software• Minimal Maintenance• Very Scalable• Simplifies packaging and deployment• Pay Per Use
Serverless Disadvantages
• Not efficient for long-running apps• Architectural complexity (business decomposition)• Vendor lock-in (FaaS is not compatible with each other)• Additional overhead for function/microservices calls• Cold-start• Missing out-of-the-box tools• Multitenancy problem
Serverless architecture with Azure Functions (AF)
Thanks to [email protected]
Serverless architecture with Azure Functions (AF)
Thanks to [email protected]
Serverless architecture with Azure Functions (AF)
Serverless architecture with Azure Functions (AF)
Serverless architecture with Azure Functions (AF)API Gateway (Proxy)
API Gateway
REST
Serverless architecture with Azure Functions (AF)API Gateway + FaaS (Host)
Blob Storage
API Gateway
REST
Host
Host
Serverless architecture with Azure Functions (AF)
Event Driven Models
Thanks to [email protected]
Event Driven – Pull Model
Thanks to [email protected]
Event Driven – Push Model
Thanks to [email protected]
Azure Event Grid
Event Sources
Azure Event Grid Event Message
[{"id": "1807","eventType": "recordInserted","subject": "myapp/vehicles/motorcycles","eventTime": "2017-08-10T21:03:07+00:00","data": {
"make": "Ducati","model": "Monster"
},"topic": "/subscriptions/{subscription-id}/resourceGroups/{resource-group}/providers/Microsoft.EventGrid/topics/{topic}"
}]
public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, CloudBlobContainer container, CloudQueue queue, CloudTable table, TraceWriter log)
{
// code
return req.CreateResponse(HttpStatusCode.OK, “Done”);}
Functions Programming Model
Thanks to https://www.slideshare.net/TokyoAzureMeetup/tokyo-azure-meetup-7-introduction-to-serverless-architectures-with-azure-functions-july-2016
Pre-Compiled Functions Programming Model
Serverless architecture with Azure Functions (AF)Distributed State Machine
Fan-out
Fan-in
Serverless architecture with Azure Functions (AF)
Event Grid
Serverless architectureEventing with Event Grid and Azure Functions
Telemetry Pre-Processing
Azure Function – Telemetry preprocessor
Azure IoT HubAzure Function
htt
ps
Service Bus
Queue
Http Response
Http Trigger
Message Exchange PatternsPost-Processing, etc.
Pull and Push Data Stream to Power BI using an Azure Function
Pre-Processing IoT Events
Azure IoT Hub Azure Event HubAzure Function
Stream Stream Stream
Azure IoT Hub Bridge to the MQTT Broker
Azure IoT Hub Azure Function
Stream
EventHub Trigger
MQTT Client MQTT Broker
Azure Function – Telemetry Pusher to AWS
Azure IoT Hub
AWS
Storage blob
Archive
Azure Event Hub
Routes
Azure Function
Trigger
Azure Storage Blob
(5 minutes)
REST API
Storing The Device Twins Changes
Azure IoT Hub Azure Event Hub
Routes
Azure Function
Trigger
Azure Storage Table
Storage table
TwinChangeEvents
Azure IoT Hub Source of InputsRouting for stream events (ESP/CEP) and message processing
Custom Endpoints
Default Event Hub
Event Hub
Service Bus Topic
R
IoT Hub
Stream Analytics
Azure Functions
/messages/events
Service Bus Queue
Stream Analytics
Azure Functions
S1: 10 endpoints100 rules
Pre-Processing IoT Events
Custom Endpoints
Default Event Hub
R
IoT Hub
/messages/events
Non-Telemetry Path
Azure Functions
…
Event Hub
Telemetry Path
Non-Telemetry
Path
https://myNamespace.azure-devices.net/devices/DeviceXYZ/messages/events?api-version=2016-11-14
Stream Stream
(Device Emulator)
Converting Telemetry Stream to the Messaging
Custom Endpoints
Default Event Hub
R
IoT Hub
/messages/eventsService Bus Queue
Telemetry Path (stream pipeline)
Non-Telemetry Path
Service Bus QueueAzure Functions
…
Azure Function - VETER Message Mediator https://xxxxxx.azurewebsites.net/api/microservice?code=xxxxx&name=HelloCloud
/messages/devicebound
Service Bus Queue
Stream Analytics
IoT Hub
Telemetry Path
C2D Message
Azure Function
IoT Hub
Routes
Event Hub AFNAFN
Validation
Enrichment
Transformation
Ingestion
Things
Downstream
Consumers
PowerBI,
Service Bus,
Storage,
etc.
Storage blob
ref
Storage blob
archive
stream
Non-
telemetry
Blob
StorageService
Bus Queue
Using AFN in the IoT Stream PipelineTelemetry Path
IoT Hub AFN
Validation
Enrichment
Transformation
Route
Ingestion
Things
Downstream
Consumers
PowerBI,
Service Bus,
Storage,
etc.
Storage blob
ref
Storage blob
archive
stream
Non-telemetry
Blob StorageService Bus
Queue
Lite IoT Stream PipelineTelemetry Path
Azure IoT Hub Telemetry and Non-Telemetry PathsDevice-To-Cloud, Cloud-to-Device, Device-To-Device
/messages/servicebound/feedback
/messages/devicebound
/devices/{deviceId}/messages/devicebound
CustomEndpoi
nt
Custom Service Bus
Queue
Stream
Analytics
IoT Hub
Azure
Function
Telemetry Path
Non-Telemetry Path
Devic
e(T
hin
g)
Rule ABC
No-RulesNo-Match
DefaultEndpoin
t
MQ
TT
, A
MQ
P,
HT
TP
/devices/{deviceId}/messages/events/{propB
ag}
Example of the Device-To-Device Communications
/messages/servicebound/feedback
/messages/devicebound
Device1
CustomEndpoi
nt
Custom Service Bus
Queue
Stream
Analytics
IoT Hub
Azure
Function
Telemetry Path
Non-Telemetry Path
Rule ABC
No-RulesNo-Match
DefaultEndpoin
t
MQTT, AMQP,
HTTP
Raspberry Pi3 + Windows IoT Core
/devices/Device2/messages/events/{propBag}
/devices/Device1/messages/deviceboundhtt
ps
/devices/Device1/messages/events
/devices/Device2/messages/deviceBound
Device2
1
2
3
4
Azure IoT Hub
Device/GW/Thing(MQTT, AMQP, Https)
DirectMethodsonDeviceMethod InvokeMethod
Telemetry Data
C2D C2DQueueing
Device (Thing) Data
Device Twin (State) Data
Direct Actuary
Stream Data
async
Queue/Topic
sync
Change notificationsvia Routes mechanism
Azure Stream AnalyticsServices
(AMQP, REST)
AzureFunctions
Updating Device twin reported properties
/messages/servicebound/feedb
ack
/devices/{deviceId}/messages/devicebound
CustomEndpoi
nt
Custom Service Bus
Queue
Stream
Analytics
IoT Hub
Azure
Function
Telemetry Path
Non-Telemetry Path
Devic
e(T
hin
g) Rule ABC
No-RulesNo-Match
DefaultEndpoin
t
MQ
T
T
/devices/{deviceId}/messages/events/{propB
ag}
Read propertiesRead/Write properties
Read, receive change notifications
1
2
3
5
4
6Read/Write properties
/messages/devicebound
C2D Messaging
Note: Step 2 can be replaced by new featurein the Routes mechanism
Azure Function - VETER Message Mediator https://xxxxxx.azurewebsites.net/api/microservice?code=xxxxx&name=HelloCloud
Designer Tool
Microservice VETER
Azure Function – Telemetry VETER Message Mediator
WCF Workflow Channel
Blob Storage
Dynamic Activity(xaml definition)
HTTP Triger Bridge to WCF Client
Event Hub Triger Bridge to WCF Client
? Triger Bridge to WCF Client
VETER Definition
ModelingBusiness, Connectivity, Hosting, …
Telemetry Data Stream with VETER Mediator
AppendixA
Thanks to [email protected]
http://www.bsp.sk/seminar