stream analytics service in azure

Azure Stream Analytics

Abhishek SurMicrosoft MVP, ASP.NET / IIS

AgendaStreaming DataUse CasesFully ManagedMission Critical ReliabilityRapid DeploymentDemo

Why Azure Stream Analytics?

A tool for real-time processing of streaming data

Growth of internet connected devices vs. connected PCs, smartphones, tablets

2009 2015 20200

5

10

15

20

25

30Bi

llions

http://www.gartner.com/newsroom/id/2905717 Forecast: The Internet of Things, Worldwide, 2013

Data in motionData at restWhat is streaming?

Why Stream Analytics in the Cloud?

Not all data is localEvent data is already

in the Cloud Event data isglobally distributed

Reduced TCO Elastic scale-out

Service, not

infrastructure

Bring the processing to the data, not the data to the processing!

The need for real-time processing

Smart grid CRM alerting sales Predictive maintenance

Real-time financial sales tracking

Real-time fraud detection

Click-stream analysis

Remote device monitoring

Connected car

Build more intelligent transportation

Reduce maintenance costs by receiving notifications when preventative maintenance is needed before equipment breaks down

Respond to and resolve traffic incidents and maintenance needs faster by integrating sensors, license plate reading systems, cameras, and social network data

Alleviate the hassle of city parking by providing citizens with an easy view of parking availability throughout the city and automatic payment systems

Increase revenue and incentivize off-peak travel by implementing dynamic toll and fare pricing based on license plate readings

Create more value withtoll and fare management

Simplify the parking experience

Increase traffic system efficiency

Optimize fleet and asset management

xx!

How do customers create a real-time streaming solution?

Infrastructure – procure and setup

Develop solution (code) for ingress, processing, and egress

Develop solutions to integrate with other components like ML, BI, etc.

Develop solutions to manage resiliency,

such as infrastructure failures

Develop solutions and infrastructure for increasing scale with business growth

Monitoring and troubleshooting of solution

Customers using Azure Stream AnalyticsFrom event or data streams to real time insights in less time with less people

resources

Development and operations resources Time

End-to-end stream processing on Microsoft AzureIngestor

(broker)Collection Presentation

and actionEvent producers

Transformation

Long-term storage

Event hubs

Storage adapters

Stream processingCloud gateways

(web APIs)

Field gateways

Applications

Legacy IOT (custom protocols)

Devices

IP-capable devices(Windows/Linux)

Low-power devices (RTOS)

Search and query

Data analytics (Excel)

Web/thick client dashboards

Service bus

Azure SQL DB

Azure storage

HDInsight

Stream Analytics

Devices to take action

Power BI

End-to-end stream processing on Microsoft Azure

Event hubs

Stream Analytics

Power BI

Introducing stream analyticsMission critical reliability and scale

Enables rapid development

Fully managed real-time analytics

Intake millions of events per secondProcess data from connected devices/appsHighly-scalable publish-subscriber ingestor

At variable loadsScale that accommodates variable loads Preserves event order on per-device basis

Real-time analytics on continuous streams of data Detect patterns and anomalies in streaming dataTransform, augment, correlate, temporal operations

Correlate with reference data

Real-time analytics

Fully Managed

No hardware (PaaS offering)Bypasses deployment expertiseUp and running in a few clicks (and within minutes)No software provisioning and maintainingNo performance tuning

Less HW and SW maintenanceSpin up any number of resources on demandUse and pay for resources only at times you need them

Expand your business globally

Procure HW Infrastructure and setup

Code for ingress, processing and egress

Plan for resiliency, such as HW failures

Design solution

Build Monitoring and Troubleshooting

Efficiently pay only for usageArchitected for multi-tenancy Small streaming jobs billed at low amountsNot paying for idle resources

Augment costs as neededLow startup costs Ability to incrementally add resourcesReduce costs when business needs changes

Lower costs

Guaranteed events deliveryGuaranteed not to lose events or incorrect outputGuaranteed “once and only once” delivery of eventAbility to replay events

Guaranteed business continuityGuaranteed uptime (three nines of availability)Auto-recovery from failures Built in state management for fast recovery

Mission critical reliability

Elasticity of the cloud for scale up or scale downSpin up any number of resources on demandScale from small to large when requiredDistributed, scale-out architecture

No challenges with scale

Only SQL queries neededDevelopers uses declarative SQL commandsSome functions take several lines of code versus thousands from other solutions

Rapid development

Versus 3 lines of code in Stream Analytics

Thousand lines of code in other solutions, such as Apache Storm

Implement temporal functionsTumbling WindowsHopping WindowsSliding Windows

Manage out-of-order events With configuration instead of code

Manage actions on late eventsUsing policy settings instead of code

Built in temporal semantics

Built-in monitoring View your system’s performance at a glanceHelp you find the cost-optimal way of deployment

Scheduling and monitoring built in

Microsoft real-time stream processing options

Complex event processing in SQL Server

Ease of development and operationalization

Flexibility and customizability

On-premises or Azure IaaS Azure PaaS Azure PaaS

No No Yes

.NET/LINQ SQL SCP.NET, Java, Python

Visual Studio Web browser Visual Studio

Demo

35

Azure Stream Analytics

© 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing marketconditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Abhishek Sur

[email protected]

Questions?

stream analytics service in azure

Software