Microsoft

Technology

Centers

Experience the

Microsoft CloudMicrosoft

Technology

CentersExperience the

Microsoft Cloud

Azure ML Data Camp

Ivan KosyakovMTC Architect, Ph.D.

Big Data

(3v)

Strategic

Analytics

Business Unit

Analytics

End User

Analytics

Advanced

Analytics

AnalystIT

Data Scientist

Top Manager

Business User

Microsoft

Technology

Centers

Experience the

Microsoft Cloud

Modern Data Warehouse

Analyst

IT

ActiveDirectory

Multi-FactorAuthentication

Key Vault

Customer

Data Scientist

Security Officer

Top Manager

Office 365

Business User

Developer

Hyperscalability

Most comprehensive

Unparalleled security

Intelligent by design

Transform data into intelligent action in the cloud

Information Management

Preconfigured Solutions

Dashboards and Visualizations

Machine Learning and Analytics

Big Data Store

Personal Digital Assistant – Cortana

Perceptual IntelligenceText AnalyticsComputer Vision APIs

Speech APIsFace APIs

Cortana Analytics Suite

Big Data Flow

Business apps

Custom apps

Sensors and devices

People

Machine Learning

and Analytics

Big Data Stores

Action

People

Automated Systems

Apps

Web

Mobile

Bots

Intelligence

Dashboards &

Visualizations

Cortana

Bot

Framework

Cognitive

Services

Power BI

Information

Management

Event Hubs

Data Catalog

Data Factory

HDInsight (Hadoop and Spark)

Stream Analytics

Intelligence

Data Lake

Analytics

Machine

Learning

SQL Data

Warehouse

Data Lake Store

Data Sources

Apps

Sensors and devices

Data

Big Data as part of Cortana IntelligenceMICROSOFT BIG DATA SOLUTIONS

Example reference architecture

Analyze Data Stored in Azure

• Track realtime data from IOT Suite: collect data from IOT

Suite in permanent store (ADLS)

• Track other Azure data (Azure Website generating web

logs) and store in ADLS

• Run Machine Learning through R Server for HDInsight to

find patterns in data

• Show results in BI tools (Power BI)

Real-Time

Reference architecture for Big Data

Data Generation Visualizations

SSRS

SharePoint BI

Excel BI

Power BI

Azure

Marketplace

ProcessingStorage

R ServerHive

Oozie

Sqoop Pig

SQL Server

Analysis Services

Azure HDInsight

Azure Machine

Learning

Azure

Document DB

Azure

SQL DB

HBase on Azure

HDInsight

Azure

Website

Azure Data Factory

Spark SQL

Spark MLib

Datazen

Azure

SQL DW

Azure Data

Lake Analytics

Azure Data Lake Store

Storm

for Azure

HDInsight

Azure

Stream

Analytics

Spark

Streaming

for Azure

HDInsight

Azure

Event

Hubs

Sensors

Bots

Crawlers

Devices

Microsoft

Technology

Centers

Experience the

Microsoft Cloud

Big Data Decision Tree

Source: https://biz-excellence.com/2016/08/30/big-data-dt

Big Data is often described as a solution to the “three V’s problem”, and how we choose right solution depends on which one of these problems we are trying to solve first:

Volume: need to store and query hundreds of terabytes of data or more, and the total volume is growing. Processing systems must be scalable to handle increasing volumes of data, typically by scaling out across multiple machines.

Velocity: need to collect data at an increasing rate from many new types of devices, from a fast-growing number of users, and from an increasing number of devices and applications per user. Processing systems must be able to return results within an acceptable timeframe, often almost in real-time.

Variety: situation when data do not match any existing data schema – semi-structured or unstructured data.

APIML STUDIO

10

SQL ServerR Services

Red Hat SUSE

Hadoop Teradata

Windows

CommercialCommunity

R ServerR Open

Cognitive

Services

Give your solutions

a human side

Microsoft

Technology

Centers

Experience the

Microsoft Cloud

Machine Learning Decision Tree

Source: https://biz-excellence.com/2016/09/13/machine-learning-dt