experfy online course - gain competitive advantage using microsoft azure data platform & cortana...

EXPERFY

Microsoft Azure Data Platform TRACK

Gain Competitive Advantage using Microsoft Azure Data

Platform and Cortana AnalyticsBy Jon Bloom

01

Data Professional, Bloom Consulting

Jonathan Bloom

Jon has been in the Data Science space since 1995. Based in Safety Harbor, Florida, he specializes in the Financial Services, Education, Hi-Tech, and Insurance industries. Microsoft Certified Solutions Expert (MCSE). https://www.experfy.com/blog/author/jonathan-bloom

1983 IBM PC / TRS-801986 High School1988 Fortan1994 Paradox/QuatroPro1996 c++, VB, Oracle, Crystal Reports1999 ASP, Actuate2004 .net2006 Java, Web Services2011 Microsoft BI2011 Microsoft Azure2012 Hadoop2014 Azure ML Machine Learning2015 PowerBI

https://www.experfy.com/blog/author/jonathan-bloom


01


Jonathan Bloom


1991 Retail1994 Banking1998 Medical Reporting Software1999 Utilities Energy Company2000 Credit Card Processing2001 Telecommunications2003 Insurance2005 Consulting2007 County Government2011 County School Board2012 Software Company2013 Consulting



01


Jonathan Bloom


2012 Microsoft Certified Professional (MCP)2012 Microsoft Certified Solutions Associate (MCSA)2014 Microsoft Certified Solutions Expert (MCSE Business Intelligence) 2012 SQL BI User Group Speaker2012-2015 SQL Saturday Speaker

– Enterprise Data Warehousing– PowerBI / Power Pivot / Power Query– Reporting for DBAs– Big Data / Hadoop

2013 – 2015 IT Pro Camp Speaker– SSIS– Business Intelligence– Big Data / Hadoop

2013 Architectural Concepts Podcast



Table of Contents① Rise of Data Culture

② Cortana Analytics Suite

③ Azure SQL Databases

④ Azure HDInsight Hadoop

⑤ Azure Machine Learning

⑥ Azure Data Lake Analytics

⑦ Azure Stream Analytics

⑧ Azure Data Factory

⑨ Azure Data Catalog

⑩ Azure Event Hubs

⑪ Power BI

⑫ Artificial Intelligence

• Data is the new Oilo We've been collecting Data for half a centuryo Data is now an “asset” to each organizationo “Monetize” data

Rise of Data Culture

Rise of Data Culture 6

• Provide Insightso Data-Driven Decisions

» Manage the business» Increase profits» Decrease costs» Streamline processes



• Data Explosiono Exponential growtho Both Structured and Unstructured



• New Data Sourceso Variety of data sources

» Open Data Sources for specific industries» Social Media data» "Home Grown" data



• Size of Datao Traditional databases could not handle huge datao New open source tools like Hadoop solve this problem



• Technology Advanceso Lower costs:

» Hardware» Software» Memory» Processing power

o Software in the hands of Developers



• Data Integrationo Mashing desperate data setso Derive new insights from existing data setso Complete picture of data ecosystem



• Agile Methodologyo Shorter time to insighto Iterative cycleso Release final product every sprinto Better project scopingo Better project control



• Data Scientisto Sexiest position

» Statistics » Math » Programming» Domain Knowledge » Curiosity » Visualizations » Storytelling



• Chief Data Officer o Reports to Senior Executives/CIO o Responsible for all Data o Data Driven Organizationo Document Ecosystem

» Servers » Applications » Databases » Diagrams



o Data Governance o Allocate resources

» Hire Consultants » Matrix Internal Employees

o Consolidate Technologies

• What is Cortana Analytics Suite?o “Cortana Analytics is a fully managed big data and advanced analytics

suite that enables you to transform your data into intelligent action.”

Cortana Analytics Suite

Cortana Analytics Suite 16

• Cortana Analyticso Azure Portalo Gather Datao Predicto Automate

Cortana Analytics Suite

Cortana Analytics Suite 17

• IaaS - Infrastructure as a Serviceo “a form of cloud computing that provides virtualized computing

resources over the Internet”

SQL Server in Azure VM

Azure SQL Databases 18

• PaaS - Platform as a Serviceo “A cloud computing model that delivers applications over the Internet”o SQL Server Database runs in Azure Public Cloud

Azure SQL Server Database


• PaaS - Platform as a Serviceo “a cloud computing model that delivers applications over the Internet”o SQL Server Data Warehouse runs in Azure Public Cloud

Azure SQL Data Warehouse


• Apache Hadoopo Hadoop is a free, Java-based programming framework that supports the

processing of large data sets in a distributed computing environment. It is part of the Apache project sponsored by the Apache Software Foundation.

Azure HDInsight Hadoop

Azure HDInsight Hadoop 21

• Apache Hadoopo Yahoo

• Doug Cutting• A better search engine • HDFS built to support Apache Nutch web search engine project• Crawl hundreds of millions of web pages• HDFS is now an Apache Hadoop sub-project



• Apache MapReduceo A Programming model for large scale data processing

• Written in Java• Send computation to the data• Parallel distributed algorithm within cluster



• Apache Hadoop NextGen MapReduce (YARN)o Splits Job Tracker into 2 pieces

» Resource Manager• Scheduler• Application Manager

» Node Manager• Agent running on each Node machine



• Apache HIVEo A Data Warehousing infrastructure built into Hadoopo HiveQL – simple SQL like language

» Ad Hoc Query Analysis» Aggregations

o No “row-level” updates or “real time” querieso Mount data: specify delimiters, file location, table name



• Apache PIGo Platform to analyze large data setso High Level languageo Parallelizationo Compiler creates series of Map/Reduce jobs



• Apache SQOOPo Apache Sqoop is a tool designed for efficiently transferring bulk data

between Apache Hadoop and structured data stores such as relational databases.

o From EDW to Hadoopo From Hadoop to Relational Databases



• Spark o Fast and general engine for large scale data processingo Open Source Cluster computing Frameworko Originally developed AMPLap, University of California Berkleyo Now part of Apache Software Foundationo Not based on MapReduce paradigm (uses In Memory)

» Up to 100x faster than MapReduce



• What is Azure Machine Learning?o “Machine learning uses computers to run predictive models that learn

from existing data in order to forecast future behaviors, outcomes, and trends.”

Azure Machine Learning

Azure Machine Learning 29

• What is Azure Data Lake Analytics?o “Azure Data Lake includes all the capabilities required to make it easy

for developers, data scientists, and analysts to store data of any size, shape and speed, and do all types of processing and analytics across platforms and languages.”

Azure Data Lake Analytics

Azure Data Lake Analytics 30

• What is Azure Stream Analytics?o Azure Stream Analytics allows you to leverage cloud technologies to find

insights in real time from devices and sensors, typically know as the “Internet of Things”.

Azure Stream Analytics

Azure Stream Analytics 31

• What is Azure Data Factory?o “Data Factory is a cloud-based data integration service that orchestrates

and automates the movement and transformation of data.”o Best of breed analytical pipelines

Azure Data Factory

Azure Data Factory 32

• What is Azure Data Catalog?o Azure Data Catalog is an enterprise-wide metadata catalog which stores,

describes and indexes information on data sources.

Azure Data Catalog

Azure Data Catalog 33

• What is Azure Event Hubs?o “Event Hubs is a highly scalable publish-subscribe event ingestor. It can

collect millions of events per second, so that you can process and analyze the massive amounts of data produced by your connected devices and applications. Once collected into Event Hubs, you can transform and store the data by using any real-time analytics provider or with batching/storage adapters.”

Azure Event Hubs

Azure Event Hubs 34

• Suite of Productso Individual Add-Ins for Excel

» Power Pivot» Power Query Formula Language (“M”)» Power View» Power Map» Natural Query Language

Power BI

Power BI 35

• Definition o “The ability of a computer or other machine to perform actions thought

to require intelligence. Among these actions are logical deduction and inference, creativity, the ability to make decisions based on past experience or insufficient or conflicting information, and the ability to understand spoken language. ” » http://dictionary.reference.com/browse/artificial-intelligence

Artificial Intelligence

Artificial Intelligence 36

• Azure Data Platform and Cortana Analytics o To get more insight and depth into each of the topics listed, please sign

up and purchase this course at:o https://www.experfy.com/training/courses/gain-competitive-advantage-

using-microsoft-azure-data-platform-and-cortana-analyticso Thanks for watching~!

Experfy Course

Artificial Intelligence 37

https://www.experfy.com/training/courses/gain-competitive-advantage-using-microsoft-azure-data-platform-and-cortana-analytics

https://www.experfy.com/training/courses/gain-competitive-advantage-using-microsoft-azure-data-platform-and-cortana-analytics

experfy online course - gain competitive advantage using microsoft azure data platform & cortana...

Data & Analytics