PowerBISELF-SERVICE BUSINESS INTELLIGENCE FOR THE DEVELOPER

Power Query, PowerPivot, Power View, Power Map, Power What…?

Hi, I’m Jeff… Founding partner Vanishing Clouds Microsoft partner for 15 years; small business/development MCTS, MCSE, MCAD, … vNext OC Treasurer

Arch of Microsoft’s BI solutions Cubes (Multi-Dimensional) vs. Tabular PowerBI is “just Excel” PowerBI Desktop Power Query and “M” PowerPivot and DAX

Microsoft’s BI Spectrum

Excel• Excellent visuals• Limited scale, etc.

Power* COM add-ins• Powerful ETL• “Stairway” to DAX• Interactive, easy visuals

Standalone• “Just” Power*• Monthly sprints

SSAS• Tabular vs. Multi-Dimensional• Install-time decision • New language, terminology

Hadoop/HDInsight• Big Data (in the cloud)• Open source, MSFT commits• “Divide and concur” using

commodity servers

Problems with Just Excel Joining tables via VLOOKUP Scale Shaping – preprocess with SQL, manual workflow or just

copy/paste Sources – “modern” REST/web Smarts – business logic beyond a pivot table

Tables vs. CubesGL Accounts Region BalanceSales North America 100COGS North America 72SG&A North America 10Op Profit North America 18Sales South America 58COGS South America 48��� ��� ���SG&A Far East 6Net Profit Far East 12

RegionsGL Accounts North America South America Far EastSales 100 58 ���COGS 72 48 SG&A 10 ��� 6Net Profit     12Op Profit 18    (pivot)

2DGL Accounts Region Scenario Balance

SalesNorth America Actual 100

COGSNorth America Actual 72

SG&ANorth America Actual 10

Op ProfitNorth America Actual 18

SalesSouth America Actual 58

COGSSouth America Actual 48

���SG&A Far East Budget 6Net Profit Far East Budget 12

3D

GL Accounts Region District Scenario BalanceSales North America NE Actual 25COGS North America NE Actual 12���         ?

Hierarchy

GL Accounts Region District Scenario Date BalanceSales North America NE Actual Jan 01 25COGS North America NE Actual Jan 01 12���          

Hyper-Cube

Multi-Dimensional Cubes Technology from Panorama; Israeli/Canadian

High performance; e.g., pre-calculated subtotals Sum quarters, then any YTD is adding at most 5 terms

Specialized vocabulary Facts vs. dimensions vs. measures; star vs. snowflake

MDX is widely seen as difficult to learn

Jan Feb Mar Apr May Jun JulQ1 Q2

Given:SUM:

YTD July: Q1 Q2 Jul

Newer Tabular Model Technology from Vertipaq (xVelocity)

Relational (like) – FKs, one-to-many, etc. Familiar “Good enough” performance DAX is “hard enough” to learn

Tabular Model

DirectQuery In-Memory

Third Party Applicatio

n

Excel

Power

View

Reporting Services

ODataFiles Cloud Services

SQL Server

Databases

Non SQL Server

Databases

Power*Power Query PowerPivot Power View/Map

Role Discover Analyze VisualizeLanguage “M” DAX N/A

Technology Oslo DSL In-memory (xVelocity)

Silverlight!

XL10/13 Install

Install (COM add-in) COM Install (COM)

XL16/Future Integrated (replace import?)

Integrated - Tab Integrated – on Insert

PowerBI Subscription Adds:• SharePoint site with engine/preview and some editing (10MB 250MB, refresh)• “Data steward” concerns: shared queries and searchable data catalog; gateway to on-prem• Mobile• Q&A – natural language

Demos Raw Excel Web Scraping Combining source: Excel and OData Slicers; Timelines

Power Query ETL – Data Access Excel – any table (not region)

PQ provides connections, but doesn’t “use” them Relational – added OLE DB/ODBC; can include instance/db/SQL

Fast Load (a.k.a. query folding) and permissions (cred cached in machine local store)

CSV/Text (including JSON)/File System (includes folders) Web – general (tables) and MSFT “indexed” like Wikipedia

Optimized for GET and tables Online Search for MSFT’s and “your” catalog OData, includes SP Azure – credentials, BLOb storage

Other sources – Exchange, AD, Facebook, SAP, …

Power Query ETL – Informally “M” Functional, strongly-typed, domain specific language

More similar to Excel functions that OOP “Control flow” (if…then…else and try…otherwise) are functions Comments // and /*…*/

Structured data types: List – an ordered sequence { … } (special form: {1..10}); also

indexes Record – “one row” of named fields [«name» = «value»]; selects

field Table – most important #table() functionExample:

Source = OData.Feed("http://...svc/ "),Orders_table = Source{[Name="Orders", Signature="table"]}[Data],https://msdn.microsoft.com/en-us/library/mt211003.aspx

Power Query ETL - Connections Don’t load to Excel unless you “have” to Immutable (can’t change after first close) Later

Refreshing Permissions – “cannot” mix Public/Organization/Private Fast Load

Publish to PowerBI portal: https://app.powerbi.com

PowerPivot Model The Excel data model—a “hidden layer” above Excel

From Data or PowerPivot tabs Column-store technology compresses most data well Scales to millions of rows (in XL13+ only limited by RAM)

Hint: Bypass Excel when loading from Power Query Business data types (Address, URL) Direct support for KPIs Excellent time functionality—but BYOC (bring your own calendar)

Data Analysis eXtensions Simpler (tabular) than MDX; “part way” between M and Excel Syntax “reversed” from M: [] around columns Statically typed with liberal coercion: "1"+1 = 2; "1"&1 = "11" Calculated Columns vs. Calculated Fields (nee Measures) It’s all about the evaluation context:

Row context – typically for calculated columns Filter context – typically for calculated fields/measures (think in a

PivotTable) Powerful functions like SUMX and CALCULATE Related() and RelatedTable() go one-to-many but not many-to-

one In-memory vs. Direct Query

We Haven’t Covered DAX and SSAS’s Tabular mode SSAS Multi-Dimensional mode (pros/cons) PowerBI Service

Dashboard (SharePoint portal) Data Steward and Shared/Recommended Queries On-premise data (gateway) and refreshing models Mobile Q&A Natural Language

Reporting (much)