analyze and visualize non-relational data with documentdb + power bi
TRANSCRIPT
![Page 1: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/1.jpg)
Analyze and visualize non-relational data with Azure DocumentDB + HDInsight + Power BIHan WongProgram ManagerAzure DocumentDB
![Page 2: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/2.jpg)
Session Objectives And Takeaways
Internetof Things
ActionableData Insights Scalable Solutions
![Page 3: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/3.jpg)
Internetof ThingsActionableData Insights Scalable Solutions
![Page 4: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/4.jpg)
NoSQL - document databases
![Page 5: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/5.jpg)
Microsoft Confidential
Modern Applications…Are developed iteratively with continuously evolving data models
Produce and consume data at unprecedented rates
Require instantaneous response times to match user expectations
Experience unpredictable and explosive growth
![Page 6: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/6.jpg)
Microsoft Confidential
Developers want…
![Page 7: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/7.jpg)
• Part of NoSQL family• Built for simplicity, scale and performance• Non-relational, no enforced schema• Fully managed
Document Database Services
![Page 8: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/8.jpg)
Not these documents
![Page 9: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/9.jpg)
{“id":“itemdata2344",“data":“TWFuIGlzIGRpc3Rpbmd1aXNoZWQsIG5vdCBvbmx5IGJ5IGhpbmd1aXNoZWQsIG5vdCBvbmcyByZWFzb24sIGJ1dCBieSB0aGlzHNpbmd1bGFyIHBhc3Npb24gZ0aGlzHNpbmd1bGFyIHnJvbSBvdGhlciBhbmltYWxzLCB3aGljaCBpcyBhIGx1c3Qgb2YgnJvbSBvdGhlciBhbmltdGhlIG1pbmQsIHRoYXQgYnkgYSBwZXJzZXZlcmFuY2Ugb2YgZGVsaW0dGhlIG1pbmQsIHRdodCBpbiB0aGUgY29udGludWVkIGFuZCBpbmRlZmF0aWdhYmxlIGdldodCBpbiB0aGUgY2bmVyYXRpb24gb2Yga25vd2xlZGdlLCBleGNlZWRzIHRoZSBzaG9ytmVyYXRpb24gb2Yga2cyByZWFzb24sIGJ1dCBieSB0aGlzHNpbmd1bGFyIHBhc3Npb24gZ0aGlzHNpbmd1bGFyIHnJvbSBvdGhlciBhbmltYWxzLCB3aGljaCBpcyBhIGx1c3Qgb2YgnJvbSBvdGhlciBhbmltdGhlIG1pbmQsIHRoYXQgYnkgYSBwZXJzZXZlcmFuY2Ugb2YgZGVsaW0dGhlIG1pbmQsIHR dodCBpbiB0aGUgY29udGludWVkIGFuZCBpbmRlZmF0aWdhYmxlIGdldodCBpbiB0aGUgY2 bmVyYXRpb24gb2Yga25vd2xlZGdlLCBleGNlZWRzIHRoZSBzaG9ytmVyYXRpb24gb2Yga2cyByZWFzb24sIGJ1dCBieSB0aGlzHNpbmd1bGFyIHBhc3Npb24gZ0aGlzHNpbmd1bGFyIHnJvbSBvdGhlciBhbmltYWxzLCB3aGljaCBpcyBhIGx1c3Qgb2YgnJvbSBvdGhlciBhbmltdGhlIG1pbmQsIHRoYXQgYnkgYSBwZXJzZXZlcmFuY2Ugb2YgZGVsaW0dGhlIG1pbmQsIHR dodCBpbiB0aGUgY29udGludWVkIGFuZCBpbmRlZmF0aWdhYmxlIGdldodCBpbiB0aGUgY2 bmVyYXRpb24gb2Yga25vd2xlZGdlLCBleGNlZWRzIHRoZSBzaG9ytmVyYXRpb24gb2Yga2cyByZWFzb24sIGJ1dCBieSB0aGlzHNpbmd1bGFyIHBhc3Npb24gZ0aGlzHNpbmd1bGFyIHnJvbSBvdGhlciBhbmltYWxzLCB3aGljaCBpcyBhIGx1c3Qgb2YgnJvbSBvdGhlciBhbmltdGhlIG1pbmQsIHRoYXQgYnkgYSBwZXJzZXZlcmFuY2Ugb2YgZGVsaW0dGhlIG1pbmQsIHR dodCBpbiB0aGUgY29udGludWVkIGFuZCBpbmRlZmF0aWdhYmxlIGdldodCBpbiB0aGUgY2 bmVyYXRpb24gb2Yga25vd2xlZGdlLCBleGNlZWRzIHRoZSBzaG9ytmVyYXRpb24gb2Yga2cyByZWFzb24sIGJ1dCBieSB0aGlzHNpbmd1bGFyIHBhc3Npb24gZ0aGlzHNpbmd1bGFyIHnJvbSBvdGhlciBhbmltYWxzLCB3aGljaCBpcyBhIGx1c3Qgb2YgnJvbSBvdGhlciBhbmltdGhlIG1pbmQsIHRoYXQgYnkgYSBwZXJzZXZlcmFuY2Ugb2YgZGVsaW0dGhlIG1pbmQsIHR
Not ideal for these documents
![Page 10: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/10.jpg)
{"name": "SmugMug","permalink": "smugmug","homepage_url": "http://www.smugmug.com","blog_url": "http://blogs.smugmug.com/","category_code": "photo_video","products": [{"name": "SmugMug","permalink": "smugmug"
}],"offices": [{"description": "","address1": "67 E. Evelyn Ave","address2": "","zip_code": "94041","city": "Mountain View","state_code": "CA","country_code": "USA","latitude": 37.390056,"longitude": -122.067692
}]
}
Perfect for these Documentsschema-agnostic JSON store
forhierarchical and de-normalized data at scale
![Page 11: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/11.jpg)
• Rich querying over flexible schemas• Automatic indexing of every document property• Easy-to-use SQL grammar• Integrated with other Azure data services
Azure DocumentDB
![Page 12: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/12.jpg)
Catalog DataPreferences & State Data Gaming Events & Logging
User Data & Content Data Exchange
Common Use Cases
Next GamesHalo5
![Page 13: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/13.jpg)
The basics
![Page 14: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/14.jpg)
The Basics• DocumentDB Resources
![Page 15: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/15.jpg)
The Basics• DocumentDB Resources
![Page 16: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/16.jpg)
The Basics• DocumentDB Resources
!=
![Page 17: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/17.jpg)
The Basics• DocumentDB Resources
![Page 18: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/18.jpg)
The Basics• DocumentDB Resources
![Page 19: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/19.jpg)
Performance levels
![Page 20: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/20.jpg)
Performance Levels
![Page 21: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/21.jpg)
Performance Levels
S1 S2 S3
![Page 22: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/22.jpg)
Performance Levels
S1 S2 S3
![Page 23: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/23.jpg)
Performance Levels
S2 S3S3
http://aka.ms/docdbperf
![Page 24: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/24.jpg)
Scaling through Collections
![Page 25: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/25.jpg)
Scaling through Collections
![Page 26: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/26.jpg)
Demo - Portal & Tools
![Page 27: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/27.jpg)
The Scenario
![Page 28: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/28.jpg)
Running a Database as a Service
CPU Utilization Storage Utilization
![Page 29: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/29.jpg)
Sample JSON data{
"TIMESTAMP": "12/10/2015 00:00:00","TimeStampEpoch": 1449705600,"Server": "ProdApp002","Counter" : "percent processor
time","UnitMeasure" : "percent","CounterValue" : 4.436914
}
CPU Utilization Storage Utilization{
"TIMESTAMP": "12/10/2015 00:10:00","TimeStampEpoch": 1449706200"Server": "ProdApp002","Counter" : "percent free space","UnitMeasure" : "percent","Location" : "d","Type" : "data","CounterValue" : 96.717065
}
![Page 30: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/30.jpg)
Goal – Pinpoint DocumentDB Hotspots
CPU Utilization Storage Utilization
![Page 31: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/31.jpg)
Solution – Power BI Integration { }
SELECT * FROManalytics
34.56%
"TIMESTAMP": "2015-12-10T00:00:00Z""Server": "ProdApp002""Counter" : "percent processor time""UnitMeasure" : "percent""CounterValue" : 4.436914"TIMESTAMP": "2015-12-10T00:10:00Z""Server": "ProdApp002""Counter" : "percent free space""UnitMeasure" : "percent""CounterValue" : 96.717065
12.09%
![Page 32: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/32.jpg)
![Page 33: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/33.jpg)
Demo
![Page 34: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/34.jpg)
Goal – Generate 50th, 90th, and 99th percentiles
CPU Utilization Storage Utilization
![Page 35: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/35.jpg)
Solution – HDInsight Integration { }
SELECT * FROMlogs201512 WHERE logs201512.Counter = “percent processor time”
SELECT * FROMlogs201512 WHERE logs201512.Counter = “percent free space”
percentile(CounterValue, 0.50)percentile(CounterValue, 0.90)percentile(CounterValue, 0.99)
![Page 36: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/36.jpg)
Solution – Full Solution { }
SELECT * FROManalytics
percentile(CounterValue, 0.50)percentile(CounterValue, 0.90)percentile(CounterValue, 0.99)
SELECT * FROMlogs201512_summary
SELECT * FROMlogs201512
34.56%
"TIMESTAMP": "2015-12-10T00:00:00Z""Server": "ProdApp002""Counter" : "percent processor time""UnitMeasure" : "percent""CounterValue" : 4.436914"TIMESTAMP": "2015-12-10T00:10:00Z""Server": "ProdApp002""Counter" : "percent free space""UnitMeasure" : "percent""CounterValue" : 96.717065
12.09% P5028%
P9036%
P9954%
![Page 37: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/37.jpg)
![Page 38: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/38.jpg)
OverallGreat fit for Logging and Event based scenarios
Adhoc, real-time queries Integration with other Azure Data services: HDInsight, Stream Analytics, Data Factory, and Power BI
Great fit for dynamic, evolving applications On the fly schema changes Automatic indexing Horizontal Scaling
![Page 39: Analyze and visualize non-relational data with DocumentDB + Power BI](https://reader031.vdocument.in/reader031/viewer/2022020213/5883dede1a28ab62358b5f21/html5/thumbnails/39.jpg)
Get Started Todayselect * from playground p where p.name = “DocumentDB”
http://aka.ms/docdbplayground http://aka.ms/docdbstarter http://aka.ms/docdbimport