directquery vs in-memory in bism - ms sql girl · the new and flashy tabular model for analysis...
TRANSCRIPT
1 MsSQLGirl.com | @MsSQLGirl
DirectQuery vs In-Memory in BISM Julie Koesmarno www.MsSQLGirl.com
Sydney SQL User Group @MsSQLGirl
2 MsSQLGirl.com | @MsSQLGirl
About Me
8+ years experience with SQL Server (since
SQL Server 2000)
MCITP 2008
Past projects: large scale 24/7 Data Integration,
Data Management and Data Cleansing system
Current project: Data Warehouse and Analysis
Services Tabular Model project
2
3 MsSQLGirl.com | @MsSQLGirl
Agenda
Terminology
DirectQuery vs In-Memory
Hybrid Mode
Out of Scope:
Sharepoint Integration
3
4 MsSQLGirl.com | @MsSQLGirl
Abstract
The new and flashy Tabular Model for Analysis Services has been highly pitched with the
in-memory (VertiPaq) capability allowing for complex queries to run very fast
What are the advantages and disadvantages of using in-memory cache to store and
query data?
Is DirectQuery the answer to address in-memory cache mode limitations?
What are the restrictions on DirectQuery tabular model?
When can a hybrid mode be used?
How would the selected mode can affect the Processing option during Deployment?
In this session, all the above questions will be answered with live demos, comparing the
two modes and the scenarios that are applicable to them. It also discusses how partition
plays an important role in DirectQuery mode and choosing the correct processing mode
when deploying a Tabular Model solution.
4 MsSQLGirl.com | @mssqlgirl
5 MsSQLGirl.com | @MsSQLGirl
v
Terminology
Tabular Model Architecture and Solution
6 MsSQLGirl.com | @MsSQLGirl 6
DirectQuery
Vertipaq
In-Memory
xVelocity Cached
7 MsSQLGirl.com | @MsSQLGirl
Tabular Model Architecture
7
Tabular Model
7
DirectQuery In-Memory
Third Party Application Excel Power View Reporting Services
SQL Server
Databases Non SQL Server
Databases
OData Files Cloud Services
8 MsSQLGirl.com | @MsSQLGirl
Tabular Model Architecture
8
Tabular Model
8
In-Memory
Third Party Application Excel Power View Reporting Services
SQL Server
Databases Non SQL Server
Databases
OData Files Cloud Services
9 MsSQLGirl.com | @MsSQLGirl
Tabular Model Architecture
9
Tabular Model
9
Third Party Application Power View Reporting Services
SQL Server
Databases
DirectQuery
10 MsSQLGirl.com | @MsSQLGirl
Tabular Model Solutions
10
Authoring
(SSDT) Deployment Option
Connectivity
Database
Properties
demo
11 MsSQLGirl.com | @MsSQLGirl
v
Execution
DirectQuery vs In-Memory
12 MsSQLGirl.com | @MsSQLGirl
Querying
12
Data Access
Query
Data Sources
12
DirectQuery
Process
In-Memory
DAX / MDX
VertiPaq SE
SQL
DAX
13 MsSQLGirl.com | @MsSQLGirl
DirectQuery Security
13 13
Role
DirectQuery Impersonation
Impersonate Current User
Default (Data Source login)
Data Source Impersonation
Windows User
Service Account
DirectQuery
SQL
Query
14 MsSQLGirl.com | @MsSQLGirl
In-Memory Security
14
Role
Row Level Security
Data Source Impersonation
14
VertiPaq SE
In-Memory
Process
Query
15 MsSQLGirl.com | @MsSQLGirl
Other Security Consideration?
Identity flow in Sharepoint 2010
http://www.sqlpass.org/summit/2011/Speakers/CallForSpeakers/SessionDetail.
aspx?sid=2027
15
BISM file
RSDS file
Simple Tabular Model Connection String
16 MsSQLGirl.com | @MsSQLGirl
v
Design
DirectQuery vs In-Memory
17 MsSQLGirl.com | @MsSQLGirl
Formula Compatibility
Semantic differences
xVelocity in-memory analytics engine (VertiPaq)
SQL Server
No Calculated Columns in DirectQuery
17
18 MsSQLGirl.com | @MsSQLGirl
DAX Functions
Some DAX Functions are not supported in
DirectQuery:
No equivalent calculations in relational engine
No equivalent SQL expressions
Performance of converted expressions
E.g. TOTALYTD, SAMEPERIODLASTYEAR
More information on BOL:
http://msdn.microsoft.com/en-us/library/hh213006.aspx
18
19 MsSQLGirl.com | @MsSQLGirl
v
Administration
DirectQuery vs In-Memory
20 MsSQLGirl.com | @MsSQLGirl
Processing Options (In-Memory)
Mode Database Table Partition
Process Default × × ×
Process Full × × ×
Process Data × ×
Process Clear × × ×
Process Defrag ×
Process Add ×
Process Recalc ×
20
Further reading:
http://msdn.microsoft.com/en-us/library/hh758414.aspx
21 MsSQLGirl.com | @MsSQLGirl
Partitioning in DirectQuery
One partition in Tabular Model
Can take advantage of Partitioning and ColumnStore
Index technology on SQL Server 2012
Partitioning / ColumnStore management done at the
source database level
21
22 MsSQLGirl.com | @MsSQLGirl
Partitioning in In-Memory
In-Memory can have multiple mutually exclusive
partitions
To eliminate unnecessary processing and
processor load on the AS servers
22
C u r r e n t
Mar 2012
Jan 2012
Feb 2012
…
In-Memory
Partitions Process
23 MsSQLGirl.com | @MsSQLGirl
v
Pros and Cons
DirectQuery vs In-Memory
24 MsSQLGirl.com | @MsSQLGirl
In-Memory - Pros
ALL functionality of Tabular Model
Calculated Columns, ALL DAX functions
Row Level security
xVelocity in-memory analytics engine
Many choices of client tool
Excel, Power View, SSRS, MDX client*
Many supported data sources
24
25 MsSQLGirl.com | @MsSQLGirl
In-Memory - Cons
AS requires Memory & CPU resources
Caching
Processing
Querying
Diligent Partitioning and Processing
Out of date data
25
26 MsSQLGirl.com | @MsSQLGirl
DirectQuery - Pros
Real time access
Scalable
SQL Server permission
SQL Server optimisation
Consistency
One Partition to manage
No processing required
26
27 MsSQLGirl.com | @MsSQLGirl
DirectQuery - Cons
Restricted DAX functions
Restricted client tools: PowerView, SSRS
No Row Level Security
Unless implemented at the Source Database level
No Calculated Columns
27
28 MsSQLGirl.com | @MsSQLGirl
v
Hybrid Mode
The best of both worlds
29 MsSQLGirl.com | @MsSQLGirl
Tabular Model Solutions
29
DirectQuery Mode
On
Off
Connection String:
DirectQueryMode=DirectQuery
DirectQueryMode=In-Memory
Query Mode
DirectQuery
DirectQuery with In-Memory
In-Memory with DirectQuery
In-Memory
30 MsSQLGirl.com | @MsSQLGirl
Which Hybrid Mode?
Query Mode Excel Connection String
DirectQuery with In-Memory DirectQueryMode = InMemory
In-Memory with DirectQuery No need to specify DirectQueryMode
30
PowerView always uses the default Query Mode
(i.e. the first bit of the Query Mode) – can’t
change this!
SSRS works with both Query Mode and
Connection String combinations
31 MsSQLGirl.com | @MsSQLGirl
Partitioning for Hybrid Mode
One Partition for DirectQuery
Processing Option: Never process this partition
Multiple Partitions for In-Memory (No Overlap)
31
…
C u r r e n t
Aug 2012
Jun 2012
Jul 2012 In-Memory
Partitions
DirectQuery
Partitions
32 MsSQLGirl.com | @MsSQLGirl
Implications
Inconsistency
Data – stale data in Cache
Calculations – semantic differences xVelocity vs SQL
Security
DirectQuery Design
Flexibility at run time / client tools
In-Memory (Excel) or DirectQuery (Real time)
http://www.mssqlgirl.com/hybrid-mode-in-tabular-bi-semantic-model-part-1.html
http://www.mssqlgirl.com/hybrid-mode-in-tabular-bi-semantic-model-part-2.html
32
33 MsSQLGirl.com | @MsSQLGirl
v
Wrap Up
34 MsSQLGirl.com | @MsSQLGirl
Tabular Model Modes
In-Memory Mode (default)
DirectQuery Mode
DirectQuery only
DirectQuery with In-Memory
In-Memory with DirectQuery
34
35 MsSQLGirl.com | @MsSQLGirl
Do I DirectQuery?
SQL Server database is optimised
Columnstore Index
Partitioning
Contains all required business rules
High Volume
Can import non-SQL data into one SQL Server
database which will be the source
35
36 MsSQLGirl.com | @MsSQLGirl
Further Reading
Cathy Dumas’ blog
http://cathydumas.com/
SQLBI – (Marco Russo & Alberto Ferrari)
http://www.sqlbi.com/
DAX resources (with links to Tabular Model)
http://social.technet.microsoft.com/wiki/contents/articles/1088.dax-resource-center.aspx
Julie Koesmarno’s blog
http://www.MsSQLGirl.com
36
37 MsSQLGirl.com | @MsSQLGirl
Q & A
37
http://www.mssqlgirl.com
http://au.linkedin.com/in/juliekoesmarno
@mssqlgirl
38 MsSQLGirl.com | @MsSQLGirl
v
Thank you!
39 MsSQLGirl.com | @MsSQLGirl 39 MsSQLGirl.com |
@mssqlgirl