msbi,data warehousing & data integrationtechniques by quontra

8/10/2019 MSBI,Data Warehousing & data IntegrationTechniques by Quontra

1/24

MSBI, Data Warehousing and

Data Integration TechniquesBy

Quontra Solutions

Email : [email protected] : 404-900-9988

WebSite : www.quontrasolutions.com


2/24

Agenda

What is BI?

What is Data Warehousing?

Microsoft platform for BI applications

Data integration methods

T-SQL examples on data integration


3/24

What is BI?

Business Intelligence is a collection of theories,

algorithms, architectures, and technologies that

transforms the raw data into the meaningful data in

order to help users in strategic decision making in

the interest of their business.


4/24

BI Case

For example senior management of an industry

can inspect sales revenue by products and/or

departments, or by associated costs and incomes.

BI technologies provide historical, current and

predictive views of business operations. So,

management can take some strategic or operation

decision easily.


5/24

Typical BI Flow

Data Sources

Users

Data Tools

Data Warehouse

Extraction


6/24

Why BI?

By using BI, management can monitor objectivesfrom high level, understand what is happening,

why is happening and can take necessary steps

why the objectives are not full filled.

Objectives:

1) Business Operations Reporting

2) Forecasting3) Dashboard

4) Multidimensional Analysis

5) Finding correlation among different factors


7/24

What is Data warehousing?

A data warehouse is a subject-oriented,integrated, time-variant and non-volatile

collection of data in support of management's

decision making process.

- Bi l l Inmon

A data warehouse is a copy of transaction data

specifically structured for query and analysis.- Ralph Kim bal l


8/24

Dimensional Data ModelAlthough it is a relational model but data would be

stored differently in dimensional data model whencompared to 3rd normal form.

Dimension: A category of information. Ex. the time

dimension.Attribute:A unique level within a dimension. Ex. Month is

an attribute in the Time Dimension.

Hierarchy: The specification of levels that represents

relationship between different attributes within a

dimension. Ex. one possible hierarchy in the Time

dimension is Year Quarter Month Day.

Fact Table: A fact table is a table that contains the

measures of interest. Ex. Sales Amount is a measure.


9/24

Star Schema

A single object (the fact table) sits inthe middle and is radically connected to othersurrounding objects (dimension lookup tables) like a star.Each dimension is represented as a single table. Theprimary key in each dimension table is related to a

foreign key in the fact table.

Snowflake Schema An extension of the starschema, where each point of the star explodes into morepoints. In a star schema, each dimension is represented

by a single dimensional table, whereas in a snowflakeschema, that dimensional table is normalized intomultiple lookup tables, each representing a level in thedimensional hierarchy.


10/24


11/24

After the team and tools are finalized, the process follows

below steps in waterfall:a)Requirement Gathering

b)Physical Environment Setup

c)Data Modeling

d)ETLe)OLAP Cube Design

f)Front End Development

g)Report Development

h)Performance Tuning and Query Optimizationi)Data Quality Assurance

j)Rolling out to Production

k)Production Maintenance

l)Incremental Enhancements


12/24


13/24


14/24

Power View and Power Pivot

These are self serve BItools provided by Microsoft. Very low on cost ofmaintenance and are tightly coupled with Microsoft Excelreporting which makes it easier to interact.

Performance Point Servers It provides rapid creationof PPS reports which could be in any form and at thesame time forms can be changed just by right click.

Microsoft also provides the Scorecards, dashboards, datamining extensions, SharePoint portals etc. to serve the BIapplications.


15/24

Data Integration

methods


16/24

RDBMS

Copying data from one table to another table(s)

Bulk / Raw Insert operations

Command line utilities for data manipulation

Partitioning data

File System

Copying file(s) from one location to anotherCreating flat files, CSVs, XMLs, Excel spreadsheets

Creating directories / sub-directories


17/24

Web

Calling a web service to fetch / trigger data

Accessing ftp file system

Submitting a feedback over internet

Sending an email / SMS message

Other

Generate Auditing / Logging data

Utilizing / maintaining configuration data (static)


18/24

T-SQLBest practices


19/24

Query to merge data into a table

MERGEdbo.myDestinationTableASdest

USING (

SELECTProductID

,MIN(PurchaseDate)ASMinTrxDate

,MAX(PurchaseDate)ASMaxTrxDate

FROMdbo.mySourceTable

WHEREProductIDISNOTNULL

GROUPBYProductID

)ASsrc

ONdest.ProductID=src.ProductID

WHENMATCHEDTHEN

UPDATESETMaxTrxDate=src.MaxTrxDate

,MinTrxDate=ISNULL(dest.MinTrxDate,src.MinTrxDate)WHENNOTMATCHEDBYSOURCETHENDELETE

WHENNOTMATCHEDBYTARGETTHENINSERT (ProductID,MinTrxDate,MaxTrxDate)

VALUES (src.ProductID,src.MinTrxDate,src.MaxTrxDate);

MERGE clause is T-SQL programmers favorite as it covers 3 operations in one


20/24

Query to get a sequence using CTE

;WITHmyTable(id)AS

(

SELECT1 id

UNIONALL

SELECTid+1 FROMmyTable

WHEREid


21/24

Move Rows in a single Query

DECLARE@Table1TABLE (idint,namevarchar(50))

INSERT@Table1VALUES (1,'Maxwell'),(2,'Miller'),(3,'Dhoni')

DECLARE@Table2TABLE (idint,namevarchar(50))

DELETEFROM@Table1OUTPUTdeleted.*INTO@Table2

SELECT*FROM@Table1

SELECT*FROM@Table2

OUTPUT clause redirects the intermediate results of UPDATE,

DELETE or INSERT into a table specified


22/24

Query to generate random password

SELECTCHAR(32 +(RAND()*94))

+CHAR(32 +(RAND()*94))

+CHAR(32 +(RAND()*94))

+CHAR(32 +(RAND()*94))

+CHAR(32 +(RAND()*94))

+CHAR(32 +(RAND()*94))

Non-deterministic functions like RAND() gives different

result for each evaluation


23/24

Funny T-SQL Try it yourself

Aliases behavior is not consistent

SELECT1id,1.eMail,1.0eMail, 1eMail

Ever seen WHERE clause in SELECT without FROM clause ?

SELECT1 ASidWHERE1 = 1

IN clause expects column name at its left? Well, not Really!

SELECT*FROMmyTableWHERE'searchtext'IN(Col1,Col2,Col3)

Two = operators in single assignment in UPDATE? Possible!

DECLARE@IDINT=0

UPDATEmySequenceTableSET@ID=ID=@ID+1


24/24

msbi,data warehousing & data integrationtechniques by quontra

Documents