ldm webinar: data modeling & business intelligence

Data Modeling & Business IntelligenceDonna Burbank

Global Data Strategy Ltd.

Lessons in Data Modeling DATAVERSITY Series

February 23rd, 2017

Global Data Strategy, Ltd. 2017

Donna Burbank

Donna is a recognised industry expert in information management with over 20 years of experience in data strategy, information management, data modeling, metadata management, and enterprise architecture. Her background is multi-faceted across consulting, product development, product management, brand strategy, marketing, and business leadership.

She is currently the Managing Director at Global Data Strategy, Ltd., an international information management consulting

company that specialises in the alignment of business drivers with data-centric technology. In past roles, she has served in key brand strategy and product management roles at CA Technologies and Embarcadero Technologies for several of the leading data management products in the market.

As an active contributor to the data management community, she is a long time DAMA International member and is the Past President & Advisor to the DAMA Rocky Mountain chapter. She was also on the review committee for the Object Management Group’s Information Management Metamodel (IMM) and a member of the OMG’s Finalization Taskforce for the Business Process Modeling Notation (BPMN).

She has worked with dozens of Fortune 500 companies worldwide in the

Americas, Europe, Asia, and Africa and speaks regularly at industry conferences. She has co-authored two books: Data Modeling for the Business and Data Modeling Made Simple with ERwin Data Modeler and is a regular contributor to industry publications. She can be reached [email protected] is based in Boulder, Colorado, USA.

2

Follow on Twitter @donnaburbankToday’s hashtag: #LessonsDM

mailto:[email protected]


Lessons in Data Modeling Series

• January 26th How Data Modeling Fits Into an Overall Enterprise Architecture

• February 23rd Data Modeling and Business Intelligence

• March Conceptual Data Modeling – How to Get the Attention of Business Users

• April The Evolving Role of the Data Architect – What does it mean for your Career?

• May Data Modeling & Metadata Management

• June Self-Service Data Analysis, Data Wrangling, Data Munging, and Data Modeling

• July Data Modeling & Metadata for Graph Databases

• August Data Modeling & Data Integration

• September Data Modeling & MDM

• October Agile & Data Modeling – How Can They Work Together?

• December Data Modeling, Data Quality & Data Governance

3

This Year’s Line Up


Agenda

• Where does Data Modeling fit with the rise of Self-Service BI?

• How Data Modeling is the “Intelligence Behind Business Intelligence”

• Creating business meaning & context

• Understand source and target data systems

• Optimize data structures to align queries with reports

• Integration with other tools (BI, ETL, etc.)

• Summary

4

What we’ll cover today


The Importance of Data Modeling in Business Intelligence

5

If You Can Read This, Thank a Data Modeler!


The Rise of the Data-Driven Business

Data, more than ever, is seen as a key business asset and strategic differentiator.

6


The Rise of Self-Service Business Intelligence

• As a result of this growing importance of data, the interest in self-service data reporting has increased among data-savvy business users.

• The availability of tools & data sets has made it easier for business people to do their own data manipulation & reporting

• Self Service BI & Data Manipulation – the tools are slick!

• Accessible Data & Open Data Sets – the amount of data available is amazing!

• Tech-Savvy Business Users – this isn’t any harder than a spreadsheet!

• While this offers great opportunities, it can also be fraught with challenges.

• Data modelers and the models & metadata they create can make the job of business intelligence easier for both BI professionals and the casual BI reporting user.

7


Users are Often Frustrated with Self-Service BI

8

What they want

What they often get


Data is Only as Good as the Metadata

9

Open Data Example: Road Safety - Vehicles by Make and Model


Metadata Matters

With Self-Service BI and Analytics, attention needs to be paid to the quality, context, & structure of data

Raw data used in Self-Service Analytics and BI environments is often so poor that many data scientists and BI professionals

spend an estimated 50 – 90% of their time cleaning and reformatting data to make it fit for purpose.(4

Source: DataCenterJournal.com

Correcting poor data quality is a Data Scientist’s least favorite task, consuming on average 80% of their working day

Source: Forbes 2016

(aka Data Models & Metadata)


Business Intelligence is a Key Driver for Metadata Management

11


Data Modeling for Data Warehousing & Business Intelligence

• What is the definition of customer?• Where is the data stored?• How is it structured?• Who uses or owns the data?

Data Warehouse BI Report:Customers by Region

• What are the definitions of key business terms?• What do I want to report on?• How do I optimize the database for these reports?

Data Modeling helps answer:

For Data Warehousing For BI Reporting

Data Modeling helps answer:

• Data Modeling is the “Intelligence behind Business Intelligence”• Creating business meaning & context

• Understand source and target data systems

• Optimize data structures to align queries with reports

Show me all customers by region

Source Systems

Relational ModelDimensional Model


Finding Balance – Model What Matters

13

• It’s important to find a balance between • Managing & modeling “trusted data sets”• Giving users the flexibility to explore.

• Most users will find these trusted data sets a welcome asset, but don’t want to be restricted from doing data exploration when appropriate.

IoT

Log Files

Data Warehouse

Master Data

Reference Data

Structure Flexibility & Exploration


Data Models Levels – Both Business & Technical

14

Conceptual

Logical

Physical

Purpose

Communication & Definition of Business Terms & Rules

Clarification & Detail of Business Rules &

Data Structures

TechnicalImplementation on a Physical Database

Audience

Business Stakeholders

Data ArchitectureBusiness Analysts

DBAsDevelopers

Business Concepts

Data Entities

Physical Tables

• For Data Modeling for Business Intelligence, it’s important to focus on both the business & technical views.


Business Meaning & Context is Critical

15

Show me all customers by region

Businessperson Data Architect

“Does this include current customers only? Or lapsed customers as well?

“Do we have to obfuscate PII?”


The Importance of Business Definitions

From Data Modeling for the Business by Hoberman, Burbank, Bradley, Technics Publications, 2009


Conceptual Data Model

• Communication & definition of core data concepts & their definitions


Data Model Metadata Can Be Used by Many Roles

18

Business Person(e.g. Finance)

What’s the definition of “Regional Sales”

Auditor

How was “Total Sales” calculated? Show me the

lineage.

Data Architect

What is the approved data structure for storing customer

data?

Data Warehouse

Architect

What are the source-to-target mappings for the DW?

Business Person(e.g. HR)

How can I get new staff up-to-speed on our company’s business terminology?


Data Model Design Layer Relationships

• Data model design layer mappings show the relationship between business terms and their physical implementations on a database platform.

19

Showing Semantic Mapping

Conceptual

Logical

Physical

Business Concepts

Data Entities

Physical Tables

Client

Customer

DB2TeradataOracle

CUST CUSTOMER CTABLE_16

In a Conceptual data model, there may be a concept called “Client” which is the term businesspeople use to describe the people they sell to and work with.

The Logical model might use the term “Customer” for that same concept.

Which may be implemented in a number of physical tables with varying naming conventions.

Conceptual

Logical

Physical

Business Concepts

Data Entities

Physical Tables


Data Modeling Creates an “Active Inventory” of Data Assets

• Know what data you have: Create a visual inventory of database systems

• Know what your data means: Communicate key business requirements between business and IT stakeholders

• Support data consistency: Build consistent database structures & support data governance initiatives

Sybase

MySQLOracle

Data Models

Teradata

Sybase

SQL Server

DB2

Teradata

SQL Server DB2

MySQLSQL Azure

SQL Azure

Oracle


Metadata Adds Context & Definition

• Metadata stored in data models provides valuable business & technical context.

21


Technical & Business Metadata

• Technical Metadata describes the structure, format, and rules for storing data

• Business Metadata describes the business definitions, rules, and context for data.

• Data represents actual instances (e.g. John Smith)

22

CREATE TABLE EMPLOYEE (

employee_id INTEGER NOT NULL,

department_id INTEGER NOT NULL,

employee_fname VARCHAR(50) NULL,

employee_lname VARCHAR(50) NULL,

employee_ssn CHAR(9) NULL);

CREATE TABLE CUSTOMER (

customer_id INTEGER NOT NULL,

customer_name VARCHAR(50) NULL,

customer_address VARCHAR(150) NULL,

customer_city VARCHAR(50) NULL,

customer_state CHAR(2) NULL,

customer_zip CHAR(9) NULL);

Technical Metadata

John Smith

Business Metadata

Data

Term Definition

EmployeeAn employee is an individual who currently works for the organization or who has been recently employed within the past 6 months.

Customer

A customer is a person or organization who has purchased from the organization within the past 2 years and has an active loyalty cardor maintenance contract.


Data Warehousing – An Example

• In the data warehouse example below, metadata for CUSTOMER exists in a number tools & data stores.

23

Sales Report

CUSTOMER

Database Table

CUST

Database Table

CUSTOMER

Database Table

CUSTOMER

Database Table

TBL_C1

Database Table

Business Glossary

ETL Tool ETL Tool

Physical Data Model

Physical Data Model

Logical Data Model

DimensionalData Model

BI Tool


Data Lineage

24

• Data Lineage shows the source to target mapping, or provenance for information.

• Many data modeling tools track this lineage through integration with ETL tools, or with internal mapping functionality.


Why Model the Data Warehouse

• Proper modeling of a data warehouse creates data sets that are:

• Easy to use

• Fast to access

• Combined with other data warehousing best practices around data integration, transformation, & governance, the data warehouse also creates data sets that:

• Contain high quality data

• Provide a broad, integrated set of data across the enterprise

25

Computing report…elapsed time 4 days, 10 hours, 27 seconds…


Modeling for BI Reporting – the Dimensional Model• A common way to model the data warehouse is Dimensional Modeling using a Star Schema, based

on methodology spearheaded by Ralf Kimball.

• For Dimensional Modeling, think of what you’re reporting “by” (e.g. by Month, by Region, etc.)• Dimensional modeling focuses on capturing and aggregating the metrics from daily operations that

enable the business to evaluate how well it is doing.

“What do I want to report by?” (Apologies to grammarians!), e.g.

by monthby regionby quarterby product

The lines on a dimensional data model represent navigation paths, not business rules.


The Star Schema

Dimension

Dimension

Dimension

Dimension

Dimension

Fact

Facts: Contain the actual values to be reported upon. e.g. Sales Figures • Few attributes (with links/keys to the dimensions)• Many values

Dimensions: Contain the details that describe the central fact. e.g. Month, Region, Quarter• Many attributes• Few values


The Star Schema• The following is a sample Star

Schema in a data modeling tool showing:

• Internet Sales (Fact) by:

• Time Period (Dimension)

• Promotion (Dimension)

• Product (Dimension)

• Customer (Dimension)


Summary

• The rise of the “Data Driven Business” has increased demand for BI reporting, particularly Self-Service Reporting

• BI Reporting is only as good as the underlying metadata, data structures, and data quality

• Data Models are a critical tool for• Understanding the business meaning of data

• Making BI Reporting more intuitive

• Improving the performance of BI queries

• Understanding source & target systems and the resultant data lineage

• Find a balance – “Model What Matters”• Modeling & metadata helps define key trusted data sets

• But not all data needs to be modeled – allow for exploration & discovery


Contact Info

• Email: [email protected]

• Twitter: @donnaburbank

@GlobalDataStrat

• Website: www.globaldatastrategy.com

• Company Linkedin: https://www.linkedin.com/company/global-data-strategy-ltd

• Personal Linkedin: https://www.linkedin.com/in/donnaburbank

30

http://www.globaldatastrategy.com/

https://www.linkedin.com/company/global-data-strategy-ltd

https://www.linkedin.com/in/donnaburbank


About Global Data Strategy, Ltd

• Global Data Strategy is an international information management consulting company that specializes in the alignment of business drivers with data-centric technology.

• Our passion is data, and helping organizations enrich their business opportunities through data and information.

• Our core values center around providing solutions that are:• Business-Driven: We put the needs of your business first, before we look at any technology solution.• Clear & Relevant: We provide clear explanations using real-world examples.• Customized & Right-Sized: Our implementations are based on the unique needs of your organization’s

size, corporate culture, and geography. • High Quality & Technically Precise: We pride ourselves in excellence of execution, with years of

technical expertise in the industry.

31

Data-Driven Business Transformation

Business StrategyAligned With

Data Strategy

Visit www.globaldatastrategy.com for more information

http://www.globaldatastrategy.com/


DATAVERSITY Training Center

• Learn the basics of Metadata Management and practical tips on how to apply metadata management in the real world. This online course hosted by DATAVERSITY provides a series of six courses including:• What is Metadata

• The Business Value of Metadata

• Sources of Metadata

• Metamodels and Metadata Standards

• Metadata Architecture, Integration, and Storage

• Metadata Strategy and Implementation

• Purchase all six courses for $399 or individually at $79 each.Register here

• Other courses available on Data Governance & Data Quality

32

Online Training Courses

Metadata Management Course

Visit: http://training.dataversity.net/lms/

http://training.dataversity.net/lms/index.php?r=coursepath/details&id_path=3

http://training.dataversity.net/lms/


Lessons in Data Modeling Series

• January 26th How Data Modeling Fits Into an Overall Enterprise Architecture

• February 23rd Data Modeling and Business Intelligence

• March Conceptual Data Modeling – How to Get the Attention of Business Users

• April The Evolving Role of the Data Architect – What does it mean for your Career?

• May Data Modeling & Metadata Management

• June Self-Service Data Analysis, Data Wrangling, Data Munging, and Data Modeling

• July Data Modeling & Metadata for Graph Databases

• August Data Modeling & Data Integration

• September Data Modeling & MDM

• October Agile & Data Modeling – How Can They Work Together?

• December Data Modeling, Data Quality & Data Governance

33

This Year’s Line Up


Questions?

34

Thoughts? Ideas?

ldm webinar: data modeling & business intelligence

Technology