data-ed: unlocking business value through data modeling and data architecture (part i of ii)

TITLE

PRODUCED BY DATA BLUEPRINT 10124-C W. BROAD ST, GLEN ALLEN, VA 23060

CLASSIFICATION DATE SLIDE

© Copyright this and previous years by Data Blueprint - all rights reserved!

EDUCATION

Data Modeling & Data Architecting for Business Value pt. 1

When asked why they are modeling data, many in the practice answer: "Because that is what must be done." However, a better approach to this question is to speak in terms that are understood in the executive suite – business results! All of our organizations are faced with various organizational challenges that require analysis. Building new systems is just one example. This webinar describes the use of data modeling as a basic analysis method (one of many that good analysts should keep in their “toolbox"). In addition, I will demonstrate various uses of data modeling to inform, clarify, understand, and resolve aspects of a variety of business problems. As opposed to showing how to data model, I will show you how to use data modeling to solve business problems. The goal is for you to be able to envision a number of uses for data modeling that will raise the perceived utility of this analysis method in the eyes of business executives. Learning objectives include:

• Understanding how to contribute to organizational challenges beyond traditional data modeling• Realizing the fundamental difference between "definition" and "purpose"• Guiding analyses through data analysis• Using data modeling in conjunction with architecture/engineering techniques• Understanding foundational data modeling concepts based on the Data Management Body of Knowledge (DMBOK)• How to utilize data modeling in support of business strategy

1

TITLE




EDUCATION

Welcome!

2

Date: January 8, 2013Time: 2:00 PM ETPresented by: Peter Aiken, PhD

Unlocking Business Value through Data Modeling and Data Architecture Pt. 1

TITLE




EDUCATION

Welcome!

Date: January 8, 2013Time: 2:00 PM ETPresented by: Peter Aiken, PhD

3

Unlocking Business Value through Data Modeling and Data Architecture Pt. 1

TITLE




EDUCATION

Commonly Asked Questions

1) Will I get copies of the slides after the event?

2) Is this being recorded so I can view it afterwards?

4

TITLE




EDUCATION

Get Social With Us!

Live Twitter FeedJoin the conversation!

Follow us: @datablueprint

@paikenAsk questions and submit your comments: #dataed

5

Like Us on Facebookwww.facebook.com/

datablueprint Post questions and

commentsFind industry news, insightful

content and event updates.

Join the GroupData Management &

Business IntelligenceAsk questions, gain insights and collaborate with fellow

data management professionals

TITLE




EDUCATION

Your Presenter: Peter Aiken, PhD

6

• Internationally recognized thought-leader in the data management field - 30 years of experience

– Recipient of multiple international awards

– Founder, Data Blueprint (http://datablueprint.com)

• 7 books and dozens of articles• Experienced w/ 500+ data

management practices in 20 countries

• Multi-year immersions with organizations as diverse as the US DoD, Deutsche Bank, Nokia, Wells Fargo, the Commonwealth of Virginia and Walmart

TITLE




EDUCATION 7DATA BLUEPRINT 10124-C W. BROAD ST, GLEN ALLEN, VA 23060 EDUCATION

Unlocking)Business)Value)through)Data)Modeling)and)Data)Architecture))

pt.)1

Unlocking Business Value through Data Modeling and Data Architecture

TITLE




EDUCATION 8

TITLE




EDUCATION

Data Modeling for Business Value• Goal must be shared IT/business understanding

– No disagreements = insufficient communication• Data sharing/exchange is largely and highly automated and

thus dependent on successful engineering– It is critical to engineer a sound foundation of data modeling basics

(the essence) on which to build advantageous data technologies• Modeling characteristics change over the course of analysis

– Different model instances may be useful to different analytical problems• Incorporate motivation (purpose statements) in all modeling

– Modeling is a problem defining as well as a problem solving activity - both are inherent to architecture

• Use of modeling is much more important than selection of a specific modeling method

• Models are often living documents– The more easily it adapts to change, the resource utilization

• Models must have modern access/interface/search technologies– Models need to be available in an easily searchable manner

• Utility is paramount– Adding color and diagramming objects customizes models and allows for a more engaging

and enjoyable user review process

9

Inspired by: Karen Lopez http://www.information-management.com/newsletters/enterprise_architecture_data_model_ERP_BI-10020246-1.html?pg=2

TITLE Agenda1. What is Data Management/DAMA/DM

BoK/CDMP?2. Why data modeling & what is it?3. The power of the purpose statement4. Understanding how to contribute to

organizational challenges beyond traditional data modeling

5. Guiding problem analyses using data analysis

6. Using data modeling in conjunction with architecture/engineering techniques

7. How to utilize data modeling in support of business strategy

8. Take Aways, References & Q&A PRODUCED BY DATA BLUEPRINT 10124-C W. BROAD ST, GLEN ALLEN, VA 23060



EDUCATION

Tweeting now: #dataed

10

1. What is Data Management/DAMA/DM BoK/CDMP?

2. Why data modeling & what is it?3. The power of the purpose statement4. Understanding how to contribute to





8. Take Aways, References & Q&A

TITLE




EDUCATION

Data Management

11

#dataed

TITLE




EDUCATION

Data Management

12

Manage data coherently.

Share data across boundaries.

Assign responsibilities for data.Engineer data delivery systems.

Maintain data availability.

Data Program Coordination

Organizational Data Integration

Data Stewardship Data Development

Data Support Operations

#dataed

Advanced Data Practices• Cloud• MDM• Mining• Analytics• Warehousing• SOA

- datablueprint.com 1/10/2013 © Copyright this and previous years by Data Blueprint - all rights reserved!

• 5 Data Management Practices Areas / Data Management Basics

• Are necessary but insufficient prerequisites to organizational data leveraging applications that is Self Actualizing Data or Advanced Data Practices

Hierarchy of Data Management Practices (after Maslow)

Basic Data Management Practices– Data Program Management– Organizational Data Integration– Data Stewardship– Data Development– Data Support Operations

http://3.bp.blogspot.com/-ptl-9mAieuQ/T-idBt1YFmI/AAAAAAAABgw/Ib-nVkMmMEQ/s1600/maslows_hierarchy_of_needs.png

TITLE




EDUCATION

Organizational DM Functions and their Inter-relationships

14

#dataed

• Published by DAMA International– The professional association for Data

Managers (40 chapters worldwide)– DMBoK organized around – Primary data management functions

focused around data delivery to the organization (more at dama.org)

– Organized around several environmental elements

• CDMP– Certified Data Management Professional– DAMA International and ICCP– Membership in a distinct group made up of

your fellow professionals– Recognition for your specialized knowledge

in a choice of 17 specialty areas– Series of 3 exams– For more information, please visit:

• http://www.dama.org/i4a/pages/index.cfm?pageid=3399

• http://iccp.org/certification/designations/cdmp

TITLE




EDUCATION

DAMA DM BoK & CDMP

15

#dataed

Data Management Functions

TITLE




EDUCATION

DAMA DM BoK: Data Development

16

from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International#dataed










EDUCATION


17


Why Modeling

18

• Would you build a house without an architecture sketch?

• Model is the sketch of the system to be built in a project.

• Would you like to have an estimate how much your new house is going to cost?

• Your model gives you a very good idea of how demanding the implementation work is going to be!

• If you hired a set of constructors from all over the world to build your house, would you like them to have a common language?

• Model is the common language for the project team.

• Would you like to verify the proposals of the construction team before the work gets started?

• Models can be reviewed before thousands of hours of implementation work will be done.

• If it was a great house, would you like to build something rather similar again, in another place?

• It is possible to implement the system to various platforms using the same model.

• Would you drill into a wall of your house without a map of the plumbing and electric lines?

• Models document the system built in a project. This makes life easier for the support and maintenance!

TITLE




EDUCATION 19

#dataed

Database Architecture Focus

TITLE




EDUCATION 20

#dataed

Data Architecture Focus has potentially greater Business Value

TITLE




EDUCATION 21

#dataed

Data Architecture Focus• Data can be shared • Redundancy can be reduced• Inconsistency can be avoided• Transaction support can be provided• Integrity can be maintained• Security can be enforced• Conflicting requirements can be balanced• Eliminates Data Dependency

– Technique used to physically stored and accessed are dictated by the application, and the knowledge of physical representation and access technique is built into the application code.

– Not desirable in database systems – Different users require different views of the same data– Freedom to change the physical representation or access technique

in view of the changing requirements• Changing record types• Physical storage location

TITLE




EDUCATION

Primary Deliverables become Reference Material

22


TITLE




EDUCATION

Data Modeling Definition

23

• Modeling = Analysis and design method used to– Define and analyze data requirements– Design data structures that support these

requirements• Model = set of data specifications and

related diagrams that reflect requirements and designs– Representation of something in our

environment– Employs standardized text/symbols to

represent data attributes (grouped into data elements) and the relationships among them

– Integrated collection of specifications and related diagrams that represent data requirements and design


• Data modeling is used to articulate data architecture components

• Data architectures are comprised of components – usually expressed as models

• Styles of data modeling exist – this is a challenge– IE or information engineering– IDEF1X used by DoD– ORM or object role modeling– UML or unified modeling language

• Data models are useful– In stand-alone mode– As components of a larger information architecture

TITLE




EDUCATION

Data Modeling and Data Architecture

24

#dataed

TITLE




EDUCATION

Models as an Aid to Understanding

25

#dataed

Models• Are usually for the

purpose of understanding

• Can be – Equations– Simulations including video games– Physical models– Mental models

TITLE




EDUCATION

Polling Question 1

26

#dataed

What is a data model?

a. Framework for understanding and design

b. Easy to validate and review

c. Structure for organizing things d. All of the above

TITLE




EDUCATION

What a model is

27

#dataedSource: Ellen Gottesdiener www.ebgconsulting.com

TITLE




EDUCATION

Use Models to

28

• Store and formalize information

• Filter out extraneous detail• Define an essential set of

information• Help understand complex

system behavior• Gain information from the

process of developing and interacting with the model

• Evaluate various scenarios or other outcomes indicated by the model

• Monitor and predict system responses to changing environmental conditions

- datablueprint.com 1/10/2013 © Copyright this and previous years by Data Blueprint - all rights reserved!29

The Role of Data Models in Rapid Development

http://www.youtube.com/watch?v=Hdpf-MQM9vY&feature=player_embedded#!360 hours or 15 days of continuous building

TITLE




EDUCATION 30

Modeling in Context

Preliminaryactivities Modeling

cyclesWrapupactivities

Evidencecollection &

analysis

Projectcoordinationrequirements

Targetsystemanalysis

Modelingcyclefocus

Activity

Refinement

Collection

Analysis

Validation

Declining coordination requirements

Increasing amounts of target system analysis










EDUCATION


31


Bed

Something you sleep in

Standard definition reporting does not provide conceptual context

TITLE




EDUCATION

The power of the Purpose StatementEntity: BED Data Asset Type: Principal Data EntityPurpose: This is a substructure within the room

substructure of the facility location. It contains information about beds within rooms.

Source: Maintenance Manual for File and Table Data (Software Version 3.0, Release 3.1)

Attributes: Bed.Description Bed.Status Bed.Sex.To.Be.Assigned Bed.Reserve.Reason

Associations: >0-+ RoomStatus: Validated

33

• A purpose statement describing why the organization is maintaining information about this business concept;

• Sources of information about it;• A partial list of the attributes or characteristics of the entity; and• Associations with other data items; this one is read as "One room contains zero

or many beds."

TITLE




EDUCATION

Data Modeling• Modeling = complex process involving

interaction between people and with technology that don’t compromise the integrity or security of the data

• Good data models accurately express and effectively communicate data requirements and quality solution design

• Modeling approach (guided by 2 formulas):– Purpose + audience = deliverables– Deliverables + resources + time = approach

34


TITLE




EDUCATION

Data Models Facilitate

2. Communication• Data model is a bridge to understanding data between people

with different levels and types of experience. • Helps understand business area, existing application, or impact

of modifying an existing structure• May also facilitate training new business and/or technical staff

3. Scope• Data model can help explain the data concept and scope of

purchased application packages

35

1. Formalization• Data model documents a single,

precise definition of data requirements and data-related business rules


TITLE




EDUCATION

Don’t Tell Them your Modeling!

36

#dataed

• Just write some stuff down

• Then arrange it • Then make some

appropriate connections between your objects

TITLE




EDUCATION

Entity Relationship View

37

#dataed

C U S T O M E R

coins

soda

machine

(adapted from [Davis 1990])

TITLE




EDUCATION

Entity Relationship View

38

#dataed (adapted from [Davis 1990])

entity thing about which we maintain information

object entity encapsulated with attributes and functions

C U S T O M E R soda

machine

coinreturn

deposits

selects

given to

dispenses

coins

TITLE




EDUCATION

Describing Data Flows and Processes

39(adapted from [Davis 1990])

money

C U S T O M E R

sufficient funds

change

dispense soda

insufficient Detect cancel

request

changecancel requestFlavor

selection

do not

have

flavor choices

flavors valid Flavor selection

can of

soda

accept flavor

count money

TITLE




EDUCATION

Keep them focused on the purpose

40

• The reason we are locked in this room is to:

– Mission: Review proposal from voice over IP providers• Outcome: Walk out the

door with the top two proposals selected and scheduled personal presentations from each.

– Mission: Discuss logo ideas for the Bore No More movement• Outcome: We will walk

out the door when we identify the top three traits that represent the Bore No More brand.

– Mission: Update all employees on the retirement plan options• Outcomes: Confirm that

all team members took part in the meeting and have access to review their plans privately with a financial consultant.










EDUCATION


41

TITLE




EDUCATION

Metadata Example

42

CRUD matrix that shows business processes and their activity typeCreating, Reading, Updating, and Deleting various data items

Business Process 1

Business Process 2

Business Process 3

Business Process 4

Business Process 5

Data Item A Create Read Delete

Data Item B Read Create Update

Data Item C Read Update

Data Item D Create Update Delete Update

Data Item E Create


As IsRequirementsAssets

As Is Design Assets As Is Implementation Assets

Exi

stin

gN

ew

Reverse Engineering

Forward engineering

To Be Implementation Assets

To BeDesign Assets

To Be Requirements Assets

43

• First, reverse engineering the existing system to understand its strengths/weaknesses• Next, use this information to inform the design of the new system

Reengineering

TITLE




EDUCATION

ANSI-SPARK 3-Layer Schema

44

1. Conceptual - Allows independent customized user views:

– Each should be able to access the same data, but have a different customized view of the data.

2. Logical - This hides the physical storage details from users:

– Users should not have to deal with physical database storage details. They should be allowed to work with the data itself, without concern for how it is physically stored.

3. Physical - The database administrator should be able to change the database storage structures without affecting the users’ views:

– Changes to the structure of an organization's data will be required. The internal structure of the database should be unaffected by changes to the physical aspects of the storage.

For example, a changeover to a new DBMS technology. The database administrator should be able to change the conceptual or global structure of the database without affecting the users.

TITLE




EDUCATION

Modeling Types

45

Logical “as-is” Physical “as-is”

Physical “to-be”Logical “to-be”

Current or Existing System

Proposed or Target System

Logical or Essential System

Physical or Implementatio

n System


Conceptual Logical Physical

Validated

Not Validated

Architecture Evolution Framework

Every change can be mapped to a transformation in this framework

TITLE




EDUCATION

Information Modeling

47

Strategic Level

Models

Tactical Level

Models

Operational Level

ModelsModels from a single data reverse

engineering project


Indexed Sequential File

Index

Program: Where is the record for person "Townsend?"

Index: Start looking here where the "Ts" are stored

Relational Database

Flat File

Program: Must start at the beginning and read each record when looking for

person "Townsend?"

Hierarchical Database

Network Database

48

5 Basic Data Structures

TITLE




EDUCATION

Polling Question 2

49

#dataed

How much non-relational database processing is out there?

a. A lot

b. Just a tiny bit

c. A significant d. None

10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

0.4%0.4%

0.3%0.3%

0%

0.5%

0%

1.0%

1.5%

5.4%

1.3%3.0%

4.9%4.9%1.7%

9.8%8.6%7.6%

15.7%

20.5%

Tota

l % o

f non

-rel

atio

nal p

roce

ssin

g


• 68% using hierarchical (typically IMS or Adabase)

• 20% reporting operational network DBMS• "the rumors of the demise of non-relational processing

are greatly exaggerated" (from Mark Twain)

• Virtually no textbook education50

Percentage of organizations relying on x amount of non-relational database processing

Percentage of non-relational processing (excluding mission-critical)Percentage of mission-critical, non-relational processing

Non-Relational Database Processing


Poor Quality Foundation










EDUCATION


52

TITLE




EDUCATION

Eliminate Entire IT Systems

53

#dataed

TITLE




EDUCATION

Eliminate Entire IT Systems

54

#dataed


Why have data structure problems been so difficult?

55


Student System Data Model


Proposed Data Model57


Running Query

58


Optimized Query

59










EDUCATION


60

TITLE




EDUCATION 61

Master Data Management

Data QualityData Governance

Technique/Technical Interdependencies


Organized Knowledge 'Data'

Improved Quality Data

Data Organization Practices

Metadata Management Practices will be inextricably intertwined with Data Quality and Master Data and Knowledge Management, (among other EIM Functions)

Operational Data

Data Quality Engineering

Master Data Management

Practices

Suspected/ Identified

Data Quality

Problems

Routine Data Scans

Master Data Catalogs

Routine Data Scans

Knowledge Management

Practices

Extraction Sources

Data that might benefit from Master Management

Sources( (Metadata(Governance(

(

Metadata(Engineering(

(

Metadata(Delivery( Uses(

Metadata(Prac8ces((dashed lines not in existence)

Metadata(Storage(

Models are required to synchronize between IT activities

TITLE




EDUCATION

Data Models and Business Rules

63

Person Job Class

Employee Position

BR1) Zero, one, or more EMPLOYEES can be associated

with one PERSON

BR2) Zero, one, or more EMPLOYEES can be associated with one JOB CLASS;

BR3) Zero, one, or more EMPLOYEES can be associated with one POSITION

BR4) One or more POSITIONS can be associated with one JOB CLASS.

Moo

nlig

htin

g

Job Sharing


• Example 1:– Our organization has lots of

employees who work multiple jobs• Example 2:

– Our organization wants to employ many part-time employees

• Requirements– We need to manage these requirements as efficiently as

possible– Each person that we track must have the capability to be

tracked as multiple employees– Each position must be capable of being staffed by

multiple persons -> employees

Person Job Class

Employee Position

Expressing Data Requirements










EDUCATION


65

TITLE




EDUCATION

Polling Question 3

66

#dataed

How do Data models support strategy?

a. Flexible, adaptable data structures

b. Cleaner, less complex code

c. Built in future capabilities d. All of the above

TITLE




EDUCATION

• Consider the opposite question:– Were your systems explicitly designed to

be integrated or otherwise work together?– If not then what is the likelihood that they

will work well together?– In all likelihood your organization is spending between

20-40% of its IT budget compensating for poor data structure integration

– They cannot be helpful as long as their structure is unknown

• Two answers1. Achieving efficiency and effectiveness goals2. Providing organizational dexterity for rapid implementation

How do Data Models Support Organizational Strategy?

67

#dataed

TITLE




EDUCATION

Data Models Used to Support Strategy

68

• Flexible, adaptable data structures• Cleaner, less complex code• Ensure strategy effectiveness measurement• Build in future capabilities• Form/assess merger and acquisitions strategies

Adapted from Introduction to Data Modeling by Clive Finkelstein in Information Engineering Strategic Systems Development 1992

EmployeeType Employee

SalesPerson Manager Manager

Type

StaffManager

LineManager

#dataed

TITLE




EDUCATION

S0: Mission and Purpose

69

#dataed

• Develop, deliver and support products and services which satisfy the needs of customers in markets where we can achieve a return on investment at least 20% annually within two years of market entry

TITLE




EDUCATION

S1: Mission Model Analysis

70

#dataed

TITLE




EDUCATION 71

#dataed

G1. Market AnalysisG2. Market ShareG3. InnovationG4. Customer SatisfactionG5. Product QualityG6. Product DevelopmentG7. Staff ProductivityG8. Asset GrowthG9. Profitability

S2: Identify Potential Goals

TITLE




EDUCATION 72

#dataed

Map Goals to Mission

TITLE




EDUCATION 73

Next Step

Market

MarketCustomer

ProductNeed

Need

CustomerProduct

MarketNeed

ProductCustomer

CustomerNeed

MarketProduct

TITLE




EDUCATION 74

Subsequent Step for Business Value

Market

MarketPerformance

ProductPerformance

Need

CustomerPerformance

NeedPerformance

ProductCustomer

Performance










EDUCATION


75

TITLE




EDUCATION

Virtually any Tool can help!

76

Testing ToolsData Profiling Tools

Data Modeling ToolsOffice Productivity Tools

Model Management ToolsSoftware Development Tools

Database Management SystemsConfiguration Management Tools



Strategic planners :storing the organizational dataarchitecture, enterprise wide models, and the strategicinformation plan, system utilizationinformation, and the strategic information plan metadata

.

Database administration :content management, cluster analyses, database design and implementation, performancenormalization metadata

Repository administration :Establish the corporate repository model,repository customization, content management,and "where used" and "how used" metadata

Project management :Estimating, tracking, and reporting metadata

Quality assurance personnel :Content verification, reconciliation, andstandards compliance metadata

Projects developers :requirement, storing requirements, analysis, prototypes,designs, tests, project management, project deliverables,code creating and impact analysis metadata

End users :policies, practices, procedures, organizations, businessrules, responsibilities, authorities, roles metadata

Methods administration :methodology evolution and customization,facilitation, technique customization, complianceand deliverable production metadata

Data administration :Standards, data assets, context and contentmanagement, data tracaability metadata

77

Data Model Users/Uses

TITLE




EDUCATION

Data Modeling for Business Value• Goal must be shared IT/business understanding

– No disagreements = insufficient communication• Data sharing/exchange is largely and highly automated and

thus dependent on successful engineering– It is critical to engineer a sound foundation of data modeling basics

(the essence) on which to build advantageous data technologies• Modeling characteristics change over the course of analysis

– Different model instances may be useful to different analytical problems• Incorporate motivation (purpose statements) in all modeling

– Modeling is a problem defining as well as a problem solving activity - both are inherent to architecture

• Use of modeling is much more important than selection of a specific modeling method

• Models are often living documents– The more easily it adapts to change, the resource utilization

• Models must have modern access/interface/search technologies– Models need to be available in an easily searchable manner

• Utility is paramount– Adding color and diagramming objects customizes models and allows for a more engaging

and enjoyable user review process

78

Inspired by: Karen Lopez http://www.information-management.com/newsletters/enterprise_architecture_data_model_ERP_BI-10020246-1.html?pg=2

TITLE




EDUCATION

Questions?

79

It’s your turn! Use the chat feature or Twitter (#dataed) to submit

your questions to Peter now.

+ =

TITLE




EDUCATION

Upcoming Events

80

February Webinar:Unlocking Business Value through Data Modeling and Data Architecture (Part II of II) February 12, 2012 @ 2:00 PM ET/11:00 AM PT

March Webinar:The Top Data Job March 12, 2012 @ 2:00 PM ET/11:00 AM PT

Sign up here:• www.datablueprint.com/webinar-schedule • www.Dataversity.net

Brought to you by:

data-ed: unlocking business value through data modeling and data architecture (part i of ii)

Technology