real-world data governance: governing data – big and small, come one come all

1

Copyright © 2013 Robert S. Seiner – KIK Consulting & Educational Services / TDAN.com

Non-Invasive Data Governance™ is a trademark of Robert S. Seiner & KIK Consulting Twitter About This Webinar at #RWDG

Robert S. SeinerKIK Consulting & Educational Services – KIKconsulting.com

The Data Administration Newsletter – TDAN.com

Real-World Data GovernanceGoverning Data – Big and Small, Come One Come All

Monthly Webinar Series Hosted by DATAVERSITYRobert S. Seiner – KIK Consulting / TDAN.com

November 21, 2013 – 2:00 p.m. EST

2



• Real-World Data Governance – Monthly Webinar Series

– December 19, 2013 – Data Governance Expectations – 2pm EST

– January 16, 2014 – A Data Governance Framework of Success – 2pm EST

– February 20, 2014 – Data Governance Policy: Components & Content – 2pm EST

– March 20, 2014 – Agile Data Governance: The Truth Be Told – 2pm EST

– Register On-Line at DATAVERSITY.net

Governing Data: Big and SmallUpcoming Webinars

3



• With Big Data being such a hot topic, a frequent question for panels atconferences and on webinars has been “is there any such thing as Big DataGovernance?”

• Bob Seiner has been asked that question many times in many venues. Inthis webinar Bob tries to answer that question once and for all.

• Perhaps you will agree with him. As always, there is a strong chance youwon’t. At worst he will make you think!

Governing Data: Big and SmallAbstract

4



• As many organizations consider Big Data solutions they recognize thatthere are aspects of big data that require specific attention.

• Some of these management issues are the same as issues with normal sizeor small data. Some of these management issues differ.

• What are the differences and what impact do they have on the governanceof that data?

• Join Bob Seiner and DATAVERSITY for a Real-World Data Governancewebinar focused on investigating the similarities and differences betweenmanaging and governing big data versus small data.

Governing Data: Big and SmallAbstract

5



• You may think I use the term “small data” with tongue firmly in cheek …

I am not and “small data” will be addressed in this webinar too.

• If you have not heard of “small data”

or you think “small data” is a made up term

or you think I made it up

or you don’t consider it different from any other data

or you think it needs to be governed like any other data …

then I suggest that you write down the fact that you first started

considering the possibilities of “small data” being something real

here on 11/21/13 in this RWDG webinar. - this was tongue in cheek.

Governing Data: Big and SmallIntroduction

6



• Klint Finley (ReadWrite.com – 5/10/11) states, “In short, big data simplymeans data sets that are large enough to be difficult to work with. Exactlyhow big is big is a matter of debate.”


7



• Webinar Introduction

– What is Big Data?

– Considerations for Governing Big Data

– Big Data Governance

– Governing Big Data

– small data


8



• What kinds of Governance have you heard of?

– Data / Information Governance


– BI Governance

– Metadata Governance

– Project Governance

– Corporate Governance

– Process Governance

– Customer Data Governance


What Does it Mean to “Govern”

Something?What

Needs to be Governed?

What Does it Cost to Govern?

What Do IGet Out of

Governed Data?

9



• Data Governance

– Data Governance is the Execution and Enforcement of AuthorityOver the Management of Data & Data-Related Resources.

Robert S. Seiner

• Data Stewardship

– Data Stewardship is the Formalization of AccountabilityOver the Management of Data & Data-Related Resources.

Robert S. Seiner

Governing Data: Big and SmallDefinitions

10




• Non-Invasive Data Governance™

– The practice of:

• applying formal accountability & behavior

• through non-invasive roles & responsibilities

• to existing and / or new processes

• to assure that the definition, production & usage of data

• assures regulatory compliance, security, privacy, protection & quality.

11



• Non-Invasive Data Governance™

– Non-Invasive describes how governance is applied to assurenon-threatening management of valuable data assets.

– The goal is to be transparent, supportive, collaborative.

– Whether that data is BIG, MEDIUM or SMALL.

– Whether that data is DESKTOP, SERVER, CLOUD.


12



• Categories of Big Data:

– Unstructured data – Text, videos, audio and images.

– Semi-structured data – Emails, earnings reports, spreadsheets, softwaremodules.

– Structured data – Sensor data, machine data and mathematical model outputs.

Krish Krishnan – Data Governance for Big Data – Information-Management.com

Governing Data: Big and SmallWhat is Big Data?

13



• Common Characteristics of Big Data:

– Volume – Any of these data categories are, by default, large in volume andvariable in size.

– Variety – The data can be available in a variety of formats, languages andsources.

– Ambiguity – The data can contain metadata about itself or have no metadata.

– Quality – The quality of data in the unstructured and semi-structured categoriesis unreliable.

Krish Krishnan – Data Governance for Big Data – Information-Management.com


14



• Big Data – Defined Again!

– “Big Data” is a term applied to data sets whose size is beyond the abilityof commonly used software tools to capture, manage, and process thedata within a tolerable elapsed time. Big data sizes are a constantlymoving target currently ranging from a few dozen terabytes to manypetabytes of data in a single data set.


From Wikipedia.com

www.allanalytics.comBig Data Quick Poll Results

15



• Big Data – The 3 V-s

– In a 2001 research report and related conference presentations, thenMETA Group analyst, Doug Laney, defined data growth challenges (andopportunities) as being three-dimensional, i.e. increases in:

1. Volume (amount of data),

2. Velocity (speed of data in/out),

3. Variety (range of data types, sources)


From Wikipedia.com – Gartner Group

16



• Big Data – Statistics – More of What is “Big”?

– “Big Data” has emerged because we are living in a society which makesincreasing use of data intensive technologies. There are 4.6 billion mobile-phone subscriptions worldwide and there are between 1 billion and 2billion people accessing the internet.

– Between 1990 and 2005, more than 1 billion people worldwide enteredthe middle class which means more and more people who gain money willbecome more literate which in turn leads to information growth.

– The world's effective capacity to exchange information throughtelecommunication networks was 281 petabytes in 1986, 471 petabytes in1993, 2.2 exabytes in 2000, 65 exabytes in 2007 and it is predicted that theamount of traffic flowing over the internet will reach 667 exabytesannually by 2013.


From Wikipedia.com

http://en.wikipedia.org/wiki/Petabyte

http://en.wikipedia.org/wiki/1,000,000,000

http://en.wikipedia.org/wiki/Gigabytes

17



• Big Data on the Net

– “Big Data” is a catch phrase that has been bubbling up from the highperformance computing niche of the IT market. Increasingly suppliers ofprocessing virtualization and storage virtualization software have begun toflog “Big Data” in their presentations.

Dan Kusnetzky – ZDNet.com (2010)

– “Big Data” is hard to do, and can be very expensive and time consuming.Integrating revenue and cost data in order to manage end-to-end businessmodels is a complicated and time consuming task. Some organizationsoutsource … how to tag and manage the process, in addition to storing thevast amounts of data required for meaningful analysis.

Peter Yared – CBS Interactive – CNET.com (2012)


18



• Big Data on the Net – A Classic Rant

– Let’s start with the basics: What is Big Data? I’m here to tell you thatnobody really knows. The good thing about Big Data is just that. So it canbe anything you want it to be.

– One of the things I noticed about Big Data is that it is always capitalizedwhen it’s written … We don’t capitalize DATA, so why should BigData be? I’m pretty sure that capitalization is a way to spot the birth of asilver bullet. Remember that when the next big thing, HUGE DATA, thenGINORMOUS DATA is announced at next year’s EDW. I guess then big datawill lose its title caps.

Karen Lopez – @datachick – InfoAdvisors

Size Doesn’t Matter. Or Does It? A Rant on Big Data Terms

DATAVERSITY.Net – May 16, 2012


19



Governing Data: Big and SmallConsiderations for Governing Big Data

• [to] gov·ern [data]

v. gov·erned, gov·ern·ing, gov·erns

– To make and administer the public policy and affairs [of data]

– To exercise sovereign authority [in data]

– To control the speed or magnitude [of data]

– To regulate [data]

– To control the actions or behavior [of data]

– To keep under control [data]; to restrain [data]

– To exercise a deciding or determining influence [on data]

– To exercise political authority [over data]

– To have or exercise a determining influence [over data]

FreeDictionary.com [Bob Seiner]

20




• Governance Means …

– Formalizing Accountability

– Bill of “Rights”

– Assuring Compliance

– Following Rules

– Understanding the Data

– Reducing Risk

• Governance Does Not Mean …

– Command and Control

– Slowing Down

– Adding Bureaucracy

– Spending More Money

– Getting in the Way

21



• Considerations for Governing Any Data and Information:

– Data Governance Basics

• Definition of Data

• Production of Data

• Usage of Data

• Formalized Accountability

• Governed Processes

– Risk Management

• Compliance & Regulatory Control

• Data Classification

• Security


22




– Disciplines

• Issue Resolution

• Data Certification

• Quality Control & Improvement

• Proactive & Reactive Governing of Data

• Definition, Production & Usage of Data

– Roles & Responsibilities

• Executive & Strategic

• Tactical

• Operational

• Support (DGT, PMO, R&C & IT)


23




– Communications Planning

• Audience

• Messages

• Delivery Tools

– Information Governance

• Process Governance

• Technology Governance

• Stewardship

• Execution & Enforcement

• Monitoring & Reporting


24



• Similarities Between Governing Big Data Versus Other Data

– Governing the Definition of Big Data

– Governing the Production of Big Data

– Governing the Usage of Big Data

– Governing the Volume, Velocity

and Variety of Big Data


25




– Governing the Definition of Big Data

• Data Redundancy – has data been defined, by who, what purpose, where, how

• Business Definition – standards, business involvement, validation

• Metadata Management – completion, storage, delivery

• Data Modeling – conceptual, logical, physical


26




– Governing the Production of Big Data

• Origination – Where it comes from

• Creation – How it is produced

• Accountability – Who has responsibility

• Delivery – How it is distributed – reports, analytics, …


27




– Governing the Usage of Big Data

• Regulatory & Compliance – what the “rules” say

• Privacy – what data they can see

• Security – who can see the data

• Distribution, Reporting, Analytics, …


28




– Governing the Volume, Velocity and Variety of Big Data

• Volume – how much data we have

• Velocity – when it is produced, how fast, delivery pipeline, …

• Variety – structured, unstructured


29



• Differences Between Governing Big Data Versus Other Data

– Use of specific technologies

• To manage

• To store

• To make available

• Webinar Poll?

– What do you see as being some of

the difference between governing

big data versus other data?


30



Governing Data: Big and SmallGoverning Big Data

• Governance of Big Data includes:

• Volume

• Velocity

• Variety

• Tool Usage

• NOSQL

• Simplicity of Use

• Complexity

• Scalability

• Structured Data

• Unstructured Data

• Social Media Data

• Records

• Meta-Data

• Compliance

• Classification

• Risk Management

• Quality Control & Improvement

31



Governing Data: Big and SmallGoverning Big Data

• Stewardship and Big Data

Data stewardship came up several times in a TDWI report, as those involved withmanaging big data are toying with the idea of adding big data initiatives to their existingdata governance ecosystem. When asked what problems are hindering big datamanagement, a lack of stewardship or governance was cited by 33 percent ofparticipants – second only to inadequate staffing or skills.

Data stewardship in a big data world – Daniel Teachey

The Data Roundtable – October 28, 2013

32



• Is there such a thing as “small data”?

• Where Karen Lopez mentioned that “BIG DATA” seems to be capitalized …

Governing Data: Big and Smallsmall data

33



• It seems inevitable, but some IT experts are now claiming that "small data"analysis is the next big buzzword because it can more quickly produce usefulresults for business managers.

– While big data is getting all the headlines, small data is the next big thing. Bigdata and the value associated with combining and culling vast structured andunstructured data sets for business insights is a worthwhile but, as the nameimplies, big undertaking.

– It may be too big for all but the largest enterprises that have the time, money and expertise to build a big data platform

Eric Lundquist – Posted 2013-09-10 on eWeek.com


34



• The possibility of individuals owning their small data would mark a startlingchange from the world of big data profiles and marketing.

• I'm not sure the business world is ready, but I'm guessing we'd rather ownour information and decide which bits to share than have big data systemsthat can scoop up those bits without our knowledge.

Eric Lundquist – Posted 2013-09-10 on eWeek.com


35



• Bob Seiner’s summary of Data Governance and small data:

– If we define small data as being data that is limited to purpose, high quality,documented, easy to access & use, … to improve organizational analytics …

– There are not to many people out there that would argue that, in order toprovide this small data, this will require governance of the data.

– Therefore if “small data” is something real, there will be small data governance.


36



Governing Data: Big and SmallSummary

• The Burning Questions:

– What impact does Big Data have on Data Governance efforts? Vice-Versa?

– What aspects of Data Governance must be altered to Govern Big Data?

– What Governance roles & processes must be directed to Govern Big Data?

37



Governing Data: Big and SmallSummary

• The Relationship Between Data Governance & Big Data

– Existing Aspects of Data and Information Governance Can Be Directly Applied

– Stay Non-Invasive in Your Approach to Governing Big Data

– Additional Technical Complexities

– Additional Process Complexities

– Additional Data Complexities

38



• Webinar Summary

– What is Big Data?

– Considerations for Governing Big Data


– Governing Big Data

– small data

• Questions & Answers

Real-World Data GovernanceWrap Up

39



Real-World Data GovernanceComing Up

• Real-World Data Governance – Monthly Webinar Series

– December 19, 2013 – Data Governance Expectations – 2pm EST

– January 16, 2014 – A Data Governance Framework of Success – 2pm EST

– February 20, 2014 – Data Governance Policy: Components & Content – 2pm EST

– March 20, 2014 – Agile Data Governance: The Truth Be Told – 2pm EST

– Register On-Line at DATAVERSITY.net

40



• Robert S. Seiner

KIK Consulting & Educational Services – KIKconsulting.com

The Data Administration Newsletter – TDAN.com

Post Office Box 112571, Upper St. Clair, Pennsylvania 15241

412.220.9643, 412.220.9644 (Fax)

[email protected]

[email protected]

Real-World Data GovernanceContact Information