data governance: the kansas approach (ppt)

Data Governance:The Kansas Approach

Education Information Management Advisory Consortium (EIMAC) Spring Meeting

May 2007Presented by: Kathy Gosa

Kansas State Department of Education

Kansas: The way we were…

Independent “silo’s” each collecting and reporting data independently

Quality of data collected is unknown and “questionable”

Minimal link or consistency in reports No agreement on “authoritative source” No agreement on definitions or policies Inconsistent technologies Work often redundant Security needs not necessarily understood or

followed

This led to….

Challenges in meeting the hundreds of Data Requests we receive

Challenges in explaining inconsistencies Difficulty in submitting to PBDMI/EDEN (no

data submitted in 04-05 SY) Confusion from schools regarding policies /

definitions / etc. Resource constraints – essential enterprise

information in the head(s) of a few individuals

Add to this….

More data! KSDE implemented Kansas Individual Data on

Students (KIDS), assigning state IDs to all Kansas students in spring 2005 and collecting student level data as basis for funding, enrollment, federal and state reporting, assessments, and accountability in 2005-2006 school year.

Enterprise Data System (including metadata) KSDE received funding from state legislature in

2006 for 3 year project to implement an Enterprise Data System.

The dilemma…

How can we quickly get on a path of organization and productivity?

One part of the answer: Institute Data Governance

“When an organization views data as an enterprise asset (transcending the data warehouse and spanning the whole organization), it establishes a … data

governance committee that oversees and guides data stewardship across the organization (and may include)

Data qualityData architectureData integration

Data warehousingMetadata management

Master data management” --Philip Russom, TDWI

Why Data Governance?

Stepped up regulatory demands Sarbanes-Oxley Act, 2002 Data Quality Act, 2002 EdFacts / EDEN (PBDMI)

Data are becoming critical for decision making. The stakes are getting higher and

questionable data quality is unacceptable. The world has changed – no one believes that

IT is a superhero!

Setting the Stage

Learn what we can from business and industry. Professional Training Webinars (on-going)

Gain executive buy-in. Focus on ROI and advantages. Communicate in their terms. Propose solutions, not problems. Demonstrate successes!

Make Data Governance part of our culture. Takes time and patience. One department at a time!

Data Governance is a process, not an event!

Steps to establishing the Kansas Data Governance Program

1. Determine our approach2. Establish a structure3. Explicitly define Roles & Responsibilities 4. Identify individuals for these roles5. Provide on-going training and capacity

building6. Identify an issue escalation / resolution

process7. Expand, reuse, and improve each year

Kansas Approach to Data Governance

Customized Learn from industry, but customize for our specific situation and

needs Buy-in (vs. Mandated)

Mandated is faster and easier to implement, but may be harder to sustain. Also requires the authority to mandate!

Buy-in may take more time to implement, but will be more sustainable since will become part of the culture.

Project Management Use Project Management techniques to establish the initial

processes and track the progress. Evaluate usefulness of Project Management for following years.

Define Success (focus on a specific problem) EDEN vs Enterprise Data System vs Data Requests vs

Communication vs Master Data Management vs ….

Information Security

Master Data Management

Policy Management

Enterprise Data System - Metadata

EDEN – Federal

Reporting

Data Requests

Data Quality

DATA GOVERNANCE

EDEN Challenges

Playing catch up Setting up infrastructure and processes (work flow & data

flow) to “get the ball rolling” 03-04 SY and 04-05 SY minimum data sets Along with submitting 05-06 data

Staff’s conflicting priorities Determining true source of data and subsequently the

data owner & steward Working with EDEN folks to work out the “kinks” Previously KSDE approached federal reporting

(PBDMI, etc) as an event Now we approach federal reporting as a process

Project Management(EDEN Coordinator)

Setting up an infrastructure for work flow and data flow that is supported, documented, and repeatable. EDEN Repository EDEN Metadata Repository EDEN Coordinator attended two day training on How to Build &

Implement a Data Governance & Data Steward Program Monitor project status and escalate as needed Create & maintain project documentation

EDEN submission plan tool Project plan with work breakdown structure Communication matrix document Roles & Responsibilities document

Lots of communication!! EDEN status meeting with core team Data Governance Board meeting Data Steward meeting Ad-hoc meetings

EDEN Metadata

Created a metadata management tool Focused on EDEN data elements only EDEN file level info (directions related to the entire file) EDEN data element level info

EDEN data element name EDEN permitted values EDEN submission used in KSDE data owner & steward EDEN definition KSDE definition & business rules KSDE source path (server, database, table, field) Transformations (crosswalk from KSDE value to EDEN value)

Training for staff Metadata Repository User’s Guide

EDEN Data Flow Processes

EDEN Work Flow Process EDEN Coordinator –

Download file specs from EDEN website Add elements to metadata repository (with Director assistance) - Identify data steward Get copy of specs to steward

Data Steward – Document metadata (ie: steward / owner, transformation values, source info,

business rules / definitions) Indicates when data is ready for reporting

ETL Programmer - Extracts data from source using metadata info and puts into EDEN repository in the EDEN format (transformations & aggregations)

EDEN Coordinator - Checks data for valid values Data Steward - Checks data for content accuracy and gives approval for

submission to EDEN XML Programmer –

Pulls data from EDEN repository and creates XML file Submits XML file to EDEN

If any errors occur they are dealt with accordingly and file resubmitted

Data Steward Program& Workgroup

Workgroup reports to Data Governance Board Created a Data Steward Program Manual Main objectives

Communication & collaboration Data quality Build capacity for ownership and accountability of data Eliminate the silo effect of working with data

Ongoing agenda items for workgroup: EDEN update Data Quality Build capacity for other Data Governance initiatives:

Student level data system Horizontal data system integration update Enterprise data system update

Data Steward Responsibilities

Identify & manage metadata Identify & resolve data quality issues (integrity,

timeliness, accuracy, completeness) Communicate data quality issues and problems to

individuals that can influence change, as needed Communicate new & changed data requirements to

necessary individuals Determine business and security needs of data Define requirements for archiving data Provide input to data analysis Ensure consistency between EDEN reporting and

other federal reporting

EDEN Accomplishments (Year 1)

Leadership support obtained Designated full time EDEN Coordinator Established technical & non-technical infrastructure

EDEN repository EDEN metadata repository Data Governance Board & Data Steward Workgroup Data flow & work flow processes initiated

Made decision to submit all data via XML Submitted minimum data sets for 03-04 & 04-05 Submitted 05-06 data Now working on 06-07 data

Plans for EDEN – Year 2

Kick-off meeting with each department Director and Data Stewards Lessons Learned Schedule and expectations Expected challenges

New format for Project Management document Excel vs MS Project

Weekly updates to Commissioners Trying new techniques with areas that had challenges

group work sessions involve additional staff

Unable to re-use as much as expected of what we did last year Early code not optimal Feds made a lot of changes No optionals – deadlines more fixed

Staffing!

Virtual (vs. Dedicated) Dedicated (all full-time resources) – allows team to focus solely on

measurement and improvement of data processes and data quality but requires significant investment from the organization.

Virtual (all part-time resources) – a more practical approach for an organization getting started, but other “job” may distract from addressing data governance issues.

How many FTEs? For the first year we estimate approximately 4.0 FTE were dedicated to EDEN

reporting and Data Governance. (Headcount approximately 14; plus 15 DGB members).

For this year we anticipate this may decrease slightly (3.0 FTE). How do we fund the positions?

First year partially funded via an NCES Special Task Order; partially funded by tying in to other (funded) projects such as Enterprise Data System.

Future years - ???? Until reporting efficiencies are realized this is an issue!

Applying EDEN Accomplishments to the Enterprise

Information Security


Policy Management

Enterprise Data System - Metadata

EDEN – Federal

Reporting

Data Requests

Data Quality

DATA GOVERNANCE

Data Governance Structure

Data Stewards and Programmers

Data Steward Program Coordinator

Data Steward Program Director

Data Governance Board

KSDE Leadership

Data Request Review Board


KSDE Leadership

Data Stewards & Programmers


Director level decision makers from each department

Meet at least monthly Mission: Support Enterprise Data System

Initiative as a source of knowledge and advocacy, provide guidance, and ensure buy-in.

Actionable agendas Learn from one another (e.g., Demonstrations from

each department, updates on EDEN and EDS) Solve problems! (Data Calendar, Policy

Management, Data Requests, etc.)

Data Request Review Board

Provides consistent treatment of data requests.

Considers, prioritizes and assigns requests for data.

Uses automated escalation and tracking process (FootPrints®).

Meetings Monthly in person Adhoc virtually Review status of requests and assign priorities to

non-urgent requests

Other Roles & Responsibilities

Executive Leadership – Commissioner and Deputy Commissioners are responsible to advocate for data governance and data quality; and resolve issues escalated from DGB or Program Director.

Data Owners – Department Directors are responsible for applications and their associated data to define and approve access; and identify data security classification.

Data Custodian – Director of IT is responsible to ensure safety and integrity of data in custody of IT; implement application and data access controls appropriate for

security classification; and provide reasonable safeguards for information resources.

Issue Resolution

Setting expectationsEscalation Process

Data Stewards and Programmers

Data Steward Program Coordinator

Data Steward Program Director


KSDE Leadership

Issu

e E

scal

atio

n an

d A

ccou

ntab

ility


Initiative to ensure that critical data subjects are managed at the enterprise level (e.g., collected and updated at a single source).

DBG assists in identifying appropriate data groups and what source should be the “master”.

Currently we are implementing MDM processes for Organization data and core Student Data.

Teacher Assignment & Licensure

Budget & Finance

Migrant

Career & Tech Ed

Special Ed

Student Data Repository

Assessments

Cor

eS

tud

en

tD

ata

……

LEA

District

&

Schools

Student Identifier

Submission Verification







Staff Training & Capacity Building

Common Authentication – Security Architecture Security & Confidentiality Policies – Security Certificates

EDEN MetaData


Organizations

Enterprise Meta Data

Designed based on lessons from EDEN Meta Data

Re-use as much meta data as possibleEnterprise Meta Data (first version) is

specific to Enterprise Data Warehouse


Budget & Finance

Migrant

Career & Tech Ed

Special Ed


Assessments

Cor

eS

tud

en

tD

ata

……

LEA

District

&

Schools

Enterprise Data Warehouse

Student Identifier










Definitions Enterprise MetaData

Enterprise Meta Data

Organizations

Data Mart

Enterprise Data System

Iterative development process Prioritized subjects:

Iteration 1 – organizations, students, assessments, accountability Iteration 2 – staff, finance Iteration 3 - programs

Data Stewards will be trained regarding use of meta data and business intelligence tools

EDEN will become a data mart of the EDS Opportunity to give meaningful data back to LEAs Significant focus on training LEA staff regarding

use of meta data use of business intelligence tools effective data use


Budget & Finance

Migrant

Career & Tech Ed

Special Ed


Assessments

Cor

eS

tud

en

tD

ata

……

LEA

District

&

Schools


Student Identifier










Definitions

Enterprise Data System: Iteration 1

Organizations

Cleanse

Integrate

Transform

Load

Extraction

&

Analysis

Data Mart

Research

MetaDataBusiness Rules, Tech Info, Data Quality

Data Mart AYP


Integrated

Time Variant

Cleansed


Budget & Finance

Migrant

Career & Tech Ed

Special Ed


Assessments

Cor

eS

tud

en

tD

ata

……

LEA

District

&

Schools


Student Identifier










Definitions

Enterprise Data System: Iterations 2 and 3

Organizations

Cleanse

Integrate

Transform

Load

Extraction

&

Analysis

Data Mart

Research


Integrated

Time Variant

Cleansed

MetaDataBusiness Rules, Tech Info, Data Quality

Data Mart

State Rpts

Data Mart

Fed Rpts

Data Mart

LEA Analysis

Data Mart AYP

…

Return Data to the LEAs

Collaboration

Eases resource constraintsProvides consistent message to the fieldHelps minimize surprises!Promotes perspective that we’re in this

together.

Policies & Guidance

Data Governance Board has adopted this as an initiative:

Establishing standard template for documenting (with version control!)

Discussing central location for policiesImplemented process for public

comment

Data Quality

KIDS Data Quality Certification initiativeData Verification tools and guidance for

districtsFuture version: Accountability Integrated

Dashboard

Our Data Governance Challenges

Linking data and policiesDealing with “distractions” Data qualityChanges in the political climateStaffing

data governance: the kansas approach (ppt)

Technology

quality of data

data warehouse

eden data elements

institute data governance

data governance committee

kansas individual data

data quality act

questionable data quality