decision suppot system

196
“Not everything that counts can be counted. And not everything that can be counted - counts.” Albert Einstein Keep in Perspective

Upload: paul-seiler

Post on 04-Nov-2014

570 views

Category:

Technology


0 download

DESCRIPTION

Slides from a seminar by Lucian Parshall to the NZ Ministry of Education on Thursday 21 January 2010

TRANSCRIPT

Page 1: Decision Suppot System

“Not everything that counts !can be counted.!And not everything that !can be counted - counts.”

Albert Einstein

Keep in Perspective

Page 2: Decision Suppot System

“Not everything that counts !can be counted.!And not everything that !can be counted - counts.”

Keep in Perspective

Page 3: Decision Suppot System

How important is a DSS?

Page 4: Decision Suppot System

How important is a DSS?

53,800 employees

Imagine the CEO of a large enterprise with:

Page 5: Decision Suppot System

How important is a DSS?

53,800 employees2,600 branch offices

Imagine the CEO of a large enterprise with:

Page 6: Decision Suppot System

How important is a DSS?

53,800 employees2,600 branch offices$6.4 billion annual budget

Imagine the CEO of a large enterprise with:

Page 7: Decision Suppot System

How important is a DSS?

53,800 employees2,600 branch offices$6.4 billion annual budget758,000 customers

Imagine the CEO of a large enterprise with:

Page 8: Decision Suppot System

How important is a DSS?

53,800 employees2,600 branch offices$6.4 billion annual budget758,000 customers

Imagine the CEO of a large enterprise with:

What strategic information might this CEO expect to be available?

Page 9: Decision Suppot System

How important is a DSS?

53,800 employees2,600 branch offices$6.4 billion annual budget758,000 customers

Imagine the CEO of a large enterprise with:

What strategic information might this CEO expect to be available?

Would an 18 month delay in finding out how many employees

left the company be OK?

Page 10: Decision Suppot System

How important is a DSS?

53,800 employees2,600 branch offices$6.4 billion annual budget758,000 customers

Imagine the CEO of a large enterprise with:

What strategic information might this CEO expect to be available?

Finding out how a customer performed on an evaluation -

six months later?

Page 11: Decision Suppot System

How important is a DSS?

53,800 employees2,600 branch offices$6.4 billion annual budget758,000 customers

Imagine the CEO of a large enterprise with:

What strategic information might this CEO expect to be available?

Not knowing the location or the age of the technology in your

branch offices?

Page 12: Decision Suppot System

How important is a DSS?

53,800 employees2,600 branch offices$6.4 billion annual budget758,000 customers

Imagine the CEO of a large enterprise with:

What strategic information might this CEO expect to be available?

We find ourselves in the Information Age with an aging

information system

Page 13: Decision Suppot System

What are the needs of the educational community?

Page 14: Decision Suppot System

Current Education Data Sets

Student

Performance

Personal

Finance

Infrastructure

Foreign

Page 15: Decision Suppot System

Current Education Data Sets

Student

Performance

Personal

Finance

Infrastructure

Foreign

Page 16: Decision Suppot System

Current Education Data Sets

Student

Performance

Personal

Finance

Infrastructure

Foreign

Promotes narrow decisions based on information extracted from one or two functional data-sets

(finance and assessment)

Page 17: Decision Suppot System

Current Education Data Sets

Student

Performance

Personal

Finance

Infrastructure

Foreign

Redundant data entry is commonDisconnected data increases resources needed

Collections become costly and inefficient

Page 18: Decision Suppot System

Educational Data WarehousingPerformance TID (10 digit)NCLB mathNCLB read. . . . . . . . AP score

PSAT math. . . . . . . . . . . . . . . . ACT enroll

Student TID (10 digit)Teacher SS#LEA Number. . . . . . . .

Stud genderStu Grade lvl

Stu FTE. . . . . . . .

Admin Unit No.

School Infrastructure Admin Unit No.

. . . . . . . . Technology

Crime/Safety. . . . . . . . . . . . . . . . Bld Age

. . . . . . . . Title I

Supply IHE Unit No,. . . . . . . .

SS#. . . . . . . .

IHE Endorsement

Finance LEA Number. . . . . . . . Per PupilTotal Rev. . . . . . . . Avg Salary

Operating Bdgt. . . . . . . .

Gov Data Admin Unit No.

. . . . . . . . Live BirthsGPS systemNum ArrestsCong Dist

Foreign Data Admin Unit No.

. . . . . . . . Employment

NCESUniversity

PersonnelAdmin Unit No.

. . . . . . . . Teacher SS#

Teacher Assign. . . . . . . .

Type of Cert. . . . . . . .

Cert Exp Date

Data Partnerships

Page 19: Decision Suppot System

Educational Data WarehousingPerformance TID (10 digit)NCLB mathNCLB read. . . . . . . . AP score

PSAT math. . . . . . . . . . . . . . . . ACT enroll

Student TID (10 digit)Teacher SS#LEA Number. . . . . . . .

Stud genderStu Grade lvl

Stu FTE. . . . . . . .

Admin Unit No.

School Infrastructure Admin Unit No.

. . . . . . . . Technology

Crime/Safety. . . . . . . . . . . . . . . . Bld Age

. . . . . . . . Title I

Supply IHE Unit No,. . . . . . . .

SS#. . . . . . . .

IHE Endorsement

Finance LEA Number. . . . . . . . Per PupilTotal Rev. . . . . . . . Avg Salary

Operating Bdgt. . . . . . . .

Gov Data Admin Unit No.

. . . . . . . . Live BirthsGPS systemNum ArrestsCong Dist

Foreign Data Admin Unit No.

. . . . . . . . Employment

NCESUniversity

PersonnelAdmin Unit No.

. . . . . . . . Teacher SS#

Teacher Assign. . . . . . . .

Type of Cert. . . . . . . .

Cert Exp Date

Data Partnerships

andProvides for a common set of definitionsBecomes the sole source of reusable dataImproves timeliness and utility of reports

a location that:Integrates information from disparate

systems into a total view and a common foundation for understanding student performance and school improvement

Page 20: Decision Suppot System

What is a Data Warehouse?

Page 21: Decision Suppot System

What is a Data Warehouse?DW is not just storage but the

tools to query, analyze and present information on the web.

Page 22: Decision Suppot System

What is a Data Warehouse?DWs have many definitions - with these similarities:

Subject oriented - gives information about a person instead of operations.

Integrated - a variety of sources are merged into a whole.

Non-volatile - provides users with a consistent picture over specified time periods.

Robust architecture - that allows concurrent access by a multiple number of users with frequent queries.

Quality data - valid and reliable data that promotes confidence in DW and forms the nucleus of information used by the educational community.

Page 23: Decision Suppot System

What is a Data Warehouse?DWs have many definitions - with these similarities:

Subject oriented - gives information about a person instead of operations.

Integrated - a variety of sources are merged into a whole.

Non-volatile - provides users with a consistent picture over specified time periods.

Robust architecture - that allows concurrent access by a multiple number of users with frequent queries.

Quality data - valid and reliable data that promotes confidence in DW and forms the nucleus of information used by the educational community.

Page 24: Decision Suppot System

What is a Decision Support System?

DSS is a process used by the educational community (with support

of the data warehouse) that transforms data into a knowledgebase

that will support decision-making.

Page 25: Decision Suppot System

What is a Decision Support System?

DSS starts with a:Problem +

administration =

Data

Page 26: Decision Suppot System

What is a Decision Support System?

DSS starts with a:Problem +

administration =

Data + dissemination =

Information

Page 27: Decision Suppot System

What is a Decision Support System?

DSS starts with a:Problem +

administration =

Data + dissemination =

Information+ social

discussion =

Knowledge

Page 28: Decision Suppot System

What is a Decision Support System?

DSS starts with a:Problem +

administration =

Data + dissemination =

Information+ social

discussion =

Knowledge+ community response =

Policy/Action

Page 29: Decision Suppot System

What is a Decision Support System?

DSS starts with a:Problem +

administration =

Data + dissemination =

Information+ social

discussion =

Knowledge+ community response =

Policy/Action+ wisdom to ask a more

complex question =

Page 30: Decision Suppot System

12 Steps to Creating the DSS

These steps are a combination of buying and building that depend on

time and money

Page 31: Decision Suppot System

12 Steps to Creating the DSS

These steps are a combination of buying and building that depend on

time and money

Education CommunityInvolvement

Page 32: Decision Suppot System

12 Steps to Creating the DSS

These steps are a combination of buying and building that depend on

time and money

Conceptual AgreementDRA/DBA Staffing

Meta Data

Security/ConfidentialityUnique ID#

Edit check/ETL

Dup Res

BI tool

Data WarehouseData Mining

TrainingDecision Support System

Education CommunityInvolvement

Time from Operation to

Analysis

DRA

DBA

Information Democracy

Page 33: Decision Suppot System

12 Steps to Creating the DSS

Conceptual AgreementDRA/DBA Staffing

Meta Data

Security/ConfidentialityUnique ID#

Edit check/ETL

Dup Res

BI tool

Data WarehouseData Mining

TrainingDecision Support System

Education CommunityInvolvement

Time from Operation to

Analysis

DRA

DBALooks linear - is multidimensional

Information Democracy

Page 34: Decision Suppot System

12 Steps to Creating the DSS

Conceptual AgreementDRA/DBA Staffing

Meta Data

Security/ConfidentialityUnique ID#

Edit check/ETL

Dup Res

BI tool

Data WarehouseData Mining

TrainingDecision Support System

Education CommunityInvolvement

Time from Operation to

Analysis

DRA

DBA

Information Democracy

Factor in the fatigue-fizzle function

Page 35: Decision Suppot System

12 Steps to Creating the DSS

Conceptual AgreementDRA/DBA Staffing

Meta Data

Security/ConfidentialityUnique ID#

Edit check/ETL

Dup Res

BI tool

Data WarehouseData Mining

TrainingDecision Support System

Education CommunityInvolvement

Time from Operation to

Analysis

DRA

DBA

Information Democracy

Factor in the fatigue-fizzle function

Escape velocity

Page 36: Decision Suppot System

DS

S 12 Steps

Page 37: Decision Suppot System

Step #1Concept Formation

Initiation Phase

Project Planning

Execution/ ControlCloseout

DS

S 12 Steps

Page 38: Decision Suppot System

Building the Framework for the DW and DSS

Page 39: Decision Suppot System

Building the Framework for the DW and DSS

Must have conceptual agreement on the:

Page 40: Decision Suppot System

Building the Framework for the DW and DSS

Must have conceptual agreement on the:

Design of support system

Page 41: Decision Suppot System

Building the Framework for the DW and DSS

Must have conceptual agreement on the:

Design of support system Policy to protect data

Page 42: Decision Suppot System

Building the Framework for the DW and DSS

Must have conceptual agreement on the:

Design of support system Policy to protect data Advisory committee(s)

Page 43: Decision Suppot System

Building the Framework for the DW and DSS

Must have conceptual agreement on the:

Design of support system Policy to protect data Advisory committee(s) Buy or build and

Page 44: Decision Suppot System

Building the Framework for the DW and DSS

Must have conceptual agreement on the:

Design of support system Policy to protect data Advisory committee(s) Buy or build and Funding

Page 45: Decision Suppot System

Building the Framework for the DW and DSS

Must have conceptual agreement on the:

Design of support system Policy to protect data Advisory committee(s) Buy or build and Funding

Page 46: Decision Suppot System

DSS Design: Best Practice

Page 47: Decision Suppot System

DSS Design: Best Practice

Who? What?

When?Data Warehouse

School Codes

Whom? Where? With?

Page 48: Decision Suppot System

DSS Design: Best Practice

Who? What?

When?Data Warehouse

Decision Support Users

Used for:OperationManagementPolicy MakersInstructionResearch

Decision Support Tools

Used how:Data miningAnalysisAd-hoc queryOff-line manipulation

School Codes

Foreign datai.e. Employment, Higher Ed

Whom? Where? With?

Page 49: Decision Suppot System

DSS Design: Best Practice

Who? What?

When?Data Warehouse

Decision Support Users

Used for:OperationManagementPolicy MakersInstructionResearch

Decision Support Tools

Used how:Data miningAnalysisAd-hoc queryOff-line manipulation

School Codes

Data Democracy Web Interface

Foreign datai.e. Employment, Higher Ed

Whom? Where? With?

Page 50: Decision Suppot System

DSS Design: Best Practice

As professionals, we need to make informed decisions, anticipate their impact on

education and design appropriate policy.

Who? What?

When?Data Warehouse

Decision Support Users

Used for:OperationManagementPolicy MakersInstructionResearch

Decision Support Tools

Used how:Data miningAnalysisAd-hoc queryOff-line manipulation

School Codes

Data Democracy Web Interface

Foreign datai.e. Employment, Higher Ed

Whom? Where? With?

Why

Page 51: Decision Suppot System
Page 52: Decision Suppot System

Steering Committee Oversight to design of the DSS

Page 53: Decision Suppot System

Steering Committee Oversight to design of the DSS Local district policy concerns

Page 54: Decision Suppot System

Steering Committee Oversight to design of the DSS Local district policy concerns Meta Data modification

Page 55: Decision Suppot System

Steering Committee Oversight to design of the DSS Local district policy concerns Meta Data modification Standard reports, and

Page 56: Decision Suppot System

Steering Committee Oversight to design of the DSS Local district policy concerns Meta Data modification Standard reports, and Long term funding

Page 57: Decision Suppot System
Page 58: Decision Suppot System

2001 2002 2003 2004 2005 2006

Cost Savings?(OCIO-USED)

Current costs (paper and mail)

Warehouse/DSS initiative

Break even

Page 59: Decision Suppot System

2001 2002 2003 2004 2005 2006

Cost Savings?(OCIO-USED)

“We spend a lot of resources on an existing data edifice that isn’t very useful”

Current costs (paper and mail)

Warehouse/DSS initiative

Break even

Page 60: Decision Suppot System

DS

S 12 Steps

Page 61: Decision Suppot System

Step #2DRA & DBA

DS

S 12 Steps

Page 62: Decision Suppot System
Page 63: Decision Suppot System

Partnership on Both Sides of the Keyboard

Page 64: Decision Suppot System

Partnership on Both Sides of the Keyboard

DBA: technical implementation of the data warehouse

environment - chairs IT group

DRA: modifies and enforces standards that sustain the

DSS environment - chairs data

managers group

Page 65: Decision Suppot System
Page 66: Decision Suppot System

DRA and DBA CollaborationUser

requirements

Time18 Mo 30 Mo

Feature expectation (DRA)

IT Development Cycle (DBA)

Critical Divergence

Page 67: Decision Suppot System

DRA and DBA CollaborationUser

requirements

Time18 Mo 30 Mo

Feature expectation (DRA)

IT Development Cycle (DBA)

Critical Divergence

18 Mo

Page 68: Decision Suppot System

DRA and DBA CollaborationUser

requirements

Time18 Mo 30 Mo

Feature expectation (DRA)

IT Development Cycle (DBA)

Critical Divergence

18 Mo 30 Mo

Page 69: Decision Suppot System

DRA and DBA CollaborationUser

requirements

Time18 Mo 30 Mo

Feature expectation (DRA)

IT Development Cycle (DBA)

Critical Divergence

Outcome of building the DW within time frame:Data Warehouse will run 12-15 years - whereasCurrent apps last 6-7 years (with patches)

18 Mo 30 Mo

Page 70: Decision Suppot System

DS

S 12 Steps

Page 71: Decision Suppot System

Step #3Define the Data

DS

S 12 Steps

Page 72: Decision Suppot System

Meta DataData about the Database in the Data Warehouse

Page 73: Decision Suppot System

Meta DataData about the Database in the Data Warehouse

School Meta Data

Manual

Finance Meta Data Manual

Meta Data promotes the -• common understanding by users • data interchange with other agencies

Student Meta Data Manual

to define Meta Data break task into logical support

groupsPersonnel Meta Data Manual

Performance Meta Data Manual

Pupil Personnel

Human Resources

Finance Office

Test Company

Facilities Manager

Page 74: Decision Suppot System
Page 75: Decision Suppot System

Meta Data Online ManualsStudent

Personnel

Fina

nce

School Infrastructure

Performance

Page 76: Decision Suppot System

Meta Data Online ManualsStudent

Personnel

Fina

nce

School Infrastructure

Performance

Employment

Higher Education

Page 77: Decision Suppot System

Name of Field

Technical Information

Number of characters: (length)

Record position: (35-39)

Field type: (alpha, numeric, character)

SIF name:

XML tag: < >

Field Number

Warehouse name:R/Ecode

Warehouse type: VCAR

Blanks: (not accepted, null)

Progam Information

Date Information

Submission:

Code format:

Definition:

Elements (variables):

Revised:

Effective:

Discontinued:

Reporting Period:

?

Edits

Error traps:

Cross field edits:

Fatal Error:

Warning:

Historical Information

Form number replaced:

Statutory requirement:

Used for:

Report number:

Meta Data Online ManualsStudent

Personnel

Fina

nce

School Infrastructure

Performance

Employment

Higher Education

Page 78: Decision Suppot System

Step #4Maintaining Security and Confidentiality

DS

S 12 Steps

Page 79: Decision Suppot System

Protection is both sides of the keyboard

Page 80: Decision Suppot System

Protection is both sides of the keyboard

System Security (DBA)Identification (confident of who)

Authentication (confident of source)Authorization (grant access rights)

Access control (user profiling)Administration (security procedures)Auditing (monitoring and detection)

Page 81: Decision Suppot System

Protection is both sides of the keyboard

Confidentiality (DRA)

System Security (DBA)Identification (confident of who)

Authentication (confident of source)Authorization (grant access rights)

Access control (user profiling)Administration (security procedures)Auditing (monitoring and detection)

Established FERPA policyUnique NSN w/check sumStatistical disclosure (<6)

Human subject review policyPurge and destruction

Set levels of access & audit

Page 82: Decision Suppot System

Step #5Unique Testing ID (NSN)

DS

S 12 Steps

Page 83: Decision Suppot System
Page 84: Decision Suppot System

Test Identification Number: Production

Record layout

Warehouse layout

First nameLast name

Date of BirthGender………FTE………Grade

Race/Ethnic………………

TID (10 digit)First nameLast name

Date of BirthGender………FTE………Grade

Race/Ethnic………………

Page 85: Decision Suppot System

Test Identification Number: Production

Record layout

Warehouse layout

• Only assigned to one student (is unique).• Number and name can be confirmed as

being correct (verified via check sum).• Meets criteria as an identifier (is valid).• Has no intrinsic meaning (is nominal).• Can be substituted for a student’s name

(is not personally identifiable).• Permanent over the life-cycle of the

student (0-21 for special education).• Is returned and used by all local

education agencies (is ubiquitous).• Issued only by the SEA (is restricted).• Accessible by selected SEA employees

only (is confidential).

First nameLast name

Date of BirthGender………FTE………Grade

Race/Ethnic………………

TID (10 digit)First nameLast name

Date of BirthGender………FTE………Grade

Race/Ethnic………………

TID rules:

Page 86: Decision Suppot System

Test Identification Number: Problems

10 digit Check Sum

Constant

Variables

First nameLast name

Date of BirthGender………FTE………Grade

Race/Ethnic………………ID#

Admin Unit #

Page 87: Decision Suppot System

Test Identification Number: Problems

10 digit Check Sum

Constant

Variables Variables change

First nameLast name

Date of BirthGender………FTE………Grade

Race/Ethnic………………ID#

Admin Unit #

First nameLast name

Date of BirthGender………FTE………Grade

Race/Ethnic………………ID#

Admin Unit #

Moves

Page 88: Decision Suppot System

Test Identification Number: Problems

10 digit Check Sum

Need other constant:Date of ImmunizationPlace of BirthBirth Cert Number

Constant

Variables Variables change

First nameLast name

Date of BirthGender………FTE………Grade

Race/Ethnic………………ID#

Admin Unit #

First nameLast name

Date of BirthGender………FTE………Grade

Race/Ethnic………………ID#

Admin Unit #

Moves

Page 89: Decision Suppot System

42 states use a unique student identifier (DQC)

How constructed (NCES)

How issued (NCES)

Combination of fields (5)

Other (9)

Soc Sec Number (8)

SSN plus algorithm (1)

Random number (8)

SEA (20)

Other (4)

School (2)

LEA (9)

ISD (1)

Page 90: Decision Suppot System

Crossing over from aggregate to single record

Data reliability and validity

Time

Aggregate collection

Page 91: Decision Suppot System

Crossing over from aggregate to single record

Data reliability and validity

Time

Aggregate collection

Single record collection

Page 92: Decision Suppot System

Crossing over from aggregate to single record

Data reliability and validity

Time

Aggregate collection

Single record collection

Page 93: Decision Suppot System

Crossing over from aggregate to single record

Data reliability and validity

Time

Aggregate collection

Single record collection

Single record collection

Page 94: Decision Suppot System

Classrooms: District AClass size! Reading! 10! 8.04! 8! 6.95! 13! 7.58! 9! 8.81! 11! 8.33! 14! 9.96! 6! 7.24! 4! 4.26! 12! 10.84! 7! 4.82! 5! 5.68

Classrooms: District BClass size! Reading! 14! ! 8.1! 6! ! 6.13! 4! ! 3.1! 12! ! 9.13! 7! ! 7.26! 5! ! 4.74 ! 10! ! 9.14! 8! ! 8.14! 13! ! 8.74! 9! ! 8.77! 11! ! 9.26

Classrooms: District CClass size! Reading! 10! 7.46! 8! 6.77! 13! 12.74! 9! 7.11! 11! 7.81! 14! 8.84! 6! 6.08! 4! 5.39! 12! 8.15! 7! 6.42! 5! 5.73

Classrooms: District DClass size!Reading! 8! 6.58! 8! 5.76! 8! 7.71! 8! 8.84! 8! 8.47! 8! 7.04! 8! 5.25! 19! 12.5! 8! 5.56! 8! 7.91! 8! 6.89

Aggregated Data can be Misleading

Page 95: Decision Suppot System

Classrooms: District AClass size! Reading! 10! 8.04! 8! 6.95! 13! 7.58! 9! 8.81! 11! 8.33! 14! 9.96! 6! 7.24! 4! 4.26! 12! 10.84! 7! 4.82! 5! 5.68

Classrooms: District BClass size! Reading! 14! ! 8.1! 6! ! 6.13! 4! ! 3.1! 12! ! 9.13! 7! ! 7.26! 5! ! 4.74 ! 10! ! 9.14! 8! ! 8.14! 13! ! 8.74! 9! ! 8.77! 11! ! 9.26

Classrooms: District CClass size! Reading! 10! 7.46! 8! 6.77! 13! 12.74! 9! 7.11! 11! 7.81! 14! 8.84! 6! 6.08! 4! 5.39! 12! 8.15! 7! 6.42! 5! 5.73

Classrooms: District DClass size!Reading! 8! 6.58! 8! 5.76! 8! 7.71! 8! 8.84! 8! 8.47! 8! 7.04! 8! 5.25! 19! 12.5! 8! 5.56! 8! 7.91! 8! 6.89

Aggregated Data can be Misleading

! Avg. classrooms != 11! Avg. class size != 9.0! Avg. reading score != 7.5

Four districts are similar

Page 96: Decision Suppot System
Page 97: Decision Suppot System

District C

5

10

10 20

District D

5

10

10 20

District A

5

10

10 20

District B

5

10

10 20

Reports using Disaggregated Data

Individual reading scores

Four districts are very different

Page 98: Decision Suppot System

Cleaning the DataStep #6

DS

S 12 Steps

Page 99: Decision Suppot System
Page 100: Decision Suppot System

Quality Data

Page 101: Decision Suppot System

Quality DataReasons for poor quality of data:Absence of definitionsUnclear definitionsLack of human resourcesInconsistent collections cycles (not ongoing)Insufficient timeInadequate training on entry and data trapsLack of data integrationFear of 'punishment' (look bad syndrome)

Page 102: Decision Suppot System

Quality DataThe key elements that improve the quality of what is being collected include:

• Consistency. Data fields must have a standardized definition so that each entity can be collected from each district in a systematic manner.

• Timeliness. There is no efficiency in gathering statewide data that reflects a one-time need or an unusual piece of information. Do a survey.

• Reliability. The data set should reflect a dependable measurement of every entity from one collection cycle to another (i.e., data has accuracy regardless of who enters it.)

• Validity. A data element must reflect a logical and meaningful description of an entity and should not be subject to interpretation (i.e., data has utility to answer the question being asked.)

The key elements that improve the quality of what is being collected include:

• Consistency. Data fields must have a standardized definition so that each entity can be collected from each district in a systematic manner.

• Timeliness. There is no efficiency in gathering statewide data that reflects a one-time need or an unusual piece of information. Do a survey.

• Reliability. The data set should reflect a dependable measurement of every entity from one collection cycle to another (i.e., data has accuracy regardless of who enters it.)

• Validity. A data element must reflect a logical and meaningful description of an entity and should not be subject to interpretation (i.e., data has utility to answer the question being asked.)

Page 103: Decision Suppot System

Step #7Resolving Duplicates

DS

S 12 Steps

Page 104: Decision Suppot System

Thresholds and Assigning ID numbersTrue False True

Non-match

Match

Page 105: Decision Suppot System

Match is true - are the same student (assign same ID#)

Thresholds and Assigning ID numbers

Pat ! Smith! M! 1/19/60Pat! T! Smith! M! 1/19/60

True False True

Non-match

Match

Page 106: Decision Suppot System

Match is true - are the same student (assign same ID#)

Non-match is true - are different students

(assign different ID#s)

Thresholds and Assigning ID numbers

Pat ! Smith! M! 1/19/60Pat! T! Smith! M! 1/19/60

Pat ! Smith! F! 1/19/60Pat! T! Smith! ! 1/19/61Patrick ! Smith ! M! 1/19/60

True False True

Non-match

Match

Page 107: Decision Suppot System

Match is true - are the same student (assign same ID#)

Match is false - are different students (assign same ID#)

Non-match is true - are different students

(assign different ID#s)

Thresholds and Assigning ID numbers

Pat ! Smith! M! 1/19/60Pat! T! Smith! M! 1/19/60

Patricia! Smith! F! 1/19/60Pat! ! Smith! ! 1/19/60

Pat ! Smith! F! 1/19/60Pat! T! Smith! ! 1/19/61Patrick ! Smith ! M! 1/19/60

True False True

Non-match

Match

Page 108: Decision Suppot System

Match is true - are the same student (assign same ID#)

Match is false - are different students (assign same ID#)

Non-match is false - are the same student (assign different ID#s)

Non-match is true - are different students

(assign different ID#s)

Thresholds and Assigning ID numbers

Pat ! Smith! M! 1/19/60Pat! T! Smith! M! 1/19/60

Patricia! Smith! F! 1/19/60Pat! ! Smith! ! 1/19/60

Pat ! Smith! M! 1/19/60Patrick! Smith! ! 1/19/60Pat ! ! Smyth! M! 1/19/60

Pat ! Smith! F! 1/19/60Pat! T! Smith! ! 1/19/61Patrick ! Smith ! M! 1/19/60

True False True

Non-match

Match

Page 109: Decision Suppot System

Match is true - are the same student (assign same ID#)

Match is false - are different students (assign same ID#)

Non-match is false - are the same student (assign different ID#s)

Non-match is true - are different students

(assign different ID#s)

Thresholds and Assigning ID numbers

Pat ! Smith! M! 1/19/60Pat! T! Smith! M! 1/19/60

Patricia! Smith! F! 1/19/60Pat! ! Smith! ! 1/19/60

Pat ! Smith! M! 1/19/60Patrick! Smith! ! 1/19/60Pat ! ! Smyth! M! 1/19/60

Pat ! Smith! F! 1/19/60Pat! T! Smith! ! 1/19/61Patrick ! Smith ! M! 1/19/60

True False True

Non-match

Match

Error Creep

Page 110: Decision Suppot System

Step #8Select a BI Tool

DS

S 12 Steps

Page 111: Decision Suppot System

Task #1:Create Model

Software & Hardware

Page 112: Decision Suppot System

Task #1:Create Model

On scalable, normalized, symmetric multiprocessing

architecture

Software & Hardware

Page 113: Decision Suppot System

Task #2: Set up a ‘road map’

Page 114: Decision Suppot System

Task #3: Choose a BI tool

Page 115: Decision Suppot System

Task #3: Choose a BI tool

Page 116: Decision Suppot System

Step #9Data Warehouse

DS

S 12 Steps

Page 117: Decision Suppot System

Benefits of DW:

Page 118: Decision Suppot System

Benefits of DW:Reduction of paper formsSavings from data duplicationBest use of technologySole source of reusable dataCommon set of definitionsIntegrated environment of core dataBreaks cycle of low quality dataAnswers that took months take daysReports that took days take minutes

Reduction of paper formsSavings from data duplicationBest use of technologySole source of reusable dataCommon set of definitionsIntegrated environment of core dataBreaks cycle of low quality dataAnswers that took months take daysReports that took days take minutes

Page 119: Decision Suppot System

Data Democracy for the Educational CommunityRe

port

s

QuerySimple - one time

Sophisticated - ongoing

Ad-hoc

Pre-defined

Page 120: Decision Suppot System

Data Democracy for the Educational CommunityRe

port

s

QuerySimple - one time

Sophisticated - ongoing

Ad-hoc

Pre-defined

General Public

Finance Officers

Audit

ors

Reporters

ResearchersLegislative Aides

Page 121: Decision Suppot System

Data Democracy for the Educational CommunityRe

port

s

QuerySimple - one time

Sophisticated - ongoing

Ad-hoc

Pre-defined

General Public

Finance Officers

Audit

ors

Reporters

ResearchersLegislative Aides

Push

Pull

As system is used one will find a need to store data not being captured

Page 122: Decision Suppot System

Push example: one time - pre defined

Page 123: Decision Suppot System

School report card• School Size: small vs. large schools• Spending: percent of budget on staff salary• Safety: rate of expulsions and degree of crime• Technology: ratio of pc's to students & connectivity• Class Size: teacher-student ratio, average size• Staff Turnover: rate and attendance• Advanced Placement: number passing test• Test Scores: gaps in State performance test• College Acceptance Rate: percent taking ACT, PSAT• Graduation/Dropout Rates: number taking GED• Satisfaction: teachers, parents and students

Push example: one time - pre defined

Page 124: Decision Suppot System

Significant Usable

Pull example: ongoing - Ad hoc

Page 125: Decision Suppot System

The largest class size in high school is the 9th grade Not really No

Significant Usable

Pull example: ongoing - Ad hoc

Page 126: Decision Suppot System

The largest class size in high school is the 9th grade Not really No

Some 9th grades have a disproportionate number of Hispanics Possibly No

Significant Usable

Pull example: ongoing - Ad hoc

Page 127: Decision Suppot System

The largest class size in high school is the 9th grade Not really No

Some 9th grades have a disproportionate number of Hispanics Possibly No

Many female Hispanics in the 9th grade are retained due to poor science skills Possibly Yes

Significant Usable

Pull example: ongoing - Ad hoc

Page 128: Decision Suppot System

The largest class size in high school is the 9th grade Not really No

Some 9th grades have a disproportionate number of Hispanics Possibly No

Many female Hispanics in the 9th grade are retained due to poor science skills Possibly Yes

Hispanics in the 8th grade had fewer computers in science classrooms and more teachers who do not have a teaching major in science

Yes Yes

Significant Usable

Pull example: ongoing - Ad hoc

Page 129: Decision Suppot System

The DW Backbone:

NCLB

The Sole Authority for the Educational Community

School Accreditation

Crime/Safety

Quality Workforce

AYP State Report Card

IDEA

Title II (IHE)

Fiscal Trends

Page 130: Decision Suppot System

The DW Backbone:

NCLB

The Sole Authority for the Educational Community

School Accreditation

Crime/Safety

Quality Workforce

AYP State Report Card

IDEA

Title II (IHE)

Fiscal Trends

Page 131: Decision Suppot System

Step #10Data Mining

DS

S 12 Steps

Page 132: Decision Suppot System

Data re-construction

Page 133: Decision Suppot System

Data re-constructionUndirected and exploratory

knowledge discovery

Page 134: Decision Suppot System

Data re-constructionUndirected and exploratory

knowledge discovery

Sequencing: order of patterns or groups

Page 135: Decision Suppot System

Data re-constructionUndirected and exploratory

knowledge discovery

Sequencing: order of patterns or groups

Framing: using past data to predict trend

Page 136: Decision Suppot System

Data re-constructionUndirected and exploratory

knowledge discovery

Sequencing: order of patterns or groups

Framing: using past data to predict trend

Clustering: assembling unforeseen groups

Page 137: Decision Suppot System

Data re-constructionUndirected and exploratory

knowledge discovery

Sequencing: order of patterns or groups

Framing: using past data to predict trend

Clustering: assembling unforeseen groups

Drilling: interactive discovery

Page 138: Decision Suppot System

Multidimensional Ad-hoc Analysis

Student

Technology Infrastructure

Performance

Page 139: Decision Suppot System

Multidimensional Ad-hoc Analysis

Student

Technology Infrastructure

Performance

Single Parent Homes

Live Births

Millages Passed

Page 140: Decision Suppot System

Multidimensional Ad-hoc Analysis

Student

Technology Infrastructure

Performance

Single Parent Homes

Live Births

Millages Passed

Ethnic change and growth by

enrollment

Page 141: Decision Suppot System

Multidimensional Ad-hoc Analysis

Student

Technology Infrastructure

Performance

Single Parent Homes

Live Births

Millages Passed

Ethnic change and growth by

enrollment

Performance by gender by PCs

Page 142: Decision Suppot System

Multidimensional Ad-hoc Analysis

Student

Technology Infrastructure

Performance

Single Parent Homes

Live Births

Millages Passed

Ethnic change and growth by

enrollment

Performance by gender by PCs

Trends and Projections

Page 143: Decision Suppot System

Multidimensional Ad-hoc Analysis

Student

Technology Infrastructure

Performance

Single Parent Homes

Live Births

Millages Passed

Ethnic change and growth by

enrollment

Performance by gender by PCs

Trends and Projections

Similar districtsthat passed bonds

by monthover past 3 yrs

by ethnicityby buildingby grade

Page 144: Decision Suppot System

Step #11Conduct Training

DS

S 12 Steps

Page 145: Decision Suppot System

The ultimate goal of training is to have everyone who touches the data at every level know what is expected of them, so that the data that are submitted will be

the valid and reliable.

Page 146: Decision Suppot System

TrainingTraining must also include detailed procedures, for example:

Page 147: Decision Suppot System

Training

Who gets notified when an error is discovered and how is the notification done?

Training must also include detailed procedures, for example:

Page 148: Decision Suppot System

Training

Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)?

Training must also include detailed procedures, for example:

Page 149: Decision Suppot System

Training

Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)? Who reviews, verifies or signs off on the cleaned data?

Training must also include detailed procedures, for example:

Page 150: Decision Suppot System

Training

Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)? Who reviews, verifies or signs off on the cleaned data? Who provides technical assistance to the end user?

Training must also include detailed procedures, for example:

Page 151: Decision Suppot System

Training

Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)? Who reviews, verifies or signs off on the cleaned data? Who provides technical assistance to the end user? What is the procedure to ensure a new copy of the data is retained for auditing?

Training must also include detailed procedures, for example:

Page 152: Decision Suppot System

Training

Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)? Who reviews, verifies or signs off on the cleaned data? Who provides technical assistance to the end user? What is the procedure to ensure a new copy of the data is retained for auditing? Who receives confirmation that the file has been received as specified?

Training must also include detailed procedures, for example:

Page 153: Decision Suppot System

Training

Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)? Who reviews, verifies or signs off on the cleaned data? Who provides technical assistance to the end user? What is the procedure to ensure a new copy of the data is retained for auditing? Who receives confirmation that the file has been received as specified? Who secures the data and maintains confidentiality?

Training must also include detailed procedures, for example:

Page 154: Decision Suppot System

Reallocation of Resources

Data collection, error checks, and clean-up

Have multiple collections -

use once disregard

Analysis Reporting Decision support and shared data

Have

Page 155: Decision Suppot System

Reallocation of Resources

Data collection, error checks, and clean-up

Have multiple collections -

use once disregard

Analysis Reporting Decision support and shared data

WantHave

Staff training - shifts from front to back end

Page 156: Decision Suppot System

Step #12The DSS

DS

S 12 Steps

Page 157: Decision Suppot System

Step #12The DSS

DS

S 12 Steps

Providing access to critical information for driving, managing, tracking, and measuring

institutional policies and goals.

Page 158: Decision Suppot System

The first decision of the DSS is to make a decision

Transactional Cyclical

Page 159: Decision Suppot System

The first decision of the DSS is to make a decision

Transactional CyclicalRealtime Points in time

Day to day operations HistoricalUpdates daily/weekly Updates quarterly

7X24 6X18Read/write Read only

Short term data retention Long-term (longitudinal)Mission critical queries Strategic-analytical queriesMore open access paths More restricted accessStandardized reports Adhoc reports

Server based Warehouse technology

Page 160: Decision Suppot System

DSS: Helps Anticipate Issues

Policy Repercussion

Policy Forecasting

Problem Anticipation

Problem Reaction

Page 161: Decision Suppot System

DSS: Helps Anticipate Issues

Policy Repercussion

Policy Forecasting

Problem Anticipation

Problem Reaction

Current

Page 162: Decision Suppot System

DSS: Helps Anticipate Issues

Policy Repercussion

Policy Forecasting

Problem Anticipation

Problem Reaction

Need to be

Page 163: Decision Suppot System

DSS: Helps Anticipate Issues

Policy Repercussion

Policy Forecasting

Problem Anticipation

Problem Reaction

Need to be

Cannot anticipate with only ‘required’

data

Page 164: Decision Suppot System

Help Anticipate Impact of Policy:Class Size

Page 165: Decision Suppot System

Help Anticipate Impact of Policy:Class Size

Achievement Issues - Does reducing the class size below grade 4 improve achievement on the state performance test?

Page 166: Decision Suppot System

Help Anticipate Impact of Policy:Class Size

Achievement Issues - Does reducing the class size below grade 4 improve achievement on the state performance test?

Teacher Supply/Demand Issues - Will the current production of teachers by IHEs meet the increased need for staff at the LEAs?

Page 167: Decision Suppot System

Help Anticipate Impact of Policy:Class Size

Achievement Issues - Does reducing the class size below grade 4 improve achievement on the state performance test?

Teacher Supply/Demand Issues - Will the current production of teachers by IHEs meet the increased need for staff at the LEAs?

Fiscal Issues - Using average salary, will there be sufficient funds for LEAs to hire additional staff?

Page 168: Decision Suppot System

Help Anticipate Impact of Policy:Class Size

Achievement Issues - Does reducing the class size below grade 4 improve achievement on the state performance test?

Teacher Supply/Demand Issues - Will the current production of teachers by IHEs meet the increased need for staff at the LEAs?

Fiscal Issues - Using average salary, will there be sufficient funds for LEAs to hire additional staff?

Infrastructure Issues - Do buildings have the space for additional classrooms?

Page 169: Decision Suppot System

Help Anticipate Impact of Policy:Class Size

Achievement Issues - Does reducing the class size below grade 4 improve achievement on the state performance test?

Teacher Supply/Demand Issues - Will the current production of teachers by IHEs meet the increased need for staff at the LEAs?

Fiscal Issues - Using average salary, will there be sufficient funds for LEAs to hire additional staff?

Infrastructure Issues - Do buildings have the space for additional classrooms?

Trend Issues - Will improved achievement impact employment, graduation or adult life roles?

Page 170: Decision Suppot System

Impact on State Standards

Page 171: Decision Suppot System

Impact on State Standards

Efficiency of SystemInputs Process Outputs

Page 172: Decision Suppot System

Impact on State Standards

Efficiency of System

Input issues:fiscal resourcesteacher supply

building structuretechnologypoverty

Inputs Process Outputs

Page 173: Decision Suppot System

Impact on State Standards

Efficiency of System

Input issues:fiscal resourcesteacher supply

building structuretechnologypoverty Process issues:

crime and safetyprof development

attendanceteacher experiencestudent performance

Inputs Process Outputs

Page 174: Decision Suppot System

Impact on State Standards

Efficiency of System

Input issues:fiscal resourcesteacher supply

building structuretechnologypoverty Process issues:

crime and safetyprof development

attendanceteacher experiencestudent performance

Output issues:college entrance

graduate numbersretention ratesemployment

Inputs Process Outputs

Page 175: Decision Suppot System

Impact on State StandardsEffectiveness of System

Efficiency of System

Input issues:fiscal resourcesteacher supply

building structuretechnologypoverty Process issues:

crime and safetyprof development

attendanceteacher experiencestudent performance

Output issues:college entrance

graduate numbersretention ratesemployment

Inputs Process OutcomesOutputs

Impact Policy

Outcome issues:works with others

acquires informationunderstands inter-relationships

allocates resources works w/variety of tech

Page 176: Decision Suppot System

Impact on State StandardsEffectiveness of System

Efficiency of System

Output issues:college entrance

graduate numbersretention ratesemployment

Inputs Process OutcomesOutputs

Impact PolicyWill no

t impact

policy with on

ly

‘required’

data

Outcome issues:works with others

acquires informationunderstands inter-relationships

allocates resources works w/variety of tech

Page 177: Decision Suppot System

Will not impact

policy with on

ly

‘required’

data

Page 178: Decision Suppot System

Finding the BalanceRequired

Data

MandatoryMeasurement in volume

(amounts, avg., ranks, percents) Realistic

Social IntegrationVocational Orientation

Use of TimeDaily Living Skills

MobilityUse of Environmental Ques

Desired Data

Page 179: Decision Suppot System

Finding the BalanceRequired

Data

MandatoryMeasurement in volume

(amounts, avg., ranks, percents) Realistic

Social IntegrationVocational Orientation

Use of TimeDaily Living Skills

MobilityUse of Environmental Ques

The DSS must help policy makers find a comfortable balance between acceptable risks and benefits.

Desired Data

Page 180: Decision Suppot System

InputIssues

ProcessIssues

OutputIssues

OutcomesIssues

General Public

Parents

Teachers

Support Staff

Admin/Boards

State Legislators

Others

Helps in Data Discovery

Standards moves from efficiency to effectiveness

Page 181: Decision Suppot System

One Last Time

District UsersUpload data formatsCorrect duplicate dataFERPA requests

Public Portal AccessPredefined Report CardsLimited queriesDashboard/Scorecard

Web Front End

Error reports

Page 182: Decision Suppot System

One Last Time

File ETL: • Student • Assessment • Finance • Professional

Deve

lope

r Ap

plica

tion

s

Error reports

Check Sum

Student IDsMatch & Merge

Audit (FERPA)

SecurityDistrict UsersUpload data formatsCorrect duplicate dataFERPA requests

Public Portal AccessPredefined Report CardsLimited queriesDashboard/Scorecard

Web Front End

Error reports

Page 183: Decision Suppot System

One Last Time

File ETL: • Student • Assessment • Finance • Professional

Deve

lope

r Ap

plica

tion

s

WAREHOUSE

Error reports

Check Sum

Student IDsMatch & Merge

Audit (FERPA)

SecurityDistrict UsersUpload data formatsCorrect duplicate dataFERPA requests

Public Portal AccessPredefined Report CardsLimited queriesDashboard/Scorecard

Web Front End

Error reports

School Codes

Meta Data

GPS

Reliable/Valid

Page 184: Decision Suppot System

One Last Time

File ETL: • Student • Assessment • Finance • Professional

Deve

lope

r Ap

plica

tion

s

WAREHOUSE

Error reports

Check Sum

Data Mart

Student IDsMatch & Merge

Audit (FERPA)

Security

DoE UsersGenerate Report CardFederal: EDEN, NCLB, IDEASkopusIssue Assessment IDs

District UsersUpload data formatsCorrect duplicate dataFERPA requests

Public Portal AccessPredefined Report CardsLimited queriesDashboard/Scorecard

Web Front End

Error reports

School Codes

Meta Data

GPS

Reliable/Valid

Page 185: Decision Suppot System

Current problem:data rich and information poor

Page 186: Decision Suppot System

Current problem:data rich and information poor

Department

Data Silos

Page 187: Decision Suppot System

Current problem:data rich and information poor

Department Educational Community

Gap:Lack of confidence No trust in systemHave a low ROI

Data Silos

Page 188: Decision Suppot System

Solution

Department Educational Community

Data Democracy

Secure ScalableFlexible

Data Warehouse

Student

Meta Data

Manual

Performance Meta Data Manual

Personnel

Meta Data Manual

School Meta Data Manual

Finance Meta Data Manual

Apply information and facilitate

decision-making

Page 189: Decision Suppot System

Solution

Department Educational Community

Data Democracy

Secure ScalableFlexible

Data Warehouse

Student

Meta Data

Manual

Performance Meta Data Manual

Personnel

Meta Data Manual

School Meta Data Manual

Finance Meta Data Manual

Apply information and facilitate

decision-making

Page 190: Decision Suppot System
Page 192: Decision Suppot System

Without Data

You’re Just Another Person With an Opinion

Page 193: Decision Suppot System

We find ourselves in an

Information Agewith an aging information

system

Page 194: Decision Suppot System

Decisionsbegin with good data

Page 195: Decision Suppot System

Most of the fun using the DSS is not finding the

answer to your question - it’s finding the new

questions you don’t have the answers to.

Page 196: Decision Suppot System