decision suppot system

Post on 04-Nov-2014

570 Views

Category:

Technology

0 Downloads

Preview:

Click to see full reader

DESCRIPTION

Slides from a seminar by Lucian Parshall to the NZ Ministry of Education on Thursday 21 January 2010

TRANSCRIPT

“Not everything that counts !can be counted.!And not everything that !can be counted - counts.”

Albert Einstein

Keep in Perspective

“Not everything that counts !can be counted.!And not everything that !can be counted - counts.”

Keep in Perspective

How important is a DSS?

How important is a DSS?

53,800 employees

Imagine the CEO of a large enterprise with:

How important is a DSS?

53,800 employees2,600 branch offices

Imagine the CEO of a large enterprise with:

How important is a DSS?

53,800 employees2,600 branch offices$6.4 billion annual budget

Imagine the CEO of a large enterprise with:

How important is a DSS?

53,800 employees2,600 branch offices$6.4 billion annual budget758,000 customers

Imagine the CEO of a large enterprise with:

How important is a DSS?

53,800 employees2,600 branch offices$6.4 billion annual budget758,000 customers

Imagine the CEO of a large enterprise with:

What strategic information might this CEO expect to be available?

How important is a DSS?

53,800 employees2,600 branch offices$6.4 billion annual budget758,000 customers

Imagine the CEO of a large enterprise with:

What strategic information might this CEO expect to be available?

Would an 18 month delay in finding out how many employees

left the company be OK?

How important is a DSS?

53,800 employees2,600 branch offices$6.4 billion annual budget758,000 customers

Imagine the CEO of a large enterprise with:

What strategic information might this CEO expect to be available?

Finding out how a customer performed on an evaluation -

six months later?

How important is a DSS?

53,800 employees2,600 branch offices$6.4 billion annual budget758,000 customers

Imagine the CEO of a large enterprise with:

What strategic information might this CEO expect to be available?

Not knowing the location or the age of the technology in your

branch offices?

How important is a DSS?

53,800 employees2,600 branch offices$6.4 billion annual budget758,000 customers

Imagine the CEO of a large enterprise with:

What strategic information might this CEO expect to be available?

We find ourselves in the Information Age with an aging

information system

What are the needs of the educational community?

Current Education Data Sets

Student

Performance

Personal

Finance

Infrastructure

Foreign

Current Education Data Sets

Student

Performance

Personal

Finance

Infrastructure

Foreign

Current Education Data Sets

Student

Performance

Personal

Finance

Infrastructure

Foreign

Promotes narrow decisions based on information extracted from one or two functional data-sets

(finance and assessment)

Current Education Data Sets

Student

Performance

Personal

Finance

Infrastructure

Foreign

Redundant data entry is commonDisconnected data increases resources needed

Collections become costly and inefficient

Educational Data WarehousingPerformance TID (10 digit)NCLB mathNCLB read. . . . . . . . AP score

PSAT math. . . . . . . . . . . . . . . . ACT enroll

Student TID (10 digit)Teacher SS#LEA Number. . . . . . . .

Stud genderStu Grade lvl

Stu FTE. . . . . . . .

Admin Unit No.

School Infrastructure Admin Unit No.

. . . . . . . . Technology

Crime/Safety. . . . . . . . . . . . . . . . Bld Age

. . . . . . . . Title I

Supply IHE Unit No,. . . . . . . .

SS#. . . . . . . .

IHE Endorsement

Finance LEA Number. . . . . . . . Per PupilTotal Rev. . . . . . . . Avg Salary

Operating Bdgt. . . . . . . .

Gov Data Admin Unit No.

. . . . . . . . Live BirthsGPS systemNum ArrestsCong Dist

Foreign Data Admin Unit No.

. . . . . . . . Employment

NCESUniversity

PersonnelAdmin Unit No.

. . . . . . . . Teacher SS#

Teacher Assign. . . . . . . .

Type of Cert. . . . . . . .

Cert Exp Date

Data Partnerships

Educational Data WarehousingPerformance TID (10 digit)NCLB mathNCLB read. . . . . . . . AP score

PSAT math. . . . . . . . . . . . . . . . ACT enroll

Student TID (10 digit)Teacher SS#LEA Number. . . . . . . .

Stud genderStu Grade lvl

Stu FTE. . . . . . . .

Admin Unit No.

School Infrastructure Admin Unit No.

. . . . . . . . Technology

Crime/Safety. . . . . . . . . . . . . . . . Bld Age

. . . . . . . . Title I

Supply IHE Unit No,. . . . . . . .

SS#. . . . . . . .

IHE Endorsement

Finance LEA Number. . . . . . . . Per PupilTotal Rev. . . . . . . . Avg Salary

Operating Bdgt. . . . . . . .

Gov Data Admin Unit No.

. . . . . . . . Live BirthsGPS systemNum ArrestsCong Dist

Foreign Data Admin Unit No.

. . . . . . . . Employment

NCESUniversity

PersonnelAdmin Unit No.

. . . . . . . . Teacher SS#

Teacher Assign. . . . . . . .

Type of Cert. . . . . . . .

Cert Exp Date

Data Partnerships

andProvides for a common set of definitionsBecomes the sole source of reusable dataImproves timeliness and utility of reports

a location that:Integrates information from disparate

systems into a total view and a common foundation for understanding student performance and school improvement

What is a Data Warehouse?

What is a Data Warehouse?DW is not just storage but the

tools to query, analyze and present information on the web.

What is a Data Warehouse?DWs have many definitions - with these similarities:

Subject oriented - gives information about a person instead of operations.

Integrated - a variety of sources are merged into a whole.

Non-volatile - provides users with a consistent picture over specified time periods.

Robust architecture - that allows concurrent access by a multiple number of users with frequent queries.

Quality data - valid and reliable data that promotes confidence in DW and forms the nucleus of information used by the educational community.

What is a Data Warehouse?DWs have many definitions - with these similarities:

Subject oriented - gives information about a person instead of operations.

Integrated - a variety of sources are merged into a whole.

Non-volatile - provides users with a consistent picture over specified time periods.

Robust architecture - that allows concurrent access by a multiple number of users with frequent queries.

Quality data - valid and reliable data that promotes confidence in DW and forms the nucleus of information used by the educational community.

What is a Decision Support System?

DSS is a process used by the educational community (with support

of the data warehouse) that transforms data into a knowledgebase

that will support decision-making.

What is a Decision Support System?

DSS starts with a:Problem +

administration =

Data

What is a Decision Support System?

DSS starts with a:Problem +

administration =

Data + dissemination =

Information

What is a Decision Support System?

DSS starts with a:Problem +

administration =

Data + dissemination =

Information+ social

discussion =

Knowledge

What is a Decision Support System?

DSS starts with a:Problem +

administration =

Data + dissemination =

Information+ social

discussion =

Knowledge+ community response =

Policy/Action

What is a Decision Support System?

DSS starts with a:Problem +

administration =

Data + dissemination =

Information+ social

discussion =

Knowledge+ community response =

Policy/Action+ wisdom to ask a more

complex question =

12 Steps to Creating the DSS

These steps are a combination of buying and building that depend on

time and money

12 Steps to Creating the DSS

These steps are a combination of buying and building that depend on

time and money

Education CommunityInvolvement

12 Steps to Creating the DSS

These steps are a combination of buying and building that depend on

time and money

Conceptual AgreementDRA/DBA Staffing

Meta Data

Security/ConfidentialityUnique ID#

Edit check/ETL

Dup Res

BI tool

Data WarehouseData Mining

TrainingDecision Support System

Education CommunityInvolvement

Time from Operation to

Analysis

DRA

DBA

Information Democracy

12 Steps to Creating the DSS

Conceptual AgreementDRA/DBA Staffing

Meta Data

Security/ConfidentialityUnique ID#

Edit check/ETL

Dup Res

BI tool

Data WarehouseData Mining

TrainingDecision Support System

Education CommunityInvolvement

Time from Operation to

Analysis

DRA

DBALooks linear - is multidimensional

Information Democracy

12 Steps to Creating the DSS

Conceptual AgreementDRA/DBA Staffing

Meta Data

Security/ConfidentialityUnique ID#

Edit check/ETL

Dup Res

BI tool

Data WarehouseData Mining

TrainingDecision Support System

Education CommunityInvolvement

Time from Operation to

Analysis

DRA

DBA

Information Democracy

Factor in the fatigue-fizzle function

12 Steps to Creating the DSS

Conceptual AgreementDRA/DBA Staffing

Meta Data

Security/ConfidentialityUnique ID#

Edit check/ETL

Dup Res

BI tool

Data WarehouseData Mining

TrainingDecision Support System

Education CommunityInvolvement

Time from Operation to

Analysis

DRA

DBA

Information Democracy

Factor in the fatigue-fizzle function

Escape velocity

DS

S 12 Steps

Step #1Concept Formation

Initiation Phase

Project Planning

Execution/ ControlCloseout

DS

S 12 Steps

Building the Framework for the DW and DSS

Building the Framework for the DW and DSS

Must have conceptual agreement on the:

Building the Framework for the DW and DSS

Must have conceptual agreement on the:

Design of support system

Building the Framework for the DW and DSS

Must have conceptual agreement on the:

Design of support system Policy to protect data

Building the Framework for the DW and DSS

Must have conceptual agreement on the:

Design of support system Policy to protect data Advisory committee(s)

Building the Framework for the DW and DSS

Must have conceptual agreement on the:

Design of support system Policy to protect data Advisory committee(s) Buy or build and

Building the Framework for the DW and DSS

Must have conceptual agreement on the:

Design of support system Policy to protect data Advisory committee(s) Buy or build and Funding

Building the Framework for the DW and DSS

Must have conceptual agreement on the:

Design of support system Policy to protect data Advisory committee(s) Buy or build and Funding

DSS Design: Best Practice

DSS Design: Best Practice

Who? What?

When?Data Warehouse

School Codes

Whom? Where? With?

DSS Design: Best Practice

Who? What?

When?Data Warehouse

Decision Support Users

Used for:OperationManagementPolicy MakersInstructionResearch

Decision Support Tools

Used how:Data miningAnalysisAd-hoc queryOff-line manipulation

School Codes

Foreign datai.e. Employment, Higher Ed

Whom? Where? With?

DSS Design: Best Practice

Who? What?

When?Data Warehouse

Decision Support Users

Used for:OperationManagementPolicy MakersInstructionResearch

Decision Support Tools

Used how:Data miningAnalysisAd-hoc queryOff-line manipulation

School Codes

Data Democracy Web Interface

Foreign datai.e. Employment, Higher Ed

Whom? Where? With?

DSS Design: Best Practice

As professionals, we need to make informed decisions, anticipate their impact on

education and design appropriate policy.

Who? What?

When?Data Warehouse

Decision Support Users

Used for:OperationManagementPolicy MakersInstructionResearch

Decision Support Tools

Used how:Data miningAnalysisAd-hoc queryOff-line manipulation

School Codes

Data Democracy Web Interface

Foreign datai.e. Employment, Higher Ed

Whom? Where? With?

Why

Steering Committee Oversight to design of the DSS

Steering Committee Oversight to design of the DSS Local district policy concerns

Steering Committee Oversight to design of the DSS Local district policy concerns Meta Data modification

Steering Committee Oversight to design of the DSS Local district policy concerns Meta Data modification Standard reports, and

Steering Committee Oversight to design of the DSS Local district policy concerns Meta Data modification Standard reports, and Long term funding

2001 2002 2003 2004 2005 2006

Cost Savings?(OCIO-USED)

Current costs (paper and mail)

Warehouse/DSS initiative

Break even

2001 2002 2003 2004 2005 2006

Cost Savings?(OCIO-USED)

“We spend a lot of resources on an existing data edifice that isn’t very useful”

Current costs (paper and mail)

Warehouse/DSS initiative

Break even

DS

S 12 Steps

Step #2DRA & DBA

DS

S 12 Steps

Partnership on Both Sides of the Keyboard

Partnership on Both Sides of the Keyboard

DBA: technical implementation of the data warehouse

environment - chairs IT group

DRA: modifies and enforces standards that sustain the

DSS environment - chairs data

managers group

DRA and DBA CollaborationUser

requirements

Time18 Mo 30 Mo

Feature expectation (DRA)

IT Development Cycle (DBA)

Critical Divergence

DRA and DBA CollaborationUser

requirements

Time18 Mo 30 Mo

Feature expectation (DRA)

IT Development Cycle (DBA)

Critical Divergence

18 Mo

DRA and DBA CollaborationUser

requirements

Time18 Mo 30 Mo

Feature expectation (DRA)

IT Development Cycle (DBA)

Critical Divergence

18 Mo 30 Mo

DRA and DBA CollaborationUser

requirements

Time18 Mo 30 Mo

Feature expectation (DRA)

IT Development Cycle (DBA)

Critical Divergence

Outcome of building the DW within time frame:Data Warehouse will run 12-15 years - whereasCurrent apps last 6-7 years (with patches)

18 Mo 30 Mo

DS

S 12 Steps

Step #3Define the Data

DS

S 12 Steps

Meta DataData about the Database in the Data Warehouse

Meta DataData about the Database in the Data Warehouse

School Meta Data

Manual

Finance Meta Data Manual

Meta Data promotes the -• common understanding by users • data interchange with other agencies

Student Meta Data Manual

to define Meta Data break task into logical support

groupsPersonnel Meta Data Manual

Performance Meta Data Manual

Pupil Personnel

Human Resources

Finance Office

Test Company

Facilities Manager

Meta Data Online ManualsStudent

Personnel

Fina

nce

School Infrastructure

Performance

Meta Data Online ManualsStudent

Personnel

Fina

nce

School Infrastructure

Performance

Employment

Higher Education

Name of Field

Technical Information

Number of characters: (length)

Record position: (35-39)

Field type: (alpha, numeric, character)

SIF name:

XML tag: < >

Field Number

Warehouse name:R/Ecode

Warehouse type: VCAR

Blanks: (not accepted, null)

Progam Information

Date Information

Submission:

Code format:

Definition:

Elements (variables):

Revised:

Effective:

Discontinued:

Reporting Period:

?

Edits

Error traps:

Cross field edits:

Fatal Error:

Warning:

Historical Information

Form number replaced:

Statutory requirement:

Used for:

Report number:

Meta Data Online ManualsStudent

Personnel

Fina

nce

School Infrastructure

Performance

Employment

Higher Education

Step #4Maintaining Security and Confidentiality

DS

S 12 Steps

Protection is both sides of the keyboard

Protection is both sides of the keyboard

System Security (DBA)Identification (confident of who)

Authentication (confident of source)Authorization (grant access rights)

Access control (user profiling)Administration (security procedures)Auditing (monitoring and detection)

Protection is both sides of the keyboard

Confidentiality (DRA)

System Security (DBA)Identification (confident of who)

Authentication (confident of source)Authorization (grant access rights)

Access control (user profiling)Administration (security procedures)Auditing (monitoring and detection)

Established FERPA policyUnique NSN w/check sumStatistical disclosure (<6)

Human subject review policyPurge and destruction

Set levels of access & audit

Step #5Unique Testing ID (NSN)

DS

S 12 Steps

Test Identification Number: Production

Record layout

Warehouse layout

First nameLast name

Date of BirthGender………FTE………Grade

Race/Ethnic………………

TID (10 digit)First nameLast name

Date of BirthGender………FTE………Grade

Race/Ethnic………………

Test Identification Number: Production

Record layout

Warehouse layout

• Only assigned to one student (is unique).• Number and name can be confirmed as

being correct (verified via check sum).• Meets criteria as an identifier (is valid).• Has no intrinsic meaning (is nominal).• Can be substituted for a student’s name

(is not personally identifiable).• Permanent over the life-cycle of the

student (0-21 for special education).• Is returned and used by all local

education agencies (is ubiquitous).• Issued only by the SEA (is restricted).• Accessible by selected SEA employees

only (is confidential).

First nameLast name

Date of BirthGender………FTE………Grade

Race/Ethnic………………

TID (10 digit)First nameLast name

Date of BirthGender………FTE………Grade

Race/Ethnic………………

TID rules:

Test Identification Number: Problems

10 digit Check Sum

Constant

Variables

First nameLast name

Date of BirthGender………FTE………Grade

Race/Ethnic………………ID#

Admin Unit #

Test Identification Number: Problems

10 digit Check Sum

Constant

Variables Variables change

First nameLast name

Date of BirthGender………FTE………Grade

Race/Ethnic………………ID#

Admin Unit #

First nameLast name

Date of BirthGender………FTE………Grade

Race/Ethnic………………ID#

Admin Unit #

Moves

Test Identification Number: Problems

10 digit Check Sum

Need other constant:Date of ImmunizationPlace of BirthBirth Cert Number

Constant

Variables Variables change

First nameLast name

Date of BirthGender………FTE………Grade

Race/Ethnic………………ID#

Admin Unit #

First nameLast name

Date of BirthGender………FTE………Grade

Race/Ethnic………………ID#

Admin Unit #

Moves

42 states use a unique student identifier (DQC)

How constructed (NCES)

How issued (NCES)

Combination of fields (5)

Other (9)

Soc Sec Number (8)

SSN plus algorithm (1)

Random number (8)

SEA (20)

Other (4)

School (2)

LEA (9)

ISD (1)

Crossing over from aggregate to single record

Data reliability and validity

Time

Aggregate collection

Crossing over from aggregate to single record

Data reliability and validity

Time

Aggregate collection

Single record collection

Crossing over from aggregate to single record

Data reliability and validity

Time

Aggregate collection

Single record collection

Crossing over from aggregate to single record

Data reliability and validity

Time

Aggregate collection

Single record collection

Single record collection

Classrooms: District AClass size! Reading! 10! 8.04! 8! 6.95! 13! 7.58! 9! 8.81! 11! 8.33! 14! 9.96! 6! 7.24! 4! 4.26! 12! 10.84! 7! 4.82! 5! 5.68

Classrooms: District BClass size! Reading! 14! ! 8.1! 6! ! 6.13! 4! ! 3.1! 12! ! 9.13! 7! ! 7.26! 5! ! 4.74 ! 10! ! 9.14! 8! ! 8.14! 13! ! 8.74! 9! ! 8.77! 11! ! 9.26

Classrooms: District CClass size! Reading! 10! 7.46! 8! 6.77! 13! 12.74! 9! 7.11! 11! 7.81! 14! 8.84! 6! 6.08! 4! 5.39! 12! 8.15! 7! 6.42! 5! 5.73

Classrooms: District DClass size!Reading! 8! 6.58! 8! 5.76! 8! 7.71! 8! 8.84! 8! 8.47! 8! 7.04! 8! 5.25! 19! 12.5! 8! 5.56! 8! 7.91! 8! 6.89

Aggregated Data can be Misleading

Classrooms: District AClass size! Reading! 10! 8.04! 8! 6.95! 13! 7.58! 9! 8.81! 11! 8.33! 14! 9.96! 6! 7.24! 4! 4.26! 12! 10.84! 7! 4.82! 5! 5.68

Classrooms: District BClass size! Reading! 14! ! 8.1! 6! ! 6.13! 4! ! 3.1! 12! ! 9.13! 7! ! 7.26! 5! ! 4.74 ! 10! ! 9.14! 8! ! 8.14! 13! ! 8.74! 9! ! 8.77! 11! ! 9.26

Classrooms: District CClass size! Reading! 10! 7.46! 8! 6.77! 13! 12.74! 9! 7.11! 11! 7.81! 14! 8.84! 6! 6.08! 4! 5.39! 12! 8.15! 7! 6.42! 5! 5.73

Classrooms: District DClass size!Reading! 8! 6.58! 8! 5.76! 8! 7.71! 8! 8.84! 8! 8.47! 8! 7.04! 8! 5.25! 19! 12.5! 8! 5.56! 8! 7.91! 8! 6.89

Aggregated Data can be Misleading

! Avg. classrooms != 11! Avg. class size != 9.0! Avg. reading score != 7.5

Four districts are similar

District C

5

10

10 20

District D

5

10

10 20

District A

5

10

10 20

District B

5

10

10 20

Reports using Disaggregated Data

Individual reading scores

Four districts are very different

Cleaning the DataStep #6

DS

S 12 Steps

Quality Data

Quality DataReasons for poor quality of data:Absence of definitionsUnclear definitionsLack of human resourcesInconsistent collections cycles (not ongoing)Insufficient timeInadequate training on entry and data trapsLack of data integrationFear of 'punishment' (look bad syndrome)

Quality DataThe key elements that improve the quality of what is being collected include:

• Consistency. Data fields must have a standardized definition so that each entity can be collected from each district in a systematic manner.

• Timeliness. There is no efficiency in gathering statewide data that reflects a one-time need or an unusual piece of information. Do a survey.

• Reliability. The data set should reflect a dependable measurement of every entity from one collection cycle to another (i.e., data has accuracy regardless of who enters it.)

• Validity. A data element must reflect a logical and meaningful description of an entity and should not be subject to interpretation (i.e., data has utility to answer the question being asked.)

The key elements that improve the quality of what is being collected include:

• Consistency. Data fields must have a standardized definition so that each entity can be collected from each district in a systematic manner.

• Timeliness. There is no efficiency in gathering statewide data that reflects a one-time need or an unusual piece of information. Do a survey.

• Reliability. The data set should reflect a dependable measurement of every entity from one collection cycle to another (i.e., data has accuracy regardless of who enters it.)

• Validity. A data element must reflect a logical and meaningful description of an entity and should not be subject to interpretation (i.e., data has utility to answer the question being asked.)

Step #7Resolving Duplicates

DS

S 12 Steps

Thresholds and Assigning ID numbersTrue False True

Non-match

Match

Match is true - are the same student (assign same ID#)

Thresholds and Assigning ID numbers

Pat ! Smith! M! 1/19/60Pat! T! Smith! M! 1/19/60

True False True

Non-match

Match

Match is true - are the same student (assign same ID#)

Non-match is true - are different students

(assign different ID#s)

Thresholds and Assigning ID numbers

Pat ! Smith! M! 1/19/60Pat! T! Smith! M! 1/19/60

Pat ! Smith! F! 1/19/60Pat! T! Smith! ! 1/19/61Patrick ! Smith ! M! 1/19/60

True False True

Non-match

Match

Match is true - are the same student (assign same ID#)

Match is false - are different students (assign same ID#)

Non-match is true - are different students

(assign different ID#s)

Thresholds and Assigning ID numbers

Pat ! Smith! M! 1/19/60Pat! T! Smith! M! 1/19/60

Patricia! Smith! F! 1/19/60Pat! ! Smith! ! 1/19/60

Pat ! Smith! F! 1/19/60Pat! T! Smith! ! 1/19/61Patrick ! Smith ! M! 1/19/60

True False True

Non-match

Match

Match is true - are the same student (assign same ID#)

Match is false - are different students (assign same ID#)

Non-match is false - are the same student (assign different ID#s)

Non-match is true - are different students

(assign different ID#s)

Thresholds and Assigning ID numbers

Pat ! Smith! M! 1/19/60Pat! T! Smith! M! 1/19/60

Patricia! Smith! F! 1/19/60Pat! ! Smith! ! 1/19/60

Pat ! Smith! M! 1/19/60Patrick! Smith! ! 1/19/60Pat ! ! Smyth! M! 1/19/60

Pat ! Smith! F! 1/19/60Pat! T! Smith! ! 1/19/61Patrick ! Smith ! M! 1/19/60

True False True

Non-match

Match

Match is true - are the same student (assign same ID#)

Match is false - are different students (assign same ID#)

Non-match is false - are the same student (assign different ID#s)

Non-match is true - are different students

(assign different ID#s)

Thresholds and Assigning ID numbers

Pat ! Smith! M! 1/19/60Pat! T! Smith! M! 1/19/60

Patricia! Smith! F! 1/19/60Pat! ! Smith! ! 1/19/60

Pat ! Smith! M! 1/19/60Patrick! Smith! ! 1/19/60Pat ! ! Smyth! M! 1/19/60

Pat ! Smith! F! 1/19/60Pat! T! Smith! ! 1/19/61Patrick ! Smith ! M! 1/19/60

True False True

Non-match

Match

Error Creep

Step #8Select a BI Tool

DS

S 12 Steps

Task #1:Create Model

Software & Hardware

Task #1:Create Model

On scalable, normalized, symmetric multiprocessing

architecture

Software & Hardware

Task #2: Set up a ‘road map’

Task #3: Choose a BI tool

Task #3: Choose a BI tool

Step #9Data Warehouse

DS

S 12 Steps

Benefits of DW:

Benefits of DW:Reduction of paper formsSavings from data duplicationBest use of technologySole source of reusable dataCommon set of definitionsIntegrated environment of core dataBreaks cycle of low quality dataAnswers that took months take daysReports that took days take minutes

Reduction of paper formsSavings from data duplicationBest use of technologySole source of reusable dataCommon set of definitionsIntegrated environment of core dataBreaks cycle of low quality dataAnswers that took months take daysReports that took days take minutes

Data Democracy for the Educational CommunityRe

port

s

QuerySimple - one time

Sophisticated - ongoing

Ad-hoc

Pre-defined

Data Democracy for the Educational CommunityRe

port

s

QuerySimple - one time

Sophisticated - ongoing

Ad-hoc

Pre-defined

General Public

Finance Officers

Audit

ors

Reporters

ResearchersLegislative Aides

Data Democracy for the Educational CommunityRe

port

s

QuerySimple - one time

Sophisticated - ongoing

Ad-hoc

Pre-defined

General Public

Finance Officers

Audit

ors

Reporters

ResearchersLegislative Aides

Push

Pull

As system is used one will find a need to store data not being captured

Push example: one time - pre defined

School report card• School Size: small vs. large schools• Spending: percent of budget on staff salary• Safety: rate of expulsions and degree of crime• Technology: ratio of pc's to students & connectivity• Class Size: teacher-student ratio, average size• Staff Turnover: rate and attendance• Advanced Placement: number passing test• Test Scores: gaps in State performance test• College Acceptance Rate: percent taking ACT, PSAT• Graduation/Dropout Rates: number taking GED• Satisfaction: teachers, parents and students

Push example: one time - pre defined

Significant Usable

Pull example: ongoing - Ad hoc

The largest class size in high school is the 9th grade Not really No

Significant Usable

Pull example: ongoing - Ad hoc

The largest class size in high school is the 9th grade Not really No

Some 9th grades have a disproportionate number of Hispanics Possibly No

Significant Usable

Pull example: ongoing - Ad hoc

The largest class size in high school is the 9th grade Not really No

Some 9th grades have a disproportionate number of Hispanics Possibly No

Many female Hispanics in the 9th grade are retained due to poor science skills Possibly Yes

Significant Usable

Pull example: ongoing - Ad hoc

The largest class size in high school is the 9th grade Not really No

Some 9th grades have a disproportionate number of Hispanics Possibly No

Many female Hispanics in the 9th grade are retained due to poor science skills Possibly Yes

Hispanics in the 8th grade had fewer computers in science classrooms and more teachers who do not have a teaching major in science

Yes Yes

Significant Usable

Pull example: ongoing - Ad hoc

The DW Backbone:

NCLB

The Sole Authority for the Educational Community

School Accreditation

Crime/Safety

Quality Workforce

AYP State Report Card

IDEA

Title II (IHE)

Fiscal Trends

The DW Backbone:

NCLB

The Sole Authority for the Educational Community

School Accreditation

Crime/Safety

Quality Workforce

AYP State Report Card

IDEA

Title II (IHE)

Fiscal Trends

Step #10Data Mining

DS

S 12 Steps

Data re-construction

Data re-constructionUndirected and exploratory

knowledge discovery

Data re-constructionUndirected and exploratory

knowledge discovery

Sequencing: order of patterns or groups

Data re-constructionUndirected and exploratory

knowledge discovery

Sequencing: order of patterns or groups

Framing: using past data to predict trend

Data re-constructionUndirected and exploratory

knowledge discovery

Sequencing: order of patterns or groups

Framing: using past data to predict trend

Clustering: assembling unforeseen groups

Data re-constructionUndirected and exploratory

knowledge discovery

Sequencing: order of patterns or groups

Framing: using past data to predict trend

Clustering: assembling unforeseen groups

Drilling: interactive discovery

Multidimensional Ad-hoc Analysis

Student

Technology Infrastructure

Performance

Multidimensional Ad-hoc Analysis

Student

Technology Infrastructure

Performance

Single Parent Homes

Live Births

Millages Passed

Multidimensional Ad-hoc Analysis

Student

Technology Infrastructure

Performance

Single Parent Homes

Live Births

Millages Passed

Ethnic change and growth by

enrollment

Multidimensional Ad-hoc Analysis

Student

Technology Infrastructure

Performance

Single Parent Homes

Live Births

Millages Passed

Ethnic change and growth by

enrollment

Performance by gender by PCs

Multidimensional Ad-hoc Analysis

Student

Technology Infrastructure

Performance

Single Parent Homes

Live Births

Millages Passed

Ethnic change and growth by

enrollment

Performance by gender by PCs

Trends and Projections

Multidimensional Ad-hoc Analysis

Student

Technology Infrastructure

Performance

Single Parent Homes

Live Births

Millages Passed

Ethnic change and growth by

enrollment

Performance by gender by PCs

Trends and Projections

Similar districtsthat passed bonds

by monthover past 3 yrs

by ethnicityby buildingby grade

Step #11Conduct Training

DS

S 12 Steps

The ultimate goal of training is to have everyone who touches the data at every level know what is expected of them, so that the data that are submitted will be

the valid and reliable.

TrainingTraining must also include detailed procedures, for example:

Training

Who gets notified when an error is discovered and how is the notification done?

Training must also include detailed procedures, for example:

Training

Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)?

Training must also include detailed procedures, for example:

Training

Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)? Who reviews, verifies or signs off on the cleaned data?

Training must also include detailed procedures, for example:

Training

Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)? Who reviews, verifies or signs off on the cleaned data? Who provides technical assistance to the end user?

Training must also include detailed procedures, for example:

Training

Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)? Who reviews, verifies or signs off on the cleaned data? Who provides technical assistance to the end user? What is the procedure to ensure a new copy of the data is retained for auditing?

Training must also include detailed procedures, for example:

Training

Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)? Who reviews, verifies or signs off on the cleaned data? Who provides technical assistance to the end user? What is the procedure to ensure a new copy of the data is retained for auditing? Who receives confirmation that the file has been received as specified?

Training must also include detailed procedures, for example:

Training

Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)? Who reviews, verifies or signs off on the cleaned data? Who provides technical assistance to the end user? What is the procedure to ensure a new copy of the data is retained for auditing? Who receives confirmation that the file has been received as specified? Who secures the data and maintains confidentiality?

Training must also include detailed procedures, for example:

Reallocation of Resources

Data collection, error checks, and clean-up

Have multiple collections -

use once disregard

Analysis Reporting Decision support and shared data

Have

Reallocation of Resources

Data collection, error checks, and clean-up

Have multiple collections -

use once disregard

Analysis Reporting Decision support and shared data

WantHave

Staff training - shifts from front to back end

Step #12The DSS

DS

S 12 Steps

Step #12The DSS

DS

S 12 Steps

Providing access to critical information for driving, managing, tracking, and measuring

institutional policies and goals.

The first decision of the DSS is to make a decision

Transactional Cyclical

The first decision of the DSS is to make a decision

Transactional CyclicalRealtime Points in time

Day to day operations HistoricalUpdates daily/weekly Updates quarterly

7X24 6X18Read/write Read only

Short term data retention Long-term (longitudinal)Mission critical queries Strategic-analytical queriesMore open access paths More restricted accessStandardized reports Adhoc reports

Server based Warehouse technology

DSS: Helps Anticipate Issues

Policy Repercussion

Policy Forecasting

Problem Anticipation

Problem Reaction

DSS: Helps Anticipate Issues

Policy Repercussion

Policy Forecasting

Problem Anticipation

Problem Reaction

Current

DSS: Helps Anticipate Issues

Policy Repercussion

Policy Forecasting

Problem Anticipation

Problem Reaction

Need to be

DSS: Helps Anticipate Issues

Policy Repercussion

Policy Forecasting

Problem Anticipation

Problem Reaction

Need to be

Cannot anticipate with only ‘required’

data

Help Anticipate Impact of Policy:Class Size

Help Anticipate Impact of Policy:Class Size

Achievement Issues - Does reducing the class size below grade 4 improve achievement on the state performance test?

Help Anticipate Impact of Policy:Class Size

Achievement Issues - Does reducing the class size below grade 4 improve achievement on the state performance test?

Teacher Supply/Demand Issues - Will the current production of teachers by IHEs meet the increased need for staff at the LEAs?

Help Anticipate Impact of Policy:Class Size

Achievement Issues - Does reducing the class size below grade 4 improve achievement on the state performance test?

Teacher Supply/Demand Issues - Will the current production of teachers by IHEs meet the increased need for staff at the LEAs?

Fiscal Issues - Using average salary, will there be sufficient funds for LEAs to hire additional staff?

Help Anticipate Impact of Policy:Class Size

Achievement Issues - Does reducing the class size below grade 4 improve achievement on the state performance test?

Teacher Supply/Demand Issues - Will the current production of teachers by IHEs meet the increased need for staff at the LEAs?

Fiscal Issues - Using average salary, will there be sufficient funds for LEAs to hire additional staff?

Infrastructure Issues - Do buildings have the space for additional classrooms?

Help Anticipate Impact of Policy:Class Size

Achievement Issues - Does reducing the class size below grade 4 improve achievement on the state performance test?

Teacher Supply/Demand Issues - Will the current production of teachers by IHEs meet the increased need for staff at the LEAs?

Fiscal Issues - Using average salary, will there be sufficient funds for LEAs to hire additional staff?

Infrastructure Issues - Do buildings have the space for additional classrooms?

Trend Issues - Will improved achievement impact employment, graduation or adult life roles?

Impact on State Standards

Impact on State Standards

Efficiency of SystemInputs Process Outputs

Impact on State Standards

Efficiency of System

Input issues:fiscal resourcesteacher supply

building structuretechnologypoverty

Inputs Process Outputs

Impact on State Standards

Efficiency of System

Input issues:fiscal resourcesteacher supply

building structuretechnologypoverty Process issues:

crime and safetyprof development

attendanceteacher experiencestudent performance

Inputs Process Outputs

Impact on State Standards

Efficiency of System

Input issues:fiscal resourcesteacher supply

building structuretechnologypoverty Process issues:

crime and safetyprof development

attendanceteacher experiencestudent performance

Output issues:college entrance

graduate numbersretention ratesemployment

Inputs Process Outputs

Impact on State StandardsEffectiveness of System

Efficiency of System

Input issues:fiscal resourcesteacher supply

building structuretechnologypoverty Process issues:

crime and safetyprof development

attendanceteacher experiencestudent performance

Output issues:college entrance

graduate numbersretention ratesemployment

Inputs Process OutcomesOutputs

Impact Policy

Outcome issues:works with others

acquires informationunderstands inter-relationships

allocates resources works w/variety of tech

Impact on State StandardsEffectiveness of System

Efficiency of System

Output issues:college entrance

graduate numbersretention ratesemployment

Inputs Process OutcomesOutputs

Impact PolicyWill no

t impact

policy with on

ly

‘required’

data

Outcome issues:works with others

acquires informationunderstands inter-relationships

allocates resources works w/variety of tech

Will not impact

policy with on

ly

‘required’

data

Finding the BalanceRequired

Data

MandatoryMeasurement in volume

(amounts, avg., ranks, percents) Realistic

Social IntegrationVocational Orientation

Use of TimeDaily Living Skills

MobilityUse of Environmental Ques

Desired Data

Finding the BalanceRequired

Data

MandatoryMeasurement in volume

(amounts, avg., ranks, percents) Realistic

Social IntegrationVocational Orientation

Use of TimeDaily Living Skills

MobilityUse of Environmental Ques

The DSS must help policy makers find a comfortable balance between acceptable risks and benefits.

Desired Data

InputIssues

ProcessIssues

OutputIssues

OutcomesIssues

General Public

Parents

Teachers

Support Staff

Admin/Boards

State Legislators

Others

Helps in Data Discovery

Standards moves from efficiency to effectiveness

One Last Time

District UsersUpload data formatsCorrect duplicate dataFERPA requests

Public Portal AccessPredefined Report CardsLimited queriesDashboard/Scorecard

Web Front End

Error reports

One Last Time

File ETL: • Student • Assessment • Finance • Professional

Deve

lope

r Ap

plica

tion

s

Error reports

Check Sum

Student IDsMatch & Merge

Audit (FERPA)

SecurityDistrict UsersUpload data formatsCorrect duplicate dataFERPA requests

Public Portal AccessPredefined Report CardsLimited queriesDashboard/Scorecard

Web Front End

Error reports

One Last Time

File ETL: • Student • Assessment • Finance • Professional

Deve

lope

r Ap

plica

tion

s

WAREHOUSE

Error reports

Check Sum

Student IDsMatch & Merge

Audit (FERPA)

SecurityDistrict UsersUpload data formatsCorrect duplicate dataFERPA requests

Public Portal AccessPredefined Report CardsLimited queriesDashboard/Scorecard

Web Front End

Error reports

School Codes

Meta Data

GPS

Reliable/Valid

One Last Time

File ETL: • Student • Assessment • Finance • Professional

Deve

lope

r Ap

plica

tion

s

WAREHOUSE

Error reports

Check Sum

Data Mart

Student IDsMatch & Merge

Audit (FERPA)

Security

DoE UsersGenerate Report CardFederal: EDEN, NCLB, IDEASkopusIssue Assessment IDs

District UsersUpload data formatsCorrect duplicate dataFERPA requests

Public Portal AccessPredefined Report CardsLimited queriesDashboard/Scorecard

Web Front End

Error reports

School Codes

Meta Data

GPS

Reliable/Valid

Current problem:data rich and information poor

Current problem:data rich and information poor

Department

Data Silos

Current problem:data rich and information poor

Department Educational Community

Gap:Lack of confidence No trust in systemHave a low ROI

Data Silos

Solution

Department Educational Community

Data Democracy

Secure ScalableFlexible

Data Warehouse

Student

Meta Data

Manual

Performance Meta Data Manual

Personnel

Meta Data Manual

School Meta Data Manual

Finance Meta Data Manual

Apply information and facilitate

decision-making

Solution

Department Educational Community

Data Democracy

Secure ScalableFlexible

Data Warehouse

Student

Meta Data

Manual

Performance Meta Data Manual

Personnel

Meta Data Manual

School Meta Data Manual

Finance Meta Data Manual

Apply information and facilitate

decision-making

Without Data

You’re Just Another Person With an Opinion

We find ourselves in an

Information Agewith an aging information

system

Decisionsbegin with good data

Most of the fun using the DSS is not finding the

answer to your question - it’s finding the new

questions you don’t have the answers to.

top related