welcome goals for dqi –establish future vision for perkins accountability –collaborate on...

Welcome•Goals for DQI

–Establish future vision for Perkins Accountability

–Collaborate on Standardization

–Establish concrete state recommended alternatives for standardizing measurement definitions and performance indicators

•Overview of Agenda and Materials

Post Data Quality Institute

•Convene forum for Skill Attainment (1S2) and conference calls for Non Trad measures

•Continue NSWG calls

•Develop report from DQI on State Recommendations

•Further explore recommendations at Regional Meetings

•Present alternatives at CTE State Directors meeting

•Establish a transition period for implementation

Accountability In Transition

Federal & State Panel

Federal PerspectiveCorinne Sauri, Policy Specialists, PRES

Perkins III

•Built on accountability provisions from the Perkins Act of 1990.

•Increased emphasis on academics and holding CTE students to same standards as non-CTE students

•Required new disaggregated reporting with system of rewards and consequences

Core Indicator Framework

•Designed by OVAE with input from states and other stakeholders.

•Not meant to be a comprehensive list of acceptable measures but as a tool for states in developing measures.

•Perkins requires four core measures for every state to report, but allows for states to develop and define performance measures.

•OVAE has limited authority under Perkins III to set measures but may provide guidance and must approve state plans.

Data Integrity?

•Data collection under Perkins III has left the program open to criticism and threat of elimination

–National Assessment of Vocational Education

–Office of Inspector General Report

–Program Assessment Rating Tool

–Budget Requests

NAVE

•“Despite serious commitment among state administrators, technical measurement and data quality problems hinder widespread use of program performance data for program management as the state or local levels.”

•The quality of Perkins performance reporting varies considerably by indicator, by state, and sometimes within a state

OIG Report

•Invalid Measures

–Many states use same indicator for different sub-indicators.

–Some states using same measure for multiple sub-indicators.

–OVAE can offer better guidance in the state plan approval process.

•57% of states do not provide complete performance data.

Lack of improvement strategies, reporting at sub-indicator level.

PART and Perkins Funding

•FY 2004: PART rating “Ineffective”

–Unclear program purpose.

–Quality of state data is deficient.

–Unavailable data on student outcomes.

–Lack of demonstrated state progress on core indicators.

•FY 2006 budget: Perkins funding eliminated.

–PART and NAVE cited.

Perkins Reauthorization

•Sec. Spellings letter on Senate Perkins bill–Cited PART results.–Requested authority to establish common measures to assess program performance for Perkins.–Requested authority to negotiate new specific measures and targets with each state.

Report to Congress

•Perkins III requires OVAE to report on States’ progress in achieving performance targets.

–Missing and incomplete data from states.–State-to-State comparison charts required.

•Impossible when concentrators are defined differently. We’re comparing apple to oranges to bananas to coconuts.

Conclusions

•Data-driven era of Accountability

•Need for standardization of data.

•Expectation of demonstrable results–Congress expects results as a return on investment.–Basic questions about data integrity cannot be addressed without common measures among states.



Federal PerspectiveSharon Miller, Director, DHSPCE

Vision for Perkins Accountability

To achieve greater consistency across the nation regarding definitions of:

Concentrator

Secondary academic attainment

Secondary completion

Secondary transition

Postsecondary completion

Postsecondary placement and retention


Concentrator

Currently about five different approaches to definitions

Needs to maximize the number of students counted without including those who take only one course

Needs to be distinguished from a completer


Secondary academic attainment

Currently about seven different approaches to definitions

Half of states aligning their measure to NCLB

Need to align both the test and the proficiency level


Secondary completion

Considerable commonality among definition

Need to include more students who concentrate (not just 12th graders)


Secondary and postsecondary placement

Roughly forty states use UI wage record to track students

Three states share information about students across state lines

Need to build capacity of all states to use administrative record systems


Results of greater consistency

Heightened ability to communicate our

outcomes to key constituentsMore support for our programs and servicesGreater ability to use data for program

improvement



Federal PerspectiveJohn Haigh, Chief, Accountability Branch, DHSPCE

History of Perkins III Accountability:

• CIF

• Rounds 1-5

• Institutes

• Evaluations

• NSWG

Perkins III: Overview


Define Key Terms: Measurement Approach (CRT)Codes (1S1,etc.)Measures (N & D)Concentrator, Completer, ParticipantBaselines Quality Indicators (scope, coverage, alignment, timing, reliability)


What is the CAR? What are negotiations? What is the process of awarding incentives & sanctions?


Roadmap for where we’ve been to date with activities:

Past Data Quality activities Past Program Quality activities NSWG discussions


Conference On Accountability & Core Performance MeasurementFebruary 4-5, 1999, Kansas City, Missouri

Reporting Results:Strategies To Improve Data QualityRegional Technical Assistance WorkshopJuly 25-27, 2000, Phoenix, Arizona

Reporting Results:Strategies To Improve Data QualityRegional Technical Assistance WorkshopAugust 14-15, 2000, Portland, Oregon

Introducing the Data Quality InitiativeData Quality UpdateNational Association of State Directors of Vocational Education ConsortiumDecember 6, 2000, San Diego, California


Data Quality Initiative National Institute: National ConferenceStrategies For Improving Data QualityJoint State-OVAE ConferenceFebruary 1-2, 2001, New Orleans, Louisiana

Improving Validity and Reliability: State Technical Assistance MeetingAmerican Vocational Information AssociationMay 13-15, 2001, Reno, Nevada

Train-the-Trainer Workshop: State WorkshopWorking With Local Educators To Collect Quality DataJoint state-federal conferenceAugust 23-24, 2001, Chicago, Illinois

National Career and Technical Education Leadership ConferenceNovember 27-29, 2001, Washington, DC


Program Quality Institute:Strategies For Improving Program QualityMay 13-14, 2002, Jacksonville, Florida Program Quality Institute:Strategies For Improving Program QualityAugust 8-9, 2002, Atlanta, Georgia National Career and Technical Education Leadership ConferenceMay 7-8, 2003, Washington, DC

welcome goals for dqi –establish future vision for perkins accountability –collaborate on...

Documents