Unit Assessment System

Teacher Education at Purdue

March 4, 2012

What is our assessment philosophy ?

CONTINUOUS IMPROVEMENT

3

All components of teacher education at Purdue engage in continuous improvement by gathering and analyzing relevant data and making decisions based on data.

BALANCE

4

The unit balances central coordination of the overall assessment system with encouraging individual offices and programs to develop flexible, creative, assessment system components that fit their mission and foster continuous improvement.

What have we done since the last visit to improve our UAS?

6

THE FIRST THREE YEARS• CI Focus: Initial licensure program candidate assessment

system (AFI = clear criteria for performance)– Reviewed the national literature and the Indiana evaluation

system for beginning teachers to identify critical instructional skills

– Created the Gate C & D rubrics to measure those skills

• Gate C – Instructional planning skills (usually assessed in methods courses)

• Gate D – Instructional skills and impact on student learning (usually assessed in student teaching)

– Worked with program faculty to align program key assessments

7

THE GATED RUBRIC• FOUR SECTIONS PLUS A HOLISTIC SUMMATIVE ASSESSMENT

I. PLANNING INSTRUCTION (also used for Gate C)II. TEACHING AND LEARNING EXPERIENCESIII. ASSESSING LEARNERSIV. REFLECTING ON LEARNING AND TEACHING

• EACH SECTION HAS…– 3-5 Unit items– Encouragement to programs to add discipline-specific programs

• ALL ITEMS USE A FOUR POINT SCALE FOR PERFORMANCE RATINGS– Clear descriptions of target behaviors in the cells

• PROGRAMS ALIGN TO THE UNIT RUBRIC IN VARIOUS WAYS– English Education– Engineering/Technology Education

8

THE PAST FOUR YEARS• CI Focus: The operational assessment/decision system (AFI =

specification of decision system)

– Created a UAS annual report template based on NCATE standards and AFI’s (Annual Assessment Summaries)

– Began implementing TEC change decisions based on the summary reports of data

• CI Focus: Advanced programs for teachers (AFI = Clear criteria for performance)

– Reviewed all graduate programs at Purdue to determine which ones aspired to be advanced TE programs

– TEC created an advanced program task force to design curricular and assessment structure for advanced programs for teachers

– Currently, advanced programs for teacher are redesigned their programs to the new unit framework.

9

CURRENT• CI Focus: Program Assessment Systems

– Submitted our first SPA program reports in 20 years in March 2011• Results of the national reviews were comparable to national averages• Most concerns related to the assessment systems, not the curriculum or

outcomes of the program

– Submitted our first State program reports in 20 years in December 2011 (one of the first universities in the state to be reviewed under the new REPA standards with the new program template)

• Results were mixed, as with the national reviews• Again, most concerns related to the assessment systems

– In Fall 2011, the TEC established Standard 2 at the program level as our continuous improvement focus for 2011-12

• Assessment Coordinator met individually with all TE Program Conveners to discuss individualized ways to improve each program assessment system

• All program faculty are currently working to improve their assessment systems

What are the features of our current unit assessment system?

11

OPERATIONAL DECISION SYSTEM• Collaborate Management and Governance

– PCC: Informal, networking and support– TEC: Formal governance and unit-level decisions

• Multiple Sources and Types of Data– OnePurdue student systems– Cooperating teacher, candidate, and employer exit surveys– Performance-based assessments of instructional skills

• Componential– Office management– Program management

OPERATIONAL DECISION SYSTEM

12

Purdue University Teacher Education Operational Assessment/Decision System Data Sources Office & Program Management Analysis Decision Makers

Registrar

Praxis (ETS)

Applicants

Program Completers

Candidate Information

Employers

Cooperating Teachers

Cooperating School Administrators

Instructor Data

Graduate Office

Office of Professional Preparation & Licensure

Office of Field Experiences

Business Office

Program Conveners

Office of Advising & Recruiting

Purdue Student Education Council

DATA

CO

LLEC

TIO

N &

AN

ALYS

IS

Office of Strategic

Assessment

UAS Annual Report

Teacher Education

Administration (ADLE,

NCATE Coordinator)

Teacher Education

Council

Dean of College of Education

DATA

CO

LLEC

TIO

N

COE Leadership Team

Program Convener

Council

Dean’s Advisory Council

Data Types Decision support data gathered include: stakeholder perceptions, candidate performances, demographic information, operational process information, etc. Data Gathering Methods Methods used to collect and store this data include: surveys, course evaluation forms, portfolio assessments, operational performance metrics, grades, etc. Data Storage This information is stored in office and departmental data repositories, both electronic and paper.

Key Reporting and information sharing Information/Consultation upon request Post-decision feedback and influence ETS: Educational Testing Services ADLE: Associate Dean for Learning & Engagement This chart depicts the way data is used to improve the Purdue Teacher Education Program.

Decisions are made and enacted by entities in green.

DATA SOURCES DECISION MAKERS ANALYSIS OFFICE & PROGRAM MANAGEMENT

CANDIDATE ASSESSMENT SYSTEM

13

Unit and Program LevelsUnit Level includes the conceptual framework and related standards, systems for assessing and managing critical transitions, and aggregated performance data from unit wide rubrics

Program Level (a) applicable program standards; (b) program assessment systems based on unit guidelines for transitions and key assessments; (c) national or state program review

17

CANDIDATE ASSESSMENT SYSTEMOnePurdue Transition to Banner

– Facilitated aggregated and disaggregated grade reports for content assessment in secondary programs

– Facilitated some aspects of Gate audits– DegreeWorks should further improve the integration of the

candidate assessment system into the university data system

Adoption of TaskStream– Old Purdue performance-based assessment system was

locally developed using a P3T3 grant– After last visit, we adopted TaskStream, an outsourced solution– We are currently researching alternative technologies to find a

better system for our needs

Questions?