Pilot Testing of Pilot Testing of NCATE’s Continuous NCATE’s Continuous Improvement: Fall Improvement: Fall 2011 Visits2011 Visits

Donna M. GollnickDonna M. GollnickSenior Vice President, Senior Vice President,

NCATENCATEAugust 19, 2009August 19, 2009

Microphone Checks will begin at 1:45 p.m.Microphone Checks will begin at 1:45 p.m.Web Seminar will begin at 2:00 (eastern Web Seminar will begin at 2:00 (eastern

time)time)

NCATE Pilot Visits in Spring 2011NCATE Pilot Visits in Spring 2011

1:45 - 2:001:45 - 2:00 Audio ChecksAudio Checks

2:00 – 2:052:00 – 2:05 Introductions and Introductions and Review of SoftwareReview of Software

2:05 – 2:152:05 – 2:15 Continuous Continuous Improvement FocusImprovement Focus

2:15 – 2:252:15 – 2:25 Institutional Report Institutional Report OptionsOptions

2:25 – 2:452:25 – 2:45 Continuous Continuous Improvement ProcessImprovement Process

2:45 – 3:252:45 – 3:25 ExhibitsExhibits

3:25 – 3:303:25 – 3:30 TimelineTimeline

Icon ReviewIcon Review

To participate in this web conference, use these To participate in this web conference, use these icons:icons:

Raise handRaise hand Yes, No Yes, No Mic Mic

Clapping Happy face Sad faceClapping Happy face Sad face

NCATE staff on NCATE staff on the web seminarthe web seminar

Donna M. Gollnick, Senior Vice Donna M. Gollnick, Senior Vice PresidentPresident

Mary Anne Kirkland, Assistant to DonnaMary Anne Kirkland, Assistant to Donna

Julien Goichot, WebmasterJulien Goichot, Webmaster

TwoTwo Options for Options for Continuing Visits:Continuing Visits:

1.1. Continuous Continuous ImprovementImprovement

2.2. Transformation InitiativeTransformation Initiative

Option 1: Option 1: Continuous Continuous

ImprovementImprovement

Focus of the Review: Focus of the Review: Continuous Continuous ImprovementImprovement The unit will engage in continuous The unit will engage in continuous

improvement between on-site visits.improvement between on-site visits.– Submit annual reportsSubmit annual reports– Describe changes in the IRDescribe changes in the IR– Submit IR at least one year before visit for Submit IR at least one year before visit for

feedbackfeedback

The unit will assess itself against the The unit will assess itself against the target level of one or more standards.target level of one or more standards.

The The Institutional Institutional Report (IR) Report (IR)

for for Continuous Continuous

ImprovementImprovement

Content of IR by Standard: Prompt #1 (“Big” question related to standard)

1. What do candidate assessment data tell the unit about candidates’ meeting standards?

2. How does the unit use its assessment system to improve the performance of candidates and the unit and its programs?

3. How does the unit work with school partners to deliver field experiences and clinical practice to enable candidates to develop the K,S,PDs to help all students learn?

“Big” question related to standard

4. How does the unit prepare candidates to work effectively with all students?

5. How does the unit ensure that its professional education faculty contributes to the preparation of effective educators?

6. How does the unit ensure that its governance system and resources are adequate to prepare candidates to meet professional, state, and institutional standards?

Content of IR by Standard: Prompt #2

Standard on which unit is moving toward to target Level (maximum of 5 pp)

Describe work undertaken to move to the target level Discuss plans for continuing to improve

OR, if this is not the standard

on which the unit is moving to the target level

Continuous Improvement (max of 3 pp) Summarize the most significant changes related to this

standard that have led to improvement related to the standard

Content of IR by Standard: Prompt #3

Exhibit links uploaded as Word document

Name of exhibit + Link to exhibit on the website

Tables in the IRTables in the IR

Standard 4Standard 4– Faculty diversityFaculty diversity– Candidate diversityCandidate diversity– P-12 student diversity in clinical practiceP-12 student diversity in clinical practice

Standard 5Standard 5– Summary of faculty qualifications & Summary of faculty qualifications &

involvement in scholarship & serviceinvolvement in scholarship & service

Options for Submission of IR

Online template with prompts in AIMS

Template with prompts written in Word & uploaded to AIMS

Holistic response to each element OR standard written in Word & uploaded to AIMS

ExhibitsExhibitsTo be available for To be available for the Offsite Reviewthe Offsite Review

Available to BOE Teams Available to BOE Teams

in AIMSin AIMS1. Program reports submitted for national review

2. National recognition reports

3. Reports from the previous NCATE visit

4. 3rd party testimony

5. Annual Reports

6. Other relevant national or state reports uploaded in AIMS

General Background General Background and Conceptual and Conceptual FrameworkFramework

1. Links to unit catalogs and other printed documents describing general education, specialty/content studies, and professional studies.

2. Syllabi for professional education courses

3. Conceptual framework(s)

4. Findings of other national accreditation associations related to the preparation of education professionals (e.g., ASHA, NASM, APA, CACREP)

Standard 1: Candidate Knowledge, Skills, and Professional Dispositions

1. State program review documents and state findings.[1] (Some of these documents may be available in AIMS.)

2. State licensure test scores aggregated by program area and reported over multiple years (Title II data reports submitted to the state for the last year must be available to the team.)

3. Data tables and summaries that show how teacher candidates (both initial and advanced) have performed on key assessments over the past three years for programs not included in the national program review process or a similar state process

4. Key assessments and scoring guides used by faculty to assess candidate learning against standards and the outcomes identified in the unit’s conceptual framework for programs not included in the national program review process or a similar state process

5. Samples of candidate work (e.g., portfolios at different proficiency levels)

6. Follow-up studies of graduates and data tables of results

7. Employer feedback on graduates and summaries of the results

8. List of candidate dispositions, including fairness and the belief that all students can learn, and related assessments, scoring guides, and data

Standard 2: Assessment System & Unit Evaluation

1. Description of the unit’s assessment system in detail including the requirements and key assessments used at transition points

2. Data from key assessments used at entry to programs

3. Procedures for ensuring that key assessments and unit operations are fair, accurate, consistent, and free of bias

4. Policies and procedures that ensure that data are regularly collected, compiled, aggregated, summarized, analyzed, and used to make improvements

5. Sample of candidate assessment data disaggregated by alternate route, off-campus, and distance learning programs

6. Policies for handling student complaints

7. File of student complaints and the unit’s response

8. Examples of changes made to courses, programs, and the unit in response to data gathered from the assessment system

Standard 3: Field Experiences & Clinical Practice

1. Memoranda of understanding, contracts, and/or other documents that demonstrate partnerships with schools

2. Criteria for the selection of school faculty (e.g., cooperating teachers, internship supervisors)

3. Documentation of the preparation of school faculty for their roles (e.g, orientation and other meetings)

4. Descriptions of field experiences and clinical practice in programs for initial and advanced teacher candidates and other school professionals

5. Guidelines for student teaching and internships

6. Assessments and scoring rubrics/criteria used in field experiences and clinical practice for initial and advanced teacher candidates and other school professionals (These assessments may be included in program review documents or the exhibits for Standard 1. Cross reference as appropriate.)

Standard 4: Diversity

1. Proficiencies related to diversity that candidates are expected to develop

2. Curriculum components that address diversity proficiencies (This might be a matrix that shows diversity components in required courses.)

3. Assessment instruments, scoring guides, and data related to diversity (These assessments may be included in program review documents or the exhibits for Standard 1. Cross reference as appropriate.)

4. Policies and practices for recruiting and retaining a diverse faculty

5. Policies and practices for recruiting and retaining diverse candidates

6. Policies, practices, and/or procedures that facilitate candidate experiences with students from diverse groups

Standard 5: Faculty Qualifications, Performance, and Development1. Licensure information on school faculty (e.g., cooperating teachers,

internship supervisors)[1]

2. Samples of faculty scholarly activities

3. Summary of service and collaborative activities engaged in by faculty with the professional community (e.g., grants, evaluations, task force participation, provision of professional development, offering courses, etc.)

4. Promotion and tenure policies and procedures

5. Samples of forms used in faculty evaluation

6. Opportunities for professional development activities provided by the unit

[1] The faculty chart in AIMS will also be used as evidence for this standard.

Standard 6: Unit Governance and Resources

1. Policies on governance and operations of the unit

2. Organizational chart or description of the unit governance structure

3. Unit policies on student services such as counseling and advising

4. Recruiting and admission policies for candidates

5. Academic calendars, catalogs, unit publications, grading policies, and unit advertising

6. Unit budget, with provisions for assessment, technology, and professional development

7. Budgets of comparable units with clinical components on campus or similar units at other campuses

8. Faculty workload policies

9. Summary of faculty workloads

10. List of facilities, including computer labs and curriculum resource centers

11. Description of library resources

12. Description of resources for distance learning, if applicable.

Presentation of Presentation of ExhibitsExhibits

1.1. Online Exhibit RoomOnline Exhibit Room

2.2. List of exhibits identified in Prompt List of exhibits identified in Prompt #3 of the IR provides links to key #3 of the IR provides links to key exhibits supporting each standardexhibits supporting each standard

***Please use NCATE’s list of ***Please use NCATE’s list of exhibits***exhibits***

The Offsite The Offsite Review 6-12 Review 6-12

months months before visitbefore visit

Offsite BOE Team

Team of 3-5 persons that meets electronically with state consultant & NCATE staff member

If state protocol calls for joint visit, state team member may join team

If possible, one of team members will be assigned to Onsite BOE Team

Offsite Review

Offsite Visit BOE Team reviews the following electronically & prepares feedback report for unit:

• Institutional report

• Annual reports

• Title II data

• Program reviews by SPAs or states

• Exhibits related to standards

Feedback Report by Offsite BOE Team

1. Statement about the evidence• Does the evidence make the case that

standards continue to be met?• What, if any, key evidence was not available

to the team?

2. Comments on unit’s progress toward meeting target level

3. List of areas of concern by standard

4. List of evidence for the Onsite BOE Team to validate during the onsite visit

Timeline: Offsite Review for Timeline: Offsite Review for CICI

12 months before visit Unit submits IR

1-3 months after the IR is submitted

Offsite BOE Team meets electronically to review IR & write BOE Feedback Report

Generally within 1-2 weeks after Offsite BOE Team meeting

BOE Feedback Report is available in AIMS

1-2 months before the onsite visit and before the previsit with the team chair, state team co-chair, & state consultant

Unit submits IR Addendum, which responds to Areas of Concern in the BOE Feedback Report

The Onsite The Onsite VisitVisit

Onsite Visit: Continuous Improvement

Smaller team will conduct shorter visit by

1. Focusing on areas of concern raised by Offsite Visit BOE Team

2. Validating that standards continue to be met

3. Providing feedback on progress toward the target level on one or more standards

Activities: SundayActivities: Sunday

Team members arrive Team members arrive onsite by 1 pmonsite by 1 pm

Team meetingTeam meeting

Orientation by Orientation by institution & stateinstitution & state

(Optional) Poster (Optional) Poster sessionssessions

Activities: MondayActivities: Monday

Visits to 2-4 schoolsVisits to 2-4 schools

Observations Observations

InterviewsInterviews

Sampling of Sampling of EvidenceEvidence

Team MeetingsTeam Meetings

Activities: TuesdayActivities: Tuesday

Follow-up interviews as neededFollow-up interviews as needed

Team meetingTeam meeting– Complete writing of BOE reportComplete writing of BOE report

Exit report to unitExit report to unit

Team leaves by 3 pmTeam leaves by 3 pm

Continuous Improvement Visit: Fall 2011 Pilot Visits

Unit submits programs for national reviewFeb 2010/ Sept 2010

National recognition reports are available in AIMS5 months after

program submissionMar-Jun

2011Unit calls for 3rd party testimony

Unit submits revised program reports for national review

Unit submits institutional report (IR)

Offsite BOE Team reviews IR electronically & provides feedback

Unit hosts previsit with BOE chair, state co-chair, & state consultant

Unit submits response to feedback report in AIMS

3-5 member BOE team conducts visit from Sunday to Tuesday afternoon

Sept-Dec 2011

after previsit

30-60 days prior to visit

Nov 2010-July 2011

Sept 2010-May

2011

First First Accreditation Accreditation

ProcessProcess

First AccreditationFirst Accreditation

Number of preconditions reducedNumber of preconditions reduced

Timeline flexibleTimeline flexible

IR to set baseline for futureIR to set baseline for future

Process same as continuous Process same as continuous improvementimprovement

IR: First AccreditationIR: First Accreditation

Unit must show how stands are being Unit must show how stands are being met to establish a baselinemet to establish a baseline

IR format options include:IR format options include:– Current online template with prompts for Current online template with prompts for

each element & 4 tableseach element & 4 tables– By standard in a Word document that is By standard in a Word document that is

uploaded in AIMSuploaded in AIMS

Exhibit list is same as for CI & TI Exhibit list is same as for CI & TI optionsoptions

Timeline: First VisitTimeline: First VisitDate Action

1-3 yrs. before visit Submission of preconditions, which are reviewed by NCATE’s ARPA Committee

1 yr. before visit IR submitted in AIMS

9-11 mos. before visit Offsite BOE Team reviews the IR & other evidence & writes BOE Feedback Report

1-2 mos. before visit Submission of IR addendum responding to Areas of Concern in BOE Feedback Report

1-2 mos. before visit Previsit to plan onsite visit with BOE team chair, state co-chair, & state consultant

Sunday to Tuesday Onsite Visit

Onsite BOE Team conducts visit & writes Onsite BOE Report

1-2 mos. after visit Submission of rejoinder to the Onsite BOE Report

Major Major Changes in Changes in

National National Program Program ReviewsReviews

National Program National Program ReviewReview

1.1. Programs have option to use current Programs have option to use current system of 6-8 assessments.system of 6-8 assessments.

2.2. Programs have option to select Programs have option to select assessments and manner in which to make assessments and manner in which to make case for their validity and sufficiency of case for their validity and sufficiency of evidence.evidence.

3.3. Programs have option of submitting only Programs have option of submitting only new or substantially changed assessments new or substantially changed assessments and new evidence for continuing and new evidence for continuing recognition.recognition.

National Program National Program ReviewReview

4.4. Program reports are submitted at Program reports are submitted at mid-cycle to provide a more mid-cycle to provide a more formative process.formative process.

5.5. Requirements for contextual Requirements for contextual information and years of data are information and years of data are reduced.reduced.

National Program National Program ReviewReview6.6. Simplified processes are being Simplified processes are being

developed for MAT-like secondary developed for MAT-like secondary and low-enrollment programs.and low-enrollment programs.

7.7. Task Force is developing common Task Force is developing common principles for SPA program principles for SPA program standards to provide greater standards to provide greater consistency across them.consistency across them.

Thanks for assisting NCATE Thanks for assisting NCATE in continuing to redesign in continuing to redesign

the accreditation system to the accreditation system to make it more efficient and make it more efficient and

valuable.valuable.

Stay in touchStay in touch

Call or email NCATE staff at anytime Call or email NCATE staff at anytime before, during, and after the visit.before, during, and after the visit.

– donna@ncate.org– patty@ncate.org– marva@ncate.org – team information – team information– maryanne@ncate.org– margie@ncate.org – program review – program review