common standard 2: unit and program assessment and evaluation

11

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing

Ensuring Educator Excellence

Common Standard Common Standard 2:2: Unit and Program Unit and Program

Assessment and EvaluationAssessment and Evaluation

October 1, 2008October 1, 200810:30 am-3:00 pm10:30 am-3:00 pm

22

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


Topics to be AddressedTopics to be Addressed

Revisions to the Common Standards Revisions to the Common Standards Edits to the Common Standards—in Edits to the Common Standards—in

process nowprocess now Common Standard 2Common Standard 2 Designing a Unit Evaluation and Designing a Unit Evaluation and

Program Assessment SystemProgram Assessment System

#1 Agenda, #2 PowerPoint

33

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


Revisions to Common Revisions to Common StandardsStandards

Commission adopted revised Commission adopted revised Common Standards in June 2007Common Standards in June 2007

Revised Common Standards Revised Common Standards implemented as of July 1, 2008implemented as of July 1, 2008

All approved institutions should All approved institutions should review and revise own response review and revise own response to Common Standards to Common Standards

44

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


Common Standards 2007 Common Standards 2007

1: Educational Leadership1: Educational Leadership2: Unit and Program Evaluation System2: Unit and Program Evaluation System3: Resources3: Resources4: Faculty4: Faculty5: Admissions5: Admissions6: Advice and Assistance6: Advice and Assistance7: Field Experiences and Clinical Practice7: Field Experiences and Clinical Practice8: Program Sponsor, District and 8: Program Sponsor, District and

UniversityUniversity Field Experience SupervisorsField Experience Supervisors9: Assessment of Candidate Competence9: Assessment of Candidate Competence

55

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


Compared to Prior Compared to Prior StandardsStandards

Evaluation System utilizing data (CS 2)Evaluation System utilizing data (CS 2)

Assess candidate competence (CS 9)Assess candidate competence (CS 9)

Focus on K-12 Content Standards (CS 1, Focus on K-12 Content Standards (CS 1, 3, 4, 7, 9)3, 4, 7, 9)

Focus on learning for all students (CS 7, Focus on learning for all students (CS 7, 9)9)

Focus on diversity and California’s Focus on diversity and California’s students (CS 5, 7, 9)students (CS 5, 7, 9)

#3 Alignment Matrix 98-07

66

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


Proposed editsProposed edits Concerns about the applicability of these Concerns about the applicability of these

standards to all types of credential standards to all types of credential programs and all types of program programs and all types of program sponsors—so proposed editssponsors—so proposed edits

No change to the content—except in No change to the content—except in Common Standard 8Common Standard 8

Broadens language to address all Broadens language to address all programs and types of sponsorsprograms and types of sponsors– Tier II, Level II, or Induction ProgramsTier II, Level II, or Induction Programs– Colleges, universities, school districts, county Colleges, universities, school districts, county

offices of education and other entitiesoffices of education and other entities

77

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


Timeline for editsTimeline for edits

Information item presented to the Information item presented to the Commission on August 8Commission on August 8

Stakeholder feedback collected Stakeholder feedback collected until September 25until September 25

COA will review feedback at the COA will review feedback at the October 10, 2008 meetingOctober 10, 2008 meeting

Edits return to the Commission Edits return to the Commission on November 6, 2008 for actionon November 6, 2008 for action

Glossary!Glossary!

88

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


StandardStandard 2: Unit and Program 2: Unit and Program Assessment and EvaluationAssessment and Evaluation

The education unit implements an The education unit implements an assessment and evaluation system for assessment and evaluation system for ongoing program and unit evaluation and ongoing program and unit evaluation and improvement. The system collects, analyzes improvement. The system collects, analyzes and utilizes data on candidate and program and utilizes data on candidate and program completer performance and unit operations. completer performance and unit operations. Assessment in all programs includes ongoing Assessment in all programs includes ongoing and comprehensive data collection related and comprehensive data collection related to candidate qualifications, proficiencies, to candidate qualifications, proficiencies, and competence, as well as program and competence, as well as program effectiveness, and are used for improvement effectiveness, and are used for improvement purposespurposes. .

99

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


Sentence by sentenceSentence by sentence

The education unit The education unit implements an assessment implements an assessment and evaluation system for and evaluation system for ongoing program and unit ongoing program and unit evaluation and evaluation and improvement.improvement.

1010

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


Sentence by sentence Sentence by sentence (continued)(continued)

The system collects, The system collects, analyzes and utilizes data analyzes and utilizes data on candidate and program on candidate and program completer performance and completer performance and unit operations.unit operations.

1111

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


Assessment in all programs Assessment in all programs includes ongoing and includes ongoing and comprehensive data collection comprehensive data collection related to candidate related to candidate qualifications, proficiencies, and qualifications, proficiencies, and competence, as well as program competence, as well as program effectiveness, and are used for effectiveness, and are used for improvement purposes.improvement purposes.

Sentence by sentenceSentence by sentence (continued)(continued)

1212

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


CTEL CTEL PrograProgra

mm

Unit Evaluation SystemUnit Evaluation System

Admin Admin ServicesServices ProgramProgram

Multiple Subject Program

1313

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


Program Evaluation Program Evaluation SystemSystem

Design the Evaluation Design the Evaluation SystemSystem

Collect DataCollect DataAnalyze the DataAnalyze the DataUtilize the Analysis for Utilize the Analysis for

Program ImprovementsProgram Improvements

1414

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


Design the SystemDesign the System Who will be assessed?Who will be assessed? What will be evaluated?What will be evaluated? What do we want to know about the What do we want to know about the

candidates and the program (s)?candidates and the program (s)? Who will provide data on the program?Who will provide data on the program? When will the data be collected?When will the data be collected? How will data be collected and stored?How will data be collected and stored? How will data be analyzed and reported?How will data be analyzed and reported?

1515

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


Assessment of Candidate Assessment of Candidate CompetenceCompetence

Candidates preparing to serve as Candidates preparing to serve as teachers and other professional school teachers and other professional school personnel know and demonstrate the personnel know and demonstrate the professional knowledge and skills professional knowledge and skills necessary to educate and support necessary to educate and support effectively all students in meeting the effectively all students in meeting the state-adopted academic standards. state-adopted academic standards. Assessments indicate that candidates Assessments indicate that candidates meet the Commission-adopted meet the Commission-adopted competency requirements, as specified competency requirements, as specified in the program standards.in the program standards.

#4 Concepts in Common Standards

1616

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


Who will be assessed?Who will be assessed? (CS 9)(CS 9)

Candidates’ competence in areas Candidates’ competence in areas appropriate for the credentialappropriate for the credential– Performance on the TPA Performance on the TPA – Performance on class projectsPerformance on class projects– Performance in student Performance in student

teaching/clinical experiencesteaching/clinical experiences Completer’s competenceCompleter’s competence

– Assessment by supervisorAssessment by supervisor

1717

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


What do we want to know What do we want to know about candidates and about candidates and

programs?programs?

About candidates? Common Standard About candidates? Common Standard 9 and the Candidate Competence 9 and the Candidate Competence Standards in the Program Standards Standards in the Program Standards

About programs? Common Standards About programs? Common Standards 1 (leadership), 3 (resources), 4 1 (leadership), 3 (resources), 4 (faculty), etc.(faculty), etc.

1818

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


What will be evaluated?What will be evaluated?

– Admission (CS 5)Admission (CS 5)– Program DesignProgram Design– CoursesCourses– Field Work (CS 7, 8)Field Work (CS 7, 8)– Advising (CS 6)Advising (CS 6)– Faculty (CS 4)Faculty (CS 4)– Recommendation for credentialsRecommendation for credentials– AssessmentAssessment

1919

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


Who has the information Who has the information we want?we want?

– CandidatesCandidates– Program completersProgram completers– Faculty/instructorsFaculty/instructors– Field supervisors—program Field supervisors—program

and districtand district– EmployersEmployers– Advisory BoardsAdvisory Boards

2020

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


When to Collect Data?When to Collect Data?

Institution and/or Program must Institution and/or Program must decide:decide:

AnnuallyAnnually Each TermEach Term With each cohortWith each cohort Beginning, middle and/or end of Beginning, middle and/or end of

the programthe program

2121

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


How to collect data?How to collect data?

Overall question – What kind of Overall question – What kind of data do we want? What data data do we want? What data collection instrument would be collection instrument would be most effective in collecting the most effective in collecting the data? data? –SurveysSurveys–AssignmentsAssignments–EvaluationsEvaluations–Focus groupsFocus groups

2222

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


Data CollectionData Collection

What kind of data will each What kind of data will each instrument provide?instrument provide?–Open-ended (qualitative)Open-ended (qualitative)–Multiple choice, scaled scores Multiple choice, scaled scores (quantitative)(quantitative)–A combination of both?A combination of both?

2323

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


Data StorageData Storage Collection procedures will impact Collection procedures will impact

storage and analytical proceduresstorage and analytical procedures If you use electronic data If you use electronic data

collection instruments, the results collection instruments, the results can be easily stored.can be easily stored.

Other collection procedures will Other collection procedures will require hand-entry.require hand-entry.

Who will enter the data?Who will enter the data? Who will have access to the data?Who will have access to the data?

2424

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


Analyze the DataAnalyze the Data Who has the skills to analyze the Who has the skills to analyze the

data, consider the questions?data, consider the questions? What tools will be used?What tools will be used? How often will data be analyzed?How often will data be analyzed? How do you want the data How do you want the data

presented? By cohort? By presented? By cohort? By program? By geographic location? program? By geographic location? All of the above? Other?All of the above? Other?

2525

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


Triangulating the DataTriangulating the Data Data from Data from

multiple sources multiple sources helps locate helps locate program program strengths and strengths and areas for growth: areas for growth: candidate candidate competence data, competence data, completer and completer and employer surveys, employer surveys, and faculty and faculty surveyssurveys

Course Evaluations

Employer surveys

Faculty feedback

TPAs

Focus Groups Key

Assessments

Program self-study

MSProgram

2626

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


Utilize the Analysis for Utilize the Analysis for ImprovementImprovement

How will results from each of the How will results from each of the analyses be shared? As results of analyses be shared? As results of course evaluations? In a meeting course evaluations? In a meeting or a written report?or a written report?

With whom will the results be With whom will the results be shared?shared?

When?When?

2727

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


Utilize the Analysis for Utilize the Analysis for Improvement Improvement (continued)(continued)

Who will ensure that changes Who will ensure that changes identified by the analyses are identified by the analyses are undertaken? How will decisions be undertaken? How will decisions be made if the data indicates that made if the data indicates that changes should be made?—Will changes should be made?—Will each professor decide? Does the each professor decide? Does the faculty as a whole? Is it an faculty as a whole? Is it an administrative decision?administrative decision?

How will changes be noted for How will changes be noted for institutional history and institutional history and accreditation activities?accreditation activities?

2828

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing



Unit evaluation system is Unit evaluation system is composed of each composed of each program’s evaluation program’s evaluation process…process…

When, what, who, how…When, what, who, how…

2929

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing



Multiple Subject Program

Admin Admin Services Services ProgramProgram

CTEL CTEL PrograProgra

mm

#5 Institution Evaluation Plan

3030

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


Revised Common Standards Revised Common Standards UseUse

GreenGreen cohort institutions may cohort institutions may respond to either the 2007 respond to either the 2007 Common Standards or the edited Common Standards or the edited 2008 standards.2008 standards.

All other institutions will respond All other institutions will respond to the edited to the edited 2008 Common 2008 Common StandardsStandards for both Site Visits and for both Site Visits and Program Approval.Program Approval.

3131

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


Responding to a Common Responding to a Common Standard Standard

Break the Standard into its component parts

The language of the standard defines WHAT the institution must do

The institution must describe HOW it is going to meet the what of the standard

3232

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


Common Standards in Common Standards in Accreditation ActivitiesAccreditation Activities

Responses to the Common Standards Responses to the Common Standards are prepared for the Site Visitare prepared for the Site Visit– Reviewers confirm the responses with Reviewers confirm the responses with

interviews and other evidence on siteinterviews and other evidence on site Also used in initial Program Approval Also used in initial Program Approval

process (CTEL, Ed Specialist, etc.)process (CTEL, Ed Specialist, etc.)– Reviewers confirm with evidence provided Reviewers confirm with evidence provided

in the documentin the document

3333

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


EvidenceEvidence

What kinds of evidence will What kinds of evidence will support the claims made in the support the claims made in the response to Common Standard 2?response to Common Standard 2?

What additional evidence from What additional evidence from programs might be needed?programs might be needed?

Differences between evidence for Differences between evidence for Site Visit and Program ApprovalSite Visit and Program Approval

3434

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


ResourcesResources

PSD NewsPSD News To subscribe, send an email to To subscribe, send an email to [email protected]@lists.ctc.ca.gov

Commission Web PageCommission Web Page

www.ctc.ca.govwww.ctc.ca.gov selectselect Program Program Sponsors Sponsors thenthen Accreditation Accreditation

3535

Co

mm

issi

on

on

Tea

cher

Cre

den

tial

ing


ResourcesResources

Biennial ReportsBiennial Reports

http://www.ctc.ca.gov/educator-prep/program-achttp://www.ctc.ca.gov/educator-prep/program-accred-biennial-reports.htmlcred-biennial-reports.html

Program AssessmentProgram Assessment

http://www.ctc.ca.gov/educator-prep/program-achttp://www.ctc.ca.gov/educator-prep/program-accred-assessment.htmlcred-assessment.html

Site VisitsSite Visits

http://www.ctc.ca.gov/educator-prep/program-achttp://www.ctc.ca.gov/educator-prep/program-accred-site-visits.htmlcred-site-visits.html

common standard 2: unit and program assessment and evaluation

Documents