Preparing IRA/CAEPPreparing IRA/CAEPProgram Reports to Earn Program Reports to Earn National RecognitionNational Recognition

Dr. Diane Kern, IRA SPA CoordinatorDr. Diane Kern, IRA SPA Coordinator

CAEP ConferenceCAEP Conference

Fall, 2014Fall, 20141

AgendaAgenda

IntroductionsIntroductions

Overview of program report and Overview of program report and IRA 2010 IRA 2010 Standards for Reading ProfessionalsStandards for Reading Professionals

Requirements of the assessment systemRequirements of the assessment system

IRA specific guidelinesIRA specific guidelines

Literacy coaching tips and coaching continuumLiteracy coaching tips and coaching continuum

Sample report sectionsSample report sections

Resources, Contacts and Q & AResources, Contacts and Q & A

2

Overview of program reportOverview of program report

Institutions must provide evidence of meeting IRA Institutions must provide evidence of meeting IRA standards based on 6-8 assessments (recommendation: 8 standards based on 6-8 assessments (recommendation: 8 assessments).assessments).

Assessments and rubrics must align with the IRA 2010 Assessments and rubrics must align with the IRA 2010 Standards for all three roles of the reading Standards for all three roles of the reading specialist/literacy coach:specialist/literacy coach: Interventionist/reading specialistInterventionist/reading specialist Literacy coachLiteracy coach Reading program supervisor/leaderReading program supervisor/leader

Assessment system must build a case that all standards Assessment system must build a case that all standards are met through utilizing the elements and potential are met through utilizing the elements and potential sources of evidence = “preponderance of evidence”.sources of evidence = “preponderance of evidence”.

Program level assessments must be administered to all Program level assessments must be administered to all candidates and should span across the program.candidates and should span across the program.

3

FormatFormat: What Option?: What Option?Option Description/Standards

Option A NCATE Report Format: 6-8 Assessments, Five Designated Assessments: Content (2) including State Licensure Data (if required); Professional and Pedagogical Knowledge (2); P-12 Student Learning (1) Additional Assessments (1-3) /

Option B Choose Your Own Assessments: Up to 8 Assessments including 1) State Licensure Data (if required); 2) Impact on Student Learning; 3) Additional Content, Pedagogical Knowledge Assessments

4

IRA 2010 Standards for Reading IRA 2010 Standards for Reading ProfessionalsProfessionals

1.1. Foundational knowledgeFoundational knowledge

2.2. Curriculum and instructionCurriculum and instruction

3.3. Assessment and evaluationAssessment and evaluation

4.4. DiversityDiversity

5.5. Literate environmentLiterate environment

6.6. Professional learning and leadershipProfessional learning and leadership

5

IRA specific guidelinesIRA specific guidelines

Program must incorporate an equivalent of 21-27 Program must incorporate an equivalent of 21-27 graduate semester hours in reading, literacy, graduate semester hours in reading, literacy, language arts, and related courses.language arts, and related courses.

Coursework must include a supervised practicum Coursework must include a supervised practicum experience – typically the equivalent of 6 semester experience – typically the equivalent of 6 semester hours.hours.

Supervision may include observations, conferences, Supervision may include observations, conferences, audiotape or videotape review of lessons, etc.audiotape or videotape review of lessons, etc.

Academic and practicum experiences must develop Academic and practicum experiences must develop candidates for all three roles: interventionist, candidates for all three roles: interventionist, literacy coach, and literacy program leader literacy coach, and literacy program leader

6

Literacy Coaching TipsLiteracy Coaching Tips

Coaching initiatives should include Coaching initiatives should include experiences in the program (such as the experiences in the program (such as the reading clinic) AND also opportunities to reading clinic) AND also opportunities to support teachers and/or paraprofessionals support teachers and/or paraprofessionals in authentic school settings.in authentic school settings.

Institutions should implement a Institutions should implement a developmentally appropriate continuum developmentally appropriate continuum of coaching experiences across the of coaching experiences across the program. program.

7

Coaching Activity: 3 Levels of Coaching Activity: 3 Levels of Intensity Intensity

(Bean, R.M., 2004)(Bean, R.M., 2004)

8

Sections of the Program ReportSections of the Program Report

Cover SheetCover Sheet

Section I: ContextSection I: Context

Section II: List of assessments (chart)Section II: List of assessments (chart)

Section III: Relationship of assessments to standards Section III: Relationship of assessments to standards (matrix)(matrix)

Section IV: Evidence for meeting standards (attached files Section IV: Evidence for meeting standards (attached files using CAEP required format)using CAEP required format)

Section V: Use of assessment results to improve program Section V: Use of assessment results to improve program (narrative)(narrative)

Section VI: For revised reports or response to conditions Section VI: For revised reports or response to conditions reports onlyreports only

9

Section I: ContextSection I: Context

Narrative or attachments, including the following:Narrative or attachments, including the following:

Any state or institutional policies that influence your Any state or institutional policies that influence your program?program?

Describe supervised clinical/practicum and other Describe supervised clinical/practicum and other experiences experiences

Attach program of studyAttach program of study

Table with three years of data on candidates Table with three years of data on candidates enrolled in the program; separate if multiple sitesenrolled in the program; separate if multiple sites

Faculty informationFaculty information

10

Section II: List of AssessmentsSection II: List of Assessments

11

Section III: MatrixSection III: Matrix

12

Section IV: Sample Assessments Section IV: Sample Assessments #1 & #2#1 & #2

Content Knowledge Content Knowledge

Licensure exams or professional examinations of content Licensure exams or professional examinations of content knowledge (must be #1 if required by state, institutions must knowledge (must be #1 if required by state, institutions must report results)report results)

Comprehensive ExaminationsComprehensive Examinations

Research ReportsResearch Reports

Child Studies/Action ResearchChild Studies/Action Research

Portfolio Tasks (that all candidates perform) Portfolio Tasks (that all candidates perform)

Evidence Base for Professional Development InitiativesEvidence Base for Professional Development Initiatives

13

Sample Assessment #3Sample Assessment #3Ability to plan instructionAbility to plan instruction

Developing lesson plans and units of studyDeveloping lesson plans and units of study

Selecting, using, and evaluating assessment toolsSelecting, using, and evaluating assessment tools

Selecting, using, and evaluating curriculum, instructional techniques, and appropriate Selecting, using, and evaluating curriculum, instructional techniques, and appropriate materialsmaterials

Establishing and evaluating literate environmentEstablishing and evaluating literate environment

Designing and implementing instructionDesigning and implementing instruction

Portfolio tasksPortfolio tasks

Literacy coaching components utilizing assessments aboveLiteracy coaching components utilizing assessments above

14

Sample Assessment #4Sample Assessment #4Assessment of internship, Assessment of internship,

practicum, clinical experiencepracticum, clinical experience

Reflective practitioner observation protocolReflective practitioner observation protocol

Coaching cycle observation, reflection and Coaching cycle observation, reflection and evaluation by supervisorevaluation by supervisor

Final school report, parent conferenceFinal school report, parent conference

Analysis of video segment with peer review, Analysis of video segment with peer review, coaching and supervisor feedbackcoaching and supervisor feedback

Practicum portfolioPracticum portfolio

15

Sample Assessment #5Sample Assessment #5Impact on Student LearningImpact on Student Learning

Case studiesCase studies

Action research with a pretest-posttest designAction research with a pretest-posttest design

Analyzing student work samplesAnalyzing student work samples

Designing and implementing instruction and/or interventionDesigning and implementing instruction and/or intervention

Portfolio tasksPortfolio tasks

Literacy coaching and leadership experiences utilizing assessments aboveLiteracy coaching and leadership experiences utilizing assessments above

16

Sample Assessments #6, 7 & 8Sample Assessments #6, 7 & 8Additional AssessmentsAdditional Assessments

Eight assessments are strongly recommended, although only six are requiredEight assessments are strongly recommended, although only six are required

Diversity, technology, creating literate environment, coaching and leadership experiences are typical Diversity, technology, creating literate environment, coaching and leadership experiences are typical areas to further developareas to further developDiagnostic portfolio for diverse learners at the elementary and secondary levelsDiagnostic portfolio for diverse learners at the elementary and secondary levelsThree levels of literacy coaching in the clinic and in school settingsThree levels of literacy coaching in the clinic and in school settingsData reports for whole school reading program and Report NightData reports for whole school reading program and Report NightSupporting teachers to select and evaluate materials and instructional methods to meetSupporting teachers to select and evaluate materials and instructional methods to meetProfessional development series, book study leadershipProfessional development series, book study leadership

17

Section IV: Evidence for Section IV: Evidence for Meeting StandardsMeeting Standards

Two-page narrative of each assessment includes:Two-page narrative of each assessment includes:a)a)Brief description of assessment and its use in the Brief description of assessment and its use in the programprogramb)b)Description of how assessment aligns with IRA Description of how assessment aligns with IRA standardsstandardsc)c) Brief analysis of data findingsBrief analysis of data findingsd)d)Interpretation of how data provide evidence for Interpretation of how data provide evidence for meeting specific standards by IRA standard meeting specific standards by IRA standard number, title, and/or wording.number, title, and/or wording.

2) One attachment for each assessment, which 2) One attachment for each assessment, which includes the assessment tool itself (the includes the assessment tool itself (the detailed directions given to candidates), detailed directions given to candidates), scoring guide/rubric, and data table scoring guide/rubric, and data table disaggregated by TWO applications (and if disaggregated by TWO applications (and if multiple sites, by each site)multiple sites, by each site)

18

Section V: Assessments/data to Section V: Assessments/data to inform program improvementinform program improvement

Narrative with no more than 12,000 charactersNarrative with no more than 12,000 characters

Summarize principal findings from the Summarize principal findings from the assessment results, faculty interpretation of assessment results, faculty interpretation of data, and how candidate performance informs data, and how candidate performance informs plans to strengthen the programplans to strengthen the program

Organized by three headings:Organized by three headings: Content knowledgeContent knowledge Professional and pedagogical knowledge, skills Professional and pedagogical knowledge, skills

and dispositionsand dispositions Impact on P-12 student learningImpact on P-12 student learning

19

Section VI: For Revised programs Section VI: For Revised programs onlyonly

Response to conditions narrative—Organized by Response to conditions narrative—Organized by the conditions—what changes did you make? the conditions—what changes did you make? Only one application of data requiredOnly one application of data required

Revised reports narrative: Describe changes or Revised reports narrative: Describe changes or additions have been made to address the IRA additions have been made to address the IRA StandardsStandards

TIP—TIP—Even if IRA did not specifically ask for Even if IRA did not specifically ask for something, you can include any (all!) sections of something, you can include any (all!) sections of the program report to make your case to earn the program report to make your case to earn national recognition. national recognition.

20

ResourcesResources

Fall IRA/CAEP Workshops and Individual 30 Fall IRA/CAEP Workshops and Individual 30 min. Consultation at Association of Literacy min. Consultation at Association of Literacy Educators and Researchers (ALER) annual Educators and Researchers (ALER) annual meeting (Sat., Nov. 1, 2014—Delray Beach, FL)meeting (Sat., Nov. 1, 2014—Delray Beach, FL)

Spring/Summer IRA/CAEP Workshops and Spring/Summer IRA/CAEP Workshops and Individual30 min. Consultation at International Individual30 min. Consultation at International Reading Association (IRA) annual meeting (Fri., Reading Association (IRA) annual meeting (Fri., July 17, 2015—St. Louis, MO)July 17, 2015—St. Louis, MO)

IRA Research Division, Accreditation websiteIRA Research Division, Accreditation website

http://www.reading.org/Resources/http://www.reading.org/Resources/ProfessionalDevelopment/Accreditation.aspxProfessionalDevelopment/Accreditation.aspx

21

Resources, cont.Resources, cont.

CAEP (NCATE) SPA LibraryCAEP (NCATE) SPA Library

http://www.ncate.org/Accreditation/http://www.ncate.org/Accreditation/ProgramReview/ProgramReviewResources/ProgramReview/ProgramReviewResources/SPAAssessmentLibrary/tabid/460/Default.aspxSPAAssessmentLibrary/tabid/460/Default.aspx

IRA CAEP Guidelines for Program Writers and IRA CAEP Guidelines for Program Writers and ReviewersReviewers

http://www.reading.org/Libraries/resources/ira-http://www.reading.org/Libraries/resources/ira-guidelines-for-program-writers-and-reviewers.pdfguidelines-for-program-writers-and-reviewers.pdf

Consultation with member of IRA Audit Team Consultation with member of IRA Audit Team when much support is needed; contracted service when much support is needed; contracted service with IRA, either on-site or email/telephonewith IRA, either on-site or email/telephone

22

Q & AQ & AContactsContacts

IRA and CAEP want your program to align to IRA Standards, to prepare 21IRA and CAEP want your program to align to IRA Standards, to prepare 21stst century century reading specialists/literacy coaches, and to be Nationally Recognized!reading specialists/literacy coaches, and to be Nationally Recognized!

Dr. Diane Kern, IRA SPA Coordinator: dkern@reading.orgDr. Diane Kern, IRA SPA Coordinator: dkern@reading.org

Elizabeth Vilky, CAEP, Director of Program Review Elizabeth Vilky, CAEP, Director of Program Review elizabeth.vilky@caepnet.orgelizabeth.vilky@caepnet.org

Dr. Stevie Chepko, CAEP, Senior Vice President, Accreditation Dr. Stevie Chepko, CAEP, Senior Vice President, Accreditation stevie.chepko@caepnet.orgstevie.chepko@caepnet.org

CAEP telephone: 202-223-0077CAEP telephone: 202-223-0077

23