BY

Karen Liu, Ph. D.

Indiana State University

August 18, 2011

1

NCATE SPA Review Process Definitions Ground Rules Response to Condition Report NCATE Course Grade Guidelines NCATE Course Grade Format Evidence for Meeting Standards EESE Assessments 1-7

2

Institutions submit program review documents to NCATE.

NCATE assigns each institution’s program report to 2-3 reviewers. (One institution may submit several program reports.)

Each reviewer will provide an independent review and submit the report to NCATE.

The lead reviewer will provide a team report. The audit committee will provide the audit

report. A staff member from NCATE will review the

audit report and provide a final review. The result of the final program review report will

be sent to each institution.

3

Revised ReportSubmitted when the previous decision required

further development or national recognition with probation

Response to Conditions ReportSubmitted when the previous decision was

national recognition with conditions

4

Program must submitCover SheetSection VI: For Revised Reports or Response to

Conditions Reports OnlyAll documentation to support what they’ve said

in Section VI Attachments in appropriate Section of program

report (usually in Section IV: Evidence for Meeting Standards)

5

Subsequent review teams cannot add any new criteria or expectations in subsequent recognition reportsMust reply solely on comments and information

in original recognition report

6

Part G: DecisionDate: The program is recognized through 08-01-

2013 (ISU EESE) The program has up to two opportunities to address

conditions within an 18 month period ISU EESE (9/15/11, 3/15/12, 9/15/12 or 3/15/13)

Addressing conditions (See the list on National Recognition Report)

Review commentsPart C: Evaluation of Program Report EvidencePart D: Evaluation of the Use of Assessment

ResultsPart E: Areas for Consideration

7

1. Make clear connections between ISU's assessments and standards rather than simply repeating the ACEI standards.

2. Provide evidence that candidates are able to use or apply the content embedded in Standards 2.5 to 2.7.

3. Provide explicit evidence for Assessment 2 that courses align with standards.

4. Provide more in-depth analysis of data.

See comments in Part C.

8

Part C: Evaluation of Program Report Evidence C.1: Candidates’ Knowledge of Content

The report indicates that Assessments 1 and 2 provide evidence of candidate content knowledge. However, more detailed alignment between assessments and standards is needed.

Assessment 2 requires course descriptions to gain a comprehensive perspective of the alignment between individual courses and the identified standards.

If grades in courses are used as an assessment, the program must follow NCATE guidelines for the construction of the assessment: http://www.ncate.org/Accreditation/ProgramReview/GuidelinesAndProcedures/DocumentingCourseGrades/tabid/456/Default.aspx

9

C.2. Candidates' ability to understand and apply pedagogical and professional content knowledge, skills, and dispositions

The report indicates that Assessments 3 and 4 meet standards in this area. However, more detailed alignment between assessments and standards is required, rather than simply repeating ACEI standards.

Activities or requirements within assessments must be set off in such a way that each task or requirement is specifically designed to measure competency on a single standard or standard element.

When the assessment and/or rubric is designed to evaluate performance across multiple standards, the resultant data cannot be used as evidence of successful candidate performance on individual standards.

10

C.3. Candidate effects on P-12 student learning Assessment 5 provides guidelines and rubrics for

the TWS to ensure that candidates have an impact on student learning. However, more detailed alignment between assessments and standards is required.

11

This program has provided examples of the use of data and how they have informed program changes, although many of the changes are still in the proposed phase. A few are listed below:

(1) proposed addition of phonics handbook or course to ensure candidates are adequately prepared to teach reading

(2) adding benchmarks to the Degree Audit Reporting System (DARS) to manage when candidates take Praxis II

(3) broaden the field placement options for students to provide a diverse body of pedagogical knowledge to candidates

12

Be sure to report data on program graduates; Maintain consistency with the "N" data that is

reported; Map standard alignment to assessment rather than

simply repeating the standard; Provide more explanation in of the alignment,

analysis, and interpretation of data and/or assessment;

Provide comprehensive evaluation and interpretation of how standards are met; and

Continue to review and refine the assessment system as needed.

13

Description of PRAXIS II Indiana licensure requirement. The content categories of the test.

Alignment with Standards Describe how PRAXIS assessment specifically aligns with

ACEI standards.

Analysis of Data Findings What have we done to improve candidates’ test scores?

Evidence for Meeting Standards (Data) NCATE asks the pass rate of 80%. (165 and higher). EESE uses “quartile” to report its data.

14

1. Courses must be required for all candidates in the program; elective courses may not be used as evidence.

2. Faculty may choose which courses will be used in this assessment.

3. The documentation of course grades-based evidence must include curriculum requirements, including the course numbers of required courses. (Documentation must be consistent with course listings provided in the Program of Study submitted in Section I of the program report.)

4. The grade evidence must be accompanied by the institution’s grade policy or definitions of grades.

5. Grade data must be disaggregated by program level (e.g. baccalaureate), grade level (e.g. middle grade and secondary), licensure category (e.g. history or social studies), and program site.

6. Syllabi cannot be submitted.

15

Format for Submission of Grades as a Course-Based Content Assessment

Part 1: Description of the Assessment Give a brief description of the courses and a rationale for

the selection of this particular set of courses. Provide a rationale for how these courses align with specific

SPA standards as well as an analysis of grade data included in the submission. (Limit to two pages).

Part 2: Alignment with SPA Standards Include a matrix that shows alignment of courses with

specific SPA standard (Reqiored).

16

Course Name & No. SPA Standard(s) Addressed by Course

Brief Description of How the Course Meets Cited Standards (if course title is unclear)

ELED 324: Emergent Literacy 2. 1, 3.1, 3.4

Part 3:  Grade Policy and Minimum Expectation Must submit grading policies that are used by the

institution or program and the minimum expectation for candidate grades (e.g., all candidates must achieve a C or better in all selected coursework)

Part 4: Sample Data Tables Data tables must provide, at minimum, the grade

distributions and mean course grades for candidates in the selected courses.

(See NCATE Course Grade documentation handout)

17

Assessments are organized into 3 areas to be aligned with ACEI standards. Content knowledge (Assessment 1 & 2) Pedagogical and professional knowledge, skills and dispositions

(Assessment 3 & 4) Focus on student learning (Assessment 5)

For each assessment, the compiler should prepare one document which includes the following items:a) A description of the assessment;b) A description of how this assessment specifically aligns with the

ACEI standard it is cited; c) A brief analysis of the data findings; and d) An interpretation of how that data provides evidence for meeting

standards o The assessment tool/instrument and/or the directions given

to candidates. o The scoring guide for the assessment.o The assessment data.

18

Focus on assessing candidates’ pedagogical and professional knowledge, skills and dispositions (Assessments 3 & 4).

Combine all four separate documents into one document and reorganize the content information presented in the document.

Map standard alignment to assessment rather than simply repeating the standard.

Provide detailed explanation to address how this assessment specifically aligns with the ACEI standard it is cited.

19

Focus on assessing candidates’ pedagogical and professional knowledge, skills and dispositions (Assessments 3 & 4).

Combine all three separate documents into one document and reorganize the content information presented in the documents.

Give a better description of the assessment. Provide detailed explanation to address how this

assessment specifically aligns with the ACEI standard it is cited.

Offer comprehensive evaluation and interpretation of how standards are met.

20

Focus on assessing student learning (Assessment 5). Combine all three separate documents into one

document and reorganize the content information presented in the documents.

Provide clear description of the Work Sample assignment (not just copy the information from the course syllabus).

Provide detailed explanation that addresses the alignment of data and/or assessment.

Give comprehensive analysis and interpretation of how standards are met.

21

Combine 2 separate documents into one. Provide better description of this assessment (May

include the objectives/goals). Follow the NCATE reporting format. (The

document should include: description, alignment with standards, analysis and interpretation).

Provide detailed explanation that addresses the alignment, analysis, and interpretation of data and/or assessment.

22

Follow the assessment report format: description, alignment with standards, analysis, and interpretation of data.

Make clear connections between assessment #7 and ACEI standards rather than simply repeating the ACEI standards.

Provide comprehensive analysis and interpretation of how standards are met base on the data presented.

23

24