the program review process: ncate and the state of indiana richard frisbie and t. j. oakes march 8,...
Post on 19-Dec-2015
213 views
TRANSCRIPT
![Page 1: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/1.jpg)
The Program Review Process: NCATE and the State of Indiana
Richard Frisbie and T. J. OakesMarch 8, 2007(source:NCATE, February 2007)
![Page 2: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/2.jpg)
Programs for which NCATE does not have standards
Art, music, dance, or drama education Business, speech, and vocational
education Some advanced teacher education
programs (e.g., Curriculum & Instruction School counselorThese programs undergo state review or
are accredited by other accrediting agencies
![Page 3: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/3.jpg)
NCATE Program Review-Specialized Professional Associations (SPAs)
The national organizations that represent teachers, professional education faculty, and other school personnel who teach a specific subject matter, teach students at a specific developmental level, teach students with special needs, administer schools, or provide services to students.
![Page 4: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/4.jpg)
State of Indiana Program Review
The state review is similar to the NCATE process. For details on the state review, see http://www.doe.state.in.us/dps/teacherprep/program_review.html
![Page 5: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/5.jpg)
Challenges for Institutions
Programs are having surprising difficulty demonstrating how assessments align with SPA standards
Programs have had difficulty selecting appropriate assessments
![Page 6: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/6.jpg)
6-8 Assessments: The Rules
Institution must submit a minimum of six assessments, unless the SPA specifies more than six required assessments
Institution may submit additional assessments when SPA does not specify all eight assessments
Five specific types of assessments are required by all SPAs
![Page 7: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/7.jpg)
Required Assessments in National Program Review
1. State licensure exam for content2. Assessment of content knowledge3. Assessment of Planning (e.g., lesson
plan or unit plan)4. Student teaching/internship assessment5. Assessment of candidate impact on
student learning or providing a supporting learning environment
6. Other assessment
![Page 8: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/8.jpg)
NCATE Unit Standard #1 Candidates preparing to work in
schools as teachers or other school personnel know the content of their fields, demonstrate professional and pedagogical knowledge, skills, and dispositions and apply them so that students learn. Assessments indicate that candidates meet professional, state, and institutional standards.
![Page 9: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/9.jpg)
Required Program Assessments
Content:1. State licensure exam for program
area (if available—otherwise another content based assessment)
2. Content AssessmentProfessional and Pedagogical
Knowledge Skills and Dispositions:3. Assessment of Planning (e.g., unit
plan)
4. Student teaching/internship assessment
![Page 10: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/10.jpg)
P – 12 Student Learning:5. Assessment of candidate impact on
student learning or providing a supporting learning environment
Program is required to have at least one more assessment, can have a maximum of 8 to demonstrate candidate mastery of SPA standards
Some SPAs have additional required types
![Page 11: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/11.jpg)
Important things learnedfrom the pilots:
Unit-wide assessments can be used as key program assessments, but faculty must demonstrate alignment with specific SPA standards
![Page 12: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/12.jpg)
Three effective strategies
SPA-specific addendum SPA-specific rubrics Coding assessment
AND including information in narrative about context
![Page 13: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/13.jpg)
Important things learned:
It is critical to document alignment between assessments, scoring guides and SPA standards
![Page 14: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/14.jpg)
Most Important Learning from Pilots: All Parts of Assessments Must Interrelate!
Scoring Guide
Data & Findings
Assessment
STANDARDS
![Page 15: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/15.jpg)
What does that mean?
Parts of the assessment are aligned with the SPA standards
These standards can be seen in the elements of the scoring guide
The data is broken down by the elements in the scoring guide
![Page 16: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/16.jpg)
Developing Appropriate Assessments: What are the Criteria?
![Page 17: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/17.jpg)
Working definitions of assessment
The term “assessment” can refer to different aspects of the program review process
Any of the 5 required types of assessment
A specific element of an assessment Assessment (assignment summary) Scoring guide Data & findings
![Page 18: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/18.jpg)
The five key principles by which reviewers evaluate your reports
Are assessments aligned with standards? Do assessments meet the cognitive
demands and skill levels of the standards? Are assessments free from bias? Do scoring guides clearly describe the
different categories? Do data provide evidence for meeting
standards?
![Page 19: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/19.jpg)
Distributing assessments across standards
Do not take a “this assessment addresses all standards” approach
Think of assessments as “primary” vs. “supportive” in meeting standards
Typically, one to three assessments, taken as a whole, meet a single standard
![Page 20: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/20.jpg)
The assessment instrument
Align to standards! Preferably with references to standards in the assessment itself.
If alignment is not embedded in the assessment, provide alignment chart in your discussion of the assessment, and a strong argument for how the assessment covers the standard. Be specific.
![Page 21: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/21.jpg)
Characteristics of a strong assessment
The objectives candidates must meet correspond to the depth and breadth required by SPA standards
Assessments are scored primarily on standards-based criteria.
The required activities candidates must perform are clearly described.
Assessment shows no evidence of bias.
![Page 22: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/22.jpg)
Scoring Guides
must be consistent with the assessment instrument
acceptable scores must target the level of competence required by a standard
must clearly describe differences among scored categories
![Page 23: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/23.jpg)
Putting it all together (with data)
The data table is how you “tell your story”
Data should be collected and used in ways that help your program analyze its strengths and weaknesses.
Data should mirror the scoring categories for the assessment (e.g. if grades are used, data should be reported as grades).
![Page 24: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/24.jpg)
Speaking of grades…
Grades in required courses are usually acceptable for Assessment #2.
Provide course descriptions and align courses to standards.
Course content must intensively and comprehensively address standards.
Show range of grades received by candidates, not just GPAs.
![Page 25: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/25.jpg)
Other comments on data
Remove identifying #s and names.Make sure the number of candidates
in data sets are consistent across tables in reports, or explain inconsistencies that might puzzle reviewers.
Note and explain any conversions of data (e.g. numerical scores converted to grades in a data table).
![Page 26: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/26.jpg)
Analyze your data
Make the most of the two pages you have per assessment to analyze and “defend” your assessments and data (Section IV). Analyze your data according to the SPA standards. Compare data across standards, cohort groups, time periods, etc. Tell what you learn from it.
In Section V, make sure to describe what you are doing or will do to address areas where data are poor or below average.
![Page 27: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/27.jpg)
How Much Data?
Fall 2008 and spring 2009, two
years of data Three years of data are optimal –
not necessary to provide more Remember: Data are only as good
as the assessment/scoring guide
![Page 28: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/28.jpg)
Resources on NCATE web site www.ncate.org, click on Institutions,
click on Program Reviews Program Report Forms, Guidelines,
Instructions, Examples of Assessments, Examples of Report Sections, Examples of Reports
www.ncate.org, click on Program Reviewers All training materials for program
reviewers
![Page 29: The Program Review Process: NCATE and the State of Indiana Richard Frisbie and T. J. Oakes March 8, 2007 (source:NCATE, February 2007)](https://reader036.vdocument.in/reader036/viewer/2022062714/56649d3f5503460f94a1949a/html5/thumbnails/29.jpg)
Who to Contact at SPAs
For questions related to the standards and what are appropriate assessments for a particular discipline:
SPA Program Review Coordinators http://www.ncate.org/programreview/spacontact.asp?ch=88