new england common assessment program
DESCRIPTION
New England Common Assessment Program. Guide to Using the October, 2006 NECAP Reports: Companion PowerPoint Presentation February 2007. Welcome and Introductions. Mary Ann Snider Director of Assessment and Accountability RI Department of Education. Welcome and Introductions. - PowerPoint PPT PresentationTRANSCRIPT
1
New England Common Assessment Program
Guide to Using the October, 2006 NECAP
Reports:
Companion PowerPoint
Presentation February 2007
2
Welcome and Introductions
Mary Ann Snider
Director of Assessment and Accountability
RI Department of Education
3
Measured Progress Service Center: 1-877-632-7774
Welcome and Introductions
Tim Crockett
Assistant Vice-President
1-800-431-8901 x2106
Harold Stephens
NECAP Program Director
1-800-431-8901 x2235
Amanda Smith
NECAP Program Manager – Grade 11
1-800-431-8901 x2259
Elliot Scharff
NECAP Program Manager – Science
1-800-431-8901 x2126
Josh Evans
NECAP Program Manager – Grades 3-8
1-800-431-8901 x2244
Tina Haley
NECAP Program Assistant
1-800-431-8901 x2427
Jennifer Varney
NECAP Program Assistant
1-800-431-8901 x2115
Mellicent Friddell
NECAP Program Assistant
1-800-431-8901 x2355
4
• To assist schools and districts in using the variety of reports and support material issued by the NECAP states in conjunction with the release of results
• To provide school and district personnel an opportunity to ask questions regarding the reports and support materials
Purpose of the Workshop
5
• Development of Grade Level Expectations
• Test Item Review Committees• Bias and Sensitivity Review Committees• Classroom Teacher Judgment data• Standard Setting Panelists• Technical Advisory Committee
Involvement of Local Educators
6
• Fall test – previous year’s GLEs• Reading and Mathematics – grades 3-8• Writing – grades 5 and 8• Variety of Item Types
• Reading: multiple choice and constructed response• Mathematics: multiple choice, short answer (one point), short answer
(two points), and constructed response• Writing: multiple choice, constructed response, and extended
response
• Common and Matrix Items
Basics of Test Design
7
Two Basic Types of Educational Tests
• Norm-Referenced: What is the student’s status or rank in a designated group?
• Criterion or Standards-Referenced: Has the student learned the skills or achieved the standard?
NECAP is a Standards-Referenced Test
8
The Family Educational Rights and Privacy Act (FERPA)
• Access to individual student results is restricted to:• the student• the student’s parents/guardians• authorized school personnel
• Superintendents and principals are responsible for maintaining the privacy and security of all student records.
• Authorized school personnel shall have access to the records of students to whom they are providing services when such access is required in the performance of their official duties.
• FERPA website: http://www.ed.gov/policy/gen/guid/fpco/ferpa/index.html
Family Educational Rights and Privacy Act
9
National Council on Measurement in Education
Educating Others
Evaluation and Research
Interpretation and Use
Scoring
Development
Marketing and Selling
Selection
Administration
www.ncme.org
10
Questions about the purpose of theworkshop and/or NECAP background?
Questions and Answers
11
Rhode Island Results
Suburban
Urban Fringe
Urban
12
Rhode Island Results
READING MATHEMATICS
2005 2006 2005 2006
All Students
58% 62% 50% 53%
Native Americans
46 49 33 38*
Asian 56 64* 52 59*
Black 35 41* 24 30*
Hispanic 31 38* 24 31*
White 68 71 59 62
Female 63 67 50 53
Male 54 58 50 54*
IEP 24 27 20 23
ELL 17 23* 14 20*
Poverty 38 44* 30 36*
* Indicates that gain exceeds the gain made by all students
13
• Student Report• Item Analysis Report• School/District Results Report• School/District Summary Report• District Student Level Data Files
Types of Reports
14
Student Report
• Two copies mailed to the school:• color copy for parents/guardians• black and white copy for local use
• Dual sided: text side and data side
• Spanish version of the student report available on the RIDE website for the Office of Assessment and Accountability
15
16
17
18
19
• Review to be sure you have a student report for every student tested in October, 2006.
• Prepare a letter to provide your interpretation of the results for your school and/or district.
• Be sure teachers are prepared and comfortable discussing individual student results with families; including any instructional interventions.
• Use your district data file to analyze growth between students you had last year and this year.
Fun with Student Level Reports!
20
Questions about the Student Report?
Questions and Answers
21
• Rationale• Inclusion of students in “Teaching
Year” reports
Teaching School vs. Testing School
22
Schools Can View Reports for Testing Year (2006-07)
Teaching School vs. Testing School
23
Teaching School vs. Testing School
Or Teaching Year(2005-06)
24
Testing Year(released this month)
Teaching Year (accountability)
25
Item Analysis Report
• Available to schools and districts on-line• These reports are confidential and the
website is password-protected. • One per content area• Use in conjunction with Released Items
documents
26
27
28
Released Item Documents
29
Grade 5 2006 Released Mathematics Item
DOK: 3
GLE: DSP 4.1
30
Grade 5 Practice Test Item (page 4)
DOK: 2
GLE: DSP 4.1
31
Fun with Released Support Materials!
• Build practice tests
• Share writing samples as models
• Mock scoring sessions to calibrate teacher judgments
• Compare released items (GLE and DOK) with classroom instruction and assessments and student work.
• Map instruction and scaffolding practices to items based on student performance.
32
Released Item Documents
• RIDE website at: http://www.ride.ri.gov/assessment/NECAP.aspx
• New information provided in the Released Item Support Materials documents includes:• For all content areas, providing the multiple-choice items
and the GLE code and text that each item is linked to• For math, indicating which items were included in session 1
of the math test by adding a “no calculator” symbol above the items
• For reading, listing the passage name and type of text• For writing, listing passage name and mode of writing
33
Questions about the Item Analysis Report or the Released Item documents?
Questions and Answers
34
• Available on-line at the Measured Progress website
• Separate report for each grade level tested
• District report contains no individual school data
School/District Results Report
35
http://iservices.measuredprogress.org
36
New thisyear
Please note:
You must always log in through this page to view confidential
reports and data
37
38
39
40
41
42
• Page 2 of the NECAP School/District Results Report
• Summary of participation and results
Grade Level Summary Report
43
44
• Review last year’s grade level summary report with this year’s
• If gains were made, identify what strategies, interventions, and programs may have worked
• If no progress was made, hypothesize why and what can be done at the school or district level to help students make progress
• Identify where your results differ from the district and state results. Hypothesize why they may differ
Fun with Grade Level Summaries!
45
Questions about the Grade LevelSummary Page?
Questions and Answers
46
Historical Data
• Data is included for two years and is combined in the Cumulative Total row
• In the Cumulative Total row for the achievement level columns:• The N columns are summed• The % columns are found by dividing the total
number of students in the level by the total number of students tested
• The Mean Scaled Score column is a weighted average
47
48
Comparisons of NECAP Scores Across Years
• School and District-Level Scaled Scores and Achievement Levels
• Student-Level Scaled Scores and Achievement Levels
• Content Area Subscores
49
• Pages 3, 5, and 7 of the NECAP School/District Results Report
• Provides information about achievement in subtopics of tested content areas
Content Area Results
50
51
Number of Scaled Score Points Denoting Minimally Statistically Significant Difference for Average Group Results*
Grade SubjectNumber of Students Tested in Group (Class, School etc.)
10 25 50 100 200
3Reading 6 4 3 2 1
Mathematics 5 3 2 2 1
4Reading 6 4 2 2 1
Mathematics 5 3 2 2 1
5
Reading 5 3 2 2 1
Mathematics 5 3 2 2 1
Writing 7 4 3 2 2
6Reading 6 4 3 2 1
Mathematics 6 4 3 2 1
7Reading 6 4 3 2 1
Mathematics 5 3 2 2 1
8
Reading 6 4 3 2 1
Mathematics 5 3 2 2 1
Writing 6 4 3 2 1
*Standard error of the mean difference with assumption n1=n2 and s1=s2
52
Percentage Difference in Student Achievement Level Classification Denoting Minimally Statistically Significant
Differences for Group Results*
Percentages of Students in Achievement Level(s)
Number of Students Tested in Group (Class, School etc.)
10 25 50 100 200
10 9 6 4 3 2
20 13 8 6 4 3
30 14 9 6 5 3
40 15 10 7 5 3
50 16 10 7 5 4
60 15 10 7 5 3
70 14 9 6 5 3
80 13 8 6 4 3
90 9 6 4 3 2
*0.05 level of statistical significance
53
Questions About the Content AreaResults Page?
Questions and Answers
54
• Pages 4, 6, and 8 of the NECAP School/District Results Report
• Provides information about achievement for different groups in a school/district
Disaggregated Content Area Results
55
56
• Compare last year’s Disaggregated Report with this year• Highlight in yellow, groups that had significant positive
increases• Highlight in blue, groups that remained stable or had decreases
• Establish cross-grade and content groups to hypothesize what the school can do to close gaps and maintain increases• Access to rigorous curriculum?• Professional development in a specific area?• Adequate support programs?• Adequate family involvement?
Fun with Disaggregated Results!
57
Questions About the Disaggregated Content Area Results Page?
Questions and Answers
58
• Separate report for each grade level tested
• Provides details about student achievement by content area
School/District Summary Report
59
School/District Summary Report
• The first line designates the year the test was administered
• The second line is the name of the report• The third line differentiates between
“teaching year” and “testing year”
60
61
• Are there consistent patterns of achievement across grades? Across content areas?
• Compare results to last year’s school summary• Highlight in yellow, areas where there has been
significant growth. Highlight in blue, areas where there has been no growth or decline.
• Establish cross-grade and content groups to hypothesize what the school can do to close gaps and maintain increases
• Develop a school/district plan to address areas of weakness and continue areas of growth
Fun with School Level Summary Data!
62
Student Level Data Files
• All demographic information for each student that was provided by the state
• The scaled score, achievement level, and subscores earned by each student in all content areas tested
• Also contain:• Performance on released items• Student questionnaire responses• Optional reports fields data
63
64
Supporting Materials
• Guide to Using the 2006 NECAP Reports and companion PowerPoint
• Grade Level Expectations• Test Specifications documents• Release Item documents• Preparing Students for NECAP: Tips for
Teachers to Share with Students• Technical Report