developing monitoring and pre-scoring plans for alternate/alternative assessments virginia...
TRANSCRIPT
![Page 1: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/1.jpg)
Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative
Assessments
Virginia Department of Education
Division of Student Assessment
and School Improvement
August 2011
![Page 2: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/2.jpg)
Purpose of this Presentation To assist school divisions in developing
plans to monitor and pre-score Collections of Evidence (COE) for the Virginia Alternate Assessment Program (VAAP) and the Virginia Grade Level Alternative (VGLA) and Course Work Compilations (CWC) for the Virginia Substitute Evaluation Program (VSEP).
![Page 3: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/3.jpg)
Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative
Assessments What are monitoring and pre-scoring? Why should divisions develop monitoring and pre-
scoring plans? Who should be involved in developing the
monitoring and pre-scoring plans? What components should the monitoring plan
include? What components should the pre-scoring plan
include?
![Page 4: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/4.jpg)
What are monitoring
and pre-scoring?
![Page 5: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/5.jpg)
Monitoring
A periodic and systematic review of COEs and CWCs while they are in the development process with opportunities for feedback and intervention.
![Page 6: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/6.jpg)
What Monitoring is Not
Checking in with the teacher
Flipping through COEs or CWCs
Reviewing COEs and CWCs without providing feedback
![Page 7: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/7.jpg)
Pre-Scoring
A detailed review of COEs and CWCs to address technical errors (i.e., grading, SEI tags) prior to scoring with opportunities for feedback and corrections.
![Page 8: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/8.jpg)
What Pre-Scoring is Not
Assigning a rubric score to the evidence
Altering student evidence or student responses
Judging the instructional quality of the evidence
![Page 9: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/9.jpg)
Why should divisions
develop monitoring and pre-scoring
plans?
![Page 10: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/10.jpg)
Monitoring Keeps division staff abreast of the status
of each COE and/or CWC
Ensures that teachers are providing instruction based on the Standards of Learning (SOL)
Ensures that instruction is on target with the division’s pacing guide
![Page 11: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/11.jpg)
Monitoring Ensures that technical issues (i.e.,
grading, SEI tags) are appropriately addressed
Provides the opportunity to address issues with ample time for correction and/or intervention
![Page 12: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/12.jpg)
Pre-Scoring Provides a final opportunity to correct
technical errors and omissions that may result in the loss of credit to students prior scoring
Results in the creation of better COEs and CWCs
![Page 13: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/13.jpg)
Who should be involved
in developing the monitoring and pre-scoring plans?
![Page 14: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/14.jpg)
Who should be involved in developing the plans?
Depends upon:
Number of COEs and CWCs
Available central office resources
Available building level resources
![Page 15: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/15.jpg)
Who should be involved in developing the plans?Central Office Staff
Division Director of Testing (DDOT) or designee
Special Education Director or designee
Director of Instruction or designee
Title III Coordinator or designee
Building Level Staff
Principal or designee
School Test Coordinator
Special Education Chairperson
![Page 16: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/16.jpg)
What components should the
monitoring plan include?
![Page 17: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/17.jpg)
Critical Planning InformationDevelop a Master Participation List for each
alternate/alternative assessment program that includes... Name of student Grade level of student School Content area being assessed by
alternate/alternative assessment Teacher(s) responsible for instruction in the
content area assessed
![Page 18: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/18.jpg)
Monitoring Plan Decisions Who will monitor?
How will monitors be trained?
When will monitoring occur?
Who will receive feedback?
How will feedback be given?
What options are available if corrections are needed?
![Page 19: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/19.jpg)
Remember!
Your monitors will provide training and/or technical guidance to teachers submitting COEs and are not allowed to score these COEs.
![Page 20: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/20.jpg)
Who will monitor?Staff with content knowledge and alternate/alternative
assessment knowledge Hired consultants Special education or testing specialists Instructional specialists LEP specialists School test coordinators Title I mathematics and reading specialists Experienced teachers Building administrators
![Page 21: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/21.jpg)
How will monitors be trained? Provide access to implementation
manuals and other resources (i.e. testing memos, VGLA worksheets, SEI tags, Curriculum Frameworks)
Conduct local training for monitors to include review and discussion of scoring rules, SOL or ASOL information, issues generated from the previous year’s scoring, etc.
![Page 22: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/22.jpg)
When will monitoring occur? Monthly
End of each grading period
At the same time as benchmark testing
Every other month
Other
![Page 23: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/23.jpg)
Who will receive feedback? Teachers
Principals
Alternate/Alternative Assessment Leaders
Central Office Staff
Trainers/Technical Assistance Providers
![Page 24: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/24.jpg)
How will feedback be given? Post It notes on COEs and CWCs
Locally created feedback forms given to teachers, principals, and/or others
Meetings with teachers, principals, and/or others about monitoring results
Other
![Page 25: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/25.jpg)
What options are available if corrections
are needed? Individualized consultation and support
Peer or small group training sessions
Division-level professional development
Access to state and local SOL and/or ASOL assessment resources
Other
![Page 26: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/26.jpg)
A Sample Monitoring Plan
for 500 COEs School-based Review Teams will be created in
each school.
DDOT and Special Education Director will train School-based Review Teams using implementation manuals, VDOE power points, division pacing charts, and other resources.
Teachers submit COEs or CWCs to the School-based Review Team at the end of each nine-weeks.
![Page 27: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/27.jpg)
A Sample Monitoring Plan
School-based Review Teams will report their finding on each COE or CWC using a locally created form. A copy of the form will be distributed to the teachers, principals and central office staff.
Central office will dispatch instructional specialist and assessment specialist to schools in need of additional support and training.
![Page 28: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/28.jpg)
A Sample Monitoring Plan
1Select And TrainSchool-based Review Teams
3Instructional and assessment specialists
dispatched to schools based on 9 Week Reports
2School-based Review Teams check COEs
every 9 weeks according to Pacing Chart and
report to principals andcentral office staff
![Page 29: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/29.jpg)
A Sample School-Based Review Team
9-Week Monitoring Form – Grade 4 Reading
Student 100 - 90 % of SOL Evidenced
89 - 70 % of SOL Evidenced
69- 50 % of SOL Evidenced
Less than 50% of SOL Evidenced
Ted Smith X
Kay Blue X
Joe Davis X
Ann Jones X
![Page 30: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/30.jpg)
A Sample Teacher Review Form
VGLA 9 Week Review SheetSchool-Based Teams
Teacher: ____________________
Student: ___________________________ Grade Level:______
Content Area:_________________________________________
REVIEW for: ____ 1st Nine Weeks ____ 3rd Nine Weeks
____ 2nd Nine Weeks ____ 4th Nine Weeks
Reviewed By:____________________________ Date:________
Collection Status: Address the following questions: (1) Is there evidence for all of the standards for the nine weeks according to the Pacing Chart? (2) Does the work submitted align with the standards satisfactorily? (2) Does the evidence demonstrate student mastery? (3) Has the student work been graded accurately? (4) Other?
Recommendation(s): Be as specific as possible
Follow-up review needed for implementation of recommendations:
___Yes ____No
Follow –up Review Date: ________________
![Page 31: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/31.jpg)
Pre-Scoring Decisions Who will pre-score?
How will pre-scorers be trained?
When will pre-scoring occur in relation to submission date?
Who will receive feedback?
How will feedback be given?
What options are available if corrections are needed?
![Page 32: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/32.jpg)
Persons selected for pre-scoring VAAP COEs can not serve as scorers.
Pre-scorers should be knowledgeable about all scoring rules and the SOL and/or ASOL content.
Pre-scorers should be detail oriented.
Who will pre-score?
![Page 33: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/33.jpg)
Remember!
Your pre-scorers will provide training and/or technical guidance to teachers submitting VAAP COEs and are not allowed to score these COEs.
![Page 34: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/34.jpg)
Training should include a list of specific issues to look for in each COE or CWC
grading
correctly completed SEI tags
detailed anecdotal records (VAAP only)
captioned photographs (VAAP only)
How will pre-scorers be trained?
![Page 35: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/35.jpg)
Other helpful questions to ask in
Pre-scoring Is the evidence organized according to the scoring worksheet?
Are all required state and local forms included and completed?
![Page 36: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/36.jpg)
Select a time to pre-score:
that will give ample time for COEs or CWCs to be complete or near completion.
that will give opportunity to return COEs or CWCs to teachers for corrections and returned by local due date.
When will pre-scoring occur in relation to
local due date?
![Page 37: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/37.jpg)
Teacher(s) submitting COE or CWC.
Building level principals
Others
Who will receive feedback?
![Page 38: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/38.jpg)
Post-it Notes on COEs and CWCs
Local created forms given to teachers, principals, and/or others
Meetings with teachers, principals, and/or others
Other
How will feedback be given?
![Page 39: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/39.jpg)
Return COEs to teachers for corrections.
Assign staff to assist teacher in corrections such as grading evidence, organizing evidence, completing SEI tags.
Options may be limited or non-existent due to time constraints
Other
What options are available if corrections
are needed?
![Page 40: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/40.jpg)
Sample Pre-scoring Plan Pre-scoring will be conducted by teachers,
building level staff and central office staff in three phases Teacher Level Building Level Central Office Level
Scoring rules will be reviewed with all persons involved in pre-scoring. Checklists will be provided.
![Page 41: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/41.jpg)
Sample Pre-Scoring Plan
PHASE 1 – Teacher Review
Conducted April 12-16
Submitting Teacher reviews COEs or CWCs using Teacher Checklists in Implementation Manuals
PHASE 2 – School-based
Review
Conducted April 19 – 26
COEs or CWCs using notes and local forms
Building Administrator reviews COEs or CWCs using Administrator Checklist in Implementation Manuals
![Page 42: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/42.jpg)
Sample Pre-Scoring PlanPhase 3 – Central Office Pre-scoring Team Review
Conducted April 27 – May 7
Central Office team reviews COEs and CWC against scoring rules. Findings noted on Post-It notes placed on evidence.
Central Office teams return COEs or CWCs to schools to correct errors and omissions.
COEs or CWCs are corrected by teachers. Teachers sign affidavits
COEs or CWCs submitted to principals or designee. Principal or designee signs affidavits.
COEs or CWC submitted for scoring.
![Page 43: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/43.jpg)
Monitoring & Pre-Scoring Results
Reduces or Eliminates COEs and CWCs with:
Missing Evidence or SEI tags
Unacceptable Evidence (textbooks, homework)
Inaccurate or Ungraded Evidence
![Page 44: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/44.jpg)
Monitoring & Pre-Scoring Results
Eliminates Surprises:
COEs or CWCs not done for students educated outside of the division
Incomplete COEs or CWCs in the division
COEs or CWCs not driven by IEP, 504 or LEP Plans
![Page 45: Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative Assessments Virginia Department of Education Division of Student Assessment and School](https://reader035.vdocument.in/reader035/viewer/2022070407/56649e165503460f94b0153c/html5/thumbnails/45.jpg)
Questions
Virginia Department of EducationOffice of Test Administration, Scoring, and
Reporting
804 225-2102
Office of Special Education Instructional Services
804-225-2932