![Page 1: Calibrating Your Grading Criteria with a Little Help from Milli Vanilli Lindsay Portnoy, Educational Foundations](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb25503460f94bb9490/html5/thumbnails/1.jpg)
Calibrating Your Grading Criteria with a Little Help from Milli VanilliLindsay Portnoy, Educational Foundations
![Page 2: Calibrating Your Grading Criteria with a Little Help from Milli Vanilli Lindsay Portnoy, Educational Foundations](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb25503460f94bb9490/html5/thumbnails/2.jpg)
![Page 3: Calibrating Your Grading Criteria with a Little Help from Milli Vanilli Lindsay Portnoy, Educational Foundations](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb25503460f94bb9490/html5/thumbnails/3.jpg)
The Problem
How do you objectively evaluate authentic assessments?
![Page 4: Calibrating Your Grading Criteria with a Little Help from Milli Vanilli Lindsay Portnoy, Educational Foundations](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb25503460f94bb9490/html5/thumbnails/4.jpg)
Qualitative Work Evaluated Quantitatively
Performance Outcome
Below Standard (1)
Approaching Standard (2)
Meets Standard (3)
Exceeds Standard (4)
Ability provide support for an argument
Does not provide support for argument; does not cite research
Provides minimal support without citing the research
Provides adequate support citing the accompanying research
Provides ample support by extensively citing relevant research and providing rich examples
Performance Outcome
Ability provide support for an argument
![Page 5: Calibrating Your Grading Criteria with a Little Help from Milli Vanilli Lindsay Portnoy, Educational Foundations](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb25503460f94bb9490/html5/thumbnails/5.jpg)
Considerations to Address when Creating Rubrics
Validity – Does the rubric measure what it claims to measure.
Reliability – Does the assessment consistently produce the same results.
Transparency – Criteria are clear enough to students so that they can assess themselves and others with roughly the same reliability.
Subjectivity – The amount of judgment used to assign a score to a student’s performance.
![Page 6: Calibrating Your Grading Criteria with a Little Help from Milli Vanilli Lindsay Portnoy, Educational Foundations](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb25503460f94bb9490/html5/thumbnails/6.jpg)
Application to Practice: What Makes a Valid Rubric
Below is the rubric we’ll use to assess the performance of two people in this film
![Page 7: Calibrating Your Grading Criteria with a Little Help from Milli Vanilli Lindsay Portnoy, Educational Foundations](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb25503460f94bb9490/html5/thumbnails/7.jpg)
Application to Practice: What Makes a Valid Rubric
P1
P1P2
P2
![Page 8: Calibrating Your Grading Criteria with a Little Help from Milli Vanilli Lindsay Portnoy, Educational Foundations](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb25503460f94bb9490/html5/thumbnails/8.jpg)
Application to Practice: What Makes a Valid Rubric
The film
![Page 9: Calibrating Your Grading Criteria with a Little Help from Milli Vanilli Lindsay Portnoy, Educational Foundations](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb25503460f94bb9490/html5/thumbnails/9.jpg)
How Valid Was that Rubric?
Memorization
Accuracy
Showmanship
Entertainment Value
![Page 10: Calibrating Your Grading Criteria with a Little Help from Milli Vanilli Lindsay Portnoy, Educational Foundations](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb25503460f94bb9490/html5/thumbnails/10.jpg)
Developing Quality Rubrics
1. Do descriptions focus on important aspects of the performance?
2. Does the rating match the purpose?
3. Are the traits directly observable?
4. Are the criteria understandable?
5. Are the traits clearly defined?
6. Is scoring error minimized?
7. Is the scoring system feasible?
![Page 11: Calibrating Your Grading Criteria with a Little Help from Milli Vanilli Lindsay Portnoy, Educational Foundations](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb25503460f94bb9490/html5/thumbnails/11.jpg)
Building More Valid Rubrics
Performance Outcome (Descriptor)
Below Standard (1)
Approaching Standard (2)
Meets Standard (3)
Exceeds Standard (4)
![Page 12: Calibrating Your Grading Criteria with a Little Help from Milli Vanilli Lindsay Portnoy, Educational Foundations](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb25503460f94bb9490/html5/thumbnails/12.jpg)
Application to Practice: What Makes a Valid Rubric
The film
![Page 13: Calibrating Your Grading Criteria with a Little Help from Milli Vanilli Lindsay Portnoy, Educational Foundations](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb25503460f94bb9490/html5/thumbnails/13.jpg)
How Valid Was that Rubric?
![Page 14: Calibrating Your Grading Criteria with a Little Help from Milli Vanilli Lindsay Portnoy, Educational Foundations](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb25503460f94bb9490/html5/thumbnails/14.jpg)
Who Knew: Errors in Ratings
Carryover effect Practice improves effort
Effect of extraneous variables Sloppiness
Halo effect When it’s better not to know
![Page 15: Calibrating Your Grading Criteria with a Little Help from Milli Vanilli Lindsay Portnoy, Educational Foundations](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb25503460f94bb9490/html5/thumbnails/15.jpg)
Who Knew: Errors in Ratings
Rater drift Less reference to rubric over time
Personal-bias error Generosity error: everyone’s great Central tendency error: everyone’s average Severity error: everyone’s mediocre at best
![Page 16: Calibrating Your Grading Criteria with a Little Help from Milli Vanilli Lindsay Portnoy, Educational Foundations](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb25503460f94bb9490/html5/thumbnails/16.jpg)
Rubric Hints and Guidelines
Use samples of student work
Share rubrics with students
Let students create some rubrics
Allow for multiple correct answers
![Page 17: Calibrating Your Grading Criteria with a Little Help from Milli Vanilli Lindsay Portnoy, Educational Foundations](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb25503460f94bb9490/html5/thumbnails/17.jpg)
Rubric Hints and Guidelines
Limit the scope of the assessment
Consider the levels of difficulty
Determine the number of levels to develop
Adjust the rubric after, not during the assessment
![Page 18: Calibrating Your Grading Criteria with a Little Help from Milli Vanilli Lindsay Portnoy, Educational Foundations](https://reader030.vdocument.in/reader030/viewer/2022032606/56649eb25503460f94bb9490/html5/thumbnails/18.jpg)
To be formative, the assessment must inform