clarifying learning goals & standards: rubrics & exemplars
TRANSCRIPT
![Page 1: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/1.jpg)
Clarifying learning goals & standards: rubrics & exemplars
David Carless University of Hong Kong
At EdUHK, January, 20, 2017
The University of Hong Kong
![Page 2: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/2.jpg)
Overview
1. Key concepts2. Students’ views on rubrics & exemplars3. Using rubrics for learning 4. Analyzing exemplars 5. Summary of key implications
The University of Hong Kong
![Page 3: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/3.jpg)
KEY CONCEPTS
The University of Hong Kong
![Page 4: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/4.jpg)
Competing assessment functions The University of Hong Kong
Stimulating productive student learning
Satisfying accountability needs
Judging student achievement
![Page 5: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/5.jpg)
Learning-oriented assessment
Prioritizing student learning as a key aim of all assessment
The University of Hong Kong
![Page 6: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/6.jpg)
The University of Hong Kong
Learning-oriented assessment framework (Carless, 2015a)
Productive assessment task
design
Student engagement with
feedback
Understanding quality academic
work
![Page 7: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/7.jpg)
Importance of learning goals
Where am I going?
How to get there?
The University of Hong Kong
![Page 8: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/8.jpg)
![Page 9: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/9.jpg)
What are rubrics?
• A.k.a. ‘assessment criteria’, ‘grade descriptors,’ ‘marking schemes’
• Rubrics combine criteria and standards in the form of a grid to provide statements of the performance needed to achieve different standards (Carless, 2015, p. 132)
The University of Hong Kong
![Page 10: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/10.jpg)
What are exemplars?
Exemplars are samples of student work used to illustrate dimensions of quality
Usually assignments from a previous cohort
The University of Hong Kong
![Page 11: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/11.jpg)
The University of Hong Kong
![Page 12: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/12.jpg)
Two relevant chapters
Carless, D. (2017). Students’ Experiences of Assessment for Learning. In D. Carless et al., (Eds.), Scaling up Assessment for learning in Higher Education. Singapore: Springer. Jönsson, A. & Panadero, E. (2017). The Use and Design of Rubrics to Support Assessment for Learning. In D. Carless et al., (Eds.), Scaling up Assessment for learning in Higher Education. Singapore: Springer.
The University of Hong Kong
![Page 13: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/13.jpg)
EVIDENCE FROM INTERVIEWS WITH STUDENTS
The University of Hong Kong
![Page 14: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/14.jpg)
Research process The University of Hong Kong
Classroom observations
Interviews with teachers & students HistoryLaw
Business
![Page 15: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/15.jpg)
Rubrics data (1)
“Criteria are rather vague to me …”
“I read the criteria descriptions some time ago and I’ve already forgotten them”
“The criteria were similar to previous ones so I didn’t study them carefully or consult them when preparing the assignment”.
The University of Hong Kong
![Page 16: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/16.jpg)
Rubrics data (2)
“The descriptions of assessment criteria are almost the same for every subject with the same key words like critical & analytic”
“The assessment criteria are a bit lifeless. I wanted to know the teacher’s interpretation of the criteria and his personal preference”.
The University of Hong Kong
![Page 17: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/17.jpg)
Rubrics data (3)
“Lists of criteria are all the same, it’s all very vague. Like for an ‘A’, it is excellent mastery and correct application and then ‘B+’ would be above average mastery and good application. You don’t know what it means so you just do what you can. Once you know how to get good academic results, you don’t need to rely on criteria any more because the criteria are always the same.”
The University of Hong Kong
![Page 18: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/18.jpg)
Exemplars data (1)
“Samples are more concrete”
“It is a reference … you can imitate its presentation style”
The University of Hong Kong
![Page 19: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/19.jpg)
Exemplars data (2)
“The exemplar served as a useful example of what a good report should be”
“There is a risk of copying but many of us try to add something more or use our own approach”
The University of Hong Kong
![Page 20: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/20.jpg)
Inferences
Rubrics are only useful for learning if students engage with them
Exemplars are attractive to students but teacher implementation is complex
The University of Hong Kong
![Page 21: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/21.jpg)
RUBRICS & STUDENT ENGAGEMENT
The University of Hong Kong
![Page 22: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/22.jpg)
Dual aspects of rubrics
1. A scoring tool for teachers
2. A potential learning tool for students as aid to self-evaluation
The University of Hong Kong
![Page 23: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/23.jpg)
Rubric options
• Analytic or holistic
• Generic or Task-specific
(see handout)
The University of Hong Kong
![Page 24: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/24.jpg)
Writing rubrics
Standards should be represented with respect to features of work, not impenetrable terms, such as ‘good’ or ‘excellent’ (Ashwin, Boud et al., 2015, p. 259)
The University of Hong Kong
![Page 25: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/25.jpg)
![Page 26: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/26.jpg)
Rubrics activity 1
Students in groups brainstorm their own criteria for an assessment task
The University of Hong Kong
![Page 27: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/27.jpg)
Rubrics activity 2
Students read the criteria for (say) a grade B and then re-write them in their own words
The University of Hong Kong
![Page 28: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/28.jpg)
Rubrics activity 3
Students use rubric to evaluate work of peer (peer evaluation)
AND/OR
Students use rubric to evaluate exemplars
The University of Hong Kong
![Page 29: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/29.jpg)
DISCUSSION 1: RUBRICS
• Share some of your thinking on using rubrics to enhance student learning
• What unanswered issues might you raise in relation to rubrics?
The University of Hong Kong
![Page 30: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/30.jpg)
RATIONALE & USE OF EXEMPLARS
The University of Hong Kong
![Page 31: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/31.jpg)
Why used?
Exemplars convey messages that nothing else can (Sadler, 2002)
The University of Hong Kong
![Page 32: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/32.jpg)
Promote self-evaluation
Illustrate what good work looks like
Benchmark for student comparison with own performance
The University of Hong Kong
![Page 33: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/33.jpg)
Analysis
Identify strengths, weaknesses & suggest how samples could be improved
The University of Hong Kong
![Page 34: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/34.jpg)
When used?
Useful for students at transition stages
When assignments are complex, innovative or hard to explain
The University of Hong Kong
![Page 35: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/35.jpg)
How used?
• Posted on LMS• Optional workshop (Rust et al., 2003)• Part of regular class• Displayed or made available• In part or as a whole
The University of Hong Kong
![Page 36: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/36.jpg)
Benefits for students Gain experience in making judgments
Apply insights to own work
Notice discrepancies between present level & target level
The University of Hong Kong
![Page 37: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/37.jpg)
The University of Hong Kong
![Page 38: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/38.jpg)
Copying model problem
May reduce student creativity
May encourage unproductive copying
The University of Hong Kong
![Page 39: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/39.jpg)
Time issues
• Time in collecting them
• Time in seeking consent for their use
• Time taken away from instruction
The University of Hong Kong
![Page 40: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/40.jpg)
It’s challenging!
Difficulties for some students in evaluating exemplars accurately
The University of Hong Kong
![Page 41: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/41.jpg)
Synthesis on exemplars
• Students like exemplars • They aid learning But • How to implement effectively• How to avoid ‘copying model answer
syndrome’
The University of Hong Kong
![Page 42: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/42.jpg)
Key inference
The quality of the dialogue about the exemplars is a crucial factor determining student learning
The University of Hong Kong
![Page 43: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/43.jpg)
DISCUSSION 2: EXEMPLARS
• Why don’t teachers make more & better use of exemplars?
• What unanswered issues, might you raise in relation to exemplars?
The University of Hong Kong
![Page 44: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/44.jpg)
RECENT EXEMPLARS RESEARCH
The University of Hong Kong
![Page 45: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/45.jpg)
TDG Project
Enhancing dialogic use of exemplars amongst 10 teachers in a Faculty of Education
The University of Hong Kong
![Page 46: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/46.jpg)
The University of Hong Kong
![Page 47: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/47.jpg)
Research question
What are the main features of a specific exemplars dialogue and how is it orchestrated?
The University of Hong Kong
![Page 48: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/48.jpg)
Context of the case study
Participants: • Trainee Science teachers • BEd/BSc double degree • Year 3
Teacher-researcher: Dr Kennedy Chan
The University of Hong Kong
![Page 49: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/49.jpg)
Data collection
• Classroom observation • Open-ended student survey• 2 focus group interviews• Interview with teacher-researcher• Teacher journal• Student artefacts: ‘exit slips’, assignments etc.
The University of Hong Kong
![Page 50: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/50.jpg)
Exemplars provided2 high quality exemplars
– Same format (i.e., reflective essays) but different content focus
– Design aimed at reducing ‘copying’
The University of Hong Kong
![Page 51: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/51.jpg)
Management of dialogue
1. Prior to class students analyzed exemplars;2. Students discussed exemplars in pairs; 3. Students elicited views from peers and made
mini-presentations; 4. Teacher-orchestrated dialogue; 5. Students submitted exit slip of reflections.
The University of Hong Kong
![Page 52: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/52.jpg)
Dilemmas
Students constructing
notion of quality VS
Telling students about quality
Time for students to talk with
peers VS
Time for developing shared understandings
Exemplars as
guide VS
Exemplars as model
Using students’ voices VS
Making teacher’s voice
explicit
The University of Hong Kong
![Page 53: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/53.jpg)
Good Exemplars dialogue
• Airs multiple & divergent viewpoints • Shows linkages between peer talk & whole-
class discussion • Evidences development of student views • Makes explicit some key qualities of exemplars(Carless & Chan, 2016)
The University of Hong Kong
![Page 54: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/54.jpg)
CONCLUSION
The University of Hong Kong
![Page 55: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/55.jpg)
Using exemplars & criteriaAssessment task
Students devise or engage with rubric & quality
Two samples read before class
Peer
discussion Teacher-led dialogue
Student ownership of
insights
The University of Hong Kong
![Page 56: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/56.jpg)
Engaging with quality
Student need to engage with what quality looks like and develop capacities in making judgments
The University of Hong Kong
![Page 57: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/57.jpg)
THANK YOU
The University of Hong Kong
![Page 58: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/58.jpg)
References Ashwin, P., Boud, D., et al., (2015). Reflective teaching in higher education. London: Bloomsbury. Carless, D. (2015a). Exploring learning-oriented assessment processes. Higher Education, 69(6), 963-976.Carless, D. (2015b). Excellence in University Assessment: learning from award-winning teachers. London: Routledge. Carless, D. (2017). Students’ experiences of Assessment for Learning. In D. Carless, S. Bridges, C.K.W. Chan & R. Glofcheski (Eds.), Scaling up Assessment for learning in Higher Education. Singapore: Springer.Carless, D., Bridges, S., Chan, C.K.W. & R. Glofcheski, R. (Eds.), (2017). Scaling up Assessment for learning in Higher Education. Dordrecht: Springer.Carless, D. & K.K.H. Chan (2016). Managing dialogic use of exemplars. Assessment and Evaluation in Higher Education, http://dx.doi.org/10.1080/02602938.2016.1211246Hendry, G., Armstrong, S. & Bromberger, N. (2012). Implementing standards‐based assessment effectively: Incorporating discussion of exemplars into classroom teaching. Assessment and Evaluation in Higher Education, 37(2), 149-161.
The University of Hong Kong
![Page 59: Clarifying learning goals & standards: rubrics & exemplars](https://reader035.vdocument.in/reader035/viewer/2022070601/5887091c1a28abf2228b4a6b/html5/thumbnails/59.jpg)
References (cont.) Jonsson, A. & Panadero, E. (2017). The use and design of rubrics to support assessment for learning. In D. Carless, S. Bridges, C.K.W. Chan & R. Glofcheski (Eds.), Scaling up Assessment for learning in Higher Education. Singapore: Springer.Rust, C., Price, M. & O’Donovan, B. (2003). Improving students’ learning by developing their understanding of assessment criteria and processes. Assessment and Evaluation in Higher Education, 28(2), 147-164.Sadler, D. R. (2002). Ah! … So that’s ‘quality’. In P. Schwartz & G. Webb (Eds.), Assessment: Case Studies, Experience and Practice from Higher Education (p.130-136). London: Kogan Page.Smith, C., Worsfold, K., Davies, L., Fisher, R. & McPhail, R. (2013). Assessment literacy and student learning: The case for explicitly developing students ‘assessment literacy’. Assessment and Evaluation in Higher Education, 38(1), 44-60.
The University of Hong Kong