Download - Innovation in Assessment and Evaluation
![Page 1: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/1.jpg)
1
Innovation in Assessment and
Evaluation
Prof. dr. Martin Valckehttp://allserv.ugent.be/~mvalcke/CV/CVMVA.htm
Ghent University
Maputo July 2011
![Page 2: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/2.jpg)
Structure
• Advance organizer
• Traditional approaches
• Trends in assessment & evaluation
• Assumptions in relation to these trends
• Research
• Practices
• Conclusions2
![Page 3: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/3.jpg)
3
![Page 4: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/4.jpg)
Advance organizer
![Page 5: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/5.jpg)
Advance organizer
5
Vorm/ manier van overbrengenVorm/ manier van overbrengen3. oogcontact3. oogcontact
Bert probeert het anders:
Denk…
Transmission of your message3. Eye Contact
![Page 6: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/6.jpg)
6
Kan ik het wel leren?Kan ik het wel leren?
Iedereen kan het leren
door ‘oefening’ + ‘feedback’
Luister maar eens naar de Getuigenis van Bert (student handelswetenschappen)(linkermuisklik in beeld)
1. Geloof in verandering!
2. Doe het ‘stap voor stap’ : 1 of 2 aandachtspunten tegelijk
3. Reflecteer zoals Bert: denk na over sterke punten en over veranderingspunten
Wat kunnen we leren van Bert? Denk even na en klik pas voor de antwoorden!
Can I learn this?
![Page 7: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/7.jpg)
The importance of assessment & evaluation
![Page 8: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/8.jpg)
8
![Page 9: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/9.jpg)
Traditional approaches
9
![Page 10: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/10.jpg)
Critical issues
• Validity of evaluation approach in view of assessment of skills and complex knowledge
• Fant et al., (1985)• Rating scales, daily logs, anecdotal records, behavior
coding, and self-assessment for evaluating student teachers.• Oral examinations, portfolio assessment, central assessment
centres, 360° assessment
– …
10
![Page 11: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/11.jpg)
Recent developments
Individual learner
Group learner
External institution
Teachers Expert eacher
Assessment systemInstitutional level
![Page 12: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/12.jpg)
Recent developments
• Stronger focus on “consequential validity”of measurement (Gielen, Dochy & Dierick, 2003)
• Stronger emphasis on feedback value of assessment
• What is the “learning potential” of the assessment approach
12
![Page 13: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/13.jpg)
Recent developments
• Stiggins (1987): performance assessment
• Performance assessment is expected to be geared in a better way to assess complex behavior in medical, legal, engineering, … and educational contexts (Sluijsmans, et.al., 2004).
13
![Page 14: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/14.jpg)
Concrete examples
• Self- and peer assessment
• Rubrics based assessment
14
![Page 15: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/15.jpg)
Self- and peer-assessment
15
![Page 16: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/16.jpg)
How to help the student to assess him/herself?
16
![Page 17: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/17.jpg)
Self- and peer assessment
• Learn about your own learning process.
• Schmitz (1994): “assessment-as-learning”.
• ~ self corrective feedback
17
![Page 18: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/18.jpg)
• See experiential learning cycle of Kolb.
• Boekaerts (1991) self evaluation as a competency.
• Development of metacognitive knowledge and skills (see Brown, Bull & Pendlebury, 1998, p.181).
• Freeman & Lewis (1998, p.56-59): developing pro-active learners
18
![Page 19: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/19.jpg)
19 The Learning Cycle Model
![Page 20: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/20.jpg)
![Page 21: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/21.jpg)
Is it possible?
21
Group evaluations tend to fluctuate around the mean
![Page 22: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/22.jpg)
Learning to evaluate
• Develop checklists
• Give criteria
• Ask to look for quality indicators.
• Analysis of examples good and less good practices: develop a quality “nose”
22
![Page 23: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/23.jpg)
Learning to evaluate
• Freeman & Lewis (1998, p.127) :• Learner develops list of criteria.• Pairs of learners compare listed criteria.• Pairs develop a criterion checklist.• Individual application of checklist.• Use of checklist to evalute work of other learner.• Individual reworks his/her work.• Final result checkeed by teacher and result compared to learner
evaluation.• Pairs recheck their work on the base of teacher feedback.
23
![Page 24: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/24.jpg)
Learning to evaluate
• Peer evaluation is not the same as Peer grading
• Final score is given by teacher
• Part of score could build on accuracy of self/peer evaluation and self-correction
24
![Page 25: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/25.jpg)
25
![Page 26: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/26.jpg)
Rubrics
26
![Page 27: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/27.jpg)
Rubrics
• Rubrics focus on the relationship between competencies, criteria, and indicators and are organized along mastery levels (Morgan, 1999).
27
![Page 28: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/28.jpg)
28http://web.njit.edu/~ronkowit/teaching/rubrics/samples/rubric_apa_research.pdf
![Page 29: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/29.jpg)
Rubrics• Rubric: scoring tool for a qualitative assessment of the
quality level of an authentic or complex activity– A rubric builds on criteria, enriched with a scale to indicate a
mastery level.
– For each level, standards are indicated that reflect this level.
– A rubric dictates both teacher and student what is concretely expected.
– Rubrics are used for “high stake assessment” and “formative assessment” (Arter & McTighe, 2001; Busching, 1998; Perlman, 2003).
• Rubrics focus on the relationship between competencies, criteria, are organized along mastery levels (Morgan, 1999).
29
![Page 30: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/30.jpg)
Rubrics: indicator-based assessment
• Assessment objective– Criteria
• Enriched with indicators in terms of observable behavior
• Limited number of indicators
30
![Page 31: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/31.jpg)
Critical thinking rubric
31
http://academic.pgcc.edu/~wpeirce/MCCCTR/Designingrubricsassessingthinking.html
![Page 32: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/32.jpg)
Assumptions about rubrics
• Larger consistency in scores (reliability).
• More valid assessment of complex behavior.
• Positive impact on related learning process.
32
![Page 33: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/33.jpg)
Critical issues• Adoption of this assessment approach is marred by
teacher beliefs about nature of evaluation (see e.g., Chong, Wong, & Lang, 2004);
• Also student beliefs (Joram & Gabriele, 1998)• Validity of the criteria and indicators (Linn, 1990), • Reliability of performance evaluation, e.g., when
multiple evaluators assess and score performance (Flowers & Hancock, 2003).
33
![Page 34: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/34.jpg)
Research about rubrics use• Review article 75 studies about rubrics usage• Jonsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability,
validity and educational consequences. Educational Research Review, 2, 130–144.– (1) the reliable scoring of performance assessments can be enhanced by the
use of rubrics, especially if they are analytic, topic-specific, and complemented with exemplars and/or rater training;
– (2) rubrics do not facilitate valid judgment of performance assessments per se. However, valid assessment could be facilitated by using a more comprehensive framework of validity;
– (3) rubrics seem to have the potential of promoting learning and/or improve instruction. The main reason for this potential lies in the fact that rubrics make expectations and criteria explicit, which also facilitates feedback and self-assessment.
34
![Page 35: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/35.jpg)
Conditions effective usage
• Check frame of reference for the rubric: tasks, objectives
• Train the users
• Use multiple assessors: interrater usage
• Developed by teacher and/or students!
35
![Page 36: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/36.jpg)
Development of rubrics
• Choose quality criteria: 4 to 15 statements describing the nature of a criterion
• Determine bandwidth to judge differences in quality related to the criterion: e.g., 0-5 or qualitative descriptors
• Elaborate descriptors for each bandwidth level: concrete operational terms
• Start from available student work!
36
![Page 37: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/37.jpg)
• Writing a fiction story
• Complex skill
• Criteria?
Rubrics: example
![Page 38: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/38.jpg)
Rubrics
![Page 39: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/39.jpg)
39
![Page 40: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/40.jpg)
Rubrics• Rubric: scoringstool voor een kwalitatieve inschatting van het niveau
van een authentieke of complexe activiteit.– Een rubriek bouwt verder op criteria die verrijkt worden met een schaal
waarop beheersingsniveaus zijn aangegeven
– Per beheersingsniveau worden standaarden aangegeven die die niveau weerspiegelen.
– Een rubric geeft zowel voor de lesgever als de student aan wat concreet verwacht wordt.
– Rubrics worden voor “high stake assessment” gebruikt en voor “formatieve toetsing” (in functie van leren).(Arter & McTighe, 2001; Busching, 1998; Perlman, 2003).
• Rubrics focus on the relationship between competencies-criteria, and indicators and are organized along mastery levels (Morgan, 1999).
40
![Page 41: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/41.jpg)
Performance assessment
• Rubrics focus on the relationship between competencies-criteria, and indicators and are organized along mastery levels (Morgan, 1999).
41
![Page 42: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/42.jpg)
Aanpak ontwikkeling rubric
• Kies criteria voor verwacht gedrag– 4 tot 15 statements die criterium beschrijven
• Bepaal bandbreedte die verschil in bereiken criterium weergeven– Bijv. 0-5 of kwalitatieve termen
• Werk een beschrijving uit voor elke waarde in de bandbreedte– Concreet observeerbare/vaststelbare kwalificaties
42
![Page 43: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/43.jpg)
43http://www.teach-nology.com/cgi-bin/presentation.cgi/
![Page 44: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/44.jpg)
44http://teachers.teach-nology.com/cgi-bin/research.cgi
![Page 45: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/45.jpg)
More information?• Overview of tools, examples, theory, bacdkground, research:
http://school.discoveryeducation.com/schrockguide/assess.html
• Critical thinking rubrics: http://academic.pgcc.edu/~wpeirce/MCCCTR/Designingrubricsassessingthinking.html
• Rubric generators: http://www.teach-nology.com/web_tools/rubrics/
• Interesting rubric sites: http://web.njit.edu/~ronkowit/teaching/rubrics/index.htm
• Rubric APA research paper: http://web.njit.edu/~ronkowit/teaching/rubrics/samples/rubric_apa_research.pdf
• K12 examples: http://school.discoveryeducation.com/schrockguide/assess.html
• General intro and overview:http://web.njit.edu/~ronkowit/teaching/rubrics/index.htm
45
![Page 46: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/46.jpg)
46
http://www.teach-nology.com/web_tools/rubrics/
![Page 47: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/47.jpg)
Statements about evaluation
• Learners should be trained to develop themselves such rubrics.
• Staff should collaborate in developing formal assessment and summative assessment rubrics
• Rubrics will help staff to be more concrete as to their teaching and learning focus
47
![Page 48: Innovation in Assessment and Evaluation](https://reader034.vdocument.in/reader034/viewer/2022051417/5681485a550346895db5678a/html5/thumbnails/48.jpg)
48
Innovation in Assessment and
Evaluation
Prof. dr. Martin Valckehttp://allserv.ugent.be/~mvalcke/CV/CVMVA.htm
Ghent University
Maputo July 2011