Ipsative assessment and motivation of distance learners

Gwyneth Hughes and Megan Crawford

2

Contents

1 Why does assessment need radical reform?

2 Ipsative assessment -what are the possible benefits?

3 Results of a study on ipsative feedback for distance learners

4 Recommendations

3

Why assessment reform?Problems with current assessment practices

– Dominance of summative assessment and grading over formative (Gibbs, 2006)

– Assessment can be demotivating (Broadfoot, 1996)– Feedback is not helpful for students so they don’t act (Lizzio & Wilson

2008)

Suggestions for improving feedback practice – More clarity over standards and criteria– More dialogue – Improved formative feedback (Nicol & Macfarlane-Dick, 2006)

4

What if an obsession with criteria and standards is at the root of the problem?

‘if we wish to discover the truth about an educational system, we must first look to its assessment …’ (Rowntree 1987)

5

Ipsative assessment as an alternative to criteria-referenced assessment

Personal best

6

7

What is ipsative assessment ?

An ipsative assessment compares current performance with a previous performance.

Ipsative feedback

Ipsative grades

8

Possible benefits of ipsative assessment• Improves the usability of feedback e.g. through closing feedback loops

– See handout• Links up generic feedback across modules in a course

• Improves self-esteem by rewarding progress

• Ipsative grades might motivate learners to act on feedback

9

Possible use with distance learners

• Can aid motivation at a distance as part of supportive open learning.

• Written feedback is particularly important to students learning wholly at a distance- can help them to respond in revisions and following pieces of work.

• The tutor can take on a coaching role in a very direct way and help their students personal response to feedback which at a distance may be even more tied up with emotions, identity and feelings of self-worth than F2F.

Study on ipsative feedback and learner motivation MA in Applied Educational Leadership and Management (AELM) is

delivered wholly online to a fixed cohort.

Our study looked at three tutors on the two core modules, where tutors had had specific training in giving good formative feedback.

Module had been commended by External Examiner for its high quality written feedback.

10

Tutor feedback profile

11

Criteria-related

Task-orientated

Criteria-related

Generic

Ipsative

Task-orientated

Ipsative

Generic

12

Examples of feedback

Typical comments comparing the work with assessment criteria and standards were:

“Have a look at the Assessment Criteria P. 37 to 39 of the Course Handbook: 1) Criteria A: you show a good level of understanding of the key issues and have begun to present extracts from the relevant literature..” (Tutor B)

“What then might have taken the assignment to the next grade level? … you might have engaged rather more in a critical analysis of material in this field.” (Tutor A)

13

Ipsative feedback: student views• Feedback was perceived as vague lacking clarity• Learners used grades and feedback to self-assess on making

progress but some would like more help e.g.

“I do think about my own progress but of course I am never entirely confident in my own judgment of this!”

“ if someone would read my first and second assignment and examine my progress as a paper writer and give me general comments about how I did or didn’t improve and what I needed to focus on that will be very helpful.”

Tutor feedback profile

14

Criteria-related

Task-orientated

Criteria-related

Generic

Ipsative

Task-orientated

Ipsative

Generic

Recommended profile

15

Criteria-related

Task-orientated

Criteria-related

Generic

Ipsative

Task-orientated

Ipsative

Generic

16

Ipsative feedback: tutor views

“…if one tutor supervises the whole process of an assignment from the draft to the final submission, then ipsative assessment can be included fairly easily”

Concerns were raised about needing staff development and that a new assessment approach might be time consuming.

17

Ipsative grades: student views

“I fully appreciate that my final grade will not be based on my efforts, motivation or progress but simply on the work I submit, and that students who seem to have made far less effort may end up with better final grades if their work is of a higher quality. That is the nature of academic qualifications and it should remain this way if the end qualification is to maintain its current status as a well-respected academic certificate.”

18

Ipsative grades: tutor views:

“ It (ipsative grading) is a radical proposal and…….It might motivate (lower achievers)…the only way I would see it working is if they get a grade for effort and for achievement. But I doubt that the effort would count towards the final mark. I haven’t had enough time to think about it.”

19

Recommendations

Ipsative feedback could be introduced both to motivate students and to reduce obsession with grades and criteria

Any ipsative grades would need careful explanation of the benefits

Ipsative assessment would need to be included in the assessment regime and not provide extra work

Assessors will probably need staff development and guidance

20

g.hughes@ioe.ac.uk

mcrawford@brookes.ac.uk

21

ReferencesBiggs, J. 2003. Teaching for Quality Learning at University. Buckingham: Society for Research into Higher education and Open University Press.

Broadfoot, P. 1996. Education, Assessment and Society. Buckingham: Open University Press.

Carless, D. 2006. Differing perceptions in the feedback process. Studies in Higher Education. 3, no. 2: 219-233.

Hughes, G. & Crawford, M. 2009. Challenging Higher Education: knowledge, policy and practice. Society for Research in Higher Education conference papers, Newport, Wales.

 Nicol, D. & Macfarlane,-Dick, D. 2006.

Formative assessment and self regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education 31 no. 2: 199-218.

Institute of EducationUniversity of London20 Bedford WayLondon WC1H 0AL

Tel +44 (0)20 7612 6000Fax +44 (0)20 7612 6126Email info@ioe.ac.ukWeb www.ioe.ac.uk