effective assessment designs

Designing and carrying out effective assessment

Masterclass, AHE conference

Birmingham June 24, 2015

David Carless

University of Hong Kong

The University of Hong Kong

Overview

1. Two assessment task designs

2. Your views on good assessment design

3. Elaboration of key principles

4. Implications

The University of Hong Kong

The University of Hong Kong

HISTORY CASE STUDY

Making History course

• Foundation level, year 1, 110 students

The University of Hong Kong

Making History LOs

Critical engagement with representations of past; interpret connections between past & present;

The University of Hong Kong

History Assessment

Fieldwork report (30%): Museum visit

Individual project (40%): draft 10%, final 30%

Participation (30%):

tutorial participation 15%

short weekly written responses 15%

The University of Hong Kong

Museum report

1000 words or podcast

Issues: key messages; use of space; coverage and omissions

The University of Hong Kong

Sample Project (1)

Relationship between film & history

The University of Hong Kong

Sample Project (2)

Walking tour of sites of Hong Kong identity

The University of Hong Kong

Project stages

• Topic brief

• Draft (10%)

• Final version (30%): 3000 words

The University of Hong Kong

The University of Hong Kong

LAW CASE STUDY

Tort Law

• Core 1st year course: 180+ students

The University of Hong Kong

Tort Law LOs

• Explain common torts and their functions

• Think critically about legal issues

• Analyse tort issues

The University of Hong Kong

Tort Law Assessment

Reflective Media Diary (20%)

1st sem test (20%) or test 10% + photo essay 10%

Final Exam (60%)

or 40% + 20% research essay

The University of Hong Kong

RMD

• identify tort law-related events in local media; track developments; provide legal analysis;

• portfolio-like: collecting, selecting, editing and analyzing material over time.

The University of Hong Kong

RMD features

Steady student engagement

Promotes reading habits

Incomplete, authentic problems

Workload friendly for teachers

The University of Hong Kong

Photo essay

Students photo tort law situations

Write a short legal analysis

The University of Hong Kong

Photo essay features

Student-identified legal issues

Creative …. Iterative

Not easy to grade reliably

Minor option so might be ignored

The University of Hong Kong

Use of exemplars

Sharing of samples to illustrate expectations

The University of Hong Kong

Same-day exam feedback

Discussion immediately after exam

The University of Hong Kong

Same-day exam feedback

Supplemented by online discussion

Novel angles may be added to marking scheme

The University of Hong Kong

SHARING

What do you see as the key features of good assessment task design?

The University of Hong Kong

Learning-oriented assessment

A major priority in all assessment should be to promote effective student learning processes (Carless, 2014)

The University of Hong Kong

The University of Hong Kong

Productive assessment task design

Student self-evaluative capacities

Student engagement with feedback

Learning-oriented assessment framework

TASK DESIGN PRINCIPLES

The University of Hong Kong

(1) Integrated with instruction and ILOs

The University of Hong Kong

Constructive alignment

(2) Spreading student effort/sustained engagement

The University of Hong Kong

(3) Mirroring real-life uses of the discipline

The University of Hong Kong

(4) Integrated and coherent

The University of Hong Kong

(5) Incorporates feedback dialogues

The University of Hong Kong

(6) Supports student in understanding quality

The University of Hong Kong

(7) Flexibility & choice

The University of Hong Kong

(8) Encourage deep approaches to learning

The University of Hong Kong

Programme-based approaches

Cumulative … integrated … coherent

The University of Hong Kong

Issues & Challenges

The University of Hong Kong

Student response

Students interpret assessment tasks based on their own preferences, experiences and motivations

The University of Hong Kong

Double duty

Competing priorities in

assessment

Reliability, Fairness,

Workload, QA etc

The University of Hong Kong

Lack of autonomy

Many teachers lack individual autonomy: pulled in different directions by competing assessment priorities (James, 2014)

The University of Hong Kong

Limited incentives

Good assessment design: difficult, needing effort but may not be rewarded (Norton et al., 2013)

The University of Hong Kong

Teacher X factor

Teacher will & determination to overcome barriers and strive for

… learning-oriented assessment

The University of Hong Kong

Trust in teachers Innovative assessment thrives when teachers are trusted (Carless, 2009)

The University of Hong Kong

Assessment literacy

Need for development in assessment (for learning) literacy of university teachers (cf. Price et al., 2012)

The University of Hong Kong

Final words

“Being a good teacher is very much about being a good designer of tasks” (Knight, 2002)

The University of Hong Kong

THANK YOU

The University of Hong Kong