closing the feedback loop

Assessment & Assessment & feedback:feedback:

rediscovering that rediscovering that the feedback loop the feedback loop

works!works!Peter Hartley, University of Bradford

p.hartley@bradford.ac.ukhttp://www.brad.ac.uk/educational-development/aboutus/

team/Full_details_27414_en.php

This inputThis input• Exploring the meaning of ‘feedback’• Different examples of innovation

o Audio feedback.o Video and audio combinations.o Clickers and response systems.o Adaptive systems. o Integrating systems and mobile applications.

o Where to go next?o Feedback-embedded curriculum design.o Programme-based assessment (PASS project) o Defining/comparing assessment environments (TESTA

project) o Exploring the assessment/identity link (following Higgins et

al).

Assessment is a Assessment is a problem:problem:

feedback is just part of feedback is just part of it.it.

• See the PASS Project Issues Papero Please comment/feedback and use.

• http://www.pebblepad.co.uk/bradford/viewasset.aspx?oid=260486&type=file

• Would highlight:o Assessment ‘drives and channels’o What/why are we measuring: the‘slowly learnt’problem.o Limitations of grading (e.g. marks are not numbers).o Implications for course structures/regulations

o .

An example to start …An example to start …

59% Excellent.

• This was the only tutor comment on a student assignment. How do you think the student reacted and felt?

The meaning of The meaning of feedbackfeedback

• Cannot we ‘recapture’ the ‘original’ meaning of feedback: enabling self-correcting behaviour towards a known goal.

• This means rediscovering the ‘feedback loop’ whereby information must be ‘fed back’ so that it:• relates to the goal.• is received.• is correctly interpreted.• enables corrective action.

cf. the work of Royce Sadler in Higher Education, e.g. http://www.northumbria.ac.uk/sd/central/ar/academy/cetl_afl/earli2010/themes/rsadler/

Assessment: multi-Assessment: multi-purpose & multi-purpose & multi-

audienceaudience

Example 1: audioExample 1: audio• The ASEL project

o led by Bradford with Kingston as partner.o various uses of audio, including feedback, in

different disciplines.

• Noted:o Technology is now easy and accessible.o Positive student reactions.o Different tutor styles and approaches.o A different form of communication?o Serendipity – e.g. feedback stimulated podcasts.

ASEL main conclusionASEL main conclusiono … audio is a powerful tool, providing opportunities

for personalising learning, promoting greater student engagement, and encouraging creativity. In introducing audio into their practice, lecturers were required to rethink their pedagogical approaches and learning design, adopting new and innovative ways to enable students to be more actively involved in the learning process. It allowed lecturers to provide more personal and richer feedback to students, and increased the level of interaction and dialogue amongst students and between students and lecturers. (Stewart and Dearnley)

Example 2:Example 2:audio and videoaudio and video

• Growing number of examples.o ALT/Epigeum Awards 2010: see the

ALT Open Access Repository

• See the winning entry by Read and Brown from Southampton:o Organic Chemistry.o Use of tablets to show solutions and working.

See at: http://www.soton.ac.uk/chemistry/media/ALT/

o Focus on self-assessment.

Example 3:Example 3:clickers are comingclickers are coming

• Student Response Systems at the moment?o They work … they can change staff and

student behaviour and performance.

• Buto can be cumbersome and fiddly.o setup time.o need strong commitment and support (e.g. see

experience at Exeter Business School).

Example 3:Example 3:clickers are comingclickers are coming

• Student Response Systems in the future?o They will radically change staff and student

behaviour.o They will be flexible and easy to use.o They will be on the student’s own device!

Example 4:Example 4:adaptive systemsadaptive systems

• PBL with consequences – you get immediate feedback on the consequences of your decisions.o e.g. The G4 project at St George’s

• http://www.generation4.co.uk/

• Adaptive assessmento e.g. the work of Trevor Barker

• https://uhra.herts.ac.uk/dspace/bitstream/2299/1726/1/901864.pdf

Example 5a:Example 5a:integrating systems: integrating systems:

CAACAA• IT4SEA project at Bradford

o New 100-seater facility.o Thin client technology.o QMP as University standard for summative

assessment.o Procedures agreed with Exam Office.o Design of room (available as cluster outside

assessment times)o Teaching potential.

The main The main CAA room at BradfordCAA room at Bradford

And the growth …And the growth …

Example 5b:Example 5b:integrating integrating applicationsapplications

• Use of mobile technologyo e.g. CampusM at Bradford:

• http://www.campusm.com/• http://www.techrepublic.com/software/university-of-bradford-about-

uob-10-mobile/2194295?tag=content;selector-1

• Integrating different technologieso e.g. Clive Barker et al:

• http://caa.ecs.soton.ac.uk/Papers/Barker-CAA2010.pdf

Where to go next?Where to go next?o Feedback-embedded curriculum design

(see recent/ongoing JISC Programmes).o Programme-based assessment (PASS). o Defining/comparing assessment

environments (TESTA project). o Exploring the assessment/identity link

(following Higgins et al).

Programme-based Programme-based assessment: PASS assessment: PASS

• NTFS group project over 3 yearso development and investigation leading to

pilots and implementation

• Consortiumo Led by Bradfordo 2 CETLs – ASKE and AfLo Plus Exeter, Plymouth and Leeds Met.

TESTA projectTESTA project• NTFS group project with 4 partners:

‘aims to improve the quality of student learning through addressing programme-level assessment.’

• starting from audit of current practice on nine programmes: osurveyed students using focus groups and AEQ – Assessment Experience Questionnaire – Graham Gibbs et aloalso using tool to identify programme level ‘assessment environments’ (Gibbs)

Assessment Assessment environment and environment and

impactimpact• Interim findings from TESTA

o variety of assessments can cause problemso Issues over understanding assessment criteria,

marker variation, and feedbacko variation across programmeso QA ‘myths and traditions’ can get in the way

The need for strategyThe need for strategy• An example finding from Gibbs

o ‘greater explicitness of goals and standards was not associated with students experiencing the goals and standards to be clearer’

• And what did make a difference?

The need for strategyThe need for strategy• An example finding from Gibbs

o ‘greater explicitness of goals and standards was not associated with students experiencing the goals and standards to be clearer’

• And what did make a difference?o Formative-only assessmento More oral feedbacko Students ‘came to understand standards

through many cycles of practice and feedback’

Typical student concerns Typical student concerns (based on PASS)(based on PASS)

• perceptions of ‘the course’ variable.• assessment experienced as ‘fragmented’.• anxieties re move to more integrated

assessment – perceived risk in terms of performance.

• Concerns about feedback and timing.

An example from An example from PASS: Peninsula PASS: Peninsula Medical SchoolMedical School

Includes:•four assessment modules that run through the 5 year undergraduate medical programme and are not linked directly to specific areas of teaching•focus on high-quality learning (Mattick and Knight, 2007)

Will PBA be the Will PBA be the ‘effective assessment ‘effective assessment

strategy’strategy’

And finally …And finally …assessment/ identity assessment/ identity

interfaceinterfaceStudents as ‘conscientious consumers’

(Higgins et al, 2002).But:• personal identity as ‘mediator’.

o e.g. apprentice (‘feedback is useful tool’) cf. victim (‘feedback is another burden’).

So need to change the mindsets of some students.

And finally finally …And finally finally …some other contactssome other contacts

PASSo Project Manager: Ruth Whitfield

r.whitfield@bradford.ac.uk

ASELo Project Manager: Will Stewart

w.stewart@bradford.ac.uk

• CAA (building on IT4SEA)o Project Manager: John Dermo

j.dermo@bradford.ac.uk