Lecturer

perspectives of

using technology

for student

feedback LYDIA ARNOLD

Rationale

The feedback challenge (Achilles heal)

Use of technology - uncritical acceptance?

Staff voices in the discourse

Harper Adams' own feedback dilemmas

Contribution to the Ed Dev community

Feedback: Definitions and features

Formal vs informal (Kahu, 2008)

Formative vs summative

Self, peer, automated

Dialogic, cyclical vs transmission (See Nicol, 2010; Barker & Pinard, 2014)

Emotional process (e.g. Pekrun, Cusack, Murayama, Elliot, & Thomas, 2014)

Market forces: Furedi (2009) feedback as flattery

Culturally routed (Chen, May, Klenowski, & Kettle, 2014)

Feedback as an extension of pedagogic values (Ellery, 2008)

"Information given on a performance which the aim of creating improvement"

Literature

The problems with feedback (not clear, legibility, mark attribution, code and

annotation and more!)

Varied responses (more of the same) (Robinson, Pope, & Holyoak, 2013).

Emphasis on student views (Evans, 2013) – 7% on lecturer perspectives

Lecturer views as an appendage

Technology as novel

The hope of technology

The risk of romantisizing (Clapham, 2012)

Research on technology by the enthusiastic few

More literature

Research as benefit conflation (Shelton, 2014)

Technology and the Halo effect (Lunt & Curran,2013)

A hunch about the impact of technology (shared by Watkins et al,

2014)

Existence if feedback orientations (Tang & Harrison, 2011) – so what?

Ideas that technology choices for feedback were shaped by

professional destinations of students

Models of technology dissemination Practice landscape (Geoghegan, 1994)

Support and policy assumes homogeneity

Conditions favour the early adopter

Alienation of the mainstream

Absence of a compelling reason for adoption

Rogers, 2003 (original 1963)

Does little to explain or

understand; infers

technology is ‘good’;

derogatory language

Questions

Why do tutors turn to technology for feedback?

What are the influences on choice and use of technology?

How does practice develop?

What is the effect of using technology in feedback?

Methodology Critical realist approach

Neither just objective, nor just subjective

Stratification – three layers of reality

Structure and agency

Individuals with identity

Aim to explain and devise theoretical models

Does not seek to generalise

Methods Narrative interviews – empathy as key

Harper Adams, Staffordshire, Keele, Sheffield Hallam, Chester, Manchester Met.

Connecting analysis case by case

Portraits (Seidman, 2013) – available online

Categorising (themes) – Maxqda11

Combined – collocated themes, strong themes, common themes

Modelling See lydiaarnold.wordpress.com

Findings

Findings: Why use technology?

Triggers

External examiner feedback

Team choices

Change of delivery mode

Realisations about writing

Change in student group (international)

Workload increase and a search for efficiency

Convergence

Industry experience

Family members

Career stage

Own experience

Passion

Market and competitor position

Recognition of learning preferences

Two underlying factors to trigger

reflection on practice

Change

External interactions

Social practice / praxis

Feedback beliefs – academic

identity 1. Universal acceptance of feedback importance

2. Varied ‘power’ to influence the impact feedback (affects volume and type and tech choices)

3. Ideas vs structure (degree of negotiation varies according to power)

4. Limited confidence in power of officialised feedback influences choices

5. Professionalism (how much feedback is part if the role)

Challenges the idea that poor feedback is apathy (linked to academic identity and beliefs about feedback impact)

What beliefs relate to technology

Where it can make a difference to student use (how so? ….. Media,

access)

Where it can make a difference to upholding professional

responsibility

Feedback as important

“if we don't give feedback, we are failing as educators”

“I think it’s probably the most important thing. Because I think that, something I found lacking, and it’s not a criticism of having gone through the course here, it’s a criticism of being at four unis doing various things and feedback generally is pretty poor” (Anna).

“In my teaching it's, I've always considered it an important part of the teaching process, of the learning process of the students…To some extent I certainly think most of the students learn something from it, either what you write, or when they come and try and speak to you about an assignment, and you can try and explain what your feedback actually means”

Officialised feedback

“I think the reason why there is so much focus on feedback is due to

the NSS [National Student Survey] and quality assurance so I think

there is this push from above and I think perhaps the student and what the student wants is being forgotten and I sometimes think the

students don’t want the feedback”

Variable use

[How the students use feedback is] the million-dollar question. It's

variable. I know from speaking to students about this, some do really

value the feedback but it depends on how it's presented to them. And it depends on lots of other things like the mark they get, whether they

like the module, whether it's an area that they want to extend in future,

and they want to do really well in that module because, that's taking

them in a direction. So I think it's incredibly variable. There are some

students who, even if they get fantastic feedback, still wouldn't take

any notice of it. They're still only interested in the mark”

Influences on choice of tool

Efficiency and enhancement

Personal media preferences

Beliefs about student learning preferences of students

Emotional concern

Fit with existing practice (rubrics)

Fit with structure or ideas emphasis

Fractured practice: A double fault

line

Tensions occurred in relation to:

Use of technology and the alienation of those not involved

Perceived poor feedback irrespective of technology being used

Perceived poor feedback and a lack of willingness to use

technology to address specific issues

Perceived reticence to try new technologies for the benefit of

students

Workload imbalances associated with trying to develop and sustain

good practice

Academic identity – technology

Beliefs about being a professional academic

Prioritise other activities over investing time in learning to use

technology

A very positive feedback orientation, he just avoided using

technology production for fear of distraction

Pace of change in technology was seen as all consuming

Spreading practice

Acculturation

Office location, campus location

Matching (knowing the options)

Technology first – externality

Course team

Impact on practice

increase in feedback volume and legibility, student engagement

and consistency

consider their assumptions and practices related to feedback

Attribution of marks

Information overload

Rubrics and the top end

Frustration

Key Case differences

Unit of the course team for managing feedback

Feedback in design

Quality assurance requirements

Technology as normal - “I don’t do anything else on paper. I’m not used to actually the physical hold of the pen anymore and I never

have ever since computers existed, or even typewriters. I never have

been able to compose with a pen. I have always written on a

machine”

Course team

1. Programme level operation:

a) have their practice recognised as a core part of pedagogy

b) exploit the particular benefits of tools in a way that is coordinated

from both a pedagogic perspective and a workflow perspective

c) strengthen meaningful networks for the development, discussion,

sharing and refinement of practice

1. Systems thinking about change (But systems which complement academic identity)

2. Forming ways to work with (accept or change) different academic identities

3. Consider the fault lines in practice (tech and feedback)

4. QA and clarity

5. External contact increases

6. STOP considering efficiencies without consideration of quality issues

7. Change as a trigger for reframing practice

8. Exposure to CPD in a way that fits with identity

9. Feedback induction and space management

Next

Spaces and feedback practice

Structure and ideas – orientations

Critical realism as a methodology

Academic identity and the role of feedback

Want to know more?

Full report available at http://repository.liv.ac.uk/2014121/

Portraits available at

https://lydiaarnold.wordpress.com/2014/10/10/using-technology-

for-student-feedback-lecturer-perspectives-in-their-words/

Internal related case study (with Carl Kennard) coming soon in The

International Journal of Assessment and Evaluation