Technology Enhanced Learning at University -How can learning enhancement be demonstrated?

Adrian Kirkwood & Linda Price

IET, The Open University

What does ‘enhancement’ mean?

Expression Technology Enhanced Learning (TEL) used frequently - but without a clear, shared meaning

‘Taken for granted’ – few explicit statements ‘Enhanced’ implies a value judgement What is being ‘enhanced’?

Teaching? Learning? Operational arrangements? How is ‘enhancement’ understood by teachers,

students, policy-makers, senior managers?

HEFCE e-Learning Strategy, 2009

Efficiency Existing processes carried out in more cost-effective, time-efficient, sustainable or scalable manner

EnhancementImproving existing processes and the outcomes

TransformationRadical, positive change in existing processes or introducing new processes

Benefits might be felt at three different levels, depending on the type of intervention:

Demonstrating ‘enhancement’?

What must be done to demonstrate an enhancement in learning?

Much depends upon the type of enhancement being sought

Interpreted in many ways - a contested issue Often difficult to discern from published accounts

of innovations or interventions Need to reveal the underlying assumptions.

Technology as Agent

The potential agent for change is often assumed to be the technology itself (i.e. device and/or software)

If teachers use and/or get their students to use a particular technology, this will in and of itself improve student learning.

5

Teacher as Agent

Less frequently, the main agent for change is seen as the teacher, through what they are trying to achieve by using a technological tool.

What matters is how teachers – as agents – have chosen to design teaching & learning with technology, using appropriate tools to achieve significant outcomes.

6

Literature ReviewA review of published articles (2005-2010) of evidence-based learning and teaching with technology

The Abstracts were scrutinised to ensure that they fulfilled the following criteria: Use of technology for specific teaching and/or learning

purposes in higher education; Use of technology for specific teaching and/or learning

purposes associated with one or more particular courses/modules or groups of students;

Provided some form of evaluative evidence of the impact of the technology use described.

7

Literature Review

These criteria enabled us to eliminate articles that were wholly or primarily about technology innovations in schools students’ attitudes to and use of technologies in

general plans for uses of technology that were yet to be

introduced with students the generalised or idealised potential or affordances of

technologies in education

8

Literature Review After applying the criteria, 47 unique articles and

papers were selected for review. Variations found in terms of ‘enhancement’:

Differing conceptions of ‘enhancement’ being used by teachers / researchers

Different methods employed to demonstrate enhancement

Differing forms of evidence considered necessary to demonstrate achievement of enhancement

Problems with the appropriateness and adequacy of evidence in relation to conception of enhancement.

9

Literature Review

Various research approaches had been used, from quasi-experimental to case studies.

The approaches and methods used to collect data provide some indications of the ways in which the practitioners/researchers involved conceptualise learning and teaching in higher education and the types of evidence that they consider to be salient.

10

Categories of Intervention - 1

Replicating existing teaching practices

a. An element of conventional teaching replicated and delivered to students via some form of technology

b. A comparison of different technologies for delivering the same teaching

Categories of Intervention - 2

Supplementing existing teaching

a. Making available versions of existing teaching that students can access and use whenever they want

b. Developing additional learning resources or tools

Categories of Intervention - 3

Transforming the learning experience

a. Re-design of learning activities to promote active learning

b. Effective use of TEL activities for richer learning

1. Replicating existing teaching practices

Nature of ‘enhancement’ sought often imprecise ‘Comparative study’ method often used (i.e. only

the means of delivering teaching changed) Learning gains usually confined to test or

assessment scores – the acquisition and retention of knowledge (i.e. quantitative view of learning)

Positive student (and teacher) attitudes sought

2. Supplementing existing teaching

Nature of any ‘enhancement’ often imprecise – often linked to flexibility for students

Self-report student attitudes & experiences Problems of interpretation The 4-stage model of evaluation (Kirkpatrick, 1976)

Some comparative studies (how appropriate?) ‘Enhancement’ often taken as a quantitative change

rather than a qualitative improvement (e.g. Säljö, Perry) Type of test and assessment used not always sensitive

to nature of enhancement sought

The 4-stage model of evaluation

Reaction to what degree participants react favourably

Learning to what degree participants acquire the intended knowledge,

skills, attitudes, confidence and commitment Behaviour

to what degree participants apply what they learned to their situation

Results to what degree targeted outcomes are achieved as a result

Effectiveness is best evaluated at four progressively challenging levels (Kirkpatrick, 1976)

3. Transforming the learning experience Interventions often responded to identified

learning / teaching issues Often associated with achieving qualitative

changes in student outcomes Qualitative analysis of individual / group

interviews about processes and products of learning activities

Scrutiny of student-generated materials / presentations for richer outcomes

But … technology or pedagogy?

When the student learning experience had been changed, what was the main driver use of technology? redesign of the learning and teaching approach and

goals? both?

Teachers’ differing conceptions of teaching and learning with technology are an important influence (Kirkwood & Price, 2012)

Differing conceptions of teaching

Perceptions of Academic Context

Conceptions of Teaching

Approaches to Teaching

Conceptions of Teaching & Learning

with Technology

Perceptions of Technological Context

Approaches to Teaching &

Learning with Technology

Review of Findings There remains a considerable lack of clarity –

and little consensus – about what constitutes an enhancement in student learning.

Considerable differences are expressed about the form(s) of evidence that are necessary to demonstrate ‘enhancement’.

A clearer articulation of what is meant by ‘enhanced learning’ in higher education might help advance the effectiveness of TEL.

20

21