diana laurillard: the conversational framework - an approach to evaluating e-assessment patterns in...

The Conversational Framework: An approach to evaluating e-assessment patterns in terms of learning theory

Diana Laurillard

London Knowledge Lab

Institute of Education

Scoping a vision of formative e-assessment

2

A common framework for representing learning theory

What does it take to learn (formal learning)?

What are the requirements for formative assessment?

Evaluating patterns of formative e-assessment

3

A common framework of representation

‘Instructivist’ - Teacher-focused

Teacher’s ideas

Learner’s ideas

Practice environment

Learner’s practice

Concepts

Actions

Adapt actions

Adapt Task practice environment

Reflect

Task goal

Answers

4

‘Constructionist’ - Practice-focused

Teacher’s ideas

Learner’s ideas



Concepts

Actions

Adapt actions


Reflect

Task goal

Reflect

Revisions

Feedback


Answers


5

‘Constructionist’ - Practice-focused

Teacher’s ideas

Learner’s ideas



Actions

Adapt actions


Task goal

Reflect

Revisions

Feedback



6

‘Social constructivist’ - Learner-focused

Teacher’s ideas

Learner’s ideas


Reflect

Other learner(s)

Ideas

Ideas


Reflect

Others’ practice


7‘Collaborative learning’ = ‘Social constructionism’?

Teacher’s ideas

Learner’s ideas



Actions

Adapt actions


Task goal

Reflect

Revisions

Feedback


Other learner(s)

Ideas

Ideas

Reflect

Others’ practice

Adapt actions

Draft outputs

Draft outputs


8

- A Conversational FrameworkWhat does it take to learn?

Instructivism + Constructionism + Social constructivism + Collaboration

Teacher’s ideas

Learner’s ideas



Actions

Adapt actions


Task goal

Reflect

Revisions

Feedback


Other learner(s)

Ideas

Ideas

Reflect

Others’ practice

Adapt actions

Draft outputs

Draft outputs

Concepts Answers

Reflect

QuestionsOutputs

9

The Conversational FrameworkAn attempt to draw on the learning theories developed over the

last century, and encapsulate them in a form that enables

educators to test conventional and digital methods against them

(Dewey, Piaget, Vygostsky, Bruner, Papert, Pask, Anderson, Marton, Biggs,

Brown, Lave, Wenger, Engestrom...)

10

Testing the value of digital learning

technologies against

the Conversational Framework

Lecture, Presentation, Book, Educational television, Audio… Powerpoint, Digital video, Animation, Podcast…

Teacher’s ideas

Learner’s ideas



Actions

Adapt actions


Task goal

Reflect

Revisions

Feedback

Other learner(s)

Ideas

Ideas

Reflect

Other learner(s)

Adapt actions

Draft outputs

Draft outputs

Concepts

Reflect

QuestionsOutputs

Learning through acquisition…

Libraries, Catalogues, Journals, Resource banks… Online resource, Digital library, Website, Search engine…

Teacher’s ideas

Learner’s ideas



Actions

Adapt actions


Task goal

Reflect

Revisions

Feedback

Other learner(s)

Ideas

Ideas

Reflect

Other learner(s)

Adapt actions

Draft outputs

Draft outputs

Concepts

Reflect

Questions

Learning through inquiry…

Tutorial, Seminar, Class discussion, Small group discussion… Online conferencing, Forum, Chat room, Discussion board…

Teacher’s ideas

Learner’s ideas



Actions

Adapt actions


Task goal

Reflect

Revisions

Feedback

Other learner(s)

Ideas

Ideas

Reflect

Other learner(s)

Adapt actions

Draft outputs

Draft outputs

Concepts Answers

Reflect

QuestionsOutputs

Learning through discussion…

Problem sheet, practice exercises, project work… Interactive simulation, Spreadsheet, Data analysis tool, Game…

Teacher’s ideas

Learner’s ideas



Actions

Adapt actions


Task goal

Reflect

Revisions

Feedback

Other learner(s)

Ideas

Ideas

Reflect

Other learner(s)

Adapt actions

Draft outputs

Draft outputs

Concepts Answers

Reflect

QuestionsOutputs

Learning through practice…

Answers

Outputs

Feedback

Answers

Outputs

Reflect

Laboratory, Small group work, Fieldwork, Workshop…

Teacher’s ideas

Learner’s ideas



Actions

Adapt actions


Task goal

Reflect

Revisions

Feedback

Other learner(s)

Ideas

Ideas

Reflect

Other learner(s)

Adapt actions

Draft outputs

Draft outputs

Concepts Answers

Reflect

QuestionsOutputs

Learning through collaboration…

CSCL environments, Wikis, Second Life, Multiplayer games…

Reflect

Essay, program, solution, design, product, performance…

Teacher’s ideas

Learner’s ideas



Actions

Adapt actions


Task goal

Reflect

Revisions

Feedback

Other learner(s)

Ideas

Ideas

Reflect

Other learner(s)

Adapt actions

Draft outputs

Draft outputs

Concepts

Reflect

Outputs

Learning through production…

Ppt, Program, Model, Website, Blog, Design, Digital video…

17

An integrated framework of our collective theories of

learning provides a powerful challenge to digital

technologies, showing that:

To support the learning process fully we need to integrate all the technological capabilities available

18

1. access explanations and presentations of the theory, ideas or concepts?2. ask questions about their understanding of the theory, etc, by providing the opportunity for answers from (i) the teacher, or (ii) their peers?3. offer their own ideas and conceptual understanding, by providing comment on them from (i) the teacher, or (ii) their peers?4. use their theoretical understanding to achieve a clear task goal by adapting their actions in the light of their understanding, or in response to comments or feedback?5. repeat practice, by providing feedback on actions that enables them to improve performance?6. repeat practice, by enabling them to share their trial actions with peers, for comparison and comment?7. reflect on the experience of the goal-action-feedback cycle, by offering repeated practice at achieving the task goal?8. discuss and debate their ideas with other learners?9. reflect on their experience, by having to articulate or produce their ideas, reports, designs, performances, etc. for presentation to their peers?10. reflect on their experience, by having to articulate or produce their ideas, reports, designs, performances, etc. for presentation to their teachers?

The Conversational Framework – Challenging a learning design

How does your pattern of learning activities motivate students to:

Shares practice attempt

19

Learner’s conception as practice = 20%

Learner’s conception =

10% teacher’s

Teacher’s conception

Other learner’(s) conceptions

The theory of the Conversational Framework - the learner’s developing experience

Teacher-designed task practice environment

Other learner’(s) conceptions as

practice

Offers answers, ideas

Questions, offers ideas

Feedback on action

Action to achieve goal


Adapts approach to task to current conceptionAdapt a Task practice

environment for learners’ needs

Reflects on feedback

Reflects on alternative practice

Reflects on learners’ practice

Hints, comments

Present concepts

Task goal

Revises action

Presents conception as product

Questions


20% teacher’s


40% teacher’s



50% teacher’s


Adapts approach to task to current conception



60% teacher’s

Discursive/theoretical level

Practice/practical level

“the best predictor of quality of output is the number of iterations”

Theory level

Practice level

Feasst: Formative assessment strategies

Strategy: clarifying and sharing learning intentions and criteria for success

Strategy: engineering effective classroom discussions and other learning tasks that elicit evidence of student understanding

Strategy: providing feedback (extrinsic/intrinsic) that moves learners forward

Strategy: activating students as instructional resources for one another, sharing outputs and ideas with peers

Strategy: activating students as the owners of their own learning, learner self-regulation;

20

21

Teacher’s ideas

Learner’s ideas



Actions

Adapt actions


Task goal

Reflect

Revisions

Feedback


Other learner(s)

Ideas

Ideas

Reflect

Others’ practice

Adapt actions

Draft outputs

Draft outputs

Hints/comments

Reflect

QuestionsOutputs

Moments of contingency Forms of feedback to the learner

Wiliam: Formative assessment conceptsConcepts

Task goal

22

Learner’s conception as practice


Other learner’(s)

conceptions

The Conversational Framework - the learners’ experience of formative assessment – collaborative version

Teacher-designed task

Other learner’(s)

conceptions as practice

Feedback on action



Adapts approach

to task

Adapt a Task practice



Reflects on

alternative practice


Revises action

Adapts approach

to task




Theory level

Practice level

Learner’s conception

Clarify learning intention

Engineering effective discussions, tasks and activities that elicit evidence of learning

Providing (intrinsic) feedback that moves learners forward

Activating students as learning resources for one another

Activating students as owners of their own learning


Aspects of formative assessment – Dylan Wiliam

Hints, commentsProviding (extrinsic) feedback that moves learners forward

Task goal

23

Learner’s conception as practice


Other learner’(s)

conceptions

The Conversational Framework - the learners’ experience of formative assessment – individual version

Teacher-designed task

Other learner’(s)

conceptions as practice

Feedback on action



Adapts approach

to task

Adapt a Task practice



Reflects on

alternative practice


Revises action

Adapts approach

to task




Theory level

Practice level

Learner’s conception

Aspects of formative assessment – Dylan Wiliam

Hints, comments

Clarify learning intention

Providing (intrinsic) feedback that moves learners forward


Providing (extrinsic) feedback that moves learners forward

dummy

1. access explanations and presentations of the theory, ideas or concepts?2. ask questions about theory, etc, by providing the opportunity for answers from the teacher, or their peers?3. offer their own ideas and understanding, by providing comment on them from the teacher or their peers?4. use their understanding to achieve a goal by adapting their actions in the light of understanding, or feedback?5. repeat practice, by providing intrinsic feedback on actions that enables them to improve performance?6. repeat practice, by enabling them to share their trial actions with peers, for comparison and comment?7. reflect on the experience of the goal-action-feedback cycle, by offering repeated practice at achieving the goal?8. discuss and debate their ideas with other learners?9. reflect on their experience, by having to articulate or produce their ideas, for presentation to their peers?10. reflect on their experience, by having to articulate or produce their ideas for presentation to their teachers?

24

The Conversational Framework – Challenging a learning design

How does your pattern of learning activities motivate students to:

TryOnceRefineOnce pattern

Lack of immediate feedback leads to fossilisation of errors; immediate feedback can hinder effective learning

“Feedback on any remaining errors is given, along with correct answer(s)”

“Students are posed questions to elicit answers that can contain multiple errors. A mark is given which contributes to a percentage of the total mark for the question, along with detailed - yet generic- feedback on the location and type of the errors.)”

“Students are permitted a second attempt in which to refine their answer. No further attempts are permitted.”

25



10% teacher’s


Applying the Conversational Framework to the TryOnceRefineOnce pattern

Teacher-designed task practice environment


Adapt a Task practice environment for learners’ needs

Feedback

Task goal

Revises action


20% teacher’s


40% teacher’s


“the best predictor of quality of output is the number of iterations”

Theory level

Practice level

Adapts approach to task to current conception

Gaps can be filled by using technology to:• Structure peer feedback to increase cycles of iterations• Provide computer-based feedback to increase practice iterations• Automate hints and comments in response to student actions

Technology (‘e’) adds speed of response, iteration, intrinsic feedback (processing), re-use, and orchestration of peer collaboration


26

Summary

The Conversational Framework can be used to evaluate and improve

patterns of formative assessment through use of technology against

established learning theory and the formative assessment criteria

established in the Feasst project.