1

THE INTO EUROPE SERIES

IN IN-SERVICE TEACHER TRAINING

Presentation by Presentation by Györgyi EgyüdGyörgyi Együd

‘‘Into Europe - European StandardsInto Europe - European Standards in Language Assessment’in Language Assessment’ Conference ConferenceBudapest, February 9-10, Budapest, February 9-10, 20062006

2

WASHBACK What is the washback effect? What

studies have been conducted? Are there preparatory materials? How are teachers trained

(encouraged) to prepare their students for the exam?

(questions by Prof. Alderson)

3

COURSES AVAILABLE:Modern English Examinations Teacher

Training (60 hours) Module 1: Introduction to skill-based

language testing & assessment Module 2: Assessing Reading Module 3: Assessing Listening Module 4: Assessing Use of English Module 5: Assessing Writing Module 6: Interlocutor Training Module 7: Assessing Speaking

4

• Assessing Receptive Skills (30 hours)

• Interlocutor Training + Assessing Speaking Skills A2/B1 (30 hours)• Interlocutor Training + Assessing Speaking Skills

B2 (30 hours)

5

•   Assessing Writing Skills A2/B1 (30 hours)

•   Assessing Writing Skills B2 (30 hours)

6

The materials used on the courses

(tasks, rating scales, interlocutor's

frame, etc.) have all been piloted in real Hungarian schools with real Hungarian

students.

7

All of the training courses have gone through the accreditation

process of the Hungarian in-service teacher training

framework. They run live.

8

Basic training principles as presented in ’In-service teacher development: some basic principles’ in ELTJ 49/3 by Hayes, D

(1995)

‘All teacher development activities should be classroom-centred.’

(Hayes, 1995: 256)

9

Basic training principles

‘Teachers should be involved in the preparation of courses.’

(Hayes, 1995: 257)

10

Basic training principles

‘Trainers should themselves be teachers.’

(Hayes, 1995: 257)

11

Basic training principles

Training methodology should be largely task-based and

inductive.’ (Hayes, 1995: 257)

12

Basic training principles

‘Training / development sessions should value participants’ existing

knowledge.’ (Hayes, 1995: 258)

13

Basic training principles

Teacher development sessions should offer opportunities for

participants to share knowledge and ideas.’

(Hayes, 1995: 260)

14

The general training approach of the courses is based on

the cyclical process of experiential learning

(Kolb: 1984).

15

Stage 1: Concrete experience

Stage 2: Reflective observation

Stage 3: Abstract conceptualisation

Stage 4: Active experimentation

16

Course design

Each course cycle contains the following recurring elements:

a demonstration of suitable alternative assessment approaches & techniques

17

familiarising participants with all the possible task- and text types of the given skill

presenting piloted, calibrated, benchmarked examples from the Into Europe series with the results from the pilots

18

opportunities to actually do the sample tasks presented

opportunities to reflect on the teaching implications

19

reference to the teachers’ own classroom experiences, examining the possible problem points, sharing experiences, offering guidance, solutions

20

Participant feedback from an awareness raising course

‘Teaching and testing are becoming friends, walking in the classroom hand in hand, like learners and teachers, with washback sponge.’

‘I can't wait to use today's ideas at home, with my students.’

21

‘‘Now I'll be able to judge critically the

listening exercises in the course book that I use.’

‘Thank you for making me aware of even those "small things" (the tricky ones) in designing tests. I will definitely be more careful when writing test papers for my students.’

22

The desired impact of the training programmes

teachers will use better teaching and testing techniques: positive washbackteachers will be better-informed

about European good practice in language testing & assessment

23

teachers’ awareness will develop in terms of what skills can be tested and how they can be tested, and also how students can perform at these tests

as a result of the above, language learners’ language skills and test performance will improve

24

COURSE PARTICIPANTS ARE FAMILIARISED WITH GOOD PRACTICE

THROUGH standardised examination

procedures demonstrated by a Model Examination developed for training purposes

examiner training focusing on both objectively & subjectively assessed skills

the use of benchmarked performances in the training courses for speaking and writing skills

25

In the Hungarian context the model consisted of

five main phases:

1.1. selecting sample performances from selecting sample performances from the pilot examinationsthe pilot examinations

2.2. selecting judgesselecting judges3.3. home marking by judgeshome marking by judges4.4. live benchmarking exerciselive benchmarking exercise5.5. editing and standardising justificationsediting and standardising justifications

A model of benchmarking procedures was designed and piloted by the project team under the guidance of Prof. Charles Alderson (the advisor of the Project).

26

The The main purposemain purpose of the benchmarking of the benchmarking

exerciseexercise: to : to reach agreement on grades reach agreement on grades

using the using the analytic rating analytic rating scalesscales

developed by the Project Teamdeveloped by the Project Team..

Relating the performances to the Common Relating the performances to the Common

European Framework couldEuropean Framework could be a be a

supplementary exercise. supplementary exercise.

For this purpose the 9-point scaleFor this purpose the 9-point scaless ( (Overall Overall

SpokenSpoken / Written / Written Interaction Interaction) of the ) of the CEFCEF w wereere used. used.

27

The Use of Benchmarked The Use of Benchmarked

Performances in the Training of Performances in the Training of

AssessorsAssessorsThe benchmarks and justifications produced The benchmarks and justifications produced

by the judges in the benchmarking sessions by the judges in the benchmarking sessions

are used for supporting the pre-course tasks are used for supporting the pre-course tasks

and the face-to-face assessor training course. and the face-to-face assessor training course.

Benchmarked performance samples Benchmarked performance samples

illustrate candidate performance illustrate candidate performance

at different levels of the scalesat different levels of the scales..

28

In the face-to-face training phase,In the face-to-face training phase,

thethe benchmarks and benchmarks and

justifications are revealed to justifications are revealed to

course participantscourse participants

in different waysin different ways

at different stages of the training. at different stages of the training.

29

The training procedures developed by the Projecthave the following aims:

to provide future examiners with sufficient information about the model language examination they are going to be trained for (outline, task types, mode)

to familiarise future oral examiners with standard interlocutor behaviour

to familiarise participants with the main principles and procedures of assessing speaking and writing performances

According to the CEFAccording to the CEF ( (Guideline 4Guideline 4), ), ffutureuture examiners should examiners should undertake undertake

appropriate appropriate trainingtraining..

30

Further aims:to introduce the idea and practice of using analytic rating scales for assessing speaking and writing performances

to enable participants to develop the necessary interlocuting and assessing skills

to ensure valid and reliable assessment of live performances through standardisation

to equip trainees with transferable skills (there is a special need for this in Hungary)

31

The Outline of the Training The Outline of the Training ModelModel

Stage 1:Stage 1: pre-course distance learning

• self-study of an Introductory Training Pack

accomplishing the pre-course tasks (analysing and marking writing scripts or sample video performances)

32

The Introductory Training Pack

contains• An overview of the Speaking / Writing Model

Examination• Guidelines for interlocutor behaviour• Guidelines for assessor behaviour• Pre-course tasks• Self-assessment questions• Appendices: Benchmarks & Justifications for the

Sample Tests, Examples of Candidate Language, CEF Scales, Glossary of Useful Terms

33

The Outline of the The Outline of the Training ModelTraining Model

Stage 2: Stage 2:

Live training courses Live training courses

(a series of workshop (a series of workshop sessions) sessions)

34

Example 1: Example 1: Speaking – Interlocutor TrainingSpeaking – Interlocutor Training

discussing the experiences of the distance phasediscussing the experiences of the distance phaseanalysing video samples of both standard and non-analysing video samples of both standard and non-

standard interlocutor behaviour standard interlocutor behaviour standardisation of the administration procedure through standardisation of the administration procedure through

simulated examination situations (role plays)simulated examination situations (role plays)demonstrating alternative approaches (paired vs demonstrating alternative approaches (paired vs

individual tests)individual tests)

35

Role-play Cards for Part 1 (The Role-play Cards for Part 1 (The

Interview)Interview)

Candidate Candidate You are a shy, not very talkative candidate who tends are a shy, not very talkative candidate who tends

to wait for guiding questions. You often reply with one to wait for guiding questions. You often reply with one

or two short sentences only.or two short sentences only.

Interlocutor Interlocutor

You are the interlocutor who asks the questions of the You are the interlocutor who asks the questions of the

first part of the speaking test. You have to elicit as first part of the speaking test. You have to elicit as

much speech from the candidate as possible. Please much speech from the candidate as possible. Please

remember to ask the questions listed in the remember to ask the questions listed in the

Interlocutor Frame.Interlocutor Frame.

36

discussing the experiences of the distance discussing the experiences of the distance phase phase

introduction to assessing oral performances: introduction to assessing oral performances: modes and techniques of assessmentmodes and techniques of assessment

familiarisation with the analytic rating scalefamiliarisation with the analytic rating scalestandardisation of the assessment procedurestandardisation of the assessment procedurecomparing performances at different levelscomparing performances at different levels

Example 2: Speaking – Example 2: Speaking – AssessAssessor or TrainingTraining

37

Stage 3: a distance phaseStage 3: a distance phase

Practical application of the acquired skills:application of the acquired skills: mock tests conducted in the participant’s own environment.

The Outline of the Training The Outline of the Training ModelModel

38

ConclusionsConclusions

It is not impossible to become a trained examiner without formal trainingformal training.

Training should involve distance and face-to-distance and face-to-face elementsface elements as well to ensure that future examiners go through each and every phase of the difficult and complex standardisation process.

One training course is not enoughOne training course is not enough.

Only further practiceOnly further practice and monitoring monitoring examiner behaviourexaminer behaviour can ensure that candidates’ language abilitiy is assessed in a standard manner, and the assessments are valid and reliable.

39

WASHBACKAre there preparatory materials? How are teachers trained (encouraged) to prepare their students for the exam?

(questions by Prof. Alderson)

40

‘Teaching and testing are becoming friends, walking in the classroom hand in hand, like learners and

teachers, with washback sponge.’