118613 assessing speaking presentation slides

8/9/2019 118613 Assessing Speaking Presentation Slides

1/72

Assessing SpeakingAn approach grounded in theory

and practice

Evelina D. Galaczi

Jill Buggey


2/72

Who are you?A. Im a teacher.

B. Im the co-ordinator/director of studies.

C. Im a learner.

D. Im a parent.

E. Im a researcher.


3/72

Where are you?

Cambridge ESOL exam centres


4/72

Assessing SpeakingAn approach grounded in theory

and practice

Evelina D. Galaczi

Jill Buggey


5/72

5

Cambridge AssessmentA leading authority

Influencing thinking through publishing

and consultancy

Leading-edge and operational research onassessment in education

Not for profit

150 yearsexperience

Delivering world-class assessment

Part of Cambridge University

Delivering Assessment

Influencing Thinking


6/72

Our reach Los AngelesMexico City

So PauloBuenos Aires

St Petersburg

Moscow

Warsaw

Berlin

Cambridge

Paris

Madrid

Bologna

Naples

Athens

Istanbul

Dubai

Pune

DelhiChennai

Shanghai

Guangzhou

Beijing

Seoul

Taipei

Ho Chi Minh City

Kuala Lumpur

Sydney

Cambridge ESOL exam centres

Office, Staff or

Representative

2,700test centres

50,000preparation centres

4 mcandidates

130countries

18,000Speaking Examiners

700Team Leaders

43Regional Team Leaders

23Professional Support

Leaders


7/72

Our task in this session to overview different speaking test formats

to explore paired speaking tests in terms

of research and practice

to explore implications for the classroom


8/72

How can speaking ability be assessed?

Direct formats

Semi-direct formats


9/72


Direct formats

interaction with a liveexaminer:

one-to-one

paired

group

scored by human raters


10/72


Semi-direct formats

computer-based:test taker speech iselicited by a machine

scored by human ratersor machine scored

Direct formats

interaction with a liveexaminer:

one-to-one

paired

group

scored by human raters


11/72

Why direct assessment of speaking? Direct speaking tests focus on production

and interaction

Computer-based testing lacksco-constructionand interaction


12/72

No best way to test speaking each format has its unique advantages

and limitations

each format is suitable for differentpurposes


13/72

Traditional approach to speaking

testsThe direct individual speaking test format

Examiner Test taker


14/72

Issues with the individual speaking test

format

More restricted tasks and types of interaction Unequal rights between examiner and candidate

Unbalanced power distribution

Examiner Testtaker


15/72

Whats the solution?


16/72

What about a paired format?

Test Taker A

Testtaker B

Interviewer Assessor

Test Taker A

Interviewer

Test taker A

Interviewer


17/72

Positive

effects of

paired

tests


18/72

Positive

effects of

paired

tests

Reflect theclassroom


19/72

Positive

effects of

paired

tests


More pairwork


20/72


21/72

Positive

effects of

paired

tests


More pairwork

Broader rangeof language

Moreevidence oflanguage

proficiency


22/72

Positive

effects of

paired

tests


More pairwork



proficiency

Allow moreinteractionalsymmetry


23/72

Potential

issuesReflect theclassroom

More pairwork



proficiency



24/72

Potential


More pairwork



proficiency


Complexity and

uncontrolled

variables


25/72

Potential


More pairwork



proficiency


Complexity and

uncontrolled

variables

Individual

scores based

on jointconstruction


26/72

Is this a fair test?

What insights does the available

research provide?


27/72

True or false? In a paired format ...

6. There is more variety of speech functions

5. The effect of background variables is higher

3. There is more collaborative development of topics

2. The test taker has more opportunities for initiating

topics

4. There is more examiner talk

1. The test taker has more turns


28/72






topics


1. The test taker has more turns True


29/72






topics



True


30/72






topics



True

True


31/72






topics



True

True

False


32/72






topics



True

True

False

True


33/72






topics



True

True

False

True

True


34/72

Interaction in paired formats more complexcognitive and strategic

processes


35/72


36/72

Interaction in paired formats more complexcognitive and strategic

processes

which elicit a richer range of language

which allows for better inferences to be

made about a candidates proficiency in

wider real-life contexts


37/72

Effect of background variablesInterlocutor effect (OSullivan, 2002)The effect of background variables on performance from other

candidates and examiners

proficiency level personality

acquaintanceship

cultural background

age gender

status, etc.

Note: both in individual and paired tests


38/72

How influential are these variables?

Effect on discourse?

Effect on scores?


39/72


40/72

Effect of background variablesproficiency level

personality

acquaintanceship

some influence on conversation, but

impact is small


41/72


personality

acquaintanceship


impact is small

interplay between task type andpersonality


42/72


personality

acquaintanceship


impact is small

interplay between task type andpersonality

interplay between acquaintanceship,

gender and cultural norms


43/72

Variability related to different characteristics

of conversational partners is all that

happens in the real world. And so they arethings that we should be interested in

testing. (Swain, cited in Fox, 2004)

What should we do about it?


44/72

Paired speaking tests present

challenges, but mostly opportunities

(Brooks, 2008)

What should we do about it?


45/72


46/72

The Cambridge approach a range of task types, only one of which

is paired:

benefit from paired format

control possible limitations

use of analytical and global criteria andscales


47/72

The Cambridge approach a range of task types, only one of which

is paired:

benefit from paired format

control possible limitations

use of analytical and global criteria andscales

training and monitoring of examiners


48/72

Which of these Cambridge ESOL tests

do not use a paired format? Cambridge English: Proficiency (CPE)

Cambridge English: Advanced (CAE)

Cambridge English: First (FCE)

Cambridge English: Preliminary (PET)

Cambridge English: Key (KET)

Cambridge English: Young Learners (YLE)

IELTS Cambridge English: Business Certificates (BEC)

Why not?


49/72

Which of these Cambridge ESOL tests

do not use a paired format? Cambridge English: Proficiency (CPE)

Cambridge English: Advanced (CAE)

Cambridge English: First (FCE)

Cambridge English: Preliminary (PET)

Cambridge English: Key (KET)

Cambridge English: Young Learners (YLE)

IELTS Cambridge English: Business Certificates (BEC)


50/72


51/72

What does this mean for classrooms?


52/72


53/72

What does this mean for classrooms?

students need to be supported with

appropriate materials

suitable classroom practice to developcollaborative skills


54/72

What is meant by collaborative skills?

Collaborativeskills


55/72

Topic development

Collaborativeskills



56/72

Topic development Turn-taking

Collaborativeskills



57/72


Interactive listening andlistener support

Collaborativeskills



58/72


Speaker nominationInteractive listening and

listener support

Collaborativeskills



59/72

Cambridge English: First

Speaking Part 3 (collaborative task) B2 level

decision-making task based on a series of

pictures candidates talk together for about 3

minutes


60/72

Cambridge English: First

Speaking Part 3 (collaborative task)Id like you to imagine that a local caf wants to

attract more people. Here are some of the

suggestions theyre considering.

First, talk to each other about how successful

these suggestions might be.

Then decide which two would attract most

people.


61/72


62/72

Collaborative skillsextract 1

Ana Yeahthis one seems to be good, the terrace, but

maybe if you have a good weather

Giallo Yes, I dont know here in Cambridge

Ana Yeah, its difficult [both laugh]

Giallo ..and then if you. have just opened a caf or a bar

maybe its not agood idea to put some place , sorry,

some tables outside. I think its better to concentrate on

music orother ideas.

Ana Yeah, yeah .


63/72

Collaborative skillsextract 2Ana Yeah, yeah . What do you think about football

matches?

Giallo Depends on the target of the public that you are

interested in. I mean if you are interested in Ana Sports?

Giallo Yeah, if you are interested in attracting men its more

than good, its a perfect idea but I dont know. You can

tell! I mean would you go in a caf with a football match?Ana: Yeah, me not but most of the people I think maybe its

a good idea.


64/72

Collaborative skillsextract 3Ana But I think that the one that would attract... the most

people, I think, is the place where you can watch TV I

mean football matches or sports becauselots of people

are interested in that

Giallo Mmm

Ana and maybe the coffee free is also important

GialloYes

Ana Young people has not a lot of money, yeah

Giallo Yeah, I dont know about television. I think when you

begin, maybe its better to put like some comfortable

sofas becauseitsan example of your caf.


65/72

Examples of collaborative skills

Ana Giallo Both

agrees with

Giallo

initiates

discussion

turn-take well

asks Giallo

questions

invites Ana to

speak

dont interrupt

each other

develops the

topic by giving

reasons for her

decisions

moves

discussion

forward

listen and

respond to what

their partner

says


66/72


67/72

Developing collaborative skills

Areas to improve on How can your students

improve?

Turn takingInitiating discussion

Giving views and

opinions

Analyse clipsRecord your students

Practise useful

language

Raise awareness ofskills and language

needed


68/72


69/72


70/72


71/72

Summary of todays webinar

overview of different speaking test formats

exploration of paired speaking tests in

terms of research and practice implications for the classroom

resources available


72/72

118613 assessing speaking presentation slides

Documents