authenticity: turning theory into tasks · interactional authenticity cognitive processes in...

AUTHENTICITY:

TURNING THEORY INTO TASKS

Andrew Kitney & Sian Morgan

Direct method of teaching

Greater focus on speaking

assessment

Communicative competence

Interactional competence

Authenticity of interaction

Why did

authenticity

become

important

in assessment?

Oral interviews and authenticity

• Natural conversation?

• Construct underrepresentation (Savignon 1985, Young & Milanovic, 1998)

• Unequal power relationship

• Unscripted

Situational authenticity

real-life

task

test

task

• the degree to which features of real life tasks are reproduced in test tasks

Interactional Authenticity

cognitive

processes

in real-life

language

use

cognitive

processes

in test task

• the extent to which a test taker engages cognitive processes when completing a test task

‘A concern with situational authenticity requires test takers to respond to contexts which simulate ‘real life’ in terms of criterial parameters without necessarily replicating it exactly.

‘A concern with the interactional authenticity in a test requires that the cognitive processes candidates engage in while completing the test task are representative of, and offer adequate coverage of, the cognitive processes which would prevail in a natural (i.e. non-test) context.’ (Weir 2013: 97)

Accounting for these will allow test designers to build a validity argument for context validity and cognitive validity, part of Weir’s (2005) overall socio-cognitive framework for test validation.

Context validity and cognitive validity

Challenges for the testing organisation

• a range of authentic real life speaking and writing tasks, which elicit skills relevant to the test taker

• ensuring reliability of the scores important for the test user

• practicality of administration ( test taker and test provider

Speaking

• Criticism of Oral Proficiency Test

• Face to face

• Paired format tasks

• Range of skills – monologue, paired problem solving, developed discussion (cognitive validity)

Speaking

Cognitive challenges for the test taker

• Time pressure under normal circumstances – most utterances require an immediate response

• Planning time minimal, and monitoring + planning occurs often when TT engaged in own turn

• Interaction is dynamic, reactive and reciprocal

• Increased difficulty for lower level

(Field 2011 , p. 85/86)

How cognitive demands of tasks increase along CEFR levels

A2 B1 & B2 C1 & C2

Domain Personal Public Abstract

Control Controlled Semi-controlled Open-ended

Support Support Some support Autonomy

Interaction Transactional + Interactional + Evaluative

A2 interactive speaking task

A2 interactive speaking visual prompt

B1 interactive task prompt

B1 interactive speaking task speaking

B2 interactive speaking task

B2 interactive speaking task

Advanced interactive speaking task

• There is a cognitive dimension to writing

• There is a contextual dimension to writing

Writing

In the case of writing, what is involved?

The cognitive dimension -producing a text

a. macro-planning

b. organisation

c. micro-planning

d. translation

e. monitoring

f. revising

(Shaw & Weir 2007)

a. assess the task requirements and audience, determine goals and how to achieve them

b. order the ideas, identify relationships between them, decide on importance

c. focus on the part of the text about to be produced

d. translate ideas into linguistic form

e. monitor

f. make changes informed by the monitoring

In L2 writing, cognitive resources may be deployed differently, depending on the expertness of the writer

• Less proficient writers > more cognitive resources to local language issues

• More proficient writers > resources available for organisation and rhetorical presentation of ideas

The cognitive dimension

The cognitive dimensionProducing a text will involve – to a greater or lesser extent...

Macro-planning Assessing the task requirements, determining goals and how these might be achieved

Organisation Ordering ideas, identifying relationships between them; determining

which are central to the goals of the text, which secondary

Micro-planning Focuses on the part of the text about to be produced – at sentence level

Translation Translating ideas previously held in abstract form to linguistic form

Monitoring Checking mechanical accuracy of spelling, punctuation and syntax

Revising Making corrections or adjustments informed by monitoring. Can be after

each sentence, each paragraph or when whole text has been written.

(Shaw & Weir 2007)

The cognitive dimensionProducing a text will involve – to a greater or lesser extent...

Macro-planning Assessing the task requirements, determining goals and how these might be achieved

Organisation Ordering ideas, identifying relationships between them; determining

which are central to the goals of the text, which secondary

Micro-planning Focuses on the part of the text about to be produced – at sentence level

but also in relation to the paragraph and overall text

Translation Translating ideas previously held in abstract form to linguistic form

Monitoring • Checking mechanical accuracy of spelling, punctuation and syntax

• Examining the text to determine if it reflects the writer’s intentions

and fits with the developing argument structure of the text

Revising Making corrections or adjustments informed by monitoring. Can be after

each sentence, each paragraph or when whole text has been written.

(Shaw & Weir 2007)

‘Writing is neither simply words on a page nor an activity of solitary individuals…When we pick up a pen or sit at a word-processor we adopt and reproduce certain roles, identities and relationships to engage in particular socially-approved ways of communicating. So while every act of writing is in a sense both personal and individual, it is also interactional and social.' (Hyland 2002:48)

Writing as a social act taking place in a specifiable context (Weir 2005)

• Setting

• Audience

• Purpose

The contextual dimension

You have just finished a three-week study and work programme in an English-speaking country. You studied English language in the mornings and worked for a local company in the afternoons.

The programme organiser has asked you to write a report about your experience. In your report, you should evaluate the programme, explaining which part of the programme was more useful, and suggest changes you would recommend for next year’s programme.

Write your report.

Write your answer in 220 – 260 words.

Designing the task

How can the task take account of these dimensions?

You have just finished a three-week study and work programme in an English-speaking country. You studied English language in the mornings and worked for a local company in the afternoons.

The programme organiser has asked you to write a report about your experience. In your report, you should evaluate the programme, explaining which part of the programme was more useful, and suggest changes you would recommend for next year’s programme.

Write your report.

Write your answer in 220 – 260 words.

Designing the task

How can the task take account of these dimensions?

A2 C2

Linguistic

Cognitive

Socio-linguistic

Levels

Features of writing tasks at different levels

You have just finished a three-week study and work programme in an English-speaking country. You studied English language in the mornings and worked for a local company in the afternoons.

The programme organiser has asked you to write a report about your experience. In your report, you should evaluate the programme, explaining which part of the programme was more useful, and suggest changes you would recommend for next year’s programme.

Write your report.

Write your answer in 220 – 260 words.

• Functions of analysis and evaluation

• Having to select two of three concepts to cover and being able to discuss their merits relative to each other requires macro-planning, organisation and the ability to generate ideas to support an argument.

• Successfully arguing at C1 level for one concept over another is likely to involve some knowledge-transforming writing.

• use of source materials and role of reading into writing

• most academic writing is not timed: students can revise and edit, refer to sources, reflect on topic etc.

• most academic writing is judged primarily on the accuracy of content, rather than use of language.

opinions provided in the task input could be seen as a stand-in for longer texts, and allow writer to position his or her opinions in relation to them

compromise between authenticity and practicality

academic literacy versus general language proficiency

(Weigle 2002)

Possible limits to the authenticity of the task

(Cheung & Elliott 2017)

For more resources visit

Our YouTube channel Cambridge English TV for videos of our webinars and Facebook Live sessions

www.youtube.com/user/cambridgeenglishtv

Thank you!

You’re going to design a task.

1. Form groups of four.

2. Decide what your context is (test users and takers)

3. Decide which level you are going to target

4. Design a task to elicit either Speaking or Writing performance

Consider:

• What abilities within your skill you want to assess

• Context of use

• Criteria for assessment (if you have time)

5. Write the instructions and prompt

Task force challenge: Phase 1

Instructions for Phase 1 (15 minutes)

1. Form groups of eight (i.e. your group and one other)

2. Explain your task and how it is relevant to a real-life context (5 minutes each)

Phase 3: Revise task based on feedback (10 minutes)

1. Revise your task if necessary, considering any feedback you received

2. Copy task out onto yellow card and stick it on the wall for group task photo.

Phase 2: Share ideas with another group

(10 minutes)