Manipulating Task Complexity: its impact on L2 production across task types and modes

Roger GilabertMayya Levkina

Universitat de BarcelonaTBLT Conference

Lancaster September 2009

Thanks to GRAL at the University of Barcelona, Ministerio de Ciencia e Innovación – HUM2007-64302) and Generalitat de Catalunya – 2009SGR137).

L2

Task –based research into performance

and acquisition

PLANNING TIMETASK COMPLEXITY

TASK FAMILIARITY

Intro: context

INTERACTIONOther

Brennan, forthcoming

L2

Task –based research into performance

and acquisition

PLANNING TIMETASK COMPLEXITY

TASK FAMILIARITY

Intro: context

INTERACTIONOther

“ task complexity is the result of the attentional, memory, reasoning, and other information processing demands imposed by the structure of the task on the language learner.”

(Robinson, 2001:28)

TASK COMPLEXITY

Pedagogic task 1

REAL-WORLD TASK

Pedagogic task 2

Pedagogic task 3

Pedagogic task n

What may be the role of task complexity in a task-based syllabus?

Long, 2005

Balanced development of L2 production and acquisition

GOAL

Simple Complex

The predictions of the Cognition Hypothesis

More complex

along resource- directing variables

More complex

along resource- dispersing variables

Less fluent

More lexically and structurally

complex

More accurate

Higher interaction

Less fluent

Less lexically and structurally

complex

Less accurate

Higher interaction

(e.g. amount of preparation time, familiarity, or multi-tasking)

(e.g. the number of elements or the amount of reasoning)

Findings of the impact of task complexity on production along resource directing dimensions

Fluency decreases (Robinson, 1995; Rahimpour, 1997: Gilabert, 2005)

Increased lexical complexity (Robinson, 1995; Rahimpour, 1997; Gilabert, 2005)

No differences in structural complexity (Robinson, 1995; Rahimpour, 1997; Gilabert, 2005)

Higher accuracy (Robinson, 1995; Rahimpour, 1997; Ishikawa, 2001; Iwashita et al.; Gilabert, 2005)

+/- here-and-now +/- elements+/- reasoning

demands

Fluency decreases (Michel, Kuiken & Vedder, 2007; Robinson, 2001)

Increased lexical complexity (Michel, Kuiken & Vedder, 2007; Révész, forthcoming)

No differences in structural complexity (Michel, Kuiken & Vedder, 2007; Kuiken & Vedder, 2007; Kuiken, Vedder, & Mos, 2005) but Révész (forthcoming)

Higher accuracy (Michel, Kuiken & Vedder, 2007; Kuiken & Vedder, 2007; Kuiken, Vedder, & Moss, 2005) but Révész (forthcoming)

Fluency decreases (Niwa, 2001 )

Increased lexical complexity (Michel, Kuiken & Vedder, 2007; Révész, forthcoming)

No differences in structural complexity (Robinson, 2007) but Révesz (forthcoming)

Higher accuracy (Robinson, 2007; Gilabert, 2007) but Révész (forthcoming)

Goals and QuestionsGoal: to explore the impact of task cognitive

complexity accross task types and modes.

Questions:

1) What’s the relationship between general proficiency and performance?

2) Can differences in task cognitive complexity explain differences in performance in both monologic and dialogic tasks?

3) Are the effects of task cognitive complexity the same across task types?

Design Participants Statistical analysis, transcription, coding

3 task types

Repeated-measures design

Latin square design

9-point Likert scale affective questionnaire

42 English-L2 volunteers in monologic study50 in dialogic (25 dyads)

2 institutions (lower-/upper- intermediate)

X-Lex / Y-Lex vocabulary size test

Descriptive statisticsRepeated-measures ANOVANon-parametric tests

CA mode of CHILDES for transcription of 252 tasks in study 1 + and 150 tasks in study 2 (302 tasks)

Intrarater (97 %//97%)Interrater (91.5 %/90%)

Experimental design

Advertising Journalism P.R.

(task-based program at Communication Studies Department at Ramon Llull University,Barcelona Spain)

Typically they deal

with storyboards

for campaign

presentations

Task selection: Needs analysis

Journaslists often have

to find their ways in

unknown cities

In crisis management, scenario planning is

an important

aspect

TV/Cinema

Task 1: narrative

Monologic and dialogicOperationalization

Begin the story like this: TODAY Mr. and Mrs. Ropper are in bed. They’re trying to get to sleep but they can hear music coming from the apartment above theirs.

Begin the story like this: YESTERDAY Mr. Festenkroud was shopping at the supermarket. He was checking his shopping list and looking at prices. An employee was putting price tags on the products.

SIMPLE here-and-now

Visual presence (here)

Present tense (now)

COMPLEX there-and-then

No visual presence (there)

Past tense (then)

Tasks and operationalization of variables

Interactive, two-way, closed, convergent, split information narrative task

SIMPLEHere-and-now

•Visual presence (here)•Present tense (now)

COMPLEXThere-and-then

•No visual presence (there)•Past tense (then)

Tasks and operationalization of variables

Results: Affective perception questionnaire

I thought this task 1 2 3 4 5 6 7 8 9 I thought this task

was easy was difficult  

I felt frustrated 1 2 3 4 5 6 7 8 9 I felt relaxed

doing this task doing this task I did not do this 1 2 3 4 5 6 7 8 9 I did this

task welltask well This task was 1 2 3 4 5 6 7 8 9 This task was not interesting interesting I don’t want to 1 2 3 4 5 6 7 8 9 I want to do

moredo more tasks tasks like thislike this

(Based on Robinson, 2001)

Dependent Variable Narrative Map Fire chief

Difficulty ,372 ,013* ,006*

Stress ,765 ,513 ,079

Confidence ,552 ,067 ,005**

Interest ,371 ,262 ,912

Motivation ,775 ,842 ,530

Task 2: map task

Monologic and dialogicOperationalization

Simple

Few landmarks

Clearly distinguishable landmarks

One axis(lateral= right, left, straight)

Complex

Many landmarks

Similar landmarks

More axes(lateral – right, left, straight– vertical – up, down – sagittal – front, back).

Simple

•Few landmarks•Clearly distinguishable landmarks•One axis(lateral= right, left, straight)

Interactive, one-way, closed, convergent, split information map task

Tasks and operationalization of variables

Route marked Route unmarked

Complex•Many landmarks•Similar landmarks•More axes (lateral – right, left, straight– vertical – up, down – sagittal – front, back).

Interactive, one-way, closed, convergent, split information map task

Route marked Route unmarked

Tasks and operationalization of variables

Wayfinding is an important and complex task.

Landmark identification

Path selection

Direction selection

Abstract environmental overviews

Chown, E., Kaplan, S., & Kortenkamp, D. (1995)

Task 3: firechief task

Monologic and dialogicOperationalization

SIMPLE

Many resources

No particular roles

Few unconnected factors

COMPLEX

Few resources

Particular roles of characters

Intricately connected factors

SIMPLE

•Many resources•No particular roles•Few unconnected factors

Interactive, two-way, open, convergent, shared information decision-making task

COMPLEX

•Few resources•Particular roles of characters•Intricately connected factors

Interactive, two-way, open, convergent, shared information decision-making task

Complex problem-solving tasks are situations that are:

(1) dynamic, because early actions determine the environment in which subsequent decisions must be made

(2) time-dependent, because decisions must be made at the correct moment in relation to environmental demands; and

(3) complex, in the sense that most variables are not related to each other in a one-to-one manner. In these situations, the problem requires not one decision, but a long series, in which early decisions condition later ones.

Quesada et al. (2005)

Experimental design: production measures

Transcriptions were coded for:

- Fluency: Unpruned speech rate A Pruned speech rate

B Pauses x minute

- Structural Complexity: Sentence Nodes x AS-Unit.

- Lexical Complexity: Guiraud Index of Lexical Density- Accuracy: No. Of errors x 100 words

Repaired to unrepaired errors

Results: Question 11) What’s the relationship between general

proficiency and performance?

Moderately strong correlation between PROFICIENCY and PERFORMANCE

Proficiency x Performance in MONOLOGIC

Simple Story

Complex Story

Simple Map

Complex Map

Simple Firechief

Complex

Firechief

Rate A ,792** ,613** ,590** ,613** ,557** ,586**

Rate B ,839** ,758** ,643** ,669** ,667** ,695**

Pauses ,271 ,085 ,336* ,261 ,372* ,197

S-Nodes x AS Unit

,208 ,035 ,479* ,336* ,130 ,269

Guiraud Index

,718** ,650** ,618** ,562** ,760** ,716**

Errors x 100

words

-,725**

-,680** -,816** -,768** -,719** -,738**

Rep to unrep

,216 ,158 ,031 -,024 -,104 ,119

Results: Question 11) What’s the relationship between general

proficiency and performance?

Moderately strong correlation between PROFICIENCY and PERFORMANCE

Proficiency x Performance in

DIALOGIC

Simple Story

Complex Story

Simple Map

Complex Map

Simple Firechief

Complex

Firechief

Rate A ,417* ,329* -,021 ,233 ,226 ,143

Rate B ,391* ,362* ,375 ,335 ,334* ,121

Pauses -,177 -,143 ,084 -,093 -,007 -,074

S-Nodes x AS Unit

,216 ,179 -,071 ,038 ,162 ,094

Guiraud Index

,346* ,478** ,415* ,404 ,269 ,356*

Errors x 100

words

-,099 -,029 ,009 -,231 -,115 -,086

Rep to unrep

-,023 ,228 -,035 ,140 -,036 ,066

Results: Question 22) Is there an impact of Task Complexity on

performance in both the monologic and dialogic tasks?

Proficiency x Performance in MONOLOGIC

Simple StoryComplex Story

Simple MapComplex Map

Simple FirechiefComplex Firechief

Rate A ,448 ,072 ,196

Rate B ,069 ,227 ,404

Pauses ,308 ,827 ,460

S-Nodes x AS Unit

,261 ,095 ,638

Guiraud Index

,286 ,002 ,087

Errors x 100

words

,001 ,000 ,777

Rep to unrep

,009 ,000 ,149

Results: Question 22) Is there an impact of Task Complexity on

performance in both the monologic and dialogic tasks?

Proficiency x Performance in

DIALOGIC

Simple StoryComplex Story

Simple MapComplex Map

Simple FirechiefComplex Firechief

Rate A ,167 ,372 ,164

Rate B ,256 ,309 ,229

Pauses ,336 ,972 ,177

S-Nodes x AS Unit

,830 ,277 ,116

Guiraud Index

,025 ,287 ,008

Errors x 100

words

,707 ,231 ,325

Rep to unrep

,162 ,353 ,468

Fluency

130,13 127,84 128,20

0,00

20,00

40,00

60,00

80,00

100,00

120,00

140,00

Complex Narrative Complex Instruction-givingtask

Complex Decision-makingtask

Structural Complexity

1,72

1,23

1,63

0,000,200,400,600,801,001,201,401,601,802,00

Complex Narrative Complex Instruction-givingtask

Complex Decision-makingtask

Lexical Complexity

5,00 4,88 5,09

0,00

1,00

2,00

3,00

4,00

5,00

6,00

Complex Narrative Complex Instruction-givingtask

Complex Decision-makingtask

Accuracy

34,05

12,07 13,14

0,005,00

10,0015,0020,0025,0030,0035,0040,00

Complex Narrative Complex Instruction-givingtask

Complex Decision-makingtask

Results: tasks compared by dimension Sig. difference

Results: tasks compared by dimension Sig. difference

As expected, in the MONOLOGIC task, general proficiency correlated strongly with performance, and particularly with lexical complexity and accuracy.

The picture is not so clear for the DIALOGIC task, where interaction seems to mitigate the effects of proficiency on performance, especially with regard to accuracy

Discussion: Question 1

In the MONOLOGIC tasks, task complexity shows an impact on accuracy in the narrative task, while it has an impact on both lexical complexity and accuracy in the case of the map task. Higher tasks demands seem to draw attention to form.

Task complexity has no impact on the decision-making task. General measures may not be able to capture such impact.

In the DIALOGIC tasks, task complexity seems to only affect lexical complexity, and just for the narrative and the decision-making task.

Discussion: Question 2

In the MONOLOGIC tasks: the map task generated less structurally and lexically complex speech.

In the DIALOGIC tasks, task complexity seems to only affect lexical complexity, and just for the narrative and the decision-making task.

Discussion: Question 3

1)As in other task-based research areas (e.g. planning time studies, task repetition, or interaction), SPECIFIC PREDICTIONS need to be made for each TASK TYPE.

2)In the same way, predictions need to be adjusted to EACH MODE, since behavior on monologic and dialogic tasks differs considerably.

Conclusions

1)Small sample sizes

2)Use of general measures only

3)Binary operationalizations of complexity (simple/complex, not a continuum)

4)Not factoring in individual differences (e.g. differences in WM capacity)

5)Not using complementary information from native speaker performance

Limitations

1)Use of more specific measures (task-related, developmentally sound)

(Pownall, forthcoming) use of “conjoined clauses” as in Révész (forthcoming) and NPs has found that ‘specific, task-related measures capture the impact of task complexity’

Ways to go from here

and, because, so

“The car” “The little boy” “The funny little boy”

If, before, after

“The girl that was reading little boy”

1)Use of more specific measures

(Pownall, forthcoming) use of “conjoined clauses” as in Révész (forthcoming) and NPs has found that ‘specific, task-related measures capture the impact of task complexity’

2) Integration of Task Complexity into SEQUENCING studies.

Ways to go from here

Thank youGràciesGracias

Members of the GRAL group: Carme.Muñoz, M. Luz Celaya, Elsa Tragant, Teresa Navés, Joan Carles Mora, Imma

Miralpeix, Raquel Serrano, Júlia Barón, Natalia Fullana, Laura Sánchez

Interns: Mayya Levkina, Mireia, Anna Marsol Catherine Daughty

Our students

Roger Gilabert rogergilabert@ub.edu

Universitat de BarcelonaSpain