interactional competence in the workplace:...
TRANSCRIPT
![Page 1: INTERACTIONAL COMPETENCE IN THE WORKPLACE: …ukalta.org/wp-content/uploads/2017/10/Galaczi_Taylor-LTF-2018-for... · Case Study: Occupational English Test (OET) • ability to communicate](https://reader035.vdocument.in/reader035/viewer/2022062915/5e9a06732bb25468744c447d/html5/thumbnails/1.jpg)
INTERACTIONAL COMPETENCE IN THE WORKPLACE: CHALLENGES AND OPPORTUNITIES
Evelina Galaczi ● Cambridge Assessment English
Lynda Taylor ● CRELLA
Language Testing Forum 2018 ● CRELLA
![Page 2: INTERACTIONAL COMPETENCE IN THE WORKPLACE: …ukalta.org/wp-content/uploads/2017/10/Galaczi_Taylor-LTF-2018-for... · Case Study: Occupational English Test (OET) • ability to communicate](https://reader035.vdocument.in/reader035/viewer/2022062915/5e9a06732bb25468744c447d/html5/thumbnails/2.jpg)
Influences on evolution and definition of interactional competence
Pedagogic
Socio-political
Theoretical
Methodological
![Page 3: INTERACTIONAL COMPETENCE IN THE WORKPLACE: …ukalta.org/wp-content/uploads/2017/10/Galaczi_Taylor-LTF-2018-for... · Case Study: Occupational English Test (OET) • ability to communicate](https://reader035.vdocument.in/reader035/viewer/2022062915/5e9a06732bb25468744c447d/html5/thumbnails/3.jpg)
The AUTHENTICITY debate
Is the speech elicited in interactional tests sufficient to assess communicative competence?
Language Proficiency Interviews:
Natural conversation? (Johnson 2001, Johnson & Tyler 1998, Lazaraton 1992,
van Lier 1989, Ross and Berwick 1992, 1996; Young
and Milanovic 1992)
Paired test format
• broader range of speech functions(Brooks 2009; Galaczi 2008, Gan, Davidson & Hamp-Lyons 2009;
Iwashita 1998; O’Sullivan, Weir and Saville 2002)
• cognitive demands (Field 2011)
![Page 4: INTERACTIONAL COMPETENCE IN THE WORKPLACE: …ukalta.org/wp-content/uploads/2017/10/Galaczi_Taylor-LTF-2018-for... · Case Study: Occupational English Test (OET) • ability to communicate](https://reader035.vdocument.in/reader035/viewer/2022062915/5e9a06732bb25468744c447d/html5/thumbnails/4.jpg)
The RELIABILITY debate
Interviewer variability
• Different support behaviours across examiners (Lazaraton 1996, Brown 2003, Brown & Hill 1998)
Interlocutor effect
• Personality, gender, familiarity, cultural background, talkativeness …(Berry 1993, Chambers, Galaczi, Gilbert 2012, Davis 2009, O’Sullivan 2002, Nakatsuhara 2013, van Moere & Kobayashi 2004)
• We are all linguistic chameleons
What level of variability do interactional tests present?
![Page 5: INTERACTIONAL COMPETENCE IN THE WORKPLACE: …ukalta.org/wp-content/uploads/2017/10/Galaczi_Taylor-LTF-2018-for... · Case Study: Occupational English Test (OET) • ability to communicate](https://reader035.vdocument.in/reader035/viewer/2022062915/5e9a06732bb25468744c447d/html5/thumbnails/5.jpg)
The testing of interactional competence …
A validity asset?
A validity threat?
![Page 6: INTERACTIONAL COMPETENCE IN THE WORKPLACE: …ukalta.org/wp-content/uploads/2017/10/Galaczi_Taylor-LTF-2018-for... · Case Study: Occupational English Test (OET) • ability to communicate](https://reader035.vdocument.in/reader035/viewer/2022062915/5e9a06732bb25468744c447d/html5/thumbnails/6.jpg)
IC = a psychological construct
Does interactional competence reside WITHIN an individual?
IC = a social construct
Does interactional competence reside in co-constructed interaction BETWEEN individuals?
The THEORETICAL debate
(Bachman & Palmer 1996; McNamara,1997; McNamara & Roever, 2006)
![Page 7: INTERACTIONAL COMPETENCE IN THE WORKPLACE: …ukalta.org/wp-content/uploads/2017/10/Galaczi_Taylor-LTF-2018-for... · Case Study: Occupational English Test (OET) • ability to communicate](https://reader035.vdocument.in/reader035/viewer/2022062915/5e9a06732bb25468744c447d/html5/thumbnails/7.jpg)
(Galaczi & Taylor 2018:227)
Interactional Competence
![Page 8: INTERACTIONAL COMPETENCE IN THE WORKPLACE: …ukalta.org/wp-content/uploads/2017/10/Galaczi_Taylor-LTF-2018-for... · Case Study: Occupational English Test (OET) • ability to communicate](https://reader035.vdocument.in/reader035/viewer/2022062915/5e9a06732bb25468744c447d/html5/thumbnails/8.jpg)
Construct definition
Test design
Scoring model
Examiner training in test delivery and scoring
How can interactional competence be measured without compromising validity or reliability?
![Page 9: INTERACTIONAL COMPETENCE IN THE WORKPLACE: …ukalta.org/wp-content/uploads/2017/10/Galaczi_Taylor-LTF-2018-for... · Case Study: Occupational English Test (OET) • ability to communicate](https://reader035.vdocument.in/reader035/viewer/2022062915/5e9a06732bb25468744c447d/html5/thumbnails/9.jpg)
Case Study: Occupational English Test (OET)
• ability to communicate in an English-speaking healthcare context
• 4 skills: reading, writing, listening and speaking
• 12 healthcare professions, e.g. dentistry, medicine, nursing, pharmacy, podiatry, veterinary science
![Page 10: INTERACTIONAL COMPETENCE IN THE WORKPLACE: …ukalta.org/wp-content/uploads/2017/10/Galaczi_Taylor-LTF-2018-for... · Case Study: Occupational English Test (OET) • ability to communicate](https://reader035.vdocument.in/reader035/viewer/2022062915/5e9a06732bb25468744c447d/html5/thumbnails/10.jpg)
OET Speaking
• 1-to-1 face-to-face
• 2 role-plays (~5 mins & 3 mins prep)
• Candidate their professional role
• Interlocutor patient, client, relative, carer
• Interlocutor script
![Page 11: INTERACTIONAL COMPETENCE IN THE WORKPLACE: …ukalta.org/wp-content/uploads/2017/10/Galaczi_Taylor-LTF-2018-for... · Case Study: Occupational English Test (OET) • ability to communicate](https://reader035.vdocument.in/reader035/viewer/2022062915/5e9a06732bb25468744c447d/html5/thumbnails/11.jpg)
![Page 12: INTERACTIONAL COMPETENCE IN THE WORKPLACE: …ukalta.org/wp-content/uploads/2017/10/Galaczi_Taylor-LTF-2018-for... · Case Study: Occupational English Test (OET) • ability to communicate](https://reader035.vdocument.in/reader035/viewer/2022062915/5e9a06732bb25468744c447d/html5/thumbnails/12.jpg)
![Page 13: INTERACTIONAL COMPETENCE IN THE WORKPLACE: …ukalta.org/wp-content/uploads/2017/10/Galaczi_Taylor-LTF-2018-for... · Case Study: Occupational English Test (OET) • ability to communicate](https://reader035.vdocument.in/reader035/viewer/2022062915/5e9a06732bb25468744c447d/html5/thumbnails/13.jpg)
OET Role play tasksTest design
• Explicit contextual information
• setting, participants, content
• Range of speech functions
• e.g. discussing symptoms and concerns, explaining cause of symptoms, recommending and exploring different treatment options …
• Implicit requirement for candidate to demonstrate empathy (Silverman 2016)
• attentive listening, facilitating patient’s narrative, reassuring a patient who is worried/anxious/angry/concerned
![Page 14: INTERACTIONAL COMPETENCE IN THE WORKPLACE: …ukalta.org/wp-content/uploads/2017/10/Galaczi_Taylor-LTF-2018-for... · Case Study: Occupational English Test (OET) • ability to communicate](https://reader035.vdocument.in/reader035/viewer/2022062915/5e9a06732bb25468744c447d/html5/thumbnails/14.jpg)
Assessing OET Speaking performances
• Post-hoc
• Audio recording
• Independent rating by 2 trained examiners
• 9 assessment criteria
• 4 linguistic criteria: 6-point scale
• 5 clinical communication: 4-point scale
![Page 15: INTERACTIONAL COMPETENCE IN THE WORKPLACE: …ukalta.org/wp-content/uploads/2017/10/Galaczi_Taylor-LTF-2018-for... · Case Study: Occupational English Test (OET) • ability to communicate](https://reader035.vdocument.in/reader035/viewer/2022062915/5e9a06732bb25468744c447d/html5/thumbnails/15.jpg)
Linguistic
criteria
Assessment criteria
Clinical
communication
criteria
Understanding
and incorporating
the patient’s
perspective
Relationship
building
Providing
structure
Information
giving
Information
gathering
FluencyIntelligibility
Appropriateness
of Language
Resources of
Grammar and
Expression
‘indigenous’ assessment criteria(Jacoby 1998; Pill 2016)
![Page 16: INTERACTIONAL COMPETENCE IN THE WORKPLACE: …ukalta.org/wp-content/uploads/2017/10/Galaczi_Taylor-LTF-2018-for... · Case Study: Occupational English Test (OET) • ability to communicate](https://reader035.vdocument.in/reader035/viewer/2022062915/5e9a06732bb25468744c447d/html5/thumbnails/16.jpg)
How is the construct of interactional competence conceptualised and operationalised in the OET Speaking Test?
![Page 17: INTERACTIONAL COMPETENCE IN THE WORKPLACE: …ukalta.org/wp-content/uploads/2017/10/Galaczi_Taylor-LTF-2018-for... · Case Study: Occupational English Test (OET) • ability to communicate](https://reader035.vdocument.in/reader035/viewer/2022062915/5e9a06732bb25468744c447d/html5/thumbnails/17.jpg)
Balancing the tension between authenticity & reliability
OET Speaking:
• rich construct definition for IC
• discipline/workplace-specific
• context setting
• role specification
• task parameters
• interactive listening
• indigenous criteria – both linguistic AND clinical communication skills
• guided interaction but spontaneous (unscripted) talk
• interlocutor training
• independent double marking
![Page 18: INTERACTIONAL COMPETENCE IN THE WORKPLACE: …ukalta.org/wp-content/uploads/2017/10/Galaczi_Taylor-LTF-2018-for... · Case Study: Occupational English Test (OET) • ability to communicate](https://reader035.vdocument.in/reader035/viewer/2022062915/5e9a06732bb25468744c447d/html5/thumbnails/18.jpg)
Taking account of the effect of interlocutor variables
OET Speaking:
• the interlocutor as part of the IC construct reconceptualising variability as construct-relevant, not irrelevant
• guidance for interlocutors to minimise negative impact of any potential effect
• inclusion of 2 role-play tasks (‘2 bites of the cherry’)
![Page 19: INTERACTIONAL COMPETENCE IN THE WORKPLACE: …ukalta.org/wp-content/uploads/2017/10/Galaczi_Taylor-LTF-2018-for... · Case Study: Occupational English Test (OET) • ability to communicate](https://reader035.vdocument.in/reader035/viewer/2022062915/5e9a06732bb25468744c447d/html5/thumbnails/19.jpg)
Acknowledging the role of non-verbal behaviours
OET Speaking:
• non-verbal and paralinguistic behaviours included as part of the IC construct (i.e. broadening of the construct)
• Clinical communication criteria:
• Criterion A4: ‘Showing empathy for feelings/predicament/emotional state – achieved through both non-verbal and verbal behaviours … use of silence and appropriate voice tone …’
• Criterion B2: ‘Picking up the patient’s cues – changes in non=verbal behaviour such as hesitation or change in volume …’
![Page 20: INTERACTIONAL COMPETENCE IN THE WORKPLACE: …ukalta.org/wp-content/uploads/2017/10/Galaczi_Taylor-LTF-2018-for... · Case Study: Occupational English Test (OET) • ability to communicate](https://reader035.vdocument.in/reader035/viewer/2022062915/5e9a06732bb25468744c447d/html5/thumbnails/20.jpg)
Possible issues to consider
• task comparability across tests/disciplines?
• ‘role-play’ vs ‘real-life’?
• socio-cultural conventions?
• personal characteristics?
• emotionally charged interaction?
• non-verbal interactive behaviour among L2 users not well understood?
• interlocutor training?
• assessor training?
• challenge for assessors, esp. when rating audio-recordings (kinesicand nonverbal turn-taking features are not visible)?
![Page 21: INTERACTIONAL COMPETENCE IN THE WORKPLACE: …ukalta.org/wp-content/uploads/2017/10/Galaczi_Taylor-LTF-2018-for... · Case Study: Occupational English Test (OET) • ability to communicate](https://reader035.vdocument.in/reader035/viewer/2022062915/5e9a06732bb25468744c447d/html5/thumbnails/21.jpg)
Further informationElder, C., McNamara, T., Woodward-Kron, R., Manias, E., McColl, G., & Webb, G. (2013). Towards Improved Healthcare Communication. Language Testing Research Centre, University of Melbourne.
Galaczi, E., & Taylor, L. (2018). Interactional competence: Conceptualisations, operationalisations, and outstanding questions. Language Assessment Quarterly 15(3), 219–236.
Lim, G. & Seguis, B. (Forthcoming 2019). OET 2.0 Test Revision manual. Cambridge Assessment English.
Pill, J. (2016). Drawing on indigenous criteria for more authentic assessment in a specific-purpose language test: Healthcare professionals interacting with patients. Language Testing 33(2), 175-193.
Plough, I., Banerjee, J., & Iwashita, N. (2018). Interactional competence: Genie out of the bottle. Language Testing, 35(3), 427–445.
Séguis, B. & McElwee, S. (2019). Assessing clinical communication in the Occupational English Test. In Papageorgiou, S., & Bailey, K. M. (Eds.). Global perspectives on language assessment. New York, NY: Routledge.
Silverman, J. (2016). Relationship building, in Brown, J., Noble, L., Kidd, J. and Papageorgiou, A. (Eds). Clinical Communication in Medicine (pp.72-75). Oxford: John Wiley & Sons.