testing air traffic controllers for language proficiency

© EUROCONTROL 2010

ICAO – Regional Workshop on Language Proficiency Requirements Implementation

Rome, March 3rd to 5th 2010

Testing Air Traffic Controllers Testing Air Traffic Controllers for language proficiencyfor language proficiency

Adrian EnrightAdrian Enright

EUROCONTROLEUROCONTROL


Rome, March 3rd to 5th 2010 © EUROCONTROL 2010

1998 - 20101998 - 2010► ICAO Assembly 32ICAO Assembly 32ndnd Session Sept. 1998 Session Sept. 1998► PRICE study Group 2000 – 2006PRICE study Group 2000 – 2006► SARPs adopted by ICAO Council March 2003SARPs adopted by ICAO Council March 2003► ICAO Assembly resolution A36-11 (Oct 2006) ICAO Assembly resolution A36-11 (Oct 2006)

extends deadline to March 5extends deadline to March 5 thth 2011 2011► March 2010: how much further to go?March 2010: how much further to go?► Only 17/56 States indicate compliance …Only 17/56 States indicate compliance …



ICAO SupportICAO Support

► Action Plan for Implementation, regularly updatedAction Plan for Implementation, regularly updated► Checklist for regulators / service providersChecklist for regulators / service providers► LPRI workshops – 7th since Sept. 2006LPRI workshops – 7th since Sept. 2006► Language samples Language samples ► Doc 9835, Circular 318Doc 9835, Circular 318



Basics of test designBasics of test design

► Assess speaking and listening proficiencyAssess speaking and listening proficiency ICAO Rating scaleICAO Rating scale Holistic DescriptorsHolistic Descriptors

► Tests must demonstrate validity, reliability and Tests must demonstrate validity, reliability and practicalitypracticality



Language testing as seen by the air traffic controller …

START

FINISH



Holistic DescriptorsHolistic Descriptors

► A test should demonstrate:A test should demonstrate: visual and non-visual communicationvisual and non-visual communication communication in the context of the jobcommunication in the context of the job

► And the ability of the candidate to:And the ability of the candidate to: check, confirm and clarify informationcheck, confirm and clarify information handle an unexpected turn of eventshandle an unexpected turn of events use an accent or dialect intelligible to the aviation use an accent or dialect intelligible to the aviation

communitycommunity



Survey on testsSurvey on tests (Professor Charles Alderson of Lancaster University)(Professor Charles Alderson of Lancaster University)

► Survey conducted in 2008 and sent to numerous Survey conducted in 2008 and sent to numerous organisations using tests for pilots and controllersorganisations using tests for pilots and controllers

► Only 22 responsesOnly 22 responses► Additionally, all ICAO States contactedAdditionally, all ICAO States contacted

17/190 responded17/190 responded► Very little evidence attesting to the quality of testsVery little evidence attesting to the quality of tests► Unclear whether CAAs have the knowledge to Unclear whether CAAs have the knowledge to

judge the quality of testsjudge the quality of tests



Report concludesReport concludes► ““We can have very little confidence in the We can have very little confidence in the

meaningfulness, reliability and validity of the meaningfulness, reliability and validity of the aviation language tests currently available for aviation language tests currently available for licensure”.licensure”.

► ““The consequences of inadequate language tests The consequences of inadequate language tests being used in licensing pilots, air traffic controllers being used in licensing pilots, air traffic controllers and other aviation personnel are almost too and other aviation personnel are almost too frightening to contemplate”.frightening to contemplate”.



But, if I can say that my …But, if I can say that my …

► test specifications and test content match the needs of test test specifications and test content match the needs of test takerstakers

► test examiners (raters and interlocutors) are appropriately test examiners (raters and interlocutors) are appropriately trained and accreditedtrained and accredited

► tests have been adequately trialledtests have been adequately trialled► test takers are aware of how and on what they will be test takers are aware of how and on what they will be

tested and how their performance will be ratedtested and how their performance will be rated► the reliability of marking is monitored the reliability of marking is monitored



Then I’m on final for a safe landing …



But I must ask myself …But I must ask myself …► Am I confident that my test is fully compliant with Am I confident that my test is fully compliant with

ICAO LPRs?ICAO LPRs?► Are my controllers adequately prepared for the Are my controllers adequately prepared for the

test?test?► What actions am I taking to reassure my What actions am I taking to reassure my

controllers, especially the older ones, if they fail to controllers, especially the older ones, if they fail to get level 4?get level 4?

► Would I accept a controller with a level 4 in Test Would I accept a controller with a level 4 in Test “XYZ”?“XYZ”?



Beyond 2011?Beyond 2011?

► Programmes to maintain language proficiencyProgrammes to maintain language proficiency► Review and audit of testing systemsReview and audit of testing systems► At least 1 LPR Workshop per yearAt least 1 LPR Workshop per year

exchange experienceexchange experience education assistanceeducation assistance maintain standards of testingmaintain standards of testing



Food for thought …Food for thought …

► What is your organisation doing to prepare What is your organisation doing to prepare controllers for the language proficiency test?controllers for the language proficiency test?

► Why are we seeing a negative approach to Why are we seeing a negative approach to language testing from some of our controllers? language testing from some of our controllers? And what are we doing about it?And what are we doing about it?

► Whose responsibility is itWhose responsibility is it to find answers to to find answers to these and other questions?these and other questions?



Ensuring that student controllers have at least level 4 proficiency will lay the foundation for the future.



Thank youThank you

testing air traffic controllers for language proficiency

Documents