european language policies and activities, with particular
TRANSCRIPT
European language policies and activities, with particular
reference to assessment
J Charles Alderson
Department of Linguistics and English Language
Europe?
• European Union• 25 member states (15 plus 10 new
members on May 1st 2004)• European Commission is the civil service,
and its directorates are the ministries• Council of Ministers makes policy
decisions• European Parliament (Brussels and
Strasbourg) - an assembly of MEPs
European Union
• Austria, Belgium, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Poland, Portugal, Slovak Republic, Slovenia, Spain, Sweden, Netherlands, United Kingdom.
• 20 official languages
Europe?
• Council of Europe
• 46 member states: the EU plus 21
• Committee of Ministers• Parliamentary Assembly
• European Court of Human Rights
• No directorates, no ministries
Council of Europe
• 25 EU countries plus:
Albania, Andorra , Armenia, Azerbaijan, Bosnia and Herzegovina, Bulgaria, Croatia, Georgia, Iceland, Liechtenstein, Luxembourg, Malta, Norway, Romania, Russian Federation, San Marino, Serbia and Montenegro, Switzerland, "the former Yugoslav Republic of Macedonia", Turkey, Ukraine.
• 2 official languages
Council of Europe
The "European Convention on Human Rights" (1950) sets forth a number of fundamental rights and freedoms (right to life, prohibition of torture, prohibition of slavery and forced labour, right to liberty and security, right to a fair trial, no punishment without law, right to respect for private and family life, freedom of thought, conscience and religion, freedom of expression, freedom of assembly and association, right to marry, right to an effective remedy, prohibition of discrimination).
Council of Europe
Language Policy Division (Strasbourg)
European Centre for Modern Languages – ECML (Graz)
Council of Europe
Aims
“The Council of Europe accords special importance to fostering the linguistic and cultural diversity of its member States. Its activities in the field of languages aim to promote plurilingualism and pluriculturalism among citizens in order to combat intolerance and xenophobia by improving communication and mutual understanding between individuals.”
Council of Europe
The notional-functional syllabus (Wilkins, 1970s)
Threshold Level (Niveau Seuil, Kontaktschwelle, etc., etc.) Trim and Van Ek, 1970s
Later: Vantage and Waystage Levels (1990)
Finally: The Common European Framework
Council of Europe
The Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Versions 1996, 1998, 2001 (Cambridge University Press).
Now available in 18 languages (Basque, Catalan, Czech, English, Finnish, French, Galician, Georgian, German, Hungarian, Italian, Japanese, Moldovan, Polish, Portuguese, Russian, Serbian
and Spanish)
Six levels: A1,A2, B1, B2, C1, C2
Council of Europe
The European Language Portfolio
A document in which to record and reflect on language learning and cultural experiences, comprising:
• A language passport • A grid
• A detailed language biography
• A dossier.
The European Language Portfolio
• A language passport which its owner regularly updates. A grid is provided where his/her language competences can be described according to common criteria accepted throughout Europe and which can serve as a complement to customary certificates.
The European Language Portfolio
• A detailed language biography describing the owner's experiences in each language and which is designed to guide the learner in planning and assessing progress.
The European Language Portfolio
• A dossier where examples of personal work can be kept to illustrate one's language competences.
European Union
Aims to become: “the most competitive knowledge-based economy in the world” (Lisbon Strategy, 2000)
Language learning is seen as one of the key issues and language proficiency as a key element in mobility of labour and learning
European Union
The Barcelona Indicators, including
• an indicator of achievement in foreign language education, at the end of compulsory education
• based on the CEFR
• reflecting member states’ test and assessment cultures
• to be developed in 2005-2007
European Union
• Enhancement of language learning, teaching and assessment through
• Socrates, Lingua, Erasmus and Leonardo Projects,
• including materials development, student and teacher exchanges, teacher education
and• DIALANG – diagnostic tests in 14
languages
European Union
Action Plan for Linguistic Diversity and Language Learning
Policy of One + Two
Encouragement of minority and less widely taught languages
Increasing professionalisation of language teaching and assessment (including ENLTA and EALTA, cf ALTE)
Manual for Relating Language Examinations to CEF(R)
Developed by the Council of Europe.Aims to:
· contribute to competence building in the area of linking assessments to the CEF;
· encourage increased transparency on the part of examination providers;
· encourage the development of both formal and informal national and international networks of institutions and experts.
Manual for Relating Language Examinations to CEF(R)
The Manual aims to
provide “reference” materials which can be used to help examination providers and others link their measures to the CEFR, including
• videos of spoken performance• scripts of written performance at all CEFR levels• CD of calibrated reading and listening items
Building a European Item Bank of Anchor Items for Foreign Language
Study (EBAFLS)• EU-funded Project (Socrates Lingua 2)
• Reading and Listening
• EN, FR, DE• 8 partner countries: NL,DE, ES, FR, HU,
LUX, SE, UK
• October 2004 to September 2007
Building a European Item Bank of Anchor Items for Foreign Language
Study (EBAFLS)• Allows for valid comparisons independent
of the specific cultural background (culture free or fair)
• Uses existing items• Can be used to evaluate own test as an
extension of the bank• Could be used as part of a national
Barcelona Indicator
Building a European Item Bank of Anchor Items for Foreign Language
Study (EBAFLS)Two facilitating projects funded by the Dutch
Government
2. A survey of the policy and practice of assessment in all EU member states
3. A study of the constructs underlying the CEFR levels, for reading and listening
Improving language education
• Conference organised by the Dutch Government, October 14-16
Themes: • Reviewing national language education
policies
• Stimulating and maintaining teacher professionalism
• Assessing and certifying language skills
The Dutch CEF Construct Project
• Do we have in the CEFR an instrument to help us construct Reading and Listening items and tests based on the CEFR?
• If the CEFR scales and the Descriptive Scheme are insufficient, what is needed to develop such an instrument and what should it look like?
The Dutch CEF Construct Project
• Project coordinator: J.C. Alderson
• Partner institutions: French Ministry of Education, Dutch CITO, German DESI, Catalan Department of Education, Finnish expert (Takala)
The Dutch CEFR Construct Project
Methodology • expert judgments on the usability of the CEFR
for test construction, • identifying what might be missing from the
CEFR, • developing a frame for analysis of tests and
specifications, • examining a range of test specifications,
guidelines to item writers, and sample test tasks at the six levels of the CEFR.
The Dutch CEF Construct Project
Outcomes • Critical review of the CEFR• Set of compilations of CEFR scales and of test
specifications at the different levels of the CEFR
• A series of frameworks or classification systems• A World-wide-web-mounted instrument, based
on the CEFR, to enable the characterisation of tests and items in relation to the CEFR.
Definition of a reading or listening item, consistent with CEF
"A task (which has an intention/ goal/ purpose) for an individual or group, requiring an understanding of a text related to a theme in a domain which requires certain strategies, under certain conditions and limitations".
Problems with the CEFR
• Terminology problems: synonymy or not?
• Gaps
• Inconsistencies
• Lack of definition
Terminology problems: synonymy or not?
Operations at A2
• Understand• Take • Get
• Follow
• Identify • Infer
Operations at B2
• Understand• Scan• Monitor
• Obtain
• Select• Evaluate• Locate• Identify
Synonyms?
• I can understand familiar names, words and very simple sentences, for example on notices and posters or in catalogues” (page 26)
• “Can recognise familiar names, words and very basic phrases on simple notices in the most common everyday situations” (page 70)
Synonyms?
Can identify the main conclusions in clearly signalled argumentative texts.
Can recognise the line of argument in the treatment of the issue presented, though not necessarily in detail.”
Synonyms?
Can find specific, predictable information in simple everyday material such as advertisements, prospectuses, menus, reference lists and timetables.
Can locate specific information in lists and isolate the information required (e.g. use the ‘Yellow Pages’ to find a service or tradesman)?
Gaps in the CEF
• A description of the operations that comprehension consists of and a theory of how comprehension develops.
• A specification of micro-skills or subskills of comprehension.
• Concepts introduced in the text but not incorporated in the scales or related to the six levels in any way.
Gaps in the CEF
• competence, general competence, communicative language competence
• activities, processes, text, domain, strategy, task• context• ludic and aesthetic uses of language• texts• text to text activities • socio-cultural knowledge• study skills • tasks, including description, performance (conditions,
competences, linguistic factors), strategies, difficulty
Gaps in the CEF
• The Task: what is it that candidates have to do with text?
• Test methods and the processing demands they create
Inconsistencies
• Operation recognise only mentioned at the levels A1, B1 and C1 and not at the other levels
• The use of a dictionary only mentioned at B2 and C1
• Clear, slow and carefully articulated speech (A1)Clear, slow and articulated speech (A2)Clear, standard speech, familiar accent (B1)Normal speed, standard language (B2)
• “Simple notices” A1, “everyday notices” A2. No other references to “notices”
Lack of definitions
• Simple
• the most common, everyday, familiar, concrete, predictable, straightforward, factual, complex, specialised, highly colloquial, short, long
• Number of words?
• Is a short text necessarily “easier” than a longer text?
Conclusions
• Inter-analyst agreement was quite promising, but needs to be improved by training and discussion before decisions are made.
• The relation between the dimensions in the Grid, and individual CEFR levels was not very obvious.
• Relatively few dimensions showed any significant association with the six CEFR levels.
• However, the collection of much more extensive data using the Grid is recommended before solid conclusions can be reached about the relationship, or lack of it, between the dimensions of the Grid and CEFR levels.
Conclusions
• The Grid is a useful instrument for the description of test items and tasks in terms of the CEFR. A series of recommendations are made for revision and development of the Grid and associated guidance and training materials, and for further research using the revised Grid.
Conclusions
• The identification of separate levels in the CEFR is at least as much an empirical matter as it is a question of the content of the tests as determined by test specifications or as identified by any content classification system or Grid.
Proposal
• A set of procedures, which involve analysing the content of the test in question using the Grid, estimating the CEFR level of items, texts and tests, and then pre-testing and calibrating the items, as well as conducting standard-setting procedures to set the boundaries of the CEFR levels on the scale coming from the calibration, assigning a psychometric level to the items, and then assigning a definitive CEFR level if the psychometric level falls within the band of the estimated level
• In other words if the estimation based on the analysed content is comparable with the psychometric value.