LEVEL 4 IAW STANAG 6001LEVEL 4 IAW STANAG 6001- A CONCEPTUAL MODEL -- A CONCEPTUAL MODEL -

Jana Vasilj-Begovic

BILC Secretary/Canada

Gerard Seinhorst

Head, Language Office, Defence Institute Security & Intelligence, Netherlands

SCOPE OF PRESENTATION

Background

Methodology

Conceptual Framework

Implications for Testing

Way ahead

Background

BILC conference 2010 – Istanbul

Formation of level 4 WG

Members: Bulgaria, Canada, Denmark, Germany, Italy, Netherlands, Shape, Sweden, USA

Methodology

• Survey and compile literature

• Develop a conceptual framework

• Find illustrative samples

• Discuss testing implications

Provide guidelines for test development

Conceptual Framework

Aims to answer the questions:o how to interpret the STANAG 6001 descriptors for Level

4?

o what are the typical and shared (linguistic) characteristics of Level 4 language users?

o what are the implications for assessing Level 4 language ability?

Purpose: to enhance the common understanding of Level 4 proficiency.

Level 4 Characteristics

• Linguistically complex

• Low redundancy

• Multiple meanings in socio-cultural context

• Rarely encountered in daily life

• Unfamiliar cultural concepts

• Highly individualized / sophisticatedinterpretation of the layers of explicit and implicit

meaning

Types of Comprehension

• Literal

• Inferential

• Evaluative Understanding meaning “beyond the lines” Evaluating the significance of the message

The Level 4 Language User

STANAG 6001: “Expert”

This level of proficiency is typically, although not necessarily and not exclusively, achieved by individuals who use the foreign language extensively to handle intellectually highly demanding issues as part of their profession or specialisation

The Level 4 Language User

• Very taxing academic and professional settings

• Deal with demanding topics at the highest levels• Examples of military-related tasks:

Spokesperson responsible for press releases and press conferences requiring nuanced, culturally appropriate communications

• Arbiter between warring factions

• Providing official interpretation services

• Analyzing the hidden communicative intent of diplomatic pronouncements

The Level 4 Language User

• Adapts and tailors language to suit the purpose and situation

• Understands/expresses subtleties and shades of meaning

• Consistently maintain high degree of grammatical and lexical accuracy

• Uses metaphors, clichés and proverbs spontaneously

• May have fossilized lapses in pronunciation and intonation

Significant Differences (1) LEVEL 3 LEVEL 4

• Understands/Can produce formal and informal language, for most everyday social and work-related situations.

• Understands/Can produce highly sophisticated language appropriate for almost all topics, and social as well as professional situations.

• Understands/Can produce language relating to abstract concepts and hypotheses.

• Readily adapts language to any situation. Shows a firm grasp of various levels of style and can understand/express subtle nuances and shades of meaning.

• Understands/Can produce extended discourse and conveys meaning correctly and effectively. Understands arguments/Can argue for and against different options.

• Uses the language with great precision for all social and professional purposes, including persuasion, negotiation, or the representation of an official point of view.

Significant Differences (2) LEVEL 3 LEVEL 4

• Can understand/express meaning “between the lines”.

• Can understand/express meaning “beyond the lines”.

• Understands/Demonstrates humour and irony, but may not fully understand some allusions, as well as implications of nuance and idioms.

• Good command of idiomatic expressions, figures of speech and colloquialisms.

• Rarely requests repetition. Has a natural flow, without searching for words.

• Consistently maintains a high degree of grammatical and lexical accuracy. Uses the language effortlessly.

• Occasional errors rarely disturb the native listener/reader.

• Errors are rare and without pattern.

Implications for Testing

Testing receptive skills (listening and reading)

Tasks

Text type

Sample

Testing techniques

Text Type (listening)Adaptation of Clifford’s Text Characteristics

Speaker Purpose Typical Text Type Listener Purpose

PROJECT lines of thought beyond the expected; connect previously unrelated ideas or concepts; present complex ideas with nuanced precision and virtuosity.

Extended discourse that is tailored for the message and the intended audience. To achieve the desired tone and precision of thought, the speaker will often skilfully use low-frequency vocabulary, cultural and historical concepts, and demonstrate understanding of the audience’s shared experience and values.

Listen “beyond the lines”, understand the speaker’s sociolinguistic and cultural references, follow innovative turns of thought, and interpret the message in view of its wider cultural, societal, and political setting.

Text Type (listening)Adaptation of Clifford’s Text Characteristics

Speaker Purpose Typical Text Type Listener Purpose

PROJECT lines of thought beyond the expected; connect previously unrelated ideas or concepts; present complex ideas with nuanced precision and virtuosity.

Extended discourse that is tailored for the message and the intended audience. To achieve the desired tone and precision of thought, the speaker will often skilfully use low-frequency vocabulary, cultural and historical concepts, and demonstrate understanding of the audience’s shared experience and values.

Listen “beyond the lines”, understand the speaker’s sociolinguistic and cultural references, follow innovative turns of thought, and interpret the message in view of its wider cultural, societal, and political setting.

Text Type (listening)Non-participatory Situations

Speaker Purpose Typical Text Type Listener Purpose

Analyse issues, present complex or novel ideas or arguments on professional, political, economic, cultural or societal issues (through public media, overheard conversations).

Lectures, broadcast interviews and discussions, briefings, conference presentations.

For example: an interview contrasting critical thinking with emotional intelligence.

Understand unpredictable shifts of subject matter and tone; understands highly sophisticated language and references embedded in a wider socio-cultural context.

Text Type (listening)Participatory Situations

Speaker Purpose Typical Text Type Listener Purpose

Elaborate on various highly abstract topics

Counsel/persuade or influence using nuanced language specifically tailored to the listener(s).

Negotiations

Debates

Professional meetings.

For example: influencing others to change a policy position

Understand highly complex discourse and all styles of speech (highly formal and abstract or highly informal/colloquial).

Respond and contribute satisfactorily, although not always in the projective mode.

Text Type (reading)Clifford’s Overview of Text Characteristics

PROJECT lines of thought beyond the expected; connect previously unrelated ideas or concepts, or present complex ideas with nuanced precision and virtuosity.

Extended discourse that is tailored for the message being sent and the intended audience. To achieve the desired tone and precision of thought, the author will often demonstrate the skilful use of low- frequency vocabulary, cultural and historical concepts, and an understanding of the audience.

Read “beyond the lines”, understand the author’s sociolinguistic and cultural references, follow innovative turns of thought, and interpret the text in view of its wider cultural, societal, and political setting.

SAMPLE LISTENING TEXT

Extracted from the Charlie Rose Show

Topic: Science and Health

http://www.charlierose.com/view/interview/12276

SAMPLE READING TEXTAN AMORAL MANIFESTO, Joel Marks (Philosophy

Now)Hold onto your hats, folks. Although it is perhaps fitting that the actual day on which I sit here at my computer writing this column is April 1st, let me assure you that I do not intend this as a joke. For the last couple of years I have been reflecting on and experimenting with a new ethics, and as a result I have thrown over my previous commitment to Kantianism. In fact, I have given up morality altogether! This has certainly come as a shock to me (and also a disappointment, to put it mildly). I think the time has come, therefore, to reveal it to the world, and in particular to you, Dear Reader, who have patiently considered my defences of a particular sort of moral theory for the last ten years. In a word, this philosopher has long been labouring under an unexamined assumption, namely, that there is such a thing as right and wrong. I now believe there isn’t.

Testing Techniques

Alignment of text and task level

Constructed response vs. multiple choice

What line of argumentation does the author present for his relinquishment of morality?

One skill vs. combined skill testing

Rating decisions

Implications for Testing

Testing productive skills (speaking and writing)

Tasks

Sample

Testing techniques

What L4 Speakers/Writers Can Do

• Tailor speech to a specific audience

• Counsel

• Persuade

• Advocate a position at length

• Elaborate on abstract concepts

• Express nuances and shades of meaning

Sample of Speaking Proficiency

• Task: discuss abstract topic

• Testing/elicitation technique (OPI): descriptive prelude

• Level of task aligned with level tested

Sample of Writing Proficiency

• Task: analyse abstract issue (NATO related topic)

• Testing technique/writing prompt: descriptive prelude

• Level of task aligned with level tested

SAMPLE WRITING PROMPT

One of the most significant trends in recent decades in the arena of international military and political affairs has been the perceived change in the scope and purpose of NATO commitments and engagements. In the post-Soviet Era, the mission of NATO has apparently shifted from “defending the free world” to “saving the innocent from ethnic cleansing or genocide”—whether at the hands of their own rulers or of neighboring nations—through intervention in an armed struggle on the side of a party chosen on the grounds of moral righteousness and justice. The policy of intervening to prevent genocide or ethnic cleansing is seen as the moral equivalent of humanitarian aid to save innocent human lives.

Sample of Writing Proficiency

In the year 2000, the UN published the Brahimi report on peacekeeping and one of its probative findings was that peacekeeping was significantly more effective, when there was regional ownership of the mission. There are several reasons for this, but one of the major contributing factors is that few conflicts can be adequately solved, unless there is a solid regional backing for the peace process. Additionally, peacekeeping soldiers from the same region and culture are generally much more attuned to the cultural particularities of any given conflict zone and as such they may be better able to understand and confront the underlying cleavages in a society torn by war and atrocities.

Implications for Testing

• Finding appropriate texts

• Selecting appropriate testing techniques

• Determining proficiency/performance ratio

• Sampling widely to elicit a representative/ratable sample of proficiency

• Reconciling requirements for wide sampling with those of practicality

• Validating (target population)

• Training testers/raters

WG Status

• Paper finalized and disseminated (available on BILC Website www.natobilc.org)

• Format and protocol for level 4 testing under consideration

• Pilot project of level 4 testing protocol planned

QUESTIONS

THANK YOU!