stanag 6001 & bat, clifford.pptx

17
50 Years of BILC: The Evolution of STANAG 6001 1976 – 2016 and the first Benchmark Advisory Test Ray Clifford 24 May 2016

Upload: vodang

Post on 05-Jan-2017

300 views

Category:

Documents


7 download

TRANSCRIPT

50 Years of BILC:The Evolution of STANAG 6001

1976 – 2016and the first Benchmark Advisory Test

Ray Clifford24 May 2016

1976: STANAG 6001, Edition 1 • NATO adopted the standards that were

drafted by BILC during 1973 through 1975.– The lack of consistent specifications limited the

STANAG’s usefulness for assessment purposes.– The brevity of the descriptions allowed for

multiple interpretations.– A testing expert later called those descriptions

“thin, haphazard, and often off the mark”.– To fill the gaps, NATO nations implemented their

own interpretations.

Negative Impact of Edition 1• National interpretations were not aligned.– National tests and ratings were not equivalent. – My level 3 might equal your level 2 or even your level 1.

• The Standardized Language Profile (SLP) for jobs became inflated. – SLP’s were often set by people-comparisons rather than by

criterion-based job task analyses. – “The last guy was a 3; and his language skills were weak, so

we need a 4.”

• The lack of language skills was identified as NATO’s greatest challenge to interoperability.

Actions Taken• 1999: BILC formed a working group on Testing

and Assessment.– Its initial goal was to elaborate on and interpret

the original STANAG 6001 descriptors.– Representatives from 11 BILC member-nations

participated.• 2000: That working group produced the

“Interpretation and Elaboration Document”.– That “Interpretation” was successfully trialed in

the new BILC Language Testing Seminar.

2003: STANAG 6001, Edition 2• 2002: The BILC Steering Committee– Approved the “Interpretation Document”.– Forwarded it to the NATO Standardization Agency

for approval and publication.• 2003: STANAG 6001, Edition 2 was released. – It was the first substantive change in a quarter of a

century.– The interpretation and elaboration document was

included as Annex A.

2009: STANAG 6001, Edition 3• After extensive study and work, Edition 3 was

published with several substantive changes.– It changed the labels for the levels so they better

reflected the content of the level descriptions.• 0 = No proficiency.• 1 = Elementary Survival.• 2 = Fair Functional.• 3 = Good Professional.• 4 = Very Good Expert.• 5 = Excellent Highly-articulate native.

2009: STANAG 6001, Edition 3(Continued)

– It defined language proficiency as “unrehearsed, general language communication ability”.

– It added descriptions of “plus levels” as Appendix 1 to Annex A.• “A plus level is understood to be more than halfway

between two base levels.”• “A plus level substantially exceeds the base level, but

does not fully or consistently meet all of the criteria for the next higher base level.”

2010: STANAG 6001, Edition 4

• Added two additional purposes for which plus levels may be used.– Edition 3: “A plus level may be added to a base

level for training or evaluation purposes … “– Edition 4: “A plus level may be added to a base

level for training, evaluation, recording or reporting purposes … “

2014: STANAG 6001, Edition 5• 2014: Edition 4 reformatted and republished

in ATrainP-5 as Edition 5.– The format and introductory pages were modified.– The standards were not intentionally changed.– But typos were introduced in the publication

process.• 2016: Edition 5, version 2 was just published.– Typos have been corrected.– Edition 5 will also be available on the BILC website.

Benchmark Advisory Test(BAT)

A Parallel Effort that Appliedthe Improved STANAG 6001

Why Benchmark Testing?• To promote consistent interpretation and

interpretation of STANAG 6001 across national testing programs.

• To provide an external measure against which nations can compare their national STANAG test results.

• To pursue a joint development project as a way of demonstrating how STANAG 6001 tests can be designed, developed, and scored.

Benchmark Advisory Test (BAT) • 2003: Possibility raised; options discussed.• 2004: Steering committee recommended the

concept to the NATO Joint Services Subcommittee Group.

• 2005: BILC launched a volunteer, collaborative effort to create a Reading and Listening BAT.

• 2005-6: ACT took accepted the BILC work and – Contracted to complete tests in 4 skill modalities.– Set an implementation date of 2009.

2009 BAT Administration

• 200 tests were allocated across 11 Nations.• Testing began in May, 2009.• Tests were administered by Language Testing

International.• Results were reported to each participating

nation, but only for their own personnel.• Overall results were reported without

identifying the test takers’ nationality.

Summary

• The STANAG 6001 revision and the BAT development project were an unparalleled success.

• You, the BILC family, are the reason they succeeded

• I have never seen an international team that worked with such a high level of energy, goodwill, cooperation, and dedication.

Summary• The STANAG 6001 revision and the BAT development project

were an unparalleled success.• You, the BILC family, are the reason they succeeded• I have never seen an international team that worked with such

a high level of energy, goodwill, cooperation, and dedication.

•Congratulations on your accomplishments!

Stop! Your time is up!

A Footnote

In case you don’t remember the BAT study, …

the overall BAT test results will be reviewedin my presentation on Thursday.