military language testing at the national defence university and the common european framework bilc...
Post on 03-Jan-2016
216 Views
Preview:
TRANSCRIPT
Military Language Testing at the Military Language Testing at the National Defence University and National Defence University and
the Common European the Common European FrameworkFramework
BILC CONFERENCE 2006. BILC CONFERENCE 2006. BUDAPESTBUDAPEST
• General language examination in five General language examination in five languageslanguages
ARMA bilingual military examination inARMA bilingual military examination in nine languages nine languages
NATO STANAG 6001 monolingualNATO STANAG 6001 monolingual language examination language examination
Interviews with officers and NCOsInterviews with officers and NCOs
Hungarian officers and NCOs Hungarian officers and NCOs serving at the NATO headquarters serving at the NATO headquarters
Job descriptions of soldiers Job descriptions of soldiers serving in NATO positions serving in NATO positions
Observance of the regulations for the Observance of the regulations for the conduct of examinations;conduct of examinations;
Continuous training and observing ofContinuous training and observing of testers testers
Continuous pretesting and renewal ofContinuous pretesting and renewal of items; items;
NATO STANAG 6001
2100
605 709 717
1391
971
050
010
0015
0020
0025
00
2002 2003 2004 2005 2006Jan. -Apr.
2006
cand
idat
es
2002
0
50
100
150
200
L S R W L S R W L S R W
LEVEL 1 LEVEL 2 LEVEL 3
CAN
DID
ATES
/ R
ESU
LTS
CANDIDATES
RESULTS
2003
050
100150200250300350400450
L S R W L S R W L S R W
LEVEL 1 LEVEL 2 LEVEL 3
CAN
DID
ATES
/ R
ESU
LTS
CANDIDATES
RESULTS
2004
0
100
200
300
400
500
600
L S R W L S R W L S R W
LEVEL 1 LEVEL 2 LEVEL 3
CAN
DID
ATES
/ R
ESU
LTS
CANDIDATES
RESULTS
2005
0
100
200
300
400
500
600
700
L S R W L S R W L S R W
LEVEL 1 LEVEL 2 LEVEL 3
CAN
DID
ATES
/ R
ESU
LTS
CANDIDATES
RESULTS
2006 Jan. - Apr.
050
100150200250300350400450
L S R W L S R W L S R W
LEVEL 1 LEVEL 2 LEVEL 3
CA
ND
IDA
TES
/ R
ES
ULT
S
CANDIDATES
RESULTS
COMMON EUROPEAN COMMON EUROPEAN FRAMEWORKFRAMEWORK
ESL Exams in HungaryESL Exams in Hungary
• University Of Cambridge ESOLUniversity Of Cambridge ESOL• Trinity College London ESOL & GESETrinity College London ESOL & GESE• The European Language CertificatesThe European Language Certificates• Pitman QualificationsPitman Qualifications• Österreichisches Sprachdiplom Deutsch ÖSDÖsterreichisches Sprachdiplom Deutsch ÖSD• Goethe-Institut vizsgarendszereGoethe-Institut vizsgarendszere• LEXINFOLEXINFO• LCCIEB EFB angol üzleti szaknyelvLCCIEB EFB angol üzleti szaknyelv• EuroEuro• ECL nyelvvizsgaECL nyelvvizsga• Bibliai héberBibliai héber• Diplomáciai szaknyelvi vizsgaDiplomáciai szaknyelvi vizsga• Certifications de l´ Alliance française de ParisCertifications de l´ Alliance française de Paris
Linking Separate AssessmentsLinking Separate Assessments
• EquatingEquating
• CalibratingCalibrating
• Statistical moderationStatistical moderation
• BenchmarkingBenchmarking
• Social moderationSocial moderation
Linking ProcessLinking Process
• Selection of appropriate proceduresSelection of appropriate procedures
• Project planningProject planning
• Coordination of the participantsCoordination of the participants
• Application of the proceduresApplication of the procedures
• Recording of resultsRecording of results
Linking ProceduresLinking Procedures
• FamiliarisationFamiliarisation
• SpecificationSpecification
• StandardisationStandardisation
• Empirical ValidationEmpirical Validation
FamiliarisationFamiliarisation
• Discussion of the Common European Discussion of the Common European Framework levelsFramework levels
• Qualitative analysis of the Common Qualitative analysis of the Common European Framework levelsEuropean Framework levels
The Specification Stage
Manual:Manual:Relating Language Relating Language
Examinations to the Examinations to the Common European Common European
Framework of Reference for Framework of Reference for Languages: Learning, Languages: Learning, Teaching, AssessmentTeaching, Assessment
(CEF)(CEF)
SpecificationSpecification::
Description of the coverage of Description of the coverage of the examination in relation to the examination in relation to
the Common European the Common European FrameworkFramework
Content analysis procedures:
filling in a number of checklistsfilling in a number of checklists using relevant CEF descriptors to relate using relevant CEF descriptors to relate
the exam to the categories and levels of the exam to the categories and levels of the CEFthe CEF
The AThe Aims of the Specificationims of the Specification
To analyse the content in relation to To analyse the content in relation to the relevant categories of the CEFthe relevant categories of the CEF
To profile the examination in relation to To profile the examination in relation to
the relevant descriptor scales of the the relevant descriptor scales of the CEFCEF
To make a claim that an examination is To make a claim that an examination is related to a particular level of the CEFrelated to a particular level of the CEF
Phase 1: Phase 1: A general description of the A general description of the examination or test tasksexamination or test tasks
Phase 2: Phase 2: A detailed description of the A detailed description of the examination or test tasksexamination or test tasks
SStage 1tage 1General Examination Description General Examination Description
FormForm (A1) (A1)What is the overall aim?
What are the more specific objectives?
What are the principal domains?
Which communicative activities are tested?
What types of test tasks are used?
What information is published for candidates and teachers?
SStage 2tage 2Forms on Different Forms on Different Aspects of of
the Examinationthe Examination The test development process (Form A2)The test development process (Form A2) The marking schemes and scoring rules for The marking schemes and scoring rules for
different subtests (Form A3)different subtests (Form A3) The grading and standard-setting procedures The grading and standard-setting procedures
for different subtests (Form A4)for different subtests (Form A4) The reporting of results (Form A5)The reporting of results (Form A5) The analysis and review procedures (Form The analysis and review procedures (Form
A6)A6)
FormForm 3A: 3A: MarkingMarking
How are the tests marked?
Where are the tests marked?
What criteria are used to select the markers?
How is accuracy of marking promoted?
Are productive or integrated test tasks single or double rated?
If double rated, what procedures are used when differences between raters occur?
The Detailed Description of the The Detailed Description of the ExaminationExamination
Forms for the Communicative Forms for the Communicative Language ActivitiesLanguage Activities
Listening ComprehensionListening Comprehension Reading ComprehensionReading Comprehension Spoken InteractionSpoken Interaction Written InteractionWritten Interaction Spoken ProductionSpoken Production Written ProductionWritten Production Integrated Skill CombinationsIntegrated Skill Combinations Integrated SkillsIntegrated Skills Spoken MediationSpoken Mediation Written MediationWritten Mediation
Listening ComprehensionListening Comprehension
Which situations, content categories, domains are the test takers expected to show ability in?
Which communication themes are the test takers expected to be able to handle?
Which communicative tasks are the test takers expected to be able to handle?
What kind of communicative activities and strategies are the test takers expected to be able to handle?
After reading the scale for After reading the scale for Overall Listening Overall Listening
Comprehension, indicate and Comprehension, indicate and justify at which level(s) of the justify at which level(s) of the scale the subtest should be scale the subtest should be
situatedsituated
Forms for the Aspects of Forms for the Aspects of Communicative Language Communicative Language
CompetenceCompetence
ReceptionReception InteractionInteraction ProductionProduction MediationMediation
OutcomeOutcome
A A narrative description of the narrative description of the examinationexamination
CCompleted versions of the relevant ompleted versions of the relevant formsforms
A graphical presentation of the A graphical presentation of the relationship of the examination to the relationship of the examination to the CEF levelsCEF levels
Thank you for your attentionThank you for your attention
top related