assessing oral proficiency and grammatical errors: secondary analysis of da interview data
DESCRIPTION
Assessing Oral Proficiency and Grammatical Errors: Secondary Analysis of DA Interview Data. Sun-Kwang Bae, Curriculum Development Division, DLIFLC David Moon, Technology Integration Division, DLIFLC. - PowerPoint PPT PresentationTRANSCRIPT
This speech/presentation is authorized by the Defense Language Institute Foreign Language Center and the Department of Defense. Contents of this presentation are not necessarily the official views of, or endorsed by, the U.S. Government, DoD, Department of the Army, or DLIFLC.
Assessing Oral Proficiency and Grammatical Errors:
Secondary Analysis of DA Interview Data
Sun-Kwang Bae, Curriculum Development Division, DLIFLCDavid Moon, Technology Integration Division, DLIFLC
• The Study Question• Background• The Study• Results• Discussions• Q & A
Presentation Outline
2
What are the grammar structures/features that learners of Korean typically control at a given level of oral proficiency?
[From the perspective of teaching & learning - 김지민 (2009); Corder (1981)]
The Study Question
3
Level 1• Able to ask and answer simple questions related
to everyday survival needs• Able to satisfy minimum courtesy requirements• Able to deal with routine everyday situations• Able to talk simply about self
Tasks that learners can handle at each level – ILR L1
4
Level 2• Able to satisfy routine social demands and
limited work requirements• Able to narrate and describe in all three time
frames• Able to give detailed instructions/directions• Able to participate in conversations on current
events as well as give detailed information about work, family and self
Tasks that learners can handle at each level – ILR L2
5
Level 3• Able to participate in most formal and informal
conversations on practical, social and professional topics
• Able to support opinion in the FLO content areas• Able to hypothesize• Able to discuss concrete and abstract topics• Able to discuss topics of particular interest• Able to deal with unfamiliar situations
Tasks that learners can handle at each level – ILR L3
6
Level 1• Structural accuracy is random or severely
limited• Almost every utterance has errors in basic
structures• Time concepts are vague• Can formulate some questions
Structural control that learners are to have at each level – ILR L1
7
Level 2• Discourse is minimally cohesive• Grammatical structures are usually not very
elaborate and not thoroughly controlled• Errors are frequent• Simple structure and basic grammatical
relations are typically controlled
Structural control that learners are to have at each level – ILR L2
8
Level 3• Effectively combines structure and vocabulary to
convey meaning• Discourse is cohesive• Use of structural devices is flexible and
elaborate• Errors occur in low frequency and highly
complex structures, but structural inaccuracy rarely causes misunderstanding
Structural control that learners are to have at each level – ILR L3
9
Data Collection• Face-to-face diagnostic oral assessment
interviews conducted by a pair of specialists (David Moon, EM, AS, JY, YR, NT)
• During June 1999 – September 2000• 55 students: L0+ (1), L1 (15), L1+ (18), L2 (14),
L2+ (6), L3 (1) • The tool: Sample
The study: data collection
10
Structural Features• Sentence patterns: Plain, Honorific, Quotation• Verb conjugations: Irregular, Special• Complex constructions: Passive, Attributive, Negation• Particles: Case, Locative, Instrumental, Auxiliary• Cardinal/Ordinal Number: Native, Sino-Korean
agreement• Pronouns: Personal, Things, Places• Conjunctions: Basic, Advanced, Conjunctive Fillers
The study: structural features examined
11
The result: sentence patterns
L1(N=15)
L1+(N=18)
L2(N=14)
L2+(N=6)
Plain: Present 73/26/0 78/22/0 100/0/0 100/0/0
Plain: Past 47/53/0 61/39/0 100/0/0 100/0/0
Honorific: Present 13/83/0 17/83/0 29/71/0 100/0/0
Honorific: Past 0/100/0 6/86/6 21/ 72/7 100/0/0
Negation: Statement & Question
40/53/7 56/44/0 93/7/0 100/0/0
Negation: Request & Suggestion
13/20/67 28/43/39 50/0/50 100/0/0
Note: The figures are percentages “Satisfactory/Need Work/Not Applicable.”
12
The result: verb conjugations & complex structures
L1(N=15)
L1+(N=18)
L2(N=14)
L2+(N=6)
Irregular verbs 27/66/7 33/67/0 57/43/0 100/0/0
Special verbs 31/69/0(N=13)
38/62/0(N=16)
83/17/0(N=12)
100/0/0(N=5)
Passive and causative 0/27/73 0/56/44 21/29/50 67/0/33
Attributive 7/93/0 33/67/0 79/21/0 83/17/0
Note: The figures are percentages “Satisfactory/Need Work/Not Applicable.”
13
The result: quotation
L1(N=15)
L1+(N=18)
L2(N=14)
L2+(N=6)
Quotation: statement and question
6/38/56 22/67/11 42/44/14 100/0/0
Quotation: request and suggestion
0/33/67 6/38/56 7/21/72 83/0/17
Note: The figures are percentages “Satisfactory/Need Work/Not Applicable.”
14
The result: markers/particles
L1(N=15)
L1+(N=18)
L2(N=14)
L2+(N=6)
Subject/Object Case 33/67/0 56/44/0 71/29/0 100/0/0
Dative/Locative 20/80/0 50/50/0 71/29/0 100/0/0
Instrumental 18/73/9(N=11)
30/60/10(N=10)
67/33/0(N=12)
100/0/0(N=5)
Auxiliary 0/100/0 28/72/0 71/29/0 100/0/0
Note: The figures are percentages “Satisfactory/Need Work/Not Applicable.”
15
The result: numbers + counters
L1(N=15)
L1+(N=18)
L2(N=14)
L2+(N=6)
Cardinal: Native Korean 33/67/0 44/56/0 71/29/0 100/0/0
Cardinal: Sino-Korean 40/60/0 33/56/11 71/29/0 100/0/0
Ordinal: Native Korean 7/47/46 22/56/22 36/21/43 83/0/17
Ordinal: Sino-Korean 7/40/53 16/34/50 0/14/86 83/0/17
Note: The figures are percentages “Satisfactory/Need Work/Not Applicable.”
16
The result: pronouns
L1(N=15)
L1+(N=18)
L2(N=14)
L2+(N=6)
Personal 53/34/13 89/11/0 93/7/0 100/0/0
Demonstrative: Things 13/54/33 72/17/11 93/7/0 100/0/0
Demonstrative: Places 26/61/13 67/22/11 100/0/0 100/0/0
Note: The figures are percentages “Satisfactory/Need Work/Not Applicable.”
17
The result: conjunctions
L1(N=15)
L1+(N=18)
L2(N=14)
L2+(N=6)
Basic conjunctive endings 40/60/0 56/44/0 86/14/0 100/0/0
Advanced conjunctive endings
0/80/20 6/83/11 0/86/14 17/83/0
Conjunctives/ Conversational Fillers
0/100/0 41/59/0(N=17)
64/36/0 100/0/0
Note: The figures are percentages “Satisfactory/Need Work/Not Applicable.”
18
The result: discourse skills
L1(N=15)
L1+(N=17)
L2(N=14)
L2+(N=6)
Repair 7/93/0 35/65/0 79/21/0 100/0/0
Vocabulary choice with precision
0/100/0 0/100/0 21/79/0 50/50/0
Negotiate meaning 27/63/0 29/71/0 85/15/0 100/0/0
Paraphrase and restate 7/93/0 29/71/0 21/79/0 100/0/0
Note: The figures are percentages “Satisfactory/Need Work/Not Applicable.”
19
Level 1• Personal pronouns• Present tenseLevel 1+• Past tense• Negation (in statements & questions)• Case markers (subject, object, dative, locative)• Demonstrative pronouns (things & places)• Basic conjunctive endings
Discussions: grammar features that learners begin to control at L1 & L1+
20
Level 2• Negation in requests and suggestions• Irregular verbs and special verbs• Attributive forms• Instrumental and auxiliary particles• Numbers and their agreement with counters• Conjunctives and conversational fillers• Repair strategy in discourse• Negotiate meaning in discourse
Discussions: grammar features that learners begin to control at L2
21
Level 2+• Honorifics• Passive and causative• Quotations• Precise lexical selection• Paraphrase and restate
The features that learners can’t control by L2+:• Advanced conjunctive endings
Discussions: grammar features that learners begin to control at L2+
22
Q & A
Implications & Applications
23
Contact Information
Online Diagnostic Assessment for reading and listening skills:
http://oda.dliflc.edu
Contact: [email protected]