workshop t7: do indonesians make mobile calls and do russians go on holiday?
DESCRIPTION
Do Indonesians make mobile calls and do Russians go on holiday? Cultural determination in tests. By academic staff of the University of Amsterdam and the Catholic University of LeuvenTRANSCRIPT
![Page 1: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/1.jpg)
DO INDONESIANS MAKE MOBILE CALLS AND DO RUSSIANS
GO ON HOLIDAY?
NHTV Breda – january 2008
Katholieke Universiteit Leuven
Universiteit van Amsterdam
E.Teijsse-Jongkind @uva.nl
The effect of culture on testingThe Certificate Dutch as a Foreign Language
![Page 2: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/2.jpg)
![Page 3: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/3.jpg)
Content of this presentationContent of this presentation- Purpose of this presentation: to be aware
of cultural bias in the classroom
- Background of the CNaVT (Certificaat Nederlands als Vreemde Taal)
- Definition of cultural bias and the effect of culture on different levels
- Task: Does this test contain cultural bias?
- How does the CNaVT manage the effect of culture in the CNaVT-exams?
- Discussion: own experiences with cultural bias
![Page 4: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/4.jpg)
The Certificate of Dutch as a The Certificate of Dutch as a Foreign LanguageForeign Language
- CNaVT (Certificaat Nederlands als Vreemde Taal)
- administration in more than 40 countries around the world: from Madrid to Jakarta and from Warsaw to Rio de Janeiro
- 2500 candidates each year
![Page 5: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/5.jpg)
Assess as direct as possibleAssess as direct as possible
- Based on a needs analysis
- authenticity: 6 exams represent 6 different domains
- task-based: exam tasks represent real life language use situations
![Page 6: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/6.jpg)
The effect of culture: cultural biasThe effect of culture: cultural bias
Definition of cultural bias:Scores of groups are distorted, because the items favour some groups over others. This distortion is not caused by a difference in (language) proficiency.
![Page 7: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/7.jpg)
THE EFFECT OF CULTURE ON THE EFFECT OF CULTURE ON DIFFERENT LEVELSDIFFERENT LEVELS(Van Baalen, Beheydt & van Kalsbeek, 2003)
On the level of:1. Words and expressions2. Images3. Subjects and concepts4. Knowledge of the world5. Relation examiner - candidate6. Classroom / test situation
![Page 8: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/8.jpg)
1. WORDS and EXPRESSIONS1. WORDS and EXPRESSIONS
verpotteren
flitspaalgedrag
speeddaten
(zit)bank vs. zetel
lopen vs. stappen
namiddag
iets afkloppen/ hout vasthouden
![Page 9: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/9.jpg)
2. IMAGES2. IMAGES
![Page 10: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/10.jpg)
![Page 11: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/11.jpg)
3. SUBJECTS3. SUBJECTS
![Page 12: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/12.jpg)
4. KNOWLEDGE OF THE WORLD4. KNOWLEDGE OF THE WORLDexample 1example 1
•You want to organize a party for your birthday.
•Describe what you want to your friend.
![Page 13: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/13.jpg)
4. KNOWLEDGE OF THE WORLD4. KNOWLEDGE OF THE WORLDexample 2example 2
vs.
![Page 14: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/14.jpg)
5. RELATION EXAMINER-STUDENT5. RELATION EXAMINER-STUDENT
![Page 15: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/15.jpg)
6. CLASSROOM PRACTICE/ TEST 6. CLASSROOM PRACTICE/ TEST SITUATIONSITUATION
![Page 16: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/16.jpg)
MINI TESTMINI TEST
• Read the test and answer the following question.
• Does this test contain cultural bias for some groups of students? Which levels of cultural bias can appear? Why?
![Page 17: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/17.jpg)
A TENSIONA TENSION
Validity in the sense that we want our exams to measure language proficiency.
Reliability in the sense that we want to be able to state that our exams are not influenced by other factors like the culture of the candidate.
![Page 18: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/18.jpg)
DIF
Differential Item Functioning
= the statistical view of cultural bias
DIF ANALYSISDIF ANALYSIS
![Page 19: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/19.jpg)
CNaVT item bias researchCNaVT item bias research
Example 1
Country: Indonesia
Item: Question in form
‘What kind of object have you lost?’
![Page 20: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/20.jpg)
CNaVT item bias researchCNaVT item bias research
Example 2
Country: Russia
Item: Question from friend in letter
(…) ‘But I often think back with nostalgia about my holiday in your country. (…) And you? What did you do during your holidays? Did you travel? Where to? Or did you stay at home and what did you do?’ (…)
![Page 21: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/21.jpg)
How does the CNaVT How does the CNaVT manage the effect of culture?manage the effect of culture?
- awareness of what we test: what (concerning culture) is relevant, what is irrelevant
- generate input on possible cultural bias from different sources in different stadia
- In case of suspicion of irrelevance: DIF analysis
![Page 22: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/22.jpg)
BUT… CNaVT is a test of Dutch as a BUT… CNaVT is a test of Dutch as a foreign language! foreign language! Difference between test situation and the classroom
Cultural bias is not something that has to be avoided in the classroom
But: teachers should use cultural miscommunications to learn from it
Intercultural competence!
Teacher and test developer: recognize and be aware of the effect of culture
![Page 23: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/23.jpg)
Tips: How to make students Tips: How to make students interculturally competent? interculturally competent? Three important principles:
Perspective
Context
Dialogue
![Page 24: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/24.jpg)
DISCUSSIE / VRAGENDISCUSSIE / VRAGEN
What is your experience with cultural bias in your own classroom/test practice? Can you give an example? How do you manage?
![Page 25: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/25.jpg)
LITERATUURVERWIJZINGENLITERATUURVERWIJZINGEN
Van Baalen, C., Beheydt, L. & Van Kalsbeek, A. (2003). Cultuur in taal. Interculturele vaardigheden voor docenten Nederlands aan anderstaligen. Utrecht: Nederlands centrum buitenlanders, 167p.
![Page 26: Workshop T7: Do Indonesians make mobile calls and do Russians go on holiday?](https://reader034.vdocument.in/reader034/viewer/2022051514/549ecce0b37959af618b4838/html5/thumbnails/26.jpg)
DO INDONESIAN MAKE MOBILE CALLS AND DO RUSSIAN
GO ON HOLIDAY?
NHTV Breda – januar 2008
Katholieke Universiteit Leuven
Universiteit van Amsterdam
E.Teijsse-Jongkind @uva.nl
Language testing and cultureThe certificate of Dutch as a foreign language