1. 1. Hangzhou, March 2011 Prof. Dr. Hora Tjitra & Mu Tong, Zhejiang University EvaluationStudies ofCross-culturalTraining A Review of the Literature From 1988 to 2000
2. 2. EvaluationStudiesofCross-CulturalTraining Evaluation Studies on Cultural Awareness Training Black and Mendenhall (1990) Deshpande and Viswesvaran (1992) Bhagat and Prien (1996) Kealey and Protheroe (1996) Bhawuk and Brislin (2000)
3. 3. EvaluationStudiesofCross-CulturalTraining 3 14yearsinGermany 7yearsinChina Bornandgrewup inIndonesia Prof.Dr.Hora Tjitra - Cross-cultural and Business Psychology Dipl.-Psych.,Technical University of Braunschweig Organizational Psychology and Human Resource Management Dr.Phil.,University of Regensburg Intercultural Psychology and Strategic Management Executive Education,INSEAD HR Management in Asia
4. 4. EvaluationStudiesofCross-CulturalTraining Key Benefits of Evaluation Learning Value Chain and the Benefits of its Measurements LevelLevel Key Questions 0 1 2 3 4 5 Input and Indicators What is the number of participants,hours, and programs and what are the costs? Reaction and Planned Action Was the learning relevant,important, useful,and helpful to participants in the job environment? Did the participants plan to use the content in the program? Learning and Confidence Did participants increase or enhance knowledge,skills,or perceptions and have confidence to use them? Application & Implementation What did the participants do differently in the job context? Was the program implemented effectively? What changes were made on the job? Impact and Consequences What are the consequences of the application in terms of output,quality, cost,time,and satisfaction? ROI Did the monetary benefits of the learning program exceed the investment in the program? Sources:Elkele & Philips (2007).The Chief Learning Officer. Satisfy client needs Justifying budgets Improve program designs and processes Enhance the transfer of learning Enhance the respect and credibility of LTD staff Increase support from line managers Strengthen relationship with key executive and administrators Set priorities for learning and development 4
5. 5. EvaluationStudiesofCross-CulturalTraining Key Benefits of Measurement and Evaluation Sa#sfyclientneeds Jus#fyingbudgets Improveprogramdesignsand processes Enhancethetransferoflearning Eliminateofexpand Enhancetherespectandcredibility ofLTDsta Increasesupportfrommanagers Strengthenrela#onshipwithkey execu#veandadministrators Setpriori#esforlearningand development 208 The Chief Learning Ofcer The ROI Process Evaluation Planning Data Collection Collect Data During Solution Implementation LEVEL 1: REACTION, SATISFACTION, AND PLANNED ACTIONS LEVEL 2: LEARNING Collect Data After Solution Implementation LEVEL 3: APPLICATION/ IMPLEMENTATION LEVEL 4: BUSINESS IMPACT Develop Objectives of Solution(s) Develop Evaluation Plans and Baseline Data Convert Data to Monetary Value Calculate the Return On Investment Generate Impact Study Data Analysis Identify Intangible Measures LEVEL 5: ROI Reporting INTANGIBLE BENEFITS Isolate the Effects Tabulate Costs Of Solution Figure 8-4. ROI methodology. (Source: Phillips, 2003. Used with permission.) 5
6. 6. EvaluationStudiesofCross-CulturalTraining Tools and methods in learning evaluation Collecting Data Isolating the Effects of Learning Converting Data to Monetary Values Surveys Questionnaires Tests On-the-job observation Interviews Focus groups Action plan and program assignments Performance contracts Business performance monitoring Control group Pre-post measurements Trend lines and forecasting Participants or managers Other experts Output data is converted to profit contribution or cost savings The cot quality is calculated Wages and benefits for the used of time value Historical cost and current used Opportunity cost External database Participant estimation from different groups 6
7. 7. EvaluationStudiesofCross-CulturalTraining 7 When to Use the Training: International Assignment Developing expatriate-profiles and conducting selection Preparation of the expatriates and their families (e.g. cultural awareness training, etc.) Conceptualization of an escorting system during the assignment (e.g. coaching, mentoring, etc.) Return planing and reintegration seminar Internal marketing to motivate employee for international assignment Marketing Selection PreparationSupporting Reintegration International Assignment Dierenttypesofculturalawarenesstrainingishelpfulforeachstageofthecircle 10%-50%ofexpatriatesreturningearlyfromtheirassignment. ExpatriatesdiculAesarecostlyforMNCs,rangingfrom$250,000to$1million. (Eschbachetal.,2001)
8. 8. EvaluationStudiesofCross-CulturalTraining Generic Design of Cross Cultural Awareness Training 8 Lunch Dinner Break Break Day 3 on the Way to Cross-Cultural Synergy Personal Charter & Learning Plan Day 1 Critical Situations in Working Places My understanding of others: Self and other perception Culture & Management: Working in International Environments Cultural Encounter Situations Learning in Action Short Lecture & Discussion Joint Dinner Case Study: Cross-cultural Leadership Day 2 Communication Across Cultures Role Play: First Meeting Case and Role Play: Conflict Management and Team Building
9. 9. EvaluationStudiesofCross-CulturalTraining How Effective is a Cross-Cultural Awareness Training in Developing Intercultural Competence? 9
10. 10. EvaluationStudiesofCross-CulturalTraining Summary of the Evaluation Studies on Awareness Training Some evaluation confirms the effect of CCT on trainees cross-cultural skill, adjustment, and performance. Others have doubts regarding the quality of these studies whether CCT effectiveness has been demonstrated. Dont establish a link between training and the measured effect on a dependent variable. The effectiveness of CCT in these studies was regarded as too positive in previous reviews. In general, it remains unclear which method of CCT is overall most effective and which methods are most effective in specific contingencies. The result for the same studies does not always correspond in different reviews due to the different perspectives taken by the reviewers. Littrell & Salas, 2005
11. 11. EvaluationStudiesofCross-CulturalTraining English language vs. Multiple languages: Eglish,French,German, Italian,and Spanish. Specific domains vs. Wide variety of disciplines: Management, military studies, human resource management, cross-cultural psychology, intercultural relation, immigrant studies, education, anthropology, political science and diplomacy. Domain of the Evaluation Study
12. 12. EvaluationStudiesofCross-CulturalTraining Findings Methods Lectures, culture assimilators and class discussions were the most often used. Training was composed of three or more methods . U.S. trainees were evaluated most. Over half of the treatment groups were students. Lack of manager subjects. Participants
13. 13. EvaluationStudiesofCross-CulturalTraining Training Methods Used in the Evaluation Studies
14. 14. EvaluationStudiesofCross-CulturalTraining Findings Focus Most training focus on a specific culture. Others were culture general or a combination. Japanese, U.S. and Canadian cultures were the most common focus. Timing The training took place either prior to, during, or independently of a sojourn in another country. The duration of the training varied from1/2 to 50 hours over a period of time ranging from 1 day to 8 months.
15. 15. EvaluationStudiesofCross-CulturalTraining Trainee satisfaction Academic achievement, contribution to company goal. Perceived well-being, health and safety concerns, adjusted to a foreign culture, perceived control of the environment. The ability to make isomorphic attributions awareness of cultural differences Cultural interest, positive attitude toward members of other cultures, trainee ethnocentrism. Problem-solving ability, the ability to deal with cross-culture misunderstandings, display of cultural sensitivity Expected Results of Awareness Training Knowledge Behavior Attitude Adjustment Performance Satisfaction
16. 16. EvaluationStudiesofCross-CulturalTraining Cross-Cultural Training Effectiveness
17. 17. EvaluationStudiesofCross-CulturalTraining Some Critical Issues of the Current Evaluation Studies Control Groups Pre- and Post-Training Test Randomly Assign Long-term vs. Short Term Effects Different Measurement Methods Placebo and Hawthorne Effects
18. 18. EvaluationStudiesofCross-CulturalTraining 18 International Project References in the Research Areas of Culture Diversity,Talent Development,and Strategic Change 14yearsinGermany Bornandgrewup inIndonesia 8yearsinChina Building Global Competence for Asian Leaders Applying Social Medias (Web 2.0) in Learning & Development. International Employability:Development of Intercultural Competence of German and Chinese Young Professionals. Cross-Cultural Learning Behavior: Effectiveness of the Western Technology Transfer and Learning Approaches in China. Comparative Studies of Chinese-Indonesian Intercultural Competence and Sensitivity. Dynamic Decision Making in Chinese and Multinational Teams. Intercultural Perspectives of International Post-Merger Integration in Europe. Intercultural Synergy in Professional Team. Complex Problem Solving in Small Groups. ...
19. 19. EvaluationStudiesofCross-CulturalTraining Conclusions and Implications Quantitative and Qualitative Time Span Wintry Doctrine Low Motivation Moderators The Paucity of theory Cooperation and Resistance
20. 20. ThankYou Contact us via Mail: hora_t@mac.com Follow: twitter@htjitra Website: http://horatjitra.com ZhejiangUniversity,Hangzhou(China) Based on: LISA N. LITTRELL, EDUARDO SALAS: A Review of Cross-Cultural Training: Best Practices, Guidelines, and Research Needs. Human Resource Development Review, 4(3), 2005. p305-334