(elisabeth prechtl, ph.d. student at university of bayreuth, germany) predictive validation of an...

Predictive Validation of

an Intercultural

Assessment Centre

(Elisabeth Prechtl, Ph.D. student at (Elisabeth Prechtl, Ph.D. student at

University of Bayreuth, Germany)University of Bayreuth, Germany)

Elisabeth Prechtl - Predictive Validation of an Intercultural Assessment Centre September 20082

Overview

1. Introduction: purpose of the study

2. Intercultural assessment centre

3. Design of the study

4. Validation strategy: Validation by prediction

5. Results

6. Implications and discussion


1. Introduction: Purpose of the Study

International companies send employees abroad

Increase in all kinds of assignments, expecially short-term assignments

(< 1 year, according to Cartus Global Mobility Policy & Practices Survey,

2007)

Success of the assignment is crucial to the company and subsidiary

more and more SME send employees abroad (e.g. frequent commuting)

Key issue for international HRM: selection of candidates,

assessment of an employee´s competences/ skills in dealing with

different cultures (before assignment),


1. Introduction: Purpose of the Study

Common practice:

- employees are sent abroad because of their previous overseas

experience and the number of languages they speak

- high reliance on self-selection of employees (in SME, Schmidt&

Minssen, 2007)

however, instruments for assessing intercultural competences are

available: - Questionnaires (Matsumoto et al., 2001, Earley et al., 2004),

- Assessment Centres (Lievens et al., 2003, Ehret,

2006)

can an instrument such as an Intercultural Assessment Centre increase the

prediction of success when other information is available?


2. Overview of the Intercultural Assessment Centre

Aim of this assessment centre: to assess a candidate’s potential for

intercultural encounters before being sent abroad or working in an

international team

Assessment of six intercultural competences (Inca-model, Prechtl&

Davidson-Lund, 2007):

- tolerance for ambiguity - Behavioral flexibility

- Respect for otherness - Empathy

- Knowledge discovery - Communicative

awareness

The candidates participate in different tasks which simulate „critical“

situations in contact with other cultures, like role-plays, group exercises

and written scenarios


2. Overview of the Intercultural Assessment Centre - Dimensions and Tasks-

Tolerance for ambiguity

Video exercise

Roleplay 1

Roleplay 2

Written scenario

Behavioral flexibility

Respect Empathy Knowledge

discovery

Communi-cative

awareness

Group roleplay


t 1 Intercultural Assessment Centre

t 2 candidate participates in an interview during

assignment concerning intercultural performance

(telephone interview)

t 3 supervisor/ collegue (from foreign culture) answers

questionnaire on candidate´s performance

3. Design of the Study

Candidates are sent abroad,

regardless of the result of this

assessment


Sample:

employees of small and medium-sized companies (+students on

internships) to be sent on short-term assignments/ commuting

assignment

t 1 112 subjects participate in 1day assessment (run in

Germany, 2004-2005)

t 2 51 subjects are interviewed during assignment in

different countries (self-rating,2005-2008)

t 3 29 supervisors/ collegues rate them (other-rating)




Countries of assignment


Approach: Validation by Prediction

-> Can intercultural success be predicted by the result of the

Assessment Centre? (predictive validity)

-> Can the Assessment Centre predict more of the success

criteria than biographical information? (incremental validity)

Method: Ordinal Regression Analysis

Parameters: R 2 according to Nagelkerke

significance of model fit

4. Validation by Prediction


Predictors:

Intercultural

Assessment Centre

(6 intercultural

competences)

Biographical data

previous overseas

experience,

number of foreign

languages

4. Validation by Prediction

Criteria of intercultural success:

Effectiveness

Adaptation to work

Commitment to foreign

organization

Satisfaction with work/ in general


Effectiveness and adaptation to work (self-rated):

variance explained by predictors

5. Results: a) Success rated by Candidate

0.12

0.34*

0.020.21

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

effectivity adaptation to work

biographical information

biographical plus interculturalassessment


Commitment to foreign organization (self-rated)


0.20

0.16

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

commitment (host)


biographical plus intercultural assessment


Satisfaction with living in the host country and

generally (self-rated)


0.180.30

0.090.18

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

satisfaction (living) satisfaction (generally)





Summary for self-rated success:

– Biographical information accounts for 2-21% of variance

– Intercultural assessment (dimensions of intercultural

competence) accounts additional 4-13% (increment)

– However, a large part of variance remains unexplained:

model of all predictors together is significant only for

adaptation to work


Effectiveness and adaptation to work (other-rated)

5. Results: b) Success rated by Supervisor

0.400.52*

0.07 0.03

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

effectivity adaptation to work




Commitment to foreign organization (other-rated)


0.55*

0.12

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

commitment


biographical plus interculturalassessment


Satisfaction with living and generally (other-rated)


0.60*0.71**

0.17 0.18

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

satisfaction (living) satisfaction (generally)




5. Results: a) Success rated by Supervisor

Summary for other-rated success:

– Biographical information accounts for 3-18% of variance

– Intercultural assessment (dimensions of intercultural

competence) accounts for additional 33-49% (increment)

– Both predictors together can explain a large part of

variance: model is significant for adaptation to work,

commitment and satisfaction


6. Some Implications…

Intercultural Assessment Center (intercultural dimensions)

does have an incremental validity over biographical data

an Intercultural Assessment Centre rating does enhance

the accuracy of prediction

amount of variance explained by predictors differs among

success criteria (e.g. more explained for other-rated criteria)

intercultural dimensions are most useful for the prediction

of success criteria, when other-rated success is

important

(e.g. when persons are rated by others from a different culture)

Discussion: is there an effect due to a self-other-rating-bias?

Thank you very

much for your

attention!!

contact:

elisabeth.prechtl(at)web.de

(elisabeth prechtl, ph.d. student at university of bayreuth, germany) predictive validation of an...

Documents

intercultural encounters

assessment centres lievens

validation strategy

prechtl davidsonlund

employees of small

selfselection of employees

employees abroadincrease

employees competences