Social-desirability in teachers’ studies based on self-reports – the case of Russia

Alena Kulikova

National Research University Higher School of Economics

Institute of Education

National Research University Higher School of Economics 2017

www.hse.ru

HSE Moscow, 2017 2

1937, G. Allport – First critic of the self-report method “Respondents could bias results of testing if they decided to do it”

Social DesirabilitySituational Motivation:

• Attitudes about desirability of misresponding in this situation

• Attitudes about allowability of misresponding in this situation

Personality:• Incentive motivation• Achievement motivation• Motivation of power• Agreeableness• Narcissism

Situation:• Anonymity• Instruction• Purpose• Administration• Construction of survey

Culture

Measurement of Social Desirability

Criterial Survey

Compare to objective data

Direct Indirect

Overclaiming technique (Paulhus et al, 2003)

- Lie scales (MMPI, CPI, etc);- BIDR and CIDR (Paulhus, 1998);- The Marlowe-Crowne scale.

-Special questions;- Response styles.

HSE Moscow, 2017

What the Social desirability

responding is…

3

Socially desirable responding (SDR) is the tendency of respondents to reply to a questionnaire in a manner that will be approved by other people.

➢ a threat to the validity of psychological and educational measures;

➢ needs to be controlled.

HSE Moscow, 2017

TALIS 2013

4

Teaching and Learning International Survey 2013

➢ 37 countries

➢ 4000 teachers in Russia, representative sample

➢ Teacher education, professional development, appraisal and feedback, school climate, pedagogical beliefs and practices

➢ Social desirability scale was included

• social desirability makes an impact on teachers’ responses

• the power of the impact of social expectations is associated with socio-economic level and educational achievement at country level (Vijver and He, 2015)

5

• Adopted version of The Marlowe–

Crowne Social Desirability scale:

– 10 questions describing SDR

in terms of school context

• Response styles:

– Extreme

– Middle

– Acquiescence

HSE Moscow, 2017

Jia He, Fons J. R. Van De Vijver, 2015

6

Aim: to discover what are the features of social desirability in teachers’ responses in case of Russia.

HSE Moscow, 2017

0

5

10

15

20

25

30

35

40

45

50

Never Once in a year 2-4 times in ayear

5-10 times in ayear

1-3 times in amonth

Once a weekor more

%

Observe other teachers’ classes andprovide feedback

International Average Russia

0

10

20

30

40

50

60

70

I took/take part in aninduction programme

I took/take part ininformal induction

activities

I took/take part in ageneral and/oradministrative

introduction to theschool

%

Induction programs

International Average Russia

7HSE Moscow, 2017

Sample• TALIS 2013 data• 20 countries

• 65535 teachers at all• 3972 teachers from Russia

Instrument• Adopted version of The Marlowe–Crowne scale

• 10 items• 7-point Likert scale

Methods• Item response theory (IRT) – Rating scale model• Correlation and regression analysis

1 Totally disagree

2 ..

3 ..

4 Neutral

5 ..

6 ..

7 Totally agree

8HSE Moscow, 2017

1) Delete 2 items: 9 and 102) Remove all respondents with the INFIT statistics more than +2 3) Remove two countries - Malaysia and Korea, because of large DIF4) Decrease the number of categories

from 7 to 4: 1223445

Psychometric quality:

- Reliability 0.8 (Alpha Cronbach)

- Unidimensional

- Categories work reasonable

- No large DIF across the countries

9HSE Moscow, 2017

Our hypotheses:

➢ SDR is positively associated with:• Job satisfaction• Self-efficacy• Age

➢ Class contingent in Russia is not homogeneous-> SDR could be related to amount of difficult students in class

10HSE Moscow, 2017

✓ Students whose first language is different from the language(s) of instruction or from a dialect of this/these language(s)

✓ Low academic achievers

✓ Students with special needs

✓ Students with behavioural problems

✓ Students from socioeconomically disadvantaged homes

▪ Based on IRT (rating scale model)

▪ Reliability 0,72

▪ Benchmarks based on the map of items

• low (from 0 to 10% “difficult” students in class),

• medium (10 to 30% “difficult” students in class),

• high (more than 30% “difficult” students in class)

11HSE Moscow, 2017

3,5

4

4,5

5

5,5

6

Age

Age groups

SDR Class contingent

Age

SDR 1 -,165** ,137**

Class contingent

-,165** 1 -,086**

Age ,137** -,086** 1

3

3,5

4

4,5

5

5,5

Class contingent

** correlation is significant at the 0.01 level (2-tailed)

Correlation analysis

12HSE Moscow, 2017

Predictor B P-value

Age -,074 ,000

Gender ,064 ,000

Class contingent -,024 ,112

SDR ,366 ,000

Self-efficacy

Predictor B P-value

Age ,103 ,000

Gender -,009 ,534

Class contingent -,078 ,000

SDR ,286 ,000

Job satisfaction

R square - 12% (without – 5%)R square - 15% (without SDR – 3%)

Regression analysis

13

Before

• SDR scale has two dimensions

• Partial invariance

• SDR negatively correlated with country affluence and educational achievement(Fons J. Van de Vijver, Jia He, 2014)

Now

• SDR scale is unidimensional scale, could be used in comparative studies

In case of Russia:

✓ Social Desirability is associated with the age, class contingent, job satisfaction and self-efficacy

✓ Young teachers – are the most trustful group of teachers (for studies based on self-report)

✓ For Russian data social desirability makes an impact on the results

Next

• Control characteristics at country level: compare countries using multilevel modelling

• Check the stability of the trend which has been found

HSE Moscow, 2017

Thank you for

attention!