MPLUS USER MEETING UTRECHT Peter Schmidt University of Giessen Approximate Measurement Invariance in Cross-Cultural and Comparative Research: Results from four studies Overview General Approach Study 1: Attitude toward Immigration in the ESS Study 2: Revised Value Scale with 19 values Study 3: Invariance of Universalism Value over time and countries in the ESS Study 4: Invariance of four Higher Order Factors of Values in the ESS Outlook: Approximate Measurement Invariance, Robustness checks, Multilevel CFA/SEM as explanatory tool. Measurement invariance - psychometric property of a questionnaire The questionnaire is measurement invariant when it measures -the same construct -in the same way -across different groups, such as countries, cultures or other geographical regions, conditions of data collection or time points Measurement invariance is a precondition for any meaningful comparison of means, correlates and regression coefficients of the measured construct across groups(Proof given by Meredith 1993, elaborated in Millsap 2011, Guenole/Brown 2015) RECENT CONTROVERSY Measurement Invariance in the World Value Study Aleman/Woods (2015): Our findings indicate that for the most part WVS orientations are not configural, metric and scalar invariant and hence comparable cross- nationally (over 89 nations), except among a small number of Western postindustrial societies (p. 21). Welzel/Ingelhart(2016) Approaches to measurement invariance 1) Assuming it ( dangerous) 2) Empirical assessment establishing full MI (rather seldom) in case of lack of MI - looking for partial MI (Byrne, Shavelson & Muthn 1989) - dropping groups or items - refraining from cross-group comparisons -looking for alternative appropriate methods to assess cross-group invariance - checking for robustness (Oberski 2014, Kouha/Moustaki 2015) Most often used approach to test for measurement invariance: 1) Multigroup Confirmatory Factor Analysis - MGCFA (Bollen 1989, Jreskog 1971) 1) configural invariance 2) metric invariance 3) scalar invariance Latent variable Quest. a Quest. b Quest. c Latent variable Quest. a Quest. b Quest. c Group AGroup B 1 1B 2B 2A 1A 1A 2A 3A 1B 2B 3B 1 2) Evaluation based on differences in global model fit indices between models (Chen, 2007) Alternative approaches in the framework of MGCFA: 2) Evaluation of exact measurement invariance based on local misspecifications (Saris, Satorra & van der veld, 2009) 1) Test for approximate (Bayesian) rather than exact measurement invariance (Muthn & Asparouhov, 2013, van der Schoot et al. 2014) Similar assumption in both approaches: allowing for some small deviation Note! Study 1 The comparability of attitudes toward immigration in the European Social Survey: Exact versus approximate measurement equivalence (Public Opinion Quarterly 2015) Eldad Davidov University of Zurich Jan Cieciuch University of Zurich, University of Finance and Management in Warsaw Peter Schmidt University of Giessen Bart Meuleman University of Leuven Ren Algesheimer University of Zurich Data and Measurements A total of 35 countries and 6 rounds of the ESS (2002/3, 2004/5, 2006/7, 2008/9, 2010/11, 2012/13) are included in the study. Not all countries participated in all rounds. Some joined early on in 2002/3 and did not participate in other later rounds. Other countries were not part of the ESS at the beginning but joined later. Data and Measurements Table 1 summarizes the number of participants in each round, the percentage of female respondents, and the mean and standard deviations of the respondents age in each country. Data in each country included both respondents with and without immigration background. We excluded respondents with a migration background from our analysis to avoid positivity bias in the scores. Data and Measurements Thus, the total sample included 271,220 respondents. The data were retrieved from the ESS website, Further information on data collection procedures, the full questionnaire, response rates, and methodological documentation is available on the ESS website. Data and Measurements Three items in the ESS measured attitudes toward immigration. They ask whether respondents prefer their country to allow more or fewer immigrants who belong to a certain group to come into the country. Data and Measurements The first group consists of people of the same race or ethnic group from most [country] people, the second group consists of people of a different race or ethnic group from most [country] people, and the third consists of people from poorer countries outside Europe. Respondents record their responses to these three questions on 4-point scales ranging from 1 (allow none) to 4 (allow many). Data and Measurements Number of respondents (N) by country and ESS round with % of female (% F) and mean and standard deviation of age 14 1st Round (2002/3)2nd Round (2004/5) 3rd Round (2006/7)4th Round (2008/9) 5th Round (2010/11) 6th Round (2012/13) N% FM age (SD age ) N% FM age (SD age ) N% FM age (SD age ) N% FM age (SD age ) N% FM age (SD age ) N% FM age (SD age ) 1. Austria (17.19) (17.91) (17.91) (18.52) 2. Belgium (18.48) (18.48) (18.86) (19.00) (19.16) (19.47) 3. Bulgaria (17.80) (17.64) (17.84) (16.95) 4. Croatia (18.25) (18.99) 5. Cyprus (17.54) (18.04) (18.91) (18.59) 6. Czech Republic (17.55) (17.88) (17.37) (17.64) (17.11) 7. Denmark (17.73) (17.78) (17.61) (18.09) (18.62) (19.22) 8. Estonia (19.48) (19.22) (18.98) (19.43) (19.41) 9. Finland (18.53) (18.67) (19.05) (18.76) (19.27) (18.92) 10. France (18.56) (18.04) (17.84) (18.96) (18.56) Data and Measurements Number of respondents (N) by country and ESS round with % of female (% F) and mean and standard deviation of age 15 1st Round (2002/3)2nd Round (2004/5) 3rd Round (2006/7)4th Round (2008/9) 5th Round (2010/11) 6th Round (2012/13) N% FM age (SD age ) N% FM age (SD age ) N% FM age (SD age ) N% FM age (SD age ) N% FM age (SD age ) N% FM age (SD age ) 11. Germany (17.95) (17.97) (18.12) (17.43) (18.53) (18.74) 12. Greece (19.22) (18.85) (16.87) (19.05) 13. Hungary (18.20) (18.09) (18.54) (19.10) (18.35) (18.20) 14. Iceland (17.71) (18.84) 15. Ireland (17.84) (18.08) (18.35) (18.29) (19.12) (18.17) 16. Israel (15.79) (16.07) (16.87) (16.50) 17. Italy (17.89) (18.09) 18. Kosovo (17.04) 19. Latvia (19.06) (18.56) 20. Lithuania (18.86) (19.46) Data and Measurements Number of respondents (N) by country and ESS round with % of female (% F) and mean and standard deviation of age 16 1st Round (2002/3)2nd Round (2004/5) 3rd Round (2006/7)4th Round (2008/9) 5th Round (2010/11) 6th Round (2012/13) N% FM age (SD age ) N% FM age (SD age ) N% FM age (SD age ) N% FM age (SD age ) N% FM age (SD age ) N% FM age (SD age ) 21. Luxembour g (19.65) (18.78) 22. Netherland s (17.13) (17.49) (17.87) (18.00) (17.66) (18.16) 23. Norway (17.22) (17.43) (18.32) (18.14) (18.76) (18.38) 24. Poland (18.51) (17.92) (18.45) (18.86) (18.74) (18.69) 25. Portugal (19.11) (19.48) (19.02) (19.87) (19.19) (19.08) 26. Romania (18.45) (17.64) 27. Russia (19.11) (19.06) (18.61) (18.12) 28. Slovakia (17.83) (17.79) (17.16) (17.39) (16.56) 29. Slovenia (18.58) (19.21) (19.06) (19.08) (18.73) (19.06) Data and Measurements Number of respondents (N) by country and ESS round with % of female (% F) and mean and standard deviation of age 1st Round (2002/3)2nd Round (2004/5) 3rd Round (2006/7)4th Round (2008/9) 5th Round (2010/11) 6th Round (2012/13) N% FM age (SD age ) N% FM age (SD age ) N% FM age (SD age ) N% FM age (SD age ) N% FM age (SD age ) N% FM age (SD age ) 30. Spain (19.32) (18.94) (19.09) (19.38) (18.57) (18.29) 31. Sweden (18.75) (19.00) (18.92) (19.33) (19.54) (19.26) 32. Switzerland (17.67) (18.50) (18.32) (18.89) (19.38) (19.32) 33. Turkey (16.74) (16.39) 34. Ukraine (18.74) (18.81) (18.68) (18.94) 35. UK (18.60) (18.92) (19.18) (18.56) (18.91) (19.24) Total38,192 44,98843,33555,52047,47941,706 Plan of Analysis 1. Testing for exact (full or partial) scalar equivalence First, we ran 6 MGCFA analyses using the full information maximum likelihood (FIML) procedure (Schafer and Graham 2002), one for each round, with all the countries included in this round. Each analysis contained three assessments for configural, metric, and scalar equivalence, respectively, with the corresponding constraints for the metric and scalar levels of measurement equivalence. To identify the model we used the approach proposed by Little, Slegers, and Card (2006) and constrained the loading of one of the items to 1 and the intercept of this item to 0 in all countries. Plan of Analysis 1. Testing for exact (full or partial) scalar equivalence If it turned out that the loading and/or intercept of this item varied considerably across countries, we used a different reference item for identification. When full measurement equivalence was not established, we tried to assess partial measurement equivalence. We used the program Jrule (Saris, Satorra and van der Veld 2009; Oberski 2009) for the detection of local misspecifications of parameters whose equality constraint should be released according to the program. Plan of Analysis 1. Testing for exact (full or partial) scalar equivalence In order to establish partial scalar equivalence, only one item could be released, because partial scalar equivalence requires that parameters of at least two items are constrained to be equal across all groups. However, results of analyses using Jrule indicated misspecifications for two or even three items in several countries. This indicated that in these countries even partial scalar equivalence could not be established. Plan of Analysis 2. Testing for approximate scalar equivalence Assessing approximate measurement equivalence using Bayesian analysis requires imposing priors on specific parameters. When testing for approximate measurement equivalence, the average difference between loadings and intercepts across countries is assumed to be zero as in MGCFA when one tests for exact measurement equivalence with one exception: Approximate measurement equivalence permits small differences between parameters otherwise constrained to be exactly equal in the classical approach for testing for measurement equivalence. Plan of Analysis 2. Testing for approximate scalar equivalence van de Schoot et al. (2013) demonstrated, using simulation studies, that variance as large as 0.05 imposed on the difference between the loadings or the intercepts does not lead to biased conclusions when approximate equivalence is assessed. We followed their recommendations and imposed the following priors on the difference parameters of the loadings and intercepts: mean difference = 0, variance of the difference =.05. Plan of Analysis 2. Testing for approximate scalar equivalence We used similar constraints to identify the model as in the MGCFA: We constrained the loading of one item to (exactly) 1 in all groups and the intercept of this item to (exactly) 0 in all groups. If the loading and/or intercept of this item varied considerably across countries, we chose a different reference item to use for identification. The latent mean was freely estimated in all countries. Plan of Analysis A measurement model with a latent variable measuring attitudes toward immigration with three items (Item 1 Item 3) and three measurement errors (e1-e3). Results Global fit measures for the exact measurement equivalence test in each ESS round 25 Chi2dfRMSEASRMRCFI 1st Round of ESS Configural Metric [ ] Partial metric [ ] Partial scalar [ ] nd Round of ESS Configural Metric [ ] Partial metric [ ] Partial scalar [ ] rd Round of ESS Configural Metric [ ] Partial metric [ ] Partial scalar [ ] Results Global fit measures for the exact measurement equivalence test in each ESS round 26 Chi2dfRMSEASRMRCFI 4rd Round of ESS Configural Metric [ ] Partial metric [ ] Partial scalar [ ] th Round of ESS Configural Metric [ ] Partial metric [ ] Partial scalar [ ] th Round of ESS Configural Metric [ ] Partial metric [ ] Partial scalar [ ] Results Countries with misspecified two or three intercepts according to Jrule (criterion >.01) with the percentage of countries that did not reach partial scalar equivalence on the second row 27 ESS1ESS2ESS3ESS4ESS5ESS6 9% countries15% countries40% countries32% countries37% countries42% countries Hungary Israel Estonia Portugal Slovenia Ukraine Bulgaria Cyprus Denmark Estonia Hungary Latvia Russia Spain Switzerland Ukraine Bulgaria Denmark Estonia Germany Hungary Israel Latvia Lithuania Norway Ukraine Denmark Estonia Germany Hungary Israel Lithuania Netherlands Spain Switzerland Ukraine Cyprus Estonia Germany Hungary Iceland Israel Kosovo Netherlands Portugal Switzerland Evaluation Criteria Were based on the ppp and the 95% C.I. Non significant ppp and a 95% C.I. that contained a zero were treated as indications of an acceptable model fit Second in the difference output provided by Mplus we inspected the loadings or intercepts of those items that are significantly noninvariant (deviating significantly from the average parameter across groups) Results Fit measures for the approximate measurement equivalence model in each ESS round ppp 95% Confidence Interval 1st Round of ESS.057( ) - ( ) 2nd Round of ESS.422(-53.57) - ( ) 3rd Round of ESS.364( ) - ( ) 4rd Round of ESS.220( ) - ( ) 5th Round of ESS.340( ) - ( ) 6th Round of ESS.320( ) - ( ) 95% Confidence Interval = 95% Confidence Interval for the difference between the observed and the replicated chi-square values ppp = the posterior predictive p-value Results Correlations of country rankings based on three methods (exact equivalence, approximate equivalence and raw scores) in six ESS rounds (ESS1/ESS2/ESS3/ESS4/ESS5/ESS6) Exact (partial scalar model)Approximate scalar model.995 /.998 /.993 /.988 /.992 /.973 Raw scores.954 /.971 /.970 /.956 /.971 / /.972 /.975 /.955 /.966 /.980 Study 2: Invariance of 19 Values 31 Schwartzs theory of basic human values 1) Structure: circumplex continuum 2) Content: 10 value types Basic values - Beliefs about the importance of abstract goals as guiding principles in life Previous findings of values measurement invariance Most of the published analyses were conducted on the ESS data (PVQ-21) by Davidov and colleagues (e.g., Davidov, 2008; Davidov, 2010; Davidov, Schmidt, & Schwartz, 2008) PVQ-21 (in the ESS) to measure 10 values with the old value model - a disappointing result (Davidov, Schmidt, & Schwartz, 2008) Lack of scalar measurement invariance PVQ-57 to measure 19 values based on the new value model - an encouraging result (Cieciuch et al., 2014) Schwartzs refined theory of basic human values 1) Values are more narrowly defined (19). 2) There is greater homogeneity of items. 3) Each value is measured by three (rather than two) items. The old value circle Tolerance Nature Concern Caring Dependability Humility Interpersonal Rules Societal Personal Face Dominance Resources Thought Action The new value circle 10 values19 values PVQ-57 PVQ-40 Schwartzs refined theory of basic human values New PVQ5x developped to measure 19 values Sample eight countries: Finland, Israel, Italy, New Zealand, Poland, Portugal, Switzerland, USA PVQ-57 to measure 19 values CountryN Finland334 Germany325 Israel394 Italy382 New Zealand141 Poland545 Portugal295 Switzerland201 Full metric invariance: 16 of the 19 values and 3 values - full or partial metric invariance across alomst all countries Full or partial scalar invariance: 10 of 19 values across almost all countries (with a few exceptions for single countries): benevolence caring, universalism tolerance, universalism concern, universalism nature, hedonism, power dominance, power resources, security personal, security societal, self-direction thought Encouraging results of exact MGCFA Cieciuch, Davidov, Vecchione, Beierlein, Schwartz, 2014 -Metric: ok! Results of the exact MGCFA Cieciuch, Davidov, Vecchione, Beierlein, Schwartz, Scalar: better than with the earlier PVQ version BUT there is still room for improvement 10 values invariant and 9 values noninvariant Is the test too strict?? Lets focus on the scalar measurement invariance and look into methods = that allow for small deviations Defined by the researcher the size of misspecification in the Saris et al. approach Defined by the researcher the variance of parameters in the Bayesian approach misspecification of intercepts >.1 variance of intercepts =.01 variance of intercepts =.05 Jrule reads output from MplusMplus For each value separately, because of two reasons 1)PPP with higher-order values and multiple values was always significant 2)We were interested only in scalar invariance test, because metric and configural invariance were already established We present one example in detail (SDT = self-direction thought) and a summary for all other values Self-direction thought Jrule Misspecification at.1 Mplus Priors: variance of intercepts =.01 Conclusions: 1) The results are very similar 2) Only two exceptions: -SDT1 in Israel: misspecified in Jrule, but not in Bayes -SDT3 in Poland: misspecified in Bayes but not in Jrule ppp =.495; CI = ( ) ( ) Self-direction thought Jrule Misspecification at.1 Mplus Priors: variance of intercepts =.05 Conclusions: 8 parameters misspecified in Jrule, while in Bayes 4 parameters are misspecified ppp =.502; CI = ( ) ( ) Conclusion Detection for local misspecification Test for approximate measurement invariance Diagnosis of ill items is quite similar BUT the treatment (therapy) is different In order to reach an acceptable model, there is a need to release the misspecified parameters It can lead to - lack of measurement invariance - or to dropping groups There is no need to release the misspecified items, if the ppp indicates an acceptable model fit Summary SDTSDASTHE Finland*--** Germany*--** Israel*--** Italy*--** New Zealand*--** Poland*--- Portugal*--** Switzerland*--- Exact MI ** = full scalar MI * = partial scalar MI - = lack of scalar MI Approximate MI ok = scalar MI Openness Summary SDTSDASTHE Finlandok Germanyok Israelok Italyok New Zealandok Polandok Portugalok Switzerlandok Exact MI ** = full scalar MI * = partial scalar MI - = lack of scalar MI Approximate MI ok = scalar MI Openness ACPODPOR Finland-** Germany-** Israel-*** Italy-** New Zealand-** Poland-**- Portugal--** Switzerland-** Summary Self-enhancement Exact MI ** = full scalar MI * = partial scalar MI - = lack of scalar MI Approximate MI ok = scalar MI Results of Bayesian analysis Summary Self-enhancement Variance =.05 Value (number of items)95% CIppp Achievement (3)-24.31; Power Resources (2)-25.38; Power Dominance (2)-24.72; ACPODPOR Finlandok Germanyok Israelok Italyok New Zealandok Polandok Portugalok Switzerlandok Summary Self-enhancement Exact MI ** = full scalar MI * = partial scalar MI - = lack of scalar MI Approximate MI ok = scalar MI Summary Conservation Exact MI ** = full scalar MI * = partial scalar MI - = lack of scalar MI Approximate MI ok = scalar MI FACSEPSESTRCORCOIHU Finland-***---- Germany-***---- Israel--*---- Italy-***---- New Zealand-***---- Poland-***---- Portugal-***---- Switzerland--*---- Summary Conservation Exact MI ** = full scalar MI * = partial scalar MI - = lack of scalar MI Approximate MI ok = scalar MI FACSEPSESTRCORCOIHU Finland ok Germany ok Israel ok Italy ok New Zealand ok Poland ok Portugal ok Switzerland ok Summary Self- transcendance Exact MI ** = full scalar MI * = partial scalar MI - = lack of scalar MI Approximate MI ok = scalar MI UN N UNCUNTBEC BE D Finland** *- Germany**- - Israel*** - Italy*** - New Zealand**** - Poland** - - Portugal***- - Switzerland-** - Summary Self- transcendance Exact MI ** = full scalar MI * = partial scalar MI - = lack of scalar MI Approximate MI ok = scalar MI UN N UNCUNTBEC BE D Finland ok Germany ok Israel ok Italy ok New Zealand ok Poland ok Portugal ok Switzerland ok ESS sample sizes for the selected 15 countries over six ESS rounds ( ) 1st Round (2002/3) 2nd Round (2004/5) 3rd Round (2006/7) 4th Round (2008/9) 5th Round (2010/11) 6th Round (2012/13)N Belgium1,8991,7781,7981,7601,7041,86910,808 Switzerland2,0402,1411,8041,8191,5061,49310,803 Germany2,9192,8702,9162,7513,0312,95817,445 Denmark1,5061,4871,5051,6101,5761,6509,334 Spain1,7291,6631,8762,5761,8851,88911,618 Finland2,0002,0221,8962,1951,8782,19712,188 United Kingdom2,0521,8972,3942,3522,4222,28613,403 Hungary1,6851,4981,5181,5441,5612,0149,820 Ireland2,0462,2861,8001,7642,5762,62813,100 Netherlands2,3641,8811,8891,7781,8291,84511,586 Norway2,0361,7601,7501,5491,5481,62410,267 Poland2,1101,7161,7211,6191,7511,89810,815 Portugal1,5112,0522,2222,3672,1502,15112,453 Sweden1,9991,9481,9271,8301,4971,84711,048 Slovenia1,5191,4421,4761,2861,4031,2578,383 N29,41528,44128,49228,80028,31729,606173,071 STUDY 3 : UNIVERSALISM in the ESS over 15 countries and 6 time Points F. Zercher/P. Schmidt/E. Davidov/ J. Ciechuch, Frontiers in Psychology, 2015. Analytical steps for the exact and the approximate measurement invariance approaches Traditional exact approach Approximate approach Steps 1. Configural model 2. Metric model 3. Scalar model 4. Partial scalar model 1. Setting different informative priors for all loadings and intercepts 2. Releasing constraints on those loadings and intercepts which are different Additional steps 5. Deleting groups which are not full or partial scalar invariant 3. Deleting groups which are not fully or partially approximately invariant Global fit measures of the traditional exact approach Chi(df)RMSEASRMRCFI Countries/ Timepoints Configural0(0)00115 Round 1 Metric54.55(28) Scalar (56) Partial Scalar64.89(24) Round 2 Metric45.23(28) Scalar (56) Partial Scalar53.28(28) Round 3 Metric49.86(28) Scalar611.49(56) Partial Scalar53.78(27) Round 4 Metric93.75(28) Scalar968.67(56) Partial Scalar87.43(24) Round 5 Metric107.04(28) Scalar925.79(56) Partial Scalar90.10(21) Round 6 Metric73.24(28) Scalar956.58(56) Partial Scalar69.26(21) All rounds simultaneously Configural0.395(0) Metric430.05(178) Scalar (356) Partial Scalar348.23(126) [ For the single rounds this refers to countries; for all rounds this is combination of country and time point. Countries still included are: Belgium ; Spain ; Finland ; United Kingdom 2012; Hungary ; Ireland 2008, 2010; Netherlands ; Norway ; Poland 2006; Portugal ; Sweden 2012; Slovenia 2002, 2006. Global fit measures for the approximate invariance test ppp ppp after releasing misspecified parameters Round Round Round Round Round Round groups groups Note: ppp = posterior predictive probability Correlations between latent means computed using sum scores (1), the exact (2) and the approximate (3) measurement invariance models for 73 county/time points Sum scores (1) Exact test (2) Approximate Bayesian test (3) ** **.844**1 Study 4: Invariance of four Higher Order Factors of Values in the ESS(submitted) 1st Round nd Round rd Round th Round th Round th Round Belgium1,8191,7341,7671,7041,6741, Denmark1,457 1,4511,5541,5481, Finland1,7581,6921,6451,8981,6382, Germany2,7852,8002,8282,6972,9432, Hungary1,5641,4071,4091,3881,4041, Ireland1,8381,1391,5821,6822,2952, Netherlands2,3011,8241,8141,6931,7541, Norway1,8061,5431,5331,3741,5181, Poland1,9821,6211,6291,5441,6751, Portugal1,4171,9872,1172,2202,0352, Slovenia1,3901,2971,3291,1721,2381, Spain1,6381,5441,8022,5201,8621, Sweden1,6771,6631,5851,5391,4571, Switzerland2,0092,0841,7581,7641,4671, United Kingdom1,7481,8062,3012,2302,3152,212 Total25,44123,79224,24924,74924,50826,385 Global fit indices for the approximate measurement invariance tests across a subset of countries ppp95% Credibility Interval Self-enhancement in 8 countries: Belgium, Denmark, Ireland, Netherlands, Portugal, Spain, Sweden 1st Round of ESS nd Round of ESS rd Round of ESS rd Round of ESS th Round of ESS th Round of ESS Global fit indices for the approximate measurement invariance tests across a subset of countries ppp95% Credibility Interval Self-transcendence in 12 countries: Belgium, Finland, Germany, Ireland, Netherlands, Norway, Poland, Portugal, Spain, Sweden, Switzerland, United Kingdom 1st Round of ESS nd Round of ESS rd Round of ESS rd Round of ESS th Round of ESS th Round of ESS 47.27 Global fit indices for the approximate measurement invariance tests across a subset of countries ppp95% Credibility Interval Conservation in 10 countries: Belgium, Finland, Germany, Ireland, Netherlands, Poland, Portugal, Switzerland, United Kingdom, Slovenia 1st Round of ESS nd Round of ESS rd Round of ESS rd Round of ESS th Round of ESS th Round of ESS 80.66 Global fit indices for the approximate measurement invariance tests across a subset of countries ppp95% Credibility Interval Conservation in 10 countries: Belgium, Finland, Germany, Ireland, Netherlands, Poland, Portugal, Switzerland, United Kingdom, Slovenia 1st Round of ESS nd Round of ESS rd Round of ESS rd Round of ESS th Round of ESS th Round of ESS 80.66 Conclusions Approximate invariance testing seems to be a useful alternative especially in the case of testing for scalar invariance in cross-cultural studies with many groups. There is a need for more empirical and simulation studies which contain: Number of groups larger than 100 including many countries and several time points Variation in the amount of misspecification Inclusion of 2 and more latent variables and items with cross-loadings and error correlations Summary table Study 1Study 2Study 3Study 4 Items Constructs11914 N of groups< Countries< Time points1161 Exact scalar 9% 42% 10 of Approx. scalarAll 73 8 12 Thank you for your attention!