just how comparative are comparative statistics? roger jowell centre for comparative social surveys...
TRANSCRIPT
![Page 1: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/1.jpg)
Just how comparative are comparative statistics?
Roger Jowell
Centre for Comparative Social Surveys
City University
LLAKES International Conference
![Page 2: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/2.jpg)
www.europeansocialsurvey.org
Why bother with comparative social statistics?
Discovering differences and similarities
“Comparative sociology is not a particular branch of sociology. It is sociology itself.” Durkheim
Understanding one’s own society
Critical for cross-national governance
Helps challenge national stereotypes
But comparisons ideally needed both between countries AND over time
![Page 3: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/3.jpg)
www.europeansocialsurvey.org
Obstacles to cross-national social measurement
Cultural incompatibilities
The language barrier
Breaching the ‘principle of equivalence’
Temptation of league tables
Clash between standards and consistency
Differences in methodological capacity
Differences in methodological habits
![Page 4: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/4.jpg)
www.europeansocialsurvey.org
Overcoming the obstacles – at least partially
Ensuring equivalence of methods in all countries:Sampling
Mode of data collection
Language equivalence
Concept equivalence
Meticulous documentation
Consultative design, not ‘safari’ method
Contextual variables
![Page 5: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/5.jpg)
www.europeansocialsurvey.org
A case history - the European Social Survey
Multinational time series, started 2001
34 countries so far
Contributes to scholarship and governance
Large training component and potential
Several substantive and methodological innovations
Widespread usage
![Page 6: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/6.jpg)
www.europeansocialsurvey.org
Five main aims
To chart and explain changes in Europe’s social, political and moral climate
To achieve and spread high standards of rigour in comparative social measurement
To establish new indicators of societal well-being to stand alongside existing factual and behavioural indicators
To facilitate quantitative monitoring of value change by academics, policymakers, businesses and the public
To create and maintain a new contextual data repository
![Page 7: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/7.jpg)
UKGreece Finland Ireland
PortugalSlovakia
Lithuania
Turkey
Switzerland Latvia
Austria Czech Republic
Luxembourg IcelandBulgaria
Ukraine
CyprusNorway
CroatiaRomania
Italy
Spain
FranceIsrael
SwedenRussia
Denmark
Belgium Estonia
HungaryPolandNetherlands
Slovenia
Germany
ESS Countries
Headquarters London, UK
![Page 8: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/8.jpg)
ESS Countries by Round
COUNTRY R1 R2 R3 R4
Austria
Belgium
Bulgaria x x
Croatia x x x
Cyprus x x
Czech x x
Denmark
Estonia x
Finland
France
Germany
Greece x
Hungary
Iceland x x x
Ireland
Israel x x
Italy x x
COUNTRY R1 R2 R3 R4
Latvia x x
Lithuania x x x
Luxembourg x x
Netherlands
Norway
Poland
Portugal
Romania x x
Russia x x
Slovakia x
Slovenia
Spain
Sweden
Switzerland
Turkey x x
Ukraine x
UK
![Page 9: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/9.jpg)
www.europeansocialsurvey.org
Funding story to date
Initiated and seed-funded by European Science Foundation
Then core-funded for five biennial rounds (to date) by European Commission
National costs of each round met by national academic funding councils
Over 30 separate funding decisions each round
Now selected as a prospective ‘ESFRI’ Research Infrastructure with prospect of long-term funding
![Page 10: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/10.jpg)
www.europeansocialsurvey.org
Question clusters – some core, some rotating
Trust in institutions
Citizen engagement
Socio-political values
Immigration
Moral & social attitudes
Quality of life
Crime and security
Value orientations
Perceptions of criminal justice
National, ethnic, religious ID
Health and welfare issues
Life course perceptions
Ageism
Work and family life
Education and occupation
Financial circumstances
Household circumstances
Demographic composition
![Page 11: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/11.jpg)
www.europeansocialsurvey.org
Outreach
32,000 registered data users to date (2500+ in UK)
On-line bibliography of publications based on ESS contains: 236 journal articles, 36 books and 90 chapters so far
Data increasingly deployed in policy debates
Training courses heavily over-subscribed
Influence on comparative methods well beyond Europe
![Page 12: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/12.jpg)
www.europeansocialsurvey.org
A few summary findings
![Page 13: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/13.jpg)
www.europeansocialsurvey.org
Education and Political Interest
Education and Interest in Politics in 22 countries
0
10
20
30
40
50
60
70
80
6 orfeweryears
7 8 9 10 11 12 13 14 15 16 17 ormoreyears
Years in education
% 'v
ery
' o
r 'f
air
ly' in
tere
ste
d in
po
liti
cs
![Page 14: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/14.jpg)
Four Dimensions of Institutional Trust…
0
1
2
3
4
5
6
7
8Austria
BelgiumSw itzerland
Czech Republic
Germany
Denmark
Spain
Finland
France
United Kingdom
Greece
Hungary
Ireland
IsraelItaly
LuxembourgNetherlands
Norw ay
Poland
Portugal
Sw eden
Slovenia
Bulgaria
Cyprus
Estonia
Iceland
Russia
Slovakia
TurkeyUkraine
Trust in the police
Trust in the police varies but is high…
![Page 15: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/15.jpg)
Four Dimensions of Institutional Trust…
0
1
2
3
4
5
6
7
8Austria
BelgiumSw itzerland
Czech Republic
Germany
Denmark
Spain
Finland
France
United Kingdom
Greece
Hungary
Ireland
IsraelItaly
LuxembourgNetherlands
Norw ay
Poland
Portugal
Sw eden
Slovenia
Bulgaria
Cyprus
Estonia
Iceland
Russia
Slovakia
TurkeyUkraine
Trust in the legal system Trust in the police
Trust in the legal system is lower….
![Page 16: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/16.jpg)
Four Dimensions of Institutional Trust…
0
1
2
3
4
5
6
7
8Austria
BelgiumSw itzerland
Czech Republic
Germany
Denmark
Spain
Finland
France
United Kingdom
Greece
Hungary
Ireland
IsraelItaly
LuxembourgNetherlands
Norw ay
Poland
Portugal
Sw eden
Slovenia
Bulgaria
Cyprus
Estonia
Iceland
Russia
Slovakia
TurkeyUkraine
Trust in country's parliament Trust in the legal system Trust in the police
Trust in national parliaments is lower still…
![Page 17: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/17.jpg)
Four Dimensions of Institutional Trust…
0
1
2
3
4
5
6
7
8Austria
BelgiumSw itzerland
Czech Republic
Germany
Denmark
Spain
Finland
France
United Kingdom
Greece
Hungary
Ireland
IsraelItaly
LuxembourgNetherlands
Norw ay
Poland
Portugal
Sw eden
Slovenia
Bulgaria
Cyprus
Estonia
Iceland
Russia
Slovakia
TurkeyUkraine
Trust in politicians Trust in country's parliament Trust in the legal system Trust in the police
Trust in politicians is the lowest of all.
![Page 18: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/18.jpg)
www.europeansocialsurvey.org
Attitudes to migration
Surges of xenophobia associated with economic downturns
New dangers of recession
But education matters
More education, less xenophobia, greater sympathy towards cultural diversity
Educated are more accepting of all newcomers, even of potential labour market competitors
![Page 19: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/19.jpg)
www.europeansocialsurvey.org
10 Laws of comparative research
![Page 20: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/20.jpg)
www.europeansocialsurvey.org
Law 1
Don’t confuse respect for cultural differences with tolerance of methodological anarchy
![Page 21: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/21.jpg)
www.europeansocialsurvey.org
Law 2
Never design questions or interpret data about a country one knows little or nothing about
![Page 22: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/22.jpg)
www.europeansocialsurvey.org
Law 3
Confine cross-national studies to the smallest number of nations compatible with the study’s intellectual needs
![Page 23: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/23.jpg)
www.europeansocialsurvey.org
Law 4
Pay as much attention to collecting aggregate-level background information about each country as to individual-level variables
![Page 24: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/24.jpg)
www.europeansocialsurvey.org
Law 5
Always be at least as absorbed by the limitations of the data as about their explanatory power
![Page 25: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/25.jpg)
www.europeansocialsurvey.org
Law 6
Assume initially that any major ‘new’ cross-national variation one discovers is an artefact
![Page 26: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/26.jpg)
www.europeansocialsurvey.org
Law 7
Resist the temptation to produce ‘gee-whiz’ league tables containing every nation in every analysis
![Page 27: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/27.jpg)
www.europeansocialsurvey.org
Law 8
Undertake collective, study-specific, multi-national development work and pre-testing
![Page 28: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/28.jpg)
www.europeansocialsurvey.org
Law 9
Routinely include methodological experiments in cross-national studies
![Page 29: Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference](https://reader034.vdocument.in/reader034/viewer/2022051516/56649e9f5503460f94ba2180/html5/thumbnails/29.jpg)
www.europeansocialsurvey.org
Law 10
Ensure that cross-national datasets are accompanied by detailed methodological reports about procedures and outcomes in each nation