russo ioe_nov12
DESCRIPTION
TRANSCRIPT
Interactions between the individual and the group:
reflections from multilevel modelling in educational research
Federica RussoCenter Leo Apostel, Vrije Universiteit Brussel &
Centre for Reasoning, University of Kent
2
Overview
Philosophy of education and empirical researchReverse the question: does empirical research look into philosophy?
Multilevel models in educational researchDefinition and examplesThe need for an accompanying ‘substantive theory’
A ‘substantive theory’ for multilevelRecent work by Little and YilokoskiMain features
3
PHILOSOPHY OF EDUCATION AND EMPIRICAL RESEARCH
4
Does PhilEd pay enough attentionto empirical research?
Phillips J Phil Ed 39(4), (2005)
• PhilEd hasn’t paid enough and serious attention to empirical research
• It is possible to study normative processes empirically
• Mutual benefit of PhilEd and empirical research to look into real cases
Hyslop-Margison & NaseemPhl Ed Archive (2007)
• Straw man: wrong selection of critics
• Counterexamples: PhilEd does pay attention to empirical research
• Problem of empirical generalisability
5
PhilSci and PhilEduPhillips (2005, p.582):
The marked change in doing philosophy of science came about when it was realised that there was much to gain by taking scientific research seriously, rather than discussing an artefact of the philosophers’ imagination. […]
The present essay is making a call for a parallel revolution in philosophical discussions of educational research, a revolution that entails taking examples of educational research seriously. […]Processes that humans engage in, in the real world, whether normative or cultural or psychological (or all three at once) can be studied—and probably ought to be studied—empirically, but they also need to be assessed in terms of the values (and if relevant the conception of education) that they embody.
Turn the question on its head
Does empirical research look sufficiently into philosophy of education? (Or, for the matter, into philosophy?)
7
EMPIRICAL RESEARCH
8
Modelling in the social sciences
Causal relations in social contextsMarital problems ⇄ migrationMaternal education → child survivalStress + physical health + …→ self-rated health
Two approachesQualitative: smaller and focused samplesQuantitative: statistical analyses of large data
9
MULTILEVEL MODELSA crash course
10
Why multilevel?
An example of quantitative methods used in empirical research in educationGoing quantitative, the new panacea for evidence
But … is it really panacea?
It models hierarchical structuresTypical of social (and education) contexts
A sounding board for the question: does empirical research look into philosophy?
11
Multilevel models
A special type of statistical model used in causal analysis to model hierarchical structures:
Individuals / family / local population / national populationFirms / regional market / national market / global marketPupils / classes / school / school systems
No a priori reason to choose the level of analysisActually, good reasons to study the interactions between
the levels
12
Traditional approaches
Holismthe system as a whole determines the behaviour of the parts in a fundamental way; the properties of a given system cannot be reduced to the mere sum of its components
Individualismsocial phenomena and behaviours have to be explained by appealing to individual decisions and actions, without invoking any factor transcending them
13
The ‘statistical’ counterparts
Aggregate-level models explain aggregate-level outcomes through aggregate-level
variables
Individual-level models explain individual-level outcomes by individual-level
explanatory variables
14
Types of variables
Individual: measure individual characteristics, take values of each of the lower units in the sample. e.g. income of each individual in the sample
Aggregate: summary of the characteristics of individuals composing the group e.g.: mean income of state residents
15
Dangers
Atomistic fallacywrongly infer a relation between units at a higher level of analysis from units at a lower level of analysis
Ecological fallacydraw inferences about relations between individual level variables based on the group level data
16
Robinson: illiteracy and immigration
1930 census in the US, for each of 48 states + district of Columbia
Individual correlation: descriptive properties of individualsPositive correlation: immigrants more illiterate than native citizens
Ecological correlation: descriptive properties of groupsNegative correlation: correlation between being foreign-born and
illiterate magnified and in the reversed direction
Explanation: immigrants tend to settle down in states where native population is more literate
17
Courgeau: Farmers’ migration in Norway
Data from the Norwegian population registry (since 1964) and from two national censuses (1970 and 1980)
Aggregate model and individual model show opposite results:Aggregate: regions with more farmers are those with higher rates of
migrations;Individual: in a same region migration rates are lower for farmers than
for non-farmers
Reconciliation: multilevel modelaggregate characteristics (e.g. the percentage of farmers)explain individual behaviour (e.g. migrants’ behaviour)
18
Types of models - summaryIndividual: explain individual-level outcomes by individual-level
explanatory variablese.g.: explain the individual probability of migrating through the individual
characteristics of being/not being farmer
Aggregate: explain aggregate-level outcomes through explanatory aggregate-level variables
e.g.: explain the percentage of migrants in a region through the percentage of people in the population having a certain occupational status (e.g. being a farmer)
Multilevel: make claims across the levels, from the aggregate-level to the individual-level and vice-versa
e.g.: explain the individual probability to migrate for non-farmers through the percentage of farmers in the same region
19
Grouping in multilevel
Units grouped at different levels, a-contextual language
Grouping may be more or less randomOnce the grouping is done, differentiation:
group and its member influence and are influenced by the group membership
20
Statistical modelling of hierarchies
0 1 2ij j j ij j ijY x z
response variable at the individual level
explanatory variable at the individual level
explanatory variable at the group level
i: index for the individuals
j: index for the group
these vary depending on the group
Errors are independent at each level and between levels
21
Goldstein:Multilevel in educational research
Study school effectiveness, examination results, …All quantifiable aspects of education
‘Statistical’ advantages of multilevelEfficient estimates of regression coefficientCorrect standard errors, confidence intervals,
significance tests for the clustersEnables measuring differences between clusters
http://www.math.helsinki.fi/msm/banocoss/Goldstein_course.pdf
22
Hierarchies in educational research
Simple hierarchy:Pupil / class / school / neighbourhood /
Cross-classified structurePupil – ethnicity // school – neighbourhood //
23
Goldstein et al: examination results and school differences
Inner London schools Response variable: examination resultsExplanatory variables: standardised London reading tests, verbal
reasoning category, gender, school gender (mixed, boys, girls), school religious denomination (State, Church of England, Roman Catholic, other)
Results:Small effect of school gender; Roman Catholic slightly better; girls better
than boys; large differences for different verbal reasoning categories.
Differences between schools in examination results depend on intake achievement and curriculum subject considered
No single dimension in which schools differ
24
Driessen: School composition and primary school achievement
Dutch primary schoolsResponse variable: language and math proficiencyExplanatory variables: parental ethnicity and education,
pupils sex and age, school composition, ethnic diversity
Results:Quite strong effect of school composition on language,
weak on math; all children, independently of background, perform worse in schools with high ethnic diversity
Question about distribution policy and other measures
25
[…] despite their usefulness, models for multilevel analysis cannot be a universal panacea. […]They are not substitutes for well grounded substantive theories […]Multilevel models are tools to be used with care and understanding.
Goldstein, Multilevel statistical models,http://www.bristol.ac.uk/cmm/team/hg/multbook1995.pdf
26
WHAT ‘SUBSTANTIVE THEORY’ FOR MULTILEVEL?
27
Modelling and explaining
What does a multilevel model model?Relations between different levels in a hierarchical structure
What does a multilevel model explain?How group behaviour influences individual behaviour (but not vice-versa)
Statistically, multilevel achieves bothBut the ‘substantive theory’ is still wanting
28
What needs the ‘substantive theory’
School religious denominationWhat social practices, norms, values are involved?
School composition and ethnicityHow do these influence peer relation among pupils?
…
What is the extra information that we need and that statistics does not give us?
29
LEVELS IN A SUBSTANTIVE THEORY
30
Levels, beyond statistics
Dan LittleLevels of the social:
Ontology what social entities?Explanation reduction?Causation causal powers? Inquiry what level? Description what level requirements? Generalisation recurrence of types?
Avoid analogies with natural sciences, don’t reify social phenomena
31
Levels, beyond the received views
Methodological individualism
• Social facts must be reducible to facts about individuals
• There is no higher level without lower level
• E.g.: Austrian school economics, some political scientists
Holism
• Social entities and structures have primacy and are independent
• Individuals are influenced by social facts, but do not influence them
• E.g.: sociologists in the Durkheim tradition
32
Methodological localismSocial structures influence social outcomes, embodied in action of socially
constructed individuals
Individuals are the bearers of social structures and causes, but individual actors are socially constructed
Emphasis:Contingency of social processesMutability of social structures over space and timeVariability of human social systems (norms, social practices, urban arrangements, …)
Cast doubt on generalisable theories across many populations, look for specific causal variation
33
Scale-based levels
Petri Ylikoski:Macro social facts are typically supra-individual
Micro and macro have a part-whole relationship, but not just mereological constitution
Difference in scale, not categorical
A heuristic, as there is no unique micro-level, context-relativeness
34
Different questions
Constitutive questions
• How macro properties are constituted by smaller-scale entities
• How the macro depends on the micro
• How the macro would have been different, had the micro been different
Causal questions
• Origin, persistence, and change of macro properties
• What the outcome would have been, had things in the causal history been different
35
THE SUBSTANTIVE THEORY, MAIN FEATURES
36
Levels and types of question
Give reality to levelsallocation is not random, not a statistical artefact
Articulate the embodied aspects of level interactionsSociology, anthropology, pedagogy, psychology, …
Look for empirical origins of variations in outcomeLarge-scale statistical studiesSmall-scale qualitative studiesINTEGRATION of explanation of socially-constituted behaviours
37
TO SUM UP AND CONCLUDE
38
In this talk:PhilEd and empirical research
Disputed question of the relation between PhiEd and empirical researchTurn the question on its head:
does empirical research look into philosophy?
Empirical researchCausal modelling widely used in social research, including educationSophisticated formalisms are designed to measure, model, explain social
reality, including hierarchical structuresDespite progress and improvements, formal methods are still in need of
‘substantive’ theory
In search of a substantive theoryRecent work by Little and Yilikoski addresses level ontology, it helps find the
main features of the substantive theory
39
Trouble shared, trouble halved?
Empirical research does not look outside statistics sufficientlyA problem shared also by e.g. social epidemiology
Statistical modelling, an alleged gold standard to generate evidenceA problem shared by e.g. evidence-based medicine
40
Remedies?
Qualitative research, philosophical investigations to feed empirical research
Dismantle evidence hierarchies and gold standards
Build integrated methodsQuantitative and qualitativeEmpirical and conceptualMultiple sources of evidence
41
REFERENCES
42
Courgeau D. 1994 Du groupe à l’individue: l’exemple des comportements migratoires. Population 1.
Courgeau D. 2007 Multilevel synthesis. From the group to the individual. Springer.
Driessen G. 2002 School composition and achievement in primary education: a large-scale multilevel approach. Studies in Educational Evaluation 28.
Goldstein H. 1999 Multilevel statistical models. Wiley.
Goldstein et al. 1993 A multilevel analysis of school examination results. Oxford Review of Education 19(4).
Hyslop-Margison EJ and Ayaz Naseem M. 2007 Philosophy of education and the contested nature of empirical research: a rejoinder to D.C.Phillips. Philosophy of Education.
Little D. 2006 Levels of the social. In The Philosophy of Anthropology and Sociology, Risjord and Turner (eds). Elsevier Science.
Phillips D.C. 2005 The Contested Nature of Empirical Educational Research (and Why Philosophy of Education Offers Little Help). Journal of Philosophy of Education 39(4).
Robinson W.S. 1950 Ecological Correlations and the Behavior of Individuals. American Sociological Review, 15(3)
Russo F. Causality and causal modelling in the social sciences. Measuring variations. Springer. 2009
Ylikoski P. 2012 Micro, macro, and mechanisms. In Oxford Handbook of Philosophy of Social Sciences, Kinkaid (ed) OUP