DEPARTMENT OF SOCIOLOGY

Using Social Science Data in ABM: Opportunities and Challenges

Edmund Chattoe-Brown (ecb18@le.ac.uk)

Plan• Why worry about this at all?

• Some “running example” ABM.

• What are the issues?

• This has to be an incomplete survey and your interests will be very wide: Please ask questions.

Why Worry?• ABM is not just the technique. Very importantly

it is also the methodology.• You need to think about “why this way” and be

able to defend it just as you would for any other research method.

• What might it mean to tackle the “wrong questions” using ABM?

• Do we need to stop talking mostly to ourselves? Most of you won’t be employed in a “Department of ABM” but as sociologists, economists and so on (probably).

Zaller-Deffuant (Z-D) Model• Agents have an attitude and an uncertainty.• Agents can influence each other only when their

attitudes lie within each other’s uncertainties.• Such influence affects both attitudes and

uncertainties.• Under some circumstances (with “extremists”) this

model can result in increasingly polarised attitudes.• This is supposed to give interest to the model

because polarisation is politically significant (and perhaps worrying). Is Z-D a plausibility trap?

Gremy/Boudon Model• Agents have personal attitudes and career

aspirations (an altruistic person tends to want “an altruistic job”).

• Social influence can occur in changing both personal attitudes and career aspirations.

• Agents that have non congruent attitudes and aspirations are more likely to be influenced.

• This model is justified (and in an awkward way calibrated) using (existing) real data at two time points.

Schelling Model• If agents have less than a certain fraction of

neighbours of their own kind they will move randomly.

Hägerstrand Model (1965!)

• A stochastic simulation (but arguably still an ABM) in which a farming innovation has a chance to spread spatially based on independently calibrated data on “typical” migration and communication distances.

The Gilbert and Troitzsch “Box”

Notes• Without a well defined target, none of this is going to

work probably. (Having a clear research question.)

• The test of adequate abstraction is similarity. It is not a “theoretical” or “disciplinary” notion.

• ABM do not “fit” data. If you want to do that it is hard to justify ABM rather than statistics.

• Example 1: Hägerstrand’s surprising assumptions.

• Example 2: Gremy/Boudon arguing from data. (Another reason it is really useful.)

No Data ABM• How do we convince people that this is actually

a model of what we say it is? (Example: Z-D.)

• How do we show it applies to a particular social settings? (Example: Prisoner’s Dilemma.)

• How surprising are most thought experiments really?

• Possible exception: “Implementing” existing theories.

ABM Matching Macro Data• Equifinality: How many different ABM could

have given that pattern of data? (Depends on the “richness” of the data. Example: Schelling model.)

• Power: Can we justify the additional complexity of an ABM relative to statistics? (At risk of being able to prove anything at all. Example: Gremy/Boudon argument.)

The Schelling Model

PP=0.3 PP=0.6

Three Type “Schelling Model”

All self liking (1 0 0) Red and blue both hate green but like each other a bit (1 0.5 -1)

Example: NY residential segregationOne dot equals 25 people (White: red, Black: blue, Hispanic: orange, Asian: green.)

Data again: What we usually have to explain is mixtures.

ABM Exploring Micro Data• Don’t want to end up rationalising potential

fictions. (Example: Z-D and voting for the Nazi party as “simple” polarisation.)

• Connection to use of existing research as data: In the first instance we may want to reproduce only strongly supported results. (It is surprising how many aren’t on closer examination. Example: Social networks are small world.)

It Can Be Done 1

Hägerstrand: Look at actual intersection points as well as shape.

It Can Be Done 2

Abdou and Gilbert (2009)

Very interesting question: Why aren’t articles like this cited everywhere?

Types of Data: Statistics 1• Even quite “low grade” data.

• Example: Z-D. Results simply look nothing like UK time series attitude data.

• A whole ABM “industry” undermined by a bit of web research.

• Don’t let it happen to you!

• A cultural problem not a technical one? Does it create disillusion with ABM?

Types of Data: Statistics 2

Types of Data: Statistics 3• Again, existing methods make presumptions that

clash with those of ABM. Example: Chattoe-Brown (2014).

• Compare a survey on political conversation reporting “daily”, “once a week” (ABM can calibrate on this) and one using “often”, “rarely” (ABM can’t calibrate on this).

• For statistics (which just correlate variables) both are about as good.

• Don’t be too apologetic when there isn’t data but there is little excuse when there is!

Types of Data: Networks 1

Medici

Preferential Attachment

Giant Component

Types of Data: Networks 2• This is “kind of” what dynamic Social Network Analysis does too

but with restrictive assumptions (and without integrating explicit micro processes. Probabilities versus behaviours?)

• Degrees of similarity: This network or a network like this? How like?

• Do we need big networks? Equifinality/power again?

• Example (work in progress): Hummon (2000) says utilitarian networks will be complete, star or empty. Certainly work group networks look nothing like any of these.

• Approach clash: What 5 network measures characterise a network with least redundancy?

Types of Data: Interviews 1• “‘A lot of my friends, they’ve all… they’re in the Army, or they’ve got jobs that, it’s a

career, not a job, if that makes sense. They’ve got that, and I’m just like, well, I don’t even know what I want to do. […] I don’t know, I just have no aspiration. I don’t know what I want to do, and when and why, and where.[…] going back to how I used to be, I just want to try and get in a job, and then look for a better one,… just so I’ve got something to spend my day with. But I don’t know really. I think that’s what I’d do, just for the fact of being able… have a job and actually getting out the house, that’s what I’m more keen about doing.’ (18-24, below Level 2 (including no qualifications), unemployed/inactive)”

• Career versus job.

• Goals of jobs (getting out of the house).

• How do people get to know what they want to do?

• What happens if people aren’t able to do the things they want to?

• Is developing aspiration itself a process in some cases?

Types of Data: Interviews 2• “I don’t like to think in terms of social class. It’s all very difficult for me. I mean I don’t

think we have classes any more. I mean working-class was like when we had manual workers. I’m not sure you can call women working-class, except I suppose if they’re cleaners or what. I suppose … well I’m not working class anymore and I’m not middle class either. To be honest I think we’re all classless now.”

• Not all qualitative research believes in generalisation in the first place.

• Some fields (like economics) just don’t “do” qualitative data.

• Some is just done badly even by its own standards.

• We have no more than outline ideas about systematic use of qualitative data in ABM. See: http://cfpm.org/qual2rule/

• Arguably, the most important thing is to “show your working” whatever you do.

• Sociology (and other areas) already have systematic techniques for analysing qualitative data i. e. Grounded Theory (Glaser and Strauss), Protocol Analysis (Ericsson and Simon). Ethnographic Decision Tree Modelling (Gladwin).

Types of Data: Experiments• Ultimatum Game: Can “calibrate” agents to

match controlled data.

• Watch out for presumptions of different fields: Pair play with recorded narratives?

• Problem of ecological validity. Link to “gaming” i. e. beer game (http://www.beergame.org).

• Again we don’t really have a state of the art yet but see good exemplars in references.

Types of Data: Experts• Can use experts in different ways: Calibrate or validate.

• Make sure you get validation not rationalisation. (Can you ask the expert to declare before they know what the ABM is doing?)

• Try to decide whether your expert really is an expert.

• Experts disagree.

• An expert’s view is a kind of theory. Like any other theory it may be wrong or inconsistent, particularly if it is complicated.

• Interviewing experts for ABM is currently more of an art than a science.

• This shades into participatory modelling which may serve wider goals like consensus building.

Types of Data: Existing Research• Looking at many articles, the state of the art is

apparently “ignore it”. However, this may actually be our dominant source of data.

• Because ABM is really new, we have to decide what to do with the accumulated history of research to date (and how.)

• For tentative approaches, see Chattoe-Brown (2013b) and Chattoe-Brown (2014).

The Challenge of “Scientific” ABM 1

• Existing methods not only have a methodology but a standard well defined “Unit of Research”.

• Should the ABM UoR be expected to include all necessary data?

• Very important: A theory is scientific if it can be tested, not only if it has been tested.

The Challenge of “Scientific” ABM 2• The toughest test of ABM is if it can show similarity

based on no “free” parameters at all (instead using best guesses from existing data). Rarely possible?

• Progressive refinement strategy: This parameter is about 10 (not about 100). This parameter is 8 +/- 3. This parameter is 7 +/- 1. (Connection to sensitivity analysis.)

• Sensitivity analysis can be “reversed” to focus data collection (or data research). Which parameter(s) is it best to reduce the uncertainty of?

Case Study 1: Chattoe (2006)• Initially a theory testing simulation: Does the “strict churches are strong” argument of

Iannaccone hold with dynamic church creation rather than partial equilibrium? No.

• But how could we evaluate this evolutionary model empirically?

• DEPARTURE: “You need to make decisions about the money and other things together as a group. But the leader started making them on his own. That’s not the way it’s supposed to be. For the leader to start rejecting what everyone else says and to only follow what he thinks is going to cause problems.”

• DEPARTURE: “My husband had died, the children were small, I had to go out to work, and so I couldn’t keep going to the church. To be more exact, I went to [name of city] to live with a man there - God doesn’t want us to lie in front of him. Because I was on that bad road I left the church.”

• Strictness doesn’t get a mention, in this study at least.

• But need sample of ethnographies (or comparative ethnography) to look at operation of strictness and its effect on departure/survival. Not very evident in the literature. (Or develop into effective survey from ethographies?)

Case Study 2: Sample Data• “High rates of participation and financial contribution were customary and normal.

Members were expected not only to attend the Sunday morning service but also revivals, fund raisers, and weekly Bible study. They were also expected to join at least one of the church’s organizations and to tithe their earnings. (I was asked to be on the Usher Board and to teach the men’s Sunday school class on the same day my wife and I joined the church.)”

• “Eastside Chapel had several structures in place for maintaining these commitments of time and money. Reverend Wright repeated the strict standards of participation and contribution in his sermons, during Bible study services, and at every other opportunity. The members themselves also upheld these norms and would often informally sanction one another for any perceived deviations. According to standard Methodist practice, Eastside was divided in “classes,” and class leaders were responsible for contacting their members when they did not appear in church. In order to maintain standards of giving, church leaders published the members’ contribution.”

• Timothy J. Nelson (“The Church and the Street: Race, Class and Congregation”).

• Almost too detailed? Again, may need “high points” only.

Case Study 3We need lots of this kind of data which is there but fiddly.

What is similarity? Better defined for some comparisons than others.

Case Study 4• Evolutionary arguments are particularly hard to

assess because they involve qualitative data on large numbers of units which are usually characterised quantitatively.

• New thinking: Standardised team ethnographies?

Evaluating Published ABM• Does it compare real and simulated data?

• Does it cite substantive empirical research on its subject? (Not pop science.)

• Does it cite (and try to reconcile) related ABM?

• Try this test: You will be surprised.

• Imagine existing research methods not citing substantive results or past research. Is it any wonder there may be growing external disillusion with ABM?

• Future proof your skills!

Research Design and Validation• Examples: Gremy/Boudon, Schelling.

• Out of sample: It’s a real shame G/B just didn’t have more data. (Also to examine robustness of transition probabilities.) But consider point by Edmonds.

• Out of “dimension”: “New research from mortgage and loan broker Ocean Finance suggests that around 60 per cent of adults have lived in the same house for more than 15 years, while one in ten say they have not moved for 31 years or more. Almost a quarter of those questioned have lived in 10 or more homes in their lives with moving unsurprisingly more common among younger people – 54 per cent of people aged under 25 have already lived in three or more places. On average, people typically move three more times before they are 45.”

• Note issue of “calibratability” again here. Can’t do anything with “more than 15 years” without making assumptions about tick lengths in Schelling but can compare “10 or more homes” (simulation gives 63% compared to 25%). It also seems “likely” that the “never moved” group is much too small.

• Right conclusions from this are important.

The Importance of Research Design• How many “goes” do you get at this before you have

used up all your data?

• Another argument for making the best possible use of existing accumulated research?

Other Tips …• Try to develop your ABM to testable even if it can’t be

tested yet. (Always think about how parameters might be calibrated later: Know research methods.)

• Go where the (diverse) data is for now?

• Let domain experts define problems? (But CM story.)

• At this stage doing anything systematic may still be much better than doing nothing.

• A day on google exploring and downloading is still quicker than doing interviews!

References 1• Abdou, M. and Gilbert, N. (2009) ‘Modelling the Emergence and Dynamics

of Social and Workplace Segregation’, Mind and Society, 8(2), pp. 173-191.• Bloomquist, K. (2011) ‘Tax Compliance as an Evolutionary Coordination

Game: An Agent-Based Approach’, Public Finance Review, 39(1), pp. 25-49.

• Bravo, G., Squazzoni, F. and Boero, R. (2012) ‘Trust and Partner Selection in Social Networks: An Experimentally Grounded Model’, Social Networks, 34(4), pp. 481-492.

• Chattoe, E. (2006) ‘Using Simulation to Develop and Test Functionalist Explanations: A Case Study of Dynamic Church Membership’, British Journal of Sociology, 57(3), September, pp. 379-397.

• Chattoe-Brown, E. (2010) ‘Is Simulation Forgetting Its History: Two Case Studies’, available from academia.edu. [Also references Hägerstrand and Gremy/Boudon.]

• Chattoe-Brown, E. (2013) ‘Why Sociology Should Use Agent Based Modelling’, Sociological Research Online, 18(3), <http://www.socresonline.org.uk/18/3/3.html>. [Also discusses/references the Schelling model and its wider significance.]

References 2• Chattoe-Brown, E. (2013b) ‘Building Simulations Systematically from

Published Research: A Sociological Case Study’, available from academia.edu.

• Chattoe-Brown, E. (2014) ‘Using Agent Based Modelling to Integrate Data on Attitude Change’, Sociological Research Online, 19(1), <http://www.socresonline.org.uk/19/1/16.html>. [Also references Z-D.]

• Gilbert, N. and Troitzsch, K. (2005) Simulation for the Social Scientist, second edition (Maidenhead: Open University Press).

• Nowak, A., Szamrej, J. and Latané, B. (1990) ‘From Private Attitude to Public Opinion: A Dynamic Theory of Social Impact’, Psychological Review, 97(3), pp. 362-376.

• Padgett, J. and Ansell, C. (1993) ‘Robust Action and the Rise of the Medici, 1400-1434’, American Journal of Sociology, 98(6), pp. 1259-1319.